Using Neon with Cloudflare Workers

Hello, everyone. Welcome to today's segment. We're going to be discussing how to use Neon with Cloudflare Workers. I am Gift Egwuenu, developer advocate at Cloudflare. And today I have a guest on the show with me, Mahmoud. Mahmoud, hi, thank you so much for joining me. Would you like to introduce yourself? Mahmoud Abdelwahab Absolutely. Thank you so much. Super, super happy to be here. I'm Mahmoud and I work as a developer advocate at Neon. And yeah. Gift Egwuenu Yeah, sounds good. I'm really excited to be showing exactly how you can connect Neon and Cloudflare Workers in this segment. A couple of weeks ago, actually, we had a developer week where we announced database integrations. So essentially, this enables developers to connect to their database of choice seamlessly using one of the database providers that we have integrations for. So right now we have one for Neon, of course, Superbase and PlanetScale. So for this session, we'll have Mahmoud walk us through how you can seamlessly connect Neon and Cloudflare workers to have a database on the cloud. So over to you, Mahmoud. I'll stop sharing my screen if you'd like to show us exactly how this works. First of all, what is Neon actually? That would be a very good place to start. Mahmoud Abdelwahab Yeah. One second, let me share my screen. So you should be able to see my screen, correct? Gift Egwuenu Yes. Okay. So Neon is serverless Postgres. It's neon.tech. And typically with Postgres, whenever you want a managed instance, you kind of need to pick a size for your database upfront. So you have to specify the amount of resources you'll need. So how much CPU, memory, and storage for your database? Now with Neon, it's serverless. So similar to the pricing model of something like Cloudflare Workers, you only pay for what you use. And this is possible because Neon introduces kind of like a new architecture where we separate a storage and compute, but no need to go into the super, super technical details on how that works. All you can kind of, like all you need to know really is that it's Postgres. It's almost like we strive to be 100% compatible with vanilla Postgres. So all extensions should work, everything. And what happens is when you create a Neon instance, you don't really have to worry about managing and scaling infrastructure. And what we want to kind of like achieve is that it just works where you don't really have to think about anything other than your queries and you building your application. Now, we're super excited about launching an integration with Cloudflare Workers because what the integration does is that it automatically sets the database connection string for your Cloudflare workers. So this way, you don't actually have to manually copy and paste them. And with a few clicks, which we'll go over right now, you can just set it up and it works seamlessly with Cloudflare Workers. So first step is actually creating an account. If you don't have one already, you can just click sign in. And I already have an account. I'm logged in. And this is what you'll see. I don't have any projects. You'll see kind of like the onboarding experience where we just go over kind of like the different features in Neon. But I'm just going to create a project right now. And you can think of a project as a container of all of the resources that you can have. So, and let's say, for example, you have different apps that you're working on. Typically, you would want a project for each one of them. This way, you can separate kind of like the concerns for each app. And when you create a project, well, the first thing you'll need to specify is the Postgres version. You can give it a name. So, we can say, what should we call it? Yes, I did. Books. Movies. Maybe movie database. I'm not sure. Actually, I think I'll create something different. Let's just call it Neon with Cloudflare. Okay. Okay. And version will be 15. We pick a region. And you typically pick a region based on where you expect your users to be. With Cloudflare Workers, they're deployed globally. So, you don't have to worry about that. But right now, for the database, you have to specify kind of like where the region is. And we plan on introducing read replicas soon. So, this will play very nicely with something like Cloudflare Workers on the edge. So, I'll just pick Europe because that's close to where we are. And then you can pick the size of your compute. We have the option of autoscaling. And in this option, you specify a minimum and a maximum size for your compute instance for the database to kind of like, you know, this is the interval. This helps you set like a maximum amount of resources to be provisioned. So, you don't accidentally, you know, have like a very compute intensive query and you end up kind of like overpaying for something you didn't need. And yeah, there's also the option of just a fixed size. You pick a fixed size for the database. That's it. The downside is if you kind of need more resources, you'll have to manually increase this. But if you pick autoscaling, which is kind of like what we are testing right now, making sure that it works seamlessly, you don't really have to think about, you know, like scaling. Everything will scale automatically. So, after I create a project, we will see the connection string. And it will have the credentials. So, I'll reset my password. So, let me just reset the password. It's kind of like right now. So, you will see the onboarding here. And you can kind of go through it if you want. It goes over kind of like the different features that we have. So, we have our own SQL editor, which I'll use. And we also like just show links, like a bunch of useful links that you can use. So, let me just skip this. And let me just reset the password. So, that and I'll just copy it. No one should see it. But actually, we don't need to copy actually the password because it will be set automatically by the integration. But now we know for sure that no one's going to do anything to the database. So, actually, the Cloudflare docs are really nice for the integration. So, there's I don't know if we is there a way to share links or do we just do it kind of like at the end of the live stream? We can add the links at the end of the live stream. Yeah. Okay. Perfect. Yeah. So, like there are the docs for databases on Cloudflare. These are the docs for the integrations that they have. So, like Gift said, there's PlanScale, Superbase, and Neon. We're just in Neon. So, I'm going to copy the SQL query here. And this just creates a table called elements. And it has four columns. An ID, an element name, an atomic number, and symbol. And let me go back to the console. And I'll go to our SQL editor. And the SQL editor enables you to queries against your database in Neon. So, I'm just going to paste it. And I'm going to run it. And everything ran successfully. So, now I can actually go to the tables page. And we should see that we have an elements table, which we do. We'll see that we have all of our columns and everything works as expected. But right now the database like the table is empty. So, we can add some data. So, actually, if I go back to the guide, we have an insert statement here that we can just copy and paste. And it should just work. Paste it. Run. And it's complete successfully. So, if I go back now to the tables page, I'll be able to view the data that's in the elements table. And yeah, it's very nice. So, that's kind of all we need, really, with like the Neon part. Typically, you know, when you're building an app, you will connect there are several ways to connect. You can use an ORM. You can use just a query builder. And we have at Neon our own serverless driver. So, it's at kind of like Neon database slash serverless. And this enables you to connect from serverless environments to the database. Now, this was before Cloudflare added support for TCP connections in beta. So, this is kind of like the problem that this solves. But, you know, it really depends on, you know, you evaluating the different approaches. Each approach has its own pros and cons. But yeah. So, one way is to use the serverless driver, which is just a wrapper kind of like over Node Postgres. And it just enables you to send queries. Let me just zoom in a little bit. So, you can just create a client and send a query. And as you can see here, like when you create a new client or here, it's like a new pool, you specify the connection string. And it's loaded from the environment variables. So, typically, when you're using the when you're writing a Cloudflare worker and you deploy it, well, we will see how you can set environment variables manually. But then go over how the integration just sets it automatically. So, this is actually one way to connect to a Neon database from a Cloudflare worker. Another like one another kind of like new ORM that's kind of like becoming more popular. I wrote a blog post about it. It's called Drizzle. And it's a TypeScript ORM. And this guide really kind of goes over everything. It showcases the integration and how you can use it. And the difference really is that the serverless driver doesn't manage the full workflow of creating tables, managing migrations, applying migrations, and writing queries. You're going to be writing raw SQL. But in the case of an ORM like Drizzle, you're actually writing TypeScript that then gets converted to SQL. So, it's kind of like a better developer experience. But, you know, it depends on your use case. So... MARIYA MOEVADA I just stumbled on this as well recently. And I thought it was cool to have like... MARIYA MOEVADA Yeah. So... ADO AKEMI Yeah. It's very nice. Because it's kind of like a toolkit, really. They have a CLI. So, it's Drizzle.team. And they have dedicated docs, I believe. Yeah, they do. So, yeah, it's a TypeScript ORM. And yeah, it's pretty new, but they're doing it very well. Like, their philosophy is really to be close to SQL. And it's really performance. So, they don't really have any issues when it comes to performance. Because it's just kind of composed to regular SQL. So, it's pretty cool. Definitely worth checking out and for it to be on your radar. And the team is very cool. And they have like different connectors. And they always make sure that the ORM works in all environments, which is very nice. So, that's kind of like what two possible ways for you to connect to a NEON database from a Cloudflare worker. And we'll share links kind of like towards the end. But now, let's say, for example, right now I'm in my Cloudflare account. I'm on the workers and pages tab. And I can create a new application. And I will choose workers. You can do either they've been recently kind of like combined. Also, should I switch to light mode? I don't know. I don't know what people are used to. I'm fine with that. I'm a light mode person. It's okay. Yeah. Okay. So, I can just create a new worker. And this is just a template worker. It has a name. We can call it also like Cloudflare with NEON. And it will be deployed to kind of like a domain.workers.dev domain. So, this has really just this is kind of like basics. And it's okay to deploy it. Like there's no like we're not connecting to a database here. But we can just click deploy. And once it's deployed, when we visit it, well, we're going to see hello world. So, hello world. It worked. That was very fast. Very fast. So, yeah. And this is like globally kind of deployed. So, I think if anyone accesses it from anywhere around the world, it will be just super fast. Because like the response will be served from like the closest region to where you are. Yeah, edge location. So, yeah. We can actually edit the code and kind of like do it manually here. But you can actually use the Cloudflare CLI that's called Wrangler to kind of manage everything related to like it's super nice to experience. I don't know if there's a link here. We'll find a link later. But you can just install CLI. You set everything up locally. You have a kind of Wrangler enables you to start a development server. And it uses it's very like it's identical to the deployed worker. So, you have the same experience. And, yeah. And then you can use the CLI to automate deployments and everything. It's very nice. So, right now we have a deployed worker and everything. And if we go to the list of workers, I have one here, Cloudflare with neon. And what will happen is that we can go to settings. And then variables. And here you can configure environment variables for the worker. And you can do it manually. So, you click add variable. You'll be like, oh, database URL. And then paste in the connection string. But it's just it's not seamless. You have to mess with connection strings. You have to copy them. And it's like, you know, it's always nice when you have integrations. So, here, integrations. They're in beta. You will see the available types of integrations. Also, the UI is a bit messed up because I'm just zooming in to make sure that everyone can see. But, yeah. I don't know. I just want to make sure if someone is watching on mobile, they'll be able to see. But you can see, like, neon as a first option. And I'll just zoom in a little bit. You can click add integration. And here you'll see actually collapse the sidebar. So, here you'll see kind of like the different steps you'll need to do. So, first thing is you'll need to, I believe, accept the condition. But I believe I already did. So, I should be redirected to neon and authorize. Or I've already did it. So, we'll see. Okay. So, I just need to authorize Cloudflare. So, this is neon's OAuth consent screen. And what's happening here is that Cloudflare is requesting access to my neon projects. So, like, so that Cloudflare can perform actions on my behalf. In this case, getting the database connection string and sending it for the worker. So, the scopes here are just reading projects. So, Cloudflare really isn't doing any editing, deleting, or updating environment variables. It's just fetching them from neon. So, all you do is just click authorize. And then I'll be redirected back to the setup flow. So, it's connected. And I get a dropdown. And this dropdown gives me the list of all available projects in my neon account. So, when I click expend, I will see neon with Cloudflare. This is the project that I created. Click continue. And after that, I select a branch. So, this is neon specific. And in neon, so, whenever we create a project in neon, we get a default Postgres cluster. And this is Postgres. So, in Postgres, you can have a cluster. And within this cluster, you can have many databases. So, for example, when we created the project here. So, when we created the project, we have a database that's called neon DB. Now, we can create additional databases if we like, depending on our needs. And these databases, so, this Postgres cluster in neon is represented by a branch. So, you can think of it this is one of the cool things about neon. It's like the first aspect is it's serverless Postgres. You don't have to worry about scaling and provisioning resources and everything. Everything kind of just works. But the second part really is related to developer productivity. So, with neon, you can create copies of your projects kind of like Postgres cluster. So, right here, like after I created my project, a default Postgres cluster got created. And in it, there was a default database called neon DB. So, I actually have a diagram that I can show that will make this a bit easier to understand. And it is this one. So, okay. After creating the project, this project now contains the main branch. And this is a default Postgres cluster. And in it, there are a lot of databases that you can create. We don't have a limit on the amount of databases you can create. And by default, there's a neon DB database that got created. And you can have many branches, as many as you like. And the use case really is like there are many use cases. But one of the primary use cases is you can create a branch for every preview deployment. And this also works nicely with workers because you can have multiple environments. So, you can have a worker with a database for every pull request that you create, which is very neat. But right now, we actually don't need more than one branch just because we're showcasing the integration. So, if you go to like branches in neon, you'll see that we only have a main branch. And yeah, we're going to select it when configuring the branches. You can configure the integration with other branches as well. It depends on your use case. But for now, we just want it for main, which typically will be our production branch. So, I can click continue. And then, as you can see, I can select the database for this branch. If I'm selecting another branch, then it will be a different database for that branch. So, database, I only have one. It's neon DB for the endpoints. So, this is also something neon specific. And like I mentioned, kind of like at the start, we separated storage and compute. So, what this means is the storage part is separate. And in our case, it's backed by object storage. So, AWS S3 equivalent in Cloudflare land is R2. And yeah, so, this is kind of like one aspect. The other aspect is the compute part. So, this is actually what makes it possible for us to automatically scale up and down resources. Because it's like we can just allocate more resources to meet the user's workload. So, each branch has its own associated compute endpoint. So, if I go back to the diagram that I just kind of like was on, it's right here. So, okay. So, each branch has its own compute endpoint. So, if you create, let's say, I don't know, a second branch, this branch will have another compute endpoint. Now, this is very interesting because the branches are completely isolated from the other. Same way with like any environment. If you make any change to a branch, well, it doesn't actually affect the other. So, that's why we need to specify which branch we want the integration to kind of like pull the information from. Because like the compute endpoint is part of the database connection string. **Samara Hansen** I see. And it's automatically generated for each branch you create, right? **Lucie** Yes. Yeah. Each branch has its own compute endpoint. And it's like this compute endpoint is how you actually access the database. Like you use it to connect to the database. So, yeah. In our case here, we only have one endpoint for this branch. So, I'm going to select it. And the project itself was deployed in Europe. So, that's why the region is EU central one. And the first part is the endpoint. And this kind of like all of this plus the username and password, this is what makes the database connection string plus specifying what database you're connecting to. So, I'll select this. And then roles. So, when you select a role, a role is in Postgres is the username and password combination. So, this is my role. This is my GitHub username. This is kind of like the default. But we can create other roles on Neon and they will be automatically loaded here by the integration. So, this is my role. So, with all of this information, the integration will be able to construct the connection string, the database URL, and then it will be automatically added to the Cloudflare worker when it's deployed. So, I'll just click continue. And, yeah. Now it's telling me, hey, configure secrets. What will be the secret name? So, database URL. Makes sense. And then the value. So, as you can see, the value is just my name. And then there's the password, which is not exposed, which is great. And then the endpoint ID that like the endpoint that we chose. And then the database name. So, that's pretty much it. Like, it took a bunch of clicks, really, to just load all this information. You click add integration. And everything works. So, it just says Neon is now integrated with this worker. We can give feedback. We can say, hey, it worked great. And, yeah. If we go now to actually variables, we should be able to see the variable, correct? So, it's there. And it's encrypted and everything works. So, that's pretty much it. So, yeah. It just works. The integration now. So, I think initially, like you can use a database connection string with a Cloudflare worker and connect. It just wasn't this convenient. And now it's very convenient. You just redirect it back. You connect it. And you have the database URL. Now, let's say, for example, you, I don't know, let's say you reset the password. Well, this connection string will not be valid anymore. So, you will need to go through the integration. But it's just a couple of steps. I think it's okay. But, yeah. That's pretty much it. I think, yeah. Like, this is the integration. I think right now, yeah, we just covered kind of like a quick recap. Neon is serverless Postgres. You don't have to worry about scaling and managing infrastructure. It plays very well with Cloudflare because it follows the same kind of like mental model with Cloudflare Workers. They automatically scale, deploy to kind of like worldwide, and you get charged per request or for the amount of time the worker has been running for. Same story with Neon. So, it's like if you want to go, if you're all in on serverless, definitely consider using Neon with Cloudflare Workers. And, yeah. We just went over the integration. Super easy to set up. You just go to integrations and the settings for your Cloudflare Workers. And then you can just click configure, go through the steps. Yeah. I have a database URL. Doesn't get easier than this. Yeah. One additional thing. We have like five minutes, Paul. Would you like to show just the how you can then consume the Neon database in a worker? Just the code example? Yeah. So, I do have a code example in the blog post that you wrote. So, let me just scroll down towards the end. So, this is actually what a typical hello world worker looks like. So, this was configured using Cloudflare's C3 CLI that was newly released. And this is what you get after you choose hello world script. So, you will see that you have an interface. This is the environment variables. When you specify them here, you'll get auto completion. And it will be very clear what environment variables this worker is using. And this returns kind of like the hello world. So, this is in TypeScript. And it's very similar to the worker that we deployed kind of like at the start of the stream. Yeah. And I, like in this guide, I use Hono, which is a framework that is compatible with Cloudflare Workers. Super nice to use. I really enjoyed working with it. But if we scroll down towards the end, because right here I go over kind of like drizzle, how to create tables and everything and how to generate migrations. The final, final part, which is actually connecting. And that's kind of how it works. So, in my example, I have a product table. And what I wanted to do is just when someone visits the worker, they will receive the just like the data will get returned from the database. So, it will be an API endpoint, talks to the database, and I use the integration to connect from the Cloudflare worker to Neon. So, in this example here, because I'm using Drizzle, the first thing I did was just create a new client using the Neon serverless driver. And this is kind of how Drizzle like is very composable. It allows you to kind of bring your own client and then it just works with it. So, I just now I'm loading the database URL from my environment variables. And because I'm using Hono, it kind of overrides like you have to access the environment variable using the argument that gets passed through this function. And then what I do is just I create a new instance of Drizzle and I call it DB. And then I just send a query by saying await DB dot select dot from product. So, this is actually very similar to how you would write a SQL query. Like if you want to return all data, you would do select star from and then the table name. This is what I'm doing here. And then I'm just returning the result as JSON. And if there are any errors, I just return the errors. And this and just in this one file, really like I was able to have an API endpoint that you can use and connect to. Of course, there'd be a bit more to it, like things like great limiting, potentially auth, but this is kind of like the hello world example where you can connect to a database using Drizzle, Neon serverless driver and Cloudflare Workers. Yeah, this is really clean, actually. I love the implementation with Drizzle. Yeah. So, it's so nice. I will definitely leave links to every resource that you shared after the segment ends. And yeah, thank you so much, Mahmoud. Pleasure. I had a lot of fun here. Thank you so much for having me and for giving me the opportunity to just talk about the integration, Neon, Cloudflare workers. I'm a big fan of the platform, especially Cloudflare Workers. But yeah. Definitely, our pleasure. I would just like to add one thing before we wrap up that we're excited about the database integration. And at Cloudflare, we're still actually doing some work there to add additional supports. For example, CLI supports with Wrangler. So, the steps where you did everything directly from the dashboard for integrating with the database, you can also do that using Wrangler in the future. And also, we're looking to add support for multiple level environment variables. So, you can have several different environment variables configurable on the integration steps that Mahmoud shared earlier in the demo. So, yeah. Those are things that the team is working to add to the database integration. And I'm definitely excited and looking forward to all of that. All right. Sounds very exciting. Thank you. Thank you so much for joining. And thank you, everybody, for tuning in, in case you get to watch this later. Also, thank you. And I hope you learned something new today. All right. With that, I'll say goodbye and see you all later. Okay. Thanks, everyone. Bye. Yeah.