💻 How Do You Distribute Your Database Over Hundreds of Edge Locations?

Hey everyone and welcome to Databases on the Edge. How to distribute your database over hundreds of edge locations. My name is Erwin van der Koogh and I'm a product manager at Cloudflare working on our works platform. You can find me at erwinvanderkoogh on Twitter and feel free to reach out with any questions or feedback you may have. Now over the past couple years we've seen a trend to see logic move from the region to the edge. And by regions I mean things like AWS's US East 1 for example, but all major cloud providers like Azure or Google have a very similar concept. And when I talk about edge, what I mean is sort of the pops or points of presence. These are spread out geographically so as many customers are as close as possible to a particular pop. One of the most important things they do is things like cached responses so your request doesn't have to travel half the Internet before we can get a response. So there's some obvious differences between the two which greatly influences how they evolved. So regions for example have lots of specialized servers and a lot of different products. They have multiple data centers that are very close to one another but they're very often sort of located like in or around a single city. So they're very close. The edge on the other hand is very very different. They don't have as many servers in a data center. They're much more uniform. That's because there's much fewer products. And every data center is basically sort of on its own and doesn't have any special relationships with any of the other data centers that are out there. And of course they're very geographically distributed. And of course there's sort of regions sort of cousin which we call sort of like multi -region. This is sort of the same thing as sort of the regional except that you do get sort of issues when you're talking about geographic sort of distributives. So first let's sort of look how we went from that edge from the region to the edge from the lens of compute. And yes I'm fully aware that the following content isn't entirely accurate but this is my presentation and not my PhD sort of dissertation. So without further ado let's go. So yeah we got sort of like the region. We got sort of multi-regions and sort of like the edge. And sort of when we talk about sort of things like sort of time sharing very early sort of mainframes did a lot of this. Sort of virtual machines came in actually way earlier than people sort of realized they were running virtual machines on top of sort of mainframes. Things like sort of cgroups and sort of related security sort of things sort of started popping up which culminated in things like sort of containers. Micro VMs sort of became a thing really tiny VMs which sort of the basis of things like AWS Lambda and sort of Azure functions and Google functions were sort of built on the sort of container model. And then sort of something interesting happened which was AWS Lambda at Edge. One of the first technologies that sort of got sort of like multi-regions right. Sort of contrary to popular belief Lambda at Edge never actually run inside sort of their points of presence like on the edge. AWS Lambda at Edge runs in every region that supports sort of Lambdas. And that means that we sort of have this sort of chasm here between sort of things that work in multiple regions but not at the edge. And there's a good reason sort of for that which we'll sort of get into. So how did things sort of start out in the edge? And here we're talking about sort of reverse proxies with things like virtual hosts right. We could now sort of be a reverse proxy for numerous different sort of websites but they're all static configurations still. And then sort of like things like VCL came along which is like the Varnish configuration language which allows for much more programmable configurations. But then sort of V8 came along which formed the basis of sort of Cloudflare Workers where we could now have hundreds or thousands of sort of instances, worker instances running on a single machine. Yeah, so the WebAssembly which is the basis of Fastly's Edge. And then we're talking about ECMI sort of Edge workers, sort of Netlify handlers all working in that sort of JavaScript sort of V8 model as far as we can tell. So one of the reasons why we know that chasm is impossible to cross because when AWS who already had Lambda and Lambda at Edge wanted to run something on the edge, they build it on top of Nginx's NGS and like their CloudFront functions are basically built on on Nginx. Right, so entirely different ecosystems in the regions from the edge. So what's that story about data? Like can we see similar patterns when it comes to data and sort of compute? So yeah, it starts out pretty similar. We've got like relational DBs in works in a region like SQL instead of MySQL and then things like AWS RDS sort of comes along. So like one database, one region works really well. Sort of no SQL became popular because sort of why not? We get things like document stores and sort of key value stores. Mongo comes along. Cosmo DB comes along, which is one of the first things that started out as it's sort of life as a multi-region global key value store. MongoDB launched sort of Atlas. So that became an interesting multi-region DynamoDB, global tables and sort of GraphQL, which is sort of like FaunaDB. So there's now a lot of databases that are sort of in the region, some of them working sort of over multiple different regions, but we still have that edge and that chasm to the edge. And none of these tools sort of made it all the way sort of over here where they're able to work over hundreds of nodes with sort of hundreds of milliseconds of sort of latency. So sort of why? And that's sort of like one of the reasons. One of the reasons is storage is cheap, but not hundreds of copies of your entire database cheap, right? But more importantly, as I mentioned, like getting things like consensus is hard enough in any situation with multiple nodes, but it's impossible with hundreds of them and hundreds of milliseconds of latency. So without that sort of concept of sort of consensus, a database is just like any distributed database is just sort of dead in the water. So how did our evolution go at Cloudflare? One of the first sort of things that we build at Cloudflare are to ourselves from scratch, basically, is a storage engine called Quicksilver. And Quicksilver is our implementation to sort of replicate configuration data over thousands of servers is one of the most sort of critical pieces in our infrastructure. So because without this, like no one can make any changes to their configuration. So the reason we sort of got through these sort of constraints is that from a storage perspective, like the configuration is relatively small and we don't need to keep like an entire sort of historical values in our sort of pops. We only need the latest set of values. And from a consensus perspective, we have centralized rights and we sort of fan out. So we don't run into any sort of issues when it comes to sort of locking or anything else. Next up, Workers KV. And Workers KV runs pretty similar in a way. It's our key value store that works really well with Cloudflare Workers. And from a storage perspective, we don't store the full data sets in every pop that we have. It's in a few locations, a few central locations and only reads are cached in the pops for like a minute or more. And again, a consensus is sort of pretty easy. There's only a few rights in the sort of centralized repositories. But even then, like they are very distributed. So there's no atomic operations, basically the last right wins. Now, some people wouldn't call that a database with the last right semantic. But the truth is, if you have an application with lots of reads, a bunch of rights and very few updates, KV is a perfect database for your use case. And it turns out a lot of web applications fall in exactly this category. But then we get to the interesting bit, Durable Objects. And so what Durable Objects are, are actor -like objects with attached persistent storage. And so the reason we get around that storage sort of problem is by making sure the data is horizontally sharded over multiple clusters. So we have a bunch of different clusters in a bunch of different set of data centers, and we horizontally shard. So your data is only stored in one Durable Objects cluster at a time. And so it's only replicated to the other data centers in that cluster. And again, that also solves our consensus problem, because we only have a few nodes that are relatively close by. And the code runs in the same data centers, set of Azure data. So one of the most important things to understand about Durable Objects is that there can be only one. There can be only one, if you're a true believer. And if you got the Highlander stuck in your head now, I'm not sorry at all. But there can only be one instance of a Durable Object per ID. And we're gonna talk about that, what that means in sort of a little bit more. But be reminded when I sort of go through and explain in a little bit more detail what Durable Objects are, like there can only ever be one Durable Object running for every ID that we are talking to. So as I mentioned before, there are multiple clusters, multiple Durable Object clusters, each with a handful of nodes that are relatively close and well -connected. And so multiple sort of users can connect to sort of multiple Durable Objects in different sort of locations. And yeah, like they're the same sort of locations that sort of like workers would run in, for example. So how do you get a Durable Object? How do you instantiate one of those? And so there are basically sort of two ways to get an ID. One of them is like ID from name, which you can just come up with a name or sort of anything else. So you can go like, give me an ID for this particular Durable Object, or you can go new unique ID, which just gives you a random new sort of identifier. So the thing to remember with like the ID from name, it allows you to create that sort of external identifier, such as a username or sort of a company ID. And this will check if that Durable Object is already created, if it already exists somewhere in the world, even in other clusters. And if it does, this will connect to that already existing Durable Object wherever it may be. But if not, it will create a new Durable Object sort of for you. Now, the new unique ID will of course give you a unique ID. So here are those two sort of statements. And then after that, it's all the same. So once you have an ID, you can get what we call a stub. And the stub is basically a client side wrapper that you sort of connect to, you can create in your worker and you can then call a fetch method on that stub, which will connect to the actual Durable Object. And you can make any kind of sort of request to that, await the response, and then maybe return your own response or do whatever else you want. So how do you use a Durable Object? It's actually pretty simple. Durable Object is just a regular class and it has a fetch method. And you can just call it and do sort of anything you want. The one thing that you see here is like the dis.stage .storage. This is a key value store that you can sort of write to or obviously sort of read from too. And this is sort of the full, the sort of the cool thing about this is this has full transactional support. So yeah, you can just go name, Twitter handle, create a transaction and then sort of have a transaction that's either completely written or not written to at all. And that's what's so powerful. You have your full transactional support semantics running in the same data center in the same class as your code and your logic. So the questions I sort of often get is like, what do you use it for? And the interesting bit is that we are just finding sort of, because it's so new, it will take time to figure out what you can use these for and what you can't use them for. But yeah, things like a per user cache would be great. You can build reliable infrastructure components on top of it. So like queues or sort of pub sub. You can do things like sort of event streaming. Every durable object can, you can connect to web sockets like one or more web sockets can connect to sort of a durable object and a durable object can connect outside to sort of web sockets. So yeah, things like collaborative editing, having one durable object per document and everyone who wants to collaborate on this document will open a web socket to it. And the durable object, because it's this single threaded sort of JavaScript event loop and it's not technically single threaded. But yeah, there's only one event being processed at the same time. So you can very sort of relatively easily do these collaborative editing sort of things where one command comes in at the same time. And then there's this thing that it's really interesting if you ever have this problem and it's not interesting at all if you don't. But what you can do in a durable object is say where this durable object has to run in which jurisdiction. So right now there's two, there's the EU and FedRAMP for the federal government. But you can say this particular durable object can only ever be stored inside the EU. And then no matter sort of what, where else the durable object, all the rules apply, go out the window, like from anywhere in the world, if you can, if you save, if you say that a durable object can only be stored in the EU, it will only be stored in the EU. But the thing is you can specify that per durable object. So for your European customers, you can say only store this in Europe. And for your US government customers, you can say only store them in FedRAMP jurisdictions, which might be one or more clusters. We can use them for multiplayer games. A couple of months ago, we released the original Doom that was built on top of durable objects. We have a blog post out today, a guest blog post where someone built a game where you use your phone to control a character on screen called Full Tilt. It's a blog post well worth reading. But as I mentioned, it's this sort of what else can we build on top of it? That is going to be interesting because it's such a different model of thinking about how you're going to be storing your data and accessing that sort of data. And this is the start of what we call Region Global at Cloudflare, where it doesn't matter where your users are. Compute is going to be extremely close. Storage can be close sort of as well. And it's just so excited to sort of see where Region Global is going. And especially data on the edge is just getting started. Like we're sort of making progress. I'm sure all the other people working on the edge are working on sort of their things. And I, for one, can't wait to see where this will take us. Thank you very much. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. When we talk about bringing new protocols to the web, we're talking about bringing it not to millions, not to tens of millions. We're talking about hundreds of millions to billions of people. Cloudflare has been an amazing partner in the privacy front. They've been willing to be extremely transparent about the data that they are collecting and why they're using it. And they've also been willing to throw those logs away. Really, users are getting two classes of benefits out of our partnership with Cloudflare. The first is direct benefits. That is, we're offering services to the user that make them more secure and we're offering them via Cloudflare. So that's an immediate benefit these users are getting. The indirect benefit these users are getting is that we're developing the next generation of security and privacy technology and Cloudflare is helping us do it. And that will ultimately benefit every user, both Firefox users and every user of the Internet. We're really excited to work with an organization like Mozilla that is aligned with the user's interests and in taking the Internet and moving it in a direction that is more private, more secure, and is aligned with what we think the Internet should be. We're betting on the technology for the future, not the technology for the past. So having a broad network, having global companies now running at full enterprise scale gives us great comfort. It's dead clear that no one is innovating in this space as fast as Cloudflare is. With the help of Cloudflare, we were able to add an extra layer of network security controlled by Allianz, including WAF, DDoS. Cloudflare uses CDN and so allows us to keep costs under control and caching and improve speed. Cloudflare has been an amazing partner in the privacy front. They've been willing to be extremely transparent about the data that they are collecting and why they're using it. And they've also been willing to throw those logs away. I think one of our favorite features of Cloudflare has been the worker technology. Our origins can go down and things will continue to operate perfectly. I think having that kind of a safety net, you know, provided by Cloudflare goes a long ways. We were able to leverage Cloudflare to save about $250,000 within about a day. The cost savings across the board is measurable, it's dramatic, and it's something that actually dwarfs the yearly cost of our service with Cloudflare. It's really amazing to partner with a vendor who's not just providing a great enterprise service, but also helping to move forward the security on the Internet. One of the things we didn't expect to happen is that the majority of traffic coming into our infrastructure would get faster response times, which is incredible. Zendesk just got 50% faster for all of those customers around the world because we migrated to Cloudflare. We chose Cloudflare over other existing technology vendors so we could provide a single standard for our global footprint, ensuring world-class capabilities in bot management and web application firewall to protect our large, public-facing digital presence. We ended up building our own fleet of HAProxy servers such that we could easily lose one and then it wouldn't have a massive effect. But it was very hard to manage because we kept adding more and more machines as we grew. With Cloudflare, we were able to just scrap all of that because Cloudflare now sits in front and does all the work for us. Cloudflare helped us to improve the customer satisfaction. It removed the friction with our customer engagement. It's very low maintenance and very cost -effective and very easy to deploy and it improves the customer experiences big time. Cloudflare is amazing. Cloudflare is such a relief. Cloudflare is very easy to use. It's fast. Cloudflare really plays the first level of defense for us. Cloudflare has given us peace of mind. They've got our backs. Cloudflare has been fantastic. I would definitely recommend Cloudflare. Cloudflare is providing an incredible service to the world right now. Cloudflare has helped save lives through Project Fairshot. We will forever be grateful for your participation in getting the vaccine to those who need it most in an elegant, efficient, and ethical manner. Thank you. Thank you.