Self-hosting Durable Objects

Well, I'm going to be talking about self -hosted durable objects, self-hosted Cloudflare even, if you want to think about it that way. I'm coming at this from a long career of thinking about JavaScript on the server, thinking about servers in general. I first presented Node.js, my project, back in 2009 in Berlin, to a surprised audience that did not think about JavaScript on the server too much. The demo at that talk was an IRC server. It was shown live on stage and people could connect to it. It was 500 lines of JavaScript, dependency-free, and parser and everything in there. It was shocking to people. Not only was this implemented in JavaScript, which was not a thing before that, but it was event-driven and it was terse. Yeah, it was a nice demo. But there has always been this nagging thing in my mind about that demo. By the way, do you guys know, have you seen this demo? Does anybody know this? A few people. All right, the old people, I see. We'll go to sip tea later on. Something about that demo has always bothered me because this was a single -threaded server. It was a single server, even. As you got more and more participants joining the IRC server, it slowed down because it could just not handle, like any server, there's a limit to how many connections it can handle. And taking that demo and turning it into something that scaled meant re -architecting it entirely. You would somehow need to split connections across many instances. You'd have to shard those connections. You'd have to have some mechanism for those instances to know about each other, communicate between them. In modern chat systems, you would have to have some message history, some state associated with it, some account storage, that sort of stuff. And although it was a great demo back in 2009, it's not really how you would build Discord, right? Discord is architected very differently than that. And I have to say, focusing on server-side infrastructure might seem like kind of a niche thing, but I think this is fundamentally important to humanity, right? Humanity, we are information creatures, right? All of our stuff is online. All of our stuff is accessed through the Internet. Your bank is a server, right? I think how you pay your rent is a server. How you communicate with your friends is a server. How you access the encyclopedia is a server. Everything, okay, not everything is a server. Rocket ships are not servers. But a lot of human infrastructure is actually servers. And it is important that we are able to wrap our heads around what exactly we are building. The state of the art for building server -side infrastructure has improved dramatically over the last decade or even since 2009. But it's still kind of gnarly, right? Like people are just kind of designing all sorts of different systems. You've got different databases. You've got queuing systems. You've got cache. You've got orchestration stuff. You have glue code that kind of ties it all together. You have different types of VMs and cloud services. And yeah, I mean, it's a whole profession kind of navigating that world, tying together these pieces. And I am just struck by the idea that this could all be done much more simply. And I think at Cloudflare, one can kind of pick at the details of various choices that they've made in their programming model. But to a very large extent, I think Cloudflare, Kenton in particular, got the programming model correct in that you are building for scale by default. DurableObjects is a product here at Cloudflare. But more importantly, it's a distributed systems primitive that is incredibly handy and really allows you to build important systems on top of it. So what does it give you? It gives you basically a little server, right? I kind of think of it as a little, like my IRC demo, like a little Node.js server. Of course, it's not Node.js, but it's a server that you can name and you're kind of guaranteed that you can route directly to that server. Handles WebSockets well, which is practical. It's single-threaded, which I think is important. It's easy to reason about. And they each have a little SQLite database attached to it. Previously, KeyValueStore, but these days, SQLite database attached to each DurableObject. And this idea of splitting your system into these little servers, each with a little bit of state, that each handles WebSockets very well, that, of course, normal requests as well. This is a very, very nice design. And this solves all sorts of problems. So, you know, I've been kind of wrestling with new server abstractions for a while. And at Deno, explored all sorts of stuff and built a service called Deno Deploy. But it just never was really what I wanted. And I think there was a long time where I was kind of looking at Cloudflare and recognizing that they were doing things correct, but, you know, had little nitpicks with various details. And I've been playing around with something called cells for longer than celld has been around. But recently, this past summer, I kind of decided, why don't we just implement the Cloudflare API exactly as it is, with the Wrangler JSON and the bindings and everything else? And, you know, really what I want to be able to do is I want to run Cloudflare outside of Cloudflare, in particular, durable objects, but, of course, the rest of it as well. And I, you know, having built this Deno Deploy cloud, I really am allergic to the complexity of cloud infrastructure and was really looking for a way to simplify this as much as possible. So the idea was to take the Cloudflare API and pair it with a storage model borrowed from Lightstream. Lightstream being a project from Fly.io, where they allow you to essentially backup your SQLite database to an S3 bucket. And so celld is kind of combining ideas from these. And, of course, light, you know, kind of goes without saying, but like V8 expertise and stuff taken from Deno itself, right? So we're kind of taking Deno concepts from the runtime. We're taking the API from Cloudflare. And we're taking Lightstream from Fly.io. And what we have then is actually an implementation of durable objects and other Cloudflare services that runs literally on top of a bucket. There is no other cloud infrastructure involved. Of course, you have to run the VMs, okay? So you can run multiple celld instances. And they all communicate and learn about each other through this bucket that they have access to. The durable objects are synchronized to that bucket. And yeah, this was essentially an experiment to see how well this model worked. Of course, this is inspired a lot by the blog posts by Kenton and others about how durable objects works in the first place. And yeah, turns out this works actually quite well. So what I wanted to do is a little demo here and kind of try my IRC demo again. So if you would, we'll see how well this works. Live demos are always a little sketch, a little sketch. So let's see. So you should be joining this channel here. And yeah, don't say any bad words because I think this is flooded. Let me increase the font size there. Across the top here, we've got a couple of different channels. And you have like node A, node B, and node C. These are different VMs. And these VMs are running in a service called Vulture, just a place where you can host some VMs somewhere outside of Cloudflare. Of course, you could do this in AWS. You could do this in JCP. You could do this in Fly. And we've got these various durable objects. Each IRC channel here is a durable object. And I'm using the term IRC very loosely, right? This is a chat system. And we're all in the Connect channel. But I don't know. We could join the Rust channel and say, you know, I love Rust. Or we could hop over to the General channel, which is actually hosted on a different VM entirely, and say hello. And yeah, you get the point. So what I'm going to do now is I'm going to just go ahead. So now I'm over in Vulture here. And these are actually the different VMs. And I don't know how much I should explore here live. But let me just grab, you know, we're all on B right now in the Connect channel. And let me just see if I can SSH into this. OK, there we go. I'm just trying to prove to you that this is actually a system that is running. So this is a VM that's running this Zeldi instance. And yeah, I don't know. You might be able to parse some of the flags here. You can see that it's connected to R2. So I've got three VMs running, all connected to R2. And I'm just going to go ahead and take this node B one here. And stop it. You know, let's pretend this thing just goes offline. And let's see what happens. Did it? Wait, you know, you actually click it, server stop. OK, so node B is now down. And what should happen is that it'll take a couple of seconds. But the durable objects actually get migrated to another node. So now we're actually on node A right now instead of node B. Node B is, of course, offline. And I don't know. Should we try it one more time? Just my tempting fate if I do this again. We'll kill node A and see if we all get migrated to node C then. That one also goes down. And OK, yay. So you get the point. You can run durable objects by yourself. And you can run them on Vulture or other places. Why do you want to do this? Millions of reasons you might want to do this, right? You might have compliance reasons that you need to run it in a specific place. You might be running in a factory. And you need to be very close to the machines at hand. You might have data residency issues. There's just, I think, in the 99% case, Cloudflare is the right choice, right? Like, it's hosted. It's available. There's operators there that make sure everything is running. But there are other situations where you might want to run this elsewhere. And as I said, this is more than durable objects. It's just durable objects is kind of the foundational abstraction on which we build other things. So you have stateless workers, of course. But you also have KV and queues. You can do R2 because, of course, you're just connected to a bucket. We can just store it in that bucket. It doesn't matter if it's R2 or S3. You can do workflows or cron or static assets, right? It's hosting that HTML page, which is, of course, static and delivered through cell D as well. So really, it's the worker's programming model that cell D is providing. So you might wonder, why is Cloudflare doing this? You can go read my blog post with Kenton about this. But this programming, I mean, from my perspective, previous to joining Cloudflare, this is just the correct programming model. This is raising the abstraction layer when most people, most application developers, most server developers should not be operating at the Linux syscall level, right? This is just too low level. It's not to say that there's anything wrong with that. That's where I operate. Nothing wrong with writing in a systems language like Rust. But there's benefits to operating at a higher level of abstraction. There's benefits to GC. There's benefit to not having a file system. You are forced into the right patterns. And you're not sitting there picking all of these random services, trying to tie together, I don't know, Redis and CockroachDB and Nginx and doing all of this stuff. You rather have this well-abstracted programming model designed by Cloudflare, but now available in the open source and ability to run it wherever you want. This is important for Cloudflare's business as well, because they don't want it to be a vendor login situation, right? That's bad for their business, right? You should come to Cloudflare knowing that you can program against durable objects and other services, but you're not tied into the Cloudflare infrastructure permanently. So you are safe to design your application, your business around these primitives with the knowledge that we're putting a lot of work into allowing people to run this elsewhere. Cloudflare has re-trademarked a phrase from Sun Microsystems, the network is the computer. I love that phrase, and I think that that perfectly describes the workers programming model. And I want to see this to be the default way that people are programming servers everywhere. It's worth talking about agents very quickly. The very exciting thing with durable objects is that this is a very cheap mechanism to run a little bit of JavaScript. And what else is an agent loop but just a little bit of JavaScript, right? I don't mean to put down all the work that goes into harnesses. I'm sure there's quite a lot of complexity there, but it's a loop that calls the LLM API, right? And it's very nice if that VM that you're running in is not actually a full Linux VM, but rather something super cheap. Here, I've got a memory comparison against a Node.js process on Linux that took about 44 megabytes of RSS and the marginal overhead of a cell D durable object, which is just under half a megabyte of memory. The memory usage is dramatically different. You can pack these things in very densely. Try to get a thousand Node.js servers on one eight-gigabyte machine. It's not going to happen, but you can for sure get a thousand durable objects on an eight -gigabyte machine with cell D. So yeah, it's particularly exciting because of agents, but in general, this, I think, is the right programming model. So we are working now to integrate, to merge cell D into worker D. Worker D, of course, is the open source version of workers, but it's really for testing more or less, right? It's not that that was the intention. I think the intention was to run your worker code elsewhere, but it doesn't really scale, right? You just get a single worker D instance, and what we're going to do is make it so that you can have many, many worker D instances all connected to a bucket and run many, many durable objects, many, many workers, and really scale this infinitely. It's a bad word, but kind of. So yeah, if you are interested in running durable objects, workers, other Cloudflare applications off of Cloudflare's network, I'd love to talk. And yeah, you'll be hearing more about our work in worker D in the coming months. Thanks.