Why modern software engineering needs Durable Objects

Hello, everyone, and welcome to my talk all about how modern software engineering needs durable objects. In this talk, I'm going to walk you through what I'm considering the renaissance period for durable objects. Durable objects have been around for a long time, but they solve some of the unique challenges that we're facing today with the way that they are across the entire world. A little bit about me. I'm Ashley Peacock. I joined Cloudflare around six months ago to work on durable objects, but I've been building with durable objects for far longer than that. I probably built durable objects maybe back in 2022, around when they came out. So I've been building with them for a long time. Before we dive into durable objects, though, I want to talk a little bit about what we're seeing in the industry. I'm sure everyone's going to be familiar, but I think it's important to reiterate because then we'll talk about how durable objects can help solve these kinds of problems. And I want to come at it from two different angles and present two different problems that durable objects solve. In the top left, you'll see this was this month from Cloudflare. The AI agent traffic grew by more than 1,700% in the past year, and that kind of surge is pretty much unprecedented on the Internet, and that presents some unique challenges for all of you building your applications. This isn't just isolated tech, though. On the bottom left, you'll see traffic trends, AI visits across different industries, so it's affecting many, many industries, all the ones that you are from here. And on the right, you'll probably be familiar with this graph. I'm sure you can guess what company is on the right-hand side. But this is GitHub's availability since the AI wave. And just to say, like, they had an impossible problem to try and deal with. Like, this is such a hard problem. They're at such huge scale. And if your application takes off, you're going to have the same problems, and Durable Objects is here to help solve and make those problems easy for you, and you can focus on doing what you care about, which is building the best applications for your end users. So traditional applications are typically built with large pieces of shared infrastructure, which comes with some challenges. So let's say you have a centralized database. Let's say you break it down a bit further. Let's say you have a database per microservice. At some point, if you have, let's say you go viral on Hacker News, for example. Everyone's plugging in your APIs, your CLI, whatever, into their AI agents. They're going to pound your application. They're going to pound your database, whether it's a database, whether it's the containers that you're running on, whatever you're running. It can be really hard to scale those. Databases in particular, if you need to shard that because you just can't keep up with the load, that's not something you can do overnight. And that's the same for containers. They're a little easier to scale, but with how expensive memory is, disk, et cetera, it can be hard to get the availability that you need on demand. And that ultimately leads to one failure potentially taking anyone down. I could right now spin up tens, hundreds of AI agents, hammer all of your services. Will it scale through that? Maybe. What if all of you in the room point all of your AI agents at the service that I've built? Is it going to handle that? Maybe. I'd rather not find out. And that's why I would build my application with durable objects because rather than one failure taking everything down, we can isolate the problems much more nicely with durable objects. That's problem one. That's keeping up availability of your services in the era of AI. But everyone is also running more AI agents. And everyone needs a place to run their AI agents. On the left, this is from Databricks where they saw a 327% increase in multi-agent systems in just the past four months. When AI first came on the scene, you had chatbots, one agent, and so on. Now everyone has these multi -agent systems where if everyone in this room is running three agents and they're all spinning up sub-agents, that's a lot of agents we need to run. And on the right -hand side is a report from McKinsey, which is showing orgs using AI, which increased from 20% to 90% between 2017 and 2025. So this number is only going to go up. We need somewhere to run our AI agents that's economical, doesn't cost a fortune. You can spin them up on demand. And guess what can do that? Durable objects, of course. So for those not familiar, the way the really short sentence for how I describe a durable object, it's a small but mighty server with its own database. It can handle up to 1,000 requests per second, which is maybe not huge. But when we look into how you build your applications with durable objects, you'll see that's typically more than enough. Rather than having a shared database, each durable object has its own SQLite database. So you can see on the screen you have user Alice and user Bob. Those could be two durable objects. They each have their own database. And they're completely isolated from one another. So if Alice is doing something and she's hammering her database, Bob's totally fine. It's not going to bring Bob's database down and all the other agents that you're running. They'll be totally fine, because each database is isolated. That means you get no lock contention. So if I'm doing a huge write to my database, it's not going to block all your reads while that write completes. And a really nifty thing is the database runs on the same metal that is running your durable object. So that basically means it's all local between your app and your database. So on average, we see reads to your SQLite databases on durable objects typically handled in less than one microsecond, which I'm pretty sure most databases are not operating at one microsecond. And lastly, and a key when we're talking about surges of traffic from AI, is that it scales pretty much to infinity. Obviously, not quite. But for all your needs, it's going to scale as far as you need, because durable objects underpin so many products on Cloudflare. We are the biggest users of durable objects. And something like 20% of the Internet runs through Cloudflare. So if we can handle that much traffic, your app's going to be just fine. You'll be able to spin up as many durable objects as you need to handle your traffic. In terms of what a durable object looks like, if you've never seen one, it's on the right-hand side. And it looks like a typical class. It extends the durable object from our SDK. And you write it pretty much as you would write any other class. It's obviously a little bit different, because rather than being local in memory, the durable object is going to run on our data centers. And you call it from your worker or wherever you want to call it from. You define methods on your durable object to implement your business logic. And then at the very bottom, you'll see that's how you call your durable object. Now the important thing to know about durable objects is that when you give it a name, so in the bottom right, we have get by name room 42, that is globally unique. So even if you have 10 places that you call this durable object, they will all be routed to one instance of your durable object. And we'll ensure there's only one instance of that durable object anywhere in the world at any one time. So it's perfect for coordination. When working with durable objects, you have to change your mental model a little bit. When you're talking about shared databases and shared app servers, everything is shared. Durable objects are completely different because everything is isolated. Let's say, for example, everyone in this room wants to run one agent. They would each be, you would each have your own individual durable object. If you want to run 10 agents, you can each have 10 durable objects. Because like I said earlier, it scales. You can create them on demand. They're not provisioned ahead of time. They are just dynamically created whenever you need them. We'll spin them up and handle the rest for you. On the left, that's the kind of app, the kind of share of the old world where maybe if you need to scale up, you can buy bigger machines and so on. And you also need to plan for capacity at the peak. Let's say you have a rush of traffic most days at 12 p.m. You need to design your application to have enough capacity to handle the peak at that time. With durable objects, they scale down to zero. So if I have an agent, I send it a request, it deals with my request, and then it just goes to sleep. It doesn't cost you anything to run when it's sitting idle outside of any storage you have stored in it, which is the cheapest part of a durable object. And the key thing here is that a busy customer cannot take down all of the durable objects. So if you're all in this room, we're running your durable objects, and I do something crazy with mine, spin up its database, all of its CPU, you won't notice anything because it's just going to affect my durable object. As with anything, there are limits. So each durable object has up to 10 gigabytes of storage in its database. As I said earlier, it has 1,000 requests per second, and it can have up to 128 megabytes of memory. Now, you might think 128 megabytes of memory isn't much, but when you're considering that rather than servers, you're having these tiny little objects, 128 megabytes goes very far if you're just having a single durable object per conversation with an agent, for example. So if we link this back to the problems I talked about at the start, we basically break those shared pieces into lots of small ones. And small doesn't really do it justice. They're kind of tiny, tiny, tiny parts if you think about what you would break apart from your shared applications. So rather than one database for everything, you get a database per object. Rather than everything scaling together, so your containers and so on spinning up and down, you spread your load across as many durable objects as you want and as you can. And in a minute, I'll talk about some of the strategies you can use to pick how you shard your durable objects. And rather than one failure taking everyone down, small objects limit the damage. As I said a few times, I can wreck my durable object. You guys are all going to be fine. You're not going to be interrupted. So you might have one bad user or whatever having a bad experience, but that's better than your entire application being down for everyone. Solving problem two. So we need to run loads more AI agents. We have the agents SDK from Cloudflare that's built on top of durable objects. And I like to think of it as a batteries included approach to running AI agents. It's completely kind of kitted out to integrate with our entire developer platform. So if you need to throw something off asynchronously into a workflow, there are integrations for that or queues, whatever. And it's also very opinionated about how it implements things like WebSockets and subagents, scheduled tasks, tools, and MCP. Everything that you need your AI agent to do is included in the agents SDK. And it even has things like you can build MCP servers with the agents SDK. Whether it's stateless or stateful, it will do both of those for you. Now, you might be wondering, why did durable objects solve the problem of running so many AI agents? And that's because we run isolates. Isolates are really tiny, small pieces of compute that isolate just your code. But when you're running a container, for example, which is a traditional way to run an agent, an isolate is roughly 100 times more efficient than a container. It will spin up way faster and uses roughly 100 times less resources. That means we are able to scale to meet the demands of all the agents that we're going to need to run in the future. As I said before, building well with durable objects requires a slightly mental shift. You have to think about it a little bit differently. So I'm going to talk about some of the common pitfalls we see. And then we'll move on to oops. I'm going to talk about some of the common mistakes we'll see. So the hot singleton is the most common one you'll see. So I'm going to take an example application. Let's say you're building a Slack competitor. You might think, oh, I'll make a durable object for each of my Slack servers. But if you have 1,000 users, 10,000 users on that Slack server, that one durable object is going to crumple because it can only do 1,000 requests per second. It just can't handle that kind of load, and it's not designed to. So you want to break it down smaller. You could have a durable object for every single channel, for example. That's breaking it down smaller. It's going to depend on your application. But the way to think about it is think about your hottest path and optimize for that. Break it down as small as you can while making sure you don't have too much coordination between your durable objects, because otherwise you get this chatty interface, but they're always talking to each other. So it's a fine balance between it. And you can use strategies to take things off the hot path. So we have workflows, we have queues, right? Take those off the main path of the durable object, or even have other durable objects that take your asynchronous work. Looking back at the Slack example, rather than, let's say you have a message go to a Slack channel, right? And you have one durable object for your channel, and there's 1,000 people in that channel. That durable object still has to broadcast that message to all 1,000 people over a website. If you have high throughput, it's probably not going to keep up. But you could easily have one durable object that takes the right durability when the message comes in, and then have 10, 20, 30 durable objects that are responsible for fanning out those messages, where each person in the room would connect to a shard, for example, that's effectively like the message broker that's just broadcasting to everyone. That's how you take load off, and you avoid the hot singleton. Another one is accidental placement. This is common if you are migrating from, say, AWS, or Azure, whatever, and you're still making calls from your back end in AWS to your durable objects. Now, the way it works by default is we will create the durable object close to where the request originates from. So, if all your requests are coming from your back end in AWS, all your durable objects are going to be close to whatever region you're running in. That's probably not what you want, because you want your durable objects to be close to the end user. So, if I'm running my agents in San Francisco, I want my durable objects to be there. I live in London, so I would obviously want my durable objects to run in London. But everyone needs their durable objects to run somewhere differently in an optimal way. Durable objects also support WebSockets out of the box. It's the easiest implementation of WebSockets I've ever seen. It's like four or five lines of code, and that's all you need to handle the WebSockets. They're actually quite a complicated protocol to handle. But the key is you need to use our APIs when you're using WebSockets. Make sure you call acceptWebSocket on it, because we have a really neat feature where once a WebSocket is connected to a durable object, it's able to hibernate, as I said earlier. It can go idle, but the WebSocket connection will stay open. We won't charge you anything for it, literally free. It doesn't cost you anything to have the connection open. You only get charged when your durable object receives a message, it wakes up, and it handles the request. Then it goes back to sleep, and so on. So it's a very efficient way to run your applications. Lastly, memory starts fresh. So as I just talked about, your durable object can hibernate, it can go to sleep. The memory of that durable object, so anything held in the properties of the class, they will be wiped every time it goes to sleep, and every time it wakes up, it will be fresh. You can use the in -memory storage for ephemeral storage, temporary storage, caching, things like that, stuff you can afford to lose, it's very, very good for. But anything critical, you need to make sure you write it into your database. Now, durable objects were already very, very good. But we are making them even better for the world that we're seeing today, particularly when it comes to running AI agents. A bunch of these features are not out yet. They might be available, but they're not publicly available to you. Some of them are deployed, and they are bedding into the system, we just haven't announced them yet. So this is the first time that anyone has heard of some of these features in this room. The top one is about pending IO. So let's say you fire off a prompt, chat GPT, for example, then you close your app. You're through the back end in that case. And in the past, that would mean your durable object would just stop doing anything after a little bit of time. But now that durable object, when it's got pending IO, stays open for up to 15 minutes. Every time you open a new request externally, you get another 15 minutes. So that means you can do these long running tasks without your durable object being evicted from memory. And we're also adding a shutdown hook. So as I've said a few times, your durable object can just shut down, because it's idle or whatever, you want to deploy some code. And now we're giving you an opportunity to take some action before that happens. So let's say you're putting something in memory, you want to flush it, you can do that with the shutdown hook. This one's really cool. You can now ship code without interrupting your applications and agents. So in the past, when you ran a code deploy, it would disconnect to WebSockets, which kind of sucked. And then your clients would reconnect, as is the protocol, but it's just not a very good way to be. So now your WebSockets will migrate from your old code to your new code gracefully with no errors on the client. And equally important, when you're doing a deploy, you're now able to run your old code for up to 24 hours on that durable object. And when it goes idle, we will then update to the new version. If in that 24 hour period, it doesn't get idle, then eventually, we will kill it after 24 hours to update. But that gives you a much longer window, because it used to be 30 seconds. This will be completely configurable. So you can do up to 24, you can do an hour, 30 minutes, whatever suits your application, or you can just force them to restart immediately. Last but not least, we're trying to add some resiliency features to the platform. So there, because of the network is distributed, you can get disconnects at any time. Sometimes it might be within our network, but a lot of the time they might be external to our network. Let's say someone cuts through a fiber, right? All the requests going through that fiber cable are going to be disconnected. And this is typically handled by a retry loop in your application. Now, we're automatically retrying those requests for you when it's safe to do so. We have measures in place to make sure we're not double executing your code. And you'll be able to opt in some of your methods to double execute if it's idempotent. So the takeaways. Hopefully, I've convinced you that you should be using durable objects, not only for your applications, but also for your AI agents. Because rather than running these big shared services that are going to crumble if you get flooded with AI requests, the durable objects will just scale to meet the demands of your application, if you follow the things I've said today. And for agents, it's going to become really expensive to run agents if you're running containers and so on. So make sure you use durable objects for their cheap, stateful execution. Break those durable objects down into users, rooms, whatever works for your application. And just get the benefits from durable objects. That's the end of my talk. If you have any feedback, please give it on the app. I have time for two questions. Then I have to go to a custom meeting. But I'll take two questions. Thank you, everyone. Thank you. Thank you for the talk. Durable objects are great. We're starting to use them. Question for you. Running at scale agents for users, would you recommend having, say, if you have millions of users, have each one of them have their own durable object that wakes up whenever they need it? Or would you actually do some sharding, as you suggested, where you actually put in a few hundred or a dozen or a thousand users on single objects, right? I think it depends what your application does. But I would break it down smaller. Because let's say your user in the future wants to do three parallel tasks. If you have one durable object per user, it's not going to be able to do those tasks in parallel, because durable objects are single-threaded, so it can do one task at a time. So I would have it per task or per conversation, something like that. There's no real downside to making it smaller, as long as your durable objects are not just talking to each other all the time. As far as you can, really. Any more questions? In the example of, let's say you're building Slack on durable objects, or Chachapiti or something, in this case, right, each channel is a durable object. But then what's handling the lists of channels, or the organization settings and such? Would you have a coordination durable object that is shared across the organization, but handles many less requests? Or how would you handle that sort of state? It's up to you. So you could use D1 is great for those kind of configurations. That's a good place to put it. You can have a coordination durable object, but it's going to depend how many requests it needs to handle. You have to think about that hot path, right? If it can handle the amount of requests it's going to get, you can put it in a durable object. But I would try and avoid chaining durable objects. That's normally not a great idea. You want it to be kind of a horizontal scale line, if you like, where all your durable objects are on the same roughly level, rather than having one durable object and then going to another one. That doesn't normally end too well. Makes sense. Thank you. We can do one more. Not a question, but I just want to say that our rolling releases are like the greatest thing for new objects of all time. Yeah, I think it's going to be a popular one, I think, yes. Do you have any idea when that's rolling out? Some of it has rolled out, so you might see the improvement already in some cases. But the user-facing changes haven't rolled yet. Look for the change log, because it should be soon. When you're migrating from containers to durable objects, what are some challenges you might face for protecting secrets that you have set inside of environment variables? We have a containers product, and it has a feature that is escaping me. It has something like an outbound proxy, some sort of proxy that is effectively like a worker. You can store secrets encrypted in your worker. What happens is, whenever your sandbox makes calls externally, where you need API keys and so on, it goes via the proxy, and you can basically sub in the secrets there. And then when it comes back in, you can pause the response if you want to take the secrets out, if there's any secrets in the response. That way, the container doesn't have any secrets whatsoever, because it's using this proxy. I forget the name, so I can't remember the name. But if you Google proxy on sandbox, it will come up with something like that. A lot of enterprises have a very large container footprint, a lot of big APIs, isolated databases, stuff like that. How do you take the concerns that are baked into a large client API and distill that down into something that is composed of durable objects? There's a lot of different pieces. The secrets were mentioned, connecting to different data stores, integrations with other services. Are you writing the durable objects in a similar way that you would the API? Or is there a whole other pattern that you have to look at with how you compose those? Durable objects aren't publicly accessible. You have to go via a worker, for example. So you'd implement your API in a worker. And then if you need coordination, state, et cetera, that's where your durable object typically comes in. I would put the API, if you have a big public API, in the worker. If you have MCPs and stuff, the SDK will handle it. It can either do stateless MCP, which is the new way to do MCP, or it can do stateful MCP as well. And then if you need to connect it to queues or workflows, other databases and so on, there are a range of products on the developer platform for that. So let's say you have a Postgres database or a MySQL database that you just can't move. We have Hyperdrive, which basically pulls the connections on our side. And it just makes the connections really quick, because obviously the worker is going to go up and down. And reestablishing the connections over and over is not good. And then, yeah, there are other products like queues and workflows. So I think you can gradually migrate those things over to Cloudflare. For the new system one models that have come out recently, would the latency be better if the durable objects are making calls directly to those than if you were doing it outside? As in pulling the models hosted on Cloudflare from the objects? Potentially. I haven't tested it, so I don't want to say one way or the other. But in theory, our GPUs run in our data centers in the same place that durable objects and workers and everything run. So in theory, yes. But durable objects are like region scope, so there's no guarantee you get. It depends how busy a GPU is. It's not quite the same. Because if the GPU near you is busy, we're going to send it somewhere else. So yes, maybe. I don't think there's going to be a downside to it. But I can't guarantee that you'll get a one millisecond latency because it's on the same data center, for example. And I'd assume there's no pitfalls to doing that sort of thing. There'd be no pitfalls to durable. No, I don't think so. Thank you. What's the actual difference between a durable object and a D1 database since they are both SQLite? So what are the use cases? So under the hood, D1 is just a durable object. But it's got some nice UX on top of it in terms of the API to integrate with the queries is a little nicer. It has things like read replicas, which gives you a little more throughput. But durable objects give you more flexibility. Because to interact with D1, you need a binding to a specific database. So if you spin a database up on demand, that's really hard with D1 unless you use workers per platform. But you can do the same similar thing with durable objects and just kind of use it in the same way that D1 does. So I think if you want flexibility, I would use durable objects. And then you can use D1 for configuration and more like a central database that is shared. That's kind of fine. Thanks. I think we're done. Thank you for coming, everyone. Thank you.