How Figma runs AI-generated code on Cloudflare

Hi, I'm Pratik Agarwal. I am one of the founding engineers on Figma Sandbox platform that we dearly call Nimbus. Today I'm going to talk about how Figma runs AI -generated code on Cloudflare. Before I talk about our request paths and code and sandboxes, just quickly introducing Figma Make, because I'm going to be using Make as a reference throughout this talk. Figma Make is a platform that allows users to go from an idea to production. So you have an idea, you describe the context, you take it to Figma Design, you polish it, you bring it back to Make, and you can create a fully functional application from it. So that's one of our numerous suits of AI products that we have right now that I'm going to keep referring so that we can relate to what's going on. Quick show of hands, has anyone here written code or wrote code through agents and then waited for a live preview to come back on the browser? Well, that's 30-ish percent. Looks like we have a lot of infra people. I'm also in infra, so I don't do a lot of this, but thankfully with agents, I am doing this a lot and I'm able to do a lot of end-to-end and full stack development. But this has three milestones essentially. You write code or the agent writes code, it sits in the file system, that's the first part. There's a process that runs on top of those files, maybe a binary or a development server, that's the second part. And the third part is connecting this process back to the browser. The browser needs to read what's going on inside a sandbox and then render a UI on top of it. Nimbus helps connect with the help of Cloudflare, of course, helps connect all of these three different milestones to accomplish a user request. Before talking about code, I want to introduce this term called namespace. If you are familiar with Kubernetes and if you've heard of declarative patterns, it's very similar to that, where Figma has a lot of product teams and all of these product teams need to run their own AI workloads. We cannot have each of these teams worry about the infrastructure that powers these AI workloads, because then they're going to spend half the time worrying about that. They want to solve business problems, they want to solve user problems. So you have to establish some kind of a contract between the product teams and the platform. This contract is what we call a namespace. It's a pretty dumb JSON file where product teams describe their intent. They specify what, let's say, the container image location is that will run inside the sandbox or the container. I'm going to be using the term container, sandbox, isolated environments a bit interchangeably. I think all these three things have been used across different companies in different ways, but I'm going to use them interchangeably. The namespace is the contract between the platform team and the product teams, where they define the runtime image that's going to run inside the container, the resources they need, like number of vCPUs, the memory they allocate, the capacity. By capacity, we mean by the number of concurrent containers or sandboxes that might run at any given point. The ports, the services that the browser needs to talk to, environment variables, if any, and most importantly, network policies to control any things that the agent might do, but you don't want it to do. Falling upon that, the first pattern I want to talk about here is allocating isolated compute. Now, there's too many things on those three words, so let's start with isolated compute. Why is isolated compute important? As I said, if you give an agent a task, it will try its best to accomplish it in any manner possible, be it just squeezing out all of your CPU, hogging all your memory resources, talking to an endpoint, and uploading public data or private data it should not. That's why we need isolation, because we don't want your user workloads to impact any other user workloads and vice versa. So, that's the isolated compute bit. Now, how do we allocate it at Figma? We use a term called a routing key. A routing key associates a single user session with a logical entity, like a Figma make file. So, let's say you prompt something today, you do your work, you come back a day later, the routing key is repeatable. We know how to associate this routing key with this particular make file and what container or sandbox we should go ahead and provision. We have this durable storage machine, or durable state machine, I'm sorry, that we have within Nimbus that takes care of this allocation. So, as a user, if you make a request on Figma make, we first mark it as pending, we do some checks, and then we change the status to either provision or fail. Fail is hopefully very rare, it is very rare. We need this state machine inside Nimbus, because as a user, you don't want concurrent requests, a user's concurrent requests to end up provisioning different sandboxes. So, as a user, you go into Figma make, you make a prompt, you want only one sandbox to be always processing it, because if you have different sandboxes, you run into a bunch of split brain situations, which means that there are two sort of authorities for this particular workload, which is like a bad situation. It can lead to data loss, data corruption. So, modeling your provisioning request or allocation request as a state machine is very helpful. On Cloudflare, what that turns out to is the provision request goes to a Cloudflare worker. The Cloudflare worker associates that request with a durable object handler, and then we start the container through this particular durable object handler. A durable object is important here to store the state of the configuration of the sandbox. Like we described in the namespace contract, there are certain things in the namespace contract that needs to be persisted, even if a Cloudflare container like slept and woke up or got restarted for any reason, we do not want to lose that state. For example, a security control that you want to talk to only these number of endpoints, we do not want to lose that information, even if containers restart. So, that's why I think that's the part where we use durable objects along with Cloudflare containers and workers do the routing to these durable objects. Okay, now we spoke about the contract. We spoke about some parts about the allocation path as to what happens when you open a Figma make and then prompt to it. What's happening inside the container? Inside the container, we have a bunch of workloads. So, product teams are the ones that define these workloads, that write code to make sure that processes start up and then the browser can talk to it. We, as a platform team, built a daemon that we call SandboxD that supervises these processes. So, SandboxD is the daemon that runs as a part of the base image of a container or a sandbox. It does reverse proxy, it authorizes requests, it does health check when the browser requests for the health of a sandbox or when the platform requests for the health of a sandbox. So, SandboxD is basically the process supervisor for all the workload processes that are running inside the sandbox. When I say health, a quick note here might seem nitpick, but the health of a sandbox does not necessarily mean the health of every workload running inside the process. So, going back to Kubernetes world, if you've heard of liveliness or readiness, you can borrow your concepts from there to make sure that all the workloads that's running inside the sandbox have this notion of liveness and readiness that you want to listen to from your daemon and make sure you're relaying that information back to the platform so that it can take actions as necessary. Okay, so we spoke about the contract, we spoke about provisioning or allocating a sandbox, we spoke about allocating isolated compute. Now, we have the compute, we spoke about a bunch of processes that's running inside the compute. These processes also include your agent loop, like cloud code or codex or whatever harness you want to use. Now, once you have this code ready inside the sandbox, you need to connect it back to the browser. The browser, let's say, again, in Figma make, when you prompt something, you see a preview. We show that preview in the iframe on the browser. So, the iframe should be able to connect to this particular sandbox. That's what we do through a preview URL. So, once we allocate a sandbox, the URL of the sandbox, which is like a public identity, is available to the browser. That's how the browser knows how to establish, let's say, a long-lived web socket connection with this particular sandbox. So, this preview URL has the container or sandbox identity encoded in it along with the service name. The browser knows this URL. Once it calls this endpoint, it goes to a Cloudflare worker. The worker can decode this public URL and figure out the exact container identity, associate the durable object handle back to it, and then call the sandbox at that particular port that the browser wants it to talk to. That's why durable objects are important, because you need to persist all the state, even if users are coming back, let's say, a day later to resume their work. A little bit of code. I don't want to go super deep into this, but I think this is the part of the code which ensures that both HTTP and web sockets both work with this preview URL or the public URL. Page load is not the only part. Once you open Figma Make, let's say, for an existing application, you will be able to see the preview of what you did before. But you also want to edit. You also want to iterate. You also want to build on top of it, and you want to keep seeing your changes as you build on top of it. So, there are two parts. One is the HTTP part for the page load, and one is the real -time interaction you're having with your sandbox, which you don't know is a sandbox, but you're having a real-time interaction with the sandbox, and it's being rendered back to the browser on the iFrame through that one single public URL. Okay, so now I'm going to quickly show a Makefile. This is a prototype thing we use all the time. I'm going to refresh this page so that all the junk here goes away. I have a filter for the subscribe endpoint, which is the web socket connection. The Figma Make browser here is establishing with the sandbox. Eventually, there is a control plane in the middle, but eventually, it is talking to that sandbox. Inside this subscribe, you can see the first call is for a workspace ensure, so this makes sure that the workspace is created. It gets an acknowledgement. You can see the Cloudflare sandbox ID here. You can see the application URL and other metadata. We have a warm true flag that I'm going to talk about soon as a part of my next slide. Now, so these two calls here make sure that your sandbox is now ready and up and running. What next? Next is the thing that you prompted. We need to create a workspace for it or workload for it, sorry. So, this is the workload. We register a bunch of different watches for your particular workload. So, the moment you prompt in the ready sandbox, we create a workload for your particular context that you added. Now, this workload registers, the browser registers a bunch of watches with the sandbox. We call it watchers, watches. It's basically something that is always talking back and forth between the browser and the sandbox, so we call it like a watcher. There are a bunch of watchers here. You can see this preview watcher. This preview watcher is the preview URL I was talking about. So, you can see the status here is stopped. The workspace status is bootstrapping. Somewhere down here, it says the status is starting. And finally, oh, we send a lot of messages. We have the status as ready. So, now once the status is ready of a particular workload, you will see that the URL here, the public URL is provided as part of that last message or that WebSocket message. And this is the URL the browser knows now it needs to talk to, to connect the iframe back to the sandbox so that it can render whatever code you want to see on the UI. Super small demo. We can go back to the slides. All right. Now, okay, we spoke a lot about, so we started with the contract. We spoke about why isolation is important. We had an isolated environment for our agents to run. We routed to that environment. We connected back the sandbox to the browser. The two milestones are done. Now, this is the provisioning part. Like, how do you manage and allocate all these sandboxes? Figma today runs tens of thousands of sandboxes across 15 different or 20 different AI workloads. That's why we needed this, you know, common shared platform because you don't want all these teams to go ahead and start hammering Cloudflare or any other provider out there. So Nimbus is, when we built Nimbus, the first couple of, for the first couple of use cases, we directly provisioned a sandbox when a user request came in. Over time, we found that to be unstable. Latencies were not super deterministic. It could take between five seconds to 12 seconds at P99. Reliability was a bit of an issue. So we decided that we're going to, like, just keep a stock of sandboxes all the time, which we called a warm pool. So for every contract, initially, I mentioned for every namespace, we have a warm pool where we stock a bunch of sandboxes ahead of time. So your container bootstrap time is gone. Your health check time is gone. Everything the application needs to do, we are doing ahead of time. And once a request comes in, we just pluck from this warm pool or pick from this warm pool and associate that user specific metadata to this particular container, which is less than a second. So you bring down your container bootstrap time from 10 seconds to less than one second because you have this background stock of sandboxes always available. Now, you have a stock of sandboxes, but they should be usable. They should be not useless. So how do you know that? So we also have this background lifestyle reconciler, which is constantly health checking all these sandboxes all the time, be it in the warm pool or be it in an actual user path. And if it deems unhealthy, it can just kill it and recreate new ones. So the third pattern here is the warm pool is, like, really, really nice if you want if you care about latencies, if you want to reduce provisioning latencies, you can have, like, this pre-provisioned stock of sandboxes. One more clever thing we do is if you go to Figma make and if you stop typing something, you will see that the sandbox is already provisioned for you. So you don't even have to, like, wait for a user to actually submit a prompt and then start the process. If a user starts typing, you get sort of a signal that they want to do something eventually. Stocking sandboxes is all good, but it also costs money. Like, I can stock in, like, a million sandboxes. Cloudflare will be happy to do that for me, but then I have to pay for all of that, right? So you need to come up with, like, the right balance of availability and cost. You having all these sandboxes helps with both availability because, let's say, Cloudflare's control plane has a blip. You have the stock of sandboxes you can rely upon that you can just pick up and then give it to a user. And it's also fast. So you always have this balance between, like, cost and availability and latency. The way we, you know, balance that cost is we measure the demand to the warm pool. How much demand is coming to this particular warm pool for this particular namespace? And based on the demand, we calculate what we call it a drain rate. But based on this demand or drain rate, we calculate how many sandboxes should we always keep. We keep it always between the minimum and the maximum that the product teams have told us to keep because they also care about cost. And then we add a CFD factor and a time threshold in our calculation for, like, how many sandboxes should always be there. So it's a simple auto-scaling algorithm, but it's a very simple equation currently in this algorithm. Interestingly, I did, like, prototype seven or eight different auto -scaling algorithms by, like, just having my agent emit a metric for them, like, in shadow mode by reading, like, a bunch of papers. But this turned out to be the simplest and the most efficient. So we just went along with this. No amount of, like, other auto-scaling research was needed here. All right. So taking a step back, we spoke about three patterns. The first one we just spoke about was have this background, warm pool, stock of sandboxes always available that, you know, helps with availability and latency of user requests when they're trying to provision something. When a user actually requests something, just bind your logical ID to the sandbox and keep them isolated. And then finally, you have to connect the browser back to the sandbox or the sandbox back to the browser that we do through the preview URLs or the Cloudflare URLs with the help of Cloudflare Workers and durable objects. Yeah. So these are just the three patterns. Again, adding it, like, small bullet points. Cloudflare was really helpful for us in this process. I have, like, two or three engineers sitting here that I message them maybe thrice a week here at this point. They were really helpful. Great customer support. You know, I think one of the things that infrastructure vendors should always remember is customer support and customer obsession really goes a long way. I feel Cloudflare did a great job on that. Native feature support. So having programmatic access in front of the sandbox really helped us. So, for example, we can inspect every packet that comes through the Cloudflare worker that helps us, you know, let's say separate put requests versus a get request. You can be more liberal with your reads versus your writes. Egress control. Like, just giving them a set of endpoints that the sandbox can talk to without, like, building a bunch of stuff on our end. Custom domain support. Like, as a customer, if you're using Figma, you don't want to see, like, random URLs. You want to see a trusted URL. So I think these kind of features were just natively provided out of the box from Cloudflare. And that was, like, really great. Yeah, that's it for my session. Your feedback is really important. I would love to hear how you've enjoyed the session or if you've learned anything from it. I am Pratik again. And thanks for your time.