Ferrisway: Ramp’s global routing layer on Cloudflare

Hey, everybody. Thank you for attending this session. Today I'm going to talk to you about routing at the edge and some of the stuff that we're doing at Ramp that's pretty cool around the routing setup that we have and the complexity of the system that we've built, which in the end is a complex system but made simple. So, okay, so that thing is not working. Okay, thank you. So this is me. I'm leading production engineering at Ramp. I've been there for a bit more than a year and a half now. And some of the things that we've been started doing since I joined Ramp is part of like the coolest work that we have, which this project is part of. But to get you a bit more sense of what Ramp is and what Ramp is about, Ramp is a company with many customers, with a lot of different currencies that are managed in the platform. But one of the things that it provides to companies is the way to save time and save money and do both at the same time. So basically, we are aiming to be the financial operating system for companies and provide you with all the things that you need in your systems to be able to optimize the way you are doing finance. At Ramp, we're probably one of the biggest customers of Ramp, too. We're dealing with a lot of contracts. We're dealing with a lot of expenses. And honestly, and I'm not saying that because I work at Ramp, but maybe I am in a bit, is like one of the systems that's been the easiest to deal with when you're managing people or dealing with big contracts or dealing with a bunch of systems. And overall, one of the statistics that's not on that slide, but that we talk about is that on average, the customers are saving about 7.5% of their money on the first year they're moving on Ramp and on years after that. So one of the things is that when you have a system like this, you obviously are attracting a bunch of customers. The company is growing crazily. And as the company is growing crazily, we are having increasing number of scaling problems that we have to deal with. And if you imagine infrastructure as basically city building, you end up having a city that you build and you have initially one neighborhood, like kind of like imagine that as like a small village where everything is located and everything is cool until you get to the point where that space is way too cramped. And then you start having problems either about like putting like more people in that city or even about like the roads that are becoming like bringing people to that city. And so slowly, slowly, you start having like traffic like this. That picture actually comes from a street in Montreal that's not too far from my place, which is known to be cramped like this pretty much every day around the same times because everybody is going to work around the same area and everybody is getting in those roads and they get cramped and there is more space. And so if you overcommit your infrastructure, you start having roads that are like this everywhere and your systems are getting slower. There is more risk to your system. And so what you want to do generally is start thinking about, oh, do we kind of like evolve that system? At Ramp, we had the same problem. As I said, the company growing, the company bringing a lot to other companies was like bringing a lot more customers. And so we initially built Ramp in a way that made sense as you're building a new product and you're not yet thinking about, oh, is it going to scale when we get to like thousands of customers? Initially, you build your product for like the first few customers and then you start growing and then you start hitting problems and start figuring out just like how to get through those problems one by one. We obviously grew Ramp in a way that was making sense as we were growing, but a lot of the things that we learned, we learned along the way. The product kept evolving. We started adding new features. We started adding a bunch of other things. And as the company is growing, as the number of engineers that you are working is growing, not everybody necessarily has the same framework about like, oh, are we going to build that new product or are we going to scale it? And finally, we needed to keep up with that demand. So as we were working through that, we were like, what is the best way that we can think about this and how do we get basically to the next step? So this is a very simplified schema of architecture where basically a customer was coming. We had like one load balancer and the generally monolithic infrastructure where everything is. We had like a few services, like mainly one huge database. And the target was getting into a system where we can have multiple targets, right? That kind of approach is basically called like a city or approach. One of the things that it brings as an advantage is that you can keep scaling as much as you want as long as you keep adding like path for reaching your system. Some of the things that we get as advantage from that were related to the targets that Ramp is having about the evolution of the future, right? One of them, as I was mentioning, is also like getting to scale. So if you add more parallel neighborhoods into your infrastructure, you end up having like roads on the new places that are not as used anymore. And you can start basically keeping scaling. You get new customers, you put them in the new cell, and then you have like more scaling on that front. Another thing that we're getting from it is also fault isolation. When you make changes to your infrastructure, as your company grows, as it gets like huge, and as you have like systems that are like basically at the limit of their performance all the time, doing any change gets very scary. You get to a situation where like, hey, I need to do database upgrade. How do we do that in a safe way? And do we have like the right processes that it doesn't affect our customer or that even if we have a maintenance window, that maintenance window is as short as possible. If you have like a code that gets deployed and actually was wrong code that shouldn't have been deployed, then all of a sudden 100% of your customers are affected. But when you get to a similar architecture, you get that isolation where if there is a problem, sorry, in one side of your infrastructure, the others are unaffected because you can get to a point where if there is an issue, whether it's because of the load of that specific system, whether it's because of a change that got deployed there that hasn't been deployed in the other places yet, you can have better observability of what's happening before affecting the rest of your platform. Finally, one of the things that we are trying to do also as we're growing and some of the things that we've announced this year is about extensions in other places. We are looking at like extending in Europe, for instance, and some of the things in there requires also being able to support data residency. So how do you support data residency if you have only one monolithic infrastructure where everything is happening? Well, generally you can't. And so you start rebuilding everything. But if your infrastructure is already supporting like a cell approach, you can just start deploying wherever you want. You can start deploying in different areas of the world and it doesn't like limit you about the geography of your deployments. The main problem with all of that, though, is as you're adding more routes, you're basically getting more navigation concerns. Because if someone is getting to your platform, where do they go? And you generally don't want the customer to be the one that has to figure out where do I need to connect for my system. I do know that a bunch of companies are doing that. I know that there is a bunch of systems where generally depending on what you're doing, you're going to have a different URL that you need to connect to. But I also do think that user experience is very important and customer experience is very important for a company. And you generally want to do things in a way that's entirely transparent to customers. One of the advantages also of making it transparent to customers is if in the future you need to repurpose or reorganize part of your infrastructure, that becomes entirely invisible to your customers. You can move them around to a different cell because that cell is old infrastructure. One of the things that I think probably a lot of you are going to resonate with that, like the holy grail of infrastructure and a lot of people working on deploying infrastructure is the idea of immutable infrastructure, where any time you make a change, you build something new, you move your customers on the other place, and you destroy the old. Because that way you never have the risk of big changes happening on that thing. And so if at some point you're like, hey, I built my architecture using VPCs or like a structure that way, but I decided that I don't want to use VPCs anymore, I'm going to use a gateway, you can do it. Because that's not like a very complex change in one place. This is you build something, you test it, you kind of like start moving customers there, and once you finish moving your customers, you get rid of the old infra. And so if you want to be able to do that, but if your customers are targeting specific address, that makes for very awkward conversations of like, hey, by the way, starting Monday after 9 a.m., please use that URL, but because we're still going to be in migration, please use the old URL for like 10 more minutes, and then we'll tell you when you can start using the new one. That gets very complicated. You're going to start having support tickets all over the place. But if you make it in a way that's transparent to customers, that makes it like way easier to communicate. And one of the things that's challenging in there is how do you handle the whole routing? And so this is mostly what that talk is about, is thinking about the routing process and the work we've done. In terms of the requests that we're receiving at RAMP, we have many, many different systems that are reaching our platform. These are generally the most common ones. Of course, like Webhooks and third-party services, which are things that we don't have control about the shape and form of what they're sending us, and we have to basically deal with how do we handle those requests. Some of them are coming from like bank processing. Some of them are coming from like third-party providers that we're using for different aspects of the system. But we need to be able to understand where is that request coming from, where is it supposed to be going, and how do we route that? Everything that's Web UI and mobile apps, we have a better control over it. So we generally don't want to make too many changes at once, but we have a bit more control about what we're doing for it. So if we want to add like any information in our request, we can do it. That makes it easier. Everything that the developer API requests is also something that's a bit in between because we technically own the contract that we're giving to developers. But as you have developers already using your platform and already calling your APIs, you don't want to change that on them. You want to make sure that whatever they're using today keeps working, and we need to handle that. Everything that's credit card authorization, I put it separately, but technically it's also like part of like third-party services. The main difference is that when you do credit card authorization, you have an extra constraint, which is how fast you need to answer. And so that means that we have optimized system to be able to answer those requests. But if you add a layer for routing those requests that is adding latency to that process, all the latency you're adding in the routing process, you cannot have to make your decision. And when you do credit card authorization, there is a bunch of system that kicks in, including systems that are supposed to be dealing with, is that a fraud request? Is that something that we should be authorizing? Is there capacity for that business to make that expense? Is that user of that specific credit card having a fund that actually is authorized to make that specific transaction? And all the decisions need to be made, and even if we optimize that system, that takes some time, but you want to give that system as much time as possible so there is no risk of expiring, like having a, sorry, like eating a timeout of the request that is made by the credit card providers. And finally, some of the things that started appearing more and more in our systems this year is all the things that are agent workflows, so all our agents interacting with our systems and the fact that there is more and more of agents interacting with systems, so more and more of requests. And everything that's also MCP tool traffic, so basically when an MCP tool is trying to eat a system, but might be eating multiple system at once or trying to kind of like do operations across multiple users or multiple like different businesses, how do we handle the routing for that and how do we make it work? So all of these are traffic that we need to kind of like handle with our system. Part of the constraints that we put ourselves in building this, which is aligned with this request, is that we wanted to be having that system that's invisible, as I mentioned before, dynamic and fast, so we want to make sure that if something moves around or was routed to a given cell at one point but needs to be routed to the other cell, it needs to happen very quickly, it needs to be efficient in happening, and we need to have control over it. Like, it needs to be something that can be reasoned about and can be understood when we're tracking our system. And finally, it needs to be compatible. Because if you understand what I'm saying about all the different ways we're receiving traffic, even the ones that we have control over, the more change you need to make in the system to be compatible with whatever you're building, the more complex it's going to be to roll out that new system you're building. And so if we want it to be efficient, we need to figure out how to route the existing requests in the path that we have. And this is where I'm introducing you to Godair and Ferrisway, which are basically like our two routing services or two routing systems. So Godair is the original service. I'm going to talk to you about it a bit more in a moment. And Ferrisway is basically the very cute service that's kind of the subject of this talk today. Godair is generally the authoritative service that knows about routing and knows about where things need to go. And Ferrisway is its friend that makes everything quicker, faster, more efficient, and closer to the user. Ferrisway has been in production at Ramp since January 2026. So it's been a few months now. And as we rolled it out, it took us a few weeks only to move it to handle 100% of our traffic. Ferrisway is a Cloudflare worker. It's built in Rust, compiled in WebAssembly. And it is currently dealing with about half a billion requests daily. It is dealing with about like more than actually 5,000 requests per second on average. And it's dealing with about 10 terabytes of traffic per day. And it's doing that in a very efficient way. This is generally the architecture that we have for our system. So as I said, Ferrisway and Godair are working hand in hand on the system. Godair is a cloud service that we have deployed behind the scenes. I'm also going to talk about the way it's working in a moment. But Ferrisway is basically the first point of receiving requests. But Ferrisway doesn't know everything. It tries to know as much as possible and it tries to make decisions. So it's extracting the information from requests. It's trying to resolve where that request is supposed to be going. And we have like multiple layers of cache that I'm going to talk about, too. And when it finds a route for the specific request, it's going to be skipping everything else and just sending the request to the place where it should be going. But when it doesn't find that, it's actually relying on Godair to be making the decision. The reason we have that infrastructure is because in every system, you don't want to have multiple source of truth. Because having multiple source of truth means that sometimes you might have different information in two different places, and that gets harder to kind of figure out what needs to happen. So what you generally want to do is having a single source of truth that's accessible from different systems. And so Ferris-Way takes care of routing the information that it has, Godair has the information that Ferris-Way doesn't. So when Ferris-Way doesn't know what to do, it relies on Godair. The way we're dealing with requests, which is good to understand exactly like the different layers that we are using here, is that depending on the request that we're getting, there is different source for the information that we have to grab to decide on where that request needs to go. Sometimes it's a request header, which is generally fairly simple. One of the things that's actually pretty nice at Ramp is that a lot of the requests that we add coming from the mobile application or the web application already are the router indicating which business those requests are for. And so that makes it easier because we can grab that information, figure out where that business is currently located, and route the request directly. But sometimes we have requests that have the business information as part of the request path or as part of the query string. So we also needed to be looking at this and be like, if I don't have the information in the header, I need to look at this in a different place. And so the request path or the query path sometimes can contain that information and I can extract it and then route based on that. But in some situations, I will be routing based on a user email or user email domain based on who is the person that's currently authenticated. And that information can be sometimes on a JED token that would be stored, provided as a header too, but needs to be unpacked to be looked at and then figure out the information in it. And in some specific cases, which is generally probably for Webhooks, like mostly for Webhooks, you're gonna have the information directly in the request body. So basically reading the body, finding the right information and extracting that. And so when you look at all of these different paths, what is happening in the routing, so this is the way Goddard is working currently. We are looking at, how do we extract the routing key for the request? So basically one of the methods I just talked about, some of them are actually a bit more complicated than what is on this slide. But we have about, I think currently about 100 different rules for routing requests based on path, based on different source. But as soon as we have a routing key, we're trying to resolve that routing key as quickly as possible. Remember what I was saying about trying to have the least latency possible on the request? So we're trying to use the local cache of the processes, which will have the information about what has been resolved recently. So we're using a tiny LRU algorithm for that, which basically tries and optimize for things that are currently being routed or requests that are currently being made. Because most of the time, when customers are interacting with the platform or when something is happening, it's gonna be the same keys coming over and over for a short period of time, and it's gonna change to another. So we try to optimize for that pattern. If it's not in memory, we have a layer that can be deployed in Redis. Actually, we're not using that layer currently, but there is the capacity of adding that if we need to kind of like add more caching between the different processes that are running. But one of the thing actually is that Ferris Way kind of removes the need to have that because a lot of requests are actually not coming to our router and dealt with directly at the edge. And finally, we're using DynamoDB as source of truth. So all the changes in routing, all the routing keys and mapping to cells are actually stored in DynamoDB. And this is our source of truth. So Goddard manages that source of truth and owns that source of truth. And every time we're trying to do a resolution, if it gets to DynamoDB, it then gets cached into memory. And using that tiny other approach to kind of like optimize the next request coming to that process. Once we have, sorry, once we have got the information, we then route to the different cells. To give the idea of anatomy of a request, this is what generally it looks like. So this is the different areas that we're looking at when we're extracting a routing key. As you can see, there is different place where that information is. So the more we're looking at the top of the request, the faster we can make a routing decision and the better it is for a proxying routine. Because the way most of our routing works, and it's the same for both Goddard and Ferry Sway, we're trying to get the information of the routing key as soon as possible, resolve where it's targeting, and then the rest of the request gets streamed. So we don't have to cache that in memory, we don't have to keep reading. But until we know where to route the request, we don't know where to start sending it. So we need to buffer as much as possible, so we get the information before routing. That's something that's gonna be interesting when talking about what Cloudflare Workers do. As you can see, everything that's about the pass of the request, everything that's about the query string of the request is generally toward the top. So fairly quick to kind of route, even if it requires sometimes some of the parsing of the string. Everything that's headers is generally fairly easy to access. Everything that's like JWT is also fairly easy to access. And then you get to load everything in the body once you finish loading the metadata of the request. That means that when you look at this, there is different costs to all of the extraction. So the small snippets here are actually accurate snippets in YAML that we use to configure the routing in both Goddard and Ferry Sway because they use the same routing rules. We have one file that is shared between both. And you can see that if I'm using a request header, it's generally fairly cheap. If I'm using the request path query, it's generally fairly cheap. JWT is medium cheap because even if it's in the headers, I still need to do some processing to unpack the JWT and then find the right information inside of the JSON that has been unpacked. So that is extra steps I need to take, which means a bit more expensive in the process. But if I have to look at the body, that very quickly becomes expensive. And one of the things that's interesting is if I'm showing you those two different snippets, which technically are basically the same. There is, however, a very important difference that impacts the routing that we're doing. It's, if you think about the way we have to do that, so I was mentioning it, we're doing like processing of the request. If my client is making a request and I'm getting an answer for the server, the answer is generally going to be fairly cheap because I know which request I've been routed. But for everything that's proxy, there is the buffering step. So if I go back to my code snippets in there, that looks like something like this, right? This is the section that's highlighted is the section that I need to read to be able to make a routing decision. So part of the things that we've tried to also optimize for is that on some requests that we are having the control of the payload, we're trying to see if everything that's contained in the request that we want to use for routing can be put toward the top because the least of the requests I read, the most efficient I'm going to be in routing. And so now that we're thinking about the way this works, this should start to make sense about why having workers deployed at the edge and making these decisions closer to the user are actually making a lot of sense for us because everything I need to buffer also means that I need to request to receive the information from the user before knowing where I'm sending that request. And so I need to keep that directly in my process. And if my user needs to send me a lot of bytes before I start sending that to the server, and imagine in a situation where actually I have a cell that's actually closer to my user than the actual routing decision is made, I end up taking a detour to be sent in the right place. And so instead of deploying routers everywhere to be trying to serve the traffic, using Cloudflare and a Cloudflare worker gives us that presence everywhere. And so everything that's happening, whether it is like transactions that are happening, whether it is like webhooks that we're receiving, whether it is like a user using the web application or the mobile application, we can make the decision as close to the user as possible and try and send that as quickly as possible to the right place. And that works pretty... Sorry, that works pretty much the same way as Goddard is working, except that we're using a lot of the Cloudflare-provided technologies, some of them that I like to say that we're like overexploiting a bit. We are doing the same extraction process. One of the big differences that we have between Goddard and FerrySway is that Goddard is actually reading the YAML at runtime, parsing the request and loading that in memory. For FerrySway, we're actually compiling the YAML and compiling those rules in native Rust right at build time, which makes the worker as small as possible and as efficient as possible in running through those requests. Part of the reason is because when you have a service that you deploy like Goddard, most of the time it keeps running all the time. But when you have a service like FerrySway, which is a Cloudflare worker, there is a lot more times where that service gets started and a lot more times when that service gets stopped. And so you want to optimize also for the start time of the service even more as everything else. If I need every time my worker starts and it's a cold request and I don't have a worker running, that the worker starts, loads the YAML, parses it and then make a decision on it, I'm adding a lot of latency to the request. Maybe not to the follow-up request, but that first one is going to be impacted a lot and that's not something that we want to accept. And so because of that, we compile all the rules directly in the code and the extraction is generally going very, very quickly. One of the things I like to mention about it too is, as I mentioned earlier, if we have bodies that are way too heavy, one of the problems that we've hit as we had deployed FerrySway and started using it for some of the heavier payloads is that sometimes it's trying to look for a routing key and sometimes there is no routing key or sometimes the routing key is at the end of the payload and it happens that some of our third-party vendors are sending pretty, like, heavy payloads. And, well, we add some things that were kind of, like, limiting the size of what we're caching. But at the time, I actually discovered, like, the way Cloudflare workers are being shared between requests and the fact that there are specific times where you can have multiple requests handled by the same worker at the same time, which means that we started having out-of-memory exceptions in some of the workers because they were packing, like, webhooks and trying to find keys in there. So we solved that by basically having, like, a very, very tight limit on how much we're buffering in FerrySway. And one of the things that's cool about having a service like Goddard that can have whatever capacity we want behind the scene is that any request that FerrySway is like, I haven't been able to parse this, I haven't been able to find a routing key, I haven't been able to find what is needed can just be then sent directly to the background process, which then has a lot more capacity to deal with it. And so that process is able to handle, like, multiple megabytes of data without issue, like, even, like, hundreds of megabytes if we wanted to, but FerrySway is not. So we just use FerrySway for what it's good at and then rely on Goddard for what it's not. In terms of the caching, we are also trying to optimize for how do we get answers as fast as possible. And so as a Cloudflare worker is used for multiple requests, we have the same approach where we are using a tiny little redeployment inside of the worker. And so we are using the isolate memory, which, again, gets wiped every time the worker restarts, but it's totally fine. And that isolate memory gets built up based on requests that are passing through. So this is the part that I like to say we're, like, a bit abusing the system because we're really building a cache inside of the worker to make it very efficient, but we're talking about, like, less than 10 nanoseconds for having answers when we're using the isolate memory, which is, like, a huge optimization if we're receiving a few requests coming from the same place. Because if you understand a bit, oh, Cloudflare Workers are deployed and, oh, they are kind of, like, built, as I was saying, there is a lot of cold start because if a user in a specific region is trying to call a worker that's not yet started, there's going to be one worker starting in the specific PUP that they are reaching out to. But that worker, if there is not that many requests, is going to keep being reused when it's needed until, after a while, there is expiration and a new one gets built. But if you think about it that way, if I have one user that's connecting from one specific region, then once I've done the resolution for that specific business, then every other request that user is making in the span of time it's making requests, which, if you load, like, the web UI or even the mobile app, there is a bunch of requests for assets and information that are made at the same time, that thing is going to be super optimized because you get to less than 10 nanoseconds resolution for every request. Then, sorry, as a second step, we're using the cache API. So same thing, we're slightly abusing the cache API because we're, like, using it for, like, caching at the locality of the PUP. So if you have your worker that got restarted because there was an expiration of its use, the next step is basically you start a new worker, but that new worker that starts will be able to resolve very quickly because it will be available in the cache API. So even if there is a cache miss in the memory, it's going to check in the cache API, it's going to have the answer, and it's going to be able to kind of, like, start doing that. And the cache API is in the range of, like, 10 milliseconds to be able to do resolution. So it's slower, but it's still fairly quick to be able to kind of do that resolution. And then when we don't have that information, we're looking at the more permanent cache that we have in Cloudflare, which is using workers.kv. So the reason we're not using, sorry, we're not using workers.kv for every request is because we're talking, when we're using workers.kv, something in the range of 100 milliseconds for requests. So it's still not very bad, but, like, when you compare it to, like, about 10 milliseconds for the cache API and to less than 10 nanoseconds for everything that is, like, memory, that can make a huge difference for, like, many requests made by people. So what we're trying to do is optimize, really, the layers of cache. Now, that workers.kv, as I said earlier, is not the source of truth, right? It's just another layer of cache because we're trying to optimize for Cloudflare having the information as soon as possible. And so what we're doing there is, if I don't have the information in the workers.kv, I'm actually just relying, again, to my authoritative source, which is Goddard. So instead of routing directly to a cell, I'm going to be routing that information to Goddard, and Goddard is going to be making the decision. So when I'm doing my resolution, there's three outputs possible. Either I have a fresh hit, so that means that thing has been resolved recently. I know it's still valid. I'm going to be doing the routing directly. I can have a stale hit, but it's still valid. So we have two layers in there. We have, like, a freshness, but we have also an expiration. If your thing is not fresh anymore but not expired, you can still route based on it, but you need to trigger a refresh, which is basically what Ferris Way is doing. So it's routing the request right away, so we're not wasting any time, but it's triggering a refresh in the background, so then a future request will have a fresh information, and we know that if anything has changed, that will kind of, like, reconvene. And finally, we have either, like, a cache miss, so that information is not there yet, or an information that expired, which currently is very difficult because I think we keep caching information for up to, like, we expire after 24 hours to improve routing. But if this is the case, then we basically hit Goddard to get the information and trigger the refresh of all our cache layers. The way the refresh is working is actually through the response that Goddard is providing to Ferris Way. So we didn't want to add, like, a complex process that's running on top of whatever is happening in the worker. We just wanted to reuse the process that we have in part of the request. And so as part of the response that is going to be sent to the user, that error generally gets stripped. But Goddard, when being it, is going to be making the routing decision, sending the information to the right cell, and when receiving the answer, that needs to be provided back to Ferris Way to be sent back to the customer, the client request. There's going to be that error that's going to be added on the request. That error basically indicates here that, like, for instance, business 1234 should be routed to cell A, and that information is fresh until a given timestamp and expires at another timestamp. That information then gets put by Ferris Way when it reads that error as part of forwarding the response and is used to be updating all of the caches. So we are updating the information in the worker's KV. We are updating the cache API, and we are also updating the isolate memory. So any information that goes back to us gets fresh everywhere. And so that means that any next request, whether it's coming from a worker in an entirely different region, could be using the KV, or coming from another worker in the same pub could be using the cache API, or could be coming through the same worker would be using the isolate memory. So this is what we're doing optimization there. Um, sorry. Um, the other things that we started looking at as part of building Ferris Way was, how do we unify the observability that we have of our platform? So, um, we basically started using, um, like, we, this is like an example trace for a request. In this case, you can see you went through Ferris Way. Ferris Way didn't have the information sent out to Goddard. Goddard ended up dealing with the routing and forwarding that to our application. And so we can see our core API in the end is being used. Um, and the way we build that is actually relying on tail workers. So for tail workers, uh, we are using, uh, well, as I said, we have, like, our main worker that is written in Rust. So we are basically generating logs that are being built, like, put into Cloudflare. And we set up a tail worker that's gonna be grabbing all of those logs and collecting the logs that are in the specific formats that allow us to create spans. And from this information, we are pushing that to our tail collector to be able to kind of rely on traces. Of course, as we're grabbing the information, we're also reading whatever, um, um, sorry, span ID that we get from the request. So we can tail, like, connect that directly to customer requests. We can tail that directly to, like, internal requests that might have been routed again through an external path. Uh, and we have also everything that's, if the worker is not behaving correctly, if we haven't got exceptions, if we have, uh, memory or CPU, uh, like, uh, issues, we are still gonna be sending that to the tail worker. So we end up getting spans for it, and we can trace that that request had a problem and is coming from this. The other thing that we wanted to deal with and that we had, uh, to-to improve on this year is also the -the deployment process. Uh, some of the things that we were doing initially, uh, was just, oh, I'm pushing my new worker. My new worker is live. And then, oh, if there is an issue, I need to roll that back because there is a problem with it. Um, some of the issues that we were having sometimes was also, like, related to, like, cache configuration because sometimes Cloudflare Workers are a bit weird with, like, the way they are dealing with cache even more when you're doing a proxy that makes requests, uh, because they kind of, like, mix the-the server caching and the client caching, and it gets very difficult to wrap your head around. And we've had some issues related to that. Um, and so the way we built this is actually taking advantage of the-the version deployment of Cloudflare Workers, uh, where we can actually deploy a canary with zero percent of traffic. And setting up a special header on the request, you can specifically override... Sorry, specifically override the version that you want to eat with your traffic. So you can deploy a canary with zero percent traffic and still funnel the number of requests to that specific canary that you fully control. And so we can run live tests on our production deployment with all the production set up without any customer eating that production deployment yet because we have full control over it. And so the way we're tracking this is actually looking at, like, having a worker return the version ID of its current deployment, which is, like, a-a binding you need to add on your worker. And when you do that, we basically deploy the worker, keep eating the worker with the specific header until we get the version of the current worker. And once we have that, we know that worker is deployed and ready to receive requests, and then we start running our tests. And if our tests are passing, we can then promote that new worker as the production deployment, which can save 100% of traffic. And if that fails, we just remove it and fail on the deployment. As simple as that. And it has been actually, like, helping us a lot, catching up, like, small issues that we would have where the worker would compile properly, all of our local tests would work, but then something in Cloudflare is not behaving as we expect it to, but now we're able to catch it early. Um, finally on this, uh, I think the takeaways, uh, to -to get from this session are that, um, using Cloudflare Workers, you can improve a lot on your routing and make things way faster than they would be otherwise. You can get closer to the-the customer, and you can still take advantage of the different, like, Cloudflare services to optimize the way you're doing caching, uh, and not directly use a product like Cloudflare initially thought about using it. Um, you still can have, like, an alternative layer which allows you to have a unique source of truth, and that's something that can work, and that's working very well for us so far. Uh, you can still make everything very observable and linking to the rest of your system, which is what we're doing with observability. Um, and you can still figure out ways to have this very safe to deploy and have all the testing done properly. So that's it for me, and, uh, we have about six minutes for questions. Yes. Thank you. applause man in audience, indistinctly man in audience, indistinctly man in audience, indistinctly man in audience, indistinctly man in audience, indistinctly man in audience, indistinctly No, so I actually didn't go straight to Rust, but I didn't go to JavaScript either. Um, the initial ID is, uh, go there, as the name states. Uh, if people haven't caught it, it's actually go there. Uh, like, it's actually a joke between the gopher, because it's a service written in Go, and the go there as a routing service. Uh, and so because that service was written in Go and everything was, like, ready, when I started looking at, oh, do I deploy that in Cloudflare, I actually wanted to use the Go WASM approach, but I realized very quickly that if you try doing Go WASM currently, it gets very bloated, it's very heavy, like, you get a worker that's super huge to deploy, which is also not very quick to deal with things, and that Rust was way better supported natively. Um, I wanted something that was quick and efficient, that was having, like, a very, very quick start time, um, and that could be checked very easily and preventing mistakes by, like, compile time verification, which Rust was lining up better than JavaScript at that time. Yep. man 2 Are you using Rust macros during build time to turn your YAML files into actual Rust? Yeah, I'm actually, like, uh, using, like, a build RS file that kind of, like, reads the YAML, which is also kind of, like, cool, because that means you don't have any run-time dependency on YAML at all. Like, it's basically just turning into, like, native Rust, um, and, um, it's basically generating, like, all the Rust artifacts as part of the build, uh, which then can be tested. And one of the things that we're doing also as part of the testing is we have the same fixture tests that are running on both routers, either written in Go or in Rust, which allows to verify that any change you're making is actually behaving the same on both sides. man 2 Okay. Thank you. man 3 There is the person behind. I think that was before. man 4 Uh, thank you. I, uh, I'm noticing the, um, the read-through cache being a way that you populate all of the optimal routes for things to take, and can you comment on whether it's, like, optimal versus correctness? And then, um, secondarily, if that were to, like, you know, could you talk about some of the edge cases of, like, those read-through caches needed to be completely invalidated or gone? Are you able to handle the onrush of, like, populating those caches? Um, so could you talk about the correctness versus optimal and then the edge case? man 2 Okay. Um, so in terms of correctness versus optimal, uh, one of the things, um, that's probably not always working for every company but that's working very well for us is that most of the time when we are doing, uh, routing, we are, like, we are not having, gonna have, like, routing changes that often. So it's like you're routing a business to a specific cell, you're routing, like, a bunch of IDs that are related to that business to that same cell. It's gonna be fairly, fairly rare that there's gonna be a change in that. Uh, the way we're kind of navigating that right now is with that difference between freshness and expiration that I was talking about where you have that, really, that optimization about, like, correctness versus, like, uh, accuracy versus, uh, sorry, like, uh, speed where you end up having, uh, we can route very quickly because that information was fresh enough that I can route it but I'm gonna refresh to make sure that it hasn't changed. Um, and in cases where we would be moving things around, generally, it's more under our control and the-the five-minute expiration that we are doing for freshness right now is generally fairly sufficient. We have also control over that because each routing key, as we're adding it in the source of truth, we can configure what is the maximum freshness and maximum expiration for it. Uh, we have a default, but, like, we can do it per key which gives us the flexibility that in time of migration you could decide that the key is never fresh and then it's never gonna get cached and it's always gonna eat the source of truth and then when you flip the switch, it's just gonna eat, like, the right place right away. Um, so far, we haven't had the need to use that. I don't necessarily see us having a huge need of using that but it's something that we built in because we wanted to have, like, some of the access. Um, in terms of your second question regarding edge cases, um, I would say that a lot of the edge cases that we've hit actually have been related to how a ramp is built and some of the ways, some of the requests we've done are and some of the interactions that we have inside of our platform. Um, a lot of the complexity that we are having, uh, in building this and getting into a cellular architecture is because we have sometimes interaction between businesses where it's expected that all the information is in the right place and we need to figure out, oh, are we gonna route that across the place and in different cells? Um, and so some of that sometimes gets complex because you end up having a request and be like, where do I route that request? Because I want that information to get there but I also want it to get there. How do I structure this? And I think a lot of that is just because, as I was mentioning at the beginning, ramp wasn't built thinking about we're gonna need that. Uh, it was built as what do we need today and how do we grow that? And now we're getting to the point where we need to solve that problem. And I think, yeah, a lot of the edge case that we've hit has been, like, related to that. Not as much about the routing structure. Um, which, to be honest, has been, like, pretty good. Like, we've had that since January. We've made some changes. There's been, like, new routing keys added and new structure because we've been converting other part of the infrastructure to use it. But most of the core of the process hasn't changed at all. Yep. I think it's probably gonna be the last question. How big is your data set for, like, all the YAML manifests that describe your routing today? And can you give us a sense of the volume and how many schema size and how complicated it all is? So you mean, like, the routing rules? Yeah. Um, I think right now we're probably around 100. 100 different routing rules. Uh, I might be off a bit, but, like, I would say it's between 100, 200. Uh, we have different routing rules, but a lot of them, as I said, like, one of the cool thing in all of that thing was a lot of the requests that we have for internal systems are actually using headers that are specifying which business they're trying to hit because everything that we have under our control already was using that header. And that was actually a blessing because it was not depending on a bunch of things and expecting that. It was, like, actually our core application was hoping to get that header to make decisions, and we're just hooking to that header to make our own decisions. Uh, the thing that have been growing the rules have been, like, other calls that we don't have control over, either, like, the developer API because it was built to be, like, very easy for people to use, but then we have to figure out what are you trying to hit with that or everything that's, like, webhooks and third parties. So that's why, like, given the size and the number of features that Ramp has, it's not that many rules that we had to do because we have, like, one big rule that catches, like, 90% of the use cases. Yeah. Do you expect that to grow as this has become, like, you're now adding more schemas and business units and, I don't know, API methods to your routing scope? And do you worry about change governance on that core routing table? So, yes. It's basically, like, people in my teams that are mostly owning the routing changes there. So we are in the loop and we're making sure things are working and that there is no weird rule that's going through that could be breaking. We also have, like, a proper layer deployment where we're deploying in QA. We have an heavily used QA that developers are playing with all the time. So if there is an issue there and it crashes routing, people are going to realize very quickly. For the size and the growth, I think we've been doing a pretty decent job at trying to find what are the patterns that we can use that are going to be sustained in the future. We've been working with some of the product teams to be like, hey, your application currently you're using this. Can you actually add a business ID in there? And so for some actually third -party vendors, we were able to work with them to have as part of the response, like, whatever field we wanted in the payload. And we were able to inject the business ID directly in the payload that we're now receiving as part of Webhooks, and we can route based on that. And so there is a lot of the work we've done has been also to kind of, like, figure out, hey, we're trying to solve the problem, but we're trying to think about where it's going to go in the future, too, and how do we kind of, like, have some control over that. So is it going to grow in number of roles? Probably, as we add new features and new things, as we have new third-party vendors. As we get to, like, other countries and expand all across, there's going to be vendors that are local that have their own way of doing things that we need to figure out. But is it going to be going out of control? I don't think so. Not at this time. Okay. Thank you very much. I think we're at the end, but I'm happy to chat with you after. Okay. Thank you, everyone. applause applause