From intent to infrastructure: Rethinking CDN resiliency with AI

My name's Alex. Thank you for attending this talk today. It's called From Intent to Infrastructure Rethinking CDN Resiliency with AI. And the word of the day is going to be intent. So I'm going to say it a lot, and I had a slide, but it got messed up. But the definition of an intent is like, what do we agree upon as the outcome? And so what we're going to focus a lot on is how we use intent and this framework that we're building around intents to define CDN behavior. So a little bit about me. I've been a Cloudflare for a few years. I focus on our strategic segment of customers to deploy large CDN architectures onto our network. All right, so let's take a step back and define the problem statement that we have right now. Managing CDNs at scale is hard. It's hard for three, boiling it down three reasons. First is migrating complex configurations from one CDN to another has a lot of layers to it. The second is that oftentimes what I have seen in practice is that there's a concentrated team of engineers who are deploying these configurations for hundreds of application teams. And the third is when you have a multi-CDN setup, having parity between these different CDNs. So, and then another quick aside is that this talk is actually going to be around a specific segment of our customers who have multiple CDNs in place. And so we're going to be focusing on the operational side of that. But anyways, of these three challenges, by show of hands, who has these challenges today? I want to see. All right, very cool. So we're all in the same boat here. All right, so taking a step back at the problem statement, why are these actual challenges for us? First is that over time, CDN configurations kind of become the default specification where we have years of behavior defined as rules. And then there's also provider defaults baked into those rules as well. And those kind of, you know, those configurations plus those defaults is the CDN behavior that we come to expect. And so when we go and patch or investigate that, we have to reverse engineer kind of the configurations that we have in place in order to figure out what is the actual behavior that you need. The second big problem that we have is around ongoing maintenance around CDNs and just edge configurations in general. There's this one, you know, we can put a lot of effort up front into kind of the initial deployment. So day zero, where we might take a set of configurations and then migrate them to or build an equivalent Cloudflare one. And that's like a one-time project. It ends at a point in time. But then there goes the transition to this phase where we have this ongoing maintenance that has to take place. And that transition to like day one and beyond is a whole set of challenges in itself. So we got two challenges. One of them is that the behavior is kind of locked into a specific CDN configuration. And the second is that it's very hard to update and maintain over time. So the kind of big idea that we've been working on with some customers is kind of separating out that behavior from the desired CDN behavior outside of the specific, outside the CDN configurations themselves. And so that's where we call this idea of intent and then this other idea of tests. So to go through kind of this framework that we're creating, an intent is like a requirement that captures the expected behavior. Tests are how we define that behavior will be observed. And then from that, that will be our source of truth on the left side there. But then from there, we will go and generate maybe a provider specific configuration. And so what's neat about this is that the same intent can guide multiple providers for us, where when we have kind of these requirements and tests separated, then we can actually use that and put it into AI, which is really good at generating configurations. But it's good at making these configurations when we define the specific behavior that we want. And so in this mentality of instead of just directly configuring a CDN, instead, if we can go outside and define these intents, define these tests, and then use that to factor in the types of configurations that we want to build. And what's really neat about this is that after we feed it into some models to do the translation to, so we have the intent and tests, we put them into the, you know, whatever configuration translation tool we might build, then we get a configuration out. But then we can use those same tests that we started with and run the new configuration that we've generated as kind of a feedback loop back to the thing that initially created that configuration. And so what was neat is that when we were in the session earlier this morning, Eric, you who was presenting on the test and kind of the infrastructure that he outlined, he outlined the same kind of things that they're doing at Void Zero, which is like building the test infrastructure that then you can then iterate deployments on. So it's a very similar kind of pattern that way. And under the hood, actually, we're using the technologies that he and his team built. So then from there, so actually, to take a step back, the whole kind of test-driven workflow that we have here then is that at the first step, what we're doing is we're going in and defining behavior. And this is led by a very human-centered approach where we can go in and say, these are the types of things that we want cached, or these are the types of headers that we want added, or whatever configuration you have. Step one can be augmented with AI tools as well, where we can take an export of a configuration and then have that analyze it and say, this is the type of behavior I expect from that. So first is defining kind of a collection of the behaviors that we need at a CDN edge. The second is that then we take that behavior that we've defined explicitly, and we generate a test from that behavior. And these tests have assertions in them that say, for this type of behavior, this is how you measure it, and this is the output that I'm expecting. And so that is what we've also found with our customers is using AI to take the behavior and then recommend tests to validate that is a pretty quick workflow. And then humans go in and then do that approval of it. And then finally, we can take that behavior and those approved tests and then build that configuration layer. And then when we have that output from there, we can validate the behavior of it. And if there's any modifications that we need to do to that behavior, we can go back to step three and build additional configurations and changes that we might need. And then finally, there's a way to capture like a report around the evidence that we want to collect to validate and show that a test has or that a configuration is valid, and that we can then make a promotion decision around sending that to production. And so we at Cloudflare have built a tool called CFSpec that applies this model, this testing framework, where we take out the behavior and the test and then create a framework around and then use that as an input and then create a Cloudflare configuration and then run that kind of feedback loop. And so CFSpec turns human readable requirements into reviewable tests. Users can approve tests to generate Cloudflare configuration. The third is we can validate behavior in a dedicated development zone at Cloudflare. So I'll go to the next slide if we want to switch over to the demo real quick. Sweet. So now we're looking at my screen here. All right. So I'm going to go through this video and pause at a few points. So first, we're looking at the, this is the tool that does that translation layer for us. There's several components of this. The first is the configurations themselves. The second is reports. So after we run the test, we'll have evidence that shows how the test ran and the types of, and whether or not the configuration passes it. Fourth is the requirements themselves. So that's the intent that we had talked about previously. And then finally is the test. And so then CFSpec takes each one of those components and maybe generates the next one. And so if I proceed here, we'll see that the first step we're going to do is modify one of the requirements. So I have a very simple example here, and I'll pause this. So right now, what we see is under here, it says requirements, response header test. And then the requirement is Cloudflare must add the response header CFSpec test with the value false to every request on a zone. And then the context is, this is where we're actually going to test that particular endpoint. I'm going to change that requirement and turn it to true. I'm going to save the file. And then the next step I'm going to do is say CFSpec sync. So sync is now evaluating that test or that new requirement, and it's deciding whether or not a new test needs to be generated. And this will take a moment to run. And then we get an output here. And under here, our tool will then go through all the requirements and then build a respective test for that individual requirement. And right here, we can actually look at, I think I highlighted in the next step, you'll see that the assertion that we're making actually is for a get request on this particular path, return with the response header of CFSpec is true. And then this is the actual code behind the test framework that is verifying that that header is present. So then we're going to go in, and this is kind of a critical step, which is that we want to approve the test. We reviewed and said, this is the behavior that we actually want. So we go ahead and click approve. And so at this point, we have the requirement and the test, and now we're going to generate our configuration. So what's happening behind the scenes now is that Cloudflare is taking that behavior definition and the test. We are using LLMs to generate Cloudflare configuration. And then right here, I'll pause actually, what we're doing is we are taking that configuration that was generated and actually applying it to a Cloudflare zone, a development zone. And so we generate that configuration, we're applying that configuration to a zone. And then this tool will then go and run those tests against the zone that we just deployed the new candidate to. So right now, the Terraform is being applied. And then we are running the test. And I'll pause it again. So the test ran, and it passed the test. And we can go in and do a little more detail here. But the configuration candidate was found. Three files were changed. And we have a bit of information here where the report is around how the test actually performed and the requirement that we had and whether it passed. And what's neat about this is that we can actually define how many loops of the validation tests that we do. So if we, in this instance, it passed on the first one. But if it failed in that scenario, it would take that feedback, give it back to the generation phase, and it would use that new feedback to iterate and then build a new configuration. So if I proceed here, we'll see that we can then review the candidate. And we'll see the diff on it. We'll see that here's the new one. And in fact, we'll see that the Terraform here does say that the CFSpec test header is true. And then we proceed. And then we can go and review the Terraform file itself. And we'll see actually just the same thing we just saw. And you'll see the same thing with the value header true there. I'll skip ahead a little bit here. And then the final step is that we can actually run the validation step independently. And that will generate a report as to how each requirement was able to pass in as far as CFSpec. So what's happening here is it's taking the configuration, reapplying it to the zone, running the test, and then generating a report. And then that report is just a nice little readable document here. So going back to the slides, then. We'll wrap up in just a minute. One sec. Cool. So we'll go back here. And so what this framework gives us is it addresses kind of ongoing challenges that we see with operating CDNs at scale. First is that it gives us this portability around when we preserve expected behavior across migrations and when we preserve the behavior in the test that we want, you can then actually take that and maybe apply it to a different provider if you are in a multi-CDN setup. So having these as independent things allow it to be portable. The second is that using this framework gives us a lot more scalability in that we can help automate the feedback loop around changing behavior and growing it from there. And the last is it's verifiable because each test is able to be approved in an individual way. So guided by explicit intent and approved tests, we can use AI to shorten the path from intended behavior to infrastructure at CDN deployments at Cloudflare. So the one key takeaway I want you to have is by using natural language and then having a test suite behind it, we're able to build this whole other infrastructure that helps us configure things like Cloudflare. And this can even extend beyond CDN. We use this a lot because multi-CDN setups are common with our customers. But if you have other types of applications, you can do the same kind of feedback loop in order to give our tooling better data on how to improve a Cloudflare configuration. So thanks everyone.