AST Parsing with AI： Better UX from Complex Code ｜ Cloudflare Engineering Meetup Lisbon

All right, we've got a full house today. First and foremost, welcome everybody. My name is Ruben. I'm a Director of Engineering here at Cloudflare, and I am super excited today for a couple of reasons. One is because this one, this engineering meetup marks the comeback of our engineering meetups. We haven't had one in like four months or so, I think. And the second reason why I'm super excited is because this meetup is actually happening in something called the Agents Week here at Cloudflare, which is a very important week for us here. It's a week that marks a lot of product launches, a lot of innovation, and something that we established as Cloudflare has dealt place to build, specifically AI agents, but we have a bunch of other offerings. We're going to have two talks today with our Andreas over there. First is going to be Andreas Wenceslao, he's going to talk about AST parsing with AI. And we're also going to have Andreas Jesus, which is going to talk about, and I know everyone is very interested in that talk, but essentially agentic readiness for websites. Please help me in welcoming the first speaker, Andreas Wenceslao, please, a round of applause for him. The mic is yours. Thank you so much. Okay, so first and foremost, welcome. We are going to talk today about AST parsing and how we unlocked a better UX for our users just by doing something that no one wants to do, which is AST parsing. It's just the worst thing that you'll ever do as a software engineer. But let's first talk about workflows. I'm an engineer on workflows. Workflows are durable primitives so that you can write your code with built-in retries, built-in timeouts. And so it's very easy to define an app that doesn't crash or that is very resistant to crashes. It's very easy to build something where even if your fetches, your API calls fail a lot and are rate limited a lot, you can just write your code. You are expecting it to fail and it will eventually work. And so this is a normal workflow, a very small one where we fetch an image, we do something to it. It's not very fun. But what's very fun about workflows is that we can do stuff like wait for webhooks. Say that you want to execute, you are an e-commerce site and you are waiting for the payment to process and then a webhook comes in and you carry on with the execution of your code. And if you look closely, you'll see that here we have a type safe API or now when you are using our bindings. And then below we have a boring REST API, which most of you won't call because you'll use our SDKs to consume the API or use our bindings. But sometimes we Cloudflare have to use our APIs on the dash. And it's not that fun when you have, say, create an instance and pass a bunch of params and you have to do it in a non-type safe way because you might not remember what your workflow takes in as params. And so we'll get to how we fix that or how we can fix it. And first, I'm just going to introduce all of the primitives that we have on a workflow. We have step.to, which is our wrapper to which we pass a callback. It has retries, it has built-in timeouts. You have exponential backoffs, you have constant backoffs, anything that you would want. You have sleeps. Sleeps lets you sleep your workflow for, say, 5 hours, 10 hours, 7 days, up to a year. We also have sleep-untils. So say that you want to run something and then wait for the next 9 a .m., you just do sleep until 9 a.m. and it will work. You also have the wait for events that I've talked about where you wait for a webhook. And so how can we give users a better experience when using our dashboard? If we are using our bindings, it's easy. It's type-safe by default. If you are using the dashboard, it's a guessing game. You have to try and understand what's happening. You don't remember anymore what the params are. Maybe you have 50 workflows, you have 100 workflows. Some people on my team and myself included, we have more than 20 workflows on our accounts. And so sometimes you don't remember anymore. And so for a while I thought about how could we give users and ourselves a better experience. And, well, we had no answer for a while. We could parse the AST, but no one wants to do that. And I personally didn't want to do it for a while. And so we just left the idea on the back burner for a few months maybe. And Celso will correct me later if I lie. But I think Celso asked me for it on the first, like, six months ago. And we just let it stew. And I was like, no, we can't do it. And to be honest, I lied. And I didn't want to lie, but I feel like I was forced to. Because we didn't have time for it. And so when Christmas came and we had more time to ship new ideas and try new things, I started thinking about all of the ideas that we had on the back burner. And one of them was parsing the AST to build what Celso wanted at the time. And, I mean, we selflessly, I selflessly didn't want to build it because it was very bothersome to build. And so we had the idea of having the diagram on the dash for your workflow execution. And so maybe I can try to build it really quickly while we are on Christmas break. But you'll see that you have all of these node types. No one wants to parse all of them. No one wants to look at all of them and see how they map to another node type of our own. And so how could we build this fast just during Christmas? No one wants to parse them. I didn't want to. But there was one thing that I really wanted to try near the beginning of December. Opus. All of you know about Opus. All of you know about GPT 5.4. And maybe with this we could parse them quickly. We could parse them easily. Or we could at least try to. And so I defined our nodes. And you have here all of the nodes on workflows. And I'll start on the right because you know about the steps now. And all of those make sense. And then we have our control flow nodes like ifs, switch cases, whatever else is there that I can't read anymore. Loops. And so all of those are very helpful to show to the user. You have a loop that runs these two steps. And then I started thinking a lot like we probably want to see how much concurrency you have between steps. Because you might end up overwhelming the memory on your workflow instance. And so it's helpful to understand how many things you are running in parallel or concurrently. And then we also have the start of your workflow which is our entry point to the graph. And you have function definitions and function calls. Because if a user wraps their steps inside of a function, they probably know the function and not the steps anymore. And so it makes a lot of sense to show the function call instead of just showing the steps. And so this was my POC, my two-day POC. On the first day, I had the API working barely. And on the second day, I had a very awful looking diagram that I cursed my colleague's eyes with for a while. But let's say that we want to make it look better. Well, the best way to do it is just parse every and all workflow that gets deployed from all users for a while. And I will get a bunch of emails with sentry errors every time something fails. And so I'm going to give you an overview of our control plane, the workflows control plane. And how we hooked the diagram generation into our process. So that we could test everything that the users were doing forever. And now I have this problem that I have to fix all of the AST bugs that we find. It's not that bad, as you'll see in a bit. So when a workflow gets deployed, when the user does Wrangler deploy or uses the API to deploy a workflow. That request, it hits our worker that resolves the API. And then the script gets sent to our account controller, which is a TO that manages an account's resources. Or an account's workflow resources. Then we have our sous-chefs, which are our control plane shards. And then we have engines that are one for every instance of a workflow. And so anytime you spin up an instance, we'll spin up one of these engines. They will register themselves with the sous-chef. And everything works. And I just glossed through a bunch of very hard stuff. But we are not here for that today. We are here for the little line that connects to Vitamix. Vitamix is the name of our parser. For those of you who don't know, a Vitamix is a blender. Like a kitchen grade pro blender. And we have a bunch of puns on our control plane about kitchens. As you can see the sous-chef, we have the Vitamix. We have some other things that we are working on that I can't tell you about today. And so we send all of our requests that come to config service, our worker. We put into a queue the script ID and the account ID. And then we send our Vitamix worker consumes that queue and then does whatever it needs to do. We didn't start with this. This is the final version. This is the good version that's very fast. We first had to have... We had a container and containers are very prone to cold starts. Especially when you are iterating very fast and you want a very quick feedback loop. And so we had the container I think for a week. I rewrote or rewrote partially so that we could have a Rust worker. Which worked great and are fantastic and are super fast. We have all of the niceties of having Rust. And you might be wondering, why Rust? If workers are mostly TypeScript or most workers that you see are TypeScript. Why are we building in Rust? The library that we wanted to use to parse on the SD is called OxyC. It's the Oxidize compiler. It's a Rust builds tooling system for parsing the SDs. And so we had no option. We could only convert the library to Wasm. Or we could just write a Rust worker that is also Wasm. But it's more manageable because you have everything in one place. And so we ended up with that. Every parsed workflow would get sent to either R2. If it parsed correctly. Or it would get sent to Sentry if it fit. And so we had this running I think for a month. So the whole month of December and part of January. Where every workflow deploy would get sent to us. We would parse it. No one knew that we were working on this. Besides the engineers on the team. And after a while the code base got very gnarly. Because we were just hammering new rules. And so LLMs are very good at some things. They are very bad at some other things. They don't like to write simple code. They will rewrite the same code a bunch. Let's say that you have a function that parses an array. They will rewrite the same function. How many times they have to call it. And so we had a big problem here where the code was not maintainable. We couldn't debug it. We couldn't even understand it. Because it was just so messed up and gnarly that we couldn't parse it visually with our eyes. We had to use Opus to understand what Opus did. And then you get into one of those loops where Opus is just doing nothing. And so I did a rewrite to keep the code base slim. And now we will talk about how we reined Opus in. And so I have my three rules. Which are Lint rules, Review loops and Common Sense. Common Sense and Reviews are not the same and we'll look at it right now. So, what's my process for solving a bug after I defined all of the nodes? My process is first, Opus needs to write a failing test. And if it doesn't write a failing test, we won't carry on. We won't do it. We'll create a new session. And we will keep iterating there until we have a failing test. After we have a failing test, we have Opus explain to us what the bug was. Maybe it's something that we don't want to solve. Because a user did something that's specific to them and that no one else will do. And they are requesting something that's just for them. And so you won't fix it. You won't even treat it as a bug. And then after the bug is found, if we agree that it's a bug and that it needs to get fixed. Then we ask it to explain what the fix is. And explaining is just writing the code in a message before actually trying to write it in the code. Because if it writes it once, it will keep reading it and try to iterate on it. Instead of actually rewriting it in a different way. And then the last step after the fix is approved is what I call the review loop. Which is I have a sub-agent, a skill, whatever. It doesn't matter. We just have the agent think about the fix that it did. And search for how it did it. So let's say that we have a step node. And the step can be created from many places. You will look for every place that the step is found. Is where the step is created or the node is created. And you'll see if you have the duplicated code. If the code is the same, then we extract it. We don't want to have that code that's repeated everywhere. And then you find a bug and you have to fix the bug everywhere. And that's where the LLM will forget something. You won't see it because you didn't write the code. And so it's a process. And then you have the human reviews. Which is what I call the common sense. And as you can see the common sense was in every one of these steps. Where we forced the LLM to do what we wanted. By saying you first have to do the test. We have to agree with the test. We have to agree with the fix. We have to agree that it's a bug. And that's where we as engineers still need to be very hooked into it. And so with this, for the month that we were running this. We had our friends from the design engineering team. Build this beautiful UI for diagrams. For the graph that we outputted with the API. And every fix that we had to do was like 15 minutes worth of work. It was just like this account has a bug on this script. Please tell me what happened. And then we just keep iterating on it for a while. Until we have the proper fix. You might be wondering. I talked about UX at the start. And a diagram is not necessarily UX. It's eye candy. Sure it helps with understanding what the code flow is supposed to do. But sometimes that's not enough. That's not really UX. That's a little bit of something that you can give to a user. So that they get a little bit of joy when they open their dashboard. But if you have a diagram. And if you have parsed the AST. You can on the dashboard, on Cloudflare's dashboard. We can give you the types for those create instances that we saw at the start. An API call. That has somewhat of a autocomplete. That's not something that happens regularly. And we can do that because we parsed the AST. We can also, if we parse the AST. Find incorrections where the user. Let's say that the user forgot to await a step. And that will affect how their workflow executes. Because they are not assuming that they have done something wrong. And we can show to them on the diagram. That you have missed something. Are you aware? And after the user says to us that they are aware. Everything is fine. Otherwise they will know that they have a bug. Other things that might get weird. And that are small little UX gains. Are say that you have a million instances a week. Or five million instances a week. We have some customers that produce volumes. And their workflow is complete. They don't error. But maybe one of the steps. Retries every time at least two times. And that's not something that you would expect. But if you write the code. The retries are there to catch our mistakes. Or to catch mistakes. They are not there to be part of the solution. They are a fix for a problem. And so we can tell to a user in the diagram. In one of those beautiful diagrams. We can say this step always retries. I don't know if that's. We don't need to say anymore. The user will look at it. And be like. Oh, okay. We have missed something. I didn't expect this to fail this many times. And so users will get a better understanding of their system. We also get a better understanding of what's happening. Because everything that we show to the customer. We can surface as a metric to us. And then you can also do stuff like say. Enforce rules where. So workflows should be deterministic by definition. And maybe a user did a math dot random. Outside of a step. And that might affect how you execute the code. And so by having these kinds of rules. We can show to the user in the code. You call before this step math dot random. And you are using the variable for a branch. And that will affect determinism. When we have to say replay the workflow. Because something failed. And so what I want you to take away from this talk. Is there are a lot of things. That are not UX. That can lead to way better UX. Just because you did something. And now you get all of these new ideas. And all of these small incremental. Unnoticeable changes keep happening and happening. And your users will be delighted with the experience. That you give to them. And Opus allows us to do this. We wouldn't ever parse an AST. And find the 10 to 20 different nodes. Where a step can be created. We wouldn't track the dependencies. Between steps, between promises. All of this is complete nonsense for us to do. Because we have to make the platform be available and ready. But if it's easy. Well we can just do it. And if you have any questions. Now it's the time to ask them. Yeah. Yeah. So just ask. Hi MX. Thanks for the talk. I'm curious on the workflow of using LLMs. Whether you're using any sort of orchestration tooling. Or you're just using like cloud, cloud directly. Like can you explain probably a bit more. About how you're doing that. Yeah so. I have my own tooling. I mean I think everyone on our team. On the workflows team. Has their own tooling. And we are trying to see what works for each and every one of us. And then over lunch. Most of the times we're like. Oh I don't like what mine is doing. I really like what yours is doing. And I feel like we are all converging towards the same thing. And so for example my flow is. I really like long sessions. But I can only do them. Now that we have 1 million context windows for LLMs. And what I tend to do is. Every time I have an idea. Instead of doing. Creating a Jira ticket. And being like. Oh yeah. I'll get to that when I have time for it. I just do. I do a CLI command. That opens. Creates a work tree. Creates a branch. And I do a prompt. And I get notifications every time. Something ends. Or that it's my intention. You still need all of those. Review loops. It's still. You still have to think about the code that you are writing. Or that the LLM is writing for you. But it really helps when you have. A way to use them. That really fits your workflow. When working. But are you doing this manually? As in you're doing like the build stage. Then the test phase. And you're just like prompting the. Yeah. I still do the builds. I still do. We still do the. But we have CICD. Like you would have normally. So every time you create a PR. And you can have the LLM create a PR for you. That's fine. For example. My tool. And Luis's tool. One of my teammates. They create stacked PR. So that you can build a feature incrementally. Yeah. And every PR has their own CI. You can have your tool. Not the LLM. Actually the tool. Track all the PR. Is getting built. Or if the CI is passing. If everything is okay. And then it auto rebates everything. So that you don't have to do all of that work. Thank you. Yeah. Any other questions? Did you help out there? No, no. It's one of my own. Mine is called. It's called Shirei for commander. Luis's is. Shirei is commander in Japanese. Luis's one is. It's called G stuff. G stuff for git stuff. Because it does a lot of things on git. And so. We end up with all of our little own tools. And then we discuss what we are feeling. That's correct. What's not correct for us. Okay. Thank you. Yeah. Any other questions? Or are we ready for the other Andre? All right. I guess that's it. Round of applause for Vincent Lau, please.