Giving an agent 10,000 tools with Code Mode

So we're going to be going over code mode, a little bit of the history of it, and then also beyond code mode, what it really unlocks, and then scaling tools of agents. A little bit about myself. I'm Rhys. I'm the founder of a company called Executor. It's a personal cloud where you connect all your accounts, and then get all your accounts and integrations into, and then you build whatever personal software you want on top of that. Cool. Let's dive into it. So I think what you'll notice is basically your agent needs to be able to do everything you can, and there's a couple of ways to accomplish that. One is like compute use, but you blow up your token count. One is kind of like raw tool access or bash, but really there's like a better way, and that's code mode. You get a typed capability layer over all of your tools, and I think there's an interesting trend you'll notice once you give people agents, and you see them doing more and more through them where they basically no longer have the patience to go click through websites anymore. I'm sure many of you have experienced that yourself where you're kind of just like you sign into a website now, and you're like, give me the API, give me the MCP. I don't want to click through this dashboard anymore trying to understand your product. And so, yeah, the agent that they have needs to be able to do everything. And so people try to give their agents everything, and at one tool it's fine. At five tools it's still okay. At 100 tools, about half your context window is dedicated to tool names, input output schemas, and really like imagine you're, you've got to put yourself in the mind of a model. Imagine you wake up, and you immediately have half of your memory stored with tools. And then at 1 ,000 tools, it doesn't even work with the context window. And so you kind of look at how people overcame this, and the first one that really stuck for people was Cloud Code. And when Cloud Code came out, everyone had this kind of like perception that was like, wow, the model only has one tool, it has a bash tool. And we just go, wow, CLIs are the best. You just want, you want to give your model as few tools as possible, give the model less tools. If you're on Twitter in February, there are all of these posts and articles about like, we need to design CLIs for agents, and everything needs to be a CLI. And I think what was most interesting there is if you kind of like think about the problem from first principles, you haven't actually given the model less tools when you've given it a bash tool. You're just shifting the execution environment. And what I mean by execution environment is it's a place for the agent to find the tools that they're looking for, chain tools together, and then filter the results to preserve their context window. And so when you look at bash, the way the bash kind of works is you get dash dash help, you get progressive disclosure, you get the ability to, you know, put a cat, like run the dev server, and then put the error into like a cat, and figure out what the result was, and then chain those tools together. But there's like nothing special to bash there. Like, there's many other ways you can express that. And one of those was code mode. And so it's kind of like, what are the problems with bash? How can we do better? Bash feels like it works great. And I'm actually not going to deny that like, there's a lot of times where I myself, I'm like, just give me a CLI. It works great. But you're like, when you depend on bash, you're kind of like, you think about the functionality of most CLIs, you're introducing a full Unix shell and all of the permission requirements in order to just call an API, which is crazy. Along with that, it's super lossy. So if you consider, you know, the GitHub CLI, you can't tell me exactly what is everything in the GitHub CLI without leveraging agents and parsing to basically go through all of the help menus, figure out that command structure. There's no kind of standard. And what's great about code mode is it solves all of this. So you get, you convert all of your tools to TypeScript. And then because you convert all your tools to TypeScript, you've got an index catalog of tools that you can search over. And then as you're scaling agents, especially the search matters even more. You also know what's returned from the calls. If you've ever watched the agents output, back when we were kind of in the single threaded agent world, you would notice that this pattern a lot where they would run a tool call, and then they would run a tool call again after they understand the shape to filter the output. And, you know, you don't really notice this as a problem today because all of the kind of time is spent on inference. But as models get faster, having to wait that 300 milliseconds for a model to do the same call twice, especially if it's a destructive tool, wastes a ton of time. And so the last part before jumping into a demo here is kind of like going over a few options of like this code mode and where to run it. So you can basically convert all of your MCP tools, really any tool type, OpenAPI, GraphQL, whatever you want, into TypeScript. And then the model can just write TypeScript to call that. But in order to do that, you have to run that TypeScript somewhere. There's some interesting options here. So one of the ones there is OpenCode. So what OpenCode did to implement their code mode is they wrote an interpreter. And so they don't actually run any JavaScript for their code mode. It should be called interpreter mode. And what they did is it just interprets the JavaScript, means that they don't have to ship a runner for it. Another one is Dynamic Worker Loaders. They were kind of made for the code mode use case, but there's some other more efficient ways you can use them for code mode. And so with Dynamic Worker Loaders, they're a lot faster than like a QuickJS, but with the trade-off that you get billed on unique billing. And then the last part is looking at something like a QuickJS. So for those of you that don't know, QuickJS is this pretty minimal JavaScript runtime. And it's really perfect for the code mode use case, as it can run in Wasm. It's super embeddable. Something like Py, it leverages QuickJS for its code mode. But it has a little bit worse performance and a worse security trade-off. And then kind of like looking at one implementation of code mode, I'm going to walk through a quick demo here of a product they built, Code Executor. Really, it's not too much on the product. It's more just looking at like, if you give the model an untrusted place to run untrusted code, you can build all of this functionality on top of it. Cool. So jumping over to the demo. OK. So I've set this prompt. We'll see what it returns, because I'm telling it to leave out the personal info from my emails. But it's got my Executor MCP connected. And while that runs, I'll just quickly show what like tools this has. So this is the Google APIs. And I think what's interesting on code mode is you can actually separate it into two parts. You can separate code mode into the actual running of the model. So the model will like write cool here, and it'll write like a path to call and how to call it. But the other place you can use code mode is actually like, if I click on the source tab here, you can see that like, if you're familiar with Eve, it's a very similar authoring syntax where you use JavaScript to define a tool. And you get a much more flexible implementation. So you kind of deploy these little mini apps. They have tools, skills, workflows attached. And a bunch of functionality on top of that. And so, yeah, it's found some emails. And then the next thing I kind of want to mention is basically getting a bit more into how this works under the hood and what you can do with it. So rather than thinking of MCP as just this like, you're reliant on the upstream MCP, and that's it. You can actually think of MCP as like one place to run untrusted code. And so if you have the primitives of storage, webhook handlers, skills, et cetera, you can basically start from a super minimal MCP and continue to add on on top of that. So one example that I'll do is I'll say, let's see, attach storage to the read email tool so I can take notes on the emails. And what I will do is I'll go ahead and write a migration file. It'll start making it so I can take storage. It'll extend the tools to have an input output on that. And what you've done here, you've essentially been able to leverage code mode and code execution on top of this to extend the functionality of what a base MCP provides. And so we've got ultra fast on, so it'll speed through that implementation. But talking about a little bit more basically about how registration tool would work here. So this is like one implementation of it in executor, but you can kind of take the concepts anywhere. So the model gets basically like a Git source to write to and it gets the ability to define an app. An app takes in requirements and then it outputs tools. And so if we output anything else in this one, we also output instructions on how to use it. And so it takes basically the Google API spec, converts it into a set of tools here. And all this just comes from being able to let the model write whatever code it wants to integrate with whatever it needs to. And so, yeah, it's found the app, it's running commands. And while that also runs, I'll talk about one more benefit of basically being able to let the model write code. And another one of that comes out to one of the biggest pain points that people have with kind of MCP and being connected to things is how many of you know the pain of when you go to call something and you're signed out or you don't know if you've used the authentication correctly? Like it's a constant hassle with this. And so what's super cool about just letting the model write code is here is part of this framework. It's written defined provider. In this case, it's defined in Google as a provider. This hosts says where the token's allowed to go. And then for authentication, it says here's the OAuth and what scopes are involved. And so you get this nice mix between trust at the model level, you get this nice mix between trust at the use level where you can kind of see high level, OK, the API token is allowed to go to the right hosts. And then you also get this like ability to have it define a health check where when you connect, you actually know that like, hey, these accounts are working and it can pull the names automatically. So it's going and running now and it's updating the read message and read thread. And then you can also actually see under the hood a little bit here some of that code mode execution. So, yeah, it's like found the path, found the signature for it. It's written TypeScript. And this is actually merging the two methods that I mentioned of dynamic worker loaders and OpenCode's code mode interpreter. So for all of my like dynamic code mode, it goes through the interpreter. It's a much more like lightweight form of being able to let the model run TypeScript for talkers. And then for the like highly repeatable ones of extending out MCPs and apps, it's going through the apps framework, which runs on a dynamic worker loader. And the model, again, writes code in the same way you'd write an XJS app and deploy it to Vercel. And so it's deploying that now. And after that deploys, I just have it show that code at a high level. So you can kind of get a feel for it. Cool. And so while that deploys, the last thing I'll mention is like thinking of things rather than like MCPs, skills, webhook handlers, UI as all being separate concepts, you can just think of it as one bundle of code. And so your skills can actually be defined in code as well. And so I'll show you an example of like defining skills in code, which is super cool. And so like here, if I look at the source of this app, you can see, hey, his skills, it's defined as dynamic skills. And rather than like having this super rigid thing that's like, do I load from GitHub and then like have to make a lock file format? You could just say, hey, load from this from GitHub, always at this SHA. Or in this case, like it's dynamic skill fetching, but it's going from the well -known DOF, which is super great. And then that becomes an index skill catalog. And so then back to this, we'll just take a look high level at like how you can, what code it wrote. And yeah, so it's just written these tools. So let's say it wrote read message, took the existing one, and now it's returning the notes where it just reads that from SQL. And it's just like bytecoding an extension based on untrusted code execution. So now swapping back over to the slides. There we go. The second, this is kind of the second part, which is like tools as a cloud is kind of what I'm thinking about it as. In the same way that you would deploy and kind of a website to VSL, you can deploy code mode and these extensions to MCP servers to your like personal cloud. And I'm really excited about this because you get all of this expressibility from the untrusted code execution. One example is like, you know, what if you want a custom approval amount? So any refunds over $20 is fine. So any refunds under $20 is fine. You can just define that as a rule that's like, hey, with approval, if the argument to that function is under $20, let it run. Same thing with like if you want to ping your phone for a 2FA approval, you can just put in whatever custom code you want. And again, the notes on emails demo, it just extends it with context and the model writing SQL. Along with this, there's like kind of different gateways you can use. Another one in the gateway world is Cloudflare's MCP gateway, where you can move all of your tools, your permission handling outside of the harness and sign into that one gateway. And then all of the harnesses you use have the same access to tools. And you can also, again, leverage code mode for that like one unified harness where it's able to script over all of the tool course you have. And then on the last part about what I mentioned about Bash and CLIs, again, they're just super lossy in terms of, you know, you've got this great information about if APIs are destructive or not. We've like had open API specs for, you know, 20 years now, but like threw all that out the window in the move to CLIs. But now with code mode, we got to get it to come back. And so, yeah, just overall, code mode lets you have a way more token efficient way of calling tools rather than Bash or raw tools. And then moving your policies outside of the agent and into a gateway lets you rapidly adopt any harness without having to worry about how it's implemented. Great, thank you so much.