How we built Command Code, the most used open-model harness, at 950T tokens

So let's start the talk, a couple of prompts. Carpet is a little bit odd here, so if I fall down, we will turn this into a stand-up very, very quickly, right? And you will just forget everything that happened, right? So today I'm going to be talking about command code. I am Ahmad Awais, founder and CEO of command code. As long as I can remember, I have been a developer. I've been writing code. And it's been a while since I have written code by hand, right? When I was like nine years old, I started writing code. And when I was in my 20s, I had my code running in NASA Mars Ingenuity helicopter mission. So I've seen the entire stack. I'm an electrical engineer with majors in computer science, so I love to dabble between both software and hardware. And that is sort of what we have done here. But I've written like 300 plus open source software, hundreds of millions of downloads. And in my last role, I was VP of developer relations and developer experience at RapidAPI. We got acquired by Nokia for some reason, beyond my understanding. But, yeah, we've done a lot of things. I'm mostly known as master of CLIs. How many people here have written a CLI in their life? Oh, awesome. My kind of crowd. So the story actually starts in 2020. These are, by the way, our investors. Awesome founders that I look up to. Founder of GitHub, PWV, Leader, Seed Round. We raised $5 million. And that, in the blue corner, you see Dean. CT of Cloudflare is one of our angels as well, right? So this story starts in about six years ago. I, because I'm me, had written a CLI to track COVID. I was traveling a lot. I think I had like 44 talks in 2020. And I knew that this thing, COVID, keeps coming up. So I, before everybody, you know, before WHO was like, you know, there's this thing. I was tracking this because, you know, refunds, right? And I got a message from Sam Altman. I didn't know who that guy was at that time, right? Pretty much ignored it. Then I got a message from Greg Buckman. Hey, dude, we are trying out your CLI. This is really good. We connected it to this thing called GPT-3. You want to get access to it? So GPT-3 was like 27 days old, mid-July 2020, when I got access to it. And the first thing I did was I built CLAI. So a CLI with AI in it. And that has now, six years later, become command code. Last year, around the same time, at Cloudflare Connect, I presented the idea of Taste 1. And it was really hard to explain what this model was. It was not an LLM. Now it's running at 100 trillion tokens a month scale. It's the same model, very similar to Jeff. We created it about a year ago. What it does is it learns what our developer prefers to do. And it does not keep any type of data in it. Also, it has a really nice scale divergence loop where the reward signal it has is it's not going to learn things that an LLM already knows, which is the hardest part to get right. In October, just a couple of weeks ago, we launched Agar. So I was born and raised in Pakistan. Agar is an Urdu word. It means ifs. So Agar is basically very similar to Jeff. It's a non-deterministic if of sorts. It's an open-weight model. We open-sourced it a couple of weeks ago. When we did open-source it, it was number one on the CN index, beating Jeff on tool calls. Right now, the Taste thing that we did, the Taste model that we launched, was for humans. And something really exciting happened last year. We stopped writing code. And as a founder of now a three-year -old startup, I was like, you know, I built this amazing thing for developers who were maybe writing 10% to 20% of their code. Now they're not writing any code. So what do I do with this? And I ended up writing a lot about this publicly. Five months ago, we started learning Taste of models. Most models are writing code. It makes sense. This is the entity that now writes code. And we started discovering some really, really interesting facts about different models. We saw that when you're writing Rust code, GLM doesn't really like a particular type of routines. And it repeats that same mistake around 6 p.m. every day, talking in PST, which was very odd. And we figured out maybe it's the peak R, start of peak R, when it has to make a lot of decisions. Something weird is going on in the kernels. But it's an open model, so it's not just one provider that is making this mistake. The open model itself is making this mistake. Like, it knows when I'm under a lot of pressure, I'm going to not do this, right? It was like a new form of slop, right? So we wrote about it, and it became a thing. It became such a big deal that we ended up pivoting our company. Command code became the main product. In five months, we now have over 50,000 customers. We have broken into a space that is extremely saturated, a coding agent space. On paper, it looks like we are a coding harness. We are a coding harness, part of that, right? But what we are doing is we are repairing models. We have the ability to repair models without fine-tuning them. And we do this 3 to 5 million times a day right now. We are processing anywhere from 100 to 150 trillion tokens. It's a massive scale. On a yearly basis, we are at a quadrillion -token scale. I think in five months, we are one of the largest buyers of open models tokens, like DeepSeek and GLM in the world. We have 46 providers, and we are running anywhere from, I don't know, maybe 1,300 to 1,800 B300 workloads, which is, if you had to buy it today, maybe $200 million worth of hardware, right? This is a young startup within five months in an extremely busy space of coding agents. What we have done is we have created a space for us. And now what I think about it is I think we are building Costco of AI, and I'm very excited about this because, again, open source guy, been building open source software. Do not like that all of the AI is being held by entropic and open AI, right? So outside of entropic and open AI, we think entropic is like Amazon. You build something really good, they will come back to you with Amazon Basics product and undercut you. Cursor, cloud code, right? Similar thing open AI did, Copilot and then Codex, right? So the thing that they will discover that you do really well with their models, they will actually do that themselves as well on every layer, app layer, cloud layer, server layer, hardware layer, compute layer, everything. They are going to come and eat your margins, right? So as a developer, as a founder, I wanted to own a little bit more of a moat on every single one of these layers. So outside of these three companies, all the companies that depend on them, like, I don't know, a bunch of coding agents that are built on top of Cloud and zipping margins, right? We are building Costco of AI. We want you to, we want to make tokens so, so cheap that you feel like, you know, instead of feeling like, you know, right now, I think when you're consuming a lot of token, it feels like you are in a supercar and tokens are few and you're burning them, right? Feels good at a cost, right? And I personally want to turn that feeling away from that supercar burning fuel to you turning on, you know, your light in your room, you know? We want to turn that token into such an abundant entity that it feels like electricity. We all pay for it. It's super useful and we are going to make so many different use cases out of it. So again, Costco of AI is what we are doing and we're going after entropic and open AI. We are not actually competing with them at all. We are right now probably the only coding agent that was purpose-built for open models, not for cloud, not for GPT. In fact, 6% of our users use cloud and GPT, 94, 92% of our users on any given day, at a huge scale, like on our self-serve, we are bigger than together Fireworks, based in put together all of their workload on open router. For self-serves, open model tokens, we are bigger than them. So yeah, how is this happening? This, again, I'm not going to bore you with a lot of details, but this is the tweet, the infamous tweet, got like 2 million people who read this. Our founder of DeepSeek tweeted about it. He's like, oh, we should build a harness and they have built a harness and we still beat that harness on independent public benchmarks. I was able to use model repairs to beat, use DeepSeek plus model repairs to beat Opus. Five months ago, this felt like a joke. A lot of people were like, are you crazy? What kind of slop is this? Are you a thread boy who's just writing content on Twitter? And now it's very much believable, right? In fact, two nights ago, I was reading that on value index, one of the biggest security index, the best model to do security work is right now DeepSeek. It is beating GPT-6 extra, right? Again, not going to go into the details of why that is happening. I'm sure, you know, GPT in Florida, really good at premium, right? So how is this happening? How are we actually doing this? Our thesis was there will be so many open models in the future. In fact, you guys in near future will be able to turn all of your workflows and talk to your agent and get it to train a model that is just for your workflows. It will be your model. The only struggle you will have is getting it to work as good as Cloud and GPT. That is what we are building here, right? It's a hardness and inference problem every time you hear somebody say things like, you know, open models take too long. They spend too much tokens. They waste a lot of time. Yeah, I know they are cheap, but they don't really work. What does this sound like? It sounds like crude oil to me. Like you put, instead of fuel, you put crude oil in your car and now you're complaining, oh, my car doesn't sound right. The engine is making weird noises. It's taking too long. The thing is, open models are being trained by so many labs, distillation or not. These labs lack a lot of data to make these models work really, really well. If you read the leaked S1 of Entropic, you would see that they claim that every single Entropic model is profitable. You know, they have, I think, 44 times subsidy on their subscription and they're saying we are profitable here. It's not really a subsidy. It's an engineering innovation of sorts. When you know what your models are going to do, when you can predict what type of workload you're going to have, you can make so many optimizations that you cannot do on the API side, which is why API is so expensive. Nobody was doing this for open models five months ago. We built this company outside of Entropic and OpenAI that does the same thing that Entropic and OpenAI are doing for their models, but we are doing it for, basically, the top 10 open models. And we have stats to prove it. It's pretty silly things like these. Like if you can hopefully read, well, when you're expecting timeout to be null in a tool call, a tool that any of you could have created, DeepSeek would randomly just send null or an empty object and your harness would be like, what is this? And the moment you tell DeepSeek that you're wrong, it's not going to listen to you. For some reason, it has some weird alpha male energy. It's like, I'm right, you're wrong. And it generally takes like 50 back and forth in a session in one million tokens to get it to do what you want to do. I often take an analogy, like think of it like you're training your younger brother how to drive a car and you see he's about to hit a truck. You have two options. You can push the brakes for him or you can tell him after the accident, this is what you should have done. That's the best thing. So what we are doing right now is in Cloud Code, for example, if you use DeepSeek in Cloud Code, you're doing the latter. DeepSeek is a different model than Cloud. Cloud Code is not built for DeepSeek. If Cloud Code tells DeepSeek you're doing this wrong, it's an afterthought. The error has already happened. You probably have already broken the cache. Anything that runs in your cache is 10 times cheaper. One of the biggest, you know, anything that runs in cache is like you're a smooth driver of a car. It's not making weird sounds. So that's just to get you to understand what this is. And these shapes of these errors change per model on the time and the type of capacity and concurrency that GPU is actually running on. So if you go home and you try and you're like, no, he was wrong. It's not going to do that. But when you see, like, hundreds of trillions of tokens, at that workload, you wonder, like, what is happening here? Why aren't these 300? Why is this tool failing 300 times every single hour? And sometimes it's not, right? So these models are actually very undeterministic. It's hard to break into why this is happening. And when this started to happen, I was like, you know, is this a thing? Is this where we are headed? So we reached out to Grok, a SpaceX team. And we tried to give them exactly the same sauce. We're like, you know, can you guys fix this for yourself? They are in our Slack. There's this guy called Elon Musk. He posted about our work with them, right? Which was good. He actually posted about us, and he went to bed. So we were like, you know, 200 million people saw us throughout the night. And at that moment, we were like, oh, shit, this is real. Grok, as a lab, did not know about this because obviously they see a lot less data compared to Entropic and OpenAI. Then MuseSpark came around. I tweeted to Alex, your cache is shit. Sorry, 17% cache is a bad idea. He was like, no, we have a lot of compute. Yeah, I know you don't care. But can we do this, this, this, and this and see what happens? And their cache went back to about 77%. Not great, but, you know, we were able to prove a thesis that this exists. They're going to discover this in a couple of months, and they're going to improve this, right? So again, we pivoted. We started doing the repairs in flight. So this is not a fine tune. This is not predictable. This is not something like you can download an open model. You cannot download this engine. This is what we have built, right? And this exists on so many different layers. It started in our client, which is the hardness, a very clear outcome for that was we became number one hardness on independent benchmarks, especially for two things, the read tool and the shell tool. The read tool is the context engine. It's going to read everything, so you want to make that it is the most token-efficient tool, no matter what. And I shared very publicly all of our research on that. Founder of Hermes actually posted about this. They adopted the same thing, and they are now, I think, doing less, 79% less tokens. So it did really help, right? And then the shell tool. Shell tool is the tool that takes a lot of actions, one of the most important tools. It's going to read your log file, or it's going to do something with your log file. So part of this exists on our coding agent, the hardness. Part of it exists in the cloud, which is mostly built on Cloudflare, which is why I'm here. And a huge part of this exists in kernels. So GPU kernels depends on the type of GPU you are using, the type of kernel, the type of model that is running in the time of day, the type of traffic you have. Again, you cannot really do all of this if you don't have the most amount of data in the world. So Kameen Core is not open source, by the way. I am an open source guy. I was hell-bent on, like, is this a real thing? Because as an engineer, it's very hard to figure out what is the real thing you go and build nowadays, right? Where is the mode? You know, what do I do that OpenAI or Entropiq is not going to come and just eat my cake, right? So I was like, I'm not going to open source this, just like cloud code is not open source. I'm going to make sure that this works because this thing is actually helping people. It's creating a lot of accuracy and it's giving people a lot of, you know, good code. We actually got blamed, like, are you burning VC money and you're sending cloud when we select deep, deep-seek for, you know, tokens? Because it feels like this cannot be deep-seek. It's doing really, really well, right? So we do these repairs. Part of our engine is actually built on workers, DOs in all of the, you know, Cloudflare stack. We are now roughly repairing a model one million times per trillion tokens that we serve off that model. That could be roughly five to eight hours of our workload. Again, number one on most amount of tasks, hard tasks that we are able to pass in an independent benchmark, this is our coding engine. We exist as a CLI, desktop, all the shebang, everything that, you know, you use cloud code for. So this is sort of what we are doing. We want you to enjoy cloud-like performance with open models, with Costco-like pricing, right? And this is how we are actually, I know it's very small, but we're able to give you $100 worth of inference for just $10. It doesn't actually cost us to do $100 of inference, right? That's the engineering harness and inference engineering that I'm talking about here, right? It's only going to improve. I can assure you probably hopefully next year I'm here and I'll be talking about, remember, now it's like electricity, right? So hopefully we are headed that way anyway. And at scale, the numbers are unbelievably impressive. Like a trillion tokens of cloud would cost you about $5 million. It costs about $50,000 or less to do the same amount of work on our production workload with top-end open models in $50,000 or less, right? Okay, I know. Yeah, so we are offering $3 trillion tokens a day. This is all paid tokens. We have a huge amount of free tokens as well. $1 million repairs per trillion tokens, best read and shell tool, and one of our non-star is 98% to 99% of our workload runs in cash, which is 10 times cheaper, which is how you get $100 of inference for $10 without us burning a lot of money. This has amassed a huge community within five months. 200,000 developers use command code now. I know a bunch of you might be hearing about it now. That means you are drinking the Kool-Aid of cloud so far, so I would like you to try out open models. A bunch of companies use us as well. Our coding agent has been downloaded a million times. People think we are doing something good. And all of this is we are trying to turn AI from a privilege that it is today into a right. That was the goal anyway, you know, six months ago, but I recently just read one of the most authentic resources, A16Z's state-of -market report that came out last week that 98% of U.S. households are not going for cloud or GPD, which is mind-boggling for me, right? So I think if AI is super, super cheap and good, we will end up using it for so many use cases. Yeah, that's pretty much it. This is command code. Let me know if you guys have any questions.