Your agent wrote it. Who’s on call?

Thanks for coming to our panel. We've got a short amount of time here, so we're going to jump in and go rapid fire with the questions. But excited to be joined by three esteemed panelists, and we're about to jump into the latest of everything happening right now in the Code Gen era, and how that's impacting DevOps. So I'll just quickly start off with a quick data point here. A recent survey by JetBrains showed that almost 50 percent of code is now being written by agents, and you have frontier labs like Anthropic now disclosing 90 percent of Cloud code was written by the agents. So I would love to just run down the line here, just have each panelist just quickly introduce themself, and then tell us do these stats line up with your internal usage, as well as what you can see at your end customers in terms of how your end customers are using code agents. We'll start with you, Dane. Yeah. I'm Dane Knecht, Cloudflare CTO. I think at this point, almost I would say by 98 percent of every MRPR at Cloudflare has some AI involvement. Whether that is planning, or generating the code, or the review. At this point, it's pretty much ubiquitous across the team. So yeah. Madison? Yeah. So hey, I'm Madison, Head of Forward Deployment at Factory, which is building autonomous software factories for enterprises for the most part. It's an important station because we are also building coding agents for organizations. So we also run on a software factory. So I would say we tend to try to build and dog food that system as much as possible ahead of driving that out to organizations. I was actually able to look into our data, and we started off the year around, I think, 30-40 percent of agents driving most of the code base and the changes. Now, I think we're around 60 -70 percent. I think that percent being as high as possible is not always the target, right? I think the goal is really to have good verification systems and really trust to the agents that are making changes on your behalf. So I think we're doing that in a really healthy way, and we're trying to teach organizations how to do that as well. Chris, how about Accenture? Yeah. So my name is Chris Wilder. I'm the CRO at Accenture, so the go-to-market organization. So I've never deployed code to production. So by default, I'm the safest engineer in this room. So at Century, we probably are in the same 50-60 percent range. We have a whole bunch of people using it. Our founder is fanatic, so he's probably at 98 percent utilization. At least what we see, so we are essentially an observability tool for developers. What we see is that the bottleneck has clearly shifted not just from writing code, but to the reviews, and then in production, what happens and how do you monitor the sheer amount of code that's being generated, right? That's kind of what's kind of visible now. So I got so excited by the questions, I didn't get to introduce myself. So I'm Aaron Jacobson, partner at NEA, a multi-agent venture capital firm, which happens to have been an early investor in all three of these companies. That may give you a sense of how this panel came together. So let's double-click into how this rise of code -gen agents is actually really impacting workflows. You obviously have an increasing volume of code being written by agents. You know, Madison, you are an AI -native company, you get to build things from scratch. I mean, how are you seeing this impact other workflows? How is it impacting CICD, review, you know, the whole development chain when you start to have this kind of volume of code being written by agents? What's breaking? Yeah, so a fun story that Aaron is very aware of is I was actually the lead investor in Factory about a year and a half ago, and underpinning that investment was a belief that the biggest bottlenecks for agent -native software development would not be the IDE and autocomplete, but more about the organizational context and ultimately the software factory. And again, that's what Factory is now building. And so after seeing that and recognizing that a lot of code would be driven by agents in the future, I recognized that one of the next pieces that would really break was CICD, because we have a system in agentic DevOps that, or sorry, in traditional DevOps that was really not built for agentic software. It is extremely difficult to use at rapid pace, and it does not handle bursts and spikes very well. And it's very difficult to take out lots of different versions of the code base and be able to merge them back together. So we're seeing GitHub obviously break in terms of reliability, but also just the process through which it works for agents. And that's why we're starting to see a lot of these CICD acceleration companies like Namespace and Blacksmith start to take off because they're really making it so much easier for agents to use existing tools in CICD Runners. Another space that we're seeing a lot of is code review and bringing autonomy to code review, making that a really agent-native process. That's something that we have a lot of internal tooling around, but ultimately identifying the opportunities to drive more agent -native code review, but then also find ways to surface where should this actually become autonomous and how and under what circumstances is the hardest part. So, Dane, how does that line up with what you're seeing at Cloudflare given the amount of code that agents are touching? Do you agree with CICD in review or what broke first for you guys? I mean, absolutely, the review side. I see some Cloudflare engineers in here, I'm sure that would agree. Obviously, writing the code is the cheap part now. It's the planning and the review part, you know, the bookends of it that really still need human input. And so, you know, we've put a lot of effort into building a review agent that helps at least speed up that process, that encodes how we like to build at Cloudflare, kind of takes the DNA of Cloudflare and taught an agent how to understand that, taught an agent to look for what our past incidents looked like, to spot patterns. And it's very good at putting all that, you know, context together and, you know, can even find things that humans can't find. But, you know, we're still not ready to let go of the final human step of merging it either. So, you know, right now, a large amount of time is spent reviewing a code, yeah. Well, I'm going to come back to that, how long humans are going to be in a loop, but let's save that for later. So I'm glad you brought up review, as that's an area to really double-click into further. Chris, you know, Century, you're seeing all this code actually be shipped in production. You guys, I believe, also have a review agent that you built yourselves too. Yeah. So how are you seeing this code behave? Is quality going up, or is AI sloppification also happening in the code world too? Yeah, I think the evidence is kind of mixed. One of the things we see is we do see more, if you're per line of code written, there are more issues, but it's more in the order of 10 to 15, maybe 20% that we've seen. I think the major issue is really that there is just way more code being written. So there's just way more code that needs to be reviewed in the first place. So there is a clear bottleneck there. And then there is way more code that ends up in production, and there's still a lot of issues in production. They actually just tend to accelerate with just code development, right? At least what I see from my perspective, from the customers I talk to, the people that have decent practices in place, they just get better. Those that are less evolved end up getting worse over time as well. So the ones that do it well, they tend to identify kind of categories of PRs, and they say, oh, there's low risk PRs, there is middle, and there is high. The ones that are high, they go to the human. The ones that are low, they get automatically reviewed, they get pushed to production, and it's a large category. So you can kind of accelerate it that way. And then once it gets to production, obviously, I mean, talking my own book to some extent, but you have to monitor what's going on. If you don't get signal about what's going on, you're not gonna know what's going on, and so I think one of the benefits of coding agents is that they've made instrumentation a lot easier. So even if you go down a path of open telemetry, like you need to migrate to that, putting that instrumentation in place is now much more cost effective, so you can actually get the signal that you need to figure out what's going on with your coding production. And so we see that shifting in that direction. Chris, a quick follow-up. Are you seeing the nature of the errors change or failure mode changing in terms of what's happening with the agents now writing the code? I think a little bit the obvious errors are like less prevalent, right? Like syntax errors are non-existent anymore because these things have learned to write code, and so the syntax is good. So there is a little bit of an element of like fewer simple things and maybe a handful of more complex things that are tending to surface, but I don't have great statistics on it. Dane, what about your experience being a practitioner and what you're seeing in the engineering team at Cloudflare? Does this match up in terms of quality, not necessarily getting better, seeing any new errors, any new things that were unexpected as code agents are now taking over the world? Well, I mean, so I think we're still in the, you know, just because of the type of code we write, still very much a human-centric engineering org. Humans are responsible for the planning and iterating with the agent on that plan. And so, you know, I think engineers that were great engineers before are still gonna be the great engineers now. You know, if they're pushing slop, it's probably because they wanna get a quick script out versus something that they knew was high value and high risk. And, you know, in most cases, I would hope that, you know, it's code that they're putting their name on. You know, we kind of said early on, you can't blame AI for issues because we still haven't, I mean, maybe one day when we take the, we take a human out of the loop, but right now human's in the loop. So, you know, own the mistake there, but also own the quality. And for the most part, I think quality actually has gone up in a lot of places. Velocity, you know, Kit DiVarta, who is a single engineer who works on the workers platform, you know, he's going from, you know, writing, you know, three to 5,000 lines a day to 20 to 30,000 lines a day. And, you know, he believes it's the same quality that he had before. You know, sometimes he has to go some circles and to bang it into shape, but, you know, even that is getting solved over time. Yeah. I mean, given that amazing quantity of code that's going up, what's life like right now for a on-call engineer? I know that's particularly important for Cloudflare giving you up times. I mean, are on-call engineers just getting crushed right now? Or are they themselves also now starting to use agents to make their lives easier? Well, the head of our SREs is here, so another question we can bring him. Yeah, I think we're probably early in our journey here. We have a lot of tooling around incident management, but, you know, part of the problem is, I mean, we are a security company at Hark, and we segregate and compartmentalize data quite a bit. And, you know, it's not easy for an engineer to get access to the production data sometimes that's needed, which actually is a great opportunity for agents, because agents can actually investigate something for you as an engineer without actually even showing you the data, showing you the... So I think there actually is opportunities to actually use it, actually, to make that part of the job better for Cloudflare, and Cloudflare is very unique in that perspective. But, you know, on more of the pre-production side, I think things like, you know, sentry and, you know, observability, agents love a loop where they can see a reward and optimize. And so, you know, I think that is where kind of data and pulling in a lot of information can be really helpful. Yeah. Chris, what are you seeing in your customers? Can you, you know, you're selling the tools to... I think by definition, if you're generating three times as much code, and you assume that the quality is the same, you're going to get three times more incidents or five times more incidents, right? So we do see that a lot of our customers are very concerned about it, because you can only wake up the senior people so many times. They can only drink so much coffee. On any given night before they decide to take another job somewhere else. But I think that's really the main concern. I don't think the solutions, I mean, are like, put some agents on top and try to figure out, like, what you can get out of the telemetry to get to a root cause faster. But there's still a big concern there in terms of, like, relying on these people. And on top of that, obviously people do not know if they haven't written the code themselves, a lot of that kind of tribal knowledge becomes less efficient as well in the process. I think there's some truth to the more code is, you know, potentially more incidents. The code that actually I think we're fighting the most incidents is actually code that was written, you know, sometimes five, 10 years ago. Yeah. The agents can now find those vulnerabilities, you know, and so if they can find those vulnerabilities now, I mean, the hope is that they're writing code and the review process now is not leaving those same vulnerabilities in the code. So, you know, I think the actual, the incidents are going out more from the technical debt and legacy code of the past. Yeah. And, you know, maybe it's wishful thinking that the code we're writing now is somewhat more incident-free. Yeah. Something I would add is we're doing, we're seeing this probably come up as the single biggest use cases for our enterprises core and deploying autonomous software factories. They really want to establish, we have a product called agent readiness where you're able to actually measure the readiness of your code base and identify opportunities to proactively get on top of security issues, documentation issues, making sure that you basically have this like knowledge base of your code that works for agents. And I actually think some of the security issues we're seeing now with legacy code languages are going to be, this is the beginning of a tornado of issues like this. And we're going to see this effort around proactive security remediation and bung triaging become one of the biggest investments and focuses of the next couple of months into 2027. So Madison, we talked a lot about a lot of the challenges that the world of code agents is introducing to folks. You obviously have the quantity is going way up, the quality, not clear it's actually improving, maybe discovering a lot more incidents historically from legacy code. Who's responsible for solving these challenges? I mean, do the agents need to get better producing code or do you think it's the review and the air handling process that needs to change to keep up with the agents? Yeah, so I would say engineers for the first time ever are finally getting to be proactive in their development and start to think less about reactive work. I think that's one of the biggest things that we're seeing now. So what needs to happen is we need to have engineers up level as much as possible to be thinking about review quality and managing their systems, managing their code bases, thinking about this as an artifact in the company that is fairly liquid in nature. How do we modernize and change the right languages and applications that are required to really effectively work? So I think the responsibility really is always on the engineer. You are the one setting in place the verifications, the tests, the QA, the evals, but using agents as much as possible to automate that work and bring you into this proactive mindset is what's gonna really unlock the next era of autonomous software development. Chris, you're on the other side, you're downstream from factory. What's your view in terms of who's ultimately responsible for solving some of these new challenges? I think it's all of the above, right? Like you wanna write better code, you wanna review the code more effectively, you wanna have better telemetry in place, you wanna feed it to an agent who can solve as much as possible. I do think that like getting more guardrails in place in terms of how you deploy software and getting more kind of, you know, effective at like splitting out populations you wanna launch your software to, deciding whether you wanna roll back very rapidly based on some very specific criteria and having that practice, which, you know, has existed for like 10 years, at least, if not 15. Like we have some of our customers have done this without agents, right? But, you know, getting into these practices for every deploy and deciding what you put, putting everything behind a feature flag, rolling it out to a very small set of people, seeing how it reacts, and then kind of gradually expanding and rolling back is probably in these guardrails is where I see the upside. Yeah, I mean, we've talked about this whole cycle, right, of agents writing code, reviewing code, testing it and deploying it. And ultimately, I think the Holy Grail is getting that back, looping that back, making it continuously, maybe even pulling the human out of the loop. I mean, where do you guys see the opportunity between, you know, factory, sentry, you know, CICD, other things in the middle to actually close the loop and have everything working more simultaneously together without violating some of the data and security issues that Dane was talking about? I mean, it's, at the end of the day, you gotta measure something, right? And then you have a goal -seek algorithm that goes and finds the best way to get there. And as long as you put the guardrails in such a way that the algorithm cannot deviate too much from the road, right? Like if you're not saying, go eliminate my test so that I can pass the test, like that kind of stuff, that's really the way to go. You see this with some of our customers, like you find a problem in production, say the latency of, I click on this button, the latency, when it exceeds a certain, you know, 0.5 seconds or whatever, then go investigate, go see what's going on, find all the evidence you can find from sentry, from other telemetry solutions, bring it together, do some kind of a root cause analysis, feed that to the agent, and let the agent kind of address the issue, right? So it's like a closed-loop measurement system that the feedback goes in, but you have to make sure that, again, the agent stays on the rails. Madison, does that resonate with you? What does the autonomous software factory look like in the future? So we like to say that your software factory is always ingesting signals from every system it's connected to, all the APIs, all the MCPs, all the existing, I mean, say sentry is something that we partner with, and we see a lot of organizations think about alerts from sentry as being automatic triggers for their software factory. So defining those signals, I think ultimately being as data -driven as possible is really the key to creating a economy in a closed loop. And so we are very much on the same page and think a lot about signal-in software factory and how we signal out is new software. So we're going to have to close things out here with our last minutes, but I want to come back to Dane as well as the other panelists on this question of how long until humans are completely out of the loop? I mean, it seems like the ground is shifting constantly. The models are getting better and better. Where do you see agent-to-code development DevOps going from here? And how long until we actually, everyone just has to hang out while their agents write and deploy and fix code without any human intervention? I mean, I still believe a lot in human judgment and human ingenuity and creativity. So, I mean, I think humans will be part of the process for quite some time. The, you know, where they sit in it, it might shift, but I don't, you know, in the next couple of years, I don't see us removing humans from the software development process. Madison? Yeah, we obviously are the ones pushing for what we call the autonomy maturity model at these organizations, but it's quite wild to see, you know, the dichotomy between, okay, you have the best AI models in the world able to do crazy tasks, but then you have organizations that are stuck on extremely legacy software and bridging the gap there is still very, very, there's so far to go. So having these systems that really bring humans into the process and build autonomy slowly with verification is going to be, it's going to be a very long process, no matter how fast AI evolves. Chris, you want to finish this off? What does the future hold? Same thing. I think judgment and the much overused taste are still going to be important. And you can teach agents to some extent, but, you know, if you think about it in the ultimate limit, this becomes kind of a self-organizing organic system software, right? And if you just let it do whatever the user, you know, directs it to do, God knows where it will end up. So you still need some level of, again, you know, guardrails in this case, judgment by humans. Like, hey, this is what I want my software to do. And this is where it needs to ultimately end up, right? So for a while still. Amazing. Thank you, Dane, Madison, and Chris for this awesome panel. Please give our panelists a hand of applause. Thank you so much. Thank you.