MCP is the door, and the control plane is the house

My name is Kenny Johnson. I'm a product manager on the Cloudflare Zero Trust team. I oversee access, MCP portals, and a couple other products. And I'm joined by Ayush, who's on our internal infrastructure engineering team. I think is the right name for the team. And we're really excited today to talk to you guys about what we're doing with internal MCP security at Cloudflare and hopefully trade some notes for you all to take back and both use Cloudflare products as well as just generally find a good approach to securing your MCPs. We'll start off with a quick question. Do you guys know every system that your agency you use can access? Raise a hand. Yeah, great. Exactly. And then even extrapolating that further, think about your employees and your team and their systems and all the things that they're trying out. Again, probably not going to see a lot of hands here. We are working towards that goal, though. And that's a big piece that we're really having a lot of luck with internally at Cloudflare by leveraging MCPs and really pushing MCPs as the primary standard. The things that we're really trying to avoid are things like direct API keys, custom scripts, homeworld CLI tools, you name it, we've seen it all. I'm guessing you guys are also seeing that. The challenge is even agents suggest that. They try to break out of the boxes that they're being put in. So we're really focused on trying to build a standard protocol and create a paved path for the team to be able to get access to all of the independent systems that they're looking to access. So if we're going with our house analogy, we think about MCP as the door and the gateway into independent systems for code, tickets, internal APIs, docs. And we've actually really focused on building and developing our own MCPs even when third-party MCPs are available. Generally, we're building them ourselves. I wouldn't necessarily recommend that for every organization, but it has been really nice for us to be able to home roll them and actually have a really direct control both of the MCP server as well as at the gateway level. And the big thing to ask yourself or that we ask ourselves is for which user, for which tool, under which security policy is it going to trigger, at what context cost, and we'll talk a little bit more about that. We ran into some really interesting challenges with context windows to start, and then what's going to be the scope of the impact of the action that the particular user is taking. So this is the framework that we've taken with Ayush's team as they've built and rolled out MCPs internally. This is becoming really cheap to add. You can go to the hardware store and go buy more and more doors. It's been really, really rapid in terms of MCP adoption internally at Cloudflare. It started off really slow. I think this slide is actually probably out of date. I'm guessing Ayush and team have probably shipped more since we've been here in San Francisco. So this brought us to one of our first big challenges that we ran into. That's a lot of tools, and if you take something like an MCP gateway and you push all of those tools to a context of an agent session, all of a sudden it's going to tip over. We had people running into situations where, like, you could ask, like, what's the temperature outside, and your context would shoot up to, like, 100,000 tokens. So we knew we had to come up with a way to improve this. So, yeah, like I was saying, we'd see, like, 35,000 tokens on the first request. It was a really high time to actually get a response because the agent was just chunking through tool calls or tool names and lists and descriptions that really didn't matter for its independent task. And we ran into problems where just the context window was expiring and we were hitting compactions way too early in a user's work stream. And then we ran into the, because we rolled our own MCP servers for all of our independent applications, and we have lots of smart people at Cloudflare, everybody tried to independently optimize for this. So every single MCP server that was being developed was coming up with a different solution or approach. And this is actually where we landed on a very cool approach called code mode, which basically allows you to reveal a handful of tool calls to your agent, and then it instructs the agent to write JavaScript and go and find and fetch the correct tool and then use a tool to go and fetch that particular tool. The problem is we were doing that at each individual MCP. So I'll let Ayush talk a little bit about how we kind of up-leveled that and improved it. Yeah, so... Oh, can we get Ayush's mic? Can we get Ayush's mic? Can we get the slides back? Can you talk to me? Yeah, just talk loud. You're good. And... Huh? Hello? Oh, there we go. So, yeah, we started adding code mode to our servers, but the problem was... So code mode has two tools, search and execute. Search is basically where we find the tool and search for a tool, and execute is when we actually... The model actually writes the script and calls the tool. But the problem is we only get search and execute on the portal, so there's no tool-level visibility there with us. And portal allows us to switch off or on tools at the portal, which admins can do, and that was taken away if we did that. Plus, all the logs will say search and execute, not the actual tool call. So this never worked. And that's why we wanted just one place where we can add all this without us having to do that at every server. So we added this. Portal was the right place for us. And, yeah, then we added it there. Yeah, and that's a big thing that we've seen is centralizing these tools, and that's a big thing that my team worked really closely with Ayush's team. A lot of the controls that we put in place into our MCP gateway, MCP portals, came out of working with his team. For example, we can take code mode, and you can, as an admin, either decide to force it on for everybody, you can set it up so that it's an optional flag, you either opt it in or opt it out, or you can turn it off at their specific situations. So then, taking the house analogy, what this ends up looking like, we use Cloudflare access as our front door. You definitely don't have to do that. There's no prerequisite to doing that. You can use your identity provider or a different reverse proxy if you have that. Then we create the house rules, basically. Using MCP portals, we can turn on and off independent tool calls, so if there's a specific situation where we want to not allow specific tools to be made available to subsets of users, we can do that at the portal level. Code mode then allows us to actually execute, and we've seen a 93% reduction in tokens with code mode. Saw some people taking photos of that. So we've seen a massive, massive payoff on that. And then we've got it set up and built so that the MCP execution is sandboxed completely so it can't go and reach other APIs or anything like that. So this is really the framework that we try to take when we're building our MCP servers and then on-ramping them into the portal. One other piece I'll call out here is that we've done this fully via infrastructure as code, so we've got this set up within the Terraform provider, and it's actually a really nice way for a developer to say, hey, I want to build an MCP server, and I want to publish it, and there's a really standardized approach so that iOcean team get a notification in GitLab when a new merge request is open to add an MCP server, and that's a really good trigger point to allow them to review and vet for security and make sure there's no issues, and then they can publish it to the portal, which then allows employees to discover an MCP. You don't have to go add every single individual MCP to your configuration. And then we've got our guestbook in terms of logging and DLP controls. You can route everything through the MCP gateway through Cloudflare's SecureWeb gateway, which allows you to do HTTP-level policies, so you can inspect and run specific data loss prevention controls as well as just, like, if you need to have it respect your egress policies for your firewall, you can do that. And then we've been really focused on locking down access to basically rogue MCPs or direct API keys or CLIs, and we can do that via access as well as we recently added a feature into gateway, we've got the gateway PM in here, that allows you to select on-ramps within gateway, so you can basically write an expressive policy that says, I want to allow MCP traffic but only via the MCP portal, and then block the rest of the user's HTTP traffic to additional MCP servers, which has been a really powerful control. We're not actually blocking it right now, we're running basically an alert mode that will alert the user and give them a notification saying, hey, you're accessing an unsanctioned MCP, please use portals. Yeah, so we touched on this piece, this is mainly just kind of a good lead-behind summary of how to focus on these pieces. And then, most importantly, I want to jump to a demo. Ayush has got this queued up, we'll look at some of the different policies that you can create, you can see a tool call succeeding, tool call failing, I'll let Ayush talk you guys through it. Yep, so here I'm just using MCP Inspector, I'll just turn it to white mode, I guess. But, yeah, so we have two users here, on the left side we have a user, both have already signed in to Access, in this case we're using Access, so yeah, both have signed, both have their own policies, and so, depending on the policies, they access their servers. So if we try to list the servers they have access to, and yeah, as we can see, they both have different servers based on the policies, so this is coming directly from Portal, not from anywhere else, and this is decided when they actually sign in. Oh, yep. Right. Yep, as we mentioned, that code mode, so on this server, as you can see, we have these five tools exposed, and all these tools, this is it. When we turn on code mode on Portal, these are the only five tools we'll ever have, no matter how many MCP servers we add behind the scenes. So, yep. And then, yeah, with code mode, in this hypothetical scenario, this is the difference that we have. Yeah. So if we actually try to, you know, run it on some... Okay. Of course. One sec. Yeah, so, I will prompt, and this is if we are... We are basically asking if this releases the ship or not. As we can see, code mode will first call the search tool, and this is a long list, but, yeah, it is basically returning all the tools that are available. And, yeah, normally it truncates everything, but it didn't in the first go, so that makes sense. And then, the second call is to execute. After we found the tools that we need for this prompt, the model will call execute and actually write a script to call all those tools. And, yeah, it ran through all that and it says, oh, okay, we can't ship that. So, if we, let's say, try to tell it to roll it back, the ship, it will again search for that tool if there is a tool which allows us to roll it back, and then it says that no portal tool can perform a rollback. But here's the interesting part. So, since agent can see the capabilities, it can recommend the capabilities, but it's the human which actually controls everything. So, here are the tools that we have on the MCP server, but we have turned off the students, one of them being rollback release. So, even though it is available, the admin can turn it off, so the agents can't execute anything on there. And that's been a really powerful tool, especially when you are working with third-party MCPs, because companies are really eager for you to use their MCPs, and they're like, great, let's throw everything into this MCP. But if you guys are still in the process of rolling out and getting an understanding of what's the actual surface area, what's the actual impact of things that we, of actions we want an agent to take, I might not necessarily want promo release in there, or say I'm plugging in the Stripe MCP, maybe I don't want create invoice to exist, only view invoice to exist. So, it gives you a lot of flexibility to control it at the gateway layer, as opposed to having to home roll your own MCP or just rely on the permissions in the underlying vendor. We've had a lot of luck and success. And I'll break a little bit of roadmap news. This quarter we are working on adding granular policies to this. So, today you can write policies at the portal level and the MCP server level. The piece we're building now is the ability to actually tie independent access policies to tools or groups of tools, as well as being able to do actually some reg X matching, so you can say, like, where contains right, or like some basic fuzzy matching on the naming of the tool and say, I want to allow all reads but block all right tool calls. If we could jump back to the slides. Awesome. Ayush, excellent job on the demo. It all works great, which is good. We didn't get demo -guided. And this is just kind of a good example of being able to run that individual script. It goes off and runs it and then returns that the user's not allowed to do that. Do you want to talk a little bit about the replay and what the sandbox reduced? Yes. So, everyone might have seen that we can redirect the assistance to do some shady work. In this case, we want to upload something to some hacky URL that I have. But the thing is, CodeMode actually runs on the portal in our dynamic worker, which is sandboxed, so it can't access anything outside of its own environment. Basically, it can't access any other URL other than its own world. So, yeah, nothing leaves that network. Awesome. Yeah, and so that takes us back to this is kind of the overall approach that we've taken internally at Cloudflare. We're in really close contact with the team that's been working on MCPs, and we're providing feedback to that team. So, yes, we're compliant to the latest spec of MCP servers where they're more stateless and a few other things got added. And as they add new things to the MCP protocol, we're going to stay right on top of that. So if you're taking this back to your teams, some of the things to think about, go count your doors, figure out what MCP tool calls are there. You can do this either using Cloudflare Secure Web Gateway or if you guys have other tools that are doing proxying of user traffic, that's a really good way to start. If not, you can look at things like code PR. You can look at merge requests, pull requests, things like that. We've had really good luck building different inventories internally, looking at what folks are using outside of MCP gateways, and we're both trying to get those things blocked and not necessarily allow access, but then also making it really easy and just bringing, working with Ayush's team to add more and more servers to our independent MCP gateway. You can create one front door, which is a big piece. That's also something that we're working really closely with Okta and Entra on to adopt the IDJAG cross-app access framework. That's not quite ready yet, but it's coming very, very soon, which will allow you to basically, instead of when you authenticate and then having to authenticate to all the downstream MCPs, we'll be able to mint OAuth tokens using the IDJAG framework. That piece is coming pretty soon. If you front them with access today, you can actually do that already, and that's something we can definitely help you guys set up. If your MCPs are self-hosted, fronted by access, you get implicit auth. If you are using it, Okta or Entra or something like that, we're going to be adding that piece. I would recommend starting off with turning rights off and locking all additional access. We've been creating, we've been enforcing code mode pretty heavily. You have to come to us with a pretty good reason why you don't want to have it on. That's something you can kind of negotiate and work on with the team. Then thinking about how to sandbox and protect the network access that you're giving these agents, as well as create a logging framework. I think we've got... We're talking about the newer MCP spec. We've got a link to our dev docs here if you guys want to give that a try. Thank you guys all for showing up, and thank you for the time.