Better visibility and control: Shining a light on shadow AI

My name is Abe Carryl. I'm a product manager here at Cloudflare. I've been a product manager at Cloudflare for about six years. I've spent the entirety of that time on our Zero Trust product suite, thinking about our networking and security products. For those of you who don't know, our Zero Trust product suite helps connect and protect users, applications, data, networks, and increasingly the agents connecting those internal resources. I'm excited to talk about the Shadow AI and how we're helping solve that problem today. I think it's a really interesting time, or at least my job has been very interesting over the past few weeks and months because it feels like maybe for the first time in my career, every single organization is struggling with kind of the exact same shape of problem. That's happened a few times within my career, but this feels like the first time where that's really true down to the application and the exact use cases. Typically, the kind of questions that come along with customers who we're talking to involve things like, have we reined in Shadow AI? What does our Shadow AI footprint look like? What use cases are people solving with AI? What applications are proliferating within our environment? How are people connecting to those? Then the next kind of is, what MCP services are people connecting to? Are those local? Are those remote MCP servers? Are they being downloaded from vetted or unvetted sources? Are they being built locally? Are people baking in API keys? How are we handling the authentication? Then maybe the most granular level is, what are the actual prompts that are being sent to those applications and what kind of data is being stored within those? I think it's a really interesting set of questions, but it feels like every single organization is trying to solve these exact three things or get visibility into those. That's exactly what we've been focusing on at Cloudflare and really kind of with three core pillars. The first is connecting, so helping users connect to AI experiences at a global scale. Second is building, which you've probably heard a lot about this week. Building full stack AI experiences directly onto the Cloudflare network. Then we're really going to talk about which is securing. Layering in Zero Trust policies by default and making sure that we're really locking down each one of those individual experiences. There's a few different use cases that you'll probably hear about this week. We're going to talk about securing workforce use of generative AI today, but there's tons of tracks around governing AI agents. If anybody went to Kenny Johnson's session yesterday or any of those sessions today, that's all about MCP clients and how they connect to MCP services, the remote MCP servers or MCP server portal. Building securely with AI, so we've talked a lot about that with our developer platform. Then of course, protecting AI-powered applications. That's more about how your end users are engaging with AI services that you're building and making sure they do that in a safe and secure way. We'll really talk mostly about securing workforce use of generative AI. Just to level set on that use case, what we're really talking about is end users connecting to AI applications and how we do that in a safe and secure way. There's two common pain points that we see. The first is around discovery and the second is around control. With discovery, it's often, I don't know what AI applications are being used within my environment and I don't know which tools are risky. Maybe you do know which tools are risky, but just having the defensibility to go to your team and say, hey, here's why I'm taking an action on a particular application and making sure that we can do that. Then the next set is controlling those, so determining which users can access which tools. That involves things like your technical team probably needs access to a very different set of tools or maybe the exact same tools but with a different level of access. Then your non -technical team might need something that's different than that. Then once you've taken that step, looking at the individual data that's being sent to those and making sure that you're securing the sensitive data and those transactions as well. That's really what we built our Zero Trust Platform to solve. Every time that I think about our Zero Trust Platform, I always think about it in the same kind of way as a bit of a three-piece recipe. You have your on-ramps, the ways you get traffic to the Cloudflare network. We have client-based means and clientless ways to do that. Today, we'll mostly talk about the client-based means. Then you have your services within the Cloudflare network. Those are mostly security and routing policies, so determining who can access what and then how they get there. Then you have your off-ramps, so ways that we either egress to the public Internet with things like our secure web gateway or ingress to your private network through things like Cloudflare Tunnel, Cloudflare Mesh, or other kind of off-ramps that we have within Cloudflare. Once that traffic arrives to the Cloudflare network, there's a whole host of benefits that you get. The first being visibility. Because you have a single pane of glass and a single point of inspection, you get visibility into every single request that's going over the network. The second is access controls. This is something that I always love to talk about, especially this week when we're talking a lot about our developer platform. Most network firewalls, actually, my coworker colleague, AJ, is doing a session right next door on building a programmable gateway. Really, what that allows you to do is rather than having static allow and block policies for your SWIG or for your access controls, being able to write policy as code with a custom worker. That's something that's really, really powerful. I won't get into that a ton today, but I think that's something that's really unique. Then you get into the data security controls themselves. Things like prompt guardrails, obviously, you want to think about the safety mechanisms of that experience. That's things like making sure that a user can't say, hey, what's the best way to cheat on my expense report? How much do my coworkers make? You want to make sure there's some sort of guardrails around that. That's something that we offer. Then the DLP or the data security policies themselves. Regular expressions, algorithmic checks for checksums proving that data is structurally valid, and then using AI context analysis for the surrounding bytes to then determine whether or not the data that we match is actually something that's relevant for your organization. There's a lot of things that tie back into having a developer platform there as well, where we can do these outbound checks for DLP that's really powerful. Rather than just defining those as this matched a particular pattern and was algorithmically verified to be structurally valid, you can also call out and say, hey, this is a GitHub key, but is this my GitHub key? I think that's something that's super powerful as well. Then the last piece is our AI security posture management tools. Those are things like with Anthropic and ChatGPT having compliance endpoints that you can hit directly and then use that to detect misconfiguration within your application, but also even if you're not using it for misconfiguration or managing your deployment, once you turn on our access controls, that really only works point in time. It's a really fast way to get started, but if you want to look at everything that already resides within that application, we can do that with an SSPM and with a direct integration through those compliance APIs. I'll talk a little bit more about that in a bit as well. So this is really kind of all underpinned by what we call our application library. And our application library is kind of a core primitive within our Zero Trust platform. So what we do is because of our vantage point on the Internet and our ability to see a lot of different traffic, we will categorize that traffic for you and catalog that and break it into different classifications. If you take an application like Anthropic or ChatGPT, there's tons of host names associated with that. An application like Netflix has 50 plus host names that we've categorized, but you can do that in order to take a really quick action on the application itself. We then assign confidence scores to those applications. So we look at things like, do they train on your user data? Can you change that configuration? Do they have ISO 4201 compliance? Are they financially stable? We look at different signals like that. And again, so that if you take an action on an application and you need to go defend that to anybody on your team, you have some sort of data to back that up. And then we also allow you to fork the definition of the application library. So we call that custom applications, but really it just allows you to define your own internal applications or take a definition that we have and append that add to it. And I imagine that our application library is something that we'll continue investing a lot in building header controls in and other and MCP services like continuing to add more and more information into this application library. That's important because it then is what underpins things like our shadow AI and MCP reports. So that classification allows us to look at your user traffic and then understand which users are going to which AI applications and the amount of data that's being transferred between those applications. We also recently released our MCP discovery report, which is really powerful as well. So that works a little bit differently. And rather than just looking at the host names, we actually look at the body of the request as well. So we'll look at things obviously like the headers for slash SSE and slash MCP to kind of classify that traffic. But we also look at the resource and tool calls that are being made in the body of the request. And when we see that over and over and over again, we can have a pretty high degree of confidence that that is probably an MCP service, which allows you to take action on that as well. And then you can determine the state of that application sanctioned, unsanctioned, however you kind of want to handle that within your environment. That then allows you to really quickly start to get control in your environment by making a policy that looks just like this. So if the application is unapproved and the category is artificial intelligence or the service is MCP, then I want to block that request. You can also redirect that. So in this example, I have that being redirected to chat.something.com. And that's my agentic workspace. So I can redirect traffic that way. There's all kinds of kind of different controls that you can do there. And if you're familiar with our MCP server portals, you can make a very similar control where you can say, if the traffic is MCP and it is not in my MCP server portal, then I want to redirect users to that. So really, really, really quick ways to start getting control over your environment with data informed decisions built off of those classifications and those reports. Once you have that, most users want to start building in data security policies. But some of the things that we've seen with our customers is that typically the most intrusive or the highest points of friction of enabling any kind of deployment like this is turning on something like TLS inspection or scanning the bodies of the request. So turning on DLP is often very abrasive action. It's something that can be really difficult. So we built this report in order to allow you to kind of turn DLP on in like a listening mode. So you can just enable certain DLP detection entries or DLP profiles. And then you can see exactly what would have been blocked before you kind of actually introduce those block policies. So in this example, you can see that there's a Visa card number that's being matched a lot. Because I work on the product team within Cloudflare, obviously I'm signing up for a lot of Cloudflare accounts. So this is something that I probably want to know way before I turn on DLP because I want to know that I should whitelist this domain and I shouldn't block this because that's going to be something that's going to be abrasive to this end user. Once you have that data, you can start looking at things like the actual policies themselves. And we do something really interesting with data loss prevention here, where you can look at the DLP profile is in AbleLink and Bio. That's like a fake company name here. And then the application is cloud. And we have what we call granular application controls. So we break down the individual kind of micro interactions of cloud itself. And then you can look at individual operations of the cloud application. So here you can see things like chat, send prompt, upload file, auto transcription, things like that. So you can determine where you want to analyze that traffic and kind of break that down. And that's something where we're constantly adding new applications and new granular application controls. But you can also do things where you can, if you see over here on the right-hand side of the screen, where you can generate and capture the actual prompts themselves and classify those. And we kind of think about DLP in two different perspectives. The first is content. So what the user is sending to the model. But then also the intent, what they're trying to derive back. And I think looking at those as two different interactions is really important. Because you might see something that you might learn something from that. As an example, if a user says, hey, here's my API key. Go take an action. That's a very different exercise and a very different follow-up ticket that you want to investigate with that user. Then if the user says, hey, can you mint me an API key that has these permissions? So you can kind of start to learn and have a little bit more granularity in the way that you get visibility into your environment. The last piece is out of band. So this is our AISPM. So I mentioned those compliance APIs that you can connect directly to to prevent misconfiguration or understand the way that your applications are being configured. So you might be able to see things like overprivileged users, API keys that have been outstanding for a really long time and haven't been used. But you can also run our DLP on the data at rest. So for everything that already resides within that application, you can start looking at the files that the model has generated, files that have been uploaded, and start to understand if there's sensitive data in there. Maybe source code. Maybe API keys. Or maybe it's just things that are unique to your environment. So those are kind of like the high-level ways. So just to kind of recap there, the very first thing that we do is everything starts with our application library and the way that we kind of classify applications and the way that users are interacting with those. We then generate reports and we kind of handle AI traffic and MCP traffic a little bit differently. That then gives you the ability to analyze the way that your users are leveraging these tools within your environment. You can then leverage those insights to create policies before you actually cause any impact to the end user. And then you can start to layer in your data security policies as well. So some of the things that we're building next here are around our shadow AI control. So we're going to continue to invest in deeper application library definition and classification, which I'm really, really excited for. But we're also investing a lot in risk behaviors for Ueba and for agent behavior analytics. I believe that Ayush is doing a session right now who's leading Ueba for us. And some of the things that we're doing there are starting to analyze the way that a user interacts with these models and look for anomalies in token spend, cost per prompt, how long the session is, different turns. So things like that. But we also are going to move from the operational side to the security side as well. So looking at how many different at keyboard and away from keyboard activities the user performs. So you can start to get insight into the way that a user behaves and the way that their agent behaves. I think that's going to be something that's really, really powerful. Around data security, I really enjoy talking about this because I think that one of the things that we've heard a lot from customers is just making sure that we give users the ability to customize and layer in their own business logic to the way that we detect traffic. So in this example, some of the things that we do is we have the detection itself, which is typically a regular expression or some sort of model that's analyzing the data in the request. But then we also have things like checksums, and we have keywords, and we have a context now. So we have a bunch of other checks on our end that can either increase or decrease the confidence in a detection. And what we want to do is start to expose more and more of that to end users so that they can define what they want. There are a lot of different ways and different versions of API keys that are supported or legacy, and we have to account for all of those. But maybe within your environment, you only use one type of that. So there's no reason for you to have to incur those false positives when that's something that you know doesn't exist within your environment. We're also extending that with custom validation logic. So one of the things that's often really difficult is understanding the materiality of the data. So whether or not this is just an API key or if it's your API key. So those workers that can make outbound requests to some sort of third -party service and then get that information back is a really good way to do that. It's very performant because that all runs inline. Something we talk about all the time is the composability and programmability of our SASE platform. So that's something that's really powerful here. And then of course around MCP governance, that's something that I'm really excited for. So today at the portal level, you can turn tools on and off. But this will give you the ability to define that at an identity and device level as well. So at the identity and device level, you can determine if different users and groups should have access to write tools or read tools. We'll do some like regex and fuzzy matching so you don't have to do that on a tool by tool basis. But you can say anything that looks like a write tool that I want to only give access to this group. Anything that looks like a read tool everybody can have access to and some things like that. And then we'll continue to build out our DLP guardrails. I think that the way that we're kind of thinking about DLP is that we want to start to use regular expressions more as a filter than the key way that we define our detections. And that we then want to use AI to validate whether or not that data is relevant to your environment. So those are some of the things that we're really focused on over the next few weeks. So some of the key takeaways, hopefully, that you're all walking away with is that you can't secure what you can't see. So getting visibility is often the very, very first step. In DLP, we should be looking at content and intent. So both sides of that transaction. And then of course, I think the thing that's most important that we talk about all the time on our team is just making sure that these are, that layering in Zero Trust controls is progressive and intuitive. And really, what I mean by that is making sure that you're never flying blind. You're never implementing a policy without knowing the exact impact of that. And I think the features that we're building, like passive detections, which I kind of think of as shadow IT, but for DLP, our shadow MCP report and our shadow AI report, that it gives you the ability to make informed decisions and know exactly what the impact of a change is going to be well before you put it into production. That is the time that we have today. Hopefully that was a very quick overview of how to get easy control over shadow AI. And yeah, thank you so much.