Deploying a WAF vs. running a WAF program: The missing manual

My name is Billy Hoffman, I'm the Field CTO at Ionix. My name is Justin Tibbs, I'm the Vice President of Cybersecurity at Presidio. And Ionix and Presidio have been working together for about a year now. And a big thing we have seen, a big belief of ours is that we're sort of entering the golden age of the WAF with frontier models. Frontier models are sort of driving now and how quickly we can find and build payloads for vulnerabilities. But change control is still going to change control and it tends to take a long time. And so our hypothesis is that web application firewalls offer a really great opportunity sort of like deploy a virtual patch while you actually fix the underlying issue. But that then led to something that Justin and I found, which is there's the reality of running a WAF posture management program at scale. And there's the theory and those two are very, very far apart. Yeah. Quick question, do you guys all run WAF? That's impressive. All right. This is going to be great. This is going to be great. Also, I've always been paranoid that I say WAF. Strangely, people are like, no, you have to spell it out, so. Do you get, does anybody spell? Serious question, does anybody say W-A-F? Dan, you don't count, you're weird anyways. Peculiar, you're peculiar. My vocab F says peculiar. Clearly, none of you have been on a call with Oracle before. Or they have. Or they have and they're forgetting it. All right, let's go. Yeah. So I thought it would be fun to start off with some falsehoods that people tend to think about with WAFs. Okay. I think the first one I would start off is that a WAF is a binary thing, right? They're like, is that behind a WAF? And it's like, okay, well, a thing can be behind a WAF, but you know that doesn't mean that it's protected, right? Is the WAF there, but not on? Is the WAF actually in logging mode versus blocking mode? Great, there's a WAF there, but like, have you guys ever seen that meme of like the picture of like the gate, but there's no fence on either side of the gate and there's just tire tracks of people going around the gate? I see this a lot of time with WAFs, where it's like, cool, the WAF's there and it's working great, but like, I can just talk directly to the origin, so who cares? So WAF being a binary thing is a real common one that I see. I think you guys can all probably relate to it. I'm old, so I referenced the Ronco commercials with the chicken cooker, where you set it, forget it. Yes, I'm old. That to me, like when I talk to clients, they're like, oh, my WAF, it's running, it's good. Falsehood, right? That one is always humorous to me, especially when we're gonna talk about app acceleration and dynamic development and all the crazy things that happen with libraries. We don't just set these things anymore. Maybe back in, there's no one from Imperva in here, is there? Maybe back in like 1985 with an Imperva WAF, where we just installed it and hope it ran, that might've been a thing. Yeah, exactly. There's certainly drift, and then also I think just the fact that you can have a couple of core rules, right? When Mont security came out in, what was that, 2005, 2004, somewhere around there, it was like we're having 1,500, 2,000, 2,500 CVs a year, and now we crossed 100,000 in September, which is sort of terrifying. Another one that I think is pretty pervasive is that the security team always owns the WAF. How many people here are responsible for the WAF? How many of you are in security as well? I saw some hands go down. This one is interesting, and I think it sometimes can sort of be, what was the initial acquisition of this product, right? I often will find a CDN team inside of marketing owns the Cloudflare portal access or Akamai portal access or what have you, because originally they bought as a CDN or they got it because Cloudflare traditionally has that really great automatic front-end optimization stuff for images and such. So someone outside of security buys it, and then security is like, oh, cool, here's a control we can use, but it's like, do you know all of the Cloudflare accounts? Because rarely there's just one. Do you have access to everything? Are you making sure that everything's all turned on? So who actually has access and sort of owns the budget item for it is often interesting too. And along those lines, a false which we'll talk about is how many of you guys have a WAF program? Or is it just random? Thank you, brave soul, for raising your hand. I deployed an app that's behind the WAF. Let's go. I deployed the app that's behind the WAF. Let's go, right? So having a programmatic approach to that is oftentimes lacking. This actually, I think, is actually a great way to kick it off is we try to think about a maturity curve, a maturity model of adopting a WAF posture management program. And what do I mean by a WAF posture management program? It's simple stuff like, do you have a policy about what should or should not be behind a web application firewall? I've talked to some companies who are just like, oh, we have a policy. Our policy is a very large airline in a city in the South where I live told me, oh, everything needs to be behind DocuMine. And it's like, have I got news for you? Because you have a whole bunch of assets and a lot of them are not behind DocuMine. So they'll say everything that's public or they'll say anything that has a login page. So I think that's sort of a first step is how many people sort of have a defined policy or a guideline of like, if it's public and it has blank, it has to be behind a WAF. Awesome, okay. Getting started, I often find that that can sort of just, planning that out can be a bit of a stepping point. So I feel like crawl is really awareness of do I know what I have? Do I know what is public? Do I have an idea of what should or shouldn't be behind a WAF? And do I know where my gaps are? And I think we'll talk about this, but I think a lot of people, this isn't a, much like WAF's not being binary, the maturity model isn't binary. I see some people who have some parts of the walking phase and then some of the crawling phase. Walking, so crawling is sort of like awareness of the problem and understanding where the gaps are. I like to describe it as that scene in the Wizard of Oz when Dorothy goes through the door. And it's like, before you had visibility into everything that was public and what is not actually in compliance, you live in the sepia toned world of everything is great. I mean, you're on a farm and it's the dust bowl. So I mean, that sucks, but you're in your own microcosm and everything's fine. But then Dorothy goes through that door and realizes, oh crap, there's a whole nother world over here and all sorts of wild colors. And I had no idea that this was here. And I feel like that really marks the sort of acknowledgement of the crawl piece is, wow, this is a much bigger world than I realized. Walking is really when you start saying, okay, great. Like what we want and what the reality is are different. Let's start to build a plan to bridge it. And it tends to be, let's build down a list of like zones we need on board, like a burn down chart. We're actually starting the process. But I think the difference between walk and run is sort of the realization that it is a program. I'm so glad you used this, Justin. It's not a point in time. It's not an audit. Here are all of the things we have. Here are the things that need to go behind a WAF. Okay, cool. We're done. No, because while you were doing that, your company bought another company that has a whole bunch of stuff you didn't know about. Or marketing just spun up a bunch of things for an event and they never told security about it. So you have sort of this, the running phase is really realizing, no, no, no, this needs to be sort of a repeatable, sustainable system that has proactive notifications of when new assets appear that are out of compliance or when you have drift and things like that. So this I think is in a nutshell, sort of the maturity model. Justin, I'm interested, when you talk to customers, where do they tend to be on this? Are they, no. Yeah, so I was gonna ask, because this is an interesting thing for me. So my background's export development and somehow I figured out how to build companies. But scope is always fun for me because every client I talk to is like, we have 15 external assets. And then I'm like, okay, that sounds great. 150 external assets is probably more realistic. And so that's often the, to the point of crawl, that's often the piece that's very fascinating is to really understand, because we develop in such, especially today, we have so many, and I hate the term citizen developers, but we have them, right? We have Codex, we have Cloud Code, we have workers, right? I can deploy to a worker rapidly, pretty quick, within five seconds with the CLI command and be up and running, right? And so now I have a web asset that potentially is outside of the scope of the WAF. So under crawl, what's interesting is figuring out like, what is your external asset scope? Do you actually know what it looks like? And to your point, is it actually behind your WAF program, right? Because oftentimes we find, even within Presidio, we have 1,200 full stack developers. And I always have to go and say, yo, you just put a new web asset online, right? I can see it, right? So I have that visibility program, and that's the piece I think is really important here. We have to have a program, but in that program, I need to understand what's out there so I can put it behind the WAF program. I have whatever that may be, right? Whatever you're trying to do there. And we'll talk a little bit more shortly about the acceleration of that and why we talked about this earlier with exploit development and automated machine response. But a lot of our clients don't really have, some of them use security scorecard. How many of you guys have used that? Or what's the other one? They're all kind of tragic. BitSight, BlackEye, third-party risk management. They'll totally hate me for that, but I think they're quite tragic. There's a lot of deprecated data on there. They associate things to you that are not yours, all that kind of stuff. So actually figuring out how to walk your infrastructure and understand what you have is gonna be important to understanding how you put things into Crawl and then get to walk, right? That's actually a really good point. Let's kind of hop into Crawl. And both Justin and I are aggressively not salesy. No. So I mean, we both have companies. I'm from Ionix, he is from Presidio. We both have booths. We would love to talk to you more, but this is not about trying to sell our products. This is about raising awareness. So everything you're gonna see on these slides, it's not about like, you're using XYZ product we sell. Like that's not a mark of maturity. And anybody who's telling you it is is trying to sell you snake oil. So really the way I like to think about this is like capacities. What is a capability that an organization has? And so being, I think one of the critical ones for understanding scope is being able to see your entire public facing attack surface. And that doesn't mean these are the IP blocks we own. Okay, cool. Marketing spun something up on, you know, WP Engine, like a third party WordPress host or something like that. Like that's not all of your public facing assets or people will say, oh, well I log into Cloudflare or Akamai or Imperva if we're gonna talk about them. And I look at my console and that's my attack surface. It's like, no, that's the stuff you know about that's already in there. That's not actually everything that's public. So defining scope is really a key part of that. And then I think a key part of this also is just, we started to talk about this as defining a policy. What do you as an organization wanna protect? Something actually I find that helps people accelerate this just as it is. A lot of people, how many people here have sort of inside their application or inside their company, like tiers of apps. Like these are our tier one most important apps. These are our secondary ones. This is really common, especially on the IT side, usually in regards for like disaster recovery, business continuity, maybe number of nines of availability. You're like, these are our critical things. Like our supplier portal cannot go down or we lose money. But like if our marketing site has some downtime or it takes longer to restore, less important. I often find organizations can sort of use these existing inventories potentially to sort of understand, okay, well maybe everything that's a tier A app that has a login page or maybe has PII or confidential data, that needs to be behind. And then essentially there's sort of a diff. It's like, okay, well, if I have the ability to see everything that's public and I have the ability to understand this is what should be behind a WAF, I can just look at the Delta and understand, okay, here's what I should focus on. And one of the key parts here is also having process. You know, we keep talking about a program, just someone in the security team or in the WAF team having a spreadsheet or a diagram saying in compliance, out of compliance. Okay, that's great, but that's not operationalized. So you need to start like documenting, hey, department, when you bring a new asset online, here's how you decide whether it needs to be behind a WAF or not. Here's the process for requesting access. Because if you don't do that early on, you're just making the problem bigger. You'll start running towards a walk and run, but like it's a leaky bucket. While you're doing all that, other parts of the organization are onboarding assets, but they're just growing the pile of stuff that's out of policy. Yeah, I do want to highlight just from an attacker perspective, like I was a developer in a past life, I get it. When you bring something online or when a developer brings something online, we see it really quick. If it has a certificate, we see it really fast, right? I can go scour the Internet and find every certificate associated with your domain name or any of your entities within 15 seconds. It's really fast, right? I can scour DNS records and see every new DNS entry that comes up in your asset class, right? And so the assumption of like nobody knows it's there I think is also misleading. I hear that a lot of times from developers, like I just put it up there. Great, I found it. I found it really fast. In fact, my agents that do that did that, right? And so I think that it's more critical today than ever to realize that the minute an asset goes live and if it has a DNS entry, a certificate entry, or anything that hits some sort of public facing scenario, we will find it really fast if we want to. I'm so glad you brought this up. How many people know what Certificate Transparency Logs are? About half of you. So Certificate Transparency Logs, actually largely through a lot of work Cloudflare has done, is every time a certificate authority cuts a new certificate for a domain, or even if you're using Let's Encrypt, it's sort of a public append-only ledger of all of the certificates. So there's a really great site, crt.sh, that lets you search Certificate Transparency Logs. And I can just be like, I'm from Atlanta, so okay, Coca-Cola. Search for all certificates issued to asterisk.coke.com. And it's like, oh, beverly.staging.testing. Some weird Kubernetes cluster.com. Like, you'd think it's not public, but it totally is. Yeah, wildly so, so. All right, so implementing the plan. So just because you understand what's out of compliance, that's okay. But let's say, I mean, for any sufficiently large organization, it is often hundreds of things that are supposed to be behind a WAF that aren't. And so there's this aspect of like, how do you prioritize them? How do you see people do, because you help a lot of people onboard big Cloudflare installations. Yeah, a lot of it is criticality, right? And talking about the programmatic approach, it's education first. Even with our developers, we bring it very clear. Hey, I will find whatever you put. If you put in GitHub, I'll find it. If you have a new worker, we'll find it, right? And so we have a program that searches every day, a program with policies with technology, I should say, that says, okay, every day we're looking for our own assets. We're looking for new assets, depending on these types of classes, whatever they are. And then we're inventorying them. Then we're going back to the team and saying, hey, this is out of compliance. You didn't actually register this with us, whatever it may be, on our digital side of the house, right? So we can keep track of all of our assets. From there, we can create criticality. And this is what our clients do as well. Once you understand your asset inventory, you can say, hey, you know what? This app isn't actually anything other than like I built a webpage, it has some new whatever. It's not as critical and it's isolated compared to my coke.com asset, which is probably my bread and butter and my biggest brand, right? And so being able to understand the business impact of the web asset or the asset that's external is the biggest piece of what goes behind a WAF, quite frankly, and then the attack classes and attack paths that can come at that asset at any type of speed as well. So we see that kind of prioritization, but again, if you don't understand what your teams are putting out, we've seen instances where developers have put stuff out with passwords and AWS keys, for example, into a worker and it was something they just stored for their own thing. And again, we discovered it in 15 minutes and I had someone's domain credentials into AD. That's not exactly fun, right? And so again, figuring out like what you have is the first thing and then figuring out the criticality of those assets and then where they get placed into the actual structure. And I feel like we could literally talk for 45 minutes about this and we gotta go fast. I do think another key port here is that you're moving from coverage to efficacy. So Cloudflare is a great example. I would say Akamai is as well. There's a difference between sort of an enterprise-grade WAF that's doing, yeah, it's gonna block attacks, but it's doing bot management, it's doing geofencing, it's doing rate limiting versus, you know, hey, I'm running mod security locally on a proxy. So the next part I think is the preventing drifts and also dealing with emerging threats, which is what I wanted to get to most importantly. Okay, cool, if we got 100,000 CVEs this year, are my WAF core rule sets blocking any of that? First of all, does any of that affect me? But how do I make sure that this WAF that I've deployed is actually blocking those attacks? How do we deploy custom WAF rules? How do we deploy custom WAF rules in a multi-WAF environment? Like this definitely becomes challenging without documentation and a very purposeful construction. Yeah, what's really interesting here, do any of you guys have your own harness for vulnerability testing or? A harness around like an agent? Around a transform model? Nobody? All right, what do you, all right, cool. If you have your own, I'll give you an example of what we do on some clients too. You have a harness, a harness looks for vulnerabilities, looks for web app issues, doesn't look for standard OS things, looks for contextualized craziness and business logic, whatever it is, right? And we have massive apps that go through, even if you're not using a harness, you're just using, I don't know, your SaaS tools, right? To your point of like this type of exposure. You can use an agentic function to auto-populate a WAF rule at speed, right? LLMs are fantastic at this. As much as they're annoying, they're also fantastic. You can have a cloud model, an open, you know, GLM 5.3 model, pick your poison, actually create a WAF rule on the fly. And we do this a lot for our clients. We do this internally for us where our models actually will find a threat and then we call it virtual patching. They'll actually write the WAF rule on the fly for the issue and issue it to our threat engineers and say, this needs to be implemented. Can you double check it? Cause we don't blindly trust everything as we shouldn't. And then that threat engineer says, yeah, this is a great deploy and it deploys. And we can do that within under five minutes, right? Or another option here is like deploy right away, but deploy it in logging mode and let it soak. And the really cool thing here is, is that I think a lot of times as WAF people, we are hesitant to deploy custom WAF rules because the ramifications of being wrong is blocking legitimate traffic, which depending on the app costs you money. And then business person comes and yells at you and business person is scary. So people tend to shy away from that. But the really cool thing about if you're using LLMs and you're doing very exacting testing, I'm not applying a rule at the zone level. I'm saying for this particular asset, for the CVE that just came out last week, it has a very specific known payload. Let's put a WAF rule in place and flip it into blocking mode and block it. And then, yeah, let's go actually do the thing we need to do to fix this thing, update the software. If it's an app security issue, stop exposing your settings file maybe. Those are all key. But our thing here is that this can be automated, but to do this, you need to have WAFs deployed. You need to understand where your gaps are. You need to work to reduce those gaps and then start to actually plumb this into a more automated system. Yeah. We're kind of out of time at the moment. It's the fastest speed talk ever. We obviously could talk for like 65 more minutes on this, but that's essentially it, right? The key premise here is like WAF is the Ronco set and forget it, is old. Everything's dynamic. Everything is running fast. Everything's being agenticized. Is that a word? I made it up. I don't know. My vocabulary app will tell me no later. So I can create exploits and I can write attacks on the fly. So I need to be able to remediate on the fly or at least virtually patch on the fly. So you should start taking advantage of using these types of agentic functions to automate threat detection engineering in the WAFs. Exactly. That's a big piece. And so we have two things. One, just a quiz that you can use to sort of self -assess where you are on that, at that chart and get other information. We did this analysis across hundreds of Fortune 1000 companies to essentially get what's hard data. How often is a WAF in logging mode instead of blocking mode? How often is it actually, there's still an exploitable issue behind a WAF there. Like having data is the only way we can get better and actually implement policies like this. Thank you all very much.