All about application security, firewalls, malware, APIs, and more

Hello everyone and welcome to This Week in Net. It's the June the 21st, 2024 edition and this week we're in our London office to talk about application security. It's a beautiful office with a beautiful view, although we cannot see the view from here. I'm your host João Tomé and with me I have Michael Tremante, who is the director of product focus on application security. Nice to be here, super happy to be doing this with you. Welcome to This Week in Net. We have the time set, so let's start. First let's start a bit about your background. You are based in London, here in our office currently, but you have actually a more broad background. Yeah, so I actually started, even before Cloudflare, I worked in cybersecurity. I used to work for a UK-based threat intelligence company called Necron, where I learned. My first day on the job they gave me a big book, which was I think the Web Hacker's Guide to the Internet or something like that. I had to read, I had one week to read through it and I started my career in cybersecurity penetration. Then at Netcraft we did Internet survey research, we tried to take down phishing sites, all sorts of things that are very relevant now, especially for Internet security. I had to move to London for personal reasons and the fun story there is Netcraft used to run a web service survey, they still do to date, and we used to monitor adoption of SSL traffic on the Internet. I had to look for a new job and I said, who's growing the most on the Internet? I looked at our own data then, if you had permission from the owner, and I saw a big spike. I looked into the spike and it turned out Cloudflare had just released universal SSL. Fine, let me see if they're opening a job. Cloudflare just opened an office in London, I applied, here I am. The rest is history. The rest is history. Then at Cloudflare I started actually on the solutions engineering team, speaking to customers and deploying our products. We only had three products then and over the years I moved to San Francisco and I switched to product, doing application security, and I get to date where I'm back in London doing application security. For those who don't know, application security is a broad term, but a lot of the Internet, a lot of what folks have in their iPads, smartphones, MacBooks, computers in general, comes from that realm, right? Yes. What should they know about application security? Well, so application security everything pretty much connects with the Internet today, and if you look at the Internet side of things, there will be a server somewhere in the cloud or in someone's basement giving you information that gets displayed in your application. Sometimes in fact native applications are just thin clients just loading things from application servers, and those servers will be holding end data, they'll be holding information about your connection, the user account, and they also need to be protected. Like a hacker of course has interest in compromising devices directly, but sometimes they even have more interest in compromising the application servers because there's a lot more data there. And that's the space of application security, at least with the Cloudflare lens. If it's an application which is connected to the Internet, be it from hacking attempts, malicious actors trying to overload the application, even simply mapping it out for future use, that's application security, and that's the space we operate in. And it touches, as you said, so many parts of the Internet use of these days. Specifically in terms of the evolution of application security in the past few years, there has been an evolution in terms of use. APIs. Actually we had this in our application report recently. More than half, I believe, of the Internet is done through APIs. So most folks don't know a lot about APIs, but they are actually using it. So how did the evolution... So when I first joined, I remember this was nearly nine years ago at this point, the application security products we offered back then were DDoS mitigation, and in a nutshell that's someone has a grudge against the business and they just want to try to take this offline. I just overload the application, I take it offline, and I basically inflicting financial damage on the business. And the interesting thing about DDoS attacks is they're not really sophisticated in the payloads, it's just dumb traffic trying to overload. And then the other service we used to offer back then was web application firewall. And again, if I have a website, a little bit more sophisticated here, a hacker may try to actually infiltrate the website, and a WAF is intended to be a layer of defense that it will spot when some requests hitting your website are potentially malicious, could break your website, and keep it safe. Now to your point, back in the day most websites were intended to be used by humans. It wasn't a lot, it was already, but it wasn't the main thing, it was computer-to-computer connection. Native applications were not as popular, and over the years we've seen a massive ramp in native devices connecting to application servers, and even just application-to-application communication. And that type of traffic is substantially different from the traffic me or you would generate by opening a browser. And it's opened up to a whole new set of potential vulnerabilities, discovery issues. I speak very often to customers, CIOs or CTOs, and because you don't see an API endpoint when you open the browser, there's just a little less understanding of what's going on. And hackers, on the other hand, know exactly what's going on, and it opened up a whole new attack vector that businesses should be taking care of. And now we see, I don't remember the exact figures, more than 50% of our traffic on the Cloud for Network is API-driven. Back in the day it was less than 10% even for us. So that's been a massive change. And then there's more recent things. Everyone speaks AI, we have to mention AI. How do we protect, it's coming up very often now, how do we protect chat, for example? Someone trying to misuse a chatbot. What if you have a chatbot and you ask it to give you personal information of another user? Do you have policies in place to do that? And that's the space that's evolving really fast, and I expect that will be interesting moving forward. And we've started building some products to help with that problem as well. And the way chatbots are powerful, and they should have those limits, right? True APIs, right. But if you want the chatbot to be able to help a user on their account details, how do you make sure it understands what's the boundary of what they're allowed to share versus not? And it's an interesting field, and I don't actually think there's perfect solutions in the market at the moment. Interesting. In terms of, and you mentioned the evolution, things change, people start using more APIs, actually APIs for those who don't know, Application Programming Interface. Good point. We use a lot of acronyms in Cybersecure. But it's actually a good name by extent, right? Application Programming Interface actually explains what it is. It does. Which is good. But in a way, there's much more now attackers know this, use this more. So when there's a changing in behavior, so more people use this type of APIs for their apps, for their applications, what happens is they use those vectors more, things change, new vulnerabilities are discovered. How do we approach this new vulnerabilities coming, new threats coming types of situations? Yeah, good question. And in fact, the interesting thing is the interface has changed. A website provides a interface that consumed by a browser, and everyone used to build and still does build websites in totally different ways. So the security model there is a, what we call a negative security model. So with a website, we try to block only the known bad. Whilst with APIs, you can flip it the other way around, because it's structured data, because it's app-to-app communication, we try to apply a positive security model, whereby we only allow known good, and we block everything else by default. However, in both scenarios, the new attack vector is the problem for both, whereby, regardless if you have an API, if you have a website, you might be using a software component, which is off the shelf, or you'd purchase from a SaaS provider or someone else, and they may have a vulnerability that you don't know about. And we call those zero days in the field, right? Before anyone knows about them, it's a zero day, and it's got a really high value. And we're seeing recently an uptick in zero day vulnerabilities in software vendors, and it's becoming a growing issue. Internally, we've done two things. So even back when I joined Cloudflare, we have this concept of managed rule, and they apply both to websites and API endpoint, by the way, whereby we have a team of security analysts internally to Cloudflare. We're constantly monitoring things like Twitter, but also more cybersecurity-specific forums. And whenever we see that someone has discovered something, either we find out about it immediately, or a CVE is published. CVE stands for Common Vulnerability and Exposures Database, where companies are incentivized to publish the vulnerability so companies are aware of it. To share the information. To share the information, right? Because you want companies to be able to react and patch their software. And as quickly as possible, and leveraging what we see on the Cloudflare network, we try to come up with new rules and signatures within our WAF and our API shul product to keep those payloads detected and blocked before they compromise the backend application. Now, the interesting point there is we did the research not too long ago. In most cases, when a proof of concept of the attack gets published, we see attack even within 30 minutes of that proof of concept being published. So really, really, really fast. And even our humans, very smart security analysts, they are fast, but they're sometimes not fast enough. So what we've done more recently is we've developed a new model. It's a classifier, so I'm not going to even use the term AI in this case, that has been trained on all the known bad traffic we see. And we've deployed the same model on the same traffic. And it's able to detect variations and bypasses of existing attack before we even have a signature written for it. And it's become very, very useful as a side effect to blocking new zero days because new attack vectors, although they've never seen before, they tend to be variations of existing ones. Older protections can also work. Right, exactly. So in fact, very often older protections also works, but when they don't, but it's similar, the ML model we call the WAF attack core has proven to be very, very good. And we actually have a library internally of CVs that the model will block payloads for out of the box without a signature being written. And we call this internally blocking zero day before zero day. And we're very happy about how well it's performing. More recently, for example, there's been one blog post you actually wrote about reducing supply chain risks with new tools there. In what way supply chain, for those who don't know, why is it relevant? Why people use them and how does it work? Yeah. So the WAF and the API protection we provide are protecting the backend application. And then the other thing that's increasingly important nowadays is, as you said, supply chain. And it's come into play a lot because, and Cloudflare is also one, a software as a service company. Software as a service companies are becoming the norm to build a business on the Internet. And every time in the context of web applications, you want to leverage another company's software. More often than not, you are integrating a library from that company, but loading that library from that company's hardware and resources. So at some point, if someone visits your website, you know, micro.com, they may load Zendesk chat app, or they may load Stripe payment provider. But those libraries are outside of my control. And hackers know that. And hackers also know that if they do compromise a Stripe or a Zendesk, they will get access to all of Zendesk and Stripe's customer. Very high value target, right? And this is where the concept of supply chain comes into play. It's much broader than just web applications. You have supply chain attacks, even in hardware, right? You know, libraries are loading or what hardware components you've got built into your hardware. But in the context of web applications, monitoring supply chain has become very important, because it's very easy for a web developer to just add another JavaScript file to your site. And to protect against that, we've developed a new feature called PageShield. And essentially, actually using open standards, we are able to ask the browser, when they're connecting to your application, what libraries are you loading? And the browser will send us information back about what third parties are being loaded. And then we can provide, you know, on top of that, monitoring tools, even providing an inventory of the assets is very valuable so that, you know, as a company, you can make sure you know who you're loading, who you're sending data to, and keep an eye on if they get compromised. Because then we have malicious detections that behavior changes, for example. Makes sense. There specifically, there's also a lot of conversations on what level of trust you should have when you start building, adding supply chain, other suppliers, different suppliers to your stack. In what way one should feel safe with what is out there, what we actually provide? I wish I had a good answer to that one. What we normally advise, and actually my, I'm sort of aligned with, so PCI, for doesn't know payment card industry, they have a standard. So if you transact and you hold credit card details or card details, you have to be compliant. And their approach, which is not a bad one, is unless really necessary, try to load the least amount of third parties in your site as possible, because otherwise your attack surface becomes very large. So that's the first thing I'd always advise. And tools like Paychill will help you reduce that attack surface. But then beyond that, of course, there's some, you know, basic best practices you can follow. Make sure that everything you loaded is coming from an encrypted channel, for example. Make sure that you've done some at least basic background checks on the company. Have they been compromised before? Do they have a security first mindset? Although we sell, you know, application security, really security should be built in in any company's sort of process out of the box. And you also understand, which often gets forgotten, what data you're potentially sending to those third parties. Because sometimes we've seen cases where even if they don't get, if you don't get compromised because they were compromised, you may actually be sending user data to those third parties. And hackers sometimes get access to that third party data. And it's still very damaging to your organization, right? And it's very easy for a developer to add a widget to a site without realizing that a widget, I'll make an example here without necessarily naming or shaming anyone, but let's say Facebook pixel gets added to a site or, you know, LinkedIn tracking. By default, all those tools send a lot of data to those services and it's actually increasing your exposure for your own end user data. So keeping an eye on that is very, very valuable. There's a lot of bad actors, a lot of malware. So old things you mentioned in the beginning, actually DDoS. It's an old tactic to do it. There's attackers still using it a lot, potentially more than ever now, but malware is around for many years now. How are the techniques right now for malware and how they evolved really? So yeah, the more we're going through all the different attack factors, I mean, you think security is still a hard topic to cover. So you're right. And in fact, even malware comes into play for application security a lot. If you zoom out a moment, malware often gets talked about in the context of phishing and, you know, social engineering. I'm a staff member of a company. I download a file from someone. I trust it, but in fact, it's malware. I open it up and then my computer gets infected with ransomware. Now, phishing and email security are often the main attack factor. However, lots of people forget that even application security plays a big role here. I'll give you one example. Let's assume you're running a website that publishes job listings. And normally if you're publishing job listings, you want people to upload CVs to your website, right? So that your users can review the CV applications and decide if they want to interview the person or not. There's lots of other examples where you're uploading files to an application and we've seen attackers try that as the vector to upload malware within an organization. And especially with the increase of API endpoint, because of this sort of attack factor, we developed what we call WAF content scanning. So in line to your application, we are able to detect file upload, which are normally intended, but block when we see that something is off. For example, the file may be either encrypted and you can decide if it's good or bad, or it potentially has malware inside of it before it gets uploaded somewhere where a user may go and open the file and then get infected. So really, when you think about malware, there's many ways that can enter the organization, phishing being one of it, or email security being one of it, but actually application security is in one other way for an attacker to try malware inside the organization. We're trying to block it at that. Makes sense. And Cloudflare has different services, email security also, but in this case, in the application layer also, it's important, right? Yes. For those who don't understand really this way of proceeding and doing things, what is the thing that they should know more about application security that potentially in 2024 is still not as understood as it should? Good question. From a user perspective, so let's say you don't actually own the application itself, the basic rules always come into play, I think, even for application security. ASOS actually helps with not getting compromised by phishing attacks. Make sure that the applications you're signing up to feel they are the legitimate application, always check that the main name you're accepting is your connection encrypted. Every time I go to an application and I make a payment, I actually make a judgment call if do I want to transact online based on even the look and feel on the quality of the payment page. If something doesn't feel right, I tend to sort of step back and think about it twice. It's very important, every time you're submitting user data to an application, make sure it's a trusted organization. I would say overall application security is getting more understood, so things are getting better. From the other side, I would call out the API security we mentioned earlier. I've seen a growing maturity in web developers when building applications, so your traditional attack vectors that would have worked 10 years ago, I'll mention the textbook ones, XSS attacks, SQL injection attacks, are often quite well taken care of. But with the API-first approaches now, developers are spinning up all of these APIs, and sometimes they're secure, but they go often unknown by the CIO, the CTO in the organization, and then it becomes a problem when they're no longer maintained. And it goes back to what we were saying zero days earlier, something that could be secure today doesn't mean it's going to be secure tomorrow. And I see that's where attackers are really finding their sweet spot now, because you might be running an API that was developed five years ago on a given software component, a CV comes out, the team has moved on to something else, and then the attacker is able to compromise it. So having a good idea of what software you're using, a good process to patch vulnerability, even after the fact, is super important, and that's where we need to become more mature. In what way, and you talk with customers, you get feedback, in what way do you think having access to our different tools for these types of process helps customers? And usually customers want to plug and play, and don't worry about it, but in what way does it work like that? So the first thing that you get really, especially using Cloudflare is the visibility. I was speaking to a customer actually yesterday, we had Cloudflare Connect, one of our big events, and they just onboarded their traffic and put their application behind Cloudflare, and they had the monitoring tooling already running, but our dashboard came to life, and we provide what we call always-on detection, so without you doing anything, you get to see the attack traffic, which endpoints traffic are hitting, potential DDoS attacks, all sorts of information we provide out of the box, and just by looking at the dashboard, they literally told me they were super happy, they found out about parts of their applications they were unaware of, and they started an internal review. So visibility is the first thing, just pure plug and play. As soon as you're proxying traffic, you get visibility, already improving your understanding, and your security posture. After that, there are some, of course, additional things. We mentioned earlier the managed rules in the WAF. If a new zero-day comes out, we are there, we're writing a rule, as long as the managed rules are deployed, we're simply buying you time that you can then fix or patch the application behind that, and with attackers exploiting them in less than 30 minutes, that becomes really important. So I would say normally we see an onboarding take even less than an hour for a standard web application, and the benefit you get in terms of plug and play is pretty good. First person to say, you always need to protect the application back, so always have some security process to make sure you've hardened your application as well. Especially to make sure that you're more protected. You can have a bit of protection, a lot of protection, but then to have the full scope, you must do also work. The one thing maybe where that doesn't hold true is DDoS mitigation. You need, unless as a business you have a lot of money to invest in hardware and bandwidth from CPU capacity, you need a service in place, and I do believe Cloudflare is the leading service for DDoS mitigation, and if you want to solve DDoS, get Cloudflare. It's the easiest thing, you just deploy it and it's done. And that's the volume game, it's not a sophistication of the attack game, and you need volume to fight volume, and Cloudflare provides that out of the box. And we've seen attackers using DDoS sometimes just to focus the attention there, and then going with other methods. Sometimes it's used as a distraction technique, yes. And in fact, if you're using Cloudflare at the very least, you're not worrying about mitigating DDoS, and if something else is happening, you can focus your attention on that. Well, sometimes a company gets attacked by a DDoS attack, and then they're worrying about keeping their application online, and in the meantime the attacker is exfiltrating data from another endpoint. And we've seen this, we've actually seen this happen. And in Portugal it was on the news two years ago, something really close to that, exactly like that actually, with a big airline. Thinking of the future of 2024, 2025, what worries you still of what's coming in this area? Yeah, so I mean, we mentioned AI earlier. I think putting AI aside, the use cases companies are using AI for, and exposing to the Internet, are super interesting. I think there's going to be a lot of value generated there. But the interface to an AI endpoint is different, because it's a natural language, normally interfaced, where a user will ask a question and you get an answer. So even putting aside the traditional attack vectors, there's going to be a lot of interesting discussions around how do you make sure a chat, go back to the chat bot example, is not misbehaving, is not accessing data or providing data they shouldn't provide to the given user, right? Things like hallucination, is a chat bot talking about politics when it shouldn't for a given e -commerce site, or using it for, you know, unintended purposes. We've seen examples of chat bots being able to even perform a good code, right? Intentionally, for some use cases, but then will an attacker be able to abuse that to perform attacks somewhere else? Are you sure your chat bot, if I ask it to check information about a given website, will actually try to open the website and cause some damage, right? There's a lot of new attack factors that are going to come up, especially as we're not mature in creating the new tools, but I think they're going to be focused for the next couple of years and are going to be super important to take care of. I mentioned one other thing, actually, looking forward that we've seen come up more and more often, which is actually going back to API discovery and network discovery. Companies are all doing the digital transformation that started many years ago. We'll still be talking about it in 15 years, probably. It's becoming very difficult for a single person within a company to really understand where your app is living, and that's causing a lot of discussion recently. And in fact, at Cloudflare, we're considering providing some scanning tooling within the platform to even find out things that might be outside of the existing Cloudflare scope that could be vulnerable to attacks. And that's starting to be quite a hot topic recently. That's interesting. If you have visibility to what you use, vulnerabilities that you can actually have, you can act better, know better, in a sense, right? Exactly. And you know, everyone now has a public cloud, multi-cloud, all these sort of big, big, big keywords. The complexity is, we go in cycles in computer science, right? Complexity is going up again quite a bit. And what excites you the most still about this area? The exciting thing for me is always changing. I've now been working in cybersecurity for 16 years, which is not a small number anymore, and I've not stopped learning. And every day there's something new coming up, right? I'll mention one other example, bot management, automated traffic. There's attackers out there building automated tools to scrape information, and there's a continuous fight going on between the security providers like us and the attackers trying to come up with the next best bot that can go unnoticed by our detections. And every day there's a new idea, there's a new thing coming up. We have things now that are also making it harder for us in a good way. Apple private relay and other technologies, because of course we want anonymity on the Internet, right? And those are all really good things. But at the same time, it makes detecting bad behavior harder, right? So we need to adapt our technologies to take that into account. We just actually, I was speaking to some engineers on the team, there's attackers out there that compromise Internet of things. So if you're running a smart fridge at home, you never know, there might be a little bot there. And then they resell that access to that bot for someone who may want to scrape someone's website. But because the connection is coming from your fridge, from your residential IP, it's very hard for us to detect that and block it. And these are just some examples. It's constantly changing. I'm still learning every day, something new. And it's the opposite of boring at this point. So yes, I'm excited about it. I don't know. Let's see what's going to come up in the next year or so. I never know. And you gave a few good AI examples that are not as the current ones that are being discussed that are interesting. This was great, Michael. Hope you liked it. Yeah, no, it was fun. We could probably talk for hours enough. True, true. With all of this. Yeah, but actually there's still time here, but we have to end. Thank you. And that's a wrap.