🔒 Cloudflare: the faster Zero Trust portfolio & Internet security trends

<v Dan Gould> Hello, everyone. Welcome once again to Cloudflare TV. Very excited to have you here today on Friday of Security Week 2023.

So much has been going on this week and actually right now, we're actually here to talk about a couple of reports we have published on our blog this week related to security.

And one really looks at security performance really through the lens of Zero Trust and the other looks at some security insights from the the unique vantage point of our global network.

My name is Dan.

I'm on the product marketing team responsible for application security. I'm joined by a couple of great guests.

I'm excited to have them. David Tuber, Tubes, and Michael Tremante.

Tubes, do you want to say hi? <v David Tuber> Hi.

That's it, right? That's all I have to say. <v Dan Gould> Yeah, maybe a little bit more.

<v David Tuber> Yeah.

I'm Tubes. I'm a product manager at Cloudflare focusing on network performance, network interconnection, performance and availability.

So I'm here to talk to you about the Security Week performance post.

Michael, you want to give a little intro to yourself?

<v Michael Tremante> Sure.

Hi, everyone. I'm Michael Tremante.

I'm also a product manager on the application security team based out of London.

Excited to be here today. <v Dan Gould> Awesome, awesome, awesome.

Well, listen, as I mentioned, we had a couple of really cool blogs that really look at both performance and security.

Actually, Tubes , why don't we start with you and we'll talk about the report on performance actually published today on the blog.

And you know, this look at performance is a recurring one.

We seem to do it every innovation week or nearly every innovation week.

Why is that? <v David Tuber> Well, gGod question, Dan.

And the short answer is really performance matters end to end, no matter what scenario you've got, whether it's, you know, Zero Trust application services or it's, you know, you know how fast, you know, we go through the WAF or whether it's, you know, how fast, you know, you can how fast you can execute code through Workers, anything like that.

Performance matters because latency at the end of the day is what drives costs, is what drives revenue.

So about like 10, 15 years ago, Amazon did a study where they showed that every 100 milliseconds of latency that they reduced accounts for about an additional 1% in revenue.

Google around the same time found that if you increased latency by even, you know, ten, 20 milliseconds, they saw about a 20% decrease in search and search times.

People just don't want to wait for their stuff.

And that's true in any scenario.

Like, think about it when like, have you ever used a work VPN?

You know, how many times have you spent?

It's kind of like rolling your eyes as it has to authenticate and connect and get you there.

Or like even when you're like on your phone and you're scrolling through like Instagram or Twitter and it's taking a long time to load and you're like, Oh my God, this is so annoying, right?

Like that actually makes money.

And that translates directly to revenue. So performance really matters and performance in the security and Zero Trust space matters, but in a slightly different way.

So, in security, performance really matters because, you know, the faster your WAF, the faster you'll be, that that direct revenue translation really makes sense.

But for Zero Trust, when it's all internal, how does that actually impact?

And the really good answer is that performance is a is a threat vector.

So if you've ever used a VPN and you've had a really bad experience, what's the first thing you do?

This is actually a question like, what's the first thing you do?

And you have bad experience with a VPN. <v Dan Gould> You try to turn it off?

<v David Tuber> You turrn it off.

Right? And when you turn it off, that really creates a serious problem for the IT team because now you're accessing services unprotected.

You are putting the company at risk.

And so if I'm an IT manager, I want to do everything I can to make sure that whenever my employees are using Zoom, whenever they're using anything that they would use in their daily life or in their day to day work, that they are as performant as possible so that it's so transparent to them that they can't even think to turn it off.

<v Dan Gould> That's incredible.

So I think, typically we think of performance. Obviously, it's bad for business.

That's when quantified. And actually, I wanted to just quickly touch on something you mentioned.

So you said that a matter of milliseconds can can have profound impacts on revenue.

Is that right? Like 10, 20 milliseconds? <v David Tuber> Yeah.

So, yeah, like it's very, very small and we're getting ever faster. I think everybody is expecting their experience to be lightning fast and to be not just lightning fast, but to really be responsive to user feedback, right?

Like when you play games or when you do anything on your phone, that responsiveness is really what matters to you.

And even when you talk to people who don't use the Internet or who aren't like, you know|, in Tech and building the Internet like we are.

Like they get that. They get that idea that like 10, 20 years ago, success on the Internet was being able to send an email with an attachment.

But nowadays, success is not just sending an email with an attachment, but sending an email with an attachment like that.

Like very, very fast. So like, the paradigm has shifted and it's always going to we're always going to drive down and we need to get faster and faster and faster.

And it's not like a we reach a goal and we're done.

It's we've got to get faster and faster and faster. It's like one of those, like, asymptotic problems where like, we're just getting the long tail.

<v Dan Gould> They feel like it.

And the reason I brought that up, people might assume that, hey, what's ten milliseconds between friends?

Well, it turns out it's everything, right?

From what I'm hearing, right? People may assume, well, it's, you know, whatever speed fast enough, probably not, given the fact that just a mere, you know, a handful of milliseconds so profoundly impacts revenue.

And then the other thing is, most of us have used VPNs.

They stink. And they're you know, they frustrate users.

And I never actually heard anybody talk about their security. Right?

Because it's so frustrating, you just turn it off and then suddenly it's this naked connection and that's a ton of security risk.

So basically poor performance equals more security risk.

<v David Tuber> Yeah, exactly.

And I think another way to think about this is that the people who are actually using the VPNs don't actually care.

Right? Like people who like like like if you have a VPN at home, you're not using it because you think it's the coolest thing.

If you have a VPN at work, you're not using it because you think it's the coolest thing in the world.

And like, I can't wait to try out this VPN, right? You're using it because you think because like your company is telling you to and like if you could not use it and still get the same performance, 10 out of 10 dentists would agree that you should do that, right?

So like it's really just a slam dunk that if you can make security and Zero Trust as transparent as possible to users.

Then you're going to be more secure because people won't mind having a VPN on if it's performant, if you don't notice it, you can't complain about it.

<v Dan Gould> They won't.

Exactly. They won't even notice. It's funny, like, you know, I may or may not have been guilty of trying to turn the VPN off, you know, in my lifetime, but and I'm not.

I don't care. I'm not the one who has to answer to the board.

I'm not the CSO. Right. Like, you know, like, what do I care? I just want, like, my Internet to go faster, right?

So I can get my work done. So that's actually a really great intro.

So as we think about this, like I guess very briefly, we'll dive deeper.

Like what in a nutshell, what did we cover on the blog today in this update?

<v David Tuber> So a lot of stuff is basically like when we talk about VPNs, the first thing like when when we at Cloudflare talk about VPNs and how much they suck, the first thing we think about is how can I get rid of a VPN?

And the answer is Cloudflare Access and Cloudflare Access is a great VPN replacement because it provides secure, secure application access protected by Cloudflare authenticated with your authentication provider.

So it's basically role based application access protected and improved by Cloudflare.

And you know, a lot of people talk about secure web gateways and how you need to go out to the internet and all that stuff.

And yeah, like that's a very important part of that stuff.

And having stuff like the Warp client or the Zscaler client or the Netskope client or whatever client you have is important, but at the end of the day.

Most users are going to be accessing the Internet through a set series of applications like they're not going to be accessing, you know, random website.com most of the time they're going to be going to Salesforce, they're going to be going to Slack, they're going to be going to Zoom, they're going to be going to all of these places.

And having those encrypted and secured with Cloudflare Access can really improve your performance while maintaining a Zero Trust security posture.

And so what we did was we looked at Cloudflare Access and we compared it to some key competitors in the field.

So we looked at Zscaler. So basically we looked at Zscaler last week, but we want to make sure that we're continuously looking at Zscaler because, you know, doing it once is great, but doing it multiple times is a pattern and we want to establish a pattern.

So we're going to look at Zscaler, but we're also adding our friends at Netskope and taking a look at them and seeing how they fare in application access.

And so what we did was we took six application hosts, sorry, five application hosts around the world.

Bless you. We took five application hosts around the world.

We had one in Toronto. We had one in Los Angeles.

We had one in Sao Paulo, one in Hong Kong and one in Johannesburg, South Africa.

And we basically had 300 catchpoint nodes from around the world hit every single one of those.

They tried to log in. So basically do a new connection, establish a new connection.

And then we had those same catchpoint nodes reconnect with an established connection.

So basically this mimics kind of two key scenarios that you tend to see.

One is when you have to log into your application and once you've logged in, then you know, you're, you're kind of reused connection.

And that's really important because those two scenarios look very, very different in terms of performance for many different reasons.

And we want to look at both of them because A) it will show us how we compare against our competitors, our competitors in terms of just, you know, authentication.

If authentication takes a long time, you're just kind of sitting there.

You might hit refresh a couple times spinning your wheels.

But at the end of the day, what really, really is going to matter is that steady-state application performance when you've already authenticated.

So we look at both and we look at them separately. And so that was the comparison test that we did against Zscaler Netskope.

And we were 50% faster than Zscaler and 75% faster than Netskope.

So definitely doing a lot of work to get really, really fast in in this application VPN replacement space, which is really, really important because a lot of people use a VPN or have or have a need for a VPN replacement.

<v Dan Gould> Yeah.

So I hear you say that like 75% faster. That also sort of translates into 75% less chance that your employees are going to try to turn it off and not not use it, you know?

<v David Tuber> Yeah.

I mean, like you could definitely say like that. I think that the way that I would say this is, is really like and when we look deep into the numbers, I think that this definitely backed it up is that Cloudflare's network is just closer to end users and that makes it really easy to access applications.

You know, these catchpoint nodes that we tested from are about as close as we're going to get to being embedded in ISPs.

So it allows us to see what an end user might look like. And so if I'm from coming from these ISPs into all of these providers, we can really see who's got the farthest reach on the Internet and who can get your traffic onboarded faster.

And the thing that we see over and over again with all of these performance blogs and these performance reports that we do, is that that's really the key driver.

Like you talk about like how does ten milliseconds matter, right? If like, so for me, I'm from Seattle and I live literally like two minutes away from the largest interconnection point in the Pacific Northwest.

And this interconnection point is probably a millisecond away from my house.

If it was ten milliseconds away from my house, if I needed to establish a new connection to that interconnection point or to Cloudflare that interconnection point, for me right now, it's three milliseconds for me, ten milliseconds away.

It's 30 milliseconds, right?

That's actually an order of magnitude of ten. And that's a lot.

That's a huge difference, right? And it's actually more than that. It's not three milliseconds.

It's about 30 times, it's about ten times, what it is because you have to do something called a three way handshake.

So you have to, you know, send a request, get a response back, and then send out another request and then and then respond that you have to do whenever you're establishing new connections, whenever you're doing, whenever you're doing TLS handshakes, all of this stuff adds up.

And the farther you are away from your users, the worse it's going to be. And so that's reflected not just in the Zero Trust space, but when we did our overall network performance report, we can see that we're faster for TCP connection time against not just, you know, the Zscalers and Netscapes, but also the CloudFronts, the Googless, the Akamais and the Fastly..s.

We get out closer to you and that makes it easier for us to get your traffic on us faster, which improves your performance end to end.

<v Dan Gould> Powerful stuff.

Powerful stuff. And I'll tell you, like. What I'd love for you to talk a little bit about the fact we're constantly scrutinizing ourselves for ways to get better, ways to get faster.

Can you talk a little bit about that, how we're always seeking to to to improve here?

<v David Tuber> Yeah.

So like this data is not just for like show, like we don't just make this data, we don't just like get this data and say like, "Hey!" You know, we don't just get this data and be like, "Oh, yeah, cool.

Like, we're faster, let's go publish a blog." We internalize this data and our engineering teams are our engineering teams and our network teams and our interconnection teams are always trying to find ways to get faster.

The network benchmark data that we have and we've been running since Speed Week 2021, we use that to infer where we should set up peering points and improve interconnection.

The comparisons between our between our competitors, even for developer week or even for our developer platform, even for our WAF and all that stuff, we use that to fuel performance improvements so that we're constantly staying faster.

This data and these tests are not just so that we can tell the world that we're faster.

It's so that we can actually be faster. And if we're not faster, we get faster and then we tell the world about it.

<v Dan Gould> Yeah, indeed.

Last question for you before we go over to Michael. I know, I believe was it last December, Tubes?

We did our last sort of check in on performance. And then here we are in March.

And I know we sort of keep very close tabs on the number of networks in which we're fastest globally.

How is that number shifted over the past few months?

<v David Tuber> Yeah.

So I think the last time we did it was November when we looked at this last, and we were at 42%.

Now we're at 47%.

So definitely a big jump.

You know, that's just a result, as as I said before, like we use this data to fuel where we peer.

We use this data to fuel how we interconnect with the rest of the world.

So it's no surprise that, like, you take a couple of months, you give it a couple months and we get faster.

So that's just kind of how we work in that space.

Okay.

Well, hey, thanks for for for the post today and thanks for all your work and keeping us faster because as we've established, I mean, you know, better performance equals, you know, less security risk overall.

So great work, Tubes and I'm sure you know, we'll see you on Cloudflare TV in the future, or you know, our next check in on this.

With that, we've got Michael all the way from London, let's bring you in here.

And you know, we've spoken about performance. Let's actually talk about sort of threat insights and security insights and something you've certainly thought a lot about sort of leading the sort of WAF product area.

So just, you know, I guess to get us started, what did we publish on the on the blog this week?

<v Michael Tremante> Yeah, great question.

So just like David talked about, performance. Security, of course, is the fundamental thing we're trying to improve.

Right? Um, so this week we actually published our second application Security Internet Trends report.

We started, if I'm not mistaken, March of 2021 with the first version of this report, and we decided to take a look.

We have millions of Internet properties behind the platform, behind the network.

This gives us unprecedented visibility and we wanted to share some of the trends we're seeing with the public, with the community at large.

So we took a look at the data.

Of course, most of the data we have access to is anonymized data. It's only, you know, top level statistics on the things we're seeing, the things we're blocking, the malicious activity going on on the Internet.

And we looked at quite a few different aspects of it, right?

Malicious versus good traffic, automated versus human.

And the good news about this year's report is, because it's the second time we're doing it, we actually have one full year of history on some of the trends we published and talked about.

<v Dan Gould> Yeah, yeah, indeed.

And actually something I know we talk a lot about this internally, Michael, you and I.

And you know, just to summarize for those watching, why do we have such a good vantage point?

Like, why are we uniquely positioned to release these insights?

<v Michael Tremante> Yeah.

And I think it goes back to also what David was mentioning. Every every month we're building a bigger, better, faster network.

We're closer to end users. And at the same time, we're getting more Internet properties behind us.

So we actually look this up.

Now besides the millions of Internet properties that have onboarded onto Cloudflare, and in this context, I'm mostly referring to web based applications, right?

So if you have a, if you have a website, a blog, you can just head over to cloudflare.com, sign up, bring your DNS over to Cloudflare and then turn it on and you get faster, better, secure.

And we of course, because we're sitting in front of your site, we get to see that traffic.

And as of last quarter, this equates on average to 45 million HTTP requests per second.

Okay.

That's a mind blowing number if you ask me. And that was up from 32 million last year.

So year on year, the amount of traffic we're seeing is increasing. This is purely HTTP based traffic.

We do a lot more than just HTTP at Cloudflare. To give you one other example, of course, we also offer a public DNS resolver.

So if you've ever heard about Google's 8.8.8.8, Cloudflare has the 1.1.1.1 and, as more and more users are just using the DNS Resolver, even for their home network, we're seeing something, if I'm not mistaken, more than 24 million DNS queries to our Resolver per second.

And even that for us is a great resource for security insights, right?

How quickly and how often are customers potentially resolving to then connect to known malicious domains?

So that just gives us really good visibility on what's going on.

I would say probably one of the best out there. Yeah. <v Dan Gould> That's a vast, vast, vast amount of traffic we see every second, let alone every day, every month, every year.

So actually, that's I've been getting to know some of the findings and I guess we'll start from the top in terms of, you know, how we're thinking about, I mean, people want to know how many threats, what have you blocked?

And, you know, on average, Michael, like what did we see in terms of the amount of traffic we're blocking or mitigating, as we say in the blog?

<v Michael Tremante> Yeah.

So let's define it first, because to make sure it's clear. Customers that onboard onto the Cloudflare platform can configure their security settings.

Of course, there are some things that we know are just bad. A typical example, this is DDoS volumetric attacks.

If you're running a site and someone has a you know, is grumpy and wants to take down your service, they may launch a DDoS attack against you.

If you're running a website and we see completely unrelated protocol traffic, not, you know, TCP or HTTP traffic towards you, our proxy is just going to just block it out, right?

Because it's definitely not something you're expecting to receive.

But then there's a whole portion of configurations that users and customers can configure and therefore actively decide to block.

In fact, in the security world, some customers are implementing what we would call positive security model, where they they only allow known good and therefore everything else is blocked by default.

But if we look at HTTP traffic that's, in this context, mitigated.

So, Cloudflare has performed an action at the edge on the network, basically about 6% of all HTTP traffic we can deem somewhat malicious or unwanted by applications in Cloudflare.

And that's a pretty substantial percentage of all internet traffic.

<v Dan Gould> Yeah.

I mean, when you think about that, the scale at which we were talking about before, that's a ton of traffic.

And we did this last year, as you mentioned. How is it how are we trending year over year?

<v Michael Tremante> Yeah, that's a good question.

Maybe surprisingly, it's actually trending lower.

From memory last year, it was closer to 10%. So I was, of course, being in the security space, I was expecting, you know, bad traffic to be going up and up as a good reason to buy more security products.

Um, I don't necessarily think, um, the, you know, 6% is still a vast amount of malicious traffic on the web.

But in fact, we actually had a bit of a look into this. The total malicious traffic if you count, you know, per million requests per second actually hasn't changed.

What we've seen is the growth in clean traffic. This can be due to quite a few reasons.

First and foremost, of course, Internet usage is still going up globally, right?

And single, you know, very large applications as we also on the Cloudflare network onboard larger and larger customers.

We do expect, you know, the total clean traffic to go up.

<v Dan Gould> Interesting, interesting.

Okay. So we you know, Michael, you know this better than most.

We've got, you know, an integrated application security portfolio. And many of these products are helpful in keeping organizations safer, right?

Protecting their APIs or applications.

And I think we can look at all of those products and how customers use them to protect their business and sort of back into some of these insights to really understand the most common attacks we're seeing.

So, you know, that said, I wanted to ask you about that.

Like, how did this sort of the attacks break down in terms of what's most common and what products seem to be really blocking the most?

<v Michael Tremante> Yeah.

And this actually dives into one of the trends we do notice, which is the sophistication of attacks is going up.

So first and foremost, not surprisingly, DDoS mitigation is the cause of the most mitigated attacks, right?

By definition, distributed denial of service or volumetric attacks.

We actually published a blog post not too long ago where we believe we've seen the largest known DDoS attack against our network, which peaked at over 70 million HTTP requests per second against a single Internet property.

So just imagine being behind an attack of that size.

But proportionally, year over year, the WAF is now mitigating a lot more.

I believe from memory, over 40% of mitigated traffic is coming from the WAF. And this means that there's more layer 7 HTTP traffic that requires more sophistication and more logic in our security portfolio, right?

WAF blocks are normally on a per individual request basis.

Whilst the DDoS attack is coming in, we can say all the traffic from this protocol or this range, you know, is deemed malicious.

WAFs are looking for malicious payloads. For example, a hacker attempting an XSS or SQL attack would be blocked by the WAF.

And and this has actually increased quite a bit, which means attackers are getting smarter.

<v Dan Gould> Yeah, indeed.

And I think, you know, with the WAF, we actually have multiple layered rulesets at work here and I believe this accounts for all of them.

Right, our custom rules or OWASP rules manage rules, all that.

Right? Working together. <v Michael Tremante> Yeah.

So, in fact, if we if we actually look at the WAF, manage rules, in my opinion, is where most of the value is.

Manage rules are the ones that are built by an internal security threat analyst team.

And we're actually deploying signatures which we test on the global network before we of course provide in blocking mode to our customers and things we look for, SQL injection attacks, all the commonly known standard attacks, but also, you know, more sophisticated remote code execution attacks, cross-site request forgeries, etc.

Maybe not surprisingly, the top vector remains the same as last year, HTTP anomalies.

A big chunk of web traffic is vulnerability scanners and vulnerability scanners are often looking for weird behavior in applications.

What happens if, you know, let's assume there's a 'contact me' form.

And rather than submit a well-formed post request with information that I'm submitting, I try to fiddle with the HTTP request format and provide some broken fields and see what's going on, right?

And, normally, these type of scanners will be looking for behavior, unexpected behavior, errors coming out from the web server.

And sometimes they become exploited for, for example, the denial of service attack afterwards.

<v Dan Gould> Interesting.

And what else? I think directory traversal, that was on our list too.

Is that right? <v Michael Tremante> Yeah.

Directory traversal is again very, very common. Lots of applications out there are set up, you know, from pre-built open source content management systems and similar, and web developers often do not check or do not know that there are hidden files in their application stack.

They don't lock them down. And very often attackers are looking for those hidden files that may contain passwords, IP addresses, internal URLs for systems.

And yeah, it's one of the most popular vectors we we block on the web.

<v Dan Gould> Indeed.

And essentially a good WAF should take care of this, right? From Cloudflare or anyone else.

These are common attacks. Obviously we take them seriously, but organizations obviously make sure they're in good shape.

So let's shift gears a little bit.

We looked at a couple of other areas and we'll talk about them both briefly, both automated traffic or bot traffic and then also APIs.

And we'll start with bots where they really quickly I think, you know, at the highest of levels.

How much automated traffic did we see on the Internet? <v Michael Tremante> Yeah.

So this is another change based on last year, but we still see about 30% of all Internet traffic to be automated.

And what do we mean by automated? We do have on our platform our bot management solution.

We are essentially scoring every individual request with a score from 1 to 99.

One means we're absolutely certain that the request was from an automated nature.

So not driven by a software used by human.

99 being the total opposite. Of course, you, me opening a browser, that's human.

So 30% is still pretty good chunk of all Internet traffic is automated. <v Dan Gould> Yeah, you know, I know we've spoken about this in the past, and occasionally people's first thought is, well, what about Google and Bing?

Like, that's automated, right?

That's good traffic. Like how does the sort of the good automated traffic stack up with the rest, which is probably you don't want near your site?

<v Michael Tremante> Yeah.

Surprisingly, the vast majority is non-verified. In fact, the Googles, the Bings, etc., only account for 8% of that.

Everything else can be, which we classify as non-verified, and our classification verified is quite comprehensive.

We have several hundred verified bots classified by our system that we are aware of.

All the big ones are in that list and yet everything else so custom made.

You know, if someone has a project they're working on with Little Crawler they've written, would of course be classified as unverified, including what I mentioned earlier, the vulnerability scanners.

And there's a lot of background Internet activity by attackers just trying to, or exploiting, you know, latest CVS and similar and all of that is 92% of the automated traffic we observe on on on the Cloudflare network.

<v Dan Gould> Yeah.

And that's obviously, you know, these numbers really sort of fuel us as we think about our bot management solution.

And obviously we block all the bad stuff and give organizations a flexibility to allow all the traffic they want to, including some some good automated traffic that's just Google and Bing.

But there are other sort of automated services they'll need to run their business.

So we give them the flexibility to really thread that needle.

Right. So before we wrap up, APIs, obviously they drive business today.

They do to an extent, make modern business go round.

Let's talk about that. So overall, like in terms of API traffic, what are we seeing on our network?

<v Michael Tremante> Yeah.

So there's, the first of all, there's a lot of it. Even there a good chunk of, in fact, depending on how you look at it, more than 50% of individual HTTP requests across the Cloudflare network are API related.

And that trend, we don't expect to be going down.

We actually expect API traffic to be increasing, especially as we're seeing more and more often.

API first native web applications, our own dashboard being a perfect example of that, where there's a light front end that's just using APIs to populate all the data in the dashboard.

<v Dan Gould> Yeah, indeed, indeed.

And you know, in terms of API attacks, you know, attackers are going after them.

What do we see? And I know we only have a few seconds left. <v Michael Tremante> Yeah, HTTP anomalies this year is the top on APIs, but immediately after that is SQL injection, right?

APIs are basically an interface to a back end database, and any attack vector trying to exfiltrate data should be top priority on API endpoints.

<v Dan Gould> So it's probably, you know, we've seen this in Web attacks, probably no surprise there trying to get into that same back end database, right?

So similar data tactics.

<v Speaker4> Exactly.

<v Dan Gould> Yeah, indeed.

Well, you know, with that, I thiwe're really wrapping things up here. Michael, I'm going to say thanks to you.

Really, you know, good insights. I know we'll do more of this.

So, you know. <v Michael Tremante> Thanks very much everyone.

<v Dan Gould> Tune in.

And, Tubes, thank you also for great work on performance. <v David Tuber> Thank you.