Threat Watch

and John Graham-Cumming, and I'm John Graham -Cumming, and I'm John Graham-Cumming. Welcome to Threat Watch on Monday, June the 15th. This is a 30-minute show about threats on the Internet, what Cloudflare is seeing, other things that Cloudflare may not be seeing around the world. Hopefully we're seeing everything. And I'm going to talk about a few different things today. I'm going to talk about Internet shutdowns. I'm going to talk about some of the trends that have been happening during the pandemic, some of the trends that have been happening during the recent unrest around Black Lives Matter, what's been happening around WAF-blocked requests around the world, what sort of ways are people trying to break into websites and APIs and applications, and then look at what's happened with DDoS around the world. So hopefully we'll get through that all in 30 minutes. First step is to jump to Burundi in Central Africa. Now, Burundi recently had an election on May the 20th, and as has become quite common in quite a lot of countries, parts of the Internet were shut off during the election. In Burundi, this lasted for about a day. It was actually on the election day, on the 20th of May. But we've seen other shutdowns have been longer around election periods. And this has become quite a habit around the world, and quite a surprising habit, especially when you think about the amount of Internet access people need during the pandemic. So in Burundi, on May the 20th, there was an order to shut down essentially social media. So for about 24 hours, Facebook was inaccessible, Instagram was inaccessible, Twitter was inaccessible, no one could watch YouTube videos, Messenger was unavailable as part of the Facebook shutdown, and WhatsApp as well. And all these measures are usually announced as a way to stop rumors from spreading and other sorts of unrest around elections. And in this case, about a 24 -hour shutdown. You can see this netblocks .org, which also has a Twitter at netblocks, has tracking of Internet shutdowns. And of course, Cloudflare sees these as well. This will give you a sense for what it tends to look like. If you look at this as the end of the shutdown on the 21st, and you can see that various services, Facebook, Messenger, WhatsApp, etc., had been unavailable, 0% availability in Burundi. And then they started to come back. And you can actually see the pattern of these things coming back at around 4.30 UTC. What's interesting is not everything comes back at the same time on each operator. And this is pretty typical in shutdowns that are ordered by governments. What you'll typically see is a government order will go out to Internet providers, and you can see here a number of the different providers, Onatel, Beatel, Lacelle, within Burundi, and they will then get the order to switch off or switch back on, and they'll not take the technical means, they'll do the work to actually re-allow access to those applications or the Internet as a whole. And so you'll see different service providers come on at different times. Often mobile comes on at a different time to broadband. You can see a slow ramp up here over about 30 minutes when everything came back online. And you see things get cut off in a similar way. This looks quite different to what happens when the Internet gets cut off because of a cable cut somewhere, which is a sudden drop of absolutely everything for absolutely all operators, or most of them, depending on the architecture of the Internet. There have been some incidences that looked a bit like a cable cut, but were actually government ordered. And there was one particularly notable one in Syria a few years ago, where the Internet went off completely, cut by different service providers in different locations. But it happened at the Internet connections into the country, and they were cut off quite suddenly. So it sort of looked like it wasn't government ordered. But actually, if you looked at the timing, it couldn't have been a cable cut because it would have been a cable cut in multiple different locations of the country at the same time. Now, also staying in Africa, in Guinea, there was an election as well. And similarly, there was this decision to shut down access to social media. This one lasted a bit longer, getting close to three days, and that was earlier this year. In March, March 21st, the same group of Internet products were shut down. So Facebook, Instagram, social media, talking to people on WhatsApp, YouTube, obviously, YouTube is used widely for getting out information, also shut down by a government order. And Netblocks is another way of looking at this, if you want to, they can actually take you deep down and see what got shut down. So what you'll see sometimes is an order to shut down, not the whole service, but let's just say image sharing. And that will might show up here as access to a particular CDN has been blocked, or access to the image hosting for a particular service has been blocked. But here in Guinea, pretty much everything was blocked, all sorts of things were shut down. If you look back at time a little bit before, if you go back a couple of years, and I'll give this gives you a sort of an insight into how these things work. There was a shutdown in the Democratic Republic of Congo, during election time period of unrest. And this was ordered by the government. Now they shut down Facebook, Twitter, WhatsApp, YouTube, and Viber. And in particular, they shut down images and videos being shared on those services. So this was a partial shutdown to restrict things. And this text is actually part of the letter that was sent to the Internet providers in the DRC, asking or rather saying that the Internet was going to be shut down. It's written in rather flowery French, asking very politely that access should be restricted during this period to images, image and video sharing on those services. And this is how it works. Government asked for it to be shut down. What's interesting in this letter is it doesn't provide any justification, it just says shut the things down until further notice. Now one thing that's really common is around elections. And obviously, there's been around unrest, which you've seen in Sudan. But another really popular thing, and that's particularly relevant around this time of year is during examinations. And you see this all over the world. You see it in North Africa, you see it across the Sub-Saharan Africa, you see it in the Middle East, you see it in the Far East. When it's examination times, particularly in countries where there are national exams of which pupils futures really reside, the Internet gets shut down. It might get shut down literally just for the time of the examination. So if there's a one hour period where everyone does the exam, there might be no Internet, or it might be for an entire examination period that happened in Ethiopia a couple of years ago, where the Internet was just shut off. This is done to prevent cheating. And that's actually a pretty common thing. Surprising as it might seem that examinations are a reason why the Internet gets shut off. Now, let's talk about some of the stuff that Cloudflare defends against and has been defending against during the pandemic, and over recent weeks. So obviously, Cloudflare is fairly well known for DDoS attack mitigation, but also for firewalling against all sorts of attacks against Internet properties, be they trying to break into them to deface them or steal information. If you look at the 2020 attack trend, there's a couple of interesting things going on here. So the y axis on this is the number of HTTP requests blocked per day by Cloudflare because they were malicious. So that could be they were part of a layer 7 DDoS trying to knock a property offline, or they were an actual attack trying to break into something, a SQL injection, file inclusion, cross site scripting, something along those lines. And if you look at the beginning of the year, we were running at about 40 billion requests a day. That's right, 40 billion requests a day were being blocked. And that kept around that level until the beginning of March. And this is really the point at which the pandemic began really to bite, and in terms of lockdowns around the world, and it began to peak up. And in fact, what we've seen is during the pandemic, there's been an increase in attacks on web properties, Internet properties in general, either DDoSing them or trying to break into them. And in fact, it peaked very recently in May at a colossal number of attacks per day, something close to 160 billion blocked in one day. And it's now settled down a little bit. We think that part of the reason for this is that every year we do see an uptick in cyber attacks during vacation times in universities. And that's because technically abled and now bored students get out their hacking tools and decide to have a go at breaking into something. This is not sophisticated, it's just volume of attacks, just the number of people who are doing it. And this really shows up in one important way, which is that we see a lot of users pretending to be an ancient version of Internet Explorer. We'll see a lot of attacks claiming to be Internet Explorer 6, which is so old, no one is legitimately using it anymore. But it turns out that a lot of attack tools hard-coded Internet Explorer version 6 as the user agent, as the string identifying the browser that they are pretending to be. And so this is a great signal that these attackers are not sophisticated. They're people downloading tools and just trying it out a little bit like a sort of virtual vandalism, a little bit of hooliganism online, but it's a definite increase with the pandemic. But the other thing that's been going on is some really sad kind of attacks. So we've seen quite a large increase in attacks on hospital websites. Cloudflare protects a large number of pharmaceutical companies, hospitals, and other parts of the medical ecosystem. And attacks on hospitals have gone up a lot. So this graph shows volume from the start of the year, and the y-axis here gives you the increase. So a peak, we're getting close to double the number of attacks we would normally see on hospital websites, which is completely insane in my opinion. Who attacks a hospital during a pandemic? Who tries to deface a hospital website? Well, somebody does, and somebody's been fairly persistent even right up until the end of May. The other thing that's happened is with the murder of George Floyd on May the 25th, there's been an uptick in attacks on websites which are against racism. Now Cloudflare has a project called Project Galileo, which protects vulnerable websites online. We have a very large number of websites that are given our high levels of service because they are worthy and for public benefit. Now some of those websites were fighting against racism, and there were organizations that were trying to stop racism around the world, and a large number in the US. And if you look at those websites at the beginning of the year, they essentially got no attacks at all. Maybe a small number, 100,000 in a day, something like that. But since the murder of George Floyd, there's been an incredible increase in attacks, up to something like over a billion in one day, which is completely incredible. And these are mostly DDoS attacks trying to knock those organizations offline, perhaps to stop people getting information about protests that are happening, about how they can deal with it. But it's really a sad reflection. One of the things that happens on the Internet, which is where there is something happening in the real world, there is often a mirror in the cyber world. All right, let's change gear a bit. Let's talk about the sorts of things that happen when people try to break in to websites. So Cloudflare provides DDoS mitigation, for which we're very well known for, but also a web application firewall. And the firewall is looking for the sorts of attacks that are trying to worm their way into websites, APIs, applications that are online. And these are the sorts of attacks that might be trying to steal information or log in maliciously as you do something, or trick other users into giving information away, or deface the website, perhaps for vandalism purposes, or perhaps for political purposes. So they contrast to the DDoS, which tends to be just high volume of data thrown at the website, trying to make it inaccessible to people. Here, this tends to be more stealthy, and is looking for a way to break in. And there are various ways to look at the types of attacks that we see through the WAF. The most interesting of these is a thing called the OWASP Top 10. So the OWASP is an organization that looks at the sorts of attacks that happen on web applications. And if you have not looked at the OWASP Top 10, it's really worth taking a look to understand the types of ways in which applications get attacked. And they have 10 in order of attack types that they list, which they think are the most important ones. And so we're going to go through them, and I'm going to show you some real data of what it looks like from Cloudflare's side. So the most common way in which websites get attacked is through a thing called injection, which is they try to trick the website into doing something on the attacker's behalf that shouldn't be done. For example, you go into a website, and the website shows your name in the top right-hand corner. For me, it says, Hello, John. That John was probably looked up in a database somewhere, and that required a database query. If you can trick the web application into doing something in addition to looking up my name, you might be able to get extra information out. So you might be able to say, by a clever crafting of a request to the website, not only get his name, but get his password, or give me his address, or give me everybody's address. And often, when you see attacks on websites, and you hear about them in the news where a huge amount of data has been dumped, this is how they got in. They got in through the front door, through a website, through an API, by crafting a message, an HTTP request, that contained some code that was injected and run on that user's behalf when it shouldn't have been. The second one is broken authentication. This is, you know, once you've logged into a website, obviously, what you want to do is remain being you, and you shouldn't see anybody else's data, and there should be no way for you to see somebody else's data once you've logged in. Once people are logged in, typically, there'll be a cookie, which is stored in the browser, which identifies who they are for that website, and allows them to continue being logged in. And every time you use the website, it'll say, hello, John, and I can look up my orders, or my account information. There are all sorts of ways in which authentication goes wrong, and session management fails. And that's the number two way in which websites get broken into, that there's a way, perhaps, of predicting the cookie, of stealing the cookie, of not using your cookie at all. Number three is sensitive data exposure. This is how data gets pulled down from websites. So, if an API or an application is not built well, it may be possible just to change the URL a little bit, and get someone else's information. That's a big threat, in general. XML is a ripe world for attacks, and in particular, think of external entities, which allows some XML to be manipulated, and then some additional XML inserted. And often, that can end up being executed as code, which means you're running code on someone's server when you shouldn't. Broken access control, very similar to broken authentication, but getting access to the wrong things you weren't meant to. Security misconfiguration is an interesting one. This is very common, which is that there's just a mistake in how the website was set up, how a framework was used, how some part of the application was used. And if it was misconfigured, maybe somebody can use it. We heard a lot many years ago about cross-site scripting, or XSS, as it's called, which is where you inject some HTML into an application, which ends up in someone's web browser and does something. Maybe it steals their password, maybe it shows an image that you wanted to see if it was a defacement, but that's what cross-site scripting is all about. And then, three interesting ones. So, insecurity serialization, which is what happens when you pull data from, say, a database or from a request that's being sent in. One of the things that programmers do is they serialize data, they store it in some format that a computer can understand, they send it to the application through your web browser, and it has to be deserialized, which means it has to sort of be expanded, a bit like a zip file is opened up to look inside to actually be used. And if it's insecure, then often you can make some sort of mistake in there, which can lead to code being executed on the machine, or an injection and data being stolen. Number nine is a really interesting one. You know, as we all do, that software needs to be updated. And, you know, common vulnerabilities are talked about all the time. There are new vulnerabilities coming out all the time. And attackers know this. And when a new vulnerability comes out, within minutes, they will start using it. And so, using known vulnerabilities to attack web applications is really common. And actually, it's very difficult to patch fast enough, which is why web application firewalls are important. And lastly, not monitoring your application to see what's going on. If you're not logging, you don't know what's happening. So what really happens on the Cloudflare network? Well, it's interesting. This is a graph of the breakdown of attack types blocked by the WAF by volume this year. And you can see that injection, the command injection, which is trying to run a command, SQL injection, which is trying to do something with a database, are really popular. That's the number one. You can also see in there that there's a couple of other things that are very popular. Cross-site scripting continues to be a popular way to attack applications. Information disclosure attacks, trying to pull down information directly, also very popular. But there's two here that aren't on the OWASP top 10, and it's worth talking about them. The first one is protocol anomalies. This is where people try to fiddle with HTTP, the protocol itself, to try to make the web server do something. They might change the headers a little bit. They might add a forwarded for header to try and trick the server into doing something. These are actually incredibly common. And you can see, actually, the second largest category of thing that we block on the edge. And you'll see this a lot. Sometimes it's from misconfigured software, but mostly it's from attack tools trying to trick an application into doing something because they're misusing HTTP. And HTTP itself is fairly flexible. But the one that's really interesting that's not included in the OWASP top 10 is file inclusion. And file inclusion can be remote or local. But what it involves is trying to get data from a server or upload data to a server through the manipulation of files and actually the paths for getting them. And this can result in information being disclosed. It can result in code being uploaded to a server, which then gets executed, which then gives you access to the server, which lets you take it over. And those things are very, very popular and actually not on the OWASP top 10, and definitely something people with WAFs need to think about protecting against. All right, let's talk about the bot threat. So one of the interesting things on the Internet is that a huge percentage of Internet traffic is actually performed by bots. Much larger, I think, than most people would realize. We think of ourselves as using the Internet directly. We're using web browsers. Maybe we're using an application on our phone, which uses the Internet as its backend. We're hailing a car in the street. We're swiping right or left on a dating application. All those things are human interactions on the Internet. And then there are some which are machine interactions, which we want to have happen. So maybe you're wearing a fitness tracker and the fitness tracker uploads information about the number of steps you've taken today or how your heart is doing to some web application. All that stuff is legitimate human behavior. But there are bots on the Internet. Some of those bots are great. So all of the major search engines have crawlers, which are going to walk around the web looking for changed web pages so they can then put them in their index, or we can find them through the search engine. But there are also a lot of bad bots. And what those bots are doing is a number of things. One is they are taking people's content and reproducing it elsewhere. So there are bots that will scrape content, let's say stock prices or news stories or something's being written on a blog, and they will take it and repurpose it for their own use. And so they will steal that information, essentially copy it and put it somewhere else. Other sorts of bots are actually buying things online. And so in some markets where there are things that are of limited supply, it's very often the case that bots are in there trying to buy them before humans can. One very common area here is sneakers. When a new type of sneaker comes out, that will go on sale, it'll be very popular, and there'll be limited supply. There is actually a battle between bots to try and put those things into carts and buy them quickly. And part of the reason for that is the resale value is so high. If the bots can corner the market in those shoes and then resell them on eBay or elsewhere, there's money to be made. And that's true of anybody doing retail online. That's a really, really big problem. So bots are actually a pretty big menace, and Cloudflare defends against them with a bot management product. Now what's interesting is to look at how we do this. And because of the scale of Cloudflare's network, we have a view of what is real human behavior, be it actually directly done through a web browser or through a device like a fitness tracker, and what is bot behavior. And there's a couple of different things we do. But the big thing is machine learning. We can look at this total view of the Internet and say, this request is coming from a human, or it's coming from a bot, and maybe it's coming from a legitimate bot like a Google crawler, but we can block it. And so that's, if you look at this, we're well up into the 80% being done by machine learning at incredible scale. And we do this for requests across the Internet. But the really interesting thing is if you go in and you say, okay, well, how much traffic globally is by bots? And here it is. Globally across all of our websites, it's something like 37% of all traffic. So if you think about that, the humans are really only in there creating about 60% of the traffic on the Internet, about 37% is by bots. And a very small sliver is by the good bots, the crawlers, the things we want to actually use the Internet. So this is quite striking that these bots are really a menace. And if you go and look at, if you then narrow down and you look at anyone who's retailing someone or anyone who's got valid content, valuable content, then this pie changes a lot. And as you get closer to 50% of the traffic they're seeing is from bots and 50% from humans. And sadly, that means the bots are locking the people out. All right, let's look at DDoS. So DDoS trends over time have changed. If you were to think back a few years ago, you may have thought about there being news stories with what seemed like ever larger DDoS sizes. So 400 gigabits per second was a lot at one point, 800 gigabits per second, 1.3 terabits per second, 1.6 terabits per second. And these attacks got seemingly bigger and bigger and bigger. And there was some famous attacks that knocked off GitHub for a little bit, knocked off a DNS service provider. And suddenly the story stopped. And it wasn't that the press got bored of the stories. It was that the size of the attacks actually plateaued. And in fact, in many cases came down. And the reason for that is we, Cloudflare and others in the industry got really good at dealing with volumetric attacks. They haven't gone away. This is a bit like the email spam world. So many years ago, I worked in email spam, dealing with it. If you think about it today, if you use an online email like Gmail or Hotmail, you don't really think about spam. The spam gets filtered into something called spam and you don't worry about it. It's being sorted out. The spam is still there. Well, the same thing has happened with volumetric DDoS attacks. They are definitely still out there. But for the network level ones, the layer three, layer four things, the DDoS vendors have got really good at fighting them. So if you have something online, get one of those vendors and this level of DDoS will not be a problem. The other thing I think that's happened is that that's caused the DDoS to move up. They're now trying to attack the applications directly themselves. And they're looking at the layer seven. They're trying to find the weakest link at the top layer. But we can look at some sort of statistics for this year. So if you look at attack sizes this year, what's interesting is there's been a gradual increase in size. And there's been some large ones, over 400 gigabits per second, actually around a half a terabit per second. And that's enough to knock anybody offline if they don't have a DDoS mitigation service. In fact, for most websites, even the small attacks would knock them offline. Something, you know, 10 gigabits per second, many websites don't have that much bandwidth available and would be knocked offline. So we've seen a gradual increase January, February, March in the size of these attacks. But these are well below that 1.6 terabit per second kind of exciting values because attackers have moved elsewhere. The other thing that's happened is worth looking at is that attack durations are very short. So under an hour for most of them. And the reason this is significant is that if you're using a DDoS mitigation service that requires you to cut over to it, the attackers will come in, it's sort of like a drive-by, they'll attack the website or the service, knock it offline, cause disruption and disappear again. And then they can come back again and do it again. In fact, we quite often see that, these sort of periodic attacks. That's because they know that the website or the application is reacting to the DDoS and they can knock it offline and come back and knock it offline and come back. So people really need things that are on all the time. But nevertheless, there are some DDoSes that are super persistent, 24 hours, continuous, gigabits per second being thrown at a website. The other thing that's happened, if you look at the packet rate, so the packet rate is the other way to think about this, which is how many packets per second are being thrown at something. And the reason for that is that networking hardware can have a hard time dealing with a high packet rate. And this is all over the place, but 4% at over 10 million packets per second, that's a huge rate for any router to handle. And that is something that will knock off a website or an application that's not well protected. It's funny to look at this, but if we have a release notes for our WAF, and on there, there is a new WAF rule on the 8th of June release for the Hulk DDoS tool. Now, the funny thing about the Hulk DDoS tool is it's been around since 2017, written originally in Python, rewritten in Go fairly recently. And we've added a rule for it because it's become popular lately. And I think this goes back to that little story about all those people downloading tools, trying stuff out during the pandemic. The last thing I wanted to talk about was L7 DDoS attacks. And in L7 DDoS attacks, Cloudflare will protect websites, but if a website is very small, we might let through enough traffic to knock it offline. And that was definitely the case until earlier this year, when we put in a new detection mechanism, which allows us to take not just the volume, but the effect that is happening on a website. So this error-based detection looks at a website that's protected by Cloudflare, and detects whether the website is struggling under the load that's being applied to it, and then turns on our DDoS mitigation tools. And you can see this has had a huge effect. If you look at the bottom, which is GateBot, which is our classic DDoS mitigation tool, it was mitigating hundreds of DDoS attacks per day. But if we add in the error-based detection, we're now handling over a thousand, something around that level of DDoS attacks mitigated on smaller websites, which need more sensitive protection than the larger websites, which can handle larger amounts of traffic. All right, that's it for Threat Watch. Thank you very much for watching. I should be back in a week with hopefully what has happened over the last week, if there's been any new interesting attacks. If you have questions for me, please email livestudio at Cloudflare.tv, and I will either answer them directly, or put them in another edition of this show. Thanks very much for watching. Transcribed by https://otter.ai Transcribed by https://otter.ai Transcribed by https://otter.ai