Unveiling new Internet trending and routing tools, disruptions, and Kevin Mitnick

Hello, everyone, and welcome to This Week in Net. It's the July 28th, 2023 edition, and we're back after a couple of weeks of vacation from New York City. This will be an episode full of Internet insights, goodies, with Cloudflare radar announcements. I'm João Tomé, based in Lisbon, and with me I have, as usual, our CTO, John Graham-Cumming. Hello, John, how are you? I'm very well, thank you. How are you, João? I'm good, too, after a couple of weeks of vacation. So we're back, and a few things on the news, on the Cloudflare news, but also on the Internet general news. For example, this week, Twitter announced that it will be X from now on, so a big change there in terms of branding of an old Internet social media brand, right? Yes, although I deleted my Twitter account quite a while ago now, and so it's effectively only that I click on links and it says X. The only thing that's really weird for me is it's using that... So I did an undergrad degree in mathematics, and that style of character with the double, sort of the stripe in it like that, we use those for sets, and sometimes for a thing called metric spaces, and so whenever I look at it, I think, oh, that's the unknown metric space X, but apparently it's not just that, it's Twitter. Exactly, there's a change there now in terms of what it meant before, what it means today from now on. So it will be, apparently, more than Twitter was all in app in terms of services, that will be for the future. Yeah, I think Elon Musk has said he would like to create something like what WeChat is in China, and I mean, in China, WeChat is a bit everything. It's social media, it's communicating with your friends and family, it's sending money, there's all that. Yeah, it's like a full ecosystem, financial, social. Yeah, payments. Let's see how it plays out. The promise is very big, because WeChat is a major player in China. If you go to China, I mean, that's how people want to communicate. I think, to a certain extent, we see that outside of China with WhatsApp, which is extremely common, but people want to use WeChat in China to the point where if you don't have it, I mean, on when I've been to China, I've had to install WeChat, because almost like culturally, it would be bad if I didn't have it and wasn't able to accept someone's contact information over WeChat. So I guess that's what Elon Musk is hoping X will be. Let's see how it plays out. Also, this past week, Kevin Mitnick, an actor, but also became a very important figure in terms of cloud security, died at the age of 59. He was very well known in the 90s, right? Yes, certainly. And also, I would say he was very well known as someone who was very, very good at social engineering, i.e. tricking people into what you want them to do by, you know, talking your way into a business or talking to someone over the phone and get them to do something. I think he was kind of legendary at doing that. And, you know, and obviously, part of that is technical, or part of those hacks technical and part of them are, are very social. Yeah, so that was, was a pretty big surprise, because he's not much older than me. And so, you know, of course, it's always a bit of a shock when people around your age die. And it was very sad to see that, that news, I saw it on Hacker News, and Hacker News said, you know, put the black bar whenever someone notable dies, if you go to Hacker News, they'll see there's a black bar at the top of the page. And it was a shock that it was Kevin Mitnick. So, you know, I guess we lost one of the, one of the legendary social engineers, hackers out there. In the sense, he was one of the, I think the first actor that was arrested because of what he did in the 90s. But then he became all of these figures, there was a big movement called Free Kevin at that time, because with the posters and all sorts of things, get people, I think, and correct me if I'm wrong, people didn't think the sentence, which was a lot of years in prison, didn't match what he did. In a sense, it was a big movement at that time. Yeah, there's a, there's an interesting thing about Mitnick, which is that when he was really young, he figured out how to travel the buses in, I think it's LA, by, he bought a ticket punch, and he figured out that you could punch transfer slips. So he found transfer slips in a dumpster, and dumpster diving was something that hackers did a lot of to try and find stuff, right, look in the trash. And he punched his own transfers, essentially. So it looked like he was transferring from one bus to another, which enabled him to travel around for free. I mean, obviously, that's fraudulent, because you're not paying for the service that you have. But I think it's a good indication of kind of the hacker kind of mindset of like, okay, how do I, how can I misuse this system, in this case, the bus service to do something I wanted to do? And how can I do it? What are the tools I need for it? In his case, it was, you know, this, this punched a hole in a ticket. Some of this stuff was really freaking, right? It was phone hacking, it wasn't Internet, right? So that kind of stuff, and, you know, getting long distance calls for frees and all this, this kind of stuff, and it, you know, Internet became, became part of that. So it was, this stuff predates, predates the Internet. And, you know, obviously, now, we have white hats, and white hats, and black hats, and gray hats, and, you know, all sorts of things. So let's dig in into some of the Raider announcements this week. There's, there was a bunch of them. Where should we start? Should we start with products that we announced inside Cloudflare Raider 2, or some trends? Well, first of all, if you don't know what Cloudflare Raider is, you can go to radar.Cloudflare.com. And Cloudflare makes available a huge amount of data about our observations of the Internet. So we see all sorts of trends, traffic trends, adoption, security, quality of your Internet connection, which domain names are trending, where there are outages, what's happening with, with routing on the Internet, and stuff like that. And so I think what's interesting about that is that, you know, it's often hard to get visibility of this stuff. And we have such a large view of the Internet, this is pretty unique. And it's not just a website, it's an API as well. So if you need to integrate this into something that you're, you're doing, I mean, it's, it's right there. I mean, it's really an interesting tool. And yes, we have been innovating a lot on Cloudflare Raider. So perhaps we could talk about, about some of those things. Sure. We have two things, main things in terms of the page. First, the routing information is now on Cloudflare Raider, but also trending domains. So we have like a set of trending domains and two types of lists that we just launched. Should we start with the trending domains? Yeah, let's start with trending domains. Everyone wants to know what's hot, right? If you think about Twitter, oh, I mean X, there's that trending thing, right? Which is like, what are the things that people are finding exciting? And in Raider, because we have a lot of data from our resolver. So 1.1.1.1 is used by a very large number of people around the world as their DNS resolver. And you can kind of see from that data, not looking at anybody's privacy, but not looking at what they're doing, but in aggregate, what's popular lately. So this is a great one. Let's look at Spain. So obviously you picked up Spain and I guess today Clash Royale is popular in Spain. We saw the other day in Spain that the newspapers were very popular because there was the election. The election was very heavily contested in Spain between the socialists and the right-wing parties. And you saw on here that kind of stuff growing. And you also see things that are trending over this week. So things that are sort of slowly growing over time. So these are two different things. Like, so today, obviously everyone in Spain is playing Clash Royale or at least they discovered Clash Royale because it's suddenly trending. And maybe Eventbrite, they're looking at events. Let's take a look at what's going on in Portugal, shall we? Sure. Got to check up on our local. Actually, by the way, worldwide, two things. Formula One had on Sunday, and this is trending this week, and this is trending today. But on Sunday was the Formula One Grand Prix. So it's here. And also on Monday in Israel, there was a big protest related to the approval of the judicial, part of the judicial reform. So there's this news website from Israel also on the worldwide perspective this week, which is interesting. And you were saying Portugal. Yeah, let's see if we can get Portugal to pop up. I just did it myself. RTP. RTP, right? So obviously people are looking at stuff on RTP, streaming it. And it's interesting to think about some of these changes that happen over time, because you've been noticing a lot of vacation related stuff. And I wonder if there's an increase in streaming because people are on vacation in Portugal, watching TV on their phone. And if you look at the trending this week, it's interesting that you've got Formula One, but also lower down sport TV as well, right? So lots of people watching sports. True. And there's the World Cup, the Women's World Cup going on in New Zealand and Australia this week. And Portugal is for the first time there. So maybe this also plays a role there. They're looking at it as well, right? And also the teams are now getting ready for the championship. So that also plays a role. So interesting to see these kinds of trends happening. Let's do one more. Let's look at where I come from originally, the UK, and see what's happening over there. So here it is, the UK. Okay, Trivago. So that's a travel website. So everyone in England is thinking about escaping the UK, obviously. And then some football. Look at Arsenal right there. Arsenal. Interesting. They played this week. There you go. See, I'm not an expert on English football, so I don't really know what's happening. And obviously this week, Roblox. And what's interesting about that is that the schools just broke up in England. So the kids finished school last week. So you can see that Roblox has been growing and Minecraft too. So I'm guessing that lots of kids have come home from school and are playing video games that they couldn't do before. So anyway, you can look at that. It's the main rankings. That's something we released. We'll keep working on this. It's a very complicated thing to get right because you're looking for signals and lots of data and some things jump up and down, but hopefully we'll give you a good idea of what's trending in different locations. True. It's challenging, but in a sense, I think it's interesting. There's a few examples here, like NBA on top of a few countries. Yeah. When the draft was happening. The draft, exactly. And then Ukraine. Yeah. Ukraine, people looking when that was the Wagner group, sort of mutiny happened. I think there's something about Croatia. Everyone was looking at the weather and this is because Croatia should be lovely weather at this time of year. And in fact, they had some very bad weather. So everyone was taking a look at what was happening in terms of the weather over there. In terms of products, we also released, actually yesterday, our routing information section on Radar. And this is detailed. Exactly. It's also a big challenge, right? Okay. So here's the state of this, right? The Internet is a network of networks. Everybody runs their own network. Cloudflare runs one, Portugal Telecom runs one, Microsoft runs one. And how do they talk to each other? In order to connect together, they're both physically connected together, but also in order for them to organize themselves into a network where people can find stuff. They use a thing called Border Gateway Protocol or BGP, everybody calls it. And the way in which BGP works is everybody who's connected to the Internet goes around announcing to the Internet, hey, I'm responsible for this bunch of IP addresses. So if you want to find these IP addresses, they're over here. And this loose collection of messages coming out gets bound together and calculated into something called the routing table. So anybody on the Internet and your ISP is doing this, when you try to connect to a website, the IP address is known, it then goes to this routing table and says, okay, I need to send it this way and it'll get there. So this is the thing that figures out how all the Internet is actually made and connected together. And so we collect statistics on that because we need to know where the Internet is. We're a big part of it. And we're now putting out a ton of information. So on a per network basis, you can look at what IP addresses, what's the routing, what's the connections between the networks. You can look at anomalies. So that's route leaks and origin hijacks. So route leaks where mistakes are made. So if you imagine that the routes are multiple hops, if I want to go from here to Porto, I go from Lisbon and I might go to say, I go to Coimbra and I go from Coimbra to Aveiro and then I go from Aveiro to Porto. I'm not sure that's the route I should take, but let's imagine that some of that route is sort of local, right? I probably don't need to know all the details. If I'm in Faro, trying to get all the way to North, I just need to go, oh, go to Lisbon and then carry on. If any of that information were to leak to the rest of the network, you could get very confusing. Wait, I should drive from Faro to Coimbra? That doesn't make any sense, right? So that can happen with a origin hijack, which is where somebody deliberately or by mistake says, hey, I'm Google. Pakistan famously did this in the past where they wanted to censor YouTube. And they said, I'm YouTube. And all the traffic from- And sometimes it's by mistake, right? Sometimes malicious players also plays a role there. Yes. Yes. Both of that situations. It's kind of amazing to see how something like that, which is really relevant, like you won't get in a specific region, YouTube because of someone messed up by mistake. Yeah. Or deliberately, right? Depends on the situation. Shows us the vulnerability also of the Internet, fragility in a sense, if things are not done correctly. So this is a very good tool for those who want to explore this They can, let me just put up the page, the specific page here. They can just simply go to this page and check specific countries. For example, Portugal, if there's something going on there. So you can work in an ISP or in a specific ASN and could be helpful for you to see what is going on, right? Yes. And we're going to add a facility where we can send alert if necessary. So people can get alerted if their network has been leaked or there's been some problem with it. Exactly. If you want to see a specific ASN, for example, let's pick Vodafone one. You can register alerts for the specific ASN, for example. I want to see things related to this ASN. That will be really helpful for those who want to monitor these things. And also you can do all this through the API as well. And it integrates with other, if you're a networking nerd, it can integrate with Monocore. I mean, there's a bunch of stuff you can do here to find out about networks. And today we're putting out there a more in-depth blog post about all of this new BGP origin hijack detection system, right? It has a bunch of information here for those who want to explore. Let's move on. We still have a few things to go over. Okay. We dealt with announcements. There's also Internet disruption summary about Q2 and also a DDoS report. Where should we go? You choose. Let's start with the DDoS that is from last week or two weeks ago. Here it is. The DDoS threat report. There's a few new things in terms of DDoS attacks in Q2, right? Yes. I mean, the thing that's striking, of course, is that DDoS just never seems to go away. I mean, it's just like constantly a problem. And you have the really big ones that are headline making. And then you have just this constant background noise of small attacks that don't make the media, but actually knock things offline. And you just see across the globe. So lots of stuff related to the war in Ukraine. Cryptocurrency companies getting attacked. That seems to happen a lot. Increased by 600%. That's a lot. Yeah. Yeah. I mean, that's, I guess, how it goes sometimes. And there's also this exploitation of a specific vulnerability, Metail, right? More than 500 surge in DDoS attacks exploiting a specific vulnerability. I mean, you see these sort of vulnerabilities come along, right? Where it's sort of, the vulnerabilities are exploited and then people sort of forget about them or they don't use them so much. And then they become popular again. In this case, this Metail vulnerability, which was from 2022, is getting exploited. But I think that attacks are continuing and DDoS is a real thing. So here's a report gives you a sense of it. But I guess the bad news is it's not going away. It's not going away. It's getting more sophisticated according to this Q2 report. There's a bunch of examples here, some charts, some quarter over quarter perspectives also. And why not just show this is also featured on radar. Exactly. In a specific report that you can browse through. So this is the DDoS report. You can, if you want to learn more, you can browse it through. And also this week, we just showed the summary of Internet disruptions in Q2, the DDoS report. A lot of disruptions also in Q2 from different types of situations in this case. Yes. Yes. I mean, what stunned me, I was reviewing that. I thought it was never going to end, that report. It's just like, you look at the number of different sorts of disruptions. It's huge. It's just huge. And, you know, I went through it. And if you look at it, like the different varieties of stuff. So you have like government directed, so you have the exam season where some countries cut off the Internet during exams. You have weather causing problems. And a lot of examples on the exams thing that we actually mentioned. Yeah. It's that time of year. Yeah. There's a lot. In terms of highlights for specific types of disruptions, you were mentioning the government directed like the exams, but also severe weather, cable damage, power outages, technical problems. Yes. Power outages are still around, right? Yes. Power outages. I mean, those things happen. There's a few on here where actually there are like the Internet went out and we don't know why. And it was probably a cable card or some technical problem, but the country didn't mention it. Problems with maintenance. I mean, it's pretty stunning if you go through and just think about how often the Internet, and these are large scale outages, right? These are like countrywide outages or maybe a region of a country going offline. But yeah, if you want to know about why something collapsed somewhere, well, this is the place to go for Q3. And obviously we'll do this to Q2, sorry. And we'll do it every quarter. And now we've got happening right now in Niger, right? And I think we haven't seen an Internet outage, right? You were keeping an eye on that. True. We haven't. Actually, we can just go here and see live data in terms of what's happening in Niger. Let's see if something happened. Last 24 hours. Okay. It's a little bit less traffic, but not a specific disruption. It doesn't look like the folks who've taken over there have cut the Internet off. So that's interesting because often you see the Internet getting cut off in these situations. Exactly. The only thing perspective we shared on Twitter or now X actually was the increase, 50% increase in social media DNS traffic. So people are doing more social media messaging, even search to see what's going on. We see this a lot, right? So we also have a deep dive from two weeks ago that we didn't mention the day my ping took counter measures from Merrick. What is this deep dive all about? Well, there's a weird thing. So ping, right? Ping is the program that you can do the most basic connectivity test, I guess, in a way at the IP level, which is you send a message to a computer called a ping and it sends it back to you. And the thing that's interesting about it is that it's very simple, right? And it gives you some timing information. And if something goes wrong with the timing, then it will tell you there's a problem with the time and it will then come back and give you this weird message, which is the time is wrong. So rather than give you a negative time, it'll tell you there's this problem and it says making, taking, what did it say? Taking, I mean, Merrick- Warning time of the day goes back, taking countermeasures. Right. So I think Merrick was like, what the hell? What countermeasures is ping taking? It's such a simple thing, right? It's like sends a message, you get a message back, gives you the timing, tells you if you got lost and it's like, what's happening? And so he decided to dig in. This is really a debugging story of like, well, first of all, it's obvious from the source code what it's doing actually is it isn't really taking countermeasures. It's just giving you this warning, but then figuring out under what circumstances it happens and why time could possibly go backwards. Well, you'd have to read Merrick's blog post to find out. It's a little bit back to the future type of thing, but also replies to, tries to answer this question. Are the bad measurements excluded from the final statistics here? And how do they test the software? Yeah, how they test it. And yeah, definitely. Something to explore if you are into those deep technical, deep dives. Where should we go next? We still have a couple of minutes. We announced a few things this past two weeks, including there was this Zenbleed vulnerability. Yeah. Yeah. I mean, this is a really, really interesting vulnerability in certain AMD processors where there are these instructions that work on a very, very long chunks of data. And it turns out it's possible yet another branch prediction kind of thing that I specter and melt down where it's possible to end up with something being placed in one of these big registers and then only half of it being zeroed out. And therefore you can get information from some other process on the system. And unfortunately, or fortunately, if you're evil, those instructions are sometimes used for string handling. And so string handling, it could be like a password or other data. And so it's possible to write some code in an assembly language that you might be able to steal information that another process is doing. And so it could be something very sensitive like a password or maybe even some secret string or something. It's fixed by a microcode update, which we've been applying across our entire fleet. So we're not terribly worried about it, but we wanted to reassure people that we were running the microcode update because it requires assembly language access. We're not too worried about it because it would require someone to write something malicious onto our network, but nevertheless, we're rolling it out. We're keeping an eye on if anybody tries to do anything funny through Cloudflare Workers or something like that, as I hope it will be soon be sorted out. Of course. And it makes sense to be aware of that. Even because it's sensitive data that can be affected in this case, right? Yeah. So even more so relevant. An announcement, Cloudflare 0S steps up general availability and pricing with transition out of beta. So there's that also. And we also had connection errors in Asia Pacific. Here it is in July 9. What was this all about? Well, we had an outage caused by us. So VeriSign runs the .com and .net top level domain name service. So if you try to go to something .com or .net, you ultimately go up to VeriSign and then get down to whoever runs the actual service. If you go to Cloudflare.com, you need to at some point know where .com is and then VeriSign will tell you, okay, well, Cloudflare.com is over here. And then you go to Cloudflare server and you say, okay, let me explain all this in the blog. So it's pretty vital those things work. And generally they work great. And then we also use a thing called DNSSEC and DNSSEC is a way of cryptographically saying, hey, you know, this reply you got from DNS telling you that Cloudflare.com is this IP address. Well, you can prove that it is using cryptography, but it hasn't been tampered with. Just like you might prove that in an email you've been sent has been signed with a digital signature, same kind of idea for DNS. Unfortunately, VeriSign on Saturday had a problem where they were they were sending out the wrong cryptographic information. So the signatures didn't validate. And so anyone who was validating DNSSEC then could not say, well, here's the response. So you've got no response or you've got an error response. You went to a .com or a .net and that's pretty important. Now for Cloudflare, the big use is that if we do a thing called CNAME flattening, or as a CNAME, so you sometimes get a situation where something like blog.Cloudflare.com doesn't necessarily go straight to an IP address. It actually goes to another name. So it might go to like, you know, blog -server.Cloudflare.com. And then you do another lookup where you say, okay, what is blog-server and you look that up. That was where it was failing because we were then saying, well, okay, we don't know what, we can't do this because we can't validate the response we got. So this was, you know, affected us. What we did was once we saw this, it was only happening in Asia Pacific. We pointed out DNS away from Asia Pacific. So DNS was still working, but we, when we needed to look something up, we went over actually to the US West coast, which if you think about it is just across the Pacific, to do the DNS lookup because VeriSign was still working in those locations. So that fixed it. But it took VeriSign a few days before they announced what they'd done. They'd made a mistake while doing maintenance. There was a server that was still serving up these old keys. And that was, that would cause the DNS sec problem. Exactly. We should also mention that we are now on threads .net. So not only on axe.com, but also on threads.net, the meta rival of Twitter. So it's also here, not for Europe. So we won't be able to see this for the time being, but it will be there for a few months. And we, I believe we're on Mastodon, of course. Mastodon and BlueSky too. All of them, all the social networks. All of them. Exactly. So essentially that's the time we had. Just final note, last week was the Oppenheimer Barbie premiere. Did you watch any of those? I have not watched either of them. I think of the two, I would, I think I'd like to see Oppenheimer in the movies, in an IMAX. And I was thinking... I'd do that. You went up to Colombo in Lisbon and saw it. I'm thinking about going and doing that, the IMAX. For the, for Barbie, I'd like to see Barbie. I'm curious, but I think I'll wait just to watch it on TV when it's streaming. Both are interesting, very different. I think Oppenheimer for our technology perspective is quite amazing to see, especially now that we're talking about how AI can disrupt the world. It's quite amazing to see that something that was brought up in a sense by science, by technology, had a major impact in the world. There's a lot to unpack there in a sense that I think is pretty much current. So I think it's a good suggestion. And IMAX definitely makes a difference. It's quite amazing, not 3D, but that immersive experience, because it was shot for IMAX, really brings something more magic to what we're seeing on the screen. So I'm kind of glad it's not 3D. I think 3D is kind of like, it just doesn't work that well. It's too much. I agree. And to be honest, you have, for 3D, you have to be on the middle of the room. If you go to the sides, the experience is very bad. And in IMAX, without 3D, even a little bit on the side, you will get an amazing experience. So 3D has some issues, I would agree there. Well, that's it for this week. All right. Cheers. Good to see you again. Bye-bye. Bye-bye.