Radar Bulletin: Q3 2024 Internet Disruptions
Presented by: David Belson, Dan York
Originally aired on November 14 @ 3:00 PM - 3:30 PM EST
Join David Belson (Cloudflare Radar) and returning guest Dan York (Internet Society) as they review notable Internet disruptions observed around the world during the third quarter of 2024, including their underlying causes and the impact that they have on affected communities of users.
English
News
Radar
Transcript (Beta)
Welcome, everybody. My name is David Belson. I'm the Head of Data Insight at Cloudflare and the Product Manager for Cloudflare Radar.
And welcome to our Q3 2024 Internet Disruption Summary Discussion.
I'm here with my colleague, Dan York, who is now the Senior Advisor at the Internet Society.
So welcome, Dan. And can you tell me a little bit about your new role?
Hi, David. And hello, Cloudflare TV watchers.
It's good to be back here doing this again, as we've done for the past several quarters.
It's fun to get together and talk about this. My role just changed within the Internet Society, where I'm doing a bit more helping with our new CEO, Sally Wentworth, is coming on board.
I'm helping a bit with her communications and doing a piece around that.
But I'm still, in this context, I'm still very involved with our Pulse platform at pulse .Internetsite.org and the measurements that happen there and the shutdowns, power outages, all those different things that are there.
So in this context, my role is still similar. I'm also doing a lot with low-earth orbit satellites and all those different kinds of things.
Cool. Cool. And we'll talk about that in a little bit.
Yeah, that's good. Yeah, I missed the folks on the Pulse platform.
I miss Sally. I definitely enjoyed my time with the Internet Society when I was there.
Well, it was great when you were here, and that's obviously you're doing some pretty amazing work at Cloudflare.
So it's cool to see what you're doing there and your ginormous post that you wrote.
Yeah, that's actually a good lead-in.
Yeah, Q3, for better or for worse, was really, really busy in terms of Internet disruptions.
And that doesn't even include all the ones that didn't make it into the post.
There were disruptions, I think, that we saw that we went up not officially recording or certainly not writing up.
But one of the things I think I've found over the last few years is that the third quarter, so the summer, at least in the US, is generally a time of a lot of government-directed Internet shutdowns, especially around exams.
There are a number of countries in the Middle East that hold exams in June, July, August, September time frame, and generally shut down the Internet for short periods of time over multiple weeks to try to, in theory, prevent cheating.
So we can talk about some of those.
Yeah, and this is something we've written about that is also very active within the Keep It On coalition, which is an organization where groups that are all trying to keep the Internet on and look against shutdowns.
But this has been, these exam shutdowns have been a feature that Iraq, Syria, Algeria, this year we saw, I think it was Mortegna, we saw a number of them.
Yeah, all were doing the same kind of thing where there's this, you know, and to put it in context, you know, and I've spoken to folks there, you know, they have these major exams, the baccalaureate exams or other exams that define your future.
So, you know, in those, in their education systems, you know, how you do on these exams determines what path you have in life, you know, whether you're going to university, you know, trade school, whatever, other different kinds of aspects.
And so they're very serious exams. But the process that we're seeing is that in order to stop people from cheating, they are literally shutting down the Internet for the entire country.
And sometimes it's, I was really probing at this with somebody, and in one of the countries, they said it's during the time they're distributing the exams.
So they're taking these central points, driving them out to the exam locations and stuff, and they're trying to prevent somebody in the middle who might, you know, take that exam.
Right. So definitely in looking at those shutdowns, looking at the timing, it often tends to be fairly early in the morning, like they start around 6am or so.
So kids are certainly not in school by then, but.
So it's when those, you know, when those exams are being distributed, they want to stop them before people are getting, you know, in there.
But yet you look in these same countries are very often trying now in 2024 to build out their digital capacity and to bring people online and to do all this and to build thriving sectors.
So, you know, on our shutdown incident pages on the Pulse site, you know, we often will include, you know, snippets from social media about people, you know, what people are saying.
And you have people saying, wait, this is 2024. Right.
What are we doing here? You know, four hours because of exams. You know, I'm trying to do my, you know, my DoorDash equivalent, you know, type of thing.
Or I have a business to run.
I have banking to do. I have, you know, medical issues. I have all those things that are now, you know, primarily online.
I think from a policy perspective, I mean, you know, is it that these countries know that it's a bad idea and just don't care?
Or is it, you know, what can we do as sort of a community or an industry to really impress upon them that you really need to find a better solution for this?
Well, I'd like to think that as the countries get more and more, you know, do more and more online and build up their online economies, I have to think this will change.
You know, I saw one of the posts when I wrote something up for one of the countries, you know, there were these stories of these people who are like, I'm doing freelance work for countries that are outside and I am losing jobs because I can't guarantee that I'll have connectivity during this time.
Somebody else is applying for a job. And he's like, what do I tell them?
You know, I've got to tell them that I may not have connectivity over the next 10 days.
Well, I mean, hello, they're going to go to somebody else.
You know, we saw in Iraq this year that the government, one of the government agencies had said, you know, we do not want this shut down.
Right. I think it was earlier in the year.
That was like an earlier round of exams, I think. Yeah. They petitioned to say like, hey, you know, we're not going to shut down the Internet.
And that's nice. And they shut down anyway. Right, right, right. So you have to think, and I think part of what we can do as an industry is keep this pressure on, keep talking about it, keep saying, you know, this is not shutting down your entire economy because you cannot figure out how to stop the cheating or stop the potential, you know, of this.
It's just, I mean, look how much the economy is suffering because of it.
And I think that's the key point there, I think, is like everything else, sort of money talks.
And I think the Internet society's cost of shutdown tool, right, is a great way to demonstrate that.
Yep. Our net loss calculator that we have is a great way to go and show that.
And I think the more we see people being able to say, look, you know, you're costing us jobs.
And that's that I think will be it.
But in the meantime, this past quarter, we did see it again and again and again.
Right, right. I know, you know, looking now at the fourth quarter, I had, I don't think I've seen any exam related shutdowns in the last month and a half.
So hopefully this quarter has actually been a little bit quieter in general, which is nice.
But yeah, you know, definitely we've seen, we I think frequently see a spike in government directed shutdowns in the third quarter.
Yeah, I mean, we did see, we are seeing already another topic which you've covered in your report, which is election related protests or shutdowns.
Right. You know, you mentioned Bangladesh and all that was going on in there, although that actually was not, that was actually not related to an election.
That was related to the economy coming back to it and the student jobs and the unemployment rates and all of this kind of thing, the massive protests that were happening all across the country.
Right. And that shutdown, those shutdowns happened, I think, in two rounds where there was an initial set of shutdowns that I think first, if I remember correctly, first hit the mobile networks and then the landline networks.
And that was sort of a complete outage in the country for several days.
And then they came back up, I believe in late July and then in late August, sorry, excuse me, early August, there was another, another round.
Yeah. Yeah. Yeah. And it's it's really interesting, you know, all that's, you know, that's going on there.
One of the challenges we've certainly seen on the Pulse side has been how do you, you know, how do you keep track of, or how do you know how much is shut down and pieces are there?
You know, you folks, obviously Cloudflare Radar have your, your ways of seeing the traffic that's coming in there, but sometimes it's harder, you know, but the good news is there's organizations happening.
Like there's a group called Shutdown Watch that is actually in Bangladesh that is working and helping us identify regional localized shutdowns and pieces like that.
There's some good groups that are out there doing that kind of work.
And, and obviously others in the measurement space, folks like UNI and IOTA and others who are out there doing great work as well.
So there's a whole collection of pieces that are helping us figure these things out.
Right. Exactly. And I think that, and I think a lot of these groups are good at communicating what they're seeing.
I think that's certainly been helpful in terms of understanding, you know, when you're, when, you know, is it just us or is everybody seeing it?
When did it probably start?
When did it probably stop or end? I think that's often the challenge. And certainly from our perspective, a lot of times we'll see, you know, if it's a real complete outage, it's easier to say like, okay, it very clearly ended here because traffic all of a sudden shut up.
Right. And then there's others where sort of like this is gradual restoration.
And you're now, it's also interesting because we're looking at not only bytes traffic, but we can also look at requests based traffic on radar.
And you can see sort of the shift in both and try to understand, okay, when do we call it?
When do we say that it ended? Right. Yeah. Cause sometimes it's super easy, right?
If they, if they drop BGP routes, border gateway protocol routes, if they've done that, that's easy.
Cause boom, it goes back up. That's when the route, yeah.
And it goes and it comes back up easy, simple, you know? But yeah, other times it's like, okay, is it coming back up?
What's the, is that, do we call it on off?
What's going on? It's hard. And then, you know, potentially watch for, you know, social media or other communications from either the providers or the government or something like that, that say, Hey, you know you know, connectivity was restored at, you know, 1800 local time or UTC or whatever, or, or, you know, power was restored to the island at, you know, such a, just in such a time.
And that often will align with what we've seen in terms of, of probably a larger increase in traffic.
Yeah. You know, over on the Pulse site, we write about government mandated shutdowns.
That's what we primarily do. So we're always looking at what's there.
So yes, it's beautiful when you get a statement like just recently in November, we've seen, you know, the government of Mauritius, you know, declared that they were going to shut down and block all social media for a period of time.
That was easy cause you had a declaration, you know, it's like, like the exam shutdowns, right.
In Iraq and Syria, they publish, they publish a schedule, couldn't make it easier.
Here's the schedule when you've got no Internet plan accordingly, folks.
And you know, that's the easy case, but then you have other issues.
Like just recently we've had the government of Mozambique has been shutting down the Internet, election related protests and stuff, right.
But there isn't a declaration.
You just know that you watch it and it drops in the evening at like eight o'clock at night, local time.
And it comes back up around five in the morning.
Yep. They were doing the curfews. But you don't have any, there's no statements that you can find that say that, that say when it's going to end or whatever else.
Whereas like with Iraq, Syria, Algeria, they publish their schedules.
So you and I, you and I can chat on Slack or something like that. Yep. See, we are, you know.
Yeah. And so, so actually the talk of schedules and the talk of, you know, what I mentioned, you know, when power comes back, I think is actually, actually a good opportunity to segue into another, I think, key cause of Internet outages, which is power outages.
You know, in some cases I found on social media, you know, if you follow the local power companies a lot of times, you know, they'll post and say, Hey, you know, there's, you know, there's, there's going to be a power outage in this section of the country, you know, from, from, you know, date time to date time.
You know, sometimes we'll see that impact in the, in the, the traffic graphs and data.
Sometimes we won't cause maybe it's too localized, but you know, I think the point though, is that without power, there's, there's no Internet basically.
And that goes obviously on the data center side and the, you know, the bits and pieces that makes the Internet run.
But, but really, I think obviously so on the end user subscriber side, you know, if I don't have power, chances are good.
I don't have Internet connectivity. Right. Right. Yeah. And, and you're right.
Sometimes you saw like Ecuador issued a statement, they were doing maintenance on their grid over.
Right. Yeah. A lot of these countries have really fragile electrical grids.
So, you know, they said, okay, we're going to have, you know, this is the period of time when it's going to be down.
And so you knew that there was a nationwide outage and saw that in the Internet outage as well.
Other times, you know, Kenya had a number of different kinds of issues that were going on there.
Venezuela had some, you know, breakages in, in, in power stations and pieces like that.
Cuba has had a several with one of their, their kind of their primary, you know, power production station, you know, just, you know, had problems and shut down, you know, so you had that, you know, in Cuba has definitely been struggling over the last few years with, with the aging power infrastructure.
Yeah. And this is kind of a, you know, across so many different parts of the world, we have this issue that the power infrastructure is aging.
And so now we're, you know, we become so reliant on the Internet, but yeah, the Internet needs power for it to work.
Right. Right. And, and, and so I, I hear you, you've written a blog post recently.
I did. Today, even on this topic. Yes, exactly.
Just saying about that, you know, no power equals no ones and zeros, you know, we've Internet resilience requires that.
And part of it was just because, you know, what you've said, your disruption reports increasingly have shown that there's more and more power outages.
Right. And, and on our end, you know, on the pulse platform, we just track government mandated shutdowns, but yet we trigger on, we see when there are drops in outages from Cloudflare radar, from IOTA, from UNI, we see these things.
And so we're looking at these and, and we've written, we sometimes will write on our, on our pulse blog.
We'll write articles. They are about outages and pieces, you know, like the whole thing around the cable cuts that happened off the East and West coast of Africa earlier this year and things like that.
But, but we, so we see this, but so we've been seeing this, this drumbeat, you know, of more and more of these power outages that are going on because, you know, you'll see that whether it's the aging infrastructure, whether it's extreme weather, you know, and you covered that in your post too, right?
We've had hurricane after hurricane after hurricane in our North American, Latin American part of the world that has hit us.
And we've seen that with, you know, barrel took out, hurricane barrel took out most of the power infrastructure in Jamaica, you know, them without, with, with very little there.
Hurricane Helene followed by hurricane Milton here in the Southeast United States took out significant areas of, of local infrastructure, of local power infrastructure and Internet infrastructure followed, you know, as far as the ability, although we can talk a bit about the resilience there, but, but you know, we've seen that, that pattern going on that says, you know, this is, this is something that, that needs to be paid attention to, you know, plus of course, you've had military conflicts across parts of the Middle East and Ukraine and other places where, you know, one of the parties has blown up the power infrastructure.
Right. We saw that a number of times over the last couple of years.
I mean, that's what we're seeing in those areas where again, you're losing Internet connectivity.
And that's, and that's much harder to, I don't want to say solve for, but it's much harder to, I think, sort of advocate from a policy perspective.
You know, you can advocate to, to authoritarian governments and say, stop shutting down the Internet because you're losing money, but you know, you can only, you can only yell at them just so much about, Hey, fix your power grid.
That's a significantly different investment.
Yeah. The, the military conflict side of things is not something that we're able to solve.
No, right. Right. No, but we can, but we can talk to, you know, I think one of the things, and one of the reasons I wrote the article was saying that, you know, we're not, we, you and I in the Internet industry and the Internet resilience space, when we're talking about this, we've looked at this, we look at alternate pathways, we look at routing infrastructures, we look at different things, but we don't necessarily take talk a lot to the power people who have been looking at this.
And in fairness, when I looked at some of their events, I went looking at, you know, what are some of their, cause like, right.
The power industry has been working on resilience for all of their existence, right?
Right. This is their thing. They need to do this. They've got the, you know, they got the trucks ready to roll and all that.
But when you look at their conferences around power resilience, their agendas, don't have people talking about Internet resilience.
So it was an interesting disconnect between both parts of these.
So part of why I'd written that article today was to say, you know, like we need to be talking to each other, we in the Internet industry and the folks in the power and resilience, right.
Right. Absolutely. You know, we're looking on the pulse side to, you know, impulse.
Well, you're familiar cause you were helping with it there, but we have our Internet resilience index where we look at that and we look at the resilience across the infrastructure, like IXPs.
We look at the, the, the market readiness, the security, the different aspects of that local content, the caching and pieces there we're looking.
That's easy for us to measure.
Yeah. We want to add electrical resilience. We were trying to, but we're, we're trying to find globally available, you know, open source data sets that we can use and bring in.
And some of that there's like industry-wide metrics, but we have not yet found.
And my guess is that's also an industry where they're not going to sort of open source, you know, the, the grid maps and the, you know, the, the, you know, like, like the, the telegeography submarine cable map, you know, you don't have that equivalent for the electrical grid because obviously that creates a whole separate set of, of, you know, attack surface problems.
Yeah. I don't, I don't know.
And I don't know. I mean, and this is maybe part of the question I ask. I mean, I, I don't know these, you know, and thinking this is where listeners, you know, if you've got ideas around this or you've got connections, let's, let's talk, let's be in touch around some of this because there may be things that we just don't know because we're not talking and we're not even using the same acronyms and stuff.
No, that's, that's, that's true. Yeah. It doesn't cross pollination would be, would be very valuable.
Yeah. So, so I think talking about resilience and talking about, you know, electrical resilience that obviously as you mentioned, the hurricanes that swept through the Caribbean and the Southeastern U S over the last few, you know, few months, you know, that obviously has done a lot of damage, not only to electrical infrastructure, but also, you know, Internet infrastructure.
You know, a lot of times, not only in the U S, but also internationally, I'll see posts on social media when these things happen, where people are sort of, you know, appealing to Elon Musk, you know, we need Starlink and, and, you know, Starlink will save us.
And, you know, I know that you've done a lot of work around looking at low earth orbit satellite connectivity.
You know, what, I guess the question sort of is how real is that appeal?
You know, how is, is, is satellite connectivity, whether it's Starlink or Project Kuiper or, you know, one of the other up and coming commercial services, like what, what I suspect that is not going to just solve the problem, but you know, what, what, what is its potential?
Yeah. So let me back up there too and say, one of the things that we're seeing is, is helping in all of that is we are seeing an amazing amount of interest in, in just in solar power.
And part of that is coming out with, there's just a flood really now of cheaply made solar panels that are coming, that are becoming available.
And so we're seeing in parts of the, in the world just recently, people may or may not know that Nepal suffered some very large flooding that happened up in there.
But some of the, we worked with a community network, getting one established up there in the Everest region, and they were able to keep connectivity because they had solar power.
Okay. Which is also true in places like Kyrgyzstan, where they brought connectivity to remote villages that just don't have electrical.
So, but it's also true in like the Southeast United States where people were able to go and continue to, to, to connect because they had some kind of solar power, you know, this distributed generation, distributed energy resources.
There's a bunch of terms that are coming out of that space, but that's the base power aspect that you're seeing.
And then yes, you're seeing that these low Earth orbit satellites or LEOs, whatever, LEOs, however you want to say it.
I say LEOs, but whatever.
They're providing this great ability to have resilience in this kind of thing.
We saw with Hurricane Helene, certainly people who had Starlink subscriptions because it's active in the, in the area that's there.
Right. No need, no need to appeal to Elon Musk because it's already active.
It's turned on. So they're, they're there.
You saw people flying in lots of Starlink, you know, antennas like that.
I have a, I know somebody who's working on, there's a project called Connect Carolina, which is trying to restore connectivity into parts of Northern California, North Carolina.
And his group is what they're doing is they're doing basically a cell tower that is only a small scale, but a cell antenna that is then connected in this case to OneWeb, which is now called Eutelsat OneWeb because it was acquired by Eutelsat.
Oh, right. That's right. That's right. But they're, but they're using a OneWeb antenna.
So they're sort of using satellite backhaul instead of fiber backhaul.
Satellite backhaul. Or the small cell mobile. Which then gets you into, so people can just use the regular old cell phones and connect to that local antenna and local thing to do that.
I work with, I'm a volunteer with another disaster relief organization here in the United States called the ITDRC or IT Disaster Resource Center.
And they have a similar technique where what they'll do, and they're active throughout the Hurricane Helene region and Milton.
They have trailers that are towed down to a region that will have a Starlink capability on there.
And they'll also have mass for cellular connectivity and Wi-Fi connectivity.
So they can come into an area, set that up and now be able to provide Wi-Fi and cellular connectivity for people who are there so that you can be able to get that connectivity back out of the region into places, wherever.
So it sounds like the 2024 version of what they used to call the cow, the cell on wheels.
Right.
Where Verizon or someone would drive in a cell tower on a truck. And those things are still there.
Now there's Internet also. Right. Yeah. Verizon, AT&T, those organizations still do those kinds of tasks.
They bring that portable system and stuff.
But ITDRC, other groups are able to also go and do this because there is this high speed, low latency connectivity coming from these lower authority satellites.
Now they don't have the capacity necessarily. So you're not going to take the entire town that's right around that isn't going to go and all be able to stream movies and everything else.
But to be able to shoot out an email or a WhatsApp message or something to a loved one somewhere else that says, hey, we're good, we're alive, whatever.
And I think as we've seen also, I think over this year at least as well, with the cell connectivity directly to satellites.
Yes. So my iPhone can send a message through a satellite to somebody else.
I think that will...
I believe that they're promising sort of more connectivity, virtual connectivity.
Yep. There's a whole world of direct-to-device, as it's called, direct-to-cell.
And part of this involves new satellites going up, which have improved radios that can communicate on the frequencies that your phone uses.
Okay.
So there's some parts of that, but as SpaceX is launching its next rev of Starlink satellites and they'll have these radios, they're already launching them.
So they're already able to go and do that.
They've received authorization to do some test experiment things.
They've filed to go and use more frequencies to be able to do this kind of thing.
Other companies and organizations who are going to operate in low-earth orbit are doing that same thing.
Again, like with your iPhone, which is actually using the Global Star Network, which is another older network of LEO satellites.
It's mostly just emergency messages. Right now, yeah. Right now. But it's moving toward where you can have more messaging and more pieces.
And you could see as you get more of a mesh, the capability to do more and more and more over time.
You have a whole separate conversation around that mesh and some environmental concerns and all sorts of other concerns, but that's another show.
But it'll be interesting to see what role the Internet plays in disaster recovery, disaster relief going forward.
Because I think right now we're at the point where there's an opportunity to use it in some ways, like you said, with the cow or with whatever the organizations are bringing in with this connectivity.
Maybe in a few years, we won't even need that because I'll just be able to, as long as I have a battery pack for my phone, I'll be able to maintain connectivity with the satellites and let people know what's going on or help coordinate or what have you.
Well, you look at what's being come out.
I don't know if you've seen the size of the Starlink mini, but it's about the size of an iPad.
And it's something you could just chuck in your backpack.
So funny you say that because I saw something yesterday. I assume it was true.
I haven't looked into it, but there is literally a Starlink backpack where it comes with the Dishy or whatever, some version of the Dishy.
Probably the Starlink mini.
Yeah. And probably some little power source or whatever.
I saw a cool Kickstarter project that somebody's done a little carrying case.
It's about the size of a little bit bigger than an iPad that you could have the thing in there and batteries, and you could just be able to go and take this out and power it by a solar panel or whatever else.
So it's remarkable. ITDRC does a lot of work out in the Western United States with wildfire support.
And so you could drive a pickup truck out with a Starlink Dish and be able to go and get that kind of connectivity.
Again, not the capacity you may have with terrestrial infrastructure, but in times of a crisis or a power outage like this, a tremendous way to have that extra resilient capability.
Because you can plug it into your truck and power it, type of thing.
It's fascinating to see what's coming on around.
To see how it's evolved. Yeah. No, absolutely. And I think you're probably around the same age I am.
I'm coming up on 30 years in the industry. And just to see, A, what role the Internet is playing now in all these things, but B, the fact that when we started Mozilla or whatever it was at that point, Mosaic was still pretty interesting.
And now what we're able to do - 1993 Mosaic, yes. Yeah, yeah. Oh, yeah. No, it was all of that, right?
When we had our 1,200-Baud or 300-Baud modems going. Anyway, let's start getting into the gray area or we'll stop there.
Old man on the Internet is another.
We'll do another one of those. We'll do another episode of that, old man.
So we've got about a minute left, so we can wind it up. So where can people find you, either online or anywhere you're going to be presenting or appearing in the next few weeks?
Sure. Sure. Well, you can find me. I'm Dan York on many of the services.
I mostly use Mastodon these days. So I'm mastodon.social slash at Dan York.
But you can find a lot of this content in the Internet Resilience Index, the pieces, the stuff I was talking about there are all at pulse.Internetsociety .org.
And we have our shutdowns. We have our resilience. We have country reports. We have an active blog.
We have a lot of different resources there for people looking to understand about shutdowns and resilience and IXPs and lots more.
So you can find us.
That's great. Great content. And I'm D. Belson, D-B-E-L-S-O-N on most platforms.
Radar.Cloudflare.com is our primary platform with all the data and graphs and whatnot.
And I'm also actually the editor of the Cloudflare blog at blog .Cloudflare.com.
So you can find a lot of these insights there as well. So thank you, Dan.
Thank you, David. Look forward to doing it again another time. Thanks.