Internet disruptions overview for Q1 2022

And we're live. Hello and welcome everyone. We're here to talk about Internet disruptions, an overview of Q1 of the first quarter. And I'm João Tomé and with me, I have David Belson, our head of data. Hello and welcome, David. Morning, João. Morning or good afternoon. Exactly. You wrote this blog post about several disruptions and outages and shutdowns that were on the first quarter. First, it was a very busy quarter, but then what is an Internet disruption? What types of Internet disruptions there are? That's a good place to start. An Internet disruption is when, as it sounds like, connectivity to a given geography effectively goes away or becomes limited. They happen for a number of reasons. One is that there may be infrastructure damage like a cable cut. Sometimes it's power outages. The power outages can impact infrastructure like the routers and the data centers. Or obviously, if your home loses power, it's going to take you offline. Other times, it's due to natural disasters like an earthquake or a hurricane. And then other Internet disruptions are probably more on the what I'd call geopolitical side. Generally, those are shutdowns where there's no sort of correlating physical issue like a fiber cut, but usually... It's a decision by someone. Right. It's basically some government agency, for whatever reason, decides, hey, we are going to limit Internet connectivity in this particular area for whatever reason. And then we ultimately see the traffic going away. Of course. And all of this data you connected in the blog post that I'll share in a few seconds, it's related to Cloudflare radar data. We have Cloudflare radar that has a bunch of data and we can monitor a lot of situations that happen through the Internet and we can see that really, right? Yep. Absolutely. Yeah. Traffic data from a platform or network as large as Cloudflare really gives a phenomenal insight into when these types of events occur. Oftentimes, we'll see either for a full shutdown, we can see traffic for a given geography or given network going completely away, dropping to near zero. In other cases, depending on what caused the disruption, we'll see traffic drop sharply and then maybe not all the way. And then that persists for hours, minutes, days, really depends on the cause. Of course. Here, I'm already showing our blog post about disruptions, the overview. Yep. It's a long blog post, of course, because it has a lot of outages and this was a busy year so far. So far, yeah. We're only three plus months in and it's interesting because there are definitely times when there are a number of disruptions happening often concurrently around the world and then you'll get several really quiet weeks where there's nothing significant that is observed or detected. So it really depends on... Yeah. Of course, it depends on the situation there is in. January was a big month there in terms of outages. We saw the Tonga outage. We wrote a blog post about that. And the Tonga is a very specific situation because it's not every day that a volcanic eruption damages a submarine cable. And in a way that it's the only submarine cable to a country, in this case, a Pacific archipelago country, island country, Tonga. And it's a full shutdown of the Internet because the country is fully connected by that submarine cable between Tonga and Fiji. It was near complete. Yeah. In this particular case, yeah. They were relying almost exclusively on the submarine cable between Tonga and Fiji. There was a little bit of traffic that started to come back about a week or so after the eruption, after the outage started. They started bringing satellite service online there. So they were able to regain some basic connectivity, some real low-speed, low-capacity connectivity that they relied on. I think Starlink, there was some discussion about Starlink helping out there. I don't recall if we ever saw significant traffic coming from Starlink from Tonga. But yeah, it took them a little more than five weeks to get the international cable repaired and bring connectivity back to Tonga. They were also saying that the domestic cables going between the islands there would take another several months, potentially six to nine months to fully repair. Exactly. It was a real problem for the country. It was really disconnected. And satellite services, from what we've seen, was very residual. I think I saw this. It was only on the main island. There's more than 100 islands. And only the main island had some Internet connectivity using satellites. Right. And if I remember correctly, they were using the satellite. They were parceling it out. I think they were focused on getting the banks and the hospitals and the government back online first before they made it broadly available to everybody so you could catch up on your TikTok and your YouTube. And I don't think they wanted to clog the limited satellite bandwidth with social media and such. Of course. And in that specific area, Japan also had an earthquake and that had a disruption, different disruption. Yeah. So in this particular case, there was an earthquake off the coast of Japan. It caused a brief kind of multi-hour disruption there and not a significant one based on our observation. This is not surprising. I think elsewhere we've seen around the world when an earthquake hits, depending on the damage it does, obviously, it can have a pretty significant impact on local Internet connectivity. And interestingly, when I was looking into this one, about 11 years prior, there was an even stronger earthquake around Japan. And that one also had a very nominal impact on connectivity there. So they've clearly got a lot of redundancy and a lot of sort of earthquake proofing of the infrastructure there. Of course, as usual there. So they have to be prepared in a sense, right? Yep. Yep. We also saw some infrastructure damage, mainly in the eight-hour outage in Gambia due to submarine cable damage. So yeah. So there were two big submarine cable cuts that caused major disruptions in the first quarter. Like you mentioned, the first one was in the Gambia. That was the ACE, Africa Coast to Europe submarine cable. And that was interesting because they lost that connectivity, but they had backup links through Senegal. Unfortunately, what they discovered is that the backup links through Senegal ultimately converged in the same place. That became a single point of failure and did fail. So that wound up taking Gambia offline for that eight-hour period. The other interesting one was in Tasmania, where there are three submarine cables that connect Tasmania to the Australian mainland. And two of those three cables were cut. So that disrupted traffic there for a little more than six hours. Yeah. Submarine cables are very important, of course. And when there's a disruption, things can be problematic in a sense, especially when countries aren't that prepared in terms of redundancy, like you were saying about Japan. And sometimes they have redundancy in place. Sometimes it's either expensive or potentially impractical, physically impractical. So yeah, it really depends on where you are and your economy and a number of other factors. Exactly. Sometimes it's just a few hours of disruption. Sometimes it's minutes. Sometimes it's days, like the time of situation, which was more serious. In terms of infrastructure, there were a bunch of them, of course. Also Yemen and Iran, right? Right. So those saw damage to data centers. So in the case of Yemen, there was a four-day outage there that was caused by an airstrike. Obviously, it's a politically unstable area. There was an airstrike on telecommunications headquarters. And that obviously damaged the data center infrastructure there and took the country offline for a while. It's interesting. It was one building, telecommunications building, but one building being damaged created this situation. Yeah. So that speaks to some other issues around centralization of Internet connectivity in a given country. That's probably just a different Cloudflare TV segment we can do. True, true. So yeah, Yemen is definitely one of those countries where I think there is more of a concentration because they have that primary state-owned ISP. In Iran, another country with sort of a similar model, there was a fire at a data center of one of the main telecom providers there. That only took them offline for about four hours. And again, for this particular one, because there's a little bit more choice, but there are multiple providers internally, the fire at this data center did not take the whole country offline for those four hours. We can see in the traffic graph there that it was a little bit more of a minor disruption. True. So different patterns there too. Yep. Yep. And then what we also saw, we've talked about submarine cable damage, we talked about infrastructure damage. It's almost an industry joke, but a lot of Internet disruptions are also caused by cable cuts. So fiber cuts. And we saw a handful of those in the first quarter as well. There was one in Cuba showing here. Cuba is another country with a very centralized telecommunications model. So there was a fiber cut there that caused a six-hour disruption. You can see that the traffic pattern there looks a little bit different than the historical patterns for that time period. In Pakistan, there were multiple fiber cuts at a Telenor Pakistan, which is one of the major providers there. And that took them down for about two hours. And then in Venezuela, there were multiple Internet disruptions in Venezuela over the course of the first quarter. In many cases, they were related to power outages. The power infrastructure in Venezuela really can leave a lot to be desired. It's very unstable. But in this case, it wound up being related to a fiber cut associated with CanTV. So one of the major Internet service and media providers within Venezuela. And they oftentimes, when you see an outage or a disruption in Venezuela, CanTV is implicated. So in this particular case, it took out connectivity to users in multiple Venezuelan states. So it wasn't a countrywide outage, but probably more focused on areas where CanTV may have a large number of customers. Of course, a different type of situation here. Very local, very related to the way the country works. It's connectivity, really. Should we move on to the power outages? Yeah. I was just thinking that was a good segue talking about Venezuela and then segue to power outages. So as I mentioned, Internet disruptions in Venezuela are often power outages related. But in this particular case, we did not cover any of those specifically within the blog post. But there were several others. The one that I thought was probably most interesting was the one we saw in January with Kazakhstan, Uzbekistan, and Kyrgyzstan. And this is what happens. So apparently, their electrical grids are all interconnected. So a problem with one of the power lines in that region ultimately caused problems or power outages across all three countries. And I guess maybe we see that in the United States at more of a state level where there's interconnected grids. The problem in one state can ripple into other states. But in this particular case, the power outages across these three countries had differing levels of impact. We saw traffic in Kazakhstan had a basic disruption for a few hours. While in Uzbekistan, the disruption was more significant and longer lasting. And then Kyrgyzstan had a pretty significant disruption but seemed to have recovered more quickly, at least more quickly than Uzbekistan. True. Yeah, different trends there, although they are connected. And we've seen that through this show. Yeah. It's interesting when you see those sort of concurrent disruptions in Internet connectivity, you need to start sure if there's no sort of obvious or promoted reason. So that there's a telecom provider or electric company provider has not posted something online saying, hey, here's what happened. Here's why there's an Internet disruption. A lot of times, you have to start looking for those areas of commonality. Are they sharing an electrical grid? Are they all connected to the same submarine cable? That at least there's public information out there about, so it's easier to tell. Or are they all connected to a similar terrestrial cable, which oftentimes is much harder to figure out. But a fiber cut like that can take down multiple providers, multiple countries. True. And I think it's very interesting to see how some countries are more vulnerable to this situation, be that power outages or like we saw before, undersea cables, or even non-undersea cables, just cables between countries. There are some vulnerabilities there in terms of some countries. Yeah. That's probably the great and the terrible thing about the Internet is it's all interconnected. So ideally, there are many paths to help maintain connectivity when something goes awry. The flip side is that you often don't figure out where the weak points are until something breaks. And you discover that both of your backup paths converge in the same place. And when that one place fails, that's a problem. Or that most of your international connectivity for a given country goes through one data center. And when that data center has problems or is damaged, connectivity for the country goes away. True. In Taiwan, it was a different situation. Yes. So Taiwan was interesting because what they ultimately did was traced it back to human negligence during repairs at a power plant. So human negligence at the repairs of the power plant ultimately caused power outages, which took the citizens offline, disrupted their power, and ultimately took them offline, causing a drop in traffic, at least in our observations. Yeah. Not very big. Probably more the redundancy there. In an area like Taiwan, probably, yeah. Yeah. Makes sense. And also the Cuba situation also caused a power outage. Right. Another outage in Cuba. Again, this one was fairly brief, fairly minor, about 90 minutes. It was significant, but not complete. So there was some connectivity maintained, meaning it probably had more of an effect on end users rather than the core network infrastructure within the country. True. And this is, in a sense, shows how people, not only us, but everyone can use the radar area where we have the ASNs. So for example, this chart that we're showing here regarding Cuba, people can go and check radar and check the ASNs area, for example. Right. And with Cuba, I think there's... TEXA is, I think, the primary ASN there. So we may see some others, but the primary international connectivity is going to be coming through TEXA. It's 27725. Here it is. Yeah. The names are more... You have to be able to translate between the acronyms and the actual name. Yeah. So it looks like things are pretty happy there right now. Yeah. This long names, in this case, Empresa de Telecomunicaciones de Cuba, with my good Spanish, is an ISP that has a different name, more short name, short version. That happens a lot in the ASNs. Right, right. And I think in this particular case, I think on Twitter, they also go by TEXA. So yeah, that's how I just refer to them in shorthand. Of course. And we also have... Should we go to DDoS attacks? Sure. Yeah. So we saw one significant DDoS attack that caused this broad connectivity disruption in Q1. And this was a target... This attack targeted a network in Israel, the Tehila Project Network, and this wound up taking them offline. So this is apparently a... At least they provide some hosting services according to published reports. So a lot of the government websites, government ministry websites were on this particular provider. And they may provide some connectivity as well. But what we saw is when the DDoS attack began, traffic from Cloudflare to that autonomous system, that network provider, dropped pretty significantly, basically close to zero. Yeah. It's a very specific situation, but it can impact, in this case, a large number of websites in the country. Yeah. And it really depends on what's being targeted with the attack. Sometimes it's just attacking a single website or a single property, and there's not really a large blast radius. Sometimes if the attacks are targeting DNS or DNS servers or routers, it may have a broader impact and take a larger population offline. Of course. Makes sense. That's a specific event, but we also have unspecific technical causes in a sense, right? Right. So this is what I was talking about earlier where some of the other ones we've discussed, it was due to a fiber cut or related to a power outage. In these particular cases, we observed Internet disruptions in these countries, but there was no associated publication of a root cause. But when we dig in, we also find out that these disruptions, the timing of these disruptions aligned with sociopolitical events within the country. So one of the early ones we saw during the quarter was in Kazakhstan where there was some protests around increased energy prices. So obviously government often don't like protests. In many cases, they will take steps to limit the organization of those protests. And more recently, what's been happening is that the governments have taken steps to basically either block access to certain sites like social media sites, or oftentimes they'll just cut off Internet access entirely either within the country or for a specific region. So that was what happened in Kazakhstan for a six day period. But interestingly, what happened was that connectivity returned briefly several times over those six days, as we highlighted in the blog post that you did, that aligned with televised speeches and announcements from the president there. So I think they said basically, hey, we'll let you back online so you can watch the government communications. But then after those were over, we're cutting off the Internet again until tomorrow. Very focused, focused comeback of the Internet for a specific reason. Right, right. And then in Burkina Faso, again, there's apparently an army mutiny that was taking place. And there were reports of heavy gunfire there. So obviously some disarray. And what we saw an Internet disruption that lasted for about a day and a half within Burkina Faso that started around the same time that gunfire was heard. So not to say that they damaged any particular equipment, but the government may have said, hey, this has got the potential to really go sideways. So we're going to shut down Internet connectivity. Because for some time, not as much as other situations we've seen even last year. Right. That's right. Burkina Faso had a bunch of, if I remember correctly, a bunch of disruptions over the definitely the fourth quarter and possibly the third quarter as well. Yeah. So they are definitely a country where there's an unstable political environment, which often leads to unstable Internet connectivity. True. We have five minutes. Should we go to Russian invasion of Ukraine? Yeah. So I mean, this has been really interesting to watch. And obviously, I know you've been watching it very closely as well internally. The Internet has become, in many ways, a really key battleground in the war that's going on there. In terms of being able to maintain access to information, in terms of enabling communication with loved ones and other countries. One of the really, I think, most interesting things that I've seen throughout this has been the work that the telecom engineers within the Ukraine have been doing in conditions that are unbelievable to try to keep the infrastructure, the connectivity available. And I've seen posts from other friends in the industry who have talked about the challenges that they've had in maintaining infrastructure. But the conditions that they're talking about trying to maintain infrastructure are nothing ultimately compared to what the telecom engineers are dealing with there. We did a couple of blog posts from Cloudflare that talked about connectivity and connectivity trends at a city level within Ukraine. In many cases, we saw disruptions happening during, as either traffic increases as people move throughout the country, as they left one area and moved to another area. We also saw a traffic increase, depending on where the fighting was, like Mariupol. We saw a gradual decline in traffic. And then ultimately, traffic has dropped to near zero. There was an article, I don't recall where it was published, but they talked about the last cell tower in Mariupol, where connectivity there was hanging on one cell tower, literally a wire. And I think that ultimately ran out. I think that ran out of diesel fuel. And that connectivity there essentially went flat. Again, a number of these other cities have seen significant declines in connectivity as well. True. I would highlight there two things. First, what you were saying, the movement we've seen from the East, where the fighting is going on more to the West, to the Lviv area. We all saw on the news, the very bad situation on Bushehr, with a lot of people killed. And we saw how the Internet was disrupted there, also on Mariupol, almost with residual Internet there, also related power outages, actually. And also some outages we've seen on the network level, like Ukraine Telecom that had some outages. And I would highlight there something they have there called national roaming. So when there's an outage in one operator, like Ukraine Telecom, the others push in. So it's free of charge and you can change of operator just to be connected. If there's an outage in one, that's a very interesting situation they have there. Yeah, definitely. And I think it's obviously great to see them banding together and collaborating and try to keep people online and connected within the country. And obviously, because the Internet has become kind of this battleground there, we've seen cyber attacks that are targeting key telecom providers. And in the blog post, we talked about two particular attacks that we observed, a 15 -hour disruption at UKR Telecom and a 10 -hour disruption at Trioland. Those, I'm sure, were not the only attacks that occurred targeting Ukrainian service providers during the first quarter, nor will they be the last, unfortunately. True. We have one minute or so. I will just highlight here. First, there's a lot of situations here, disruptions and outages here. And some are continuing. So for Q2, we'll have already some highlights. We published some of these on our Twitter handle, Fault Flare Radar. Everyone can follow. For example, we had this Turkmenistan situation with a shutdown visible, right? Right. Yeah, we believe this one was a shutdown. It follows a lot of blocking that they were doing, and there was no external discussion of fiber cuts or cable cuts or power outages or anything like that. So yeah, this was likely a government -directed shutdown. Exactly. It was on the news that they were blocking social network and VPN providers, so it makes sense. A lot to discover here, I would say, not only from Q1, but also from the things that are happening not only in the Ukraine, but also everywhere. So our time is up. Yeah, I'd say just encourage folks to read the blog posts, follow us on Twitter, and we'll obviously keep an eye on things. And stay tuned for the second quarter summary that we'll come up with in early July. Thank you. Bye-bye.