⚡️ Speed Week Spotlight: Internet quality on Radar

Alright, hello and welcome to the last Speed Week Cloudflare TV segment. My name is David Belson. I am Cloudflare's Head of Data Insight and I'm here today with Vasco Asturiano, Carlos Rodrigues and David Tuber to talk about the new Internet quality page that we launched on Radar today. Vasco, can you start by introducing yourself? Yeah, sure. Hello everyone. My name is Vasco. I'm a front-end engineer in the Cloudflare Radar team and I worked as part of this project of building this page, all the data visualization part. And I was also the original developer of the speed test, our speed test application. So we'll talk about that in a bit. Carlos? Well, I'm Carlos. I'm a systems engineer in the Radar team. In this project, I did data exploration. So I played a bit with the data scientists and then I worked with Vasco to make this thing happen live. Excellent. And Tubes? Hi, I'm Tubes. I'm a product manager for networking, performance and availability. And now I'm here to talk about network quality, what stuff we collect for the speed test, and how we got the data that Carlos uses to explore and create the visuals with Vasco. Excellent. Cool. Welcome, guys. Let's jump right in. So we published today, and I actually didn't bring that up, but I will. We published today a blog post that introduced the new Internet quality page on Radar. Let me share my screen real quick. So this is a really long, really comprehensive blog post that talks about the page itself, what we just launched. It talks about our partnership with MLab, and maybe Tubes can touch on that a little bit later. Why the data we're presenting today is important. I think this is something that, you know, it's important to also understand not only the speed information that we're providing, but also the latency and jitter information that we're providing, because really all of that taken together makes up the quality of the connection, even though it's the speed, the bandwidth that really usually is featured in advertisements and whatnot. And then within the post, also, we talk about how to navigate the page. I think we'll just jump over to the page live, and we can talk about that, you know, as we go. So it's available at radar.Cloudflare .com slash quality. And the underlying data, of course, is available within the Radar API, if you're into building your own tools. So when you first land there, you get to see a worldwide view. And this provides two areas of data. And actually, we probably should talk about that a little bit first. The Internet Quality page on Radar includes Internet Quality Index data. So this is data taken from some benchmark tests that Carl's will talk about, as well as data from Cloudflare speed tests. And we'll talk to Tubes about that. So the worldwide level, what we've done is aggregated the data at a continent level, or aggregated the Internet Quality Index data at a continent level. So we're showing bandwidth, so essentially connection speed data. We're showing the latency, so roundtrip times, and DNS response time. And we'll talk about why this is an important metric as well a little bit later. And then also at the global level, we're looking at the speed test data aggregated at a country level. So here you can use the Choropleth map, you can scroll around in it, you can look at a particular country. And in order to drill down further, what you can do is either click on one of the countries in the list, or click on a country itself within the map. So to dig in, let's say to Portugal, we click on Portugal. And we can see what's going on there. This also honors the time selector in the upper right hand corner. So right now we're looking at the last week. And something interesting we see here is within the bandwidth drafts for the Internet Quality Index in Portugal, we saw a really interesting spike over the last day. So sometime last night, it looks like. So it would be interesting to drill down and see what's going on there. But the dark blue line shows the median, and the light blue band shows the 25th and 75th percentiles. The latency graph, similarly, the dark blue line shows latency, or excuse me, median latency, and then the same banding applies there. And the same thing with the DNS response time graph. And then within the connection quality part of the page, where it's the speed.Cloudflare.com data, we have this radial graph, which is really intended to give you a sense of connectivity, connection quality at a glance. So really here, when you're looking at these graphs, what you want to see is larger green wedges and smaller yellow and red wedges. So the lighter green is upload speeds, upload bandwidth. The darker green is download bandwidth, download speed. Yellow is the latency, and we measure that both idle and loaded latency. And then the pink and red is jitter, which is also measured as loaded and unloaded. And then we also have a graph here that shows both two graphs, in fact, shows bandwidth and latency. And the histogram basically shows the buckets for speed test measurements over the last 90 days. So when you look at it, you can start getting a sense of maybe where the most popular connection packages are for a given country or a given network provider. So if we dig in further to a network provider, so Vasco or Carlos, should we look at Mio or Nosh? Mine is the first, so. First one. Okay, we'll dig in there so you can speak to that one. So if we look at Mio, again, we can see that basically it shows the same data as the country view, but it shows it just obviously for that autonomous system. So Mio looks like it has a median bandwidth of about, what are we looking at here, about 44, 45 megabits a second, median latency around 17, 18 milliseconds. That's pretty good. Median DNS response times around 25, 26 milliseconds. And then here we can see in the connection quality graph, this is nice to see. So it looks like download speeds are about twice measured upload speeds and fairly low idle latency and fairly low jitter. And then when you look at the histogram for bandwidth, you can see that there's probably a package they sell around 100 megabits. There's probably another package they sell around 250, and then probably another one around 500, because we see these clusters of the speed test results there. When we look at the latency histogram, we can see that the latency really is very, very tightly clustered, pretty low on the scale, whereas jitter is slightly larger clustering and has a much longer tail. And then down at the bottom of the page, we've got some Internet quality related blog posts to look at. So with that, I think what I want to do is dive into some questions for Roscoe and Carlos and Tubes. So Tubes, we'll start with you, and I'll flip over to the speed test page. But when did we launch Cloudflare Speedtest? How long has it been around, and why did we launch it? Yeah, so Cloudflare Speedtest has been around since about 2020, so a little over three years now. And I think one of the things that when we started it, we just kind of wanted to measure your throughput and just measure your Internet connection. I'm surprised you're able to run this. I realize this may not be the best idea to try to do this while I'm on Zoom. You may have goofed. We'll see. I just stopped it. So when we launched it, that was the goal. But I think over time, one of the things that we've kind of adapted the speed test to do is we've adapted it to show users what their home Internet is actually good for. And that's generally done through the network quality scores that we show that are currently under waiting to finish measurements. If you had not refreshed the page, you've got to see that David actually has quite good Internet, although his video chatting maybe could be a little better. And so one of the reasons that we kind of did this is that you look at the Internet speed test page, and we published a blog about this a couple months ago. We look at the speed test page, and there are just so many numbers. I think a lot of normal users, there is a lot of stuff here. And it's really hard for normal users to kind of understand, OK, so what is my Internet good for? These numbers mean something, presumably. Our ISPs will tell us that more bandwidth is better. But if I'm running, if I'm, let's say, if I'm now trying to watch a movie on Netflix and I'm seeing buffering problems, is that a problem with my Internet connection? Is Netflix having an issue? I want to know what's going on here. And is my Internet actually good for the things that I need it to be good for? And that's kind of how we've kind of tried to shift the speed test from where it was to where it is now. And I think that we're really, I think that a lot of the stuff that we're putting on radar is really, really valuable. And I'm super excited. I'm super excited that y'all did this, because this is really great information that people can see to kind of understand, you know, I have my experience. How does my experience compare with other people who are using my Internet or who are using my Internet provider in my country? Right. And I think we're, and I think with more of the data that we're giving them there, too, it gives them a little bit of ammunition if they needed to go back to a provider to say, hey, my connection is good for this, but not great for that. But I'm also finding that I have high latency or I'm having, you know, you can solve the bandwidth problem in some ways by maybe relocating your laptop or buying a larger, you know, bigger speed package. But if it's other issues, it gives you, we're giving them a little bit more ammunition to go back to their provider to say, hey, look, there's something wrong in your network. You know, I've done what I can. A little bit, yeah. I think that a really cool fact that I think probably a lot of people don't know is that a little over 50% of support cases filed by customers to the ISPs complaining about performance are solved by just getting closer to your Wi-Fi access point. And that sounds like that obviously to Internet, to people who are in the Internet community, that obviously makes sense. But I think that one thing people don't understand is Wi-Fi is actually kind of generally determined by the house that you live in. So for example, one of my friends, he was saying to me, you know, like, I just bought this new house and it's got all of this really cool exposed brick and my Wi-Fi sucks. And, you know, before I could get my Wi-Fi anywhere, but now I can't even, you know, connect to the Wi-Fi for my bedroom, which is like on the other side of the wall. And I was like, well, that exposed brick wall is really like thick and like you're late, your signal just can't get through. So you should get an extender. You should get like one of the Google mesh Wi-Fis or the Eros or all the stuff like that. They sell stuff like this. And so it actually, if you look at the tool tip for the network quality, one of the things that it will say is this isn't just that latency, high latency could actually be a product of, sorry, the learn more actually is where you go. The learn more will actually tell you this high latency could actually be a by-product of your poor Wi-Fi signal. And so a speed test is a pretty good approximation of just kind of understanding, you know, in your current conditions, what can you do, but it's more than just, you know, Oh, go, go complain to your provider. It's actually, there are things that you can do yourself, right? Like you could upgrade your router. You could switch to a wired connection, getting closer to your router. Sometimes you just need to simply like, we start your computer. Really like, yeah. Like sometimes like as a network engineer or someone who works in network, you know, sometimes rebooting just fixes the problem. But like, there are a lot of things that you can do. And a lot of what we do on the speed test now is trying to give users tools that they can do. Just like say, if I'm not having a good experience, what can I do to get better? That kind of democratizes the support so that you don't have to sit on a call with, you know, your tier one support person for 20 minutes for them to be like, okay, now restart your router. They're just doing their job. This is kind of like get a little bit like, like try and take proactive steps to improve your situation. And that's really what this is about. It's that, and also just kind of showing what the landscape looks like. Like that's why the radar stuff, this was really, really great. Like we created the speed test so that you could see points in time, but we aggregate this stuff and we've partnered with MLAB and we've got this huge, like, I like the phrase data lake of data now. So we can go see all of this stuff in aggregate and the radar stuff that y'all have published is really a great way for users to figure out what the Internet looks like around them. And another scenario where this could be useful is let's say that, you know, I live in Seattle, right? I live in Seattle right now, but let's say that I moved back to Boston. Cause I'm originally from Boston, go Celtics. And I am like, all right, I just bought a new house. I want to pick my Internet provider because I am hopefully moving to a situation where I can, you know, pick my Internet provider. And I am like, well, what do I think? Well, I go to radar and I go to the quality page and I can see, you know, Oh, let's look at all of the, I'm in the United States. Let's look at the providers. Let's pick the one let's, let's scroll through them and analyze them and pick the one that has the lowest loaded latency. Right. Instagram. And that's like a really good place for me as somebody who cares about latency as someone who cares about that stuff to figure out, you know, where are we, where, where do my needs, where are my needs best filled at any given for any given provider. Right. That's a great, no, it's a great, great use case for this. And I want to shift to, to, to Carlos. So what I'm going to do actually is also highlight the fact that on the speed test page. Now you're the, the network information links directly to the Internet quality page for that provider. So I on Verizon Fios. So we'll look at the, the IQI section here. So Carlos, what is the, the IQI section measure or what does it estimate? Since we talk about estimated download speeds and such here. Yeah. So when you, when you are like reading the news on CNN or scrolling through Facebook, you're not, you're actually not using our Internet connection to its full capacity. Right. And the performance you get is less dependent on the speeds that your ISP promises to sell you and more dependent on general network network conditions and in particular latency. So if you have very high latencies, your speeds will be limited by the latency. That was actually a good blog post on our, on our blog, like a few weeks ago. Yeah. Yeah. And it's like, does link from a launch blog post. Yeah. It's at the bottom of the page. And so what IQI is, is, is doing is, is showing up, showing you numbers that you would see if you happen to go and measure your traffic while you are browsing the web. So not, not doing like heavy downloads or expressing your connection while you're just doing one, just browsing. If you measure your, your, your traffic, you'll are likely, or the is very high probability that you'll see numbers very, very close to those that IQI is showing, or at least within that blue region that shows like where you're more, more likely to be expected bound. Yeah. Well, this contrast to what speed test speed test shows you because speed test is about pushing the limits, seeing about what your connection can, can do. But most of the time you won't be pushing the limits. So it's important to know what your connection can do at regular times. And that's mostly driven by, by, by latency. So you usually see when latency goes up in those graphs, you'll see bandwidth go down. When that, when that doesn't happen, usually it's indication that someone else, something special is wrong in the, like in the network. So there's congestion or something, or some, some link changed into a slower bandwidth link. And it's kind of making things go slow without moving to transit or something like that. Yeah. Yeah. Something changed, but usually under normal conditions, the two graphs will follow each other. So And then we you know, I think many of us are familiar with, with bandwidth and we're familiar with latency, but we're also in the IQI section here showing DNS response time. How should we think about that metric in terms of our, our Internet experience? Well, DNS, when, when you try to go to Google or Google.com or any other destination, the first thing that your device needs to do is go look up in DNS, what's the true network address of, of that destination. So, because the network address that it needs to contact is not the name, it's, it's a number. So DNS is at the, it's at the beginning of everything. And when you have slow DNS, then everything will be like a bit slower. And there's a saying in technology, which, which is, it's always DNS that many, many, many incidents you, when you trace them down to their roots, it turns out it's something happening with DNS. So that's where we should. So, so it's really just highlighting that this is really an important, an important, but sort of behind the scenes part of the Internet experience. Usually don't think about it. Yeah. Right. Because usually in theory it just works, but not always. And then, you know, we talked, Tubes talked about the, the speed test and, and, you know, we, we've highlighted that section of the page, but with the IQI data, where does that come from? Where are those measurements? Where and how are those measurements being made? Well, we constantly measure our, we as Cloudflare, we constantly measure our, our real world performance against other industry players. We do that all the time. Just yesterday, we published a blog post about that called Network Performance Update. We have published a few others before, that's why it's an update. And what we do for IQI is we, we take those measurements plus, plus others that are actually not mentioned in the blog post, but they're the same thing. And we aggregate them as a, like a whole and per country and per network, like on a continuous basis. So you can think about IQI as like a stock market index. You have a subset of relevant industry players that we chose to represent the performance of the whole, of the whole thing. In IQI, the set that we use for IQI, it's 10 different industry players, some large and some others like a bit smaller. And yeah, so they are meant to represent, that set is meant to represent the, the thing that you will see in a country or a network. Right. And because they're, all of these targets have sort of different deployments and different mapping technologies and techniques, there's sort of a, there's no, no one provider or no one, you know, POP is necessarily overrepresented in this data because we're aggregating it all. And, and it's, the providers are, if you go and see like most of the traffic that you get when you browse the Internet, the set of 10 providers is like most of the traffic you get. So it's uniform across the world. And well, if in cases where it's not uniform, then you really have a bad Internet experience. Right. So, so yeah, that's a relationship we want to show. Yeah. Great. Cool. I'm going to shift to Vasco now. I've got a couple of questions for him. So I'm going to scroll down here. I know as we were developing this page, you know, there was a lot of heated discussion around how to present this data and the design of this graph. So can you talk a bit about how users should interpret this radial graph? Is radial graph even the right term for it? Yes, actually we like to call it flower plot because it's a little bit like a flower plot as seen from above. So what we wanted to do with this is answer a little bit, give you like a little bit of an overview of like, what is, what is the aggregate of this country or of this network look like in terms of the quality of the connection? And this is actually a multidimensional question, right? You need to take into account like a set of different metrics in order to really answer that question. So this chart is like an attempt to try to put all those like relevant ingredients together and show it in a way that it kind of like makes sense as an overview, as something that you, you, you look at this for a certain country and then you look at it for a different country. You should be able to see from like the distinction exactly what, how those two countries compare. So the way to read this is essentially each of these wedges represent a certain metric and that metric is actually the median of the aggregate across the whole population that is selected here. So in this case for this particular autonomous system, we have all the tests that were performed and, and there's a wedge representing the download speed, the median of the download speed, another one for upload, et cetera. So on the top part, on the top hemisphere, if we will, we're representing the throughput because of throughput. And then on the Southern, on the bottom hemisphere, let's call it, we're representing like the, the latency related metrics. So the top part is actually maxed cap at one gigabit per second. So if some network or country as a median that goes above that, that would be a good place to be. But at that point, it will just be trimmed over the whole graph, but that's, that's really unlikely to happen. And for the bottom part is cap that one second. And another thing that's relevant here is that, so the colors, we chose a color scheme that's more like on the greenish side for the top part, because this is understood as like throughput, the more throughput you have, the better it's supposed to be. So, so we'll call it as green while the bottom part is, it's all like latency related metrics. So they're all like within the yellowish, orangey, reddish. Sort of the more of the danger colors. I think I was going to bring up Starlink here just as a contrast where, you know, here it's a, it's a very different sort of graph than what we were just looking at for Verizon Fios. Yeah. That's certainly different. It's expected for the satellite provider. It has higher latency, but not, not too high. So it's a, it's a usable service. And the download is much larger than the upload, which is also expected for it. But the takeaway from this graph though, would, would arguably be that Starlink may not be on average sort of spectacular for like video conferencing, because you've got the high latency and some jitter, which is going to lead to, you know, stuttery video. And because you're, you're uploading, you, you have very limited bandwidth to do that upload. That would certainly be the takeaway that I get from this graph right now. Oh, wow. Their upload is very limited and very clustered. So somebody, and I just want to say, you know, somebody who kind of like came up with the metrics and the stuff that we wanted to measure, like seeing all of the graphs that we've put together are so cool. Like I love the flower graph and I love all of this stuff. Like you test too, like, and you know, maybe eventually we can, but like, this is just the coolest stuff. Like we've got, we've got a lot for Vasco to work on first, but maybe we'll give them back at some point. No, this is great. So thank you to the three of you for participating in this conversation today. We've got about three minutes left. So a few other things that we want to talk about. So future plans for the page, you know, I know Tubes on the, the speed test page. We, we show packet loss. So here we show packet loss data and we also show, we would show the aim measurement scores here had I not stopped the speed test. So our goal is to incorporate that into an upcoming version of the quality page. And I think keying off of something that was a piece of feedback you heard at NANOG when you presented something you just mentioned here about, you know, the hypothetical loop back to Boston, we'd like to also be able to present more granular geography on the, the Internet quality page so that, you know, when Tubes does move back to Boston, he can look at it and say, for Massachusetts or for even Boston specifically, show me the providers that have a, a corpus of tests there and, and show me the quality that's been measured at, you know, for the provider in Boston. I think that, you know, could be able to get that granular will be really, you know, it's a unique perspective from, from an industry perspective. I think it'll be really, really valuable. And I think, you know, in closing, you know, we've got speed.Cloudflare.com, what we're showing here. So, you know, if you want to test your speed measurements or your, your, your Internet quality, you know, go here and run a speed test. We urge you obviously to check out Cloudflare radar, not only the Internet quality section, but also all of the other capabilities we have and that we've, we've launched over the last couple of years. We are at Cloudflare radar on Twitter and radar at Cloudflare.social on Mastodon potentially coming soon to other social platforms as soon as I have the time. And that's about it. Yeah. We've got about a minute left. Any other comments guys so that we don't have to cut the commercial? Well, my, my, my comments about that is also, if you want to dig yourself into the data, if, if you're that kind of person, we also, you have both in API, but you can get the same data that the page is showing and, and a bit more. And also at least for the speed test side of things, you have the raw data that we use unprocessed being published through MLab. And you can query there BigQuery instance, or we have a table there with all the data. Thank you for that reminder. Yeah, it's absolutely true. So yeah, measurement lab is a great partner of ours, and I think we're really excited to work with them on making this data accessible and available. And as Carlos mentioned, we've got the API. So thank you all for joining us.