Introducing new Cloudflare Radar traffic insights

all right hi everyone my name is nikita kano i'm product manager for cloudflare tunnel and warp connector here at cloudflare and we are almost wrapping up a very special week here at cloudflare we're celebrating 15th birthday and we are thrilled to have you join us so as you remember for us birthday week has always been more than just a celebration we're giving back to our custom to the community and to the internet itself by launching technology that pushes the entire industry forward and helps everyone to be safer and more secure on the internet. So today's announcement is very special in that regard. We have Cloudflare Radar team with us and I'm going to introduce them one by one to you so that you could get to know people behind this great product and then we will deep dive into what exactly we are presenting. this Friday. David, let's start with you. When did you join Cloudflare and why and what are you currently doing? I joined Cloudflare back in January 2022. Why did I join? That's a long way in Cloudflare. I'm running on four years, so it's been great though. I joined because it was a great opportunity to work with Radar, which had launched just before I joined. For the last 17, I don't even know, I can't count, like 17 years, I've been working with various, you know, internet -related data sets, internet measurements and so on at various organizations, various companies. So when the opportunity came about to join the Radar team and work with the great data, the great team that Radar and the Cloudflare has. uh i obviously jumped at it uh my role is my title technically is head of data insight uh my role is at the moment um a product management or product managing radar amazing this is this is a dream job yeah it absolutely is i think it's it's it's just so cool to be able to you know when somebody has a question be able to be like yep we can just dig into that and you know ideally it's we can just dig into it on radar sometimes we need to dig into it behind the scenes but we're constantly of that behind the scenes data and bringing it forward amazing and since you've managed that you're product managing cloudflare as a fellow product manager i'm going to ask you a question first and then i'll ask the same question to everyone here because it's kind of foundational to what we do so why providing this level of internet visibility uh that we're doing with radar is so core to cloudflare's mission to help build a better internet why do we have radar in I think a lot of it goes back to the, you know, one of our core tenets, which is transparency. So the company demonstrates that in a lot of different ways from, you know, the near immediate root cause analysis blog posts when we have an issue to weekly bureau meetings to, you know, everything like that. I think the opportunity to give the community better insight into what's going on in the Internet, I think, is definitely. um you know along the lines of just being as transparent as possible yeah and for whoever doesn't know your meeting is our kind of weekly all -time meeting um and uh this just again shows that it's not just something that we do for uh everyone on the planet like we are literally using radar every day ourselves to answer questions about the internet and it's just amazing that we're investing in like this because it improves daily life. Yeah, absolutely. And I'm always excited to see, you know, radar references externally and I'm always happy to see them internally when somebody posts something in Google chat or, you know, I see a radar graph on a slide or something. You know, I know that we're doing the right thing. Yeah. And we'll talk about how researchers use it a little bit later because especially like, at the very least, one of the features there will be very interesting for research. Luke, let's discuss the same things with you. When did you join Cloudflare and why? Yeah, so I'm on the research team, not actually on Radar, but fortunate to work with the Radar folks on a lot of things. I interned in 2018 and then joined full -time in 2019. So I guess, yeah, six plus years now at Cloudflare. And yeah, so when I joined the Radar... team didn't exist yet so a lot of a lot of what i did in my my phd program and stuff was um well i worked on a paper with nick sullivan who was the head of the cryptography team at the time and we used cloudflare data in that paper and and i was like nick how can i like can i get more of this cloudflare data it's amazing like there's so you can get so much insight into the internet with this so he's like okay well like come intern with us and you can you can work on projects and then And once I joined full time, a lot of what I did for for a couple of years was like working with academic groups and trying to figure out how we can use Cloudflare data and distill it and make it, I guess, put it in a way that we can share it publicly in research papers. So I was very happy to see Radar come along because this is exactly what I was trying to do manually, like one off, one off projects. And then now we're just sharing this like near live. um uh near live data all the time and in the amount of data we show has expanded so much so so i'm yeah very very excited about so essentially what you're saying is if radar existed in 2018 you may you might not have joined cloudflash because you would come back maybe i would have just joined radar directly at that point yeah that was good and i i guess i um and what i've worked on now that that's not my full -time job working on sharing data etc um i've worked on a lot of different systems at cloudflare and certificate transparency is one of those systems that um that we've worked a lot on it over the years for the research team and we're um i'm happy to see uh some of that being able to share some of that data on radar now amazing yeah we'll talk about certificate transparency um as well here today uh Before that, Andre, some questions to you. When did you join? Why did you join? What are you working on right now? Yeah, so I joined in April, 2024 as a summer intern. And yeah, I joined the Radar and Data Insights team and now I'm a systems engineer there. So I mostly work with Radar API, the Radar website and other Radar related projects like URL scanner. And now... monitoring and I'm also part of the AI crawl control team rather product interesting product related to AI and yeah I joined because I used Cloudflare before in personal projects and in school and I saw the the summer internships were open and I sent the CV and Then I got to know Raider, and I must say that I fell in love with Raider. Yeah, it's a really nice project. And it's amazing that we have all this traffic coming through Cloudflare, and we provide these insights and these trends for free for everyone to use and to understand how the Internet works. It's kind of a good plug that we're... We've committed to hiring 1111 interns next year. This was my favorite announcement on Monday. And when we were chatting with Karin, my partner in crime, who also organized this year's birthday week, we were like, this is amazing. And Andrei, you've hit both of those things, which makes me very happy. So love all the intern references because we're hiring 1111 interns in 2026, but also we're going to work. in a way cloudflare developer platform for free for a year for all the students starting with the edu platform so uh andre if you weren't now the uni or at school you would get all of those things combined but you don't need them anymore you work for cloudflare so let's see uh how the new generation of developers can use these things and radar as well so now let's move on to what we are doing with the radar So one of the two big announcements that we are making for the Raider this birthday week is a more localized, we can call it subnational, we can call it regional perspective. Let's start there. What specific event was happening that made you realize that going from a country level to a state or provenance level view was the necessary next step for the Raider? Maybe something prompted. to think about this yeah so so it's been um it's not just a single event i'd say uh you know one of the things that i spend a fair amount of time doing is tracking uh internet disruptions so internet outages internet shutdowns um and and oftentimes those are taking place at like you said a sub -national level uh you know one uh example that i guess didn't wind up happening today though is um there's a region called Kamchatka and they were going to apparently there was gonna be some submarine cable repairs that was supposed to take that region offline so you know it may be the case that when I look at Russia traffic as a whole I don't you know it's not enough traffic to make a meaningful impact in the graph but I'd like to be able to see you know what happened locally and you know so previously we would have the ability to look at Internal, we have the data, we just didn't expose it. So we would run what we call Jupyter Notebooks. So a local analysis. And like I said, look, here's the graph. It shows, you know, X, Y, Z. But you on the outside have no ability to get that same view. So that's one reason is, you know, from an Internet disruptions perspective, bringing the ability for a user to see the impact more locally. The other is, you know, on the flip side is sort of the events. How do, you know, how does human activity drive shifts in traffic? And one of the things we look at frequently is like the Superbowl in the U S so, you know, around the Superbowl, how did traffic in, you know, I, and this is where I show my ignorance of football, you know, how did traffic in Texas and Pennsylvania change? Assuming that the, you know, the Cowboys and the Eagles were playing, which may not even be possible. I have no idea. But. But, you know, looking at that and saying, okay, you know, you can see the traffic, you know, rose at a certain point in the game or dipped in a certain point in the game. Oftentimes we've looked at that and said, ah, that's where halftime was because everybody shifted from, you know, paying close attention to their television or the streaming platform to going back to sort of surfing the web briefly for a variety of things or going to social media sites or going to AI sites or whatever it is. And we can see those shifts in human activity. So being able to also. view that on radar as well. This is amazing. It's completely like a new level of visibility for everyone and for the researchers. And one must think that there are probably some engineering challenges in taking a database and mapping IPs to specific regions and then using it to accurately map billions of daily events all while rigging. Preserving user privacy. I guess that's one of the goals as well. So why can't you engineer this? I mean, so we, it wasn't really a huge departure from what we were doing previously. So what we're doing basically is within, we got a request, we log the request, we log the information about it. And then our IP geolocation tool that we use will assume... that particular IP address with a prefix. So it says here's, you know, this IP address is part of this larger group of IP addresses. This larger group of IP addresses is associated with what's called a GeoNames ID. So GeoNames is a publicly accessible database of ID numbers to geographic locations. And it goes everywhere from, you know, the country level all the way down to, I think, you know, probably parks and streets and things like that. So we're not going quite that far, but so what we're doing is saying, okay, this particular IP address is part of this prefix that maps to this ID. And then what we can do then is map the ID number using GeoNames data to a country and a region. So it's, it's, you know, obviously we're, we're trying to, you know, continue to maintain privacy. We're not going down to some level that is incredibly creepy. We're still keeping. it aggregated. I think the other challenge also is that the more granular you get, the accuracy begins to suffer. And that's something where for us, accuracy of the data is really paramount as well. I think the challenge, honestly, has been more on the geopolitical side, where there are some disputed territories and making sure that we are representing those territories appropriately. uh both in the naming and on the the maps um as a person who did the terrible code that syncs our status page with the colors that we have and the century mapping to this i know exactly what you're talking about yeah it's you know i i spent a fair amount of time on calls with policy team last week making sure that we're getting exactly right so we didn't get anybody in trouble good good and uh maybe last question on that um Because you also can now, there is a data explorer in Raider, if you didn't know this existed, now you know. And now it allows someone to filter regional traffic by a specific network as well. Right, I was going to mention that as well, yeah. Which is also, I think, pretty cool. That's one of the things where we've seen in the past as well, I don't remember which provider, one of the major providers had regional issues. So they were saying their network was... having problems in you know texas and ohio and florida and new hampshire and whatever so now that gives us the ability to go in and say show me you know as701 verizon fios or as7922 comcast or whatever in a particular in a particular state for instance in the u .s um you know we can also look at it and compare so we can say show me 701 and 7922 in massachusetts uh so we can say oh look you know this the you know Horizon is working okay, Comcast is having problems, or vice versa. This reminds me, last night I was running into an incident of one of the platforms, and as always, sometimes it takes time for the teams to put this on their status page, and you're wondering, did the variable break, did something cache locally? So now I think the life hack for me is, if I see some discrepancies in my internet connectivity. first thing i'm gonna do you can go check and see what what does your provider look like in your area exactly and and you know always is cloudflare seeing a larger drop in traffic from them indicating an actual network issue or is it this is a great use case i wish maybe a feature request from me maybe we have these uh speed tests and connectivity tests um that measure your um quality of the internet connectivity maybe if we see that there is a drop from and we show some kind of notification on that speed test that, hey, it's not you, it's your provider, maybe call me. Yeah, we're always looking for more ways to extend the speed test data. Cool. Amazing. Now for the second major range, and this is the Certificate Transparency Dashboard, as Luke already mentioned. So this is a very opaque and obscured part of the Internet for many, I assume. So maybe let's start. Let's start by explaining the real -world risk this tool is designed to prevent. We know from the recent Quad 1 incident that this is just a theoretical threat for anyone on the internet. So, how and why we are externalizing certificate transparency? Yeah, so certificate transparency is best thought of as like a safety net for the internet. So... The way that a website authenticates itself to clients that are visiting it is it has to present a TLS certificate. And that TLS certificate has to be signed by a certificate authority. So certificate authority would be like Let's Encrypt, DigiCert, Sectigo, and there are quite a few more. A lot of, so Chrome and other browsers have like 100 plus certificate authorities that if they've signed it, a certificate then um a web then the browser will trust um a website presenting that certificate um and the fact that this list is so long is a little bit scary because now you're trusting these uh cas are kind of the gatekeepers for the secure internet and you're trusting um the weakest link of all of them and so uh back in 2011 there was a famous instance where DigiNodeR, which is no longer a CA, but it issued a, or it was hacked and somebody uses a private key to issue a certificate for google.com. And Google caught this because of some other mechanisms, safety mechanisms they had deployed, certificate pinning. But they thought this is really bad, like we need a way to catch when there are misissued certificates like this. So they invented. certificate transparency, which is this ecosystem that basically makes a way for every TLS certificate to be publicly logged and auditable. So a browser won't trust a certificate unless it has proof that it's been publicly logged to certificate transparency logs. So that's kind of like what certificate transparency is. And Cloudflare has been involved in this. system since like incorrectly 2017 and we we operate certificate transparency logs the Nimbus logs and then in the past year we launched a new series of logs called as RAIO or based on that as based on Cloudflare workers and then we also operate a certificate transparency monitor which is responsible for looking at all of the publicly logged TLS certificates and it checks that logs behave correctly in the ecosystem, but it also provides a way to alert website owners when there's a certificate issued for their website that is kind of suspicious. It's not the CA they normally use, et cetera. So we offer this service called Certificate Transparency Monitoring in the dashboard. that you can opt into. And then very recently, I guess in the past few weeks, there was an incident where FinSEA, which is a creation certificate authority, mis-issued certificates for 1.1.1.1, which is Cloudflare's encrypted DNS service. And the only reason we caught this is because those certificates were in certificate transparency so we were able to there's a kind of a public audit trail so we were able to catch this and now kind of moving on to like what what we're announcing in this blog post so we've offered a public certificate transparency dashboard called Merkle Town which I think we launched it in 2019 but it hadn't been updated a whole lot since then and it kind of just showed kind of broad broad statistics on the CT ecosystem. And so what we've done now is give a new face to all of that data. And I think Andre can probably give a good overview of kind of what you can do with this new data source or new dashboard now. Yeah. So we have some goals with this migration to Raider. So first I think it makes sense to move this information to Raider because Raider is like the hub in Cloudflare to show data, this internet insights and trends. And the main goal was not only to move the existing trends to Raider, but also to enrich and extend those. So as Luke said, we already had some summarized data of the ecosystem, but now... for example, allow users to fetch history, historical data, which is a thing that we like to do in radar. So allow users to explore how the ecosystem evolves to time. And also we extended the ways that users could explore the data. So this is currently the largest radar data set right now. with several filters and ways to break down data so for example we added new ca pages so if you go to radar and search for a ca or a fingerprint used by the the ca you can see a page with the ca information and and the behavior of that ca we also added the dimensions real to certificate coverage that's also new so you can see breakdowns by the TLDs covered by the certificate of or if the certificate covers IPs or uses wildcards to cover different domains and I think that those are great features and great additions to this dashboard. I think one of the cool things that we saw in there as well was being able to track the issuance by CA and like we were able to see the... the less encrypt issue, the outage that they had a couple of months back. So I think that's interesting if you, you know, if there's a CI that's having problems, in theory we should be able to see the volumes shift on our dashboard. Yes, and we can also see, for example, recently less encrypt switched to the new certificate. And we could see the certificate insurance from the old ones decrease and then ones increase so right also interesting one one trend i'm excited to follow is um just a couple months ago the ca browser forum which is kind of the one of the governing bodies of um the web pki uh they they approved a ballot to reduce the maximum certificate lifetime so it was i think 397 days and it's going to be going down over the next few years and increments down to i think it's 46 or 47. days by 2029 I believe so we're going to be able to see this in the radar data we're going to go to track how as we move to these shorter and shorter lib certificates and also let's encrypt is offering seven day certificates or six day certificates as well which so we're going to be seeing seeing more of those as well let's just explain to everyone why if somebody doesn't know why a short -lived certificate is a better thing for your internet property and for the internet in general yeah so so uh yeah essentially yeah certificate is signed by a certificate authority um and it has kind of built in in that signed data is the um has two dates like a not before and not after date which kind of cap how how long that certificate is valid for um so if that private key for that certificate gets compromised like i don't know a weekend to a year long certificate lifetime then then the attacker can continue to use that certificate for the the entire rest of that lifetime and maybe the certificate will be revoked but not every client gets revocation information and um it's uh revocation is a kind of a big it's not perfect um so So by reducing the certificate lifetime, it reduces how useful that compromised certificate is. And it can only be that compromised private key is only a threat for a much shorter period of time. Amazing. So given all of that for all the developers and people who... have internet properties watching right now, what is the single most important action they can take using the new information on the radar to protect themselves and their brand? Yeah, so I guess I should say with the reduced certificate lifetimes, make sure you set up automatic certificate renewal. You don't wanna have to be doing this manually. And plus when post-quantum signatures come along in the next few years, you just that if you have it all set up automatically you don't have to worry about doing manual upgrades the second is CloudFlare offers a certificate transparency monitoring service so you can get an email when a new certificate for your domain is issued so you can you can find this in the in the dashboard enroll and and yeah and use this to to make sure that nobody's issuing a certificate for your domain that should not be doing that. And I believe those, those domains don't even have to be registered through Cloudflare or served by Cloudflare, right? I think my, my recollection is I'm getting emails for, for domains, for sites that live elsewhere, but they, they're certainly useful. And since you mentioned a post-quantum shameless plugin here, so my friends, my friends from Zero Trust, Sharon, Goldberg, and Coco secured warm. to Cloudflare Edge with PostQuantum recently. So take a look at that announcement if you missed it. And at the same time, my friend from application services, Alex Krivet, upgraded 6 million zones to PostQuantum Secure from Cloudflare Edge to the origin. So now if somebody ever captures your data in transit and by the time lovely PostQuantum computers will be... um generally available nobody will be able to decrypt uh your information because this conversation is uh post -quantum secure now which is another good reason to follow cloudflare blog and you know make sure that you um take the best out of the innovation that we have here uh and keeping an eye on the post -quantum adoption levels on on radar uh we showed that i haven't i haven't looked this week much but i know Late last week, it was around the 40-ish percent. It's hovering at 49% for the past two days. So we're almost at the halfway point. Yeah, maybe later today, we'll hit 50%. Was it because of the Apple? I think I've seen something. So I've been tracking that. I think since iOS 26 came out last week, we were tracking. It was previously about, I want to say about 2%, 2 .5 % of the requests that we were seeing from iOS were PQ secure. And then you can see basically the inflection point was last Monday afternoon and started to grow since then. When I checked earlier this week, Monday or Tuesday, it was up to about 14 or 15%. And I think as more devices upgrade, we'll continue to see that grow. We appreciate you. We see you. And we will keep monitoring this. So now I think my last question. Since you're, I will, I will label you all as internet cartographers, if you don't mind, since this is what I see radar as sort of a cartogram of the internet. Since you are looking at this data yourself quite regularly, and it reflects major global events as you've referenced, elections, conflicts, disasters, everything as it unfolds. What is the most surprising or profound thing you've learned? about the internet and its resilience from radar and i want each and every one of you to answer this david let's start with you good question um i tend to look at i think in the other direction of where is it failing um you know but i think i think the the flip side of that though is that you know we do see we do see it come back uh you know because they're uh you know in in most places uh i think these days there are multiple paths there are multiple providers uh you know we will see outages we will see disruption shutdowns uh but the traffic does come back um i think the thing that surprises me the most to be honest is that governments are still using wide -scale internet shutdowns as uh you know a tool of control um it's really unfortunate And some of the reasons they're doing it are still kind of ridiculous, like trying to prevent cheating on exams. But, you know, they're using it as sort of a social hammer to, that was a terrible analogy. They're using it as a, you know, a tech, sorry, they're using it as a technology tool to address a social issue. But, yeah, I think, you know, we continue to see through all the data we have that the Internet remains resilient. Nowadays, people will be able to just have a local AI model to help them with exams. So, no need. Well, yeah, right. And it's going to become more and more compact day by day. So, we need some other solution to this problem. Luke, what about you? What's the most profound thing you found in the radio data? So, I think it's just that you can see the same event through so many different lenses with all of the data. we have available. So one data set I helped to work on and add to Radar last year was TCP resets and timeouts. So you can see kind of at the network layer when weird things are happening for traffic from a particular region or anything. So if there are middle boxes that are injecting TCP resets to break connections, we can observe this passively with our... network and and you also see like when you see spikes in tcp resets you also see a drop in successful http requests and you you can correlate and then you can look at dns to see if um see what what dns says about about this so i think that's just that's the um i mean that's one of the biggest value propositions for radar it's just that it aggregates all of these different data sources together and you can see the same you have the same underlying event um so many different ways to paint the full picture yeah and then that was just another widely used blunt technique and it's very obvious when you see the data and you see a spike in tcp resets and you see um something weird happening with dns and you know yeah this is exactly what's happening right and our policy team you know frequently tells me that they're they always you know they use radar a lot to conversations that they're having about you know here's what we see your country doing or here's what we see your network doing yeah andre what about you what is the most interesting piece of data you found yeah so i think the most interesting piece of data or the most profound thing that i learned with radar and i know it's kind of obvious but sometimes we forget is that the internet is backed by people so all the data that we see in in radar and throughout the different data sets that's all a reflection of how we use the internet. Even if it's related to bot traffic or not, the bots are there because we use them and we made them. So I think that's related to what Luca and David said, but that's the most interesting thing that I want to add. And I also like different perspectives. Like all of you had slightly different angle and this- great this is why internet must be open for everyone because different perspectives shape a different world and we do want an open safe and secure world for everyone and i love that radar plays part in that cool then thank you everyone thank you for watching us uh this is almost a wrap -up of birthday week 2025 we are celebrating 15th birthday if you missed anything or if you were living under the rock and you didn't even know this was happening blog.glasflare.com, cloudflare.tv, all of the blog posts and announcements are there. We had more than 40 incredible announcements this year. So take a look. There is a chance you've missed something and that you are not using the technology to its fullest. And make sure to follow our blog outside of the week too. We publish great deep dives there too. Happy Friday, everyone. We'll be right back.