Speed, Eurovision voting triumph, and a bit of history: Alan Turing 101
Welcome to our weekly review of stories from our blog and other sources, covering a range of topics from product announcements, tools and features to disruptions on the Internet. João Tomé is joined by our CTO, John Graham-Cumming.
In this week's program, we go over our Speek Week recap, mentioning a few blog posts we didn’t cover last week, including how Cloudflare scaled and protected Eurovision 2023 voting with Pages and Turnstile. Next, we talk about a deep dive with insights into the process of investigating networking issues and how to begin debugging dropped packets issues from negative header lengths. There’s also an explanation on how Cloudflare Zaraz now supports JSONata, enabling precise data customization for enhanced workflows.
We also have “A bit of history” moment, to celebrate and remember Alan Turing’s importance to mathematics, computing and AI, given that this month it's the 111 anniversary of his birth (June 23, 1912).
At the end, we have in our new “Around NET” short segment Matthew Bullock, based in London, from the Product team, to talk directly from our London office about some of the products announced during Speed Week.
You can check the blog posts:
- Recapping Speed Week 2023
- How we scaled and protected Eurovision 2023 voting with Pages and Turnstile
- Lost in transit: debugging dropped packets from negative header lengths
- Cloudflare Zaraz supports JSONata
And the Speed Week Hub (https://www.cloudflare.com/speed-week )
Hello everyone and welcome to This Week in Net. It's the June 30th, 2023 edition. We'll have a bit of our Speed Week spoils and also some Alan Turing one-on-one, given that Alan Turing would be 101 years old this month.
I'm João Tomé, based in Lisbon, and with me I have, as usual, our CTO, John Graham-Cumming.
Hello, John, how are you?
Hello, I'm fine, thank you. New week, and it's the after Speed Week week. We had a wrap-up at the beginning of this week.
It was not a full week full of blog posts, but it had a few.
One of those was the wrap-up of Speed Week. Do you get good feedback from the week?
A lot of blog posts, a lot of announcements. I can't remember how many blog posts there were.
It was like 35, 37, something like that. It was a lot of blog posts and a lot of stuff in Speed.
I think the big feedback is, it's a good reminder to people that Cloudflare does a lot of security stuff.
We have a developer platform, but speed really, really matters to all of us.
There are so many different aspects of speed.
Some of it can be how fast does a web page load, but it can also be how good is my ISP?
In fact, how good is my ISP can mean bandwidth, but it can also mean latency and jitter.
That can affect things like having a video call with the usual, or are you playing a game online?
There's a lot of aspects of what goes into making something fast.
There's a lot of complexity into making...
In fact, the funny thing about speed is a bit like security is people, the end user shouldn't think about it.
In fact, if they start thinking about it, something's gone horribly wrong.
From a security perspective, there's been some security problem.
They don't trust you. From a speed perspective, it's like, well, why is this slow?
We want that to never happen. Yes, Speed Week was all about that.
We also launched on Radar, and we talked about this last week, a tool that allows you a page that allows you to see different aspects of performance, of latency, of download, upload, things like that.
That's an average, but sometimes your problems with the Internet could be the distance that you are in terms of the router in your home.
It could be the router in your home.
One of the things I thought was interesting on the Internet quality stuff on Radar that we launched was you can see the difference in latency when the system is unloaded and when the system is loaded.
Why does latency change? You think the speed of light doesn't change.
The distance between you and the server you're dealing with doesn't change.
What's actually happening is there's a thing called buffer bloat, which is the routers and often the router in your home is buffering data before it gets sent out to the Internet.
You end up with this situation where the actual speed of light latency might be just fine, but there's some very large buffering happening along the way, which gives you a mess, which gives you a long latency.
It's definitely a concern about how big the buffers are, and particularly in routers.
That actually shows up in some ways to do with the load on the network. When there's load, you see a lot of buffering.
There's a lot of metrics here for people to explore.
In terms of the SpeedWeek in general, we have the already mentioned Recapping SpeedWeek blog post.
It's one of those that Sam Marsh wrote that we usually do, where it allows you to see all of the different things and products like Observatory or Digital Experiencing Monitoring or Timing Insights that we did.
It is organized by topics, right? Yes, absolutely. It's a nice way to break it down if you're interested in a particular topic, AI or Zero Trust or web performance, whatever you can get in here and see.
I particularly like this one called Measuring What Matters, which is there are lots of things you can measure, but do those measurements actually show the end user something that is useful and also actionable?
On the Internet quality stuff, we were just talking about latency is a very important thing.
Jitter might be a very important thing. In terms of measuring the web, lots of people use time to first byte.
Time to first byte is an interesting indicator, but it doesn't tell you really what the end user experience is like.
Looking at when it makes sense to look at TTFP is important. Then there's this new core web vital, INP.
INP is something we're going to be looking at very carefully because it is going to replace FID next year.
It's important in the Google search algorithm.
Also, it's important because it's trying to really capture that interactive experience for the end user, which is how responsive is the website?
What INP captures is the time between an interaction, i.e. you do something, and the paint, i.e.
when something changes on screen. If you think about what we do fundamentally with the web is we interact with it.
We type something in a box.
We click on something. We hover over something. Just as a user, that interaction between I did a thing and when does it respond to me, that's such a vital.
That's what INP does. Measuring that's really, really important. There's lots of interesting stuff in there.
Speed made simple. How does Cloudflare actually make things faster?
We make websites very, very much faster just by using our service.
How does that actually work? Here's some answers to that. That's a lot to explore here.
But wait, there's more perspective. Of course, there's also this little segment about Speed Week.
Yes, us talking about Speed Week last week where we were so overwhelmed by blog posts.
I think at the end, we started trying to run through them hyper fast.
We were getting speedier and speedier throughout that week.
If you want to see us trying to get through 37 blog posts in 30 minutes, go back and watch that one.
True. Actually, there's a few we didn't mention specifically like the Eurovision one with pages and turnstiles.
The Cloudflare page is the fastest way to serve your sites.
Also, the speeding up API is a big, big question.
The Eurovision one is pretty fascinating. The Eurovision Song Contest, the biggest non-sporting event in the world, 162 million people.
I think something like that.
We're tuned in all over Europe and beyond Europe. If you're not a Eurovision fanatic, you will not know that what usually happens in Eurovision, or at least over the last few years, is that there is a jury vote and there is also a public vote.
The public vote is done by calling a phone number, a premium number, and you pay a certain amount of vote.
Those votes get aggregated.
There's a very long drawn-out process at the end where they slowly reveal.
Honestly, they could reveal the winner in one go, but it's this whole- One hour just to reveal it, yeah.
One hour of just all this kind of stuff. The interesting thing is that this year, they decided they would allow voting by non -Eurovision countries.
If you think about Eurovision, all these countries in Europe and a little bit beyond because Australia is involved, but there are lots of other countries that some people in the US enjoy watching it, some people in South America, and they allowed voting.
That was not done through premium numbers. It was done with an app.
This little company called Base.io built the backend for that. The interesting thing about the voting is what happens is there's this, all the songs are sung and repeated and all this dramatic happens.
Suddenly, they're like, now you can vote.
Across Europe and Australia as well, I guess, people are calling in on the premium numbers, but everyone else was now using an app.
That app was entirely on Cloudflare using Pages.
They built this backend system. As you can see, they were very concerned about the number of connected users they could potentially have because people are in there voting.
When you looked at the graphs, you can actually see it in here.
I mean, HTTP requests per second, boom, annoyed. That's what happened when the grand final voting happens.
It's like, yum. We could see in different countries like UK, Sweden, France, this spike happened in more than one country.
Right at the moment they said, you can start voting, it just goes crazy.
This is a blog post about how this company, which has a very interesting backend database, real-time database they built, how this company built this and made use of the many, many different aspects of how Cloudflare works, the caching, load balancing, how fast we can update DNS, which was critical to this, Pages serving all those millions of requests per second.
It's interesting to see this incredible peak of voting happens in the first minutes of that.
Then it tails off and some people continue voting, but it all went well.
Their system and the power of our system, and I really love this graph.
This is requests per second going through our system on the worker's platform, so on the developer platform.
We do a lot of traffic, but to give you an idea of how big Eurovision is, you can see it cause a peak, which just shows you how big it is.
By the way, they also use Turnstile, which is our capture replacement to protect payments as well.
It really gave our system a workout and it went well.
It was a good test. It was a very good test.
Congratulations to them. It's a little company with only a small number of people who ran the actual voting platform behind it with their homegrown database system, which is very interesting.
This is the kind of stuff you can do on Cloudflare.
They put up Cloudflare Pages and they didn't have to worry about that landing page causing a problem.
That was a good use case. Should we move on for this week's blog posts more than Speed Week?
Let's get off of Speed Week as fast as we can.
Exactly. Pun, pun. Lost in Transit, Debugging Rob Hackett. Go on then, let's talk about Terran.
Let's go. So these are the kind of blog posts I love.
Not that I don't love all of them like I love many, if I had hundreds of children, if I loved all my hundreds of children, but this is a technical blog post about a debugging story.
I think developers have to debug. What was interesting was we were faced with a problem where packets were getting dropped by our system.
Some, not many, but some, because they had negative header length, which of course is completely impossible.
Header can't be negative. So when a lot of packets drop, that means a bad experience in terms of- Yeah, a connection gets, something has to get resent, causes a speed problem.
So Terran started looking into this, like why is this happening?
And as is, as quite often, if you've been around networking enough, one of the things that happens is encapsulation, which is where you take some packet and you stick it inside some other packet to send somewhere else, or just in layering as well, you get packet encapsulation and well, it goes wrong.
Basically, that's my summary of it. But Terran takes you through, write down like how, first of all, we noticed it and then we went in and we start debugging it.
And it's quite a long way down in this thing called GU, Generate UDP Encapsulation.
And once again, one of our favorite, our old friends, the Maximum Transmission Unit, which is a thing that causes an enormous amount of problem when transmitting stuff over the Internet, because you have to have the size of the MTU, which for truly historical reasons now, ethernet is 50 years old.
It's pretty much pegged to the size of an ethernet packet, which is kind of crazy, but that's the way things are.
And so Terran goes through like what are the situations in which you can end up with a packet, which has a negative length header?
And well, you can go read the story. He tells you exactly how they found it and where the bug is and what the patch looks like.
So if you're into weird debugging stories, tracing stuff in Linux, crazy fixes, here it is.
It's one of those deep dives.
Yeah. It's one of those. And it's also one of those, if you operate something, like if you operate as a single Linux machine at home, you just would never notice this, right?
But once you get to Cloudflare scale, one of the things that happens is that the long tail of weird behaviors suddenly becomes quite a fat tail.
You're like, oh, wait a minute, what's that weird behavior? So we had one, we had another debugging story not very long ago about us resending stuff.
And it's like this, it's like yet another one of these examples of like, well, why is this happening?
And of course it changes. We see the latency change dramatically because we stopped losing packets.
Exactly. And read all about it here. Read all about it.
It's a long one. Nice one, Terran. It's a long one. And it shows you an old protocol still debugging an old protocol.
What's funny is you'd think that these networking stacks were so highly used, they'd be completely stable and bug free.
And it turns out that they're not, especially when you operate at our scale. Exactly.
And you can find those little things there. And then we have Cloudflare Xeras supports JSON data.
This reminds me of Pastel de Nata, Portuguese Delce. Because of the Nata part.
It's got nothing to do with Nata, Joao. I know. But there's this motto from a company here in Portugal that sells Pastel de Nata called the world needs Nata.
Okay. Well, look, we'll just accept that Pastel de Nata are very good and not to start any fights, but I think that Pastel de Belém is overrated.
And I think Mantegaria is better than Belém.
I know about that opinion you have. There's a very strong opinion about this.
Yeah. I'm still a good fan of Pastel de Belém. There you go.
All right. So let's not talk about that. Let's talk about JSON data. For those who don't know, what is Cloudflare Xeras?
And because they are third parties, they create a security problem and they create a performance problem.
Security problem because if somebody hacks one of those, they essentially could run code within your website, in someone's browser, and that can cause security problems.
And we've seen this in the past where stuff gets and also can cause performance problems.
And so the idea of Xeras is, well, don't do that. Load them on the Cloudflare network, which is close to the end user, and then only send back the information you need from the web browser to Cloudflare to then do like Google Analytics or ads or whatever.
And so that's great, but quite often you need to have custom information.
Let's suppose you're going through the purchasing flow on a website.
You may want to keep analytics about where people abandon their cart, and you may want to enrich that with that information about maybe where they were located.
Oh, this was so that you know that cart abandonments in the EU are higher than the US or something.
And JSON data is a way of adding data to it. And so what you can do is you can have data added within the Xeras system, and then you can have this querying done through it.
So we're adding support for it.
Xeras is a really interesting product. If you're including third-party tools on your website, think about using Xeras so they're gone from there.
And what this allows you to do is not be held back by the complexity of the systems, because this JSONR stuff really adds a lot to the offering.
So a good new addition in terms of support.
That's all the blog posts we have for this week. Why not go to the a bit of history part?
It's a bit of history that Alan Turing would be this month, June, 101 years.
So why not do a one-on -one on Alan Turing, in a sense, with that binary thing part there?
And I'll start to ask you, and you have a bit of history with Alan Turing, not that you've known Alan Turing, but you have a bit of history there.
But before we go through your history with Alan Turing, why was he a mathematical genius that made contributions to what is now called modern computing?
Yeah, I mean, I don't know if I go around calling many people geniuses. I'm not going to say that I think he was a genius, but he was a very, very, very able mathematician.
Let's put it that way. So he's probably most popularly well-known for having worked on code-breaking at Bletchley Park, in particular on taking work that the Polish code breakers had done on breaking Enigma, because they had quite a lot of success with Enigma before the Second World War.
And they built a machine which they called the Bomb.
And then Turing worked on a British version of that, extended it with others for slightly more complex versions of the Enigma that the Nazi Germany started using.
He also worked on some other stuff. One of the things I think is quite interesting is he worked on a lot around statistical methods applied to code-breaking.
So a lot of code -breaking stuff he's very well known for. But prior to the war, one of the reasons why I ended up at Bletchley Park was as a mathematician at Cambridge.
And he worked on a very theoretical piece of mathematics.
So within mathematics, there were some questions that came up at the sort of beginning of the 20th century about to what extent could mathematics be made systematic.
And there was sort of these, well, there's a guy called David Hilbert, a very famous mathematician who laid out this long set of things about, it was around unsolved problems in mathematics.
And some of those were around some obscure things in mathematics, but others were, there were questions actually about like, was mathematics complete?
Were you going to be able to get to somewhere where you could have a set of axioms that were completely consistent, and you could sort of, if you like, do all of mathematics within that.
And a guy called Kurt Gödel came along and said, no, actually, that's not the case.
That if you, no matter what system you define, there's always some problems in mathematics, it's going to be like outside of that system that won't be provable within it, which was quite shocking.
And there was a related thing, which was to do with computability, which is like, could you have some systematic way of doing something?
And that problem is something that the Turing worked on. And the thing is, when he worked on that problem, he had to define what something systematic was.
And a systematic thing was some sort of machine.
And if you go all the way back to Leibniz, he had thought about mechanical calculating machines.
And so in Hilbert, when he said, this problem of like, sort of what we think of as computability now, a bunch of people worked on this, trying to figure out whether there was some systematic way of proving things in mathematics, using formal languages and stuff like that.
And in a little bit of a similar way to Gödel did the completeness theorem, Turing showed that there were always going to be algorithms which were what was called undecidable.
We wouldn't know whether they would terminate or not, for example, whether a program would ever stop.
A bunch of other people worked on similar stuff, a guy called Alonzo Church, and then also a guy called Post.
We either think about, you sometimes say things are Turing complete, which means they can compute all the things that computers can compute.
We sometimes say Church-Turing compute.
Sometimes we say Church-Post -Turing compute, complete. There's these sort of awkward things.
But in the paper on the Entscheidungsproblem, Turing invents this machine.
He calls it a machine. It's this thing with a tape that goes through it.
They have symbols on it. We call that a Turing machine now, and it's fundamentally used as an idea, basis for computing.
Finally, he got interested in artificial intelligence and started thinking about what it would mean for a machine to think, and a bunch of other stuff.
It's very wide-ranging mathematics. He's particularly interested also in why do animals have the patterns on their skin that they have?
Think about a Frisian cow, the black and white. How does that develop?
Or spots on a leopard, or patterns on a frog? He looked at that from a mathematical perspective.
There's like many... Mathematology, look at the mathematical perspective.
That's quite something. Yes, indeed. Lots and lots of stuff.
He was a mathematician, very well -known. Of course, he killed himself, cyanide poisoning, in the 50s because he was convicted of being homosexual.
That's kind of his story, in a very short version of it.
Of course. You had a role, in a sense, a few years ago, more than 10 years ago now.
That's a long time ago now. In terms of doing a petition related to this, right?
I'll tell you what I thought. Not the pardon.
I wasn't involved in the pardon. I thought that, although Turing was well-known among mathematicians and computer scientists, I thought that it would be, if Britain apologized for the treatment of him and, by extension, other gay men, two things would happen.
One is people would hear about Turing. By having spoken about this sad situation, we would then move on from it and talk about all the other things he did.
And so that was in 2009. What became a blog post became a petition, which actually became an apology by the British government.
Gordon Brown, there it is.
Gordon Brown was Prime Minister at the time. Yes, absolutely. But in a sense, it was also to emphasize his role in terms of computing.
No, I thought so.
I always felt that he didn't get talked about much in public sense, because there was this very sad end to his life.
And as a British person, we're kind of bad at speaking about things.
We don't like to talk about uncomfortable subjects. So I was like, well, if we talk about the uncomfortable subject one time, everybody, then it will help us move on and we'll be able to speak about everything else he did with a sort of clear head.
So that was the idea of the apology. And some other people extended that into a pardon.
And I think the big thing for me was after the apology, the government under David Cameron actually introduced a law which got rid of all of the convictions that gay men had had in the UK, because there were many there were men alive convicted under these laws who were criminal conviction, which was quite extraordinary.
And so now there's, yes, this isn't the Alan Turing law, which actually, in the end, pardoned all of the men who had had convictions for being homosexual.
And there's also the Turing Award that permeates... The Turing Award is ancient.
You see, this Turing Award is a good example of how the ACM had the Turing Award since the 60s.
Exactly, 66. And because within the computer science community, Turing was very, well known as one of the pioneers.
And we've been giving that award for a long time.
And my goal with the petition was really to make sure that everyone else realized that there was somebody exceedingly bright who had contributed in a number of areas and not just the Second World War.
And it became like a Nobel Prize for AI, in a sense.
I guess. And recently, there's been discussions about the future role of AI with Yann LeCun.
Geoff Hinton as well. And Yoshua Bengio.
Sadly, the Turing Award winners are an interesting group of people.
Absolutely. That was a bit of history there about Alan Turing, someone you should know.
There's a movie that came out at the time you were doing that request, that petition.
Last but not least, before we go, I just want to show that in last weekend, there was a mutiny with the Wagner Group in Russia.
And Matthew Prince, our CEO, tweeted about it.
A huge uptick in the use of messaging apps inside Russia.
Because mostly, it was communicated by the Wagner Group leader by telegram.
So there was a big increase in traffic, DNS traffic, to those types of websites.
But also Russian news had a big spike during late Friday. And of course, on Saturday, when the mutiny was happening.
Well, just a few trends there. Pretty people in Russia were very keen on getting the news.
Exactly. And already this week, there was actually a disruption to tell an Internet provider, a Russian Internet provider that serves the Russian military, on this Thursday, supposedly related to a cyber attack.
And we also showed a few metrics there. Well, that's it.
See you next week. Good chatting with you. Bye. That's a wrap. Bye-bye. Before we go, it's time for our AroundNet short segment.
And also, I forgot to mention to John that Alan Turing could have been his teacher at Oxford University.
About AroundNet, this week, we're going to travel to London again. So here's from Matt Bullock from our product team.
My name is Matthew Bullock, and I'm product manager for Speed and FL, or Frontline, at Cloudflare in London.
And that's where I'm currently coming from today, the London office.
See Westminster Bridge, Houses of Parliament, and Elizabeth Tower with Big Ben in behind me.
Yeah. And today, I'm just going to highlight some of the blog posts post Speed Week.
So a few blog posts that I wrote in conjunction with a couple of other people.
So we've got Observatory, which is now available in your dashboard to allow you to monitor your website's performance, understand how performing your website is for users, and the recommendations of Cloudflare products you can enable to improve that.
We also talked about Snippets Alpha and how we've built that.
And that's actually the reason I'm in the office today, just to work with the engineering team to go over some of the finer details before we release that to our alpha testers in the very near future.
And then also about Brotli and how we now, or very shortly, are going to support end -to-end Brotli in the coming weeks.
So you can reduce bandwidth from your origin, improve compression, which again, reduces bytes.
So allowing you to serve content to your users a whole lot, yeah, a lot quicker, which is really exciting and a feature that's been requested by customers in our community for a long, long, long time.
Fun fact from me originally from a place called Ashby de la Zouche, which sounds very posh, but it's actually just in the middle of England, very small village, well, town, but it's almost a village.
And moved to London around 10 years ago, been at Cloudflare coming up to seven.
And yeah, sort of just getting to work with a load of smart, talented people building some great products and helping making your sites as fast as possible.
So any sort of feedback, please let us know in the Cloudflare community about Speedweek.
Happy to read it and to, yeah, get feature requests and sort of what you want us to work on next.