🎂 Introducing Speed Brain: helping web pages load 45% faster

Hello, everybody, and welcome to Cloudflare TV and a special session during Birthday Week.

My name is Alex. I'm a product manager at Cloudflare. I'm joined today by William.

I'll be joined by Alex.

I'm an engineering manager at Cloudflare, looking after the speed team and the rule sets team.

And we're here today to talk to you about an exciting announcement that we're making today called Speedbrain.

And the blog, I think, just probably went live a few minutes ago, which is exciting.

So we encourage everybody to read that.

But maybe we start off with a really big, wide view here.

And I will maybe ask the first question to William, but we'll sort of go back and forth here.

William, what is Speedbrain? So Speedbrain really is a way of speeding up web pages on browsers for users who are browsing the Internet.

With Speedbrain, we essentially try to make sure the next page that you're going to navigate to is already downloaded and ready to be viewed on your browser before you actually navigate to it.

So it's this sort of predicting the future product, which allows us to speculate which pages you're going to navigate to, load them into your browser, and then by the time you navigate to them, it's an instant navigation and you can browse the web much more quickly.

It sounds fantastic. That sounds awesome.

I'm quite interested in the predicting the future aspect of that. Maybe could you talk to me a little bit more about that piece?

How do we make sure that we're predicting the right future?

Yeah, this is where it gets really interesting with Speedbrain.

We obviously don't know exactly where users will be navigating to, but we can leverage a very interesting feature from the Chrome browser family, and that feature is called Speculation Rules.

And it essentially allows us to gather data from the browser in order to predict where a user might navigate to next.

So if you imagine you're on the products, imagine you're on a homepage of a website and there's a big button in the middle that says, go see the products.

If you hover over that link, you're probably quite likely to make that transition to that next page.

So the browser could say, hey, why don't we load up this page and we can navigate to that one.

There is obviously a chance, as you said, that the user won't navigate to that page, they'll hover over a button and then they'll go elsewhere and navigate to some other page, in which case we might be predicting the wrong future.

The way this works for Speedbrain in our initial release is we're actually waiting for people to click down, mouse down onto specific links.

And at that point, rather than the point when they actually release that click, we start loading pages into the browser.

And PowerFlare's network facilitates this process by integrating seamlessly with this Speculation Rules Chrome API so that PowerFlare customers don't even have to do any setup for this.

They can just click a toggle and all of their pages will be rendered slightly faster.

That's incredible. That's awesome. It sounds like there's a balance sort of what you're saying between trying to make the page fast and trying to make sure that we're not wasting a bunch of resources.

And so there's almost like a speed because you have to make that decision on the fly in that sort of hot path, in the path that the actual data and the request is being transited through.

And sort of a correctness angle as well, in terms of like, hey, we can't be super, super, super aggressive in prefetching because we're just going to be doing a bunch of work and not having or showing any benefit.

How do you think about balancing that speed versus correctness sort of angle?

Yeah, it's a really interesting topic.

If we go to the far end of the spectrum, you could imagine that a user navigates to a website, a web page, and there are multiple links on that web page.

And the browser could just load all of the pages from all of those links.

Let's say there's 20 links on that web page. We could just load 20 HTML pages ready to go.

And any navigation the user makes will be there waiting for them.

But as you say, this is going to initiate 20 requests for resources, it's going to hog the bandwidth, it's going to clog the pipes.

The far other end of the spectrum is what the web is like at the moment where we actually wait for a user to initiate that navigation in order for us to download the next page.

So what we're doing is really just moving just a little bit from this place, the web is right now towards that other, slightly crazy other end of the spectrum, where we'd be, as you say, wasting a lot of resources, because 19 out of 20 pages wouldn't be used in any navigation from that web page.

Yeah, I mean, I think that walking that balance is really, really important and really difficult to get right.

And I'm glad that we're thinking about it and that we're starting here and if things are looking good, we can sort of continue to move towards that more predictive future.

And I think that's the goal here, because the more predictive we get and the more right that we get, I think, in that predictive sort of endeavor, the more instant the web becomes.

And that's really exciting. Sort of turning back to sort of the work here, what sort of challenges did we face, did the team face in terms of building this out?

And to be clear here, this is on by default for millions and millions of zones across Cloudflare.

So what sort of challenges did we encounter when we're trying to bring this technology to this large group of websites that maybe didn't even know it existed before?

Yeah, as you said, we've rolled this out as enabled for millions of websites.

Really, we were erring on the side of caution.

We wanted to make an incremental improvement to a large number of websites rather than a large amount of improvement to a small number of websites.

So we've erred on the side of caution and we've tried to make this as safe and almost, you might use the word conservative as well, like as conservative as possible, which want that incremental performance gain for a very, very large number of websites.

So I think in terms of our engineering discussions, we were considering how could we how could we make sure this was robust and safe and testable?

But also, we've been thinking about in the future, how could we try and find more performance gains for some of those sites that could benefit from it?

So at the moment, we've done sort of the lowest common denominator across as many sites as possible.

I think looking forward, the engineering decisions will be about how can we create more of a hybrid solution, which has this baseline approach for as many sites as possible, but also some more effective and maybe slightly more, a bit more permissive or a bit more liberal structure around how we prefetch all this content.

Yeah, that's and maybe it's maybe a very out of order question and I'm sad that I didn't ask it earlier.

But two sort of words that we've been throwing around here are a lot in terms of the underlying functionality of Speedbrain are prefetching and I believe we've also mentioned pre-rendering as well.

What for people maybe that aren't aware how browsers work maybe, how do how does prefetching and pre-rendering sort of make things faster?

Sure, so if you're on a web page and you navigate to another page, we can call this fetching, you're fetching the next page and then that page is fetched and then you view that page in your browser.

So what's prefetching? Prefetching is when we fetch that page before you've navigated to it.

And when I use the word page, what I really mean when I say prefetching is just fetching the html document.

So we're just fetching the actual raw html which is what web pages are made from.

Rendering is the process the browser goes through where it takes that html document and then it grabs all of the images that might be in it.

It grabs all of the javascript that's bringing the page to life.

It brings all the css which is the styling which can add colors and fonts etc to your page and then it renders it.

It brings all of those different document types together and renders it into your browser.

So prefetching is just fetching that html before you've navigated to the next page.

Prerendering is the process of actually rendering that entire page before you've navigated to it.

So it would actually render with the images. Obviously you can't see it, it would be rendering in the background of your browser.

But what that means is when you make the navigation you could then see that page fully rendered.

The browser wouldn't take any time to even render that page.

It's worth mentioning that for Speedbrain we have not offered prerendering for our first release.

We're just interested in prefetching at this point.

But it's certainly a potential we could look at in the future with how we could offer prerendering as a solution to some of our customers.

Yeah so I mean that's such an interesting piece to throw in there because it's not only that as an engineering team building Speedbrain that you have to think about how the data is flowing between a browser client and the Cloudflare's network and servers sort of scattered around and distributed around the world.

But you also then have to think about that timing aspect, making sure that you're prefetching the right thing so that there is a performance improvement and sort of that browser semantics as well.

That's you know lots of different I think things to think about and lots of different engineering to sort of do in the right way here which is fascinating.

Yeah looking and maybe one thing that you sort of alluded to in your answer here is like looking towards the future of Speedbrain.

I know that today we're announcing a very conservative model, something that is prefetching content when somebody mouses down on a link and it's for cashable content in the same page.

And I think a lot of that decision is for safety that we've said previously, you know making sure that traffic to people's origins doesn't explode to an unwieldy amount.

But sort of looking towards the future here, what sorts of things have we been exploring as like next steps in the Speedbrain journey?

Yeah so there's two different ways that we can predict what the next page is going to be.

One is that we look at the user's behavior on a web page and they can move their mouse around and we can and see whether they click down on a link, see where they hover over a link and we can react from the user's behavior on the web page.

But there's another way that you can instruct the prefetch or speculate on pages and that is by essentially explicitly saying to the browser these could be the next pages that the user might navigate to.

So the server would actually tell the browser these are the pages that might be navigated to next.

One thing we're looking into is could we with Speedbrain, could we actually use some sort of machine learning to create that list of next pages that the browser should prefetch.

So we are currently working with a research team internally at Cloudflare to understand if there's any opportunities to make more informed, more smart prefetch for next pages within Smartbrain by using some sort of ML or AI based approach.

Yeah that's I mean I think that piece is really really exciting. I know that that was sort of as we were exploring opportunities with prefetching and pre-rendering that was sort of the idea from the beginning and something that we had talked about over the summer at various developer conferences is how good that model was performing.

I think initially we were testing it on Cloudflare owned websites like our developer documentation and other things that had a lot of links that you could click and that you can go to from sort of the same website and they're all pretty static content that we were delivering and so trying to guess or have an ML algorithm sort of a guess of any individual user's navigation path through that and making sure that it was prefetching the things in a safe way was exciting to see and I think the model if I'm not mistaken was pretty good.

It was in the high 90% in terms of accuracy being able to understand where somebody was going to go next and being able to grab that content and I forget sort of the the LCP improvement numbers for that the performance improvement numbers in the browser but I think they were maybe like 40 or 50 percent something like that which is you know an incredible number.

Yeah absolutely so there's definitely a lot of potential there.

I think one of the challenges is that every single website is organized differently in terms of their URL structures and how people navigate between those pages so there isn't a consistency that we can lean on that we can benefit from by saying oh if people are navigating this website in this way they're likely to navigate these other websites in a similar way.

There might be a little bit of that with content management systems but really every website has the option to name its URLs, its paths in different ways so I think something we'll have to work on is how do we build an ML model that works great for one site which is what we've proven so far but that also can be replicated across millions of other sites and offer those same performance gains.

Yeah and all with zero configuration, zero input from the customer and I think that's really big and something that we've seen with other products that we've worked on.

For example early hints for example requires sort of a consistent updating on your server of what should be hinted what's safe to be hinted and what's effective to be hinted and I think that I see that as one of the really main benefits of Speedbrain here which is just the ability to push a button and it sort of makes the right decisions for you and it you know if once we get sort of the ML piece integrated the Speedbrain will hopefully start you know learning over time and understanding patterns about what is safe and what's the most effective thing to pre-fetch at various eagerness levels and you know can eventually start dipping its toes into pre-rendering as well which will be exciting.

Yeah absolutely. Awesome. Any final thoughts William on Speedbrain or its development?

Oh actually we should talk about the ROM integration.

Oh yes that is a good point that we forgot.

I will maybe to make it easier for the editor we'll get back over yes.

Okay William any final thoughts here going into you know the announcement of Speedbrain?

I think one thing that I'm really excited about is the tight integration we have between Speedbrain and the ROM and web analytics that we offer at Cloudflare.

It's one thing to make your website faster but you really want to know how much faster and what I think sets Speedbrain apart is that we can enable it with just one click of a toggle but we can then also start to show you how it's affecting the performance of your website in our analytics dashboards inside Cloudflare.

So I think for me that's the most exciting thing is developing our customers understanding of how this product is affecting the speed of their web pages.

They start to see it in their metrics, in their dashboards, in their analytics and then they can start to build this virtuous cycle where they can say okay this product is improved by this much what about if I enable this product how will that affect my performance?

So I think that's something that is worth calling out at this point.

Yeah that's I think a really good point and something that I think we generally recommend sort of across all products on Cloudflare is to enable the web analytics feature to help show you how Cloudflare is helping your end users.

It takes real user measurements and it displays those back in sort of an aggregated way to you and so you can see like hey I know that I turned on Speedbrain at this point and I can see how my metrics are reacting or I turned on a different Cloudflare feature early hints or Argo or something else we've talked about today and I can see the impact that it's having on my metrics and my users and I think that point is really important and really well said.

You can't get faster it can't get better unless you're measuring something and so having you know consistent measurements is incredibly important so I think that's a really good point and something that I'm also glad that we're thinking about with you know that tight integration with Speedbrain and so yeah definitely recommend people turn that on.

Well I think that will probably do it for William and I today. We're really excited to hear your feedback on Speedbrain.

Definitely let us know in the community or on Twitter if you have feedback and you know we're happy to keep building here.

I know that there's a lot more announcements for birthday week so please stay tuned and watch those as well but from William and I wish you the best day and see you around.