Reducing payloads with shared compression dictionaries

Hi. Welcome. How's everybody's Connect going? Good? Yes? Thumbs up all around? Sweet. Cool. Well, welcome to reducing the payloads with shared compression dictionaries. My name is Alex Krivit. I work in product at Cloudflare, joined by Tim Kadlec, who also works in product at Cloudflare. We're really excited to talk to you about how we can do this, some really new and exciting protocols that are coming into vogue right now. All right. So as a good product manager, we're going to start with the problem here. So your application on one side maybe has 272. I forgot the number I wrote. Yes, 272 KB. It's massive. But the good thing about it is that everybody's already downloaded that version of it already. Sometime later, maybe something happens. You have to make a one-line code fix somewhere, fixes a bug, fixes something. And you have to go back, make that change in the JavaScript. That's fine. It's on the wire, compressed in gzip, 92 kilobytes down to the browser. It's cached. But because it's that one-line change, the browser sort of has this amnesia. It's a bit like Jason Bourne waking up, and you're like, oh, my gosh, who am I again? It has to do that every single time. Every single time you make that one-line change, which is bad because with AI, with deployments, how many times a day are we deploying now? Many, many times a day. And if the browser wakes up as Jason Bourne each one of those times, you're sending that 92 KB down the wire a lot. You're compressing a lot. This is a problem. So we're going to talk about how we can help the browser. Remember that it already has most of that file in the browser already. And that's the big hack here. So with that, Tim is going to go into some of the basics of compression now. Yeah, so to understand why compression doesn't actually fix this, we do have to understand a little bit about how it works. So basically, every format of compression that we use since 1977 has been using the same optimization. Before that, it was basic just entropy encoding. The idea that like, OK, E is more common than X. The is more common than xylophone. And so we just kind of replace those common words with other tokens and things like that to try to condense. But in 1977, L77 comes along. And the idea here is we now have a sliding window where we're able to look back at the text that we've already seen within a particular chunk of bytes. And we're able to replace that with pointers. So we could say, for example, go back 46 characters and then select nine of them. And then instead of passing the full characters along, we just pass those pointers. There's still an entropy encoding phase alongside of that, where we go through and we find the most common patterns of those pointers. And then we replace those with tokens as well to further condense. But the idea is we've always had this sliding window, again, since 1977. And the way you can think about that is it's like, if I were to say to Alex, do you remember that joke from this morning? Alex is probably going to say yes. If I were to say, do you remember the joke that I told you the other week? There's way too many and everything like that. I'm way too funny of a guy. He's not going to be able to tell you which one it is. Because again, we've gone beyond that sliding window. Now the formats, the optimizations up to this point mostly have been about how far back can it look, like how big is that window? So GZIP can remember up to 32 kilobytes. Brotli can go up to 16 megabytes. Now in theory, that's a really big window. In reality, it's just worth noting that when we inflate that window, we incur a lot of processing and overhead and things like that. So there are trade-offs that come with it. Getting the hang of the clicker. Pressing the wrong button. There we go. Now the problem with this is we're still isolated to the individual requests that we're looking at. Compression has this concept of basically amnesia. As soon as we go back to another response, we go to another request, we have to start fresh again. There's no maintained context between version 1 of your JavaScript file and version 2. A dictionary actually helps to fix that. What a dictionary does is it brings bytes to both sides that we've already seen, and we can use that to compress. So now we're not just using the sliding window context and basing it on what have we seen within this individual response, but we can actually get pretty precise with this. So Brotli ships a generic dictionary for everyone. Again, an improvement of it in and of itself. Brotli between this and the context window will actually be much more efficient in terms of compression in most cases than gzip. ZStandard lets you bring your own dictionary, and this is where the magic actually happens. Because the best dictionary is the file the browser has already seen. When we use that previous build, we can say, OK, browser, you've seen this. Server, you've seen this. We've both seen v1. v2 comes along. We change one line of code. But because we've seen everything else, we can use that as a dictionary, and suddenly everything that we pass through, any unchanged code, just becomes little pointers. And the only thing that actually has to be passed over the network is the delta. So when we do this in a lab environment and we test this, we saw a 272 kilobyte JavaScript bundle gets reduced to 92 kilobytes with gzip. And then once we've got this shared dictionary of the prior version of the library that we passed along, it goes all the way down to 2.6 kilobytes. So a 97% reduction over gzip alone because we're able to use that custom dictionary. Now, the way this works in the browser is a combination of basically we have three headers and two encodings. So again, there's some communication that has to happen. I've been trying to talk Alex into getting matching decoder rings because I'd like to be able to communicate them in a way where the rest of the team doesn't know what we're talking about. It's on the shopping list, I believe, yes. But you can think about it this way. We have to be using the same one or else this is not going to work. And so what happens is the first time, the server is actually going to send back and say, hey, use this dictionary. And then it provides a pattern that you can match this against. And what it's telling the browser is you can take this resource that I've sent you. I'm going to let you use it as a dictionary for further requests. And it's going to match against anything that matches that pattern. So in this case, JavaScript slash app. And then the asterisk is our wildcard .js. So now let's say that we do a new deploy. We get a new version. A new version of the JavaScript comes on. We've made that one line change or whatever it is. The browser says, OK, cool. I've got to make a request for this resource. Actually, I have a dictionary that I've stored as a SHA-256 hash. I have a dictionary for this thing. It matches this pattern. And so it comes back and it says to the server, hey, I'm requesting this resource. I need the new JavaScript. But it passes along that SHA-256 hash and says, I have a dictionary available. Then the server can respond. And it also says the DCB and DCZ is basically just saying, I know how to speak this. I know how to decode that. And then the server comes back and it says, well, cool. You have the dictionary. It's a valid dictionary. You know how to decode this. It's going to be safe to do this. And so it can just pass back that small, hyper-truncated version. And the browser is going to be able to figure it out from there. It also passes back a very header that tells every cache along the way that this is dependent on that dictionary. So it's a very simple mechanism, really. It's just like, again, let them know that this is the dictionary. Browser says, cool, I've got it. And I can speak the language. Server says, cool, we can talk now. Now, the big difference here is this is not the first time we've tried to do something like this. And I see a few head nods. Chrome did try some concept of shared dictionaries before. SDCH, which I believe is pronounced sandwich, which is fun, I guess, from a food perspective. But it was one browser that did this. There was no standard around it. There was no good answer to a lot of the actual issues that we had, particularly from a security perspective, things like compression attacks. So it didn't get much traction, despite being around for, like, what, nine years or so. The difference here is that this is RFC 9842. This is a standard. It is built around the browser security model. That means it's same origin only. It's HTTPS only. It's hash checked. It's cleared with your cookies. There's a lot more built into the browser to make sure that this is safe to do and efficient to do. As a result, we see about 72% of global browser usage already supports it. Chrome, Edge, other Chromium browsers, basically since Chrome 130. Firefox has it available. And nightly, Safari is working on it. So we would expect that browser usage is going to go up pretty soon. We also see that one in every 10 Chrome page loads on the web have a dictionary already available. Now, this is kind of inflated. You've got that head of the web versus the tail of the web. And the head is comprised of a lot of very popular sites. And so a lot of those popular sites are the ones using it. But again, this is not like something that doesn't exist. This is something that we're already seeing out being used by some of these larger sites on the web. Google Search is one of them. So Google Search has actually experimented with this for their search result pages. And what they've seen is when they have this dictionary available, it's a 23% smaller HTML payload that they're able to pass over the network. And that has a trickle down effect. So largest contentful paint, one of the most important user facing performance metrics that we have on the web. If you're not familiar, it's a metric that tells us when is the largest piece of content on a page. When does that get displayed? So that could be your image, your text, whatever it is. When they ran this same experiment, again, they saw that 23% improvement over Brotli alone. But there was a trickle down effect where LCP improved by 1.7% and up to 9% on high latency networks, which again, makes a lot of sense. Right? If we're suddenly passing back 2.6 kilobytes instead of 92, we can fit more in a single round trip. We're probably having to incur a lot less of that latency overhead. And again, you would expect that the impact here would be even more beneficial depending on how much that initial bundle size was compared to if we can get it down to one single round trip because it's small enough to fit in that size. So it's very impactful. And it's exciting, honestly, that this is here and that this is being actually supported and worked on. But if it's so great and it's so beneficial and it's so impactful and all it is is a couple of headers, the question becomes, why are we not seeing this more widespread? Why are we not doing this at the CDN level? Alex, why is this hard? It's hard for a number of reasons. Good, fantastic transition. OK, so it's hard. Why is it hard for CDN? Don't really want to bring this down into a woe is us because we are actively working on remediating these things. We're going to get to later. But these are some of the foot guns and things that we've seen and run into as we're actively developing this. First thing is that it's quite expensive for cache. You have to not only store variants of the same asset, but you also have to store versions of that asset so that you have something to do the delta compression against. So you have multiple dimensions that you need to store in cache just in case you need it. You need to pull that diff down. There's a second naming piece that's a little bit difficult. As Tim was saying, browsers oftentimes say and describe the dictionaries that they have by a hash. Whereas if you're talking to a cache, realizing that those words are quite similar when I'm saying them out loud, we oftentimes use URIs, URLs, to find those things. They're parts of cache keys. So that can be quite hard where you're trying to translate various hashes into cache keys. How do those things operate? Very hard problem for resource location and understanding there is a dictionary available. What I was saying as a piece of the first thing where your cache explodes is that you need to keep a lot of your old entries. If a browser comes back maybe after a month or two months and they're saying, hey, I have version 6 of this and we're on version 10 of this and we're trying to pull a delta compression against that, we need to make sure that those things are compatible, that we can catch those up. That may also mean, again, that we have to store that version longer. So we have something to delta off of to catch them up to that latest version, but is sort of the corollary to CN problems. Everything, obviously, is costing so much CPU these days. If we're doing this delta compression for people building these dictionaries, for people it takes CPU on every single side of that. The last two, for time, will sort of mash together in so far as that the bytes have to match exactly. And breaking that page, trying to send them something that they're unable to decode is quite difficult because you're breaking things from a CDN perspective that can be a silent break, or it's a break that needs to have a run beacon fired to tell us that something is broken. So lots of things to consider to make one of those things a little bit more clear and transparent is that, yes, one URL can have many answers dependent on who is coming and asking for that, and where we need to catch them up in terms of both what the encoding is and which version that they can go to. In this situation, this version 9 app that everybody is getting can have many, many different answers that you might not expect. So I think there's 12 or 16 that I put in there. But yeah, you can see how there is that major CAS explosion. And again, the problem at the bottom is that if you're serving the wrong variant that the browser cannot decode, you're breaking pages. That is not great. So how are we going to do this at Cloudflare? The first version that we shipped back in April, I believe, end of April, we call pass-through mode. And pass-through mode is essentially saying, OK, Cloudflare is going to allow this to happen. But insofar as it happens on your origin, you must do the delta. We can cache things at the edge. Send us a available dictionary. And if we get some DCZ back from your origin, those things can get cached and be returned as hits in very particular circumstances. But a lot of the work in terms of actually building those dictionaries and sending them through needs to happen at your origin. You have to bring your own dictionaries. We don't touch anything. We're just caching things, sort of a dumb pipe in this situation. And we're going to cache every single variant. So we have that explosion problem that we've been talking about, which is why this is hard. If we sort of look forward and making some forward-looking statements now, a thing that we are currently building here is called manage mode, which is, how do we take that load off of your origin and do it for you? So this makes it such that a lot of things that you would maybe do at your origin sort of come into Cloudflare. We're solving a lot of the cache key and mesh problems by appending some responses with an ID so that we can look up those hashes and cache keys and find where in the cache they are, build those deltas off of them to send back to the browsers when they're available. That helps the problem of pointing where in the cache we need to look for that dictionary conversion so that we can build that delta. And yeah, we built the delta once, which is fantastic. We don't have to store these widespread amount of variants. And so the ID is sort of the big innovation, I think, that we're looking towards here because it helps us find it in cache. The delta once problem is incredible because that explosion isn't there. We verify everything is in cache before we go to your origin, which should reduce load and CPU on customer origin side, which is fantastic. And if there is no match, if we can't delta, if we can't find things, we're going to fail open. We're going to push back down to classical or, I think, what I call normal compression in this situation. And you can sort of look to see a very demo-y version of the delta compression today and this manage mode on canicompress.com. It should sort of walk you through the process, how all of this works, if you're trying to get a better feel of it. And yeah, lots of JavaScript bytes that you can see deployed and reduced, which is fantastic. And you can do it today on your any Chromium or Edge-based browser. So things you can do when you get back to your laptop, turn on pass-through mode if you want to play with it. Obviously, your origin needs to support it. If you want to wait for the manage mode, that should be shipping soon. I was told I cannot give dates, so that is going to be soon. Make sure you're marking your dictionaries. Obviously, make sure that you understand the RFC, your origin is cooperating, our cache is cooperating, test things, very important. And then, yes, manage mode on the way. I think these slides will be available. We tried to help with some links at the bottom that will be beneficial for you as you're investigating and looking into this. I'm seeing people take pictures, so I'm going to stall for a little while. But thank you guys so much for coming to our talk. I know it's late in the day on the second day, so really cool to see everybody here, and thank you. Thanks. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you.