💻 Moving Pictures: A Look Into the Future of Image Formats for Jamstack

Do not adjust your set. Good afternoon, Cloudflare TV. This is Cloudflare TV and not BET. But I do hope I'm going to be just as entertaining. So let's go. We're here to talk about images. It's actually one of my favorite topics. So I hope you'll join me as we sort of discuss how we're here to sort of manage them because they are a bit of a tough management. What do we want out of images? We want high fidelity, we want low weight, we want small sizes. So it's actually not the easiest endeavor. Now, I'm going to stop quickly and talk about the fact that today is April 13th. 4.13, correct? Yeah. And for all you sort of nerds out there, I know Cloudflare, you have a bunch of erudites, a bunch of enlightened engineers, HTTP status code 4.13, let me know what it says. I'm just going to wink right now. So how did we get here? Well, let's think about it. Well, I'm going to actually point to two people in particular. First person I'm going to point to is a gentleman by the name of Steve Sassen. Now, Steve Sassen is the inventor of the digital camera. Now, he's holding the very first digital camera here in all its 3.6 kilograms of glory, 0.01 megapixel, and actually recorded the image onto a cassette that you can see right there. It actually took 23 seconds to record the image. I'm actually also going to point the finger to Mark Andreessen. Yes, that Mark Andreessen. In 1993, he wrote this email. It read just like this. I'd like to propose a new optional HTML tag, IMG. And that was February 25th, 1993. Now, what he believed is that bringing images onto the net, onto the web would promote experimentation. And you know what? He did get a little bit of pushback because some believe, actually, Tim Berners-Lee believed that it would lead to some potentially unsavory content. But beyond that, experimentation went on. And in 1995, HTML 2.0 was introduced. And with it came the IMG tag, the image tag. So you know what that means? It means that the image tag turned 25 last fall. So my name is Henri. You can find me on Instagram at Henri Helvedica. I'm from the greatest city on the planet, which is Toronto, Canada. Toronto, which is the home or where Commodore was founded. That's correct. Commodore 64 was founded in Toronto. You are very welcome. And when I'm not sort of making these presentations or doing my little thing, I like to sort of relax, take care of my mental health and physical being. And I share my runs on using this hashtag, devs who run. I'm a distance runner. Yes, I am. So I'd like to welcome everyone to my presentation today. It's Cloudflare's developer week. This is called moving pictures. A look into the future of image formats for the JAMstack. And I mean, I am going to talk about the JAMstack for a number of reasons. But you'll see what I'm talking about in a hot minute. Now, you'll be able to take a look at these slides. If you go to bit.ly moving pictures underscore Cloudflare, you'll be able to download those slides right now. It's in a PDF. It's about 13 megs. So I have to apologize ahead of time. So this is where the story starts. We're going to talk about images. But when I talk about images, I like to do a bit of a retrospective to have an understanding of where we were or where we are right now and where we're about to go. So let's start. Let's start with the classic graphics interchange format. Yes, the GIF. Lossless in nature. So out the gate, it was already big. There was some transparency. GIF 87, GIF 89A, or pardon me, GIF 89A is where we got that transparency. We also got the animation, which is what is really keeping this format around today. 256 colors. No big deal. But you know what? It came up in a different era. It's a little old. Following that, we're going to talk about the portable network graphics. Now, there's a big story here. I'm not going to get into all the details. But once again, we're looking at a lossless format. And it came in two well-known flavors, PNG 8, PNG 24. Mostly both very big. We also had an alpha channel this time around, which was also awesome. Believe it or not, you could actually get some animation out of the PNG. I do not recommend it. It's grossly overweight for an image format. But this format's also a little aged. I believe it's about 24 years old. Now, let's talk about the scalable vector graphic. Now, this is out of the four that I'm going to talk about. This is the only one that is not a raster format. Text-based, it can be compressed beautifully, whether you're g -zipping or broadly. Scalable, great for some graphics, logos and whatnot. But believe it or not, the SVG can get out of hand at times. You can get some transparency out of it. And it's turning 20 this year. Now, the last of the four or the quartet that I'm going to talk about is this one right here, the Joint Photographic Experts Group, better known as the JPEG. Well-known as the format that is pretty much in greatest amount on the web. It's the one that people use mostly for photography because that's what it's good at. But the best part, it's a lossy format. So this is why it was likely this popular on the web. Believe it or not, you can get an alpha channel out of it if you want, depending on that flavor, whether you go JPEG XR or JPEG 2000. And you know what? This format is also a little long almost 30 years old. So what does this mean? Well, we've had these four formats and they've been around for a long time. In fact, the rumor is that Methuselah used to use them in high school. Like they've been around that long. But all jokes aside, combined, they are over 100 years old. Actually, I think it's 104 to be precise. It is time to move on to some newer formats because we have concerns when it comes to these older formats. Let's go through them. So our concerns, we're going to listen like this, codecs, congestion, and costs. And let's start from right at the start, codecs. Now, I love this quote from Colt McCandless at Google. The foundation of the whole Internet is data compression. Nothing happens online without compression. Just keep that in mind. So I like to look at this photo as well because to me, it defines everything that we're trying to do today. Compression is a kind of a delicate mix of art and science. If you have too much art, you end up with a beautiful photo, full fidelity, and probably not well compressed. So that's not good. It's going to come in too big. If you have too much science, you end up with something like this. Compressed, colors are washed out, and it's full of artifacts. So the perfect codec, the perfect compression is going to be a bit of an amalgam between art and science. Now, when we talk about congestion, I really mean bandwidth. And bandwidth is absolutely important. And it's never been more important than today, in fact. For the last year and change, we've unfortunately been the victim of a pandemic. And what has happened, you may remember sometime last year, the European Union had put out some warnings and asked some companies to sort of do a bit of throttling because they were very concerned. Pardon me. They're very concerned. That was a fruit fly. I don't know what's going on now. They're very concerned with the fact that everyone is going to be at home. And why? Because we are suddenly all going to work from home. We're all going to learn from home. Shout out to their parents out there. And we're all going to conference from home. That was going to be, they believe, fairly taxing on the wire. And what better time to start to look at new formats when we have these kind of conditions. Now, let's look at some quick data. And this is coming from the HTTP archive, a fantastic treasure trove of data of how the web was built. Now, you can see right here, the total image bytes per web page, they're pretty much trending upwards and have been for the last several years. They are getting large. Now, if I look at some data, again, from the HTTP archive, and this is the total image request per web page, these are trending downwards. Now, normally, that would be great news. But once you combine the two and you do the math, we know that we now know that images are basically getting bigger and bigger. And again, this is adding to that congestion that I talked about. Now, the last thing I want to talk about are costs, patents, and royalties. Now, we know that when it comes to being online and, you know, shuffling in large images up and down the wire, there are several costs involved. We know that, you know, no one likes overages, you know, when you get that bill. But the one thing that people do not realize or do not think about a lot are patents and royalties. Now, this may ring a bell when I bring this up, HEAF. I thought it was a fantastic format when Apple kind of introduced it to the world. They're like, hey, here's what's going on. We're going to use this new format now. But guess what? It never made it onto the web. Why? Because patents and royalties, it was just a complete nightmare. So it was ultimately not really usable, even though it was a fantastic format. And if we take it back to history, some of you may remember that the PNG came about because of a patent issue with the GIF. This is from 1995. You'll be able to read it. I think I left the link. Anyways, I can get you the link on Twitter if you want. Anyhow, this is all to remind us that there are some costs associated with some of these formats, and we're trying to manage that. Now, we've looked at the JPEG, the PNG, the SVG, and the GIF. But it's really time now to start talking about the newer formats. Let's talk about the future. Now, modern development is essentially what we're going to evangelize, the future of development. And the future of modern development is going to come as well with a future in modern formats. Now, seeing that the images are the most abundant resources online, they can provide the most amount savings. We should agree to that. And since the JPEG is the most abundant format, it will provide the most abundant savings as well. I think we can agree to that as well. So, I'd like to introduce everyone to the JPEG Excel. You might be thinking, what is this all about? Well, let's find out. The JPEG Excel came about on the 25th anniversary of the JPEG, in fact. There was a call for papers, seven submissions, two were selected, and the two were pick a project that Google is working on to replace the JPEG and Fleef, which is the free universal image format that was created by the good people at Cloudinary. So, they combined forces and created the JPEG Excel. Now, why is the JPEG Excel going to be a great format? Well, alpha channel, we love that. Lossless, we obviously love that because we get fidelity. Animation, we can't do without that these days. Modern compression, obviously, high bit depth, legacy friendly, backwards compatible is what that meant. Universal, responsive because responsive images are important, and royalty free, very important. Now, earlier this year, great news, the bit stream was frozen, which means at this point that software applications are now free to use the format or at least to incorporate it into their applications. Major milestone, as you said. But unfortunately, we do have some challenges as of right now. The JPEG Excel support is not the greatest, but just announced a few days ago, even, maybe weeks, but we do have some breaking news, and Chrome actually added decoding support to the browser. Now, it's going to be behind a flag. They're hoping for sometime the next few releases, but this is actually great news because they're at least trying to experiment with the format. Let's talk about this next format here, AVIF, A-V-I-F. Now, this is a format that came about from the AV1 video format that was put together by the Alliance for Open Media, a bit of a conglomerate of companies that are trying to create royalty-free formats. Now, what do we have here in terms of details? Now, like I said, the AV1 file format, which is the AVIF. The bit stream was frozen in February 2019, so fairly recently, but what do we get out of it? 8, 10, and 12 bit depths? Awesome. Lossy and lossless? Awesome. Modern compression? Of course, a must. Alpha channel, high HDR, animation, chroma subsampling. For those who may not understand what that is, it's essentially when you get to play a bit with color data without, well, it's kind of like trickery. We're trying to remove some color data to drop the size of the image without you noticing, so it's kind of like playing with your human visual system. And again, royalty-free. I put in this little asterisk simply because there are rumors that there are some patent trolls out there trying to cash out, but they're apparently well-equipped for it. Now, in terms of data and results, this is from some Netflix studies, and the AVIF in comparison to WebP JPEG 2000 HEIF still came out on top, on top meaning smaller images. That's why the AVIF is definitely something that's being favored as we speak. Now, guess what? Cloudflare supports AVIF formats. Fantastic. That's good to know, and shout -outs to Cornell Luzinski, who I'm going to talk about a little later as well. Fantastic images engineer. So that could be, you could go check that out on the Cloudflare blog. Now, AVIF support looks like this. Desktop Chrome, check. Opera Chrome, I mean, Opera Desktop, check. Unfortunately, Firefox is behind the flag, and it was delayed. It was supposed to go stable, but the rumor is that it should be this month or next month for stable AVIF. So, keep your eyes peeled on that. Other good news is that Android support is coming this fall, 2021. They're in developer preview as we speak. They're looking to move to beta sometime in the summer, and they hope to have that done, sealed, and delivered by fall of this year. So, that's fantastic news. You could actually read about that on the Android developer blog. You can see details right here. Now, like, drumroll, please. This is amazing, because AVIF support is being experimented with in Safari. Now, a few of you have just spat out your coffee, because you cannot believe this is happening so soon, but they are doing it. And in fact, just a few days ago, they actually updated the libraries to current versions of David and libavif. Now, this is fantastic news. Again, we're not sure what's going to happen, but if it shows up in STP out of nowhere, you know where you heard it first. All right? So, let's keep it moving. Now, let's talk about the WebP. Now, WebP, again, is a format that came from a video format. A still format that came from a video format. So, the WebP. It's been around 10 years, actually. So, I don't want to say it's long in the tooth, but, you know, it's had some experience. At one point, Chrome believed, or the good people at Google believed that it could potentially replace the PNG. I'm not sure about that, pound for pound, but it had an aggressive codec. Animation was possible. In fact, Tumblr uses animated WebP. 8-bit depth, alpha, 420 chroma subsampling, and it was made for the web. And this is something I want to spend a little bit of time on, because 10 years ago, they believed that they needed a new format specifically for the web. To an extent, it is there, but these days, 10 years later, we want a little bit more fidelity. So, 8-bit depth, maybe not everyone's cup of tea. You know how that goes. One more thing. iOS 14, Big Sur, Safari, they now support WebP. That was announced last summer at WWDC. It went stable, I guess, fall of 2020. So, the WebP has glorious support. And in fact, just as a quick example, the good people at Shopify announced that they're going to start serving WebP for their merchants as well. Always good news when that happens. So, at the end of the day, the WebP has definitely established itself with regards to web performance. So, you cannot really talk about web performance without talking about the WebP today. Now, why is this all important as I race through the last few slides? Well, let's talk about that. Well, I talked about concerns, codecs, congestions, costs. Well, let's talk about another C, contentful paint, specifically the largest contentful paint. For those who follow metrics and who are all about or at least like to use Lighthouse, it is 25% of your Lighthouse score. Now, unfortunately, images and poorly managed images tend to hammer your Lighthouse score. Now, at P75 on mobile, the average Lighthouse score, the median Lighthouse score is 2,600, which is milliseconds, 2.6 seconds, which is at the top part of average. On desktop, since it's under 2 ,500, it gets a green check. So, these are things that we want to keep an eye on because improperly sized images are a problem. In fact, 58%, only 58% of images, or we'll say 59, are actually sized properly. So, that means that there's four out of 10 images that are not sized well. So, what does that lead us to? Well, we need solutions. And since we're talking about modern development, there are modern solutions taking place as we speak. Now, since we are talking about the JAMstack, let's talk about some of these companies, these frameworks, these CMSs that deal well in the JAMstack, Gatsby, Next, Contentful, Leventy, and so many more. All these companies are now using modern formats from WebP to WebP and AVIF. It's actually a beautiful thing right now happening. Shout-outs to Knut who spoke earlier, you know, Sanity is here on the list, and so many other great companies. Now, when it comes to tooling, well, we do have some tooling available. Squoosh, ImageMagick, Photoshop. There was a plugin that was designed by Google, and just a couple days ago, someone put an AVIF plugin for Photoshop for Windows only right now, but that is actually fantastic news. And also, you have GIMP that was out in the mix. Now, of course, when it comes to management, image management, some people like to sort of like let that off to third-party companies, you know, they don't want to handle that workload themselves. Well, thank God there are some companies, and companies like Cloudflare. Now, they have a host of services available, or should I say a suite of services available, and when it comes to image optimization, they can get the resizing done because, as I said, that's a major problem online right now. They have Mirage to handle some of the delivery so that, depending on what kind of device you are, because, you know, the ecosystem of devices across the globe swings left and right. So, you'll have the iPhones, like, and you'll go right down to, you know, inexpensive Androids with smaller screens. So, these are things that you want to take into consideration. And obviously, you have services like, or products like Polish, where, you know, you'll remove some of that useless, or I shouldn't say useless, but yeah, maybe I will say useless, useless metadata that you may not need. Some, like, I don't know, geotagging, a service that I'm sure someone like John McAfee could have used 10 years ago, easily. Anyhow, let's get down to some conclusions here. In conclusion, we're here to move the web forward. Yes, we are. So, we have the formats that are going to enable us to move the web forward. WebP, Aviv, JPEG, Excel. We have the tooling that we can use today to move the web forward. We have the services to move the web forward today. So, this is the part of the modern sort of development conversation that I love, because we have the ability to make things a lot better today. Now, let me race through my last few slides. Last fall, I had this series of talks around images. Pretty heavy. I had a good man like Cornell Lezinski come in and sort of share his love for Aviv. So, if you want to see that again, you can actually click on that little link. You'll see there are so many more conversations that took place there as well. I got to give a shout out to Rita Kozloff, who's going to join us for this event called Lazy Load that I'm co -presenting as well. So, if you want to hear about or see a little glimpse about the future of web performance, she's going to give us a little, little something something about that. You can find me at the JAMstack Toronto meetup. You can make a quick announcement. I have this event coming up. Click deployconf.ca, where I'm going to talk about more modern deployment. I want to thank everyone, especially Regan, Jen, Rita, Ms. Blossom, and all the good people at Cloudflare. And I'm out. So, we're betting on the technology for the future, not the technology for the past. So, having a broad network, having global companies now running at full enterprise scale gives us great comfort. It's dead clear that no one is innovating in this space as fast as Cloudflare is. With the help of Cloudflare, we were able to add an extra layer of network security controlled by Allianz, including WAF, DDoS. Cloudflare uses CDN, and so allows us to keep costs under control and caching and improve speed. Cloudflare has been an amazing partner in the privacy front. They've been willing to be extremely transparent about the data that they are collecting and why they're using it, and they've also been willing to throw those logs away. I think one of our favorite features of Cloudflare has been the worker technology. Our origins can go down and things will continue to operate perfectly. I think having that kind of a safety net provided by Cloudflare goes a long ways. We were able to leverage Cloudflare to save about $250,000 within about a day. The cost savings across the board is measurable, it's dramatic, and it's something that actually dwarfs the yearly cost of our service with Cloudflare. It's really amazing to partner with a vendor who's not just providing a great enterprise service, but also helping to move forward the security on the Internet. One of the things we didn't expect to happen is that the majority of traffic coming into our infrastructure would get faster response times, which is incredible. Zendesk just got 50% faster for all of these customers around the world because we migrated to Cloudflare. We chose Cloudflare over other existing technology vendors so we could provide a single standard for our global footprint, ensuring world-class capabilities in bot management and web application firewall to protect our large public-facing digital presence. We ended up building our own fleet of HAProxy servers such that we could easily lose one and then it wouldn't have a massive effect, but it was very hard to manage because we kept adding more and more machines as we grew. With Cloudflare, we were able to just scrap all of that because Cloudflare now sits in front and does all the work for us. Cloudflare helped us to improve the customer satisfaction. It removed the friction with our customer engagement. It's very low maintenance and very cost effective and very easy to deploy and it improves the customer experiences big time. Cloudflare is amazing. Cloudflare is such a relief. It's very easy to use. It's fast. Cloudflare really plays the first level of defense for us. Cloudflare has given us peace of mind. They've got our backs. Cloudflare has been fantastic. I would definitely recommend Cloudflare. Cloudflare is providing an incredible service to the world right now. Cloudflare has helped save lives through Project Fairshot. We will forever be grateful for your participation in getting the vaccine to those who need it most in an elegant, efficient, and ethical manner. Thank you.