Latest from Product and Engineering

Hey Usman, how you doing? I'm good Jen, how are you? Good, I'm Jen Taylor, I'm Chief Product Officer at Cloudflare. And I'm Usman Muzaffar, I'm Head of Engineering at Cloudflare. What do those fancy titles mean, Jen? What does Chief Product Officer do? It means I work with you to help us figure out what we should be building, delivering to our customers at scale. And Head of Engineering, what does that mean? Well, I actually have to, that actually is my job to actually build all this stuff that your team comes up with. Well, we've built some pretty cool stuff lately. Yeah, we have. And so I think it's one of the coolest things about Cloudflare is just the incredible variety of things we're working on. In fact, just keeping track of how much stuff we work on is a challenge in and of itself. And one of the things we do is we send out emails and we send out notes to the whole company saying, this is what we just shipped in the last seven days alone, check it out. And we thought we should quiz each other on what we shipped just in the last seven days and talk about why it's interesting to customers and to users of Cloudflare. Well, one of the ones that I am super excited about is the work that we've done on cache analytics. It's something that we've heard consistently from customers that it's a tool that they would really value. And the feedback on this has been incredibly positive. So but let's back up before we dig into cache analytics. What is a cache? What is a cache? It is not green paper in your wallet. But although there are engineers who have a cool shirt that says cache rules everything around me with cache C-A-C-H-E, which is the cache we're talking about. So a cache is simply your copy of something. So if you ask my phone number and I give you my phone number and you write it down, that's your copy. You have cached a copy of my phone number. And this turns out to be super important in computers all the time because computers are all about getting information. And so the question is, well, where did you get that information? Well, it had to get it from somewhere. A user had to type it in or maybe it made a request over the Internet to fetch information. And even inside a computer, that's basically what it's doing. It's moving information around from one place to another. And so the trick is, how do you make it just in the same way that if you ask for my phone number, that you didn't have to come and ask me again the next time because you have your own copy? How do you make it so you have your own copy of the information? A copy that is nearby and close to you so that you can access it more easily. And right away, that gives you the two big benefits of a cache, which is it's faster. The whole way you make a computer faster, the whole way you make anything faster that involves information moving, is don't make me go so far to get it again. Just let me have my own copy. And that's why when you talk about the specs of a computer, it tells you how big the hard drive is, but it tells you how big the RAM is because RAM is mostly just a cache of everything that was on your disk. And then in the CPU, there's a cache of what's in RAM. And so the whole thing is designed to just have copies of things that are closer to the action. But the copies are smaller. They can't be as big as that you can't have a copy of every phone number in the world, no matter how big your iPhone gets. And so you just try to keep a copy of what's interesting to you. And so your question is, well, so why are we talking about cache? How does this affect Cloudflare? Well, one of the things Cloudflare runs, one of the services we provide, is called a content distribution network, CDN. And the idea behind a CDN is we can cache the contents of our customers' websites at our data centers around the world. There are 200 data centers around the world. And by having a copy of it closer to where the visitors of that website are, that means we can serve it even faster. And so that is why cache is so important. It's our copy of customer website content that is now closer to where the visitor might be able to access it. So even though my website, osmandonuts.com, which I will create someday, and we're going to have it on osmandonuts.com, is here, let's say, in the Bay Area, and my parents are accessing it in Chicago, they will get a copy of that website served from our Chicago data center, and it will be that much faster because you literally won't have to make the trek all the way back across the continent of the United States to get the information. So that's what a cache is. And that's why they're so important to how computers work and why it's fundamental to how Cloudflare works. But so as a customer, it sounds to me like cache is going to be really helpful to me because it sounds like it's going to make my experience really fast. But what you also mentioned is that I get to push it out to the edge. So I get to put it on Cloudflare's network, so it's not hitting my origin. So it's also going to save me money, right? That's right. So it's not just about speed. It's about cost because the more that we can – the more that Cloudflare can serve the content, the less Cloudflare has to go back to the origin. Now, if we don't have the content, we're not just going to make up the contents of the donut shop. We're not going to make up the contents of your website. We have to go back to the origin, which, again, origin means the web server that the customer is running. And so that is a critical part of the story. The thing is most customers that run websites, they get a bill for all the information that leaves. They're going to have to pay for that. And so that becomes important cost savings for them to minimize the amount of traffic that goes back to their origin. Well, I mean, this doesn't sound pretty hard. Like why don't we just cache everything? Because – great question. Why can't we just cache everything? Just cache it all. Cache it all. Well, for one thing, it still takes up real space. There's an actual hard drive somewhere that's filling up. And so it's important to actually – there is a cost to this. But also you have to figure out what to cache. And so a big part of this is giving customers the control to say what should be cached. Some things you totally want to cache. The logo of your company probably doesn't change very often. So that's a great example of something that you cache. But, you know, the current contents of your shopping cart, if you're an e-commerce site, you definitely don't want to cache that. That's going to be specific to users. If you cache the wrong stuff, you can wind up serving the wrong content, which is way worse than serving content slow. So giving control over what you cache is a really important part of this story. So, but, I mean, don't I just, like, you know, push some buttons and just cache some stuff and, like, just go? And, like, isn't it going to just be awesome? It is awesome in many cases. And that's part of the job of Cloudflare is to make it that easy to do. But we still want to make it possible for customers to have control over. You should cache this or you shouldn't cache this. And that is what the real headline of last week's ship is, is that we are working on cache analytics. And what that means is customers can now see, wait, what got cached? And just like we touched on a second ago, we worry about how many times does the Cloudflare edge have to go back to the origin? And you worry about how much content. So it's like how many times do I pick up the phone and how long is each phone call? How much content and how often do we go back? Because every time we go back, that slows things down. And the more content we go back for, that makes things more expensive. And so right there are the two big numbers that you want, that we want our customers to be able to see. And that's what cache analytics is about. Show me the two big numbers. Show me how often did the Cloudflare edge have to go back, number of requests that were cached missed, that had to go back to the origin, and how much content, how much data, just in raw bytes, how much of it had to go across the wire. And so then the customers can start to ask the question, hey, how could I make this better? How can I make it fewer requests and how can I make it less data? Got it. And so cache analytics enables me to basically get a view into what's happening with my traffic, either by request or by data, and then visualize that and drill in. That's right. In fact, it has got all kinds of buttons and switches and knobs on it. It will let you slice this information in so many different ways. It gives you those great roll-ups. It gives you trends over time, which is another really important thing you want to see. As you roll out new properties, as you add new pages to your website, you kind of want to see, well, how's it looking? How's it going? What's changing? You want to be able to get down to the individual files, what files are getting cached. That's what's called a top -end facility, which is out of all the pages on my website, which one hit the cache the most or missed the cache the most. And so being able to actually play with the numbers here and give our customers something that's more like a cockpit with all the buttons and switches all around the site so they can actually see what information is going through the cache, that's what we built. And that's why we're so proud of shipping it last week. Got it. So this will give me the information I need then to basically go back and optimize my page rolls to improve my caching efficiency. That's right. That's right. Exactly. Okay, but why did it take us so long to ship this? Why was this hard? Spoken as a true product manager to an engineer, I don't get it. I can describe it, so what's so hard about it? I did the hard part. I can draw a picture of it. I mean, I know exactly what it looks like. Hey, ship it. Programming, right? Just code it. You guys know how to code. Why can't you just code it? Why was this so hard? This is hard if you remember two other numbers. 14. 14 million requests per second. Go to the cloud branch. 14 million requests per second. In the time it took me to say this sentence, we filled up another hard drive worth of information just in information about the information. That's how busy the Cloudflare edge is. Why is it that busy? Two other numbers. 200. 200 points of presence around the world. All the major cities where there are human beings, there is a Cloudflare data center. And 27. 27 million websites. 27 million websites, 200 points of presence, 14 million requests a second. It's not that simple. Even though in conceptions like just add them up and show me a graph, how do you do that in a way that can actually scale up and down with the demand? How do you do that in a way that makes it easy for any one customer, regardless of how small they are? Because mom's donut shop is only going to have five visitors a month. Maybe my mom will come see it. Maybe, maybe me when I'm hurting it, but that's it. Right? So like, how are you going to see that show up at the same time that we have sites that have literally millions of hits over that period of time? And how do we, how do we summarize that? No matter how fast a computer is, you still have to make it what engineers call performant, which is it's got to be able to show those results in the time that you play with those, with those, with those switches and those buttons in the cockpit. And that's, that's the trick. That's the fun part. How does the system that can do that? Well, it's like finding my needles from my haystack within the context of all of the haystacks in the universe. It's complicated. I get it. Okay. So how did we solve this problem? I mean, that sounds like a really big, that's a lot of data. That's a lot of, that's a lot of problem. It's a lot of problems. There's an old joke I got in the 99 problems we have, you know, 14 million requests a second, but the logging isn't one of them. So how do you, how do you keep track of this? So two fundamental technologies and you know, one of the best parts of this industry is that we all learn from each other. Like, so we all stand like as the culture of open source and how any part of this industry shares information with other parts, it's such a, such a great, some of the fun of being in the Valley for the last 20 years is how we're all learning from each other. There's two key technologies here that I think are really worth talking about. One is GraphQL. So things that end with the word QL, it usually stands for query language, structured query language came out, I think in the seventies, maybe even earlier. That is, it talks about how you can, how you can interrogate a database. And GraphQL came out of Facebook just in the last decade. And the idea is, can you give programmers a special language where you can ask complicated questions? Like, I want to see the top five results that meet the following criteria, but it's not just about being able to ask a detailed question. It's about insisting that the information come back in a way that makes sense to you. Because one of the challenges of working with databases is that they store information that makes it easy to process the information, but not necessarily easy to display it or to work with it. And so what GraphQL is, is a very interesting way of thinking about it, which is make an API, which application programmers interface, a way of interacting with a computer system that says, I want you to show me the following information, but when you send it back to me, please put it in the following format. And what this means is user interface designers can just have a field day. They can work so fast because they don't, you no longer in the world where every time you design a new feature, you have to swivel your chair and negotiate and buy cupcakes with the folks on the data side. You basically can just design the query and the response exactly the way you want and get it back, which meant that our UI teams were able to build. Once we had this, we unlocked a Renaissance in analytics. And that was what allowed us to build the API. But the other big technology is how do you scale a backend where there is a difference of nine orders of magnitude, nine orders of magnitude. I mean, it's just, it's just mind boggling, right? It's like the analogy we were using at one point was a friend asked you to move and you're like, great, how much time should I give? How much are you moving? And they're like, well, you know, should I bring my Prius or should I rent a truck? And they're like, well, you know, some of the things are the size of a sugar cube, but some of them are approximately two thirds the size of Everest. That's nine orders of magnitude. That's what the data team at Cloudflare was facing. And so what they did was put engineering into the way that the data is captured that lets it scale up and down automatically. And in some ways we're all used to this technology. Whenever you're watching, whenever you're watching a show on, on your favorite streaming channel and, and someone else starts using the Internet and then, and you, you, you maybe your connectivity goes bad and then the show doesn't stop, but maybe it just gets a bit, it loses resolution. It gets a little bit blurry so that you can still keep going. That's what, that's what is called adaptive bit rate. And that's what we built, which means we can approximate when we have millions of requests, but we can be detailed when we only have a few requests. Result everybody wins because I don't need the top 9 million. I need the top 10. And so I don't need to keep track at the extraordinary detail level when there's already millions of requests coming in. But if it's only a small website, then you want to have the best of both worlds and that, that it's those two technologies that just had to be built. And it was once we built ABR and we built GraphQL, then cash analytics enters onto, onto front center stage. And it's like, check this out. Now we have all the tools to build something really amazing. That's amazing. And I know that that GraphQL is also opened up for our UI team, the ability to really enable our users to engage and work with finer grain detail and control, right? It's enabled us to do much better filtering. It's enabled us to do zoom mobile requests. I think it's just, you know, what we have here are some foundational pieces of, of kind of analytics technology that we see born out in cash analytics, but I think we're going to continue to see really like blossom across cloud for over the course of the next few weeks and a few months. We're excited about what we're going to do. We have, we have all kinds of interesting data. One of the amazing privileges is right. And a responsibility of being able to be able to see all this, this much traffic is to be able to give our customers that visibility back so they can see what's actually going on. It's their, it's their website. It just so happens to be running for our system. Right. Well, and we live in such a unique part of a customer stack where, you know, you do see all the traffic. So we are really interesting aggregation point. We have, again, a lot of touch points, both from the eyeballs and to the origins, that we can surface to our customers and help them understand and optimize their sites. Yes. It's really interesting. Now I know the other thing is like, we're not just doing analytics for cash. You know, one of the things that starts being important is you get more insight is how do you manage the cash? And one of the things we're continuing to play with is, is how we enable our customers to, to handle some of that. And I think we've got some interesting developments on the factory floor on that one as well. Yeah, we do. We do. All right. Now I'm going to do my best Jerry Seinfeld ready. Here's my joke of the day. Okay. I'm bringing, I'm bringing it up. There's only two hard problems in computer science, cash and validation, naming things and off by one errors. Applause please. Yeah. So the reason that joke is it's supposed to be funny is cash and validation is so tricky. It sounds really easy to cash things. Remember that back to the analogy, like I'm just going to have my own copy. So I gave you my phone number. You got your copy. Now I changed my phone number. How do you know? How are you supposed to know that the copy you have is wrong? If the cash is doing its job, you don't have to go back to the wherever you got the information. So right away, like this is a deep fundamental problem. That joke is from like this, like the seventies and it's never going to go away because if you have a copy of something that is disconnected from where you got it, then that copy could go stale. And in the land of, of CDN, that trying to tell the cash, blow away your copy, start over, ask the origin again, because whatever you have is wrong is called purge. Purge is super important. Let's suppose you're the marketing person for your company's website and you've just put an awesome bio for your CEO and you misspell its last name. You know, so how are you, how are you going to, are you going to, and it's already cashed. That means that around the world, everybody's visiting that page is picking up the misspelling. So you want to be able to purge that page to say, start over, pick that up again. And in the past, right? As soon as we built CDN, we gave our customers the ability to say just the complete bulldozer option, like completely erase the cash and it'll start over. But what does that mean? It means that origin is going to get traffic from 200 data centers as cash is reload everywhere. Or we give them a scalpel, this page, this particular picture, but what the way websites really work is whole teams work on directories, whole teams work on a whole set of pages. So what you really wanted was something that's more like a shovel, everything under this corner of my website, everything under this room, everything that matches this prefix, this beginning of this name. Let's, let's ask Cloudflare to please throw away its copy so that when customers, when visitors request that information, it goes back to the origin that's called purge by prefix and another highly in high demand feature. And that's, that's what we built last time and some great technology behind that to make that efficient and yet performant. I really love the combination of how these things come together. Again, giving customers greater visibility, providing some technology behind the scenes that enables us to do it in a really efficient, effective way, giving people rich visualizations, and then giving people new tools on which to act when they see the information. It's great. It's the same pattern we see all right. First, make it fast, make it safe, right? Make it fast, make it safe. That's ground rules. Then give them control, then visibility, then rinse and repeat. Yeah. Yeah. It's fantastic. Well, and the other one that we've started working on to give folks more control is the advanced certificate manager, which has been fantastic. I mean, if you step back and you think about it, we've been on such a phenomenal journey as a company around certificates, but just stepping back for folks who might not be fully plugged in. When we talk about a certificate, what are we talking about here? Yeah. It's that green lock. Well, it used to be green. It's so important now that they took away the green, all browser manufacturers to say, look, it's so important. We will instead alert you when it is missing. That's how much the web has changed in the last five or 10 years, right? A certificate is cryptography. It's a lock. It's literally making sure that the transmission between you and the website is secure, that you are really connected to the website you think you're connected to. And no one is trying to intercept or hack your connection. It's so important. It's what has allowed the web to become a fundamental part of how communication, how commerce, how all of us trust the Internet with so much information about our actual real lives. What makes that safe is a cryptography and what makes cryptography works are certificates. So a certificate really is just a big number. It's just, it's, you can think of it as a digital lock. It's half of a digital lock and it used to be that you had to pay for them. I had a startup about five years ago and I put up a website and, and I knew I wanted to have it secure, but it was just a marketing website. So in those days it was, it was a little bit extra. And it was going to cost me 300 bucks to buy that lock, to buy that. That's crazy. And at the time I was a Cloudflare customer. I was not a Cloudflare employee and Cloudflare put out a blog in 2014 that said, we are announcing universal SSL for everyone for free. And my jaw dropped that this, what this meant was I could secure my website paying $0 and 0 cents because Cloudflare was going to get me a certificate, a real one, a real one that my browser and the rest of my customers' browsers would recognize as yep, that's legit. And so we gave our customers universal SSL. The way we pulled it off in 2014 is we, we bundled it. It was our certificate that we shared a whole bunch of websites would share together. You know, when you get a certificate, you can click on it and say, who's who's actually behind this? And you would see your website name, but you'd see 10 other website names. So it was kind of like, you know, being forced to share your, put your luggage with the overhead compartment with a whole bunch of other people. And you kind of want a little more branding of your own. So that's when we introduced dedicated certs, which is to say you can, you, we, we can give you a certificate that is specific to your domain. But advanced customers want way more control. They want to control when it expires. They want to control what kind of cryptography does it use in there? It's a, it's a trade-off. You can get the most advanced stuff, slows it down a little bit. It means you have your clients have to be a little more advanced, or you can have the older stuff. It's that's what there's a, there's a knob there that we want that's different for different websites. We want to be able to, for them to control the name. We want to be able to then use the same cert at multiple different sub websites. So all of a sudden there's like, it's not so much about, it's no longer the story about getting a certificate. That's old news. Everyone has a certificate, but now it's about the control of that certificate. How do we give them advanced certificate management, ACM? And that's, that's the other thing that came with, that went live in the last couple of weeks. And we're very proud of it because it just, it gives our, it gives our advanced customers all the control they need, but still makes easy things easy. And that to me is like one of the key principles of software is like the hard things possible and the easy things easy. Well, that's one of the things I really love about working at Cloudflare and building product at Cloudflare, right. That we're focused on making things simple and easy at scale, really helping to build that better Internet, giving universal SSL to everyone, but at the same time, also appreciating and understanding that there are advanced workflows that need to come on top of that. And then offering kind of those advanced capabilities with things like ACM. That's right now. We talked a little bit about certificates and I understand it uses an underlying technology called, called TLS, but you know, I also understand that like people are starting to use TLS in, in different ways. And one of the things you talked to me about earlier this week was mutual TLS. What's that? Yes. Mutual TLS. So TLS, remember transport layer security, that's the technology that underlies and confusingly the industry renamed some of the technology. So it used to be called secure sockets layer, not TSL, SSL, TLS, anything which has got an S and an L and a good, pretty good chance we're talking about security. So mutual TLS is a really interesting thing. Usually if you think about it, we have been so concerned with making sure that no one is faking the website, right? It's, I want to make sure I'm really connected to my bank's website and not some imposter website. That would be really bad. And so all the technology is about proving to the, that the website proves to the end user, honest to goodness, I'm who you think I am. And that's why the green lock shows up, the gray lock shows up, et cetera. Normally you don't worry about the inverse. If you're a website, I'm like, I want everyone to come see my website. Everybody, I'm not going to worry about who you are. I just, I want you to come see my website. That's great. But what if you have a website that's really not meant for everybody? What if you have built a thermostat or a toy or a gadget, some kind of smart device. And the only things that should be coming to your website is your device. Actually, if anything else shows up, it's probably a hack attempt. So now I want the green lock in the other direction. I want the client to show ID. So rather than, rather than, it's almost like up till now, we've been saying, you know, I'm going up to a bar. I really want to make sure this is the bar. I think I'm going to, so I get a drink that I know isn't going to accidentally poison me. Now it's the other way around. It's like, no, I mean you just show ID before you're even allowed in here. And it's almost the same problems. How do I know that the ID you have isn't expired? How do I know that you have a valid ID? How do I know that you even showed up with ID? How do I know that it isn't an ID for a different website? And so that's called mutual TLS, because you're not going to take away the first part. We're still going to ensure that the server proves its identity, but now it's mutual. The other way too, the server goes, Hey, let's see some, let's see a certificate. Let's see what you got. And if the server, if Cloudflare looks at that certificate and goes, no, that's not what our customer asked us to check. We're not letting you through. Our customers want to go, Hey, we kind of like you to tell us Cloudflare when you did that. We kind of want to know how many times did someone try to come to our web property that you turn? It's good that you blocked it. Great. But can you please give us the information? And that's back to that logging. It's a different kind of analytic, but this time it's, how do we make sure that the customer, so now you can, you can deconstruct the title of that feature. Mutual TLS customer logging means as TLS, as mutual TLS is configured and Cloudflare realizes this particular connection should or should not go through, that we keep track of that information and we send it directly to the customer because the kinds of the folks that customer size that are interested in this are almost always the security team and they've got their own big databases of systems they use to keep track of this stuff. They're going to want to suck all that information into that security system that they can analyze it and act on it. That's going to enable me to figure out like if I've got, you know, enable me to identify what's gotten through successfully, what didn't get through. And then for the stuff that didn't get, you know, help me figure out for the stuff that didn't get through, did we block the right stuff or do I have a configuration issue? And maybe I actually need to. Maybe too much got through or maybe like this fire, like, Oh my gosh. All the shipments of the thermostat in November have a bad certificate. We've got to fix that. Got it. So it kind of comes back to you saying before, in terms of kind of creating the power and the control and then providing the visibility on top of that to enable our customers to be able to do that kind of configuration. I have a question for you, Jen. Sure. One of the things we shipped last week was called regional services. What is that? So I'm so glad you asked. So if you step back and you think about it, part of what makes Cloudflare amazing is that we've got presence in over 200 cities and 90 countries globally. And the way that we've built our architecture, it means we can process your traffic dynamically anywhere. We're going to process your traffic wherever it is the fastest and easiest. So if you're connecting to your website in Chicago and Chicago is really busy, we might bump you into the Toronto data center and process your traffic there, which like for many folks is fine, right? I just want it fast and I want it to happen. However, there are definitely some customers that we're working with that are starting to want to have more control over where their traffic is processed. They may have contract. Physically where it's processed. Physically where it's processed, right? They may have obligations to their customers to only process their traffic in certain parts of the world or certain geographies and stuff like that. So when we started talking to customers about this, we sort of scratched our head. We're like, well, we've built this amazing network that is fast and easy and does it anywhere. How do you then turn around and offer customers this configuration? And so part of what's so cool about Cloudflare is that we enable you to set configurations and push them dynamically to the edge in a matter of seconds. And what we've done is created for certain customers who need this capability, the ability to define and only process traffic. And for this point, we're defined two regions, the US and the EU. So if you as a customer subscribe to this feature and configure your traffic such that your traffic is only processed in the EU, it means that we keep all of your TLS, all your security keys in the EU, all of your traffic. We do on the edge. We look at volumetric attacks. We just make sure you're not going to get doused with a huge DDoS. We process that. But then we actually route that traffic back into the region and we make sure that all of the processing of that traffic, all of the decryption and the security measures happen within the context of that region so that the customer can be compliant with the obligations they may have for the customers. That's great because those obligations are changing, right? There's a lot of rules around this stuff. Constantly changing. The Internet is such a fascinating place because at once we're sort of this amazing sort of deeply interconnected community, but we're also all residents of local territories, local regions that are managing and working with and responding to changes and threats and perceptions of security and privacy in different ways. And so one of the things we really want to do is make sure that customers have the ability to have a highly performant, highly secure web experience, but they also have the control that they need to be compliant to obligations, again, to customers and regulations. So I think it's exciting. Again, as I said before, it's a feature that as an enterprise customer, you can subscribe to. Today we handle regions in the EU and in the U.S. And I think I'm really curious. It's early days on this. I'm really curious to see how the market responds to it and really where our customers take us in this. Yeah, this is fascinating stuff. It involves geopolitical regions. It involves rules and laws and policy and the Internet and performance. All that stuff matters, right? If you start sending the traffic somewhere else, it may not be cached to that other location. So there's interesting synergies in all these features. Well, this was fun. Super fun. Yes, and I think we're going to try to keep doing more of these. Cloudflare ships a lot of stuff. Jen's product managers are constantly dreaming up things and asking us, so can you have it done? When can it be shipped? And I'm proud to say that the Cloudflare engineering team is very talented and builds a lot of great stuff and ships at a tremendous pace. So we'll be back on Cloudflare TV to talk more about this soon. Thanks so much. Thanks, Jen.