Latest from Product and Engineering

Hello. Here we are again. It's been a while Usman. Yes. How are you? I'm good. Nice to see you again as we get started on our latest from product and Eng, I should say.

And I'm actually really thrilled with some of our guests. We've actually had both of these guys on at various times before.

So one of the cool things is that we've been doing this now for almost six months and we're getting to have a second round of conversations with our colleagues.

But Jon and Achiel, Jon, why don't you go ahead and introduce yourself?

Sure. Thanks. Good to be here. My name is Jon Levine and I'm product manager for data and analytics at Cloudflare.

And my name is Achiel.

I'm product manager here for the edge of the edge, focusing on a lot of our latest protocols.

Cool. Awesome. Well, and so what's part of what's so exciting for me is not only that we have these guys back, not only that I love talking with these folks, but that both of these folks launched things as part of our birthday week at the beginning of the quarter.

And, you know, we've seen phenomenal adoption and use of some of the stuff that we've launched then.

And so I thought it'd be really interesting to have them back on to kind of give us an update on kind of how the product is sort of soaking in and what they're seeing and stuff like that.

And even better, when we're talking to Jon about it, he's like, I'm not just going to tell you about it.

I'm actually going to show you. So Jon, show us.

Jon's setting a new bar for the latest from product management. All right. If this turns into a demo requirement, you're going to get real popular fast, Jon.

Uh-oh. Well, I have an easy job because I'm demoing analytics, which are so visual.

They just beg to be demoed. You can't just talk about analytics. You've got to show people.

We're very lucky that we're able to show you our Cloudflare blog, which is kind of cool because then you also get some intel on how our blog's doing.

But yeah, so one thing we launched during a birthday week, our whole refresh analytics experience for Cloudflare Zones.

And we're really proud of this. Kind of a culmination, really, of years of work on the backend side.

But what we're really trying to do here is take Cloudflare analytics and go beyond just kind of understanding what Cloudflare is doing but understanding what's happening on your website.

So we still show you kind of HTTP requests and data transfer and firewall analytics and cache analytics are still there, which we'll talk about.

But the goal here is to really show you kind of who's coming to my website, what are they doing there?

And so it's pretty simple. Hold on a second. John, remind me again. Why did we do this?

Yeah. Well, one thing we found is that customers, like, they really trust us and they really like that we've got these analytics at our edge.

And what's cool about that is we see every single request that comes in, for good or bad, even the malicious ones or even the ones that we block, even ones that don't execute JavaScript or that they're using uBlock or an ad blocker to block other analytics products.

So people were kind of already using our analytics product this way, and what we wanted to do was give them insights more into how their websites were being used.

Got it. Got it. Well, and also that we were able to do it in a really privacy-sensitive and privacy-aware way.

Totally. So one thing that's interesting about our approach is we're counting visits.

We don't do that with cookies.

We don't use it. We don't, like, use an IP address to, like, fingerprint people and track them.

We're actually doing it. It's a pretty interesting approach.

We're basically looking at how many page views do you have that came from another site or didn't come from your own site?

And the basic idea of that is you get this really valuable insight into what people are doing, but you're tracking them.

The slogan is kind of like, you know, measure, you know, track your website, not your visitors, right?

We want to know what's happening on the site, not what, like, people are doing, right?

Yeah. It's not our business. Yeah. Yeah. Super interesting.

Sorry, I interrupted your flow. Oh, yeah. And, like, having done demos before, like, I know you're doing this live.

And, like, doing a live demo is kind of like doing an acrobatic act with, like, without a safety rope or a net.

And, like, I'm screwing with you.

I'm poking you as you're getting on the trampoline. No, no, no.

So keep going. That's all good. That's all good. That's level. It's all right.

That's the pros of the pros here. That's great. Yeah, this is great. You know, actually, one of the other things that I like to talk about, it's, like, because Cloudflare is sitting in front of everything that is coming to your website, it's not just that we are the only way to get this.

Like, we sort of have this moral responsibility to expose this to our customers, because how else can you actually see what is going on on your website?

And so, you know, this was one of the great achievements, I think, John, that you and the team worked on was providing this great level of visibility so that it's not just do I restore the things I would normally be able to see if I had, you know, if Cloudflare wasn't proxying requests, but that I see even more, that I get even more insight than I would have had if I was running my own analytics on my own host.

Totally. Yeah, that's exactly right.

Cool. So I'll just kind of talk through kind of some of, like, how we thought through what to show here.

We wanted to show in this first view, in terms of visits or page views, we're just kind of core stats about your website.

So how many visits were there, like, over time? Where did they come from? If you've seen our analytics, you've seen this pattern where we have a lot of what we call top ends.

So you can see the top referrers, the top URLs that people are visiting, the host names, the browsers that they're using.

My favorite feature is actually the ability to kind of, we call the group by functionality.

So I can actually switch the referrer right here.

And then the graph updates, like, in one second, which I think is so cool.

And this is all powered by GraphQL, right, which is the.

Yeah. You said it sort of unlocked a revolution in analytics inside Cloudflare because by making it easy for our front end teams to build these really interactive cockpits, almost, where you can explore the data.

Totally. Like, it just became all this great innovation showed up on the dashboard.

It's so much fun to see.

Yeah, totally. The GraphQL, it's really interesting. So, like, as people know, maybe you've tuned in before, like, we use the same APIs we expose to our customers to build our dashboard.

There's nothing secret you can build this to.

And the GraphQL API is really amazing. It's almost kind of like a SQL-like interface into the data.

And it allows us to build an ABR technology, which we should talk a little bit about too, which is kind of, which is also amazing.

It allows us to build these really flexible analytics where I can zoom in, zoom out, time range.

I can add these arbitrary filters. I can, so I selected, you know, just traffic from Google search, right?

And then I can filter by country. And then, you know, I can, like, click a time range and I can, like, zoom into that time range interactively and scroll down and see, like, what the top URLs are.

So, having that kind of granularity and dimensionality is, like, super powerful way to investigate, you know, traffic anomalies, anything happening on your website.

Cool. So, that's just for visits. You can do the same for page views. And, of course, one thing we are hearing from customers is people, like, you know, they come to Cloudflare, like, I think, Jen, you were saying, like, they want to see everything, right?

And so, like, they want to see all the requests. And don't worry, they're all still there.

You can see all of them, even the ones that get blocked. And, yeah, if we have time, we can also go over to kind of firewall and cache analytics, which kind of give you a little bit different views into some of the same data, which are also really interesting ways of kind of exploring how your website is being visited.

Yeah, so cool. Yeah. And then, as you guys were doing this, like, you know, I mean, you talked a little bit about sort of kind of from a customer perspective or a user perspective wanting to see visits and page views.

You know, what are some of the considerations that you guys had to take?

I mean, you're dealing with massive volumes of information.

You're publishing, you're talking about viewing it in a variety of different ways.

And I believe that we make this available to people or we're pitching, making this available to people who aren't even Cloudflare customers.

So talk with me first about, like, what do you guys have to wrestle to the ground?

And then kind of like, yeah, you start there.

Yeah. We have an exciting announcement coming on that front very, very soon.

But I think when I talk about, say, the PM for the data team, I'm also saying really the PM for like the scale team, all we think about all the time is how do we take the fact that we have, I think the public number is like 25 million requests every second that come into our edge and do useful things with that information, do useful things, you know, for, for internal teams, so we can mitigate DDoS attacks and tech bots, but useful things for our customers like serve analytics.

What's challenging is that our customers range in scale from maybe they get one user a day, or maybe they get a billion users every day.

To be able to deal with like nine orders of magnitude.

Nine orders of magnitude. That's the cash rate.

It's like, it's mind boggling. It's, it's like, it's hard, it's hard to wrap your head around how difficult that is and to, to do something meaningful for all those different customers.

And we have really interesting technology that lets us do that.

It's called ABR. And the trick is that we just store the data at all these different resolutions.

So we store, you know, the full copy, 10% of it, 1% of it, 0.1% of it.

And the idea is we can just kind of adaptively pick the right resolution of data to work with, to answer any given question.

So very high level questions in a very large website, you don't actually need to look at a lot of data to get an answer of like we're actually showing a lot of syncing figures, but we probably could show, you know, 144 ,000 pages is a really good answer.

You don't need to know like the exact total number.

And that insight lets us do this and makes the dashboards really fast and makes them scalable.

ABR is one of those fantastic win-win things where it made, it made perfect sense for the customers and for the users to be able to see it as fast as possible and as reliably as possible enough information in the analytics dashboard to make an actionable decision.

And then in the backend, like just transformative to the, to the engineering teams, to the SRE teams, to the capacity planning teams who are responsible for collecting this, this, this, this, this ocean of information that is coming at it at, you know, 25 million requests a second.

And being able to ensure that in the face of a spike in the space of an unexpected load, you know, we just, we just exited black Friday in the face of that, that these systems can keep up.

And the fact that they automatically adjust their resolution and which is an experience we're all familiar with.

You know, you're, if you're watching your favorite movie on your favorite streaming service and your local Internet provider has a hiccup, you see the screen get blocky for a second, but you can still get, watch the action.

You're still with it. And, you know, and so like in, in here, the, that, that cost of that blockiness, that's important.

Like you said, 144.0, 144.8, like as far as the person who's responsible for watching that number, they've got the answer.

They've got the data. I just have one question on this.

You keep on calling it ABR, but is that a made up acronym? Well, it's, it's funny.

I used to work at YouTube and that, and that team, I don't know if it came from YouTube, but from the whole video streaming world adapted bit rate, adaptive bit rate, which is exactly what it is.

It's just different rates, but yes, good call.

Yeah. Don't make up abbreviations without explaining. And that's what we call it inside engineering as well.

It's the ABR tech. It's inside the cluster. Now and then, you know, John, I know we haven't announced it yet, but I, you know, as I mentioned, I know one of our goals is to make this available to folks who don't even use don't even aren't even, as we say, internally orange cladded don't have, don't host their site or don't, don't deliver the not host.

Don't, don't have their site on Cloudflare today.

How are you going to do that? Yeah, it's really interesting.

So the all the analytics that we're showing you right now are gathered at our edge, which we were just saying, it's really cool.

Cause we see everything doesn't matter what it is.

We see all the information, most of the analytics and the way, how are we able to do that?

Well, if you're already a Cloudflare customer, you know, you change your DNS server.

So that traffic, we proxy all your traffic, all the, you know, all of your customers are gonna go through Cloudflare before going to your website.

And that's how we see everything. And that's awesome. Obviously there's lots of benefits in doing that, making your site faster and more secure, but not everyone can do that.

And so people who are using, you know, popular analytic services, Google analytics, kind of the most biggest one out there.

The way those work is you just take a little snippet of JavaScript, drop that into your HTML or use like a tag manager or something like that.

And then you send out these little beacons and we actually have a product that does that.

That's really just focused on the performance side of things.

Cause if you want to measure speed, you kind of have to put the speed measurement like where the client is to do that accurately.

But what we really know is like, there's, what we've seen is there's this huge demand just for analytics, just this experience.

And you know, there's like measuring on the client is like totally valid.

It's like, it's a good way to measure things.

It's a different, it's a different view from the server or the edge measurement.

You're measuring something slightly different, but it's a valid way to look at it.

And we do want to make that available to anyone. We think the web could use a free privacy focused, fast, powerful, scalable analytics system.

And we're planning to do just that. So it's coming very, very, very soon.

I like the very, very, very, very soon. Okay. So then while I have you here, and before we pass the baton over to Akil, so Akil you can start stretching your synapses for your GRPC.

But the other thing we've done, John, this year that like, I would be remiss if I didn't give you an opportunity to give us a shot at would actually be showing us the analytics we're doing for caching.

Yeah. So I'm really, I'm really proud of this.

I'm really excited about this. So you know, since I've been at Cloudflare, one of the biggest things, I mean, just one of the biggest things Cloudflare does, we say, we make your website faster and more secure.

Well, how do we do that?

One of the biggest ways we do that is by caching content.

And it also helps you save money because you're not sending data out of your margin that doesn't need to leave.

There's a t-shirt in there. There's like caching is cash.

Yeah. Cash money, cash rules, everything. So caching is tricky though, too, because if you cache the wrong stuff, it's really bad.

I'm sure a lot of engineers watching this have like a cache collision bug, horror story time that ruined their evening or holiday weekend.

It's scary. And so we have to be somewhat conservative.

We can't just be like, Oh yeah, everything's cached on Cloudflare.

We have to like try to cache most things, but our customers kind of need to tell us what should be cached.

So we don't, we don't catch the wrong stuff. What's hard is before this, it was hard to see, okay, how do I optimize my cache at rates?

How do I cache more stuff? How do I make things even faster? Have even less egress from my origin.

And there's a lot of optimization that can be done like TTLs, the details of the cache headers.

There's, there's, there's quite a lot there.

And we have the controls for it, but before this, we didn't, we kind of, we didn't really have the details of being able to analyze that.

So that's what we've done with cache analytics.

So we have these two views requests and data transfers.

So think of requests as corresponding to, to speed. How many like individual times did someone have to get an asset from our edge versus going back to the origin, which takes a lot longer.

And the data transfer is really about cost.

So how much are we serving from our edge in terms of bytes versus bytes that egress from origin?

So if you use a public cloud provider, that's where they get you. They get you on that data transfer costs on the egress.

It's really expensive, but for Cloudflare, it's, it's not that expensive.

So yeah, we have these two views where you can, where you can kind of go between these and see, and then it has all the same kind of power and flexibility that I showed you.

So if I go back to requests, so our blog is pretty good.

You can see actually all the distributed caching is a little bit complicated.

There's some nuance there. A lot of hit that's good, but there's all this like revalidated stuff.

So like I can go filter on that.

Like what is revalidated? So revalidated means something expired. Like someone told us to cache it for like a day and then it wasn't in cache and someone else fetched it and we went back to the origin server.

And okay, that's okay. That's, that's not the end of the world, but that basically means that that's content that could have been cached for longer.

So what's cool about this is that this is a really great signal for folks to say, Hey, look, this, this like index that JS thing is being revalidated.

We're going back to the origin server all these times just for the origin server to say, Hey, I'm good.

Like it hasn't changed. So this is a great signal for a customer to look at and say, Hey, I should make my cache TTLs longer, catch things for longer.

So super powerful tool to be able to drill into what's happening with caching and does still require a bit, a bit of expert knowledge until like understanding how this stuff works.

So we'll be working on it.

Unlocks that question right now. You can ask, you can start to study that thing and dive through it and be like this, you know, this is, this is where you can do it.

Cause you can't start to troubleshoot if you can't see it. You can't see it. So the step one is the visibility.

It's the visibility. It's like having the windshield of your car.

It's an important feature. True that. True that. Yeah. And the other thing it's really cool is like changes that people think configuration changes.

Like did that, did anything happen? Like what? Like, can I see that now they can zoom in and see that.

So that's awesome. That's super cool. Yeah. And I think one of the, this is, this is a feature that's available for customers prone above.

Is that, is that right? Yeah. That's right. Pro business enterprise. Yep.

Totally. That's great. I would love to. Yeah. Yeah. Yeah. Yeah. For now. Cool. Anything else we should cover while we have you with the screen up, is there anything, anything else we, we should, we should cover, or should we pass it over to Akil to GRP, GRPC, our world.

Yeah. I mean, our analytics have, like, I love that we have analytics and like almost all of our tabs now.

And we're just like, it seems like every week, we're trying to launch a new analytics product, which is really exciting.

And yeah, just, you know, let us know if there's more things you want to see ways to improve this.

And yeah. Yeah. Thank you so much for having me.

I appreciate it. Thank you so much for having me. I appreciate it.

Thank you so much for having me. I appreciate it. Thank you so much for having me.

I appreciate it. Thank you so much for having me. I appreciate it.

Thank you so much for having me. Thank you so much for having me. I appreciate it.

I appreciate it. I appreciate it. I appreciate it. Thank you so much for having me.

I appreciate it.

Thank you so much for having me. I appreciate it. Thank you so much for having me.

I appreciate it.

Thank you so much for having me. I appreciate it. The most widespread one is And you can kind of imagine, let's say you're using a food delivery service or something.

So your phone is constantly trying to figure out, like, hey, what's the status of my order?

And like, the way it used to work is like every second or so, taking the server, like, hey, can I update the user?

It's like super inefficient.

You can imagine, like, for companies like that, or even car manufacturers, like, there's a lot of smart cars now that have really smart capabilities.

And all these analytics, don't be happy, up to the cloud. And all they're using, which is a little bit inefficient.

So five years ago, Google came out with a new protocol, which is gRPC, which does kind of two things.

One, it's a lot, lot more efficient because it's in a binary format.

It doesn't have, like, actual characters, but it has pure ones and zeros, which is a lot more efficient.

But the other really cool thing is it allows you to stream data. So the server, you can just set up a connection.

Your phone is up, and it doesn't have to ask all the time, like, what's the status of my order?

The server can just, like, push a message and say, oh, by the way, your order is ready.

The tricky part, I think, on our end is that gRPC has this kind of nasty requirement that you need HTTP2, which is like the way web browsers and devices commonly speak to servers, all the way from end to end.

And with Cloudflare, what we do is while we always support HTTP2 on our edge, we've kind of downgraded to HTTP1 before sending it off to your server, your origin.

That broke the gRPC flow, which is the- Wait, wait, wait.

Why? Why did we do that? Why did we downgrade? Downgrading? That sounds terrible.

That does not sound like Cloudflare. We do not downgrade. We upgrade, man.

That's a great question. It's actually largely due to just historical reasons.

Most of the reverse proxy software out there, we use for most of our pipeline engine X, and the only way it reaches out to origins is by downgrading.

KPL probably has a lot more background here. Well, I mean, I would cut ourselves a little bit of slack.

Like, HTTP2 wasn't really designed for, like, a reverse proxy, right?

It was designed for web browsers fetching a lot of- loading websites that have lots of requests from, like, one host name.

And so I think for the- when you think about how you would design a CDN, like, maybe you wouldn't use HTTP2 as a protocol.

Great protocol for client-to-edge, but edge-to-origin doesn't fit into that quite as well.

That's why we need HTTP4, but Ikea will talk to you about that the next time we show up.

All right. Okay. So, okay. So, sorry, I interrupted you.

We had a problem. We downgraded. Yeah. So, that's how we, like, the majority of people that run the HTTP server, like Cloudflare, operate.

They downgrade to HTTP1.

So, what we did in the end is we kind of had the option to do HTTP2 through our entire pipeline, but in the end we kind of figured out, like, a really cool hack in the sense that we can just downgrade to HTTP1 as long as the connection to your origin, to your server, is to HTTP2.

So, that's literally what we do.

We have a really cool blog article detailing exactly what happens. We still secretly downgrade to HTTP1 inside our network, but then right before we hop off to the origin, we upgrade again to HTTP2.

So, to anyone using it, it's a complete black box.

You just see HTTP2 and gRPC just works, and you can use gRPC. Yeah. So, it's fantastic, right?

Because the RPC is Remote Procedure Call, right? So, like, we're just talking about, let's clarify all our acronyms.

So, that, you know, and that lowercase g in front of it is, you know, Google Remote Procedure Call.

And so, like, here, we're delivering exactly what our clients want, which is the ability to use that gRPC protocol that streams the content all the way through and goes through the Cloudflare edge and comes back and picks up our security and performance, you know, our security and actually our routing, our performance features as well.

And so, we've given all that in keeping it compatible with the Cloudflare architecture on the inside.

So, it's actually a really great technical story of how we're doing.

I'm curious, Nikhil, do you have any stories about how our customers are using this and how it's being adopted?

Yeah, that's what I was just about to say.

So, that's a great point, then, that for a lot of our customers, this is, like, take the food delivery service, which a lot of people in that space use this protocol.

This is literally their bread and butter, right? If they can't, like, push orders or, like, inform people that their sandwich and bread and butter is done, they're done, though.

And we heard over and over from not just one, but a lot of people asking about this, like, please allow us to put this on Cloudflare because we want those security benefits, we want to be able to apply WAP, block management, we want to do rate limiting, we want to get DDoS protection, but also, like, just reliability and performance features, right?

You want those updates to go over the line as quickly as possible. So, support for hardware smart routing is amazing.

Reliability with load balance also, which is supported out of the box.

You can just move all of that and manage your entire pool of origins that are handling these things.

Again, this is not, or often, more often than not, not just one or a few clients.

We're talking, like, thousands, hundreds of thousands, millions of clients all connected together.

That's why they choose this super efficient protocol.

That being said, though, like, the interesting thing to me was I was, when we launched this, we generally heard more from those types of customers that they were using it.

We've also actually seen a surprising amount of interest from lower tier clients.

I mean, we have, like, thousands of users in our beta program, but a lot of them are just on pro, free, pro, and biz, just using this for either their hobby project or a small business, right?

It's really a protocol that's a lot more popular than we initially even thought.

Yeah, it's so interesting when you talk about kind of the implementation of gRPC and why people were sort of asking us to do it, right?

And this is so kind of quintessential Cloudflare.

It's like, we have this system, we have this network, and then we have this whole thing.

And it's like, if you can find a way to plug into it, you get the goodness of everything else that happens on the network.

Yeah. Which is just, it's such a beauty of the way that our system is designed, that it's like you inherit, when you take innovation and you plug it in, that innovation inherits all of the power of all of the capabilities that we have in our offering.

It's cool. Yeah, that's a great point. I think it's something we're really actively chasing here.

We have this great network. How can we get, like, as many protocols and as many stuff on there so you can take advantage of that?

Yeah. Another question I had is, Akhil, as someone who also spent a great deal of time with our Spectrum product and thinking about that, part of Spectrum's mission was, in fact, embedded in the name is the Spectrum, right?

Like, all the other protocols that are running on all those other ports.

You have 65,536 ports, at least in TCP before.

And so, 80 and 443, sure, those are HTTP. Like, let's build a product that covers everything else.

Like, has there been any interactions or discussions with customers about how Spectrum fits into things like gRPC and those protocols that are HTTP2 -based?

And, like, is there an interesting overlap or underlap in how some of these services relate to each other?

Oh, that's a good question.

So, yeah, for sure. In the sense that, for everyone tuning in that's never heard of this, Spectrum, like any protocol, but it's just a TCP or UDP reverse proxy.

Getting, like, feedoffs and protection reliability are a smart idea. Because it doesn't look at the actual bytes and the HTTP requests, we actually supported gRPC with Spectrum.

That's what I was going to add, trying to add to this. Just for the audience, Akhil and I did not rehearse this, and he absolutely said precisely what I wanted him to say, which is, at some level, we've supported this all along.

So what does it mean for us to actually nail this with explicit gRPC support?

Go ahead. Please continue. You're doing great. So we had customers using that already.

I think we just wanted to offer more. People want deeper layers of the inspection.

It's definitely going to be an ongoing focus for us to see, like, which protocols and what types of inspections you want to go forward.

But to your point of, like, other interactions between HTTP traffic and Spectrum, we also have customers that, for whatever reason, don't want it to decrypt at all.

But they do want, either they want to just hide their servers, or they want just feedoff protection.

And Spectrum is a great solution for that that we've used a lot. And all the other interesting use cases, we've also seen pop up more and more is people using us as, like, an anonymization service.

A company wants to promise that they can't see who did the HTTP requests, but they still want to serve the HTTP requests.

And Spectrum is a great solution for that. Because while we can see that, we can just promise not to share that information, and they can still offer that service.

So definitely a lot of really interesting overlap and cases popping up all the time.

And just to tie all these conversations back together, how do customers get to see what Cloud First Edge is actually doing?

John was showing off all these amazing graphs and analytics.

Like, what's the vision for how GRPC and Spectrum and all these kinds of analytics, how will they show up to our customers?

Do you want me to take this one?

Either of you. Yeah, sure. Yeah, so Spectrum today, we do expose through our Cloudflare locks, which is really cool.

So we will get all their Spectrum events, push it directly to Storage Bucket, their SIEM, wherever they want to view that.

Definitely would like to have better analytics in the dashboard for Spectrum, too.

Something we're definitely thinking about.

And then, correct me if I'm wrong. What's that? Did you forget about network analytics?

Of course we have network analytics. How could I forget network analytics?

Which, of course, for Spectrum customers, they can see all the network -level traffic, L3, L4-level traffic.

And by network, we mean at a level lower than the notion of a website, right?

Exactly. So it's actually like your network footprint.

And as Cloudflare has gone lower in the stack, it's been more interesting and more important and very interesting for us to see how our product has evolved to think of as Cloudflare as your front door of the lowest level of the network stack.

We're at 29 minutes past the hour. And as always, I can't believe how fast this time has gone.

Akhil and John and Jen, it was so awesome to talk to you about our latest stuff.

And we will be back again soon. Now we know the period of this sine wave.

It's going to be in a couple of quarters. We'll be back here and talking to you guys.

Thanks, all. Bye. Bye.

Bye.