Originally aired on May 30 @ 5:00 PM - 5:30 PM EDT
Join Cloudflare's Product Manager Daniele Molteni, and Director, Product Marketing Daniel Gould to learn more about all the work that has gone into Unmetered Rate Limiting.
Read the blog post:
Visit the Birthday Week Hub for every announcement and CFTV episode — check back all week for more!
Hey, everyone. Hello. Welcome to another segment of Cloudflare TV. It indeed is Birthday Week. My name is Dan. I'm on the product marketing team here at Cloudflare. I'm joined by one of my esteemed colleagues, Daniele, who is in product management. management. Hello, everybody. My name is Daniele. I'm the product manager for the Web Application Firewall, and I'm based in London. Indeed, indeed. And we are here today to talk about rate limiting because there's actually people think they're familiar with rate limiting. But we've been very, very busy in both in terms of adding innovation to rate limiting and really updating how customers can consume it. So really exciting stuff. And Daniele, I'm excited to talk to you about it for a few minutes today. So, you know, it probably makes sense, I think, to start at the beginning and really remind people what is rate limiting, right? And why do we still use it today? Why is it still such a valuable security tool? So what do you think? Like how would you summarize rate live? Yeah, I would say weekly meeting is one of the most used tool of the WAF, so it's a very used, very great last defense for everything that has to do with application security. So essentially the product is a way to enforce, to create rules, to enforce a maximum rate of request that you want to allow to reach your server, your origin. So essentially the idea is, is that I can let individual visitors, individual user to make, let's say, five requests per minute maximum. And after those five requests, then I'm going to block this user to for a period of time for a timeout period that we call it, like for a couple of minutes, 5 minutes, and we prevent this user to overflow server or an application with too many requests. And this often blocks certain types of attacks. So it's essentially a rules based system like many other things in the world. Yeah. And it's really useful, we think, just about just abuse in general traffic. You don't want and it's just a really, really good way to address that. There's a lot of configurability. You can set the threshold yourself. What makes sense? You can I think you can even account for anticipated peaks, right? If there is a spike in traffic, you can you can account for that and make sure that business runs smoothly and abuse doesn't overwhelm servers or, you know, knock things off track. What do you think when you talk to customers, when you talk to organizations, what are the primary reasons they tend to use it today? Are there are some use cases, things that stand out. Yeah, Yeah, absolutely. So the first one is targeted volumetric attacks. So when we think about volumetric attack, we often talk about DDoS, right? DDoS, you're flooding an application with a huge number of requests and then you're trying to bring down the application entirely, the server entirely. You take it basically offline and of course you can counter those type of attacks with like our DDoS product, for example. But when it comes to other types of attacks, like, for example, takeover account takeover attempts or bots trying to break into an application. So targeting a login endpoint or in general targeting expensive API endpoint, expensive to run, then we run into a situation where you're not sending perhaps DDoS, a massive DDoS attack, but you're still sending enough high volume requests to a very targeted portion or surface of your application and you can bring down the entire application or take over an account. So still sensitive data or basically just break into an application and do harm. So those are like typical use cases. We hear time and time again from our customers. And for those specific use cases, rate limiting becomes a very, very effective tool. Yeah, I know we've done webinars for customers in the past and maybe these attacks can be devastating in that they are like an account takeover attack. If you don't have the right tools in place, like a rate limiting solution, if that is successful, right, there's enough credential stuff and eventually no one is successful that can be devastating to a brand, right? I mean, the real consequences are not getting this right. And, you know, something like that can be really, really useful. So that said, I think that's really we'll talk more about some of these use cases, APIs, things like that, which sort of ties into a lot of the innovation we've been thinking about to actually continue to advance rate limiting. But let's talk a little bit about specifically what we're doing this week during Birthday Week to advance rate limiting or advance advanced rate, advanced rate limiting for the whole world. So let's talk about that a little bit. Yeah, sure. So I'm very excited. This is one of like one of my most exciting product that I have launched here. Cloudflare. And the reason is because it's going to it's going to affect a lot of our customers or I would say everyone basically on our platform. So let me first mention something. I think let's go back in the days before I joined Cloudflare and I think even before you joined Cloudflare in 2017, we launched DDoS. And when I joined Cloudflare, there was a lot a lot many people talked about that as a kind of a moment when where we basically brought revolution into the business world and DDoS was giving protection to every customer, whether they were a free customer probes. So whether they were small or all the way up, of course, or enterprise segment, they were giving admitted free data protection. So a very small site could benefit from the same level of protection that the big bank has. And that's really a revolution. And since then we've seen that we've mitigated a lot of big massive attacks to different parts of the Internet. And this arguably has probably increased probably has increased the security of the Internet. Because no doubt, no doubt. About it, sheer size of how many customers we have on our platform. And we started thinking about can we do the same for weight limiting? And this is where we started thinking about this. And what we are launching today is essentially unlimited version of weight limiting for every customer. So every customer today, whether they are free per base, of course, enterprise, they will get as part of their plan some level of protection with unmetered rate limiting, with free rate limiting. That's yeah. So this is going to be included in basically in everyone's plan. They will we can discuss later about the packaging and what every plan will receive. But the point is like everyone would be able without additional incurring additional costs to deploy rate limiting rules. Yeah. And something else that's really cool here is it's not just the packaging packaging to make it more accessible. They are actually updating rate limiting right for everybody. It's not just for enterprise customers and we'll talk about that, but also for our paid or free pro business plans. They're getting our most advanced most up to date rules engine ruleset engine, which I think is really, really powerful, just like you said, bringing a meter dash to everybody. Now we're bringing the most advanced rate limiting in the world to Everybody. Yes. Yeah, absolutely. So this is something like when you read the title of the of the blog post, of course, it is something that will catch everyone's eye. But the additional innovation is what we are limiting we are giving is not just our previous product. What people are used to today is actually our brand new we're committing based on the rules. There's a pretty big mouthful statement. But what it basically boils down to is that we are being building rate limiting as part of fire rules. So for people familiar with our dashboard and the way we interact with the far rules is a very descriptive way of defining what requests are going to match on or going to match on our specific rule. So you can use all the parameters of the HTTP request and also some metadata, and then you can decide what action to take. And until today, the common action for fire rules were a block log or challenge, while on the other hand always limiting was very much focus on path your right path only so you could create a rule that only applied to slash logging, for example. And so there was no other way to be more granular. And what we are doing today is launching basically merging the two things. So we're limiting and merging our rules engine. And this is what new limiting is. So you can define the request. You want to create limit through the filter of a final rule. So using and leveraging all the parameters of the HTTP request. And then of course, you can define the the rate you want to enforce. And we have additional, of course, characteristics or features we can discuss later for users of what we call now advanced limiting, which is of course the our enterprise grade version of the product. Indeed. Okay. So think about this and we've done rate limiting for a while just to make sure that for viewers we establish like how is this different like before? Because I think about this, we conventionally people think about rate limiting the way you said, right. There's a path and then they will look to measure or count traffic based on like a particular IP address. Right. And primarily that was the primary means of doing rate limiting. Right. You see where the traffic is coming from and there's a structural there and that is that. But now, as you said, really any header in the request can be used to count traffic. So this is I think, a big step forward in terms of the quality of it. And the other thing is you mentioned before sort of usage based in something we're moving away from. So this is another big, I think, advancement here and that previously it was like for, for say a pro or business plan, they would have to add it on to the plan and then pay per million requests. And this is something that you talked about the unmetered this is really what it means is at this point there's no longer paying for those requests or I think it was the $5 every million requests. That is no longer the case. This is what we mean by unmetered. You have it, you use it and you don't have to pay for extras or add ons or anything else. Yes, that's very correct. So that's the other big revolutionary. So we have customers now that they use their credit cards. You're talking about free pro and these customers, they have to use their credit card to pay for rate limiting charges. And sometimes the big problem here that sometimes they can't predict how much is going to because it's going to cost the other components. I think already we were kind of like we're already considering what's basically what are we going to charge. So even before for the old weight limit in the previous version, we were charging based on good requests. So we had this idea that if you were subject to an attack and if you're blocking, let's say, 80% of the request, then we would have charged you only the 20% that were gone through because we didn't want you to customer to, of course, be penalized by being under attack. And I think while this is this being very, very customer focused and customer friendly, I think we can take this one step further. So I thought we can actually make this a part of everyone's plan so that we basically help build a better Internet, right? So everything starts from our goal isn't done. So tell us a bit more. Yeah. What's the goal of Cloudflare and how this fits into that? You know, if you're just tuning in, you've never heard of Cloudflare before. You know, there's a mission, something that Matthew and Michelle talk about all the time, and that's to help build a better Internet. And I think that I see pieces of news like this. And yesterday we spoke about Turnstyle, which are which is our free CAPTCHA replacement, right. Which is also something really powerful. What people ask is, why do we do this? We're just given interesting, innovative technology away. If I were unmarried rate limiting now or giving away free cash alternatives. Why do we do these things? Well, it's because of that mission. Because this is the right thing to do to really help the Internet at large, to help build a better Internet. And I think that's one of the most exciting things about Birthday Week for me is like oftentimes we'll innovate something, we'll build it, it'll be really cool, really useful. And instead of doing our very best to squeeze every penny out of it, we'll make it available for free at large for the whole Internet, because that is part of it. That's the right thing to do. When you think about our mission, so happy Birthday Week I guess. And now it's means unmetered rate living, which is, which is awesome. And we talked about all the attacks this helped with. And so I think if it's dos, if it's stopping captures and now it's really stopping abuse across the Internet with all those attacks you mentioned. So actually, just now that we're getting people excited about rate limiting, let's actually talk about what they get in the PAYGO plans. I believe free plans that we've got free for those who are tuning in, free pro and business, and then we've got our enterprise customers. But those PAYGO plans who pay per month, they pay on an ongoing basis to PAYGO, I believe debt free accounts get one rate limit. Is that right? But as you go up the stack, how many rules again do you get for being a base customer or pro partner? So for pro, you get two rules and base you get five rules. And of course, our enterprise customers are going to get 100 rules on their on their zones. But of course, rules is only one side of it. So as you move up plan, you get, of course, access to more fields and features you can you can use. And we're going to show you the dashboard and what you can do in a second. But just to give you an idea. Free customers will be able to write rules only on path. So pretty much like our previous rate limiting product. And they are going to be able to define a rate only on 10 seconds window. And then as you move to pro, you get access to more parameters you can write your rule on. So it's not just path but the entire URI and method and you get a little longer window. You can you can define the limit the rate on and then if you go to bes, then of course you get a bigger list. So on top of method and on top of all the URI your query, you get also IP, the source IP or the source the IP of the request and you get a longer time frame. You can define your rate on the IP. The source IP opens up a number of new use cases. So you can, for example, in this case you can exclude certain IP ranges from the rule. So you can say I want to run those rules only on IP that are not in this list, for example, and that allows you to define a list of IP, for example, of your headquarter or IPS of your of your partners that you know for sure that no attacks nor malicious intent is coming from. And of course, when you move to enterprise, then you get a couple of different options so you can stay on what we call the limiting the basic rate limiting, which gives you access to the entire all the HTTP request fields, but then also the then you can also go into the advanced rate limiting. And then tell me a little bit more advanced with limiting how why is that exciting? Well, we said before for many years and rhythm has been very, very useful. We don't want to disparage what came before, but there was relatively simple in terms of we define a path, for instance, and then we look at where the traffic is coming from. And if a particular IP address is going above the threshold, that's where we throttling. And that was generally sort of the simple way of doing rate limiting. It worked pretty well, but it turns out the world has moved on. There are new sort of projects organizations are doing and we really felt that innovation really, I think we could put more innovation into rate limiting to make this advance rate limiting that would make it even more applicable to emerging use cases. And that's for me. When you ask why is it exciting, it's limiting, which now again, we call advanced rate limiting for enterprise customers is really, I think, more suitable for more use cases, particularly use cases that we see in 2022 and beyond. And I think one that comes up quite a bit is, is APIs and we know on any given day and generally. We've we've run the numbers that you did in a blog post. More than half the traffic on the Cloudflare network is API related. If we look at some of the headers and so that just I think makes clear the need to give organizations the right tools to keep APIs safe and productive. And so you talked about being able to use any requests we're building or any, any part of the request as we build rules, that's where we can really use the advance rate to protect APIs, given the fact that if we can sort of measure or count based on an attribute of the API request, like a key or a cookie or a token, that makes it, I think, more applicable for, say, like an authenticated API call, right? Where if we just block the entire IP range or IP addresses, that is sort of, as we say, using a sledgehammer instead of a scalpel where we just need to cool off a particular connection versus blocking it outright. And so I think those things like that are really, really exciting. Now something else that I think is really cool is, is the this concept that until we started thinking about this or you shared sort of parties and things, the notion of complexity based rate limiting, which I hadn't really thought of much. Can you tell us a little bit about what that is? Because I feel like that's something new and useful that people I think, get a lot of value out of. Yeah, absolutely. So before describing complexity, let me take a step back and expand on something you mentioned that is very important. So always limiting or traditionally limiting was making the assumption that every visitor had a different IP. So you were you were calculating the rate based on IP requests. So ten requests per minute were referring to the same IP. So once the rate was exceeding that, those ten requests per minute, then you would start blocking requests from the IP that exceeded that rate, not from everybody else. And this is true still for the majority of rules, but the big innovation or one limiting, as you said, is going beyond the IP based. So advance rate limiting, which again is our enterprise or flagship version of limiting, allows you to completely ignore the IP and define characteristics you are going to count or you're going to use. We say count internally, but you're going to use to define the rate. So as you said, you can use the an API key or a cookie to define the rate and calculate the rate. And so every person we done with a cookie or the key will get will be will be basically their rate is going to be computed. And as soon as it exceeded, then you're going to block request from that particular session. And this is the very powerful bit. And as you said, this is very what allows to protect API authenticated traffic API traffic so well, because then you don't care if the IP or if the visitor is behind or not or not obscures the original IP. You have one IP with many users behind and then you get can get really confused. Why do I see so many requests? How can I distinguish one visitor from from the other? With this solution? With advanced rate limiting, you can pinpoint what user behind the map is making those requests and block only that offending visitor. And this is very important and I think this is very step change innovation with the answer is limiting complexity based. I think that's something very interesting. And I remember when we started thinking about this and we basically created this for a few customers of ours that had GraphQL APIs, right? With GraphQL, you simply you have one endpoint that takes all the requests, regardless of what's the goal of the request. And so it can be a very can be very, very simple request to serve like it's a simple look up in a database like give me this ID or can be something very complex that you need to process. You need to run on the server to provide a response. Like, for example, think about running analytics or pulling some pulling some data. And so every request doesn't look like the next. So every request is a different load that you are putting on the on the server. And so what you actually want to limit here is not the number of requests, but is the amount of complexity you are pushing on to the server. So you don't want that every visitor, every user makes too many complex requests because if this happens, then the server can't serve all these requests and goes down. So we introduced a concept of complexity base, which means that the server can basically give a weight to the request, like this is a complex ten instead of 100. And then we track for every request the complexity. And when we exceed the complexity, regardless of the number of requests, then we start blocking the visitor. That is really that is the innovation, that is innovation. And we're seeing a lot of success here with GraphQL customers because this is where really you can tailor the product to the actual stack that they're using, the actual framework that the customers are using. And I guess a comment in general across all advanced limiting is that we are seeing we're seeing the product getting closer and more integrated with the actual back end. Interesting. Interesting. We've got about a little over 5 minutes left. Why don't we look at some of this in action? I think that that picture is worth a thousand words. Yeah, let's do that. So let me share my screen so we can I can quickly show you how the new Unmetered mitigation meeting looks like in our customers dashboard. So here I have this, the one page. This is a base zone. So if this customer is listening to us, So this is something that you're going to see if you right now go into the dashboard. So under read limiting, you'll see two sections. The one at the bottom, you will see that we had a label called previous version. This is the old way of limiting. So here if you see this is one of the classic all the activity rule which is scoped to a specific endpoint or path is a logging endpoint, and you are limiting a number of requests where to request below ten requests per minute. So we are still running the old system here. And then at the top we have just introduced the new one. So here you see you get five new rules, unmetered rules within your system. This feature is unlimited. There is a link to our blog post. If you want to learn more about us, then build what you get, like the details of every of the feature for every plan. But yeah, in here you can create new rules. If you are a free or pro, you'll see, of course, fewer rules free. By the way, we won't get this this week. So if you're a free customer. Yeah. Wait a little longer. Next week you're going to see the new cards going. Up just one way. It's just one week. Yeah, just one week. One thing to call out is that the one at the top is a meter, as we said. So if you're writing rules here, you're not going to get charged. But if you are still using the old one, then this is going to work as before. So if you have rules that are being billed, then you'll still get charged because of that usage. So if you want to take full advantage of the new system, you will need to rewrite the rules on the card at the top and delete or disable the old one. I would strongly recommend to do that as soon as possible so you don't risk to incur additional charges. So let's take a look at how the new regime looks like. So if you create a rule, this is going to look familiar. This is like the final rules pattern. So at the top you can define what parameters of the request you want to scope the request to. So you can say, okay, again, my login point and perhaps I want to the method to be opposed and then perhaps I want to exclude, as I said, specifically by Right. So use the IP source and then in here you can decide to block. For example, you can also customize the response exactly like all great limiting does. So you don't have to block right there other. All right. You don't have to block you can you can challenge you can log or you well different types of challenges. I would strongly recommend to use the challenge here that we decide what challenges to give. But yeah, it is not just block and then you customize the response. And finally, here is the for how long you're going to block. And here you can define the rate as you can see here with the same IP. This is what we discuss. This is what how we are tracking different users for these pro and free. This is grayed out because you can only use IP if you are an enterprise on advanced limiting, then you're going to get access to different things like, as you said, cookie headers, path, horse names, even the body. Now we are opening like to JSON fields in the body. So you can, you can really track that the use of a specific value of a field and associate the rate to that. Yes. And that's that's there are, of course, additional features like the counting expression. If you're interested, I really strongly recommend you to go and read out our new blog post where there are plenty of information. Yeah. So on that note, we've got about 45 seconds left. How do people get started if they're on today, a PAYGO plan? Do they have to have to start using the new plan and move rules over or their old rules? So, yeah, so if a customer doesn't have any old rate limiting role, you will only see the new one. You can just start creating rules there. Go and have fun. If you have an old limiting user, then you will see both for a period of time, probably for the next year. We will keep both running together, so just familiarize with the new system and then move over your rules. So you're going to you're not going to get charged. Perfect. We've got 10 seconds left, Danielle. Thank you so much. Happy birthday week. Exciting advancements with rate limiting. Congrats on the launch and many thanks. Thank you then. And thanks, everybody.
