Project Fair Shot

Good afternoon. Hi, this is Alissa Starzak. I am Cloudflare's head of public policy and I'm here with Brian Batraski. And I'm going to let him introduce himself, but we're here to talk today about Waiting Room. Hi everybody, I'm Brian. I'm the product manager for Waiting Room, load balancing and some of the health checks here at Cloudflare. Really excited to talk to you today. So Brian, let's start with sort of very high level. What is Waiting Room? Yeah, great question. So Waiting Room is part of our reliability products and solutions here at Cloudflare where it protects customers, organ servers and infrastructures from being inundated by too many requests that would normally take the application down. And so in turn, we take that excess load of requests coming from end users like you and I and place them into our Waiting Room queue. So therefore we provide a very seamless and expected user journey for any end user that's trying to get to an application, but also make sure that the application overall is reliable, stable and available for all users to interact with. So that's great. So let's put that in sort of terms that a nice public policy person can understand. So if I'm trying to visit a website, what happens? So yeah, let's say you wanna go to alyssa.com and alyssa.com is really, really popular. Everyone's trying to get access to it right now. And so normally without a Waiting Room, because we've seen this huge transformation of in-person brick and mortar sales and interactions for folks moved to online, normally the amount of requests that you normally be able to handle is skyrocketing from the normal cadence and patterns. And so you would then, instead of going directly to alyssa .com, would be actually go into our Waiting Room and therefore it can be our default or it can be branded to make sure that has the same look and feel as alyssa .com as well. And then it'll show you the estimated time for you to be able to get into the application. Wait a little bit, just keep that tab open and available. It'll refresh by itself. And once things have slowed down a little bit in the application, a little bit of room has been freed up, then it'll automatically load up for you. Alyssa.com will show up and then you can do and browse however you normally would like. That's great. So I wanna ask, how did you come up with the product in the first place? So where does it come from? How long has it been around or how long has it been in development? Yeah, great question. So it's a funny story. Last April, so I, also the product manager for our load balancing team, which governs a lot of how traffic kind of flows through our stack and our pipeline, our network. And so as a person who had a good sense about what traffic kind of moved around our network looked like, someone came to the idea of their own personal website kind of being hammered too much during COVID-19. And so they came to me and said, hey, Brian, what do you think about this idea? We thought about like maybe a waiting room and what are your thoughts? And I thought it was a fantastic idea and basically took that from a few of our internal peers and hit the ground running. And since then we've been very attentive away at developing and creating this amazing product. And as you can imagine, there are some very interesting edge cases and use cases that come up when you build a centralized product in a decentralized network, one of the largest in the world. And so it's been very, very interesting and great for learnings of how to build this and make sure it's successful and easy for folks to use day in and day out. So I wanna talk about that, but I actually wanna go to the use case first cause I think we have, it's also a time when we have these incredible use cases for it. So everyone's using the Internet right now. Here we are in time of COVID-19 and lots of people visiting websites that aren't used to getting a lot of visitors and particularly lining up for lots of things, including what? Why don't we talk about Project Fairshot? Absolutely. So, we wanted to build a solution, a reliable solution that was very flexible for any type of industry, right? As you can imagine, e-commerce, government, webinars, gaming, telehealth, a number of different industries are being hammered by more requests than they've ever kind of seen before. And so we want something that was very flexible and can be used across different vectors in the market. But one of these industries and one of these segments of the market that are being hammered more so and are pretty important right now is the healthcare segment and the ability to register for your vaccination. And so we came up with this project called Project Fairshot, which is where we're going to provide the waiting room service for free for all customers that are responsible for the registration and or administration of the COVID-19 vaccine. So we can do our part where we can make sure at least those applications are available, they are stable, they are reliable, and that folks have a very fair way to be able to get into those queues, be able to make their appointments for their vaccine and make sure that they're staying healthy and can take care of themselves. And so even though we can't be the ones that administer the shots, we can't be the ones to be driving the refrigerator trucks, but at least it's something that we could do and play our part, not only to make the Internet a better place for everybody, but also try to help this process and make it smoother for everyone involved. Well, I think all of us have an experience where we have someone in our families, my parents have tried to go get vaccines and they can't get in because the site has crashed or the site is just not accepting anyone else. And there's no way to actually queue up. So this is actually something that will solve that problem, which is pretty amazing. So how does it, from a practical standpoint, are there limits? Are there, you know, you talked about building something that is centralized on a decentralized, well, let's actually talk about that part. Yeah, absolutely. So, you know, we at Koffler, you know, we believe very fundamentally about, you know, building things that will work for everybody. And so that includes ourselves. And so we absolutely dog food our own products because if it works, you know, for other businesses and other customers, then there's no reason it shouldn't work for ourselves. And so we actually have built the waiting room product on top of workers and recently released durable objects. And so we actually have the power and the scale of our workers product and durable objects product where, you know, we really can scale up and scale down pretty, not only endlessly, but almost endlessly. And so we do have an incredible scale. We have, you know, again, the largest ending cast network in the world. And so we have yet to run into a capacity limitation in terms of waiting room thus far. And, you know, I'm very excited to continue pushing on those edge cases, pushing on the capabilities of what we can do. So then we can really make something truly, you know, scalable for any and all folks, all businesses across the world. So that last issue, you know, one of the reasons that we've seen some capacity issues from some of those entities. So the reason my parents can't sign up for vaccines is because they're using sort of legacy systems where they're not used to needing a lot of access, right? Or having a lot of people visit. So how do we build, how do we layer that? How do we layer waiting room on top? Why does that make it work? Is it a, does it fill the long-term gap? I mean, how do you think about it from a product standpoint? Yeah, yeah. You know, you bring up a really good point. You know, there are a lot of industries that traditionally have been a little hesitant to adopt new technologies. And, you know, healthcare is definitely, definitely one of them. Banking, it's understandable, right? They're dealing with a lot of really important data. They have something that works today and there's a lot of compliance regulation around that to make sure that privacy is kept intact and that, you know, everything is kept in line. And so we wanted to build something that didn't require any code changes and that could very easily be integrated with the existing architecture of whether it be kind of an older stack, more traditional, or something that was more cutting edge and new. And so there are no code changes required for Waiting Room. There are no JavaScript tags you need to add into your application because anytime that someone needs to go and commit a new set of code or do a new migration, anything of the sort, it introduces a new vector of risk, right? It's an unknown, unknown of what could happen. I think we've all kind of been in a situation at some point or another where we deployed some bit of code thinking everything was gonna go great. And, you know, there was some part of the stack that it touched that we weren't aware of. And so we wanted to remove that from the equation altogether. And so being able to deploy your Waiting Room is incredibly easy. All you need to do is have your DNS records and proxy through our network onto Cloudflare. And then from there, the setup, your configuration, which I think I've timed myself to set up within under two minutes. And so if you're the architect, engineering architect of your company, or, you know, a business office analyst or anything, a customer support representative, doesn't matter where, it's almost certain that you'd be able to set up a Waiting Room in a matter of minutes. And one thing that is really impressive that we're seeing is that we're seeing a lot of great traction from Project FairShot, from a lot of folks in the healthcare industry wanting to take advantage of this service so then they themselves can be reliable, be available for their customers, for end users so people can actually take advantage of their services. And so I'm hoping, and I think we're seeing some really indicators of some movements into adopting more technology through Cloudflare and, you know, overall in the market for these industries that are a little bit slower moving to adopt these new cutting edge capabilities. So what does it look like? So here I am on the, my parents signed up for somebody who's got a Waiting Room. What does it look like for them? What do they see? What is it, what's the experience like? Yeah, so, you know, they want to go into their health provider to sign up for a COVID-19 vaccination, for example. And let's say that application is a little busy right now. There are more people that are trying to access it that it can really handle. In fact, that's definitely the case right now. Yep, yep, which is what we're seeing left and right. And so as an end user, as someone that I'm going to, let's say CVSpharmacy.com, a pharmacy local here in California, and instead of, if it's too busy, instead of being able to go directly into the appointment slots, I'm gonna see a Waiting Room. What it's gonna say is, hey, you're in line in Waiting Room, thank you so much for being patient. Things are a little busy right now, but here's your estimated time for you to be able to enter the application and not to do anything, just sit tight, keep this tab open or this window open, and we're gonna take care of the rest for you. And as soon as Space opens up, that page will then reload. We actually refresh it every 10 seconds on your behalf. So you don't have to do anything. You can go browse elsewhere or, you know, feed your dog, you know, make some food, whatever it may be. And then once Space has kind of opened up in the application, things have slowed down a little bit, it'll automatically load for you, and then you can do whatever tasks and make sure you can register for that appointment. Well, so how does that work, Jack? So you don't know how many people are in line, but you have a sense of what the wait time is. How do you figure that out? How do you, what does it look like? It's a really interesting technical problem. And so it took quite a bit of, you know, getting the best minds together and the brainstorming of how can we make sure that this is not only accurate, but that we weren't arbitrarily, you know, killing folks when we didn't need to or so on and so forth. And so by using workers, by using durable objects and a few things that we call, you know, aggregation syncs across all of our colos across the world, we're able to get a almost real-time understanding of what is the inbound request coming to each of our different colos to a particular application, and then to have a aggregation across those in almost real time to see, okay, we see that there is, you know, a large influx of folks coming from North America, but things are looking okay a little bit in Asia Pacific. And so, you know, from there we make, we with workers and with durable objects, we make the decision of, okay, where do we let folks in and where do we kind of queue them up for a little bit? We also do a lot of calculations in the backend to have an understanding of how many folks are in the queue at any given point, how many, what is the inbound flow and like what are the traffic patterns we're seeing? So then we can always update the estimated time, make sure that it's dwindling down and keep that as accurate and up-to-date as possible. So does that mean when it really is based on order that someone visits the website? So from a practical standpoint, so anywhere in the world, or how does that look as a practice? Absolutely. So right now, as of today, we support a FIFO or first in first out queuing method, but that's only for today, right? We absolutely have plans for a near term iteration of the product because, you know, we're always wanting to make it better. We're always wanting to expand on what we're building. And so we are going to be creating different queuing methods that could be random or overclocking lottery. So then you can directly align the way and behavior of your waiting room to make sure that it is fair in the sense of how you want your end users to interact with your waiting room and how they should expect to be entered into your application. So today we do offer first in first out, but we are quickly going to follow up on that with a number of different queuing methods. So again, you can directly align this with how you want your customer flow to take place. Well, so what about the appearance of it? So when you go into the waiting room, what do you see actually? Is it, what does it look like? Can you brand it? How to describe that a little bit? So we give two options out of the box, right? We, if you don't, if you need something that'd be done very quickly and maybe you don't have as many resources of different creative marketing or brand design teams, we do offer a branded Cloudflare waiting room that works already ready to go. But if you do want to brand it, we do give full customization options to our Project Fairshot users. And so you actually will be able to change the HTML and CSS as you please, have it have the exact look and feel of what you want your overall application to be. So then when your end users actually go into the waiting room they don't actually really realize that they're still somewhere outside of your application that they would then think and have the experience that, oh, I'm still in alyssa.com and I'm just waiting a little bit, a few extra minutes before I can actually go engage and do my tasks. So we do provide full customization options to make sure that fonts, colors, icons, images, logos, all that look and feel is as seamless as experience as possible. On top of that, you can, we do support adding multiple languages. And so again, if you have different types of users then we can make sure that you cater to their experience to make sure that they get all the information they need. Well, so I see this and I think about all of the different applications of things that we can use it for. So when you and I were chatting before, I mentioned that these issues come up all the time where people have legacy systems where the problem is really inside. It's not big enough to address that sort of demand. So we saw it with voter registration. So again, I'm on the public policy side and we run the Athenium project. I can think of all sorts of uses for it. So what do you see as sort of the biggest potential? Where do you think it's gonna get used most? Just thinking ahead, what are the things that you think are gonna, you can be, you can do with it potentially? Yeah, yeah, great question. There are a number of different applications in the immediate near term. No, because we are launching Project Fairshot, the healthcare and specific for COVID-19 vaccinations is really our forefront in the near term, right? We wanna make sure we're really doing our part and it's incredibly important for the kind of the health of citizens around the world that they can have fair access to these programs and to these vaccination spots. And so that's what we're kind of really starting with. But again, the applications are really endless. E -commerce and retail are absolutely huge, huge folks that we definitely expected to adopt this. PS5s are maybe not as important as COVID-19 vaccinations, but people still really, really want them on top of sneakers and other types of retail items. On top of that, we see a definite large amount of interest from gaming as well, since gaming is taking a huge surge during COVID-19. We do see this for gambling, we see this for telehealth. We definitely see verticals for webinars, any sort of marketing events that take place. So far as we introduced Cloudflare Radar a little while ago to see and show the patterns and the updates that we're seeing around the Internet overall. And so one thing we have seen is a large increase in marketing campaigns that are taking place across the world to really drive people into their applications. And so any sort of type of, again, a webinar or any type of event that you're taking place digitally, Waiting Room is going to be a fantastic, fantastic solution to make sure that you don't get hammered or inundated with too many requests all at once. So there are so many things that you can continue to do. You mentioned that you're going to think about different ways of queuing. What else is next for the product? So there are some segments where people need to understand not only that people are inbound and they're being queued, how many people are being queued? What is the pattern of the traffic they're seeing? So we definitely want to come out with analytics to make sure that we give understanding of the patterns that people are coming in and out of the Waiting Room, how long they're spending in the Waiting Room, how long they spend in the application. And so we really want to arm our customers so that if any issues do take place or so that they can better and efficiently utilize their infrastructure, we want to provide that data to them as we do with other analytics products here on Cloudflare. And so we're definitely going to create that. Another item that's very high up on the roadmap is a VIP or Weighted Queue. So there may be individuals where, for example, maybe COVID-19 vaccines or for a concert when those come back, or webinars where you may want some folks to be either ahead in the queue or get specific access ahead of time. And so we are definitely going to build out something where we can have a Weighted or VIP queue where you can pass them a token or some sort of URL parameter where they can kind of jump the queue and have a little added weight into their preference of getting into that. So the whole point of making it customizable, right? So you can do it based on whatever criteria you want. Ultimately, you can figure out what the Waiting Room should look like, right? It's sort of kind of like the real world. You can decide if you want your velvet rope or you, there's lots of options, right? Exactly, exactly. And on that note, there are some other things that I think would be really fun where when you're in a Waiting Room page, there's a lot of opportunity for engagement. And so, I think we've all kind of seen where when you're on Google and then AirAsk come up, they have that little dinosaur game where you can kind of go through. So we're definitely thinking about adding little games or interactions like that, specific to different industries or verticals. So if you are, for example, a healthcare company, you could have something that aligns directly with healthcare. If you sell outdoor gear, maybe a game where someone is like climbing up a little mountain, also availability for showing your social media, your Twitter, your marketing events. There's a lot of opportunity for what can be shown in the Waiting Room pages and also how we can continue to help our customers drive engagement with their applications and with their products. Well, and in the COVID-19 sense, you can imagine people who are waiting for vaccines, there's actually a fair amount of information you do want to get to them, right? So even from a public standpoint, or even things, a public service announcement, that's the term I was looking for. You can imagine the things you might want to tell them as they sit and wait to sign up for the vaccine, because there's lots of information about the vaccine. There's a description of how you, what it looks like. It's sort of being in the airport and having to watch the video of, welcome to such and such airport. These are all the things that you need to know, right? 100%, absolutely. And links to the CDC, or again, there's full capabilities, right? We want to make that flexibility. So whatever is shown in the Waiting Room, the behavior, the logic really aligns with that particular group, that particular company. And so there's really no one shoe that fits all, right? And so we want something that not only is easy to use, very fast to set up, but again, can align directly with their given use cases. So how do you think you get the word out about this? Because I think what we're seeing right now, and this is something I deal with all the time on the public policy side, we have all sorts of entities that don't know anything about, that there's even a product like this out there in the market and they can't figure out how to solve a very practical problem, which is that their website keeps crashing or people can't get into it and they just don't know what to do. So how do you think we get the word out and what have you found that people have talked to you about when they've signed up? Yeah, well, definitely we're getting a lot of email leads. Whenever there's a tweet, and it comes from Matthew, our founder, that definitely gets a lot of traction and a lot of engagement. Also, the announcement that we made at Project Fairchild has been gaining a lot of traction. I think when things like this kind of come up, the word kind of spreads quite quickly and word of mouth goes very quickly. And also, it's really interesting how big the world is but how small the intermining circles can be. And so we are seeing a lot of engagement through our marketing campaigns, through Project Fairchild announcements, through our blog posts that we recently sent out. We definitely plan to follow up, continue pushing that message out. Because again, we want to help people. And that's the goal at the end of the day here. We want to make this available. We are offering this for free because we don't feel that it's right to charge people for something that is dire in a time of need where they just need help. And so for those candidates that do qualify under Project Fairchild, they are gonna make sure that we are helping them and we are a partner to make sure that they're successful. But in terms of engagement, getting more of the word out, more social media outreach, great case studies and quotes from our customers who have a great experience with us and our product. And so there are definitely some folks that have already been onboarded onto Project Fairchild for an emergency situation to make sure we can help them. And they're having a great experience so far. So can you talk about those yet or is it still too early? Can you give an example? There's one that I can talk about. And this is the county of San Luis Obispo located here in California. And so they had a problem where their vaccine registration site was getting inundated by too many requests. Everyone wanted to get access and they couldn't keep the application up. And so we jumped in right away. They engaged us and said, hey, we need some help here. We saw the Project Fairchild announcement and what can you do for us? And so we had a meeting with them a few days ago. We got there a very quick 30 minute meeting. They were able to get everything branded which looked amazing. It had the exact look and feel of their overall application and not only did we keep their entire website up for the entire duration of filling all their spots, not a single error was shown to their customers and every single person had a fantastic experience making sure that no wait times were above about 15 minutes and every single one of their vaccination spots were filled up through the day. And so they had a wonderful experience and they were able and so nice to actually share a quote with us that we've actually been kind of sharing with the rest of the world so that others can see what great experience that they've had and how they can also have a good experience with our products as well. Well, and it does show that this is something that can be done, right? It's the incredible power of technology when you can build something that actually helps people in real time with a very concrete problem. So there are lots of things we can't fix in this pandemic but the fact that we can help fix that is just a really good thing across the board. I completely agree. It's incredibly rewarding. I don't wanna say it's a stretch to say that we're helping save lives but we are doing our part from a networking industry and Internet standpoint where we are helping those folks have the access into those programs and it's incredibly rewarding to be able to be a part of it. Well, I hope we hear from both, not only the people who actually sign up for the service but from the, maybe they'll never know that we're actually helping them but from the people like my parents who try to sign up and can't get in and then can because waiting room has been used. It really does make the difference in Project Fairshot in particular. Well, so Project Fairshot, let's go into the scope of it a little bit. So it's global, right? We're doing it for anyone around the world. Absolutely, yeah. As long as you are responsible for the registration of COVID-19 vaccine, you absolutely qualify. You know, we do have some marketing pages where you can sign up and let us know that you are a vaccine distributor or a registrar and we will absolutely engage you as fast as possible and make sure we can partner with you to make you successful. And yeah, as long as you are responsible for that area, you will get those services for free. And then you figure out how you wanna distribute and you figure out who can sign up. You set your own criteria and then we can help you make it work technically, right? Absolutely. And so one thing to also call out is, you know, every product in Cloudflare needs to be able to interact with other products here, right? We are a full platform. And so waiting room does integrate and interact very cleanly and nicely with our DDoS mitigation, our WAF, our firewall, our bot mitigation management, our rate limiting page rules and redirects, all that integrate quite seamlessly with waiting room. And so if you do get those products as well, that'll get all your illegitimate traffic out the door before it ever reaches waiting room. And then only the folks that are actually legitimate users who need to sign up for that vaccine will actually make it into the queue. So then it won't arbitrarily raise your estimated wait times. And again, all these products are gonna work hand in hand to make sure that you're successful. And then in the future iteration, something that we are thinking about and we believe is a good idea as we're talking to customers is an integration with our load balancing product as well. Don't have that just yet, but as your origin servers go down or inundated, instead of serving a 502 error or a 503, instead we want, so you can conditionally show a version of a waiting room with conditional messaging saying, hey, something went wrong a little bit or whatever kind of copy or text that you'd like, hold tight, we're working on it. And that is a much better experience than just showing this bland 502 error page where that can tarnish brand trust, re-increase cart abandonment and all these negative things. So we think that will definitely help not only our end users, but our customers as well. All right, well, so that's the integration that's coming, right? It's the, it is funny how much that you can influence things just by what message you show. And this is the same point in the branding, right? It's the idea that you can decide exactly what it looks like and figure it out on the front end is a big deal. Absolutely, a hundred percent agree. So what else? So what else can we do? What else should we, what else have, what have I forgotten to ask about waiting room that I should have asked because I am not technical enough? No, I mean, I think we've covered quite a lot of it. You know, we went through the main, a lot of the main use cases. You know, we, one thing that we do have coming out for our enterprise GA that is coming in Q2. Again, we're going to come out with more of those queuing methods. It's something where we call an event schedule where if you, again, if you have those marketing campaigns, you have those webinar events, anything you want to really drive traffic to, it doesn't, it's a different use case than what we call unplanned spikes, which we're covering today, where you have too many requests coming in when you didn't really expect it or forecast it, and we're protecting you now, but that doesn't mean the behavior of your waiting room is going to be the same when you have some sort of marketing or online event. And so we are going to provide event schedules in our later release coming in the second quarter where you can change the logic specific to that event, set it up, forget it, and then know that in however many weeks ahead that that event takes place, waiting room will automatically kick in with that specific type of logic and behavior. And then once that event is over, it will then kick back into the unplanned spike behavior that you've already set up originally on that waiting room. So again, this goes back into the easy access, the easy setup, and the really flexible nature of waiting room to make sure it works, not only for each individual customer, but for each event or timeline those customers go through as they continue to grow and scale their business. So how many of those can you set up? So say you have multiple events coming, how much do you plan in advance and how much reverts back to this sort of norm? How much can you do? Yeah, so we're still figuring out a lot of those fine -pointed details, but right now, you'd be able to set up at least a few weeks in advance, and then the event can go as long as now, up to a week, if needed, but again, we want to provide that flexibility. So as we continue to build this out, we're going to continue to engage our customers, validate our hypothesis with them, really understand what they need, because there's no better way to find out what to build and make sure that it's really going to be adopted and be right, other than talking to your customers and having them tell you what really is important. So we're going to continue to uncover that, make sure we validate this across the market, and then we're really excited to push it out very, very soon. Well, and this is a set of customers to do it with, right? I mean, this is the set of customers who actually can really think about, when Project Airstot in particular, they're going to be thinking about how to make sure this works, because they're going to have to keep doing it. It's just the first round of vaccines. We have a long, long way to go, unfortunately, but the more we can do it through Project Airstot, I think the better off we'll be. Exactly, exactly. The more of these applications can be available, the more folks that we can get into these spots, the more availability, the more reach that we can kind of have across different areas of the United States and other countries in the world, that's only going to help speed up the time in which we get more people vaccinated and really try and play our part to end this pandemic and have moved to the next chapter of all of our lives globally and personally as well. Well, Brian, I think this project is amazing, and I'm super excited about it. It makes me very proud to be a Cloud player. And so I want to thank you for it. But also, I know we're sort of coming to the end now, but this is great. I'm really excited about it. And so just thank you for all of it. And thank you for Project Airstot and all the work you're doing coming up, so. And thank you for all the folks at Cloud Player that have spent so much time to make sure this is a success. You know, it really takes a village, so thanks so much, and looking forward to talking to you all soon. Thank you. Thank you.