What is Cloudflare Customer Support

Hello. Hi, welcome to Cloudflare TV, everyone. My name is Shane Ossa. I'm the technical training program manager for the support team here at Cloudflare. I've been here about three years. I'm a bit of a technical training geek. And I'll let Otto introduce himself. Hey, everybody. I'm Otto Imken. I'm the head of the support team here at Cloudflare. And I've been here four and a half years next month. And we're here to talk to you today about Cloudflare customer support. It's the key to the company. And we're excited to kind of give you details and talk to you about how things work. Shane. Cool. Yeah, Otto, I was just going to do, before we launch into it, just a super quick bio so people get to know us a little more. I've been in the Cloudflare for three years. And before that, I was with a solar manufacturing company in the Bay Area for a couple years doing technical training with them. And before that, I was in the solar industry at a local EPC for about seven years. And it was there that it was in both of these companies that I got into training itself and started studying it from the meta sense, how to be a good technical trainer, how to create training curriculum, how to create training products, how to train partners. So I got really into it, obviously. And then Otto found me and the Cloudflare team found me and brought me in to build a training program for the support team. And when I joined, we were about 25, 30 people. And now we are about almost 90, almost 100 people. So it's been a really exciting adventure. And I'm so excited to be part of this journey. How about you, Otto? Do you want to give us just a quick bio? Sure, sure. And I'm going to say it was one of the best things when we started a training program on the support team at Cloudflare. Before that, it was tribal. People would sit next to each other and watch people do tickets until they knew how. And ever since we had a training program, it's allowed us to really grow and keep the intelligence on the team across training 90 people. So training has been really great. So me, yeah, yeah. I started doing customer support in 1997, which I guess is 23 years ago at an ISP in San Francisco doing support for 28.8 modems, 56.6 modems, Windows 95 on the phone with people trying to set up their TCP IP and connect to the Internet. And worked at a few ISPs as level one support, level two support. Worked at an email list hosting company called Topica. It was my first customer support manager role back in 2001. And a couple of other startups and ISPs in the Bay Area. Worked for a couple of big companies like HP, was the head of support at Snapfish, their online photo division. I worked at Viacom, Nickelodeon, which is a lot of fun. Spongebob. We had actual green slime in the office, which was very dangerous. And yeah, yeah. You had to watch out when people were in a frisky mood on Fridays. And before this, I worked at Liftopia, which is a company that sold ski lift tickets online. And that was really exciting to do support there. I had to learn how to ski to feel the pain of the customers quite literally. And came to Cloudflare about four and a half years ago. A little bit before you, yeah, I was about 20 people on the support team, about 300 employees. And it's really been exciting to see the company grow. Nobody really knew who we were four and a half years ago. And it's really been amazing. And yeah, really happy to be here. It's been a privilege to help grow this team. Yeah. Cool. Well, let's dig in. Let's talk about customer support at Cloudflare tech support. Yeah. If you're a Cloudflare customer, what kind of support can you expect to receive? Yeah. So when you, so our support model is follow the sun. We do support 24 seven, 365. We have a team all over the world. And you can contact us via phone, email chat. And we have so many customers, so many domains that use Cloudflare. We really have to kind of differentiate the different levels of support. And so we have millions of free customers and we love our free customers. And we have to, there are so many, well, I'm kind of jumping ahead. We have to really allow free customers to help themselves as much as possible. We have paid service models. We have our enterprise customer support and sorry, I'm losing myself, but the follow the sun model really allows us to help people 24 hours a day. And depending on your plan level, there's different levels of support. We can kind of dive into more. Right. Yeah. And I should say there's a couple of different ways to contact us. Of course, you can just email us at our email address. But one of the cool things that we have for contacting us is the ticket submission form, which you get to from our help center on our dashboard. The ticket submission form is something we built ourselves. It's not just your average web form. It actually starts to run diagnostics on your domain. As soon as you type your domain in, it starts beginning to suggest a knowledge-based articles that would fix anything that it found wrong with the test that it ran, even before a human being is going to come to help you. Cause we know that customers ultimately want, would like to be able to fix their own problems without, you know, going through the motions of a support troubleshooting experience. So they can contact us via the ticket submission form, which is really cool, or they can just email us of course. And then certain customers, paid customers, are entitled to chat support 24 seven with a live human being. And you already mentioned 24 seven phone support for enterprise customers as well. They can call us and get a live, highly trained human on the line to help them with their emergency any day of the year, including weekends and holidays and everything. So that's our basic model is just be there for our enterprise customers on the phone or on chat or via email all the time. And so, like you said, we follow the sun, we have people in Singapore, we have people in Austin, we have people in London, and now Lisbon is a new office that is scaling up as well. And San Francisco where we're both based in order to achieve that. Yeah. No, I love the ticket submission form. So yeah, when you're logged into Cloudflare in the dashboard in the upper right, you can click on support and it takes you to a great page with all of our resources where you can see the status page to see if there's any maintenance going on or any known incidents. It can take you directly to the community where you can talk to hundreds of thousands of Cloudflare power users for posting all day 24 seven there. It can take you to the knowledge base where we have 500 different articles on anything and everything Cloudflare related. And so I've always said customers don't really want to talk to customer support unless they really have to. And so if they can find the answer themselves, or better yet, if the product can just work, and they don't have to troubleshoot it, that's all the better. And we constantly work with product engineering to give them that feedback on what's working and what's not. But if you do need to talk to us, the ticket submission form is something we built ourselves. And it really tries to help you diagnose the problem before you submit the ticket. Because again, you don't want to have to sit and wait. And even if you're an enterprise customer, and you get a reply within an hour, you still don't want to sit and wait an hour you want the product you want to fix now. So yeah, the ticket submission form will as you start typing in your info, it'll suggest an answer. And a good way to look at it is most questions, 80% of the problems that we see have a known answer. If you're trying to figure out how to set up your DNS records, or how to install an SSL certificate, millions of people have already done that. And there's a really good way to do that, that our technical writers and our senior support people have looked at a million times. And here is a really good step by step instructions on how to do it. So it'll suggest those first. We're adding more rich content to that as well. We're sorry, cutting in, we're adding, you know, we're adding multimedia content, shout out to our content team that's building, you know, recorded videos, embedding, you know, screenshots as much as possible to make this, you know, support customer facing content really good so that they don't even have to talk to a person unless they really have to. And another thing that's cool to mention about the ticket submission form is the same service that's running in the background, diagnosing and running tests on someone's domain when they type that in, make the submission form is the same service that we actually have hooked up to respond to them if they do go forward with a support request, which is our automated response service helper bot. Maybe we can just kind of jump ahead to that because that's something that's our support operations team has developed, which is really, really cool as well, right? I mean, we're a tech company and the question I get a lot is like automate, automate, automate, you know, let the robots do what the robots are good at, let the humans do what the humans are good at. And that's been a big project of ours. Yeah, yeah. So the automation, I mean, one way to look at it and take a step back is we always start where a small team, you're going to manually have tools and you're going to have to figure it out yourself. And we just hire really smart people who can figure out problems. But over time, then you want to replicate that with internal tools. So internally, our team can run a test and figure out what the problem is. And once those tools get pretty good, we want to expose those to customers. And so the ticket submission form, for example, runs a bunch of different diagnostics. So we can see if there's something broken with the way you've set up your SSL certificate, or you don't have a DNS record set up or, you know, several, there's a seven or 10 of them right now. Over time, we want to put those in front of you in a dashboard so you can do those yourself. But right now, HelperBot, when you're going through the ticket submission form, will run those tests for you and show you the results and show you green, green, green, red, there's something you need to look at this might be why you're trying to contact us. HelperBot also runs anytime you email us or chat with us or submit something through the submission form. And it will send you an automatic reply saying, hey, it looks like we have a problem here. It'll also insert information into the ticket. So when a support engineer is looking at it, it'll say, we ran some diagnostics on auto's domain, and it looks like he has a problem with his SSL certificate. So it gives us a jump start, it tells you, it tells us, it starts fixing the problem before a human actually starts looking at it. Right. And it's really cool what that team has done. I mean, we have data scientists and researchers using, you know, machine learning, natural language processing services that uses like sophisticated keyword matching. So we're really, I mean, this is a team that's presenting at international, you know, meetings of people that do this type of research. So we're really kind of on the cutting edge in terms of automating as much as we can and trying to resolve a customer's issue as quickly as possible and try to give them a great experience. And, you know, it's been ramping up. I'll share a little graphic for you there if you want to. I think you're showing the outline instead of the slide. Yeah, no, I couldn't be happier with the operations team. It's four people today and we're adding more people all the time. And a lot of customer support will automate things. Yeah, that's perfect. So like if you contact an e -commerce company or your phone company or ISP, they really, it is really the same 10 or 50 questions over and over again. And they can really send you the same thing about how to apply for a refund or how to reset your password. And so a lot of companies have out-of-the-box automations that really makes sense for them. Our products are so customized and so complex and your situation is always, our customer situations are always pretty unique that we haven't been able to really use those out-of-the-box automations. And so the support operations team built this from scratch. And every time I tell people at, you know, conferences or in meetings with different vendors, they, someone told me last week, I've never heard of any of our customers building this themselves. This is something that big vendors like Salesforce or Zendesk or ServiceNow spend a lot of time and have lots of engineers doing this. We've got a team of four people who built HelperBot. And this slide is, kind of shows you just how amazing the job is. So this is kind of showing the past 12 months of ticket touches. We might get 15 to 20,000 tickets per month. And this is showing that in May, 4,163 of those were helped by HelperBot with zero human touch. And so again, it's not a dumb automated generic response. It's going through and running tests on your domain. It's pulling info from our account system. It's checking your DNS, your SSL and seven or eight other things. And then using machine learning on all the texts of the ticket that you're putting in, in order to give you a really good first response on what your problem might be. And again, just reiterate, like 80% of problems are the same things over and over again. And so when it's one of those, it's pretty confident sending out that response. If it's not one of those, if your site is under attack and you're saying urgent, urgent, I need your help. It's not going to send that auto response for the most part. It might send you some generic info, but it's smart enough. And it's gotten smarter over the past two years. Machine learning does that so that it knows this is something we need to mark urgent and put in front of a human. This is something that's the same old DNS setup that we see all the time. And I'm confident, me HelperBot, I'm confident sending you this. And it really worked. And so that 4,000 zero touch tickets in a month is a huge benefit to customers and to us. And so it was worth spending that time instead of using a generic out of the box solution. And that's great. And that allows our highly trained and smart and hardworking team to focus on the issues that require an investigation, that require some debugging, or that require a human touch and looking at someone's settings and suggesting how they could configure something better. Right? Exactly. There's a lot of problems that we see that are super complex and require years of training and sophisticated tools and talking to our engineering team or our SRE team, site reliability engineers, and really kind of deep dive troubleshooting. And in order to do that, we've got to free them up from doing the repetitive tasks that a bot can do. That's really what bots are for. And that's another really important role in support for support of this kind of company is to aggregate that data, what customers are contacting us about, what feedback are we getting from the customers so that we can pass that information to the product and engineering teams so that they can improve the product. We tend to see, we feel the pain that the customers may be feeling. And we see if there are any issues or if there's something that can be improved, we're typically the first people to see that. So not only are we trying to solve the issue, but we're trying to then do our part to make sure that that information gets up to the product team, the product managers, and the engineering team so that they can get it on their roadmap to fix that thing or improve that or add another feature. We deal with feature requests all the time as well. Yeah, that's definitely a core part of our team's job. And I get requests all day long from product managers, engineers, from my boss, from people all across the company. What do customers say about bot management? What do customers say about stream? How are customers in India dealing with our network? We get that all day. And that's a huge part of our job is to look at that data, collate it into something usable and actionable, and be the voice of the customer back to the rest of the company and give them something that they can act on. So we have a couple of people on the team, Kevin and Zane, who worked on the team for several years and have recently become product specialists. And that's a core part of their role is to gather that data, take it to the product team, and prioritize. There's always a million bugs with any software. And so we have our top 10, top 20 list of this is what customers are telling us are their biggest pain points. And this is what's going to have the biggest impact if you fix X, Y, and Z. And I won't go through our dirty laundry of all of our bugs, but every software company has plenty of things that could be better. And so if you're a customer, come and tell us. Say, this would make my life easier if this part of the dashboard was easier to use, had better pull-down menus, had better instructions. We can make the instructions better. I'd love to be able to drag a section of this analytics dashboard and filter this one thing and drill down. We heard that, and then we released that. And so a lot of these requests come through our customers. This feedback comes to our customers. And I think that's a good segue. You mentioned our product specialist team, which is on the support team. So as part of the journey here, it's been really cool to see the way the team has matured into providing different career paths for people that have career goals. The majority of our team are tech support engineers, and these people are extremely technical, very driven, empathetic, hard-working. But over time, we've had to specialize. Some people have joined me and are helping me train the next generation of tech support engineers. And some people are really interested in educating and helping people grow, and they join the training team. Some people are really interested in improving products and doing the work that happens behind the scenes to take that feedback and data, the ticket data. We got 100 tickets on this one issue, and they want to make sure that the engineering team is going to prioritize fixing that issue. They can go on to join the product specialist team. We created another internal team to handle our escalations from the support team up to engineering teams as a buffer so that tech support engineers aren't going directly to an engineer who's in the middle of a sprint and trying to code and saying, could you fix this other bug? And then that engineer has to drop everything and reproduce the issue and do everything properly. And we created what we call the escalation engineering team to be the buffer in between our frontline tech support engineers and our engineers in the back room to reproduce the issue, validate the issue, say, oh, yeah, there is something going on with this one server in this one data center. And hey, engineers, SRE team, can you go reboot that server? And there are more. Yeah, and I've gotten so many compliments on the escalation team that we just started January 1st from product and engineering and SRE and network. I get those weekly and pass them along to them that they have been such a big help where before we focused on break, fix, solving tickets or feature requests, and there was no one to bridge the gap. And they've really been there to take a ticket, really deep dive with an engineer and give them the info they need so that they can prioritize and fix those problems. And that's been huge. And we're going to keep expanding that team for sure. Yeah. Yeah, these are the smaller teams that have sort of grown organically within our team out of their need. We have another one, the support operations SOPS, which we've already mentioned, developed helper bot, developed the ticket submission form, the chat widget and chat bot we have running, a bunch of our tooling that we use and countless other little services that we're responsible for. We have an escalations engineering team, we have a training team, we have a product specialist team. We've also decided to make, you know, I mentioned earlier that the tech support engineers, which is the bulk of the team, are extremely technical. We decided that, hey, we'd really like to have a little bit of a more diverse team, and maybe we can bring on people that maybe have slightly less experience in their career, and bring people on a little earlier. So we created a customer support agent role as well to try to allow us to lower the bar a little bit to tech support engineer, bring in people that we, and now that we have a training program, bring in people that, you know, that we can, you know, eventually develop into full tech support engineers. So lower the bar, I'd say move the bar sideways. So we got people with a different skill sets. And yeah, we need those too. Yeah, maybe just with less industry experience, right, that don't have five years of tech support experience at a tech company yet. You want to be able to allow people who just came out of college with a computer science degree or something. Yeah, customer support is one of the best ways to, especially at Cloudflare, to learn everything about Cloudflare's products and customers and features. But at most companies, it really is, if you can solve any problem that's thrown at you during the day about any of our products, you can really use that knowledge to do anything. And so we've had the mixed blessing of people coming in and really excelling at support and really being great people, people having those soft skills and then learning those technical skills. And they know more, you know, I would argue more than almost anyone at Cloudflare about the breadth of all of Cloudflare's products, because they do have to be ready for anything that comes at them. Whereas there's some of the smartest people you'll ever meet at Cloudflare on other teams who are really focused on their area and their products or their marketing, you know, area. And in support, they learn everything. And so over time, we've had the mixed blessing of people moving on to other teams at Cloudflare. Everyone in the company kind of recognizes, oh my gosh, they're really good at all kinds of things. I can use them. And we've had people move to almost every other team in the company. SRE, product, engineering, SE, you name it. Yeah. And so then we have to go back to the starting board and train new people, which is why you're here. But I also see it as we have representatives of support and the customers on every team in the company. And so I can go to engineering and talk to former support people, and they know the pain of the customer. They know how we do things when we try and troubleshoot and solve tickets. And they'll listen. And it's really, I've never worked anywhere where the support team has such a level of respect within the rest of the company. And people really do listen to us. They come to us for advice. We have a shadowing program where we advise everyone to sign up and come. If you're in marketing, come sit in support for an hour and sit with an agent and ask them questions and say, hey, I'm in marketing. I'm really interested in how people sign up or what do they do in their first 30 days or whatever marketing people are interested in. How did that campaign work? How's that new bot management product? Are people using it? And they can come sit and ask questions and really look at customers' words. We send out weekly metrics to the entire company, and it gives them a kind of high level overview of are tickets going up or down? Is customer satisfaction going up or down? But it also puts in front of them the real words of customer feedback of what they really don't like and what they do like. And I love that being put in front of every employee every week. And then I get feedback from every week about that. I really love seeing this. I had no idea that customers were saying that. And it's really important. And I love that people take it seriously. Yeah. And it's important to build relationships internally, right? We work really closely with the success team on the enterprise accounts. And we work really closely with the solutions team and giving them the opportunity to come and spend time with us, get to know what the customer experience is like, get to know what the customer support experience is like, builds the relationships with other teams so that then we work together better ultimately. And we've had, yeah, we've had loads of success team members spend time with us and solutions and engineers and product. And it's really great. And I want to, I guess I'm going to kind of segue us a little bit. We were talking about follow the sun and being all over the world. You want to talk a little bit just for a short while about, you know, providing support in other languages, which I know is a big goal of ours. Sure. Absolutely. So follow the sun. Just give you a brief picture. Every eight hours we hand over from one region to the next. So at 5 p.m. Pacific time, we have a video handover between the US team and the Singapore team. And there's an on-call person in one and an on -call person in the other. And we'll tell them what's going on during the day and keep an eye out for this, or we need to report on that. At 1 a.m. Pacific time, the Singapore team will hand over to the European team. And then at 9 a.m. Pacific time, it'll come from Europe back to the US. And so it's, I love the video part of it because we get to see everyone and everybody gets to build a relationship between Lisbon and Austin, between Singapore and San Francisco. Yes. I like the actual document that we pass over, right? We have a, we have a wiki page that we create every day with our handover notes to the next team. I wake up and I, one of the first things I do is open the handover notes. It's like reading the paper from the day before, like this customer needs this and this customer needs that. And we resolve this issue. And there was this other issue that came up. And so it's like every day I can kind of grab my coffee and pick up the handover notes and see what happens. That's very, that's very nerdy of you, Shane. Very nerdy. Well, I like documentation. You know that. Guilty pleasure. It is, it is great. There's always like 10 announcements and you can really get a snapshot of what's going on with the company as a whole every day, reading that. You're right. It is like a little newspaper. So yeah. Along with the handovers, like you're saying, we hand over to the Singapore team and we have a great multilingual team over there. Multinational, multilingual team here in Singapore. Yeah. We have 90 people on the support team today and there's people from 20 or 25 different countries and people that speak about 20 different languages on the support team. And so it's been a big push as the product team has built dashboards in German, Japanese, Mandarin, on top of English. They're working on some other major languages right now and we have a steady stream of dashboards coming out in those languages. We've in conjunction been building up our language support. And so we started with the knowledge base. And so if you go to the knowledge base support.Cloudflare.com and click on the upper right, there's a pull down and you can see most of the top, you can see all of the top articles in French, German, Spanish, Portuguese, Mandarin, and Japanese and English. And that's been a huge help because we have millions of customers all over the world and we give away our product for free. And so we get tons of people from every continent, from every country. And we're really popular in Latin America and Africa and Asia and Europe all over. And a lot of people speak English and we kind of default to that because we're from the US, but a lot of people don't. And a lot of people do, but it's so much easier when you can see the details in your own native language and be confident that, oh, that's what they meant because any little not or but can be confusing in any language. So we started with a knowledge base, also our macros, which is what we call our email templates, which is what agents send very similar to helper bot. We have a known answer to all of our basic things. If you want to change your password, there's simple instructions on how to do that in eight different languages. And so we can tell you, you know, in Mandarin, how to change your password. The next step is people. So over the past couple of years, we've been hiring a lot of people who speak a lot of different languages. And we have eight Mandarin speakers on the team, five Spanish speakers, a bunch of Portuguese speakers since we've opened the office in Lisbon, which has been great. A bunch of German speakers since we opened in Munich. And so we're kind of building up that expertise over time. We have a strong demand due to our big partnerships in China with JD.com and Baidu. We have a lot of demand for Mandarin speakers, not just in mainland China, but all over the Chinese speaking parts of Asia. We have a big demand in Korea, Japan, everywhere. So we're trying to meet that need. Helper bot is also a system that tags all the incoming tickets. So I can look at reports and see how many Portuguese language tickets we got last week, how many Mandarin, how many Spanish, et cetera, and really see that demand. And I definitely hear the demand every day from salespeople. When are you going to have my language or this other language? So we're hearing that and we are answering tickets in a bunch of languages today. So that's been really big. It's really helped us become more of a global company. Yeah. Yeah. So out there, if you're listening and you speak languages other than English and you're really interested in helping to make a better Internet and you'd like troubleshooting computers, contact us. Yeah, definitely. And I think we're doing a bunch of sessions on Cloudflare TV in different languages too, which is really exciting. And it sort of follows a lot of things at Cloudflare. It starts with the sales team getting customers in different languages. And then there's the success and solutions engineers who help set up that customer and manage their accounts and then support. So if we go to whatever country, if we opened up in, I'm trying to think of a neutral example, in Thailand, we opened an office in Thailand and sold to a bunch of Thai customers. We would have salespeople, success and solutions engineers who spoke Thai. And we do. And based in Singapore and Thailand. And then if we have a critical mass of customers, we would hire a bunch of support people who spoke Thai as well. So I look forward to that. Yeah. We also have local phone numbers in almost all those countries as well. That's right. If they want to call in support, it used to be that they call the 650 area code, which the Bay Area people know that's Silicon Valley. That's right. And then we heard from our customers who if I'm calling from Germany, or if I'm calling from Italy, or if I'm calling from France, I'd like to be using a local French number. So those numbers, we created those. And then those all come into our phone support line. So I want to segue us to contacts actually. So contact channels and contact volume. First on channels, we already discussed, we have the web form, the ticket submission form, which is smart form. We have email, people can just bypass that and email us directly. People can chat with us if they have a business plan or any plan. And they can call the phone line anytime. And then there's the community as well out there. I want to ask, how many contacts do we see a week? Sure. Yeah, absolutely. It goes up and down. I think I was lying on it, but four to 5,000 contacts per week overall. And that's across all plan types. So roughly half of those contacts are from free customers. And that's great. We love our free customers. We want to help them and get them using Cloudflare successfully. Split across that, I think the majority now come from the ticket submission form, which is great because like we're saying, it adds a lot of value when you submit through there because it suggests known answers. It does diagnostic tests and it routes you to the right person and puts you through a bunch of helper bot diagnostic, a bunch of tagging, which will market. It'll notice a lot of things. So the ticket submission form, we've really built up over time. We get a lot of emails still, but as we push people, it's probably 40% ticket submission form, 30% email, and then 20% chat. A lot of you may not know about the chat because it is available only to our business and enterprise customers. So it's, Oh, I was wrong. So about 15% are chats as of this one. All right. It's in the ballpark. And chat is very popular with those. And it's great for small specific questions. It's not great for, you know, long form. How do I configure this really complicated setup? Oh yeah. Try pasting a big code block into chat. It doesn't format very well. Exactly. It's just like when you're chatting with any other company, you want to ask a quick question and get a quick answer mainly in plain text. But yeah, API there on that chart is the ticket submission form. And that's because we built it ourselves using the API. So have you noticed that we've gotten more contacts over the last few months, you know, with everyone, a lot of people in the world needing to use the Internet a lot more? Sure. So March was our biggest month ever. And as a Internet infrastructure company, everyone uses, well, the Internet has always been important for the past five or 10 or 20 years, but now more than ever. And so as a infrastructure company that does a decent percentage of all traffic on the Internet, we've been discovered by a lot of people and people are already using us or using us more. So we saw our busiest month in support in March and it's gone down a bit in April and May. So it's, people are getting used to the new normal of working from home and we have a lot of products that help you work from home, like Teams is a VPN replacement, which allows you to authenticate and have secure traffic from outside of a physical box. A lot of Cloudflare's offerings are replacing things that used to require an expensive box 10 or 20 years ago and putting that in the cloud, but giving you all the benefits of a completely secure connection through our edge network. And so people are discovering that, people are needing that more, people are using it more, people are just spending more time on the Internet. And so our slower months, my favorite month of the year is December. A lot of people, a lot of our big partner customers have code freezes, they go on vacation, things slow down. And so every year, we kind of go into a valley of tickets and December, and then it jumps back up in January. We've been growing nonstop for 10 years since we started. And so I look at the graphs of customer ticket volume, it'll go up and plateau and then go up again and reach a new plateau. And we've been really lucky that we have not gone in a linear growth so that for every new customer, we get X number of new tickets. We've really been able to be smart about the growth and give people resources to help themselves build products that are pretty good and that don't break at the rate products did six or seven years ago. That's a bad mouth products from seven years ago. Things get better. This is another fun one in terms of, we get thousands of contacts via the channels that we've mentioned. And Otto was mentioning December is a slower month, and that's when we can catch up. But it's also fun to look at what time of day and what days of the week that we see a lot of contacts. This is in Pacific time. The busiest time for us tends to be at around, you can see 8am, 9am, 10am Pacific time, which is when the East Coast of the United States starts to wake up and starts to use the Internet. Yeah, yeah, this is this is really important for us to hire and schedule correctly to schedule our meetings so that we don't maybe have meetings at that busiest time of day. Although actually think about we do, we do have meetings then. But yeah, the graph on the right will show you a Pacific time. The midnight Pacific time is when the, you know, the European team is working, taking over. And it's, you and I have both been to the London office a few times. And you can see in the morning, it's kind of nice and slow. And they check in, they have their coffee, they make sure everything's running okay. Sorry, you're right. Well, some have tea, some have coffee, you're right. They brew a pot of tea, they make sure everything's running okay. And then they can maybe work on projects or have meetings or whatever, go to lunch at the nearby market. And then it starts getting busy in the afternoon. And they really see this and every single person on the team will tell you this, like, Oh, yeah, by 2pm in London, it's really busy. And I can't do anything. I've got to really keep my eye on the queues. We put all our tickets into queues. And there's a queue for different areas. And yeah, based on the topic, we have a billing queue. If your ticket is about billing stuff, we have an enterprise queue if you're an enterprise customer, etc. Yeah, so we can talk about that more. Sure. But then throughout the day, we we had an office in Austin about three years ago, I think. And that was a huge help because those hours before 9am pacific time, before we started with an office in San Francisco, London and Singapore, five years ago, those were our three big offices with support in them. And when we added Austin, it was really huge help because of those hours that you're just highlighting our busiest hours of the day. Now we have Austin support engineers come in at 9am Austin time, and they can really jump in and help with that huge amount. It used to overwhelm the team on the busiest days of the week. And that's the other thing is that those days of the week, everyone on the team will also be able to tell you Tuesday, Wednesday, Thursday are, you know, it can vary which one of those three days. But yeah, that varies sometimes. Yeah. Yeah. And but those are always the busiest. And it's a natural function of our customers and other companies. People, you know, that's when they do their Cloudflare work when they're configuring things when they're using us the most. On the weekends, they're not. And on Friday, they're kind of doing other things. And so it peters off, you know, late on Thursday. So that's where we have to schedule people really well. And so if you're a support engineer in San Francisco, for example, you might work one weekend a month. And that we just have a few people working on the weekends because that volume is so much lower. But we do need to be there for incidents for urgent under attack issues for customers. And we prioritize right at that point when we're on the weekends. Yes, there are fewer contacts from customers, but the ones we do get, we make sure we prioritize our enterprise customers and we prioritize customers that are having urgent issues, right? If their site is glitching, if their customers are seeing errors, if they believe that they're under attack or, you know, they're getting anomalous traffic, then we're gonna prioritize. And we do that during the week, too. We don't just do that on the weekend. Like you mentioned, we have all of our customer contacts, which we call tickets, segmented into different views in our CRM, which we call our queues of tickets. And you mentioned some are billing related. We put those in one queue. Some of them are enterprise customers, which we, of course we do first come first serve. So we work on, you know, the ticket we got first, we work on first, which is fair. But if you have a more urgent issue than me, then your ticket's going to get worked on first. So if your site is offline, we're going to work on your ticket before we work on this other person's ticket who just has a question about how load balancing works, right? We prioritize and we have to, because as you saw, we get 5 ,000 tickets per week via these contact channels and we have 80 people, right? But that's total global, 90 people global. So we only have, you know, 30 people working per geo, you know, with our follow the sun model. And of those only 25 are tech support engineers. And of those, a few are on vacation and a few might have appointments. And so we really have to be thoughtful and mindful and get all the data that we're getting around when tickets are coming in, what are the priority issues that we're making sure that we're working on the most important customer issues first. Yeah. And I think we've gotten a lot better and a lot more methodical about that in the past year. Cause you have to look at what you just said, you know, several different angles of, of data, you know, who is the customer? What is the priority of their ticket? We call it a P1 or P2 for urgent or high. If it's a P4, P5, that's just an information question. Like, Hey, I was wondering about this, not urgent. Just tell me what you know about this thing. And then how old is it? You know, did they write us in the past five minutes or two days ago? And when you put all this together, we have an SLA and where you can, if you're a support engineer, you can look at a queue and see this ticket is approaching its SLA of four hours for this P level and this business plan level and this level of how old it was. And so that gives us, yeah. And that allows us to look, it used to, there'd be just a big lump of a hundred tickets waiting to be answered. And you would feel that pressure of, Oh, I auto have to answer a hundred tickets all at once. It's like, no, there, this is more important. This person is under attack and they need an answer in the next five minutes because their side is down and they're losing money. This person is just asking, Hey, I heard you do bot management. Tell me more about it. Yeah. That's less, less urgent. That's a good segue. So another one of our talking points is what customers contact us about, right? We've talked about how much they contact us and what channels they contact us on and how we handle those contacts, but what, what do we see them writing in a lot about? I mean, you definitely get a lot of the normal account related stuff. I lost my password, which we've almost entirely automated. And those types of things, we try to automate those account lines. I need to, those basic ones, I lost my password and billing ones where they want to talk about their subscriptions and stuff. But what are the other sort of top things that a Cloudflare customer writes to us about? What does the support team do? Yeah. One easy way to look at it is what are our biggest, most fundamental products and then what on a website, you know, breaks is more likely to cause issues. It's almost always DNS. Yeah. Yeah. Yeah. Yeah. So we're one of the largest managed DNS providers in the world I would say the largest, but I'm not positive. So I don't want to make any claims, but we do a lot of DNS requests every day. And so that's, you know, always been one of our top issues. And it's, you know, if you have a whole large company full of network operations, DNS is like one, two, three for you. If you're a mom and pop, you know, websites, DNS can be kind of weird and complicated. And so everybody's got a question about that. And if it's not working, it breaks everything. Right. So if your DNS records don't resolve, your website's not going to, people are not going to be able to find your website. So. Yeah. Yeah. And you can set up, yeah, you can set it up correctly and take down your site. And what happened? I need support. Yeah. Yeah. So we end up seeing a lot of questions about how to configure, you know, my 10,000 records correctly. And I mean, the DNS engineering and product team has done amazing work recently to, you know, give us better scanning. You know, when I started to scan my domain for all of the DNS records, that's gotten way better. The actual interface itself has gotten better. We've added support for more types of records. I mean, there's loads, I can't even name the amount of features we've added just in the short three years that I've been here to make our service better. So the fact that we see a lot of things about DNS speaks nothing to our product, but more just to the, like you're saying, the fundamental nature of DNS, if you don't get it right, it breaks everything. So we see DNS, of course, we see a decent amount of SSL related issues too. Yeah. Yeah. I'd say SSL is usually number two. And that, again, is one of our fundamental products. We, several years ago, started giving away SSL certificates for free, which was a major. Encrypting the Internet for free for the whole world. Yeah. Yeah. That's not an easy engineering challenge. It's a fun engineering challenge to try and solve. Right. You know, shout out to Lee Allaway, the third co-founder and other numerous people that try to encrypt everything. Absolutely. Yeah. Lee and our engineering team saw that it was a fundamental benefit to have everything encrypted on the Internet. And so it used to cost $500, $1,000 to buy a certificate to get the little green lock on your website. And over through business and through engineering, we've been able to give that away for free. And so when people discover that, every, that's why I always say everybody who has a website should be using Cloudflare for free, at least, even if you don't want to give us money, that's fine. It's going to make your site more secure and faster and better. And that's one of the fundamental reasons is SSL certificates. So we get a lot of questions about that. It's, it can be very easy just to turn it on and it works, or you can have a very sophisticated implementation that is, needs our help. So it's, it used to be our number one, I would say, you know, two years ago, three years ago where we had set it up and there was such high demand, there was pipeline issues and there would be delays. So when you try to- A day, a day. That's a lot of SSL certificates to provision at once. And so that's, you mentioned the pipeline, the SSL engineering team has done amazing work to build entirely new pipelines and reduce the amount of issues that we see, but it's still every now and then it goes wrong. And it's another one of those things when the SSL isn't working, then whoever's browsing your website gets thrown in there or they get, you know, Chrome and Firefox and the other major browsers out there are now starting to, you know, tell people when you go to an insecure website, Hey, be careful about entering your credit card on this website because when you enter your credit card, does your credit card number is going to travel on the network of the Internet unencrypted. And I'm glad for that. And so it's just kind of an amazing challenge of scale. And, and so yeah, that's another issue we tend to see. Yeah. And a big shout out to our security engineering team. They've done a fantastic job of creating that, that pipeline is what we call it over the past couple of years. And it's a really great example of listening to the voice of the customer and listening to our feedback. And for the most time, it worked great, you know, two years ago and very seldomly there would be a backlog and people would have to wait an hour for their certificate for their free certificate. I must emphasize, and we targeted that and we identified that as an issue and we've cut those tickets in half, even as we've, you know, doubled, tripled the number of customers we have. We get half as many tickets as we did a couple of years ago because of the security engineering team and the support team focusing on that and customers love it. So, so yeah, so DNS, SSL, CDN page caching type questions are always popular. So that's kind of our third biggest area of products is making your website faster, caching your static assets and how that works. And so, yeah, that's one of the things we, and you can just turn that on and it works pretty well and gives you a lot of benefits. But once you see those benefits, then you want more and more and more. And you see that it's saving you money by not hitting your origin as much. And it's speeding up the download of your website and, you know, far away continents. And so then you want to customize it. How can I, how do I do this? How do I, does it work on this type of, you know, asset? And so we get a lot of questions around those. Let's say, and then just in general, just everything on the dashboard, like how does this work? How can I customize this? I'm using this a little bit, but I want to be smarter and use it more. And so all across the dashboard. So that's one way that we kind of categorize tickets is we did a whole taxonomy project to be smarter about what area is this ticket about? And then you get the customers who write about five different areas at once, like, tell me about this, tell me about that. And so you got to help them too. Yeah. Yeah. One of the other types of issues that we get every now and then, which I always find really fascinating are the under attack tickets. And I remember when I interviewed at Cloudflare three years ago, I got to attend one of our company, full company meetings that we have every week. And we have various presenters at these meetings. And one of the presenters at the time presented on an attack that we had seen that last week. And of course it was anonymized. So no one could see which customer it was, but they kind of dove into the traffic and the protocols and how we're able to stop the attack. And I just thought that was really cool. And I was really excited to work at a company that did that. I mean, these tickets are fun. They can be very fun. I mean, they're high pressure and it's not fun if it's your website that is under attack, but these types of tickets come in and customers are reporting seeing some sort of anomalous traffic to their website. And our tech support engineers get to kind of dive into the data that we have. Our customer support engineers have, like you mentioned earlier, we're trying to provide as much data as possible to our customers and it's getting better and better. I mean, the analytics team and the various engineering teams are releasing every week. It feels like more data to the customers. But of course, as you would imagine, someone that works at Cloudflare has access to even more. And so that's our tech support engineers. And so they get to pull these logs, analyze the data, and try to help the customer mitigate these attacks. So that's another type of ticket. It's not the most common type of ticket that we get, but it is one of the more interesting ones that we see every now and then. Yeah, definitely. Definitely. And we have built up a team of experts on how to defend yourself from DDoS attacks. And I just pulled up, when you mentioned that, I pulled up the public-facing records, because I want to be sure I'm talking about public-facing things. But Q1 2020, Cloudflare blocked an average of 45 billion cyber threats per day, which is crazy. Yeah, this isn't just DDoS attacks. We're talking about cross-site script injections, SQL injections, you name it. And we also recently released Magic Transit, which is a layer three and four DDoS protection, which is a whole other animal really. And that's been another thing that's been really exciting to tackle as a team, how we can retrain our team. We were great at HTTP level seven, mitigating SYN floods. And then we had to learn a whole new language when we learned about GRE tunneling and everything else that's happening on those layer three and four. Yeah, it's a spy versus spy battle, where as we get smarter, hackers and botnets try new things. And those 45 billion threats per day, there's not a person blocking those. It's we've built up the intelligence over time, where we have thousands of rules. And that's the whole secret sauce of Cloudflare, is you put your network behind us, all your traffic goes through our edge first. And it knows through all this experience and seeing all these requests that this is bad traffic, and this is good traffic. And so there's not someone manually blocking those every day. We have really sophisticated rules at the edge to do that. But then on top of that, some stuff does get through due to various things. And you do need to talk to a human, or sometimes the customer has it misconfigured, and they're not using all the tools we give them. But they need help. And so those can be the more exciting ones, get your blood pumping, your adrenaline pumping. I can see this attack, and you see these crazy graphs where the number of requests just went through the roof starting at this point in time, and now they're getting 100,000 times the requests they normally do. How can I help them? How can I block that? So those are exciting. Yeah, cool. Well, I want to be conscious of time here. It's been a great conversation. We have about five minutes left before the next show on Cloudflare TV. We could do a little bit of recap, or we could talk a little bit about one of our other talking points. We could mention for a few minutes some of the metrics and KPIs that we look at as a support team internally. We look at, of course, how many contacts we're getting. We look at how many comments that our people can make per hour, or per day, or per week, or per month. We look at our customer satisfaction. Customers get given a survey after they interact with support, and we ask them for their feedback, and they can rate it. This is an industry standard. Yeah, customers, please fill those out. Every ticket that gets submitted, we do send you a survey after, and we do read all of them. We look at those numbers. We watch them really closely. We use it to calibrate HelperBot. Was that really the right answer? It makes HelperBot smarter every day. It makes our whole team smarter. Like I just said, we send those comments to every person at Cloudflare. If you put in something, be nice, be fair, be honest, and that's going to go to the CEO, and to the head of engineering, and to Shane and me, and we're going to read all those. We're a very data-driven team, and we tag everything. We have a lot of data on what the customers are writing in about. We're able to slice that up and see what do we need to improve internally. Can we go faster here? Can we get more efficient there? Do we need to do additional training on this product because we haven't been getting great CSAT customer satisfaction when we answer questions related to a certain topic? There's a lot of metrics that we look at all the time. Yeah. I guess a couple of things I'd want to plug, if we just have a couple of minutes, are the knowledge base and the community. Again, this is the knowledge base, support.Cloudflare.com, is the collected intelligence of the whole company. These don't just come from us off the top of our head. We've been a lot more sophisticated the past couple of years, and we have the product managers themselves writing articles about their new products. We have a really great technical writing team and video team putting together really great content, and it's gotten better and better over time. I'm not trying to deflect people from talking to us. If you're under attack, come talk to us, but a lot of the stuff that we're going to tell a customer is there in the knowledge base in a much nicer format than I'm going to say with bullets and links and videos. Take a look there. There's really great stuff there. Support.Cloudflare.com. In six different languages. That's always a benefit. The other is community.Cloudflare.com. The community has been going two or three years, maybe three years now. Yes. We have an amazing group of passionate users. We have 15 or 20 MVPs who are amazing people who know more about Cloudflare products than you or I put together because they've looked at thousands and thousands of posts and really dug in and helped people all around the world on everything. This is a great place to put in your strange edge case that you're using an off-brand flavor of Linux on your server and you're doing something really weird and pop it in there and you'd be surprised. There's another Cloudflare customer in there that tried to do the same thing with that. Yes. We have millions of customers with every combination of software using on their websites and you'll find an answer. It's really amazing when you pop in there and really friendly people who get excited about, oh, I can't believe we just blocked this attack or I can't believe this worked so well or I'm upset that this is broken and somebody helped me fix it. It's really great. Well, cool, Otto. It's been great to talk to you about customer support. Good to see you. I haven't seen you in person in the office in months. I miss it. I know. I can't wait to get back. We sit right next to each other. Yes. We'll sit next to each other again sometime, except on Zoom. I'm missing my bike ride into the office now. Yes. You just ride your bike in circles now? Yes. Still ride, but it's every day back and forth. That was my main thing. Well, our time is at an end. Enjoy all the viewers. Enjoy the next segment of Cloudflare TV. Yes. Good to see you, Shane. Thanks, everyone. You too. Bye, Otto.