How Cloudflare Built This

Today I'm joined by a very special guest, Matthew Prince, co-founder and CEO of Cloudflare, and we're going to talk about how Cloudflare built Cloudflare. Very meta. It is great. Yeah, and for those of you that haven't watched previous weeks, I'm Rustam. I'm a director of product here at Cloudflare, and excited to talk to Matthew today about how all of this sort of came to be. This is the background. It's when you were a big rock climber, and that's from a hiking trip in Alaska. Yeah, this was last year. The only reason I have this up is because my living room is less studio ready than a grand vista in Alaska. That looks nice. My laptop's not powerful enough to do the virtual background, so I've got this random wall. It looks pretty good. It looks better than my living room, that's for sure. Cool, so in general, before we've been diving into how specific products or whatever were built, and the decisions that were made that went into that, I've been just starting with what did you do before Cloudflare, and what led you to join the company, and in your case, start the company. Yeah, so we don't have to go back into sort of prehistory here, but what did your career look like before starting Cloudflare? Sort of a circuitous wandering path that frustrated my parents deeply. So I went to college thinking I was going to study computer science, because I was pretty good with computers, and then decided for reasons that don't make any really good rational sense that instead I would study English literature. Despite that, I had job offers at the end of college to go work at, and this was in the mid-90s, and I had job offers to go work at companies I thought weren't going to go anywhere like Microsoft, and Yahoo, and Netscape, and I said I don't want to sit and write computer code the rest of my life, so I applied instead to law school, and about a semester in, because I thought what lawyers did was what Perry Mason does on TV, and what I realized was lawyers are just like computer programmers. They spend their entire time sitting in front of computers writing what is effectively code, but they don't have a compiler or a debugger to see if it works well, and so I went to, between my second and third year law school, to work for a law firm out in San Francisco, and I was working as a securities lawyer, which is the people who help take companies public, and it was the summer of 99, and it was so much fun, because we were just taking company, after company, after company, after company public, and got an offer to go work there, and went back to school, and thought I'd be all set, and then the first dot-com bubble burst, and the law firm called and said, good news, bad news, good news, you still have a job, which actually a lot of my fellow classmates didn't, but bad news, we don't need any more securities lawyers, but we've got lots of room in the bankruptcy practice, and it's basically the same thing, which is something that only a lawyer would say, because it didn't feel like the same thing, and so I was pretty lost. I didn't know what to do, and then one of my law professors, his brother was starting a company that was an insurance benefits brokerage, it's called Groupworks, it was basically the same business model as Zenefits is today, and we raised like six million dollars, I was the first employee, hired a team, tried to build the product, made a million just disastrous decisions, and it crashed and burned within about 18 months, but it was so much fun to create something from nothing, that it became really hard to ever imagine that I'd go back to being an attorney or something again, and so for the next, oh my gosh, for the next nine years, I kind of tried to just dodge phone calls from my parents, who were like, when are you gonna get a real job? I did a bunch of just, I was a LSAT test instructor for a while, I signed up to be an adjunct law professor at a local law school in Chicago, and just tried to do almost anything else. I finally started a company called Unspam Technologies, which was, I mean basically, it was just a research project that the next thing you knew turned into a company, and that's where I hired Lee, who ended up being one of Klaffler's co-founders, and we'd hired him straight out of college as just this young, smart engineer, and the Unspam product was really simple, it was basically a do not call list, but for email, and we would work with state governments to get it implemented, and we had this great small engineering team, and there wasn't that much engineering that had to go on once you built it, and so we would do these interesting fun side projects, so we predicted the winners of the Sundance Film Festival in advance using Bayesian statistical analysis, that's actually how I met John Cumming, our CTO. We built a browser plugin that would allow you to swap your Google cookie with other people's, and the idea was if you just put enough noise in the system, it was impossible to pick signal out, so it made any kind of targeting hard, and then one of the other things we did is there was a conference called the MIT Anti -Spam Conference, which was run by Paul Graham, who later went on to start Y Combinator, and he would invite people to come out and give talks at MIT on how to solve what was sort of this interesting machine learning problem at the time, which was how to defeat spam email, and I got invited, and I needed to come up with a talk, and so I went to Lee, and I said, hey, can we build a system that tracks how spammers operate online, and Lee was like, yeah, I think if you build the front end, I'll build the back end, and so he went away and built the back end of the system, and I basically went to what was the hot startup at the time, which was LinkedIn, and, you know, right click, view source, select all, copy, you know, paste, and so the project Honeypot, which is what we ended up creating, UI to this day still has references, if you look in the source code to LinkedIn, all over it, changed the colors a little bit, and that, and gathered a bunch of data on things that nobody knew the answers to before, like how long does it, from when your email gets harvested to when you get your first spam message, how long is that interval, and, you know, does the same spammer, which is sending out solicitations for, you know, Viagra spam, are they the same ones that are sending out for diploma spam, and so we kind of collected all this stuff, and I gave the talk, and then we kind of forgot about it, and what happened in the meantime was about 100,000 people signed up for Project Honeypot, and we expanded it to be able to track more than just email scrapers, and, but it was just this hobby that was kind of over in the corner, and I'd taken a sabbatical from Unspam to go to business school, which is where I met Michelle, and, you know, Michelle was sort of the perfect kind of yin to my yang, or, you know, whatever, where, you know, what I, what she was really good at was the opposite of what I was good at, and, and so I was trying to answer. So, like, how did you figure that out? Like, you know, business school is a funny thing, where, you know, we were at HBS, and you take all of your courses the first year with the same 90 people, and by the end, you can finish each other's sentences, and professors would play games with that, where they would be like, you know, Matthew, how would Michelle answer this question, and you can, you actually, before somebody even gets called on, you know exactly what they're going to say, and, and so, you know, you get to know each other super well, and she was, which was unique for, you know, business students, that she had sort of a sense of ethics, and right and wrong, and, and also, you know, had almost zero vanity, and, you know, she was, she was really just looking for what the right answer was, not trying to score points, and, and so, and I, so I was trying to convince her that we should start a company together for, gosh, almost a, almost a year, and finally, I, I was, I was telling her about Project Honeypot, and she's, and, and she was like, that's the idea, we should use that data, and the original plan was, we were going to build, you know, firewall boxes, and we had gone on a school trip together out to Silicon Valley, and, and, you know, I think Michelle, it was, it was, it was 2008, 2009, and, and so the, you know, the economy was falling apart, and job offers were getting pulled, and she had worked as a, as an intern between her first and second year at Google, and it was the one year Google didn't extend any offers to any of their, their summer interns, which I'm, I'm grateful for, and, and so you, we were just talking about this, and, and, you know, again, I think we started out with the idea of let's build a box, and use the Project Honeypot data in order to make it smart, and very quickly, we realized, you know, the, the boxes are going to go away, and, and that, that's, that's how we, that's how we came up with this. How was that clear to you, right? Because that was, this was, what, 2008, 2009, AWS, you know, earliest forms were a year or two before that, the cloud was not here in any big way. What, what led to that intuition? You know, the SaaS, so Salesforce, and, and NetSuite, and, you know, there were those companies, and it was clear, that was, that was right about the time, you know, that was 10 years into Salesforce, and it was right about the time that it was clear that the world was shifting in that direction, and so we, I don't think we saw AWS, but we did see, we did see that. I think the other thing was that, you know, we, when you're starting a company, you kind of think about, like, what's the unfair advantage that you have that somebody else doesn't, and we had the data from Project Honeypot, but we also had all the users of Project Honeypot, and, and we had, you know, a relationship with them, and by and large, you know, they weren't running, you know, their own infrastructure. They were on, you know, shared hosting that was out there, and, and we were like, well, if we build a box, and we've got this mailing list of 100,000 Project Honeypot users who have said they want a service like this, if we ship them a box, you know, when they're using HostGator or something, there's no, or DreamHost, there's no way that they can do that, so how can we stick ourselves in, in that, that world, and so even before the cloud, I think, you know, the shared hosting world of small, small providers, I think we felt like we understood a little bit better, and so it was sort of like, how do we, how do we, how do we service that, and I think that then made it clear that, you know, more and more and the world was going to, was going to shift to, to that. Got it, and so, so it sounds like you were, you were playing with different ideas, how to, how to turn this data into, and, and, and customer list or user list into something real. How did that, I mean, that could have gone in a bunch of different directions, whether that's Firewallbox or what became Cloudflare, how did you sort of zero in on, on what ended up getting built as, as the sort of thing that launched? You know, so about the same time that I, that Michelle, because it started as a school project, and it wasn't, in fact, I, there, you know, Michelle sent me an email of, and this is very Michelle, but here are the, you know, six things I want to get out of it, and here are the six things I don't want, and, you know, and it was like, you know, I want to learn how to write a business plan, I want to get better at pitching to investors, I want to build a better relationship with, you know, the advisor that we had, those are some of those, and the, on the things we're not going to do, one of them was, we're not going to actually start a company, like, like, don't, don't, don't kid yourself, and, and she had lots of options of, you know, other, other things to do, so she had a, she had an offer from LinkedIn and a bunch of other, a bunch of other companies, so I, but every day as we would work on this, it just became clearer and clearer that this was, that there was a real opportunity here. I think the thing, you know, the school had given us a little bit of money, because we entered the business plan competition, and they gave us a little bit of money to help, help us, you know, with various things, it was like a thousand dollars, and we signed up for the SurveyMonkey, whatever their sort of base plan was that you, that you could send out a survey, and, and we sent the survey out to a bunch of Project Honeypot users, and, and being good business students, you know, the first eight questions were things like on a scale of one to ten, you know, how would you rank this, that, or the other, but the last two questions were, we had, we got ten questions with the, the base SurveyMonkey plan, and we'd, we'd only done eight, and we were like, well, we have to do something else, and so the last two questions were something like, you know, how does, how does the problem of cybersecurity make you feel, and, and, and something else that were these sort of, you know, fill in the blank, blank questions that were free form for people to answer, and the answers we got back were just unbelievable, where people were like, you know, the content scraping is what makes me believe in the death penalty, and, you know, cyber attackers, you know, belong in Guantanamo, and I mean, you just, there was this incredible visceral rage that was under the surface, and I think Michelle, who, you know, was, you know, today is one of the foremost experts on how the Internet works, but at the time, like, wasn't a part of this community, and hadn't seen it, I think that when she saw that, she was like, wow, there's, there's real passion here, there's an unmet need, those first eight questions were completely worthless, and we threw that away, but we, I mean, we, we won the business plan competition on the quotes that we got in that survey, we ended up raising the first money that we, that we ended up raising the first two million dollars, and our entire pitch deck was basically just quotes from people that had filled out, filled out that survey. Interesting, and, and so it sounds like there was never any, like, moment where Michelle was seriously thinking about taking that LinkedIn offer, or you were going to go do? I was, you know, I, when the, I remember it was, it was in, and I only, I've only said this to her fairly recently, but so we, we graduated from school in, you know, call it, call it June of 2009. We'd gotten a entrepreneur in residence for the summer thing out at, out in, in, in Palo Alto at Highland Capital, so we moved out to do that, and, and she deferred the offer, and I remember, and, and then we raised money in, in the fall, in, in October and November of 2009, and I remember in early 2010, like March of 2010, the phone rang, and the caller ID came up, and it said LinkedIn, and I had this sinking feeling like Michelle was gonna leave, and I told her this, you know, the other day, and she's like, of course I was gonna leave, like, we were, at that point, we were, we were in this, but it was, you know, it was, it was still, you know, it was so nascent that, that, that, that I, I, that I worried about it, and she, you know, had, had lots of, lots of different opportunities, so. Interesting, but you, you were all in on this, you were like, this makes sense, I've wanted to start a company for a while, let's, let's go. Um, my, uh, so, um, so my dad, uh, my dad had a family business, which was, which was restaurants, um, and not very good restaurants, um, I mean, perfectly nice, but like, you know, Applebee's, and Famous Dave's Barbecue, and he ran the, the Hooters in Utah, and, and I couldn't imagine anything worse than running my dad's Hooters, um, but he was getting older, and he, you know, he wanted, he was like, son, it's time for you to come back and run the family business, and so I was, I was, I mean, at some level, I figured eventually that probably would be what I, what I did, but I, I didn't, I really didn't want to, and so I was looking for, you know, anything that I could do that would be, that would be something, something else, and so I, you know, whether it was going back to unspam, or starting, or starting this, or what it would be, you know, I was definitely looking around for something, and, and I, I still couldn't, you know, again, after the, after you've started a company, it's just, you know, it's, it's so, it's so hard to go, and for me, at least, it's really hard to go, you know, work for someone else, I mean, I think that you, I don't understand how you tolerate me, frankly, because like, you've seen, you've, you've, you've, you've launched two, you know, companies yourself, and it's, and that's, there's something that's, that's, there's something that's pretty, pretty amazing about, about, about doing that, although it's, it's not, it doesn't, it's, it's, it's rare that it, it's, there are, there are perks to having a boss too, though, right, that's, that's one of the things I, I took away from the entrepreneurial experience, is like, people always say, oh, I wish, I just want to be my own boss, and that's really nice at some point, but it's also, it's also stressful in all sorts of ways that, you know. Yeah, I, I, I, I'm glad, I, I hope that's true, and, and, and I, and, but, but it, but it is one of those things where, I don't know, I was just, I was never, I was really never good at working for other, other people. So, okay, so, so you have all this data from this spam project that, that seems interesting, you have all this really visceral user feedback that says, we hate people that screw with our websites, how did, how did that turn into DDoS mitigation as the first problem that, that you wanted to solve? Yeah, so DDoS wasn't, I mean, at some level, certainly when we launched, DDoS was not the first problem we solved, and in fact, you can search around the Internet and find lots of examples of us saying Cloudflare is not a DDoS mitigation service, because DDoS was what we were terrified of. Now, now I'll say that I remember really clearly pitching Lee that he, that he should, that he should join this little motley group that we were, that we're starting this together, because he, he had been working at my old company, but he was, he was, there weren't a lot of technical problems, so he was getting bored, and he was getting offers to go work at Facebook, which was, you know, just getting started, and, and, and Google and a bunch of other, you know, companies, and, and Lee was just somebody who I always wanted to have on the team. And, and so I pitched him, and I remember at the end, like, it was like, it was like a 30 minute soliloquy of like, here's how it'll all work. And at the end, he was sort of quiet for like, 45 seconds, and he was like, yeah, that'll work. And you could just, I mean, you could almost hear his mind turning. And at some point, he said, you know, at some point, if we get big enough, maybe we'll get to a point where when somebody is under a DDoS attack, we'll just say, we'll step in and stop it. And we'll talk about, you know, what to charge them after the fact. But it took us, it took us a while to get to that point. And in the early days, we were terrified of DDoS attacks, because we just didn't have a very big, a big network. And so it was a race to how could we get big enough to be able to mitigate those challenges? And how do you have enough customers that sort of makes the entire thing work without getting any customers that are so big, that all of a sudden, you're going to be, you know, terribly embarrassed by, you know, how flaky the service was in the beginning. And that was, I mean, again, it's, that was just this tightrope that we were walking. And there was some point when, you know, in 2013, or 14, where we realized that we'd gotten to a scale where I remember talking to this, there's a guy who runs the edge of Google's network. And I was like, what's the largest DDoS attack you could you think you could stop? And he said, you know, that's the wrong question to ask. He said, we got executives at Google asked me that all the time. And what I say is, we can now stop attacks, they're large enough that if, if it starts to cause us a problem, a lot of the rest of the Internet will be breaking as well, because we'll have overwhelmed, you know, all these peering ports and other things. And somewhere in 2013, 2014, was where we where we cross that point. And today, I mean, I like it would, it would, it would be a very significant attack that would that would cause massive disruption across the rest of the Internet, to be able to take take us down. Why did why do you think that, you know, spam in the early 2000s was obviously a big, big unsolved problem, right? Like everyone on the Internet back then remembers the Viagra ads and all that. And today, that's largely a solved problem, right? Like spam definitely exists, but it's not the daily nuisance that it used to be. Why has the same thing not happened to DDoS attacks yet? Like kind of, well, so I remember really clearly, and in, I think it was 2000. So I must not remember that clearly, if I don't remember the date, but sometime in the kind of 2004 2005 timeframe, when Bill Gates said, spam will be a solved problem in five years. And all of us in the industry were like, haha, he doesn't know what he's talking about. All right. And then five years go by, and spam was largely a solved problem. And what happened in that intervening time was people gave up on running their own mail servers. And they switched to, you know, using, you know, the Microsoft's managed exchange, which is now part of Office 365, or they switched to using Gmail and G Suite and all that. And at the end of the day, like spam was just a problem of statistics. And what you needed was data on who the good guys were, and who the bad guys were. And once everyone was centralized behind a relatively small set of email providers, it became a very tractable problem to solve. I actually think DDoS is largely a solved problem as well. You know, for our customers. I mean, something's gone terribly wrong if there's a DDoS attack that affects them in any way. And part of the reason for that is it's, it's again, it's just a statistics issue. And, and that's been one of the things that's driven, you know, so many people to go behind, whether it's it's Cloudflare, or one of one of our competitors, and, and that the the scale that we have just means that we can continuously get better and better. And so, you know, that's been the thesis behind what we've tried to do at Cloudflare from the beginning, which is that the exact same thing that happened to spam email was going to happen to, you know, initially web and today, Internet security as a whole. And, and, but the key was, how do you get enough enough customers behind, behind your network. And, you know, today, we're up to over 27 million sites that sit behind us, you know, north of 13% of all the web, we're able to spot trends and see and see things and actually use that to stop it. And, and that's why, you know, a few years ago, we were able to say, Listen, we're not we're not even going to charge anything more for DDoS attacks. And our free, you know, our free version of Cloudflare on a fairly regular basis, as you know, stops even nation state sponsored levels of attacks. And my hunch is that, you know, fast forward, a few years from now, and people will think of DDoS the same way that they think of, you know, Viagra spam. Hmm. So we're at 2627, whatever million today. Back back when, you know, clever started, we were obviously at zero, how did how did we go from zero to 100 or 1000? Like, what did that early user growth look like? Yeah, so you know, that's, that's, that's a super hard problem for anyone. What we did was we was we actually reached out to the project honeypot members. And we did it, we did two different things. Um, first was, you know, we, we didn't have any money, like, I, I was, I was totally broke. And, and, you know, we didn't want to, like, go borrow money from our parents and, and, and, and we needed some servers to actually be able to build the first version of Cloudflare. And, and we were trying to figure out how we could get some servers and AWS wasn't really a thing that you could use at the time. And we and we also knew that we wanted to have our own infrastructure. And, and at some point, Michelle, and in, you know, again, this very Michelle, she was like, always talking about how, you know, loyal the project honeypot use community is why don't we just email them and see if any of them have any extra servers we can have. And Lee and I both looked at her like she had two heads. And we're like, that's the dumbest idea. And she's like, well, we just try it. And I was like, okay, fine. So we knew early in Project Honeypot's history for whatever reason, and I actually don't remember the rationale, we'd cut we'd ask people when they signed up to give what their postal code was. And so we looked for anyone who is in a in a with a postal code that was within 50 miles of the Bay Area, which is where we were, where we were entrepreneurs and residents. And we emailed them and said, Hey, do you have any extra servers lying around? And with the response rate, what we emailed like, you know, 200 people, and the response rate was like, 50%, or something. And people were like, I don't but my neighbor does. I'm trying to get him to give it to you and all this. And so Michelle, like she got in her little Jetta. And she drove from place to place to place to place to place, just picking up servers, and none of them worked. But we were able to actually cobble together, you know, two functional kind of Franken servers. From that, the thing that was actually more important, though, was throughout that process. And again, this is very Michelle, she she talked to all those people. And she was like, Hey, we're thinking of building this, what do you think? And they all gave a little bit of feedback that caused us to adjust what our what our what our plan was. And, and those, those, those people sort of became, you was in Chicago. But sometimes we call that we named named pops after airport codes. And so we named it after O'Hare Airport. So it's ORD. And internally, the the servers that are sort of the brains of any pops are called hordes. And the reason why is because the guys kind of Ian didn't know what to call it. So he was just like, Oh, let's call it an order. And because then it's actually from that first pop in Chicago. And so the very first project honeypot users, what we did was we did the same zip code trick where we looked up people who are using project honeypot that were all within, you know, a certain distance of where where that location was, because we didn't want to introduce additional latency, and we only had one location. And so we emailed them. And and again, got this just astonishing signup rate, and we knew what their domains were. And the initial onboarding for CloudFloat, the way it worked was, you'd go to a page, and we and it would say, what is your GoDaddy username and password. And you and you type in your GoDaddy username and password. And behind the scenes, we'd go slurp down all the DNS records from GoDaddy, import them into our DNS system, propagate that out. And again, it was all just in one data center. So there wasn't there wasn't a really a propagation time, and then log back into GoDaddy and change the name servers. And the whole process start to finish took about 30 seconds. And we'd have a little spinner that would be going while it was done and go through it. And the crazy thing was, so anyone who like had identified with project honeypot should be somewhat security conscious. And so we'd email people out, and we'd be like, hey, we you know, we just started this new thing, log in here and put your GoDaddy username and password and we'll let you into the into the beta. And without asking any questions, people would just go do it. And then like, an hour later, they'd email back and they'd say, Hey, did I just fall for a phishing scam? Because I mean, it was effectively what it was. And, um, and it was and there were some things like one time the crawler messed up and it and it was for whatever reason, not copying over MX records, which are the mail exchange records. And turned out that the person who had signed up the only way we had to communicate with them was by email and they and and the email they use was the same domain that they just signed up. And so and suddenly we broke all their email and we had no way to reach back out to them tell them switched over and we hadn't stored their old settings. And so it was I mean, it was it was great. The first 10 people who signed up we we broke their websites in, you know, 10 different ways. And what was amazing was people be like, Hey, you broke it. But you know, here's here's what I think is is going going wrong. And you know, if you just change the change this, I'll go try to try it again. And they were incredibly patient. And it was this really eclectic group of people. And there was one guy, his name is Bob. And Bob, Bob ran a we referred to him as kinky Bob, because Bob ran a site that sold adult toys that were fashioned to look like animal genitalia. And we actually built and he found every bug. I mean, it was the craziest thing. And but but at the same time, like, you don't really want to be staring at these, but so we built actually a browser plugin that would blur all the images, because Bob was always writing in with other stuff. And at some point, he wrote in and, and he said, Hey, um, would you mind writing me a letter that says that I've been helping you debug your service, because I actually this is my sort of hobby project on the side, but I actually work for the giant tech company. And we get credit for providing services as individuals to nonprofits. And there's no way Cloudflare is ever going to actually be profitable, or make any money. So you must be a nonprofit. And so if you do this, I can I can take it. And I was and I'm just thinking, I wouldn't have the guts to tell to tell someone that, you know, my side hobby was was was what Bob's was so but it was strikes me about all these stories, like, you know, lean startup is kind of like, you know, rightfully so trendy book in Silicon Valley, and this ecosystem, and it's, you know, question every assumption and start small and iterate from there and all this. And I'm sure you guys haven't read the book when when you were doing all this, but you didn't have any assumptions of question, right? You were going into this sort of totally cold and just talking to users and asking, like, what can we do to help you and then doing that, right? Like Tom Eisenman, who is our who's our advisor at HBS, I remember Michelle and I were like, we said, Oh, we're not going to build boxes anymore, we're going to pivot, we're going to build a service. And we described how it was going to work. And everyone's objection was, you know, if you had a bump in the road, if you're providing security, you had a bump in the road, you're going to slow things down. And so we became obsessed with this. And we're like, well, it's computer science, you know, one of the tools you use to speed things up in computer science is caching. So maybe we'll just cache stuff. And we'll do all these other optimizations and other things. And we went and described it to Tom. And, and Tom said, Oh, you're building a CDN. And Michelle and I looked at each other and, and we're like, and we look back at time, we didn't want to look stupid in front of our, you know, our advisor and professor. And so we're like, yeah, that's what we're doing. And we walked out in the hall, and we're like, you need to look up what a CDN is. And we had, I mean, we had no idea. And so as a result, I think, you know, we always thought of ourselves as competing with boxes. And if you install, you know, a checkpoint firewall, and you get, you know, zero attacks, checkpoint doesn't charge you less for the firewall. If you install a checkpoint firewall, and you get a million attacks, checkpoint doesn't charge you more. And so we always thought that, you know, having fixed predictable pricing, if you're competing with hardware, you had to you had to line line things up with hardware. And I think that a lot of and at the time, I mean, I mean, Tom should know that CDN space, his wife was on the board of Akamai at the time. And, and I think we would talk to people who are in CDNs, and they were like, well, you have to charge based on bandwidth, because, you know, that's going to be a variable cost. And we were like, well, why does it have to be a variable cost, maybe we can just get a lot of peering relationships and drive that and make it a fixed cost. And, and I think that, you know, when I when I see other entrepreneurs, oftentimes, the people who really are able to disrupt industries are people who are sort of interested in the problem, have sort of a tangential kind of knowledge of it, but aren't totally deep in it. And it allows you to see kind of different different perspectives. I mean, I think that the collisons over at Stripe were are similar, I think that, I mean, it's certainly the Airbnb founders, I mean, that they come from the hotel industry, right. And so that I think is allows people to see different solutions sometimes. And I think that was the case with us. What's crazy, though, is, like, a lot of startups pivot and change directions and do different things over time. Like, if you go back and read the business plan that we had on file at, and it's still on file in the library in, in, in, in Cambridge, like the very first sentence was Cloudflare is a service that makes the Internet more secure, more reliable and faster. And it's like, it's kind of like, I don't think that we foresaw, you know, Cloudflare for teams, I don't think that we foresaw workers, I don't think we foresaw some of the other things. But, but that that core, you know, for certainly the first five years of Cloudflare's life, we were on a pretty kind of straight trajectory. And that was how we and we really didn't pivot. Yeah. What did your day to day look like? Once you were in that, like 1000 users plus things really starting to take off a bit? Were you writing code? Were you? Yeah, yeah. I mean, I did I did some of the Lee hated doing front end. So I did some of the original front end design until we could until we could hire some people who did that. I did a lot for it took, it took until about 2000. Late 2012. Before I wasn't the customer service agent that had answered the most tickets. And, and so I still do occasionally, customer service, swings and, and, and help help that team out. And it's just I mean, that I feel so inadequately prepared to answer people's questions. They've gotten so much more complicated now. And, you know, and so there was a lot of that what what really was was, you know, we might if if there was a problem anywhere on the network, Sri Rao was our very first technical operations SRE role, who's still who's still with us. His phone would go off first. And if he didn't, if he didn't respond within a certain amount of time, mine would go off second. So I, it was a lot of just tech ops, because, you know, once we launched it, just every day, hundreds and hundreds of new sites would sign up. And they kept getting bigger and bigger and bigger and more and more and more complicated and stuff would break, you know, all the time. And so I don't think from 2000, you know, from the September of 2010, through, you know, probably, probably the end of 2013, I don't think I got a solid, you know, more than two hours of sleep at any given time, because it was just you are constantly fighting, fighting fires. And, and it was, it was, it was really, it was, it was hard, initially. Yeah, Jerome Fleury, who runs our network team now, was telling similar stories last week. And it sounds like him and you and others found that experience exhilarating. And I know, I know, some others said, Yeah, this is cool, but but not for us. I mean, it was tough. And it was, you know, there's a there's a case that they now teach it at business school. And it's set at a time when we were like 30 employees and, and, and, and five people quit. And one of the people who quit was, was was supposed to be running, you know, tech ops shifts. And he was like, I can't I got to sleep. And which is I think is totally reasonable. I mean, this wasn't, we weren't, but we just didn't know how to, we didn't know how to like, how are we going to stop like it was, once it was once the ball was rolling, like you couldn't you couldn't sort of you hit oil and the well is gushing. Like, what do you do? Yeah, yeah, what do you do? And so and I mean, I remember being back to help teach that the case at some point, some kid who was a student, you know, raises his hand. He's like, you know, when I was at General Electric, what I would have done is just called down to HR and hired like 30 more SREs. I don't understand why these guys don't do that. And it's just like, we were I mean, because nobody knew who we were. Um, you know, I just in the beginning, you asked what my day to day was, I mean, the thing that Michelle and I spent most of our time doing was just trying to hire people. And it was so hard. I remember there was a guy who who's I won't use his his actual name, but let's say it was like his name was like, I don't know, flower or something. And it was like flower, what's your last name? And he's like, I don't have a last name. My name is flower. And it's like, okay, and he showed up to the interview without wearing shoes. And and, you know, and clearly hadn't showered in a ton of time. And, and Michelle's like, seriously, this is the engineer you found? And I'm like, yes. And we're begging flower. And he's like, yeah, you know, I don't know, I've got a bunch of other offers. I might go here, I might go there. And, and, and he was just like, it's so hard to recruit early on. And it's one of those things that, you know, I think you forget, pretty, pretty quickly, because, you know, once you get to about 50 people that it there starts to be some momentum behind it. But those I feel like I just owe forever, you know, debt of gratitude for those those early employees who, who joined because it you know, it was it was not clear. It's gonna all evidence was it wasn't going to work. So. So it was it was great that they that they, you know, stuck, stuck with us. So you know, it sounds like, you know, early product, found some traction really quickly through through a lot of customer focus, and things were sort of taking off. That product was free, right? You there was no, no. Yeah, we had. So when we so the pre launch, we launched it TechCrunch Disrupt in September of 2010. Prior to that, it was everyone was free. When we launched, we had launched with a $20 a month plan. And, and then one of the other things that was that we thought was going to actually be, at least initially, the primary driver of revenue was we thought that when we put challenge pages up, that we would actually put ads for antivirus software on those challenge pages, because the theory was, you know, yeah, a lot of bots would hit them. But sometimes it would be people who had, you know, actually an infected computer. And if you could say, Oh, your computer is infected, we know it's infected, you know, go buy McAfee or whatever, then then it would then it would, you know, that that would actually be and we can generate, you know, some some revenue from that. And, but and we tried to do deals with all the big antivirus companies, and none of them would talk to us because it wasn't, you know, we didn't have any, any scale. And so we just put up like an AdSense ad. And I remember sometime in early on, all of a sudden, we woke up and we had made like $400,000 that month, from the AdSense ads running on on the on the pages. And we're like, Oh my god, this is really working. And it's really great. It's incredible. And, and then we knew something was wrong when Google was like, Okay, we're gonna not pay that out to you quite yet. And what had happened was there was a handful of we are our bot detection kept getting better and better and better. And we started blocking what were the ad click bots that would go on to sites and automatically click the ads. And so they'd hit the challenge page, they'd see an ad and they'd click on it. And so all of a sudden, there's this completely fraud. I mean, again, we weren't we weren't creating, we had no idea we weren't creating this, but the that the bots would hit the page, they click on the ads, it generated a ton of revenue. And, and then we were like, Oh, that's not gonna work. And so we, we, we went once that once that happened, we stopped doing doing that. And we've never, never really gone down that path, you know, since but yeah. And then like, how, you know, obviously, we started at $20 a month, and then eventually added $200 a month, and then sort of have gone up market and added higher price points since then. What motivated like, what, when was it time to start to add another tier? Yeah, um, you know, why 20? You know, it was what we I was, I mean, it was it was it was sort of, um, you know, I think that we had, we had sort of seen that there was, I think what's somewhat unique about our business is we keep getting pulled up. And the general rule of thumb has been that every 18 to 24 months, our kind of average contract value goes up by an order of magnitude. And that's, and so and that's kind of where where that where that happened. Um, we saw customers who wanted, you know, more largely was support driven, they wanted a higher tier of support. And so we're like, if we're going to staff that, then we should we should charge more for it. And so that was when we added the $200. Why 200? Because it was 10 times 20. I mean, it was so that's how much logic that was behind it. And then I remember it when it was in late 2011. The phone rang, and I picked it up. And, and it was the head of technical operations from Bain Capital. And he's like, we need to sign up for Cloudflare. And I was like, great, go on, you know, sign up, you can do the free version, the $20 mod, the $200 mod, just put it and they're like, we're Bain Capital, like, we're, there's no service that's worth our time. That's where you know, that's only $20 a month or $200 a month. And I was like, okay, well, what do you think you should pay? And they were like, well, we think maybe, you know, we'd be willing to pay like $50,000. And I was like, great sense to check, you're our first enterprise customer. And we'll create a tier of service for you. And that and they they still to this day, you know, are using are using using Cloudflare. And, and so I think that that's sort of how we've kind of marched up, up, up the up the stream around that same time, and someone else came and said, you know, we're really excited about what you're doing. We'd love to, we'd love to use you, but and we'll pay you a million dollars. And we need the following, you know, seven features or whatever. And I remember, you know, Michelle was like, we're not ready for that. And, and so we actually said no, to that customer. And what we said was, we're heading in that direction. But it's going to take us an 18 months to get there. We'd love to check back in with you when we're more ready. But until then, you know, we're not. And they were blown away. And, you know, we, we kind of worked our way there. And we got the features to the point where we were comfortable, and we called them back up. And they were our first, and they ended up being our first million dollar customer. It was just two years after they had initially called us. So yeah, it sounds like both the features that evolved in the product, and the pricing and all that evolving was really customer driven from the start. You mentioned earlier that, that, you know, the first five years of the company really focused on that core product suite that, you know, was in the business plan and that you launched with. How did things evolve from there? And how did the product suite proliferate sort of five years in? You know, I think we kept, again, listening to customers, seeing what, what people wanted, and we kept adding it kind of to those, those basic plans. And for a long time, Cloudflare was all you can eat, like you paid one amount, you got everything. It was probably around 2015, where we started to realize, actually, you know, there's enough complexity here that we should charge based on different, different SKUs. And so, you know, I, I think that that was the case, you know, how I think about the company is, you know, fundamentally, what Cloudflare is, is we're a network, and we can move data from any point on earth, any other point on earth, faster, more reliably, more securely, and more efficiently, meaning cheaper than anyone else. And, and so that's, like, that's what we've built as that core network. And then the question is, what are the opportunities that can sit on top of that network? And yeah, you know, web security was, was a was a good place to start. But turns out there are all these other things that if you have a network like that, that you can add on to it. And so I think we're always looking out and saying, where are there places where you're moving data, and you want to do one or more of those, those four things fast, reliable, secure, or, or inexpensive? And can we help help provide that? And I think that's been kind of the, you know, that's been the the touchstone to what what it is that we're that we've that we've built. And I think that that's how we think about things. And, and there's incremental kind of product development, which we do. But then I think the thing that that I think is really exciting about what we're doing is, we're also saying, okay, what's something what's something that is, is, you know, could be really revolutionary that you could build on top of the network. So I was on a call with, we were talking about, you know, we celebrate birthday week at Cloudflare, which is the week where we launched, and we've made it a tradition that we announced a bunch of kind of really innovative products during during that time is when we when we did universal SSL, which is making SSL available to all our customers, even free customers, it's when we did unmetered DDoS mitigation, it's all of those sorts of things, which are these sort of big change the world products. And john, our CTO, you know, said, you know what I want, I want a Cloudflare SIM card, plug in my phone instantly faster, instantly more reliable, instantly cheaper works anywhere in the world, you know, and just works. And I think there were certain people on the on the call that, you know, almost fell out of their chair, because that seems, you know, so crazy. But But, you know, I think that's, that's the sort of thing that I hope that we continue to be able to do. And I think it's the sort of thing that our network allows us to do. I don't think we'll have it by by September 2020. But I thought you were gonna say that warp is almost a step toward that incrementally. Yes, I mean, I think that that's, that's exactly me. You know, the interesting thing is, you know, Magic Transit, which is one of the products in your portfolio, you're sitting in a Cloudflare office in San Francisco, say, you connect to the Internet over, you know, so your laptop connects to a Wi Fi access point, which connects to a switch somewhere, which connects to an aggregated switch, which then has a piece of fiber optic cable, which is essentially dark fiber that goes from there to our nearest location. And at that point, you're on the Internet, but you're on Cloudflare. If you're then from from the Cloudflare office in San Francisco, you know, visiting a Cloudflare customer that's based in Frankfurt, there's a decent chance that you can get from one side of the Internet to the other side of the Internet without ever touching the Internet. Because we've now built out the backbone that we use for a lot of different routes, we've built out things that allow that to happen. And that then starts to allow us to really dramatically improve some of the technology that underlies the Internet and make it significantly faster. So like you've been working on, can we increase the frame sizes that we're sending across? And the answer for most of the Internet is no, because you know, you can't trust that the path is there. But if we can, because it's either our own backbone, or it's someone that, you know, we have, you know, the direct contractual relationship with, and we know that you're going to be on both sides, maybe you can start increasing, you know, the packet sizes and the frame sizes. And maybe you can really, you know, fundamentally improve some of the underlying ways that the Internet works in ways that I think a lot of people would be, wouldn't even consider trying. So which is... Yeah, this is something John Graham-Cumming, our CTO, loves talking about is Metcalfe's Law, right? The more things connected to a network, the more valuable it becomes. And the more, I mean, the way I've interpreted that is the more within our sort of ability to affect change at a huge scale, right? Not just at a small scale. No, it's, it's, you know, it's, it's been, and it's, it's one of these things that I think still takes us a while to realize where, you know, it's, gosh, it would be great if the browser, if a browser supported this, and, and, and, and I catch myself saying things like that, and someone on the team will be like, well, do you want us to call the Chrome team? Or do you want us to call the Apple team? Like, we've gotten now to a scale where we can actually start to, you know, make things better. I was, I was talking to the CTO of, of, of Mozilla the other day. And, and he, and he said, this is, this is a probably not a perfect quote, but he said something like, hey, are you excited for how much better the Internet's going to get really quickly? And I was like, what do you mean? And he said, well, you know, it's a little scary that there are only 10 companies that matter on the Internet anymore in terms of its, of its infrastructure. But the good news of that is we all know each other. And that's, I think that's exactly right, that, you know, it is actually pretty scary that so much of the Internet is centralized behind a relatively small group of organizations. But, but we can all cooperate and work together to improve some of these fundamental challenges that, that people for a long time thought were intractable. Jerome, last week on your show, you know, talked about, is BGP safe yet? People thought, oh, BGP is this terribly insecure protocol. It's totally trust-based, but we can never get everyone to, to change. Well, it turns out, you know, if you, if you, if you do some things and you prod people in the right way, you can dramatically affect sort of the trajectory of, of improving the underlying Internet. And, you know, that all of the big tier one providers, I would, I would guess will have BGP security in place, you know, by the, by a year from now. And that's, that's, that's the sort of stuff that you put in your obituary. I mean, that's, that's pretty, what did you do, you know, over the course of your career? Oh, I fixed routing on the Internet, right? That's pretty incredible. And, and, and I think it's one of the reasons that I, you know, it's, it's fun to come to work every day. But how do you balance, and how do, how does Cloudflare as a company balance investments in these moonshots and these things that really move the needle for the Internet and the world versus the sort of incremental meat and potatoes feature improvement bug fixes that sort of keep our customers happy? Obviously, the moonshots make our customers happy eventually, right? But there is a kind of tension there. Yeah, yeah. I mean, I, I keep looking for a good answer to that, that question. And, you know, we've been doing these internal fireside chats with people who we admire and, and, you know, it's been, it's been fun. And we had Eric Schmidt do one a little while ago. And I asked him, I was like, at Google, how did you decide, you know, what, what projects you work on? And he was like, and what project and what things you don't? And he was like, well, that was easy at Google, we just wanted to do everything. And I and so I like, there's a little bit of me that's that way too. I hope I but and obviously, he was being a little a little flip in saying that. But it's, again, I think that, that we're in, we're in, we're one of I think, the really rare technology companies that has a market opportunity that's big and broad enough enough that we can keep creating new things that live on top of, you know, this, this fundamental network that we've that we've built. We got a couple minutes left. Last question for you. The cat is angry over there. So I'm at my I'm at my in-laws, and they have two cats that I'm very allergic to. And one of them is, is, is talking to me. So I'm on camera. And you're free to answer the cat's question instead of mine. But what advice do you have for someone thinking about starting a company right now? Like, obviously, economic circumstances are a little nuts, a lot of just general uncertainty. Not that dissimilar from from from when Cloudflare started. What would you say to someone? Thank you. Yeah. Um, you know, at some level, I think that some of the best companies get born out of these times when it's when it's when it's hard. And, and I think that, you know, it separates people who are doing it because they, you know, have a real idea and a real passion from people who are doing it just because it's fashionable. And, you know, struggling to figure out how you're going to raise money, makes you appreciate money a lot more and makes you appreciate that, like, if you run out of it, at a company, you're dead. And so, you know, I think that we've always been very frugal as an organization. And, and I think that that is, that's a real strength that we have. And when I look at other companies that have been super successful, that that frugality is a real, is a real asset. And so, you know, I think just we are going to go into a time where it's going to get harder to raise money, we're going to go into a time where people are going to be more risk averse. If I'm not sure how my wife is, is, is working on on starting a company and just like, it's, it's hard if you're not if you and your co founders aren't in the same room to be able to fight and, and you can't, it's just you can't do it over a zoom call as easily. And, and so I think that that's, I think I think it's going to be hard, but I think some of the entrepreneurs that come out of this time will be the ones that build, you know, the next Google or the next Facebook or the next Cloudflare. We're at time. Thank you so much for joining and see you all next week on How Cloudflare Built This.