How Cloudflare Built this

Welcome to the inaugural episode of how Cloudflare built this. So my name is Rustam and John. My name is John Levine. And we're both product managers at Cloudflare. And we're excited to sort of first host all of you watching. And second off, just sort of peel back the curtain a little bit on how Cloudflare actually builds products and sort of takes ideas from ideas on paper or on a wiki page and turns them into software that our customers around the world use every day. So to go a little deeper here, you know, we introduced ourselves by name, but John, do you want to talk a little bit about what you do at Cloudflare and sort of how you got here and then and then I can do the same. Sure. Yeah. So I'm product manager for data and for analytics, which basically means taking all the data that our customers and users generate on our products and making that useful. So making it accessible to engineering teams to do their jobs and making it accessible for our customers in the form of analytics or logs that we deliver to them. Before that, at Cloudflare, I was a product manager for CDN. So making sure that we got content quickly and in a low cost way all over the world. Got it. Cool. And so I'm Rustam. I'm actually a director of product here and I manage our performance and networking team. So that includes the CDN as a product and a bunch of other things, including Magic Transit, Argo, Spectrum, and some other things, including our data pipeline. And full disclosure, I am John's manager. I promise he is here voluntarily and And I've been here. I've been at Cloudflare for about four years, you know, joined when we were a much smaller company. It's been really, really cool seeing us sort of grow into what we are today. And prior to Cloudflare, I was at a couple early stage startups. I tried starting my own company in 2013 and prior to that I was at Microsoft for a couple years. Cool. Now that intros are out of the way. John, you want to talk a little bit about what we plan on talking about in future episodes of the show and what the general sort of outline is for what the formula is? Yeah, so we want to make this into a series and we realized there's so much we can do to talk about how our products came into existence, like Rustam said earlier. There's so many products, even really small things that had an incredible amount of just thought that went into them. And we do like to share that publicly, you know, in our blog posts or webinars, but we thought we could go much deeper into what that process actually looks like. So I think in this first episode, which is like extra long, we have time, we want to talk a little bit about the meta process. So we're actually going to share like the PRD template that we use. PRD just stands for product requirements docs, pretty standard thing across a lot of software companies, which we'll talk about what the role of a PRD is and what goes into that, which helps drive a lot of product development. And then for the second half, we want to actually talk about a specific feature that I worked on called the cache API, which is a workers feature and talk about how that came to be. And then just do a little bit of a retrospective. I'm like, what do we think went well? How do we think the process worked for that or didn't work? The process that we should have applied or things we should have thought about and how that's doing today and where we think that's going to go. We hope to do this kind of on a regular basis and talk about more products from the small to the very big. Yeah, I'm excited to share all those details with folks. So just to make this super meta, in the five minutes before the show went live, John and I were talking and I had put together a little brief on what I wanted the show to be, who my target customers were, how I knew I was doing a good job, sort of like just applying all the PM block and tackle to the show itself. And here's that document. So a series brief, Cloudflare builds and launches lots of products. Let's tell the stories behind them. So just rough ideas on questions we could ask, like, what were the light bulb moments that sort of happened in the course of developing the product? Who was the product design folks? Who was the show design folks for the people watching? And that includes folks we'll talk about in a second. What were the best decisions that were made in the course of building and designing the product? And what were the worst? And just what does that process look like very tactically, right? What were the steps and gates that a product idea had to go through from idea to shift? And then what are the actual software delivery teams at Cloudflare look like? And what are the different roles and responsibilities of each of the folks on the team? I think one of the really interesting things about Cloudflare is that the answers to these questions are different for every product we ship, right? We're really empowered as product managers and engineering managers and engineers and designers and all the other folks involved in building products to figure out the working style and methodology that makes the most sense for our given problem space and then execute them in the way that makes the most sense to us, right? So hopefully in each of these episodes, you'll hear some things are sort of constant across all the things we do as a company, but some things are very different from team to team. And so I'm excited personally to even go deeper and learn more about how different folks at the company think about these different things. And I'm also interested in hearing about the bad things, right? Like what went really poorly when we built this product and launched this product? Those are the stories that don't come out in blog posts that we don't talk about as much. And I'm really excited to have this venue to sort of talk through the hard things about being a product manager and about building software, because we all know there's lots of those. In terms of who this show is targeted at, you know, I'm really excited to talk to Cloudflare customers about, you know, how the products they themselves use every day come to life. And I'm also excited to talk to the industry at large, right, and hear from other PMs and engineers and other folks in similar roles and similar companies, hear from them about what's really interesting about things we do, what's like totally crazy and like we should think about changing, etc. And then how do we know we're doing a good job? Well, I think one thing is like hopefully you come back next week and you watch the next episode, right? Retention is super important to me as a product manager. I'm interested in driving adoption as well. I'm interested in hearing if this show makes you happy, if it makes you sad. But please, please reach out. And, you know, I think just the amount of engagement we get, the amount of questions we get from folks watching will be a really good signal for whether or not we're doing a good job. So please reach out and tell us if we're doing a good job or not. I have my, I have the live stream stats open in another window so I can tell in real time, you know, audience sentiment. Okay, this is great. It's like watching the presidential debates and seeing the like why. Cool. Okay. And then scrolling down in this document, I was just jotting down episode ideas. So you're watching the first episode, intro episodes of JPL, John Levine. We've introduced ourselves. We've talked about what we're going to talk about. We'll walk through goals. Let's talk a little bit about what each of us did before Cloudflare. You know, Cloudflare is not either of our first jobs. And I think it's really interesting sort of putting in perspective how software development works at Cloudflare by talking about how it's different from what we've seen in the past. So, so John, you mentioned, what did you do before? Yeah, sure. So Cloudflare is like my second employer as a, as a college graduate, I guess. Before this I worked at Google, and I worked on a number of teams at Google. I started out in the search organization working on search UI as a product manager. And then after that I worked at YouTube for a number of years. I worked on a lot of different things. I worked on playback quality, security, YouTube and education, the YouTube embedded player. And I spent a couple years working on a new app called YouTube Go, which is about making YouTube more accessible in emerging markets, places with either slow or expensive Internet data speeds or both. One thing I always say about, you know, Google is a massive place, I think it has like 100,000 employees or something. And I remember going to the West Coast PM offsite at Google towards the end of my time there and I think there were just more people who attended that, more PMs in the West Coast of the US than there are employees at Cloudflare today. So just it's a huge organization and there's, it's hard to generalize about that. But one thing about all the things I worked on is that they were all consumer facing. And I like to tell people like being a consumer facing product manager and being a product manager on, you know, products that are used by large organizations and businesses, it's like a totally different ballgame. And so I had to learn a lot, how to do a lot of things differently when I came to Cloudflare and I think that'll be a theme, because I have a consumer background and it's a theme we'll talk about a bunch on the show. Yeah, so it's interesting. So, you know, I think one of the interesting things about product management generally, but also even that split between consumer versus sort of more enterprise and B2B focused product management is, you can read about these things in a book, right? Or you can watch a TV show on the differences between these things or how to be a product manager or whatever, but like, you really have to sort of live it, right, to sort of internalize what the actual differences are. You would describe that as, we're talking right now about how to write voice notes. You can't actually. Yeah. Let me do a TV show about how to play guitar, but you're not allowed to touch a guitar. Well, hopefully that just means we need to do a really good job. Okay, so let's focus on the B2C versus B2B thing for a second. If you had to summarize the differences there, just a couple bullet points, what would the big takeaways be? Yeah, and maybe it's sort of like, it's already been hard to say what they have in common, but I'll start by talking about what are some of the obvious differences. I think the main one is how you define success. Let's talk about what the business goals are for Google or for YouTube. You can be very cynical about it, but the goal of YouTube and Google is to have adoption measured in billions, right? Billions of people using the product. At YouTube, it was watch hours per day. I want to say that in 2016, YouTube announced that they passed a billion hours watched per day. So think about the growth that YouTube has gone on since 2016. So the scale of that is absolutely massive. And we can talk, we can have a whole episode on business models and why it's important for YouTube to reach this ultra humongous, hard to reason about scale. But that means you think about product development very differently. So when you have a billion users or billions of users, you can't really talk to all of them. You can segment them. You can think about them at eye level. We thought about users who are in countries with expensive Internet access or slow Internet access. We can think about users like that. But more commonly, you fall back on statistical measures and thinking about the population. So everything at YouTube, when you work on a scale, is basically done by experiment. So an example of that is, I mean, a classic example is like, you know, the algorithm. How does YouTube decide what to show you on the homepage, right? And so that is all driven by basically running experiments. There's, you know, a model that predicts what you'll want to watch. And that model is tweaked and they're constantly trying out new things and A-B testing them and making, finding 0 .1, 0.2, 0.3% improvements and launching those. And those 0.2% improvements add up to being this kind of, you know, world striding colossus. That's actually pretty good at knowing what videos you want to watch. That's a great approach when you have a billion customers. And you're trying to get to 1.5 billion. That doesn't really work when you have hundreds or thousands or zero customers and you're trying to get started and ship something. So I think that's like the core difference. Are you talking, can I actually just talk to my customers or do I have to kind of run experiments and treat them as a population? So how much do you talk to, that sort of begs the question, how much do you talk to customers on Cloudflare? All the time. So at YouTube, you know, we'd run user studies. Actually, when I was, you know, I fly to India and, like, interview people, interview a handful of people, and that was really instructive. And, you know, folks, you know, we interviewed folks in the U.S. too, right? And product testing is an important part, especially for product design if you're building a new UI flow. But at Cloudflare, it's such a core part of the job. I mean, I talk to customers multiple times a week to understand what they're doing, how they're using the product. And it's okay to build things for one customer, right? And that took me a while to learn. And it's okay to build things for one customer because the art here is understanding, is that the vanguard? Are they going to do something that is going to represent what a lot of people are going to want to do soon? Or are they kind of off to the side and maybe doing something weird, or that's not going to be as popular later on? So when you're interviewing customers, how do you figure out, like, oh, this is some snowflake. And this customer we're talking to wants something totally off the wall. And this might make sense to them, but it doesn't make sense for anyone else. So it's like, oh, this is really interesting. And, like, there's something generic here that we can build and sort of, you know, deliver a lot of value to a lot of customers. How do you go about teasing that apart? Yeah. And this is definitely the riding a bike part of being a PM. That's so hard to boil down, I think, to principles. But I think there's, like, much like riding a bike, there's no way to do this until you have just talked to a lot of customers. And there's no way to describe, like, an algorithm for how to do this. If there's an algorithm, it's, like, talk to one customer, talk to another customer, talk to another customer, right? Like, keep talking to them. Take an idea, you know, form hypotheses all the time, right? Like, have an idea about a way to solve their problem, and then chew on that, right? And then talk to the next customer. And I frequently will talk to customers and say, hey, what if we did this? Does this work for you or make sense if we tried to do this? Well, later in the episode, I think we'll make it less abstract. Hopefully we can talk about an example of doing that. And if that makes sense to more than one person, you know, those are good. You know, if you can keep talking to more customers and refine that idea, eventually you'll come up with something that makes sense. But I think, like, you have to have that expectation that, like, if you just talk to one customer, you're probably not going to have a solution that, you know, that works for everyone. Yeah, that makes sense. Talk to me about that. So you went from Google, sort of, like, huge, huge company to Cloudflare, much smaller company at the time. You know, still smaller than Google. I don't know whether that's a good thing or a bad thing. Different story. But was that transition scary? What prompted you to take that leap in the first place? Yeah. Because, you know, Google, I think still has a reputation as a pretty good employer, in spite of, you know, also problems that are publicized. It's definitely, it's a great employer. I think my main motivation, personally, was just not wanting to be a Google lifer. Like, it was my first job out of college, and I didn't want to only work at Google. And I just had this sense that, like, I was getting, I didn't want to, you know, just be, like, a Google guy. And then, you know, again, personally, like, I don't think this is about a comparison between two companies. It's more about what I wanted at that time. I was looking for a place where, you know, the company was up and coming and where I could grow with the company. And where there was a lot of opportunity, I felt like I could have a lot of impact. And so Cloudflare felt like, and I should mention a domain that was interesting to me and that I felt like I can contribute to. At YouTube, I was often, like, a techie PM. Like, I was involved in a lot of infrastructure efforts. So coming to Cloudflare was a natural fit for me, personally. The fact that the company was growing was and still is a really good fit. Cool. Should I flip it around, Rustam? Do you want to talk about your career path? Yeah. Tell me about what did you do when you graduated from college? Yeah. So I was an economics major in college. And all through, like, childhood, I was always, like, tearing apart computers, putting them back together. My family was super into Macs, like, in the 90s, which was the dark old days of Apple. And for some reason, I decided that I needed to figure out how to run all sorts of exotic operating systems on these decrepit PowerPC Macs. So I had to learn how to program to get, like, net PSD running on a 75 megahertz, you know, PowerMac 7200. Because, like, that just who knows why that was the goal. But to get there, I had to figure out how to write a little C and, like, put all this stuff together. We're going to bleep that in the recording. Yeah, bleep that in. Anyway, so I went to school. I was an economics major. And this is wild to say this, but I never considered computer science or programming or software engineering or anything techie as a career until very late in college. And I was convinced that I needed to go to grad school to get a technical degree, to be taken seriously, blah, blah, blah. My parents still ask me when I'm going to get my PhD, you know. Yeah, yeah. So my mentor at Wisconsin, where I went to school, Professor Ramsey, he goes by one name, convinced me I was going to do a PhD. I was like, yeah, that sounds great. I'm going to do a systems PhD in computer science. And I had just gone to this career fair on a lark. And I handed out resumes to, like, a handful of random companies. I happened to walk past the Microsoft recruiting booth. And there was no line, which is very important to this story, right? Because if there had been a line, there is no chance I would have waited in it. And, you know, actually, as I- What year is this? This was in 2011. This was, like, before Microsoft was cool again. This is really kind of amazing, right? Because, like, Microsoft's cool again. Like, it's 1997 again. Yeah, yeah. It was not cool at the time. It was not. 2011, you were ahead of the curve. And I was this, like, obnoxious 21-year-old, 22 -year-old, whatever. And I had used a Mac my entire life. I had never used Windows, period. You know, as little as I could get away with. Anyway, I handed my resume to the Microsoft recruiter and chatted with them for a little bit. They asked some basic questions. And then they were like, what job do you want to apply to? And I was like, I don't know. And they were like, you strike me as a product manager. I was like, I don't know what that is, but sure. You know, there's no way, you know, ever I will work for Microsoft. So, you know, do whatever you want. Put me down as a product manager. Anyway, I won't tell this whole story, but fast forward six months. You know, my bud is in a seat in Redmond, Washington, as a product manager, program manager, as they call it, working on Exchange. And that was a fascinating time to be at Microsoft. You know, you alluded to Microsoft was not cool then. I think Microsoft, a lot of people point to the transition between Balmer and Nadella as the sort of like turning point. I think that's a little unfair to Balmer, actually. Like, you know, I think Nadella has done a lot to, you know, change the trajectory of the company. But a lot of the pieces that you can point to at Microsoft today that are key to their success were already sort of in play by the time I got there in 2011. That was two years before that CEO transition happened. And so, like, Exchange was fascinating to be a part of because Exchange was this extremely lucrative business for Microsoft and entirely on-prem at the time, right? So, you know, every year, every two years, Exchange would ship a new release. People would buy boxed DVDs of Exchange, and then they'd buy per-seat licenses, and this thing just printed cash for Microsoft. DVDs in a box. We should think about that for a second. Like, your career started shipping DVDs in a box. Yeah, and this directive had come down that we're going to burn this on-prem business to the ground, and we're going to move to the cloud, whatever it takes. And people were like, this is crazy. Like, what does the cloud even mean? And so I was actually there. We were already shipping the same code to our hosted environment and then also trying to figure out how to push, go to golden master phase for Exchange 2013, what was called E15 at the time, or internally codenamed, the 15th version of Exchange. And so watching the, like, box software process happen and, like, zero bug balance and all these, like, sort of this finely-tuned machine that Microsoft had built over, you know, 30, 40 years to ship box software, and then them figuring out how to ship that same code as a service was fascinating to watch. And I was actually on the service side of the business, and some people were like, oh, like, what does the service team even do? You've got, like, not that many customers, and you're not making any money. Like, it feels like a dead end. Yeah, yeah, yeah, exactly. And, you know, credit goes to Microsoft, but they were committed to killing the old use and hashing a new one. So we'll talk about the other, your other things you did in a second, but for just comparing Microsoft, for Microsoft in that era specifically, what do you think is the, you're talking about the transition, but, like, what do you think is the essential difference between, between Microsoft then and maybe where Cloudflare is now? I mean, aside from, like, the fact that there's infinite differences, but what's maybe the most salient one to you? I think, you know, in some ways the similarities are interesting to talk to. Microsoft was and is relentlessly customer -focused, right? And so just the way they approach, like, some, I remember getting this bug report, it was like, or seeing a bug report, and it was like, if your, like, Windows locale is set to a right-to-left language, and then you launch the Exchange installer, you know, in English or left-to-right, these crazy things happen. And, you know, if I were looking at that bug, it would have been like, who cares, right? Like, you know, like, and they were like, no, we have to fix this. Like, this, the customer can't install Exchange. We need to fix this. I was like, oh, okay, you know? And I think that customer focus I see at Cloudflare as well. For sure. And actually, there's a nice segue here. I think a lot of the lessons I learned at Microsoft around how to build software and how to orient around users, we practice as sort of part of our DNA at Cloudflare, but then I've also taken a lot of what I learned there and tried to install that as process on our PM team and sort of tried to share those, that tacit knowledge with folks internally. And some of that has turned into our product requirements document. Microsoft actually published a really nice book on how they approach thinking about software development. They have lots of books on this, but one of them is called Scenario-Focused Engineering, and that's basically a textbook on how to interview customers and how to talk about their problems and how to turn that into documents that guide software development. And a lot of the stuff in that book I find really valuable and is captured in things like our PRT template. So you also left a giant, comfortable company to do something maybe less known. Scary. What made you want to change that, and what did you end up doing? You talked about being a Google lifer. There were so many Microsoft lifers. You'd walk into their office, and Microsoft gives you a crystal every five years. It's literally a piece of glass that gets bigger every five years. And you'd walk into people's offices, and there'd be a 30-year crystal, which means they had been working there since 1982. And I was like, that's really cool. They got a big piece of glass in their office, but also that's a little terrifying, right? And so Microsoft is a super comfortable place to work, really awesome place to work. I learned a lot there, a lot of really, really smart people. And I left to start a company. And then Ballmer left like two weeks later. He heard that Rustem quit and was like, oh, no. Yeah, I left in the middle of 2013 to start a company. I had a pitch deck. I sent it to a couple of investors in San Francisco. One of them said, yeah, I'll write you a check if you quit your job and move to San Francisco in the next 12 days or some really arbitrary deadline. And I did that. I had no idea what that meant. Moved down here, and the rest is history. That company ended up folding after a year. This episode is not long enough. I want to keep our viewership, so I won't go deep into that story. Different episodes, cool. But yeah, I spent a couple of years doing really early-stage startup stuff. I went through a combinator with a company called Spire. I was just building breath-tracking wearables. And I got some exposure to B2C stuff in that context and just saw how powerful things like acquiring users through Facebook ads was. That was when lookalike targeting was first coming about, and I was like, this is magical. Upload a CSV of your users and tell Facebook to find users like that, and then they do a really good job of doing that. There's so many ways we can segment this job. I talked about consumer-oriented versus business-oriented, but there's almost like giant billion-user consumer -oriented versus zero. You're just trying to bootstrap a consumer business. Those are also completely unrelated different things, which we could also talk about. Yeah, yeah. And then taking a product from launch to 10 users, to 1,000 users, to 10 ,000 users was super interesting. And I realized I actually really liked the intersection of that more B2B stuff that I was doing at Exchange, at Microsoft, and then the B2C, very tactical, growth -focused PMing that I was doing at Spire. And I realized the next job I wanted was something that sort of mashed the two together. And I think one of the really special things about Cloudflare is like, yes, to your point, we are a B2B business at our core. But our roots are in this sort of like pay-as-you-go, self-start business. We have more scale from a customer account perspective than most B2B businesses ever dream of. And, you know, there's other companies in this genre, right? Like Stripe, Twilio, a lot of these companies that started sort of late 2000s. 2009, 2010 was like the era. Right. And you can even put AWS and similar products in that category, where it's almost like a business-to-developer go-to-market motion, right? Yeah. And so that, I felt, combined a lot of the things I liked about the B2C stuff with the sort of like B2B talk to your customers, and you can actually just solve their problems. You don't have to just do a million experiments to figure that out. Yeah. We've talked about that a lot. This job would be a lot less fun without having so many customers that we can use as an experiment, in addition to the enterprise customers we can talk to. It's pretty special. Yeah. And so that was my thesis looking for jobs. I ended up at Cloudflare, and that's really turned out to be true, right? Like we're able to do the really sort of like qualitative research and ask customers what their problems are, and then figure out what solutions are, and also combine that with really quantitative research because of the large customer base. And that's a killer combo, and it leads to really good products. For sure. So do you want to transition in talking about that process that you're trying to install at Cloudflare for Microsoft? Should we talk about what is that PM process at Cloudflare? Yeah. I think that's a great idea. So many ways we can approach this. We were going to talk about this at the end, but maybe should we actually just treat this whole thing with a case study? Should we start by talking about our case study now? Yeah. Sure. Yeah. I think specifics are always better. So I want to talk about that cash API, which is a relatively small feature that I worked on when I was new here. Is this an API that you ask for money, and then it spits cash? Cash API, yeah. It's our ATM API, yeah. C-A-C-H-E. The first rule about working on content delivery at Cloudflare is that you have to be able to make cash jokes. Not very good cash jokes. The drop of a hat, right? That's right. So I can talk a little bit about that, and maybe we can talk about the Stash API. That would have been such a better name if we had only thought of that. Oh, really? Okay. That's good. Thank you, Kerry. Shout out to Kerry for that one. So maybe what we can do is talk about – we can kind of imagine ourselves back. This was in 2018, about two years ago when we did this. Use the kind of current process that we're on now and maybe talk about how we would apply that process to cash API. Does that work? Yeah, that sounds great. Cool. Russ, were you presenting? Do you have our PRD template? Is that a good place to start? Yeah. Cool. Maybe while you're pulling it up, we can talk about what is a PRD? What is it for anyway? Yeah, what is a PRD and why is it important? Why is it important? Why do we write PRDs? What's the answer? Should I take a stab first? I think one thing that we haven't done in this show is acknowledge – we should be a little careful about PM hero culture. The old joke I learned was if you fire all the salespeople, no one would sell anything. The company would go out of business. If you fired all the developers, no new code would get written. Nothing new would happen. If you filed all their SRE or ops people, the service would go down. But if you fired all the PMs, what would happen if you got rid of all of us? I think that's maybe one of the clearest ways to think about what a PM does. What we really can do is influence, encourage other people around us to do the right things, to sell the product well, to build the right software. We try to do that by getting everyone rowing in the same direction. To me, a PRD is just one written way of doing that. There's many, many, many ways we do that. PRD is kind of an industry standard thing. I think that document plays a different role at different companies and comes in a different life cycle. Maybe we can talk about how we use that document at Cloudflare and apply maybe the Cache API to it. Give me one second. Let me figure out a way to present this. PRD stands for product requirements document. Maybe I'll just give a little preamble. At some companies, a PRD is maybe what we would describe at Cloudflare as a spec. It specifies precisely everything about how the product is supposed to work. You could throw it over the wall, so to speak, give it to any developer, and they would know exactly what code to write. Or give it to a designer, and they could design a user interface flow for the product. In some cases, I think it's meant to be the precise source of truth. It's something that you update with every detail about the product. You maintain it maybe even after the product launches so that it continues to define exactly what the product does and how it works. On the other end of the spectrum, I think, you know, in Rustman I joined, we were not the best PRD writers. I tend to write ones that are maybe fairly short. And I think what we realized is this document has a really important role, but we wanted to scope what that role is. And I think there's a right time to write one. You don't want to be too early. If you're still thinking about the problem and understanding what the problem is and who it's for, it's probably a little bit too early. But once you've thought about who you're building for, and you've talked to enough people to understand that, that seems like a great time to write the PRD. So, great. Now I have the PRD of Rustman. I still get crap for one of the first PRDs I wrote. Driving to standards compliance on a specific thing was important to me. And the PRD I just linked to the RFC on the ITS website. That's good, right? Some people thought that was a good PRD, and other people were like, what is this? Easy standards compliant. RFC. Cool. Let's talk about what we think belongs in here. Yeah. So, you know, to your point on, there's a bunch of bullet points up here. It just sort of talks about what's going on. There's some sample text in here that describes Railgun, a sort of older product at Cloudflare, and what a PRD for Railgun would actually look like. So... Let's actually use it. Maybe let's use that sample and talk about Railgun. Then we can talk about how it applies to Cache API. Sure. Yeah. So, I think the most important thing, you know, in the life cycle of a product, especially as you transition from like, oh, you as a PM have done a bunch of research, and you've talked to a bunch of customers, and, you know, you have some thesis on, here is a discrete set of customer problems that we can solve with technology. The most important thing as you transition from, I have this idea in my head, to we need to start building this, is getting everyone else on your team and even at the company on the same page about, like, what is it that we are trying to accomplish, right? And so, you know, Cloudflare has a very strong writing culture. We blog a lot externally. We blog a lot internally. And so, I think the first section, well, actually, the first section in the PRD is like, is literally a red tape section, right? Like, how are we going to price this? Has the review been scheduled? What is the legal and privacy? What are the legal and privacy implications of this product, right? Privacy by design is important to us. It's literally the first thing that, or one of the first things that shows up in the document. And then the next thing that shows up is intro to the blog post, right? Like, if you're writing a public blog post about the feature or product you're about to ship, or proposing we build, what does the introduction say, right? Like, and if you've done a good job writing the blog post, ideally the introduction elaborates on, you know, who your target customer is, what problems they have, and why is their life better using this new product, right? So, I cheated here, right? This template literally copies and pastes from the actual public blog post. Well, my first question was going to be, what do you, like, you haven't even done this. Like, how different is the actual blog post you publish from the PRD blog post, or what do you think is different about it? That's a really good question. It's funny. The thing I think about Magic Transit launched last year, and I think we'll try and do an episode on Magic Transit specifically in the future. I wrote this blog post. I wrote more than the intro, right? I wrote, like, a big chunk of the intro blog post, like, about a year before the product actually shipped. Maybe a little less than that. Six months, maybe. And then I kind of put that to the side, and it sort of sat in the corner of our wiki, and no one read it, and then we were sort of preparing to launch the product, and we were, like, arguing about, not arguing, but, you know, debating how we should position the product, who, you know, how we should talk about it publicly, et cetera. What the name would be. What the name would be. That's its own episode. And then, you know, I started writing a blog post, and I didn't actually look at the old blog post because I had written it so long ago, and I was, like, there's no way that's relevant. You know, I'm just going to start clean sheet. I wrote a new blog post, and then at some point, like, prior to publishing it, I went back and I looked at the original one, and I was, like, these are the same. This is crazy. And it was actually really cool. I was, like, either that means I am clairvoyant, which I certainly am not, or it means that the process works, right? We did a bunch of research. We deeply understood what our customers' problems were. We had an early idea on how to solve them, and then this document that I wrote six months before the real document was effectively the same. Yeah. I think we could probably do a whole episode. Like, I alluded to this earlier, but, like, when do you write this thing, right? Because, like, clearly you don't write it the week before the product ships, but you couldn't have, like, if you'd written it a month before you actually did, you wouldn't have known it now. So, like, when does this happen? Yeah, that's a great question. You write it when you start. I don't know how to. How do you ride a bicycle? You know, I think there's a life cycle from, like, you know, I think the first lightbulb moment is you're talking to a customer, and they're like, you know, I have this problem, and it's really difficult to solve. Or I have this problem, and I've solved it in this really, you know, specific way. And you're like, that's really interesting, right? Like, there's always that moment where you're like, wait, say that again. And that, you know, at least at Cloudflare, starts in a customer interview, right? Some customer says, like, I have this problem. And then, you know, you talk to another couple of customers, and you hear variations on that same thing. And it's not necessarily, oh, you hear the same thing from three customers, or five customers, or 20 customers. It's like, at some point, you're like, there's a product here, right? Or there's a feature here that I can build. Well, so you hear the problem enough times, and you realize that, like, you see that there's a way to get there. Somehow you see that there's, you know it's in our capacity, like, as an organization, to solve that problem. Yeah, and I think some people would call that vision, or whatever. I'm like, I think that term is kind of bullshit, right? Like, I don't know what that means. But, yeah, no, to your point, like, the pattern matching, I think there's some Steve Jobs quote about, like, being able to connect dots, or something like that. Yeah. Really corny. But, yeah, at some point, you see that there's, like, there's something that, some generic product that you could build that would give our customers leverage, right? Maybe let's talk about Cache API with this. Well, I guess I should answer my own question. How do we apply that to the Cache API? First off, what is the Cache API? Oh, yeah, what is the Cache API? Well, actually, let's sort of go through this lens of, like, how did we decide that we needed a Cache API, or, like, you know, what is that? So we should start by talking about workers, which is the context that this lives in. Workers certainly deserves its own how we built this episode. So Cloudflare Workers is our serverless computing platform. So when a request comes into our edge, rather than just, like, try to proxy it or return a response really quickly, we actually run code written by our customers, typically in JavaScript. And it's meant to be fairly lightweight and mostly stateless code. And even, you know, in the early stage of developing it, we realized that for a lot of things to be useful, they need access to some state. And we spent a lot of time deciding how to add state. We built a lot of products. We have Workers KB. There's a lot of stuff coming there. But one of the core ideas of Workers in the beginning was that we wanted to support a lot of APIs that are available in the web browser because, for a lot of reasons, one is the environment that many developers are familiar with. You could literally maybe offload code that you had from web browsers to Workers directly. And we saw customers doing this. We still see customers do this. And finally, like, we think that people who put together the web APIs were, like, pretty smart and, like, thought about how to do API design and thought about how to expose state in a useful way. And that's really where the cache API came from. It came from us saying, we have this API which exposes state in the browser. How can we expose that in a Worker? So, yes, so applying this, like, how did we get here? Well, when we were building Workers, you know, the idea of cache API came up, you know, from the team. You know, Kenton Varda is kind of the, it was, you know, he's an architect. I don't know the right word. He did a lot of the initial. Person working on it. Did a lot of the initial, I think, vision and also, like, just coding development of it. Ah, there we go. Here's some product docs. And, and also, you know, we, many of us, the company were talking to customers who were very early workers, doctors when it was beta. And we just talked about what the problems were that they were trying to do. And now, initially, when we were building Workers, the problems were what I would say are very CDN focused. So we're trying to get things done with our CDN that were maybe pushing the limits of what ours could do, or maybe even what a traditional CDN could do. And so a couple of examples of things like that were customer tell us they wanted to cache requests with an HTTP method other than get. So just really quick HTTP has this idea of methods and get is the most common one. When you type a URL in your web browser uses the get method typically refers to things that, that actually don't have state. But if you want to, you know, fill out a form on the web, you might be submitting a post request, right. Or you might upload something with a put request. And so people might want to actually cache those requests and responses, which is a little weird. And there's no real standard or pattern for that on the Internet. So that's one thing. Another thing people might want to do is they might want to modify a response they got from the, from their origin, but they might want to save some parts of the response they got on. It sounds like these are literally things that were coming out of customers mouth. Yeah. Yeah. So, Hey, Mr. Customer, Mrs. Customer, like what is it that you want to do with Cloudflare that you're not able to do? And they're like, I want to cache put requests. And then you're like, well, that's weird, but okay. And then you talk to someone who wanted to cache, like we want to add cache tags. That's a whole story about what a cache tag is. Now you do that, but basically they wanted to modify a response to a thing that was already being cached. So the thing was in cache, but they wanted to modify it when it left cache. And yeah, to your point, These use cases were so specific. They weren't even necessarily about workers. It wasn't even like, no one came to me and said, I need state and workers. Right. Or me. I mean, this, these conversations were with many, many, many, many people across Cloudflare Rita. Cause of deserves a lot of credit. She kind of was the co product manager for this. A lot of folks are involved, but they would talk to us. And they would just talk about the specific problems they had. And so how did, I'm trying to think how this idea took shape. I think Zach was one of the people who I think proposed this specific solution of like, Hey, there's already this. You could do it in a worker if you had a little bit more state and there's already this API that kind of works. And I think that's where that seed was planted. And I was initially super resistant. I was like, I don't really know what this cache API thing is. This doesn't make sense. Like, what does the API for cache even mean anyway? But I learned about it. And so we documented here. It's pretty simple. It's like, you can write things to cache. You can read things from cache. You can delete them. It's a pretty. Okay. So switching back to this PRD template, there was presumably some, I don't know where the PRD for this is. We should have done more prep here, but whatever. So anyway, so, so, so initially the blog post, you know, the intro was like now, now actually we can probably find cache API. Oh man. Oh yeah. Should we find that? Yeah. Yeah. Yeah. I think I read that blog post. Well, yeah, January, 2019. Wow. That's still a long time ago. Okay. Yeah. I think the worker's cache API gives you control of how your content is cached. Caching, a basic explanation. And then now it integrates with the cache. This is really interesting. When it comes to the blog post, like, I don't think I could have written this when we, when I at the time the PRD was written, because I don't think, I think I didn't totally know what we were building until we were underway building it, which is kind of interesting. I think this happens for me anyway. Maybe I'm, maybe I just am missing the vision, you know, but it happens more often than not, or you see a pattern, you're trying to solve for things and then it comes together and it, and it builds and like, those are the things that actually launch. If something doesn't make more sense while you're working on it, typically doesn't actually ship. But so you're laying out some use cases, example, customers maybe. Yeah. And then walking through how it actually works. Oh, this. Yeah. There's some cool art. I wonder who made these diagrams. Yeah. And this is a basic, basic, like, how does this thing work? Cool. And then, so, okay, let's keep, Oh, I'm just confused by my windows right now. Okay. So, so, okay. We've got the angel of the blog post then goals. Like what are we actually trying to accomplish with this product? So the goals of the cash API are really about, we're pretty clear because they were really just about adoption of workers. We were really explicit early on. So we will actually, let's talk about business goals. So business goals are actually, we've been, we've been trying to be pretty user focused in this, but at some point, like we are a business we're trying to like, you know, like make money, continue on as a business. Our business has something we're trying to accomplish. And so in the case of, of cash API, we're trying to make more people, you be able to use workers. And we did that. I think that, you know, maybe a non-goal for us was trying to make money off the cash API. We did float that idea at the time we talked about, Oh, would it make sense to like charge people every time they write to the cash API or charge for how much your store. And we can go into why we decided not to do that. But I would say the core reason is like, we just want to, we have a, we have a pretty robust pricing model for workers and we want to work. Yeah. Yeah. And then, okay. So success criteria and product KPIs, it sounds like just driving workers, adoption, number of workers users generally was, was a success metric. And then you know, did it matter actually how much people were in the cash API? That's an interesting one, right? That's a really interesting question. We talk about this all the time. Like does adoption matter? It can be so easy to just get narrowly focused. You know, I do a lot of, you do a lot of PM interview questions and I always ask about this and like, you have to be really careful about like, it doesn't matter if people use your products or it doesn't matter if they're trying to accomplish their own goals. So I would, I would argue for cash API. The important metric is really do people use workers? Not do people use the cash API? If it allows a new customer to onboard and start using workers and make use of the product, then it's successful. But if that means a relatively small number of customers do it, but those customers are valuable, that's, that's much more important. All right. The next section is target customers and users. Like what, what's captured in this section generally? Yeah, this is a really interesting one. This is actually a hard one for cash API, I think. So as you know, as we showed here, the idea is to talk about who is going to use the product and talk about, you know, what should we, what should we know about that person that's, that's selling for building a product? So if I were going to do this for cash API, I would talk about different, maybe developer personas and maybe levels of experience they have either, either programming or using workers or using clockware. So I might talk about, if I were just going to do this now, I would say maybe let's talk about an existing workers developer who knows Cloudflare really well, is using workers, understands like the web APIs pretty well, and is just trying to accomplish this specific task. And then maybe I talk about someone who doesn't know any of those things, but still wants to cache book requests. What would they do? What problems do they face? And then use cases. This is sort of, I, so I, I realized this is a little confusing because the sample text is all talking about. Oh yeah. We've got some smart, smart viewers. That's right. We don't figure out the discussion. So, so it sounds like, you know, caching put requests, et cetera, would have, would have gone in use cases. Then how do you figure out what the difference between an MVP, like how do you bucket MVP to everything else? Like that's, that's a critical question, right? This is, this is again where the art comes in. It's like there's, there's, there, you can't decide what you're building and how you're building it. Like those are, those, those questions are fundamentally intertwined. Right. And so you have to take into account the fact that, you know, there's an, man, we're, we're, we're going to have multiple Steve jobs quoted in the segment, real artist shit. Right. Like at the end of the day, you have to launch something. And it's really important that your product gets out there. And so this is about working with your engineering team to understand what's actually feasible and what is the, what is the baseline thing that people want to use? I think one thing we do at Cloudflare that took me a while to come around on, but is effective is to pick a date. Maybe you have a date when something has to come out that really just creates this like mental focus of like what actually has to happen for anyone to use it. I don't know if I've ever seen anyone. I mean, like almost no one errs on the side of being too minimal and not having enough stuff done for MVP. People always put too much stuff into the MVP. We can switch things up. We can pick a different cliched person to quote. Paul Graham here, right? If you're not embarrassed when you ship, you've, you've, you've shipped too late. I think one of the really interesting things about scoping MVPs is like, there's all this conventional wisdom about things that need to be in an MVP, right? Like, Oh, you need, this is going to make various folks at the company cringe, but like, you know, I can think of all sorts of very obvious, like you need this, like we launched products that we want to charge money for that have shipped without the billing system in place. Right. And like, that's like, how, why did we do that? And well, it turns out like, we can deliver a lot of value to customers even if we can't charge the money. Right. So let's do that. Or, or whatever the case might be. Right. Like this is sort of discarding that dogma about like, Oh, obviously you need X or like obviously you need to Y and really, really tightly thinking about like, what is the problem? And how can we solve it? And, and what is the minimum valuable thing to a customer and what would, what, what would minimally make them happy? Right. Absolutely. It was important. So I thought maybe we could leave the purity behind. We have another eight minutes or so. One thing that people don't do a lot, I think when they talk about purity user product development is talk about, okay, then what? And what's interesting about the cash API is we're talking, that's something that shipped almost two years ago. And so I thought it'd be useful to talk about like, okay, so what do we think about this purity process? Was this? Yeah. Okay. So we, we had those, we have those, we have those questions. Okay. So we've sort of covered light bulb moment. What were the best decisions made around the launch of the product? Yeah. Well, if you go really into the weeds here, if we talked through the developer docs, I mean, there were a lot of, one thing we didn't talk about is like, after you write the purity, there's so many micro decisions that get made that are so small. And it's hard to know in the moment, which of those are good and bad. And I can, it's probably not worth going into those in too much detail, but I think some of them, I think most of you did a pretty good job. There are a couple that I would change. I'd say, I think like our general idea of like we need to add state rather than build a feature for everything that has state, like let's, let's pick an API and let's pick a web standard was, I think those are all really good ideas. What was the worst, worst decision? Yeah. I think two years on from the cash API, the challenge I have, or if I hesitate recommending it to someone is that it's just hard to use. And we wanted to solve people's problems and we gave them, it's interesting because that's, that's literally the other side of the coin you said was the best decision. We gave them, we gave them some tools. We gave them a fishing rod. We said, have fun. And this wasn't, I think what's one thing, I think it's good is that it's a building block that does not preclude us. Like I think a lot about path dependency. I think about if we ship this, then what can we ship afterwards? What's going to be really hard to do after we do this, what becomes easier because we did this. And I think the cash API makes it easier for us. And I hope we do spend more time building tools for people to manage their state in a worker. Let me just give one example to make this more clear. I mentioned that cash API has a really simple interface for developers. It's like, is this thing in cash? If so, get it out of cash, put this into cash or delete. And that's great. The problem is that, if you're trying to replace like Cloudflare's cash with this API, it's a little bit more work than that. Like the way our cash actually works is quite complex on a heuristics and logic. And, and a lot, a lot of thought and blood, sweat and tears have gone into actually making it work. There's so much more that goes into that. And frankly, as a, as a sort of person who is just starting working on caching on Cloudflare, I didn't, I didn't, I'd appreciate the complexity of that. That's fine. It's actually fine. I think it's okay. With my perspective now, I think it's still like a building block, but I would like, I would love to see us build on that by giving people a little bit more structure and how they use it. So that could be for starters, just better code examples about best practices, about how to use it. Eventually it could be libraries in the runtime itself to say, okay, rather than call these low level APIs directly, we suggest calling this higher level abstraction, which may be ships with workers. Got it. No, that makes a lot of sense. I think we do have one sort of question from the field. So let's save a little bit. Maybe let's, let's go through one more of these in terms of how the team separates roles and responsibilities. So, you know, obviously in the cache API, there are a lot of, a lot of different folks involved. How, how, what, what was, what did the actual, like who did what? Yeah. Yeah. I think actually it was really across, you know, organizational success. Like we had, it was like a workers team, which has product managers, engineers, and designers. There's a, there was a cache engineering team, which had product manager me and, and engineers. And I think everyone came together really well on this. And this was the kind of classic thing where like the cache team is building basically backend for the workers team. So they're like someone at some point, had to write a spec with like an API interface and kind of both teams built up to the middle and that worked. It's pretty good. We kept it simple. I think we had really, really good folks working on it on all sides. One thing that was maybe challenging is maybe outside just product engineering. Actually, one thing I also want to mention, I think we did a really pretty nice job in the documentation. I'm happy that it turned out Harris Hancock, a developer who on the worker side did most of the documentation, I think did a really great job. But I think in terms of now, people have to use it. Right. And so where I think maybe we could have done better or engage better with the organization was spending more time with our sales engineering team, with our developer relations team to make sure that once it was out there, people knew this existed and knew how to use it in the most effective way. That's a whole PM story unto itself, right? Like if a tree, if a product falls in the forest and no one's there to use it or see it, then well, yeah, it kind of speaks to some of the stuff we were talking about earlier. Cool. Okay. Let's, let's come to this question from Alex. What type of PM is best and why? PMs who wear plaid shirts. Like salmon colored t-shirts is clearly the, yeah. Yeah. No, I, so I, I think it's a good question. I think, I think one of the slightly different question that I get often is like, Oh, do you have to be technical to be a PM? I think that's a, that's almost the wrong question, right? Like I can think, I'm just thinking about PMs at Cloudflare and a lot of us don't have like classic technical backgrounds. I mean, I know you have a CS degree, but you also have a linguistics degree, right? Which of those do you use more as a PM? Tough to say, which applies more to my job. I think it depends. I think at Cloudflare is weird. I don't want to overgeneralize from Cloudflare. Like I kind of use the CS degree at Cloudflare. It's kind of helpful, man. Like that's, that's unique to, I think, we've been talking about a cache API for the last half hour or so. But then beyond the technical versus non -technical, I think there's also the like, some PMs are really good at project management. Some PMs are really good at data analysis. Some PMs are really good at food and very few PMs are good at all of those things. Is any one of those sort of like secret sauces more valuable than the other? Yeah, that's a great question. I would say that the key is to like, is to know you're good at it and specialize. So I think like, like, like of all the things you just mentioned, like good at data analysis, check, not very organized. I'm like, okay. I'm like mediocre at designing. The manager is trying to hide his facial expressions. So I can tell you I'm good at it and not good at it. So I think like company role, product selection, like the things, like control the things you have control of that take advantage of the things you're good at and maybe excuse the things that you're not as good at. Yeah. Excellent. Great. Well, I think we are, we are at time. The next part about a product launch is always the retrospective, right? What went well, what didn't we'll, we'll talk about that offline. Yeah. Thanks for, thanks for joining John and thanks for watching everyone. Thanks everyone. And yeah. Yeah.