How Cloudflare Built This

on this bi-weekly episode of How Cloudflare Built This. I'm your host, Guy, just kidding, Rustam Lalkaka. I'm a director of product here at Cloudflare and I'm joined today by Angie Kim. You want to introduce yourself? Sure, yeah. Hi, I'm Angie Kim. I'm a product manager here at Cloudflare and I'm coming to you live from my closet, which we were just talking about. It's a very nice looking closet. This has a window. It has a window. It has a door. Unfortunately, it does not lock, so I have an intrusive three-and-a-half -year-old that will barge in, but yeah, I've got windows. This looks nicer than most San Francisco bedrooms. Cool. You work on some really critical pieces of infrastructure at Cloudflare and all things related to billing and making sure our customers can pay us for the services we offer. I'm excited to talk to you today about how some of that stuff gets built. In general, we've opened these episodes just learning more about what folks did before they ended up at Cloudflare. Walk me through your professional path from wherever you started to here. Sure. I think most people start off where they went to college. I went to University of Illinois in Champaign-Urbana. So many Illini. This company is crazy. I know. It's awesome. It is so cool. I went there, studied engineering, didn't know what I wanted to do. Most people go to consulting. I was there for a couple of years, mostly focusing on QA types of work. I was always curious about systems and how things work. I didn't really want to do engineering coding, per se, but I definitely did want to stay technical. I was doing that for about four years. Then I transitioned over to one of the big banks. Total different mind shift in terms of career. But did QA there for a little bit, a while, but then I got the opportunity to do product management. Never really heard about product management, even at my previous employer, because I worked on the tech team, so we didn't have real product managers. We're just building out software that the clients needed. Whatever the client asked for, right? Exactly. But going to a bank, obviously, it's a little bit more structured, regimented, and I got to work on some cool things. I worked mostly in the cash management sector, so I learned way more than I wanted to about wires and ACHs and all that fun stuff. In terms of the technology I worked on, I worked on our portal. I worked on our mobile app, which is pretty cool. My boss at that point, he made fun of me because I didn't even have a mobile phone, but yet I was a mobile product manager. That was the running joke that I was his worst disappointment because I didn't use the tech that we were building. But then I also worked on some client workstation applications as well. I did that for 10 years. One of the interesting things I'm always curious to hear folks' stories on, especially earlier, I think people these days have more exposure to things like what a product manager is, but when did you first hear about product management as a discipline? Was it obvious that you wanted to do that job? It wasn't until I worked at the bank that I even heard of product management as a discipline. I didn't even really know it was an option for me because I had no previous education and training, but then it turns out that nobody really does, at least not back then. But I think the reason why I got the opportunity was just because my natural tendency to ask questions and understand what is it that's trying, what is the goal, what is it that we're trying to build, what is the benefit, just asking those natural business questions. And then also, I really enjoyed the opportunity to work with engineering and try to figure out the solution. So yeah, it was never something that was on my career path. I remember taking one of those surveys in high school in terms of what you're going to be and it definitely did not come up. But yeah, I think it just kind of fell into my lap just because I had good mentors that saw me as a good fit and kind of threw me into the deep end and said, hey, you should try this out and I couldn't refuse. And once you actually got into it, did you like it or were you like, oh my god, what am I doing here, imposter syndrome set in? What was that like? I mean, initially at first it was kind of daunting because I was making the decisions and calling the shots and I was like, well, I have data that supports it, but am I really doing the right thing? So there was always a moment of doubt, but the more feedback that you have with customers and just getting feedback from the sales team and in terms of what the regulator just saying that was required at that point, it kind of justified the rationale in terms of why I was making those decisions. But yeah, at first it was very daunting to be able to tell a group of people, this is what you're going to build and you're going to build it this way. And having all of that trust being put on you, it was a lot at first to handle, but I kind of love it. Not so much, well, a little bit because I'm a little bit of a control freak in some ways so that I can kind of get, I can steer the direction in terms of what's being built, but the collaboration and then just working with people and then seeing people use it, like that's pretty awesome. Yeah, how did you figure out, I think one of the hardest things from my experience sort of growing up as a product manager is like figuring out when to speak up and when to shut up, right? And like when to let the control freak like out of the cage and when to just like sit on your hands and say like, do whatever you think. How did you calibrate that? So as long as I didn't see things headed towards failure, I try to kind of rein in on that because there's some things that people just have to like learn on their own. And again, you don't want people to fail, but you want people to learn from their mistakes too. Also, if it didn't add any big difference to the outcome, those are things I had to learn and figure out what to really hold on to versus what I could let go of. So yeah, if it was just a matter of like this pixel is off, like that's not my expertise, like I'll just let it go. But if it still ended up with the outcome that I wanted or that I was expecting, then I kind of, I, yeah, it's something that you very much have to like learn and and read back on. I still find myself like, it's like punctuation. So you were at a bank, you were sort of like learning the ropes. Were you in San Francisco at that point? No, I was still in Chicago. So my husband got an opportunity, he couldn't refuse. And I was debating at that time, do I continue working for the bank because there's offices all over the place? Or do I just totally take a leap in terms of embracing like the Bay Area culture? And you know, I was like, I've been at this job for 10 years now, I should try something different. And I thought, okay, I'm just gonna embrace it. I have to say, when I came over to Cloudflare, like, it was a shocker. It was just so far beyond anything that I ever experienced. It definitely put me out of my comfort zone. I was definitely in a much more fast paced environment. Like at the bank and at my previous employer, you know, you couldn't submit anything into production to one to like three rounds of change controls, like all these rounds of testing, you know, sign off by like leadership, but here, like people would submit a PR and it'd be in Prague, like, really? So it was cool to see the amount of progress that was being made and how quickly. But then also some of the securities that I had in terms of, you know, all the different checks and points and balances that were in place were kind of being lifted. Also, just like the different ways of communicating was just totally beyond what I was used to. So stepping back a little, how do you go from like bank to, I want to work at a tech company to Cloudflare? Like what, how did you find Cloudflare? Yeah, so Cloudflare had an opportunity to work on billing, and I felt like it was a good segue into, you know, in terms of my career. So I was struggling with, do I go into fintech, where that is the main product that you work in? Or do I go into more of like a service related function? But I have an opportunity to learn more about how things work. And I wanted to take a step back and not be like the primary product, because I didn't want to become so specialized that I didn't get an opportunity to learn different types of things. And I felt like in that way, Cloudflare was a good opportunity. And also just the mission and what Cloudflare was doing at that point was just beyond and so different from what I had seen before. It just felt like a good place for me to come. Got it. How big was the company when you joined? I think there were like three, less than 350 employees. I joined after you, and you're still smaller than that. So anyway, in that like, I'm not sure, kind of startup, but not. Yeah, yeah, yeah. I mean, my orientation class was a total of three people. And surprisingly, we're all still here. So it's awesome. Yeah. And so you're, I interviewed Pat in the last episode, and you're, he's the, I think, the longest tenured PM at Cloudflare now. Yeah. And you're the second. So you mentioned like, you came in and it was like, oh, my God, this has been moving quick. Yeah. Is it like, yeah, how does the experience today versus then sort of compare? What stayed the same? And what's, what's changed? I guess what's stayed the same is like the level of trust that we have in terms of what we can contribute and how we can form the way that things operate. So that's still the same. One thing that has changed is just the sheer number of people and the teams and the disciplines that they have. So there's a lot more people that can help lend their expertise in the way of things that should be done versus way back when, when you were doing basically most of the heavy lifting. So, you know, it's great that we have people that have very specific domain knowledge that can be contributing. So that's cool. But then also on the other hand, because there's so many more teams, there's so many more things I need to support. So good for the company. And I guess it keeps me busy that the company is just growing as it is. Yeah. And then, right. Like as the team grows, as teams grow, like more connective tissue, it's like, it's, it's, it's interesting to me watching all this happen because like you read about these things in books, right? You're like, oh, you guys, you got more people. It's harder to communicate. And then like over the past four or five years, like actually watching that happen has been fascinating. Yeah. We are able to get bigger things done, but it takes longer because it does. Yeah. Yeah. Yeah. Cause there are so many people. Yeah. Yeah. One of the things that one of my mentors had taught me in the past is that everybody has an opinion, but not everybody's opinion matters and trying to sift through all that, especially when there's a lot of people that have a lot of opinions. It takes a lot of work and also just balancing expectations as well. Yeah. Yeah. Makes sense. We, we, we mentioned upfront that you, you work on billing systems. What does that actually mean? Like what are the, what are the components that your teams own and build? Yeah. So I would, the best way I would translate it is just think about what it is like your own retail experience, right? So you go to a website, you want to order something, right? So you're basically putting it in a cart or some place you want to purchase the, the item. So you'll have to put in your payment method, whether it's your credit card or some other, some other thing. And then that information gets stored somewhere else. And then there's the actual charging of that payment method. And then once you go ahead and have your, the funds collected, then you get a receipt back after that. So that's the invoicing piece after that, then there's actual fulfillment of it, right? So somebody has to ship it or send it to you or it gets emailed. So that's the last piece that I work on is more of the enablements of the different features that customers are signing up for through our dashboard. Got it. So the actual, like, right. There's no physical product being shared, but you still do have to make sure that like the customer gets what they paid for. Right. Yeah. And so, and also because we're a reoccurring subscription-based company, you know, after a month passes, we need to recharge you again for the next month that's going to happen. We also have a lot more of like usage-based products. So we bill in arrears. So meaning we pay, we charge you for the previous month's usage because we want to bill you, bill you on what you've actually consumed. And so there's a prepaid component and then there's a, the arrears component. Interesting. So that all makes sense. And I think like any CS 101 student is like right now thinking like, oh yeah, I know how to model that. Like there's like invoices and orders and customers and like there's some lines and easy. Right. And that was certainly my impression when I got to the Cloudflare. It's like billing, it's easy, right? Like count some things and then you let count and then email and then money. Right. What's actually so hard about this? So if you take like each step and decompose it, there's actually a lot of stuff that happens. So there's the, first of all, the thing that people are buying, right? You have to define what is the pricing scheme? How are you going to charge customers? Is it on a tier base or is there no volume discount? Is it going to be prepaid or is it going to be an arrear? So you have to think about like what is it that they're buying and how do you operationalize the actual purchasing of that particular object? And then there's the act of, okay, I'm going to go ahead and put in my credit card information. Well, there's a lot of personally identifiable information about that. So that's how do you store and capture that information so that if there's a hack that happens that that information doesn't get exposed to the bad actors. Also, you know, we have a system where customers are entering in their credit card information. How do you keep the bad people at bay that are trying to just test credit cards, right? So there's a lot of fraud that can happen. So how do you figure out who you can trust versus those that you can't? The actual act of purchasing it. So there's things like taxes that we need to also consider. It's not the most glorious thing to talk about, but it's, you know, there's different regulations and if you're going to do U.S. taxes versus other taxes and how do you represent those taxes? And those tax codes often don't actually reflect like SAS realities, right? So you're doing these things. Yeah. And they differ based on the, is it the city versus the state? It all differs. So there's a lot of complexities that happens with that. And then also with taxes, like we need to ensure that the information that we have on file is valid, that we're texting you appropriately. And so there's like edges verification that needs to happen. So, you know, along the steps, there's a lot of different things that happen. And then in terms of, you know, especially in a case where people can't pay, like when do you shut them off? You know, we have a grace period where it says, we're going to give you like five more tries. And then at that point, like, you know, how do we communicate that to customers? How do we let the user experience look like? Exactly. Yeah. So it's all about, you know, checks and balances in terms of, you know, letting the good people through, collecting their money, giving them a chance to pay us if they haven't been able to, and then keeping, you know, the bad people away from us, I think. It all makes sense. I mean, right. Each of those individual pieces is not that complicated, but then you combine them all and you have to do them flawlessly. Instantaneously, right? Instantaneously, that becomes very complicated. So, you know, if we were to model this as like a set of customer personas and then their problems, who are your personas? It sounds like obviously people trying to buy things is one. And, you know, the clients are customers themselves and they're probably the most important, but then who else, who are the other stakeholders who have all these like random demands of your team? Right. Yeah. So you got to think about the other groups within the company. So we've got our customer support team. So they're the ones that feel the brunt of a lot of the different questions that come from customers. There may be occasions where, for whatever reason, their credit card failed. There's only so much information that we get back from the banks, because they have an obligation to protect their customers as well. And we can only expose as much information as we have. So when we try to explain to a customer, oh, sorry, your credit card didn't go through. It looks like you may have run out of insufficient, you have insufficient funds, like maybe we'll try it again. But we're not going to tell people that through the dashboard, because we don't know if we can really trust you or not, especially if you're just testing credit cards. So there's a lot of just like messaging and communication in terms of training for our customer support team. They're also the ones that are, you know, at times issuing refunds, there may be service was not as expected, or there was a huge degradation in service. You know, there needs to be a goodwill credit that's being issued. So training in terms of guidance as to how that's done and providing them the tools for that. Trust and safety, again, in the case of, you know, the bad actors, how do we shut these customers off, the ones that are abusing our system? There are one, obviously, accounting and finance, because there's a lot of revenue recognition implications. It's a huge KPI that we have to be able to measure and report back to, you know, everybody, because we're a public company, right? So being able to give accurate numbers. So that's important. And then marketing, they're always trying to launch different initiatives to be able to offer different programs, whether it's the policy team launching Galileo or Athenian. The developer community, they have something for startups and hatch incubation programs. These little programs are some like, not necessarily huge, but some amount of like custom logic in the billing system. Right, right. And also be able to trace back and justify why these customers are getting service for free. It's, yeah, there's a lot of different flags and exception cases that we have in trying to build traceability in terms of who these customers are and why they have it. Yeah, right. It turns out once you build a really robust billing system, giving things away for free is harder than charging for them. That's one of the other eye-opening moments I've had. That all makes sense. So, I mean, okay, so that's the like explanation of why this is hard for someone who's coming in totally cold. But then I'm sure we have viewers who are like, oh, well, like, why don't you guys just use Stripe like, you know, they have all these like, or there's any number of other companies out there that have these like invoicing in a box system. You just like hit some API and everything magically works. Why, why can we not just use one API and then have things just work? So there's some things that they just don't cover. So one of the things that always is like a little thorn in my side, but very important is we, so within the U.S. there are certain entities that we're not able to have business with. That's a very U.S. centric notion. And some of the other companies that are outside the U.S., they don't have that restriction. So, you know, countries like Iran, North Korea, and the list goes on and on. Like we had to build a solution around that so that we're not even sending that card information through our network. We're just stopping it immediately up front. So it's like little things like that, that we need to account for, especially because these are very strict requirements that I've gotten from other teams that it's not going to be a one fit solution for everything. Like it would be great. If you're running a really simple storefront and you're selling like socks out of your, your house to Americans. Right. Yeah. One of the APIs is great, but as soon as things get more complicated, there's some random set of requirements that disqualifies. Yeah. Yeah. Especially because we do have such a global customer base. The amount of money as well varies as well. So we've got customers that are buying, you know, you know, spending $5 a month. And then we've got customers that are like, you know, in the thousands that are using a lot of the usage-based billing products, because that's what they need. Right. And then we have customers on the enterprise side doing one of that. Yes. So we were talking about different stakeholders and you conspicuously left out product managers. I'm smiling and you're laughing because it's like a super familiar. Oh yes. We've had our run-ins too. Yeah. Yeah. Right. Like I think our first time working closely together was like, after I started, I was like, I need to launch this product. I'm like, I think I need to charge money for it. And so you're the person to talk to. Right. And then you're like, yeah. And then, and then you were like, what are you trying to ship? And I was like, in two weeks. I guess, first off, why, why do people always think about billing last when it seems like one of the most important things? Yeah. Well, I think it's funny because well, everybody's just narrowed in on, you know, what they are aware of. And I think a lot of different product managers, they focus on the feature functionality, making sure it's of quality that it's simple to use that there's going to have great adoption. And it gets usually launched as like a beta just to see, you know, what the customer feels. Yeah, exactly. Right. And then once they, they get good signals in terms of, okay, we're ready to go GA, they're like, okay, let's go. And billing is always, was always kind of like the last consideration. We're getting into a better cadence where it's one of the first questions that comes up, especially because now we have like an organized pricing council where we, you know, question the strategy, the upsell, especially when the pay-as-you-go customers and the enterprise customers, like what's the differentiation. So there's a lot more like business types of discussions happening up front. But yeah, people just assume, like you said, that billing is just easy. You just count it and then you're done. But it's not always the case, especially if it's a usage-based product. Like there needs to be some unit that you're counting, right? Billing can't just grab that value out of the air. It needs to come through some organized fashion. And we integrate with the data team to build, collect that information. So yeah, what comes in needs to come That last minute, like, oh crap, we need to charge money for this. Angie, how do we do that? First off, it's not like we're joking about it. So it's clearly not that bad, but I'm sure that that last minute sort of like pressure to figure out how to capture revenue for a given product has resulted in some, I mean, like interesting situations, right? And I often hear like you and the billing team talking about like, oh, like we got this like crazy, gnarly technical debt where like we can charge for some things, but not other things. And then like applying a discount to that, but not that is difficult. And a lot of that ends up just building up because we're making quick decisions, right? We're trying to like unblock a given product. Yeah. I think given where we are at the time that the conversation happens, you know, there's definite trade -offs that we make, like given the deadline, because there's a lot of marketing and different campaigns are happening to communicate things out and, you know, press releases, we want to hit a certain deadline. So unfortunately, you know, we do make those trade-offs, you know, we figure out, well, can we release as soon as possible? And then we figure out how do we unravel that and figure out in the long run, what's the best approach? So like in the case of Argo, like we launched it pretty quickly and then we went back and had to make a lot of refactoring to do the right. Yeah. And just to be clear, right? Like it didn't impact customer experience. It didn't impact the actual bills that went out. It just made your team's life more difficult building additional functionality. Yeah. Yeah. We're at a place where we have a lot more maturity in the system so that it's not nearly as bad as it used to be. There's definite things that we can do. And a lot of the products that we launched have followed kind of like in the same pattern. So we haven't seen any big surprises. I see that. Yeah, that's actually interesting, right? Like early on, like we were inventing new ways to charge for things all the time. And it's like, we need to charge for things this way. And it's like, oh yeah, we could probably do that. And now it's like, we're going to charge for this the same way that we charge for that thing, right? Exactly. Yeah. Yeah. How do you, and this is a more generic product management question, but like how do you trade off between fixing some of that debt and building the new features that you want or users are asking for? Yeah. So it's always a funny story. Like the list of things that I want billing to do is this long. And then the list of things that I get from the organization is this long. And it's just trying to figure out how do you balance those two things. And sometimes we have to just go with the shortcuts for a little bit of a time and then figure out how do we unpack things. And so one of the things that I've learned is that you don't need to be perfect at the initial launch. You need to make those small incremental updates for like the biggest lift as possible. So it's a matter of just trying to like weave your way through it and figure out how to get there. Yeah. And know what's possible to trade off and what's not, right? Like sending bills that are inaccurate is not right. Not okay. Not okay. But then like, do we need a hyper detailed invoice at day one or is it okay to say like Argo, right? Yeah. And that's kind of how we did launch, so the best we could do was tell people this is how much you consumed. And it raised a lot of questions from customers saying, well, how did I get to this point? And it took, unfortunately, almost a year for us to actually build usage-based analytics to show day over day how much usage was being consumed. And it was one of those things that we knew we needed at first to launch because the questions were going to happen. But after we rolled out Argo, I think load balancing was soon thereafter. And the amount of products just kept queuing up and we couldn't address it right away. But it was one of those things that we had to just, unfortunately, wait for a year. And then not too long ago, I would say earlier this year, we launched alerts. So, that's like almost two years later. Again, things that we knew had to get rolled out when we launched Argo for the first time, but it just had to take a backseat. Yeah. I'm always surprised, or surprised is the wrong word, but it's always interesting seeing what features seem like, oh, obviously, you need that, right? Some people are like, obviously, you need alerts if you're going to charge based on usage. And yes, ideally, we would have alerts. But it's always interesting studying what the M in MVP needs. Yeah. But in the case of alerts, you needed the infrastructure to actually trigger the alert, configure the alert. Those are all things that didn't exist anywhere. And was it on the onus of my team, or was it the right place to be placed within a different team to build it? That's a fascinating question. So, alerts is a great example of us getting bigger as a company, and then all of a sudden, now we have specialized alerting teams, and they built a bunch of infrastructure. Now you can build alerts really, really, really, really quickly, right? Instead of doing all that yourself. How did those conversations happen inside the Cloudflare product team? How do we figure out this needs to be core infrastructure, and then everyone builds on top of that versus this team should just do it themselves and see what happens? Well, I mean, what you want to avoid is teams building their own thing, right? You don't want to have six versions of essentially the same thing that is just a waste of engineering resources, right? Also, trying to expect a customer to understand, well, this is where you're going to configure an Argo alert. This is where you're going to configure a billing alert. This is where it's just a total whack-a-mole for customers. Our UI is not the best, admittedly, right? And trying to expect customers to figure that out on their own, it just doesn't make sense. So, if it's a shared service, it should be built by a team that focuses on those types of things, and billing was not the right team, nor had that right expertise to be able to put that together. The flip side of your example, though, obviously having six versions of the same service is better than having no versions of that service, right? So, I think that the existential trade-off is, do you want one? But then you get inconsistencies, right? So, then it's like, well, billing alerts, they get triggered at this time of day. And then it's just confusing to customers and confusing to people within Cloudflare, too. Yeah. No, I think this sort of like, do you build a tool as a platform, or do you empower individual teams to go do their own thing, is one of these age-old organizational management, especially as it pertains to software management problems that doesn't have perfect answers. Cool. Let's talk about some of the products you've actually brought to market. We've touched on usage-based billing a little bit. Not to keep asking the same question, but like, why was that hard, right? Like, what was the sort of like sea change that that represented, and why did that take the world on? So, it was hard because it was a totally new concept that didn't exist. We always had products that were sold on a prepaid basis. So, there was a reliability in terms of a customer knows exactly how much you're going to get charged month to month. Usage -based billing, completely variable, right? So, one month it could be charged five bucks, another month it could be charged 500 bucks. And trying to build that trust meant that the data that we're collecting had to be in solid. Also, being able to explain that to customers. One of the things that we always knew that we needed was to be able to estimate how much the bill could be. We still don't have it, like to be honest. I would love it. I don't know when we're going to get there. But to be able to build that transparency and have that mind shift was something that we had to figure out in terms of the enablement flow. So, you'll see some messaging, you know, do we scare customers right away and tell them pricing up front, or do we wait a little bit later in the flow? So, just that alone was one thing. Another thing was the mechanics of, again, counting things, right? So, we didn't have that pipeline and that pipeline needed to be built for the first time. And to be right now, I think it aggregates it on an hourly basis for every single zone, for every single customer. And that's a ton of data going through. And then being able to accumulate it and aggregate it on a daily basis. Also, making sure that it's in alignment with their billing period. So, we don't bill everybody at the first of the month or the end of the month. We bill based on the first time that they enter their credit card. So, your billing date is going to differ than my billing date. And to make sure that we're in alignment with that period is also, like, another tricky point of business. We also have, like, a tier that's usually included for a lot of the usage-based products as well. And so, again, trying to aggregate it and then remove or subtract that free allotment was also something different as well. So, all of those things combined, plus all the different vectors and the way that the teams are counting things, because we're relying on the other teams as well to make sure that they're counting things accurately. Because we have no visibility into all the different products that are being used and how they're being, like, what's a good request versus a bad request and how is it getting routed? Like, we don't know all those details. So, it was a lot of just, you know, collaboration with the different teams and all the dependencies that we had. Because, you know, the teams are focused on building the products and then the counting of it usually happens towards the end because you first need to build the thing that's going to trigger the activity and then count it later on. So, there's just a lot of just tight integration and coupling when it came to the usage-based billing bit. Yeah. So, it's the same theme over and over again, right? None of these things is super complicated on its own, but you add up, like, 15 different steps and then you have to do it thousands of times a day or millions of times a day or whatever it is and all of a sudden it gets really hairy. It does, yeah. Yeah. Have these projects gotten easier over time? Like, do you think you've gotten more experience as a product manager and you can, like, see around these corners more or it's the same, like, blocking and tackling every time? No, yeah. So, we've definitely gotten much better. One of the things that had come up in the past, especially because we were at one point launching almost a new product every quarter at some point, that one of the complaints was that billing couldn't keep up. And, you know, we've got… From a launch perspective, right? Correct. Yeah. And now we've got registrars. So, there's different types of TLDs that are being sold. We also have our marketplace where third-party developers are making submissions and we have to, like, support the selling of those products. And the constant theme was that billing just couldn't keep up. So, one of the things that we had done was, how do we build this out as a service? Like, you know, so that's available for self-service. So, the team… The product managers can go into some, like, billing store and say, I want that billing product to support my product without having to do a bunch of, like, requesting custom work from you guys, basically. Is that the idea? Yeah. Making it more self-serve and not, like, consulting. Right, right. So, yeah. So, we built out kind of, like, a catalog, if you want to say. So, it's basically just a different bunch of parameters that you can define. And you can basically make your submission for, this is how I want to package it. This is how much I want to sell it for. And it's, again, a PR to get submitted. And it's, like, almost a two -day turnaround just because we have to review it and just go through testing to make sure that it's in all the right systems and then push it to prod. So, yeah, it's good muscle memory that we've got. And it's becoming a lot more easy. Also, with that power that we've given, we're also a little bit wary about the way that certain things are being configured as well. So, there is a lot of just still consultation that's happening. But it doesn't take an entire quarter. It's down to, like, just a matter of dates. I can smell the finger wagging in me. Cool. So, no, I think it's fascinating to just hear, you know, it's, like, yeah, right. It's easy to talk about how, like, things like load balancing have matured over the years at Cloudflare or whatever. But, like, it's super important and cool to hear about how things like our billing systems have matured so that a bunch of engineers don't have to write a bunch of custom code every time. Yeah, we have a bunch of components that are available for our UI component library that people can use. So, basically, we're just trying to work ourselves out of a job. Yeah. The problem is the problems keep showing up faster than you solve them, right? That's true. That is very, very true. Blessing and a curse. Yeah, I mean, to your point, like, we were launching one new SKU, effectively, a quarter, and that felt, like, overwhelming. And now we're doing, I don't even know what the number is, and it's fine. Yeah. Yeah. Cool. Let's talk about something a little different. Actually, almost exactly a year ago, Cloudflare went public. What changed, if anything? I mean, you mentioned earlier, like, now the numbers that we produce get consumed by other people, but were there specific things that changed in the run-up for that? Yeah. So, we were using a vendor for invoicing application, and they did not have all the compliance that was needed for us to be able to hit that, like, I'm going to call it IPO standard. And so, we had to essentially do a huge migration of all of our customers. So, I think one of the analogies that Matthew uses is trying to change the wings of a plane as you're trying to fly it, right? So, we had to, because we couldn't stop collecting money, and we couldn't stop charging customers, right? So, we have new customers that are signing up all the time, and then we have our existing customers that are trying to pay us. We couldn't stop that from happening. So, trying to time all of that, as well as remapping everything, because the systems are completely different, it wasn't easy. And this was the first time that we actually had to do, like, something I'm going to call revenue recognition, which I don't know if people know what that is. I don't need to talk about it. That's boring. But yes, it's not glamorous, but it's important. Having to do all of that and, again, keep the business going while supporting new product launches, it was a huge effort. So, just trying to get all of the dependencies lined out, and all of the, you know, hundreds of thousands of accounts that had to get migrated over, as well as each of their subscriptions mapped over correctly, while not overcharging them or undercharging them, again, it was not a daunting task. While also trying to train customer support, well, these customers are going to be in the old system, and these customers are going to be in the new system, and then we have to run catch-up files, and also, we had accrued a lot of bad data hygiene over the course of X number of years, just because, again, we're giving things away for free, and we didn't know why. I specifically remember the email from you that was like, why does, like, Rustam, why are all these zones getting things for free? They seem connected to you in some way, and I was like, yeah, I was just, like, testing some stuff, and I clicked, like, yeah, yeah. But now we have KPI. It's like, well, you know, these number of zones account for X number of traffic that's going through our network, and again, trying to be able to clean up the data was not easy. I remember, for tax purposes, we needed to make sure that the addresses we had on file were accurate, and, you know, you're from New York, and the spelling of New York was just atrocious. Like, you'd have New York, lowercase, uppercase, mixed case, NYC. It's just, like, trying to sift through all the data. So, I think, I guess, maybe the theme of this episode is, like, important things that people don't talk about, right? Like, I think a lot of product managers, and if you read blog posts about product managers, like, oh, like, interview the customer and stay focused on their problems, and also, like, at the end of the day, project management is the super, like, unglamorous but ultra -important thing to actually getting things done, and it sounds like, you know, this project was an extreme example of that, right? Like, the dependency mapping and just timeline mapping and making sure that things are running on time was critical. How do you actually get good at project management, right? It's not like you can just read a book on it and then say, like, oh, I know how to do this. I mean, it comes with practice, right? The more you do it, the better you get at it. I think also just communication. As most other teams do, we have daily stand-ups every single day, just trying to keep on top of how things are going, and then we have our PM dev. You know, it's funny, you bring up daily stand-ups as, like, an obvious thing, but I feel like every time I've run into a situation where it's, like, information is not flowing properly, it's, like, well, are people talking enough, right? Obviously, it's possible to, like, have meetings too frequently and overwhelm and distract, but, like, so often, it's, like, this project is not going well. Well, are we talking regularly? No, right? Like, okay, should we try a daily stand-up? And then that magically makes things better, right? Like, so I think you're selling yourself short a little bit. You're, like, oh, these things are, like, obvious, like, yeah, right? But, like, not always obvious. Well, I think also active listening is important too. It's, like, you know, an engineer may deliver the message that he's still working on a JIRA. It's, like, well, you've been working on it for three days, like, what's going on? You know, just trying to, like, yeah, right, like, yeah, just sometimes people don't know when to escalate either, so there's a lot of time boxing that happens, right? Just focus on this for, you know, an hour. If you can't, then raise your hand and ask for help, but, yeah, just communication and setting expectations is really important too, so if you're able to deliver 85% of it, you know, can you declare victory or do you have to push the date? Just trying to, yeah, be transparent about how things are moving along, so and being organized too, like, organized through chaos, right? So, you do, you just do your best day by day, but, yeah, communication is important. I think, I think the other thing, just writing things down is so underrated. I mean, people say it's important, but, like, it's actually important and then I think, I think, as it relates to some of this that we were talking about earlier, like, the time horizons on unwinding a lot of the problems we're talking about are, like, actually doing all the stuff we want to do as product managers are often measured in, like, months or years, despite us wanting to do everything up front and, like, it's super cool going back and reading project plans and being, like, oh, yeah, it took us a little while, but we actually did every single one of these things. Yeah, yeah, it is awesome. It is awesome, yeah, but then it's also kind of disappointing where it's, like, oh, you thought it was going to take six months and ends up taking years too, so, yeah. Yeah, yeah, yeah. One other thing I wanted to touch on, so one of the things that I think makes Cloudflare special is that, you know, we obviously have a big enterprise business and these are folks that, you know, pay us lots of money and have, like, account managers and that whole enterprise -y thing going on, but then we started life as a company with a sort of you sign up on the website and give us a credit card and then, like, magical things happen kind of business and we still put a lot of emphasis on both of those things, right? How does that make your job, how does that change your job? So, we've spent a lot, at least from the billing side, we've spent a lot of time making sure that we have a good experience for our pay as you go customers. Just because the sheer volume and the scale at which we have, there's no way it could be done manually, so we've done a lot we could do and built a lot of features. To be frank, we haven't done the same with our enterprise customers. Just because there's so much more complexity in terms of their contracts, you know, they may have, these customers may have a soft cap saying that this is what they're going to pay us up front, what happens if there's an overage, and then there's other customers that are purely variable and the contract terms, they differ as well. It's not straightforward. It's like almost like every customer, well, we try not to have this happen, but like it's like every customer is like a different product, right? Exactly. Yeah, and also, you know, depending if you're like a reseller versus a partner versus a direct customer, the terms, they just differ. So, it's for those, for the enterprise customers and the contract customers, it's been a very manual process which, again, doesn't scale. So, one of the things that I'm taking on is challenging is trying to figure out how do we automate this? How do we improve it so that we don't have the situation where like you turn on Argo for 10 zones and you forgot about it or we have customers that are on trial and then a year passes and we're like, oh, we forgot to turn it off or we're not, you know, it's just the data hygiene has been not that great. Also, the amount of time that we have from the sales organization, like manually going in and setting the different flags on and off and putting calendar reminders to like turn it off for an XY customer, it's just not a good use of people's time. So, yeah. Opportunity for sort of software leverage. Exactly, exactly. Try to replace the things that are manual and human error prone and replace it with software. Changing topics sort of a little drastically and to talk less about or actually kind of on that note though, you know, I feel like this conversation is common with folks who have been around the company for a little bit. They're often moving into sort of more people management roles. How has that transitioned in from being a sort of like individual contributor PM at Cloudflare to managing PMs at Cloudflare? What's different or what's different between those two experiences and then what has matched your expectations making that jump and what surprised you? Sure. So, I love it. It just means that there's more work that can be done and it doesn't fall just on me. So, it's been a good experience. I have been lucky that I had, I was able to hire somebody into my team that basically covered for me on both of my maternity leaves. So, he's accustomed to working on the team and so it's not a surprise in terms of what the day-to-day is like. So, he was well aware of what he was walking into. I think also it's just for me, you know, I've managed people in the past. It's really cool just to see people grow and be able to impart my words of wisdom on them and be able to, you know, help them, you know, in whatever capacity that I can. So, that's a nice thing. The hard thing for me is, I mentioned this before, you know, being the control freak, right? So, especially because I've been at Cloudflare for over four years now and letting go is hard for me. But again, it's for their benefit. It's for the benefit of the company just because more things can be done and make that, I don't want to micromanage, like I'm not a believer in that at all, but trying to just be there and providing the support, also keeping them accountable of the work that they need to get done as well. So, you know, those are all the things that you have to deal with people and that's probably the hardest thing that people don't realize. Like, people just assume, oh, in order to progress in my career, I need to become a manager. Well, I think it's actually harder than being an IC because it's the people aspect, right? You don't know how that person is going to be on that day. Are they going to be in a bad mood? Are there other things outside of their life that is influencing their perspective? Are they being motivated? Are they challenged? Like, it's all things that you don't have full visibility into that makes it more difficult. Yeah, no, there's also those, I find it super, I mean, I agree with you. It feels harder than being an IC. Like, with an IC, or like, as an IC, you know, I see a problem, I'm like, oh, yeah, I know how to solve that. You just talk to this person and then, like, write some code and the problem goes away, right? And as a manager, there's a lot more nuance to it. Like, yeah, people are more complicated than software. And then that moment, like, oh, I actually do know, I have some advice and I think it's actually useful. It's like a really rewarding experience. Yeah, especially if you don't want them to make the same mistakes that you've done. It's like, oh, here, I've got a perfect example and being able to share that and just be like, avoid this. Yeah. Yeah, but then on the flip side, there's just those things where, like, yeah, there's no way I can explain to you how to avoid this mess. You're just going to have to crash the car once in a while. Yeah, yeah, right. Just make sure, right, then your job as a manager is to make sure the car crash happens in a safe and controlled way. That they're able to walk away, right? Yeah, exactly. Make sure the airbag is turned on. Exactly. Have fun. Good luck. Yeah. Like, I guess you've been at Kloppler for a little bit. Your role has changed. The company's grown a lot. What keeps you around? Like, what's exciting and, yeah, like, what do you wake up, like, stoked, stoked about? So, one of the things I always say, it doesn't matter where you work. Like, you could be working at, you know, the most leading tech, leading edge company, or it could be, like, the most, you know, giving company in terms of entrepreneurial spirit and goodwill. It all comes down to the people, right? So, no matter what, where you're working at, it's always going to be the people that make each day worthwhile getting through. So, here, some of the leaders that we have have been fantastic. Like, you can tell that they really care about the company and also the people. So, they're very genuine. They're very sincere in terms of the messaging that they send to everybody. Also, it's just the level of trust, like I said before, because it is still small enough where you can influence the way the company is going to go or certain things should be built. Being able to have that much autonomy is awesome. Also, it was, like, one of those things where when I first came to CloudFlow, it's like, really? You don't have X, Y, and Z? And it's still on my list of things I want to accomplish. So, selfishly for my own personal, like, goals, those are things I want to, you know, hopefully see to fruition. But, yeah, I think just overall, the people are some of the most creative and smartest people I've ever worked with and just the amount of energy. And people still get excited every single day. Yeah, no, I think, it's funny after spending a while at a place and I'm like, I think that, like, can do, like, I was actually, I was at a virtual career fair earlier today and some students were asking me and some folks, other Cloudflare folks, like, oh, like, what makes Cloudflare special? And it's like, nope. It sounds simple, but, like, we look at a problem and we say, how are we going to solve this? Not, like, crap, like, what do we do? Right? And I think that sort of, like, can do, we're all in this together spirit, like, a lot of things have changed, but that one hasn't, right? Yeah. It's pretty cool. And I appreciate the people that, like, you know, with that can do attitude, like, people are still scrappy, even though we've got, like, all this process and, you know, all these people, people come up with the best path forward. And again, it could be a shortcut at that point in time, but people still believe that they can solve all the problems. So it's, it's awesome. Yeah, yeah, no, I, it sounds almost cheesy talking about it now, right? Like, oh, like, just like all the smart people and the attitude and all that, but, like, that really, that really is it, right? Like, yeah, none of the technology or the data centers or any of that stuff matters. Yeah, we weren't actually, like, yeah, it felt like we were in a position to succeed. We got a couple minutes left. What advice do you have for folks who want to be like you when they grow up? Like, and that sounds really cliche, but, like, I think, I think there, I've given this some thought in the context of career fairs and stuff like that. How would you answer that question? I would say always be curious, like, keep asking questions, like, why, how, what, you know, what is it that people are doing, what motivates them, just trying to get to the crux of, like, what is the core problem, and then what is it that you're trying to solve for. So just, you know, asking questions, and then also on the other side, you have to listen, right? You have to be able to absorb the information and be able to digest it to something, whether it's, like, a requirement or just, like, a brainstorming, you know, opportunity just to, like, collaborate and figure things out, and being, like, a natural problem solver or at least the desire to build. Yeah, so that's one thing. I would also say don't play it too safe. Like, if you're launching a feature that doesn't get you excited or a little nervous, that means it's not really worth launching, right? So it could be, like, something really small in your mind. Like, one of the things that we launched was the number one Zendesk ticket that we got from customers or complaints was, I want my invoice. So years ago, we used to be able to email the invoices out, and we switched to a different system and couldn't, and then we finally had the opportunity to potentially do that again, and it was, like, one small thing that we rolled out, and I was, like, yes, we're going to, like, see the Zendesk tickets, like, plummet, right? Because it was huge in terms of how many we'd accrued over the past, like, two years, and we were excited to launch it, and we rolled it out, didn't announce it. We put, like, one little blurb in an email, and I think that first week, we got over, like, 5,000 users to adopt that feature, so it was something really small, and it made me nervous because it was, like, well, I don't know how people are going to receive it. Are people going to, like, notice it? Like, right? But it worked out fine, so you should be launching things that you really care about, no matter how big or how small, and be open to just listen. Like I said, people have lots of opinions. Being able to sift through and figure out what is the most important will help you figure out what the MVP is, and it doesn't need to be perfect. Again, like, I think a lot of people get into, like, you know, for lack of a better term, analysis paralysis, right? They need it to be 100% accurate, but you just need to get something out there and get the feedback, right? So either it's going to get adopted or it's not. It's better to put something out there when it's early versus when it's all fully done and you can't make any changes to the system, and it's part of that, you know, listening aspect and just making sure that you have your ears to the ground. I think you touched on, like, a couple really, really important points there. The, like, starting from first principles and just really trying to understand problems at their most basic level and just keep asking why until, like, there's no more whys to ask, right? Like, it's super important. I think the flip side of that, though, and it goes back to the people point, is, like, you need to surround yourself with people that encourage that, right, and want you to get to the bottom of things and, like, really actually understand things. Because I certainly don't, you talk to people where you're, like, you ask these basic questions and people are, like, oh, you don't know about, like, how some complicated thing works? You're, like, no, I don't, and I'd really like to understand, so please explain it to me, right? I think one of the things that makes Cloudflare successful is, like, that vibe is not present. Yeah, I mean, I've learned a lot of things just in my career here. Like, if you asked me four years ago if I would be able to run a postmanner class, I would be, like, what is that? But, you know, engineers have shown me, it's, like, little things along the way, and it's been good for me, and, like, I'm writing queries all the time, a skill is something I kind of knew, but I never really used, and now it's, you know, like, second nature. So, I've learned things along the way, and hopefully I've taught other people things as well. Yeah, that's cool. We've got a minute left. Thanks for joining. Any words of wisdom, any feedback on the interview? How can we make next episode better? You need to explain to me what the background is I'm looking at. Oh, okay. So, first off, if I turn off the virtual background, you see my very disorganized living room filled with surfboards and bicycles. That would still be a nice background. Maybe. This right here is from a backpacking trip I did last year in Alaska, and so, actually, right, this virtual background is not going to work very well. Right here is the tent I was sleeping in, and then, that's a tent right there, yeah, and then, actually, right there, just out of the frame, was Denali, the actual mountain. So, yeah, it was pretty cool, and then, like, right over that ridge right there is the Muldrow Glacier, which is this, like, crazy, like, half-mile-wide, mile-wide, maybe, glacier that flows off of Denali and into the McKinley River. So, it was really cool. Alaska's really cool. If, once people can travel again and do stuff, I'd love to go back. It's on my list. Did you do the glacier hike, or just look at it? I did. Anyway, till next week, all.