Latest from Product and Engineering

Hi, I'm Jen Taylor, Chief Product Officer at Cloudflare. I'm Usman Muzaffar, I'm not chief anything. I'm head of engineering of Cloudflare. Nice to see you, Jen. We've got two amazing guests with us this time. Very excited. Can you guys please introduce yourselves? Hey, my name is Gurjinder. I'm the engineering manager here at Cloudflare. Hello, my name is Justin Raczak. I'm a product manager at Cloudflare. So welcome, Justin and Gurjinder to Latest from Product and Engineering, our weekly talk show about product and engineering at Cloudflare. Wow, it sounds really fancy when you say it that way, Jen. It does. It really does. I'm working on my marketing spiel. There we go. So you guys are responsible for some of, I think, the most interesting aspects of the product, because there are aspects of the product that pretty much every user, whoever comes to Cloudflare, touches. What do you guys work on? Yeah, so Gurjinder and I work together on our onboarding team and our UI platform teams, which, as you mentioned, collectively is just about everything that customers see in the dashboard outside of some of the analytics. And for the onboarding piece, we have a really broad definition for onboarding at Cloudflare, where I think most people, when you hear onboarding, you kind of think of sign up. Maybe there's a couple of tool tips around the UI to tell you what to do and then, you know, job's done. But the reality is Cloudflare is a big platform. There are a lot of products. There are a lot of features that's become true over the past few years and will be, I think, even truer a few years from now. And so we also, when we're thinking about onboarding, think about helping customers figure out how to navigate the platform and figure out what products and features are appropriate for the problems that they're trying to solve or the pain points they're trying to mitigate. So when you think about onboarding, kind of, I mean, just to kind of put a finer point in that, like, where do you think it starts and kind of where does your job end in this universe? Tricky question. We'll give the answer, I think, for a start. So really, the customer journey obviously begins before we see them. So we partner with our friends in marketing for kind of everything before signups. But I think where we really consider ourselves to start driving the customer's experience is on that signup page. And then onboarding, where that ends kind of depends on the customer, which is one of the interesting challenges, I think, about the way we think about the broader definition for onboarding. So for some customers who have very simple CDN, let's say, and that's kind of the beginning and end of their needs, onboarding might be a short journey for them. For customers who are launching a new venture and their needs six months from now are different than they are today, the finish line for them in what we would consider onboarding is much further down the road. So it's kind of hard to say where the end is. I think the end is when the customer has kind of grown into everywhere they'll grow into. Yeah, I usually, when I talk about onboarding to my friends, my family, my people who I work with, so I usually say onboarding is the first touchpoint on the Cloudflare experience for any customer. Without onboarding, they cannot start the journey with Cloudflare. Now, the definition of when it ends, it really depends on the company. Cloudflare is so heads-on into making new products every day. And users need change every day as well, right? Today, I might want CDN, but tomorrow my needs change. Now it's a work-from-home experience. I might need more products which are more applicable to my current situation. So it's up to the onboarding team to figure out the ways to get those things ahead of customers, tell them, like, hey, these are the things that can benefit you, and help them in onboarding onto those products as well. So it really depends on the situation of a customer and what kind of customer we are dealing with. Yeah, I think one of the great things for the longest time, Cloudflare's website front page was give us five minutes and we'll supercharge your website, right? And that was as much of an invitation as it was a promise. Like, that's all it takes. And I remember when I was a Cloudflare customer, it really was that simple. Like, all you have to do is, like, get in, create an account, okay? The clock is ticking. You're at, like, one minute and 30 seconds. And now, you know, change name, servers, hit go, and bang. And that's up. And, you know, some of that oldest part of the code is still there. But, like, it's way more complicated than that. So, like, I should start with Justin. So it used to be five minutes, supercharge your website. So, you know, I've changed name, servers. My domain is now being proxied through Cloudflare. What's more complicated in 2020 than this was in 2010 when we launched? It's a great question. I'm wishing I had a really good analogy for this year. But, you know, I think I've seen analogies around across some of the teams talking about kind of a Starship dashboard, right, or a cockpit of a Starship. And I think, you know, the very simple onboarding experience made sense for all customers when the platform was a little more simple. When the onboarding experience maintains that simplicity, but what you get on the other side of the door is the bow of the Starship enterprise and these, you know, here you go, customer. Good luck. Don't break anything. You know, try things out. That's not going to lead a lot of customers, I think, to the type of success that they're looking for. And so what we want to do is guide them along those different controls, right, and help them discover what's appropriate for them or what might not be appropriate for them. As Virginia mentioned, the product is changing all the time. And so if we don't highlight new things for customers who've been with us for eight, nine, ten years, that's difficult. And they kind of find themselves sticking with what they had originally and missing out on all this new value and new opportunity. And so figuring out the right ways to say, hey, there's a new thing, and we think it's appropriate for you, check this out, making that simple and easy for them to do. Hey, there's a new thing is a great challenge, right? Because at Cloudflare, one of the things we're so proud of is how prolific we are. You know, we just have announcement week after announcement week, shipping new products, you know, attacking so many different parts. The vision is if there's a packet on the Internet and there's a way to make it faster, safer, more reliable, Cloudflare should be there to help you make that packet faster, safer, more reliable. So talk a little bit about how did we try to make something obvious that wasn't there? Like you should know we have a new product here, we have a new thing here for a customer to try. What was involved with sort of surfacing that and showing it to a customer so that they could enable a new product or a new feature? Yeah, you want me to take this one? Yeah. So, again, we kind of tried to keep a tab on like we did not have much insights before, and we were working constantly on figuring out what works for customers in this onboarding flow by making sure we have gathered enough data around what customer journey looks like on dashboard. And what are the pain points of those dashboards, learning about them and improving those pain points to surface more data on those pain points. And that's had been a part of journey in my last one and a half years at Cloudflare when I was working in this onboarding team to enable product and everybody who works on this product to make sure they have enough data to surface to make the kind of decisions that they want to make to figure out where is the pain point lies for our customers. And I think now we are at a point where we are kind of now doing research based on all those data points. And we have done some experimentation around showing the recommendations on the customer dashboard. When customer signs up, we show them kind of recommendations like, hey, have you considered doing this? Have you considered doing that? Which gives customer a big dip into like what other products are available within Cloudflare. That boils down from one main pain point that we discovered is that customer when they sign up for Cloudflare, because Cloudflare is packed with so many products. It was like Justin was mentioning, it was simplistic sometime back, but now it's so complex like a spaceship. So customers do not have any idea what they're getting into. So onboarding is trying to get up to the job where we recommend certain things. And we have done another experiment where we are talking about when customer onboards, we show onboarding guide. We walk the customer through many different settings that might be applicable to them and could secure their website more so that they can make a decision on that. And I think Justin can explain more about how that experiments ran and how the data looks like so far on the high level. But yeah, that's enabling Justin and anybody else on making sure we have enough on doing those journeys. Spoken as a true engineering manager, hey, we built what you asked, you decide, you tell us if it worked or not. I think your agenda touched on the really important part, I think. So to your question is how do we figure out the right way to do this? The short answer is we ask our customers, but we do that through experimentation. So we try different ways of highlighting products embedded inside dashboard. And we kind of say, you know, was this helpful? Did this get more people from noticing something like an Argo product to actually deciding it was right for them and activating it for their domain? And if we put something out there and it seems, you know, 0.0001% of people are clicking on it. We either didn't talk about it in a great way or we put it in the wrong place. And so we said, OK, that right there, that was not the strategy for customers. And again, interesting challenge that is particularly with the higher level parts of the Cloudflare dashboard is that these interfaces have been around for a long time, particularly in our account level things and our domain level roll ups like the domain overview. And so they they had a purpose when they were built and everything kind of changes in the context around them. And so something that we're having to look at right now is do these context still make sense? And asking those questions to customers through those experiments. So what is the purpose when it's just CDN? Way back in the day when it's just CDN, what is the purpose of a page that tells you everything about your domain? Now, when we have the robust offering of products that we have today, does the purpose that we decided for originally still make sense here? Or do we completely reimagine something new? Is it something that guides customers to the 30, 40, 50, 60 different features that they might be interested in checking out rather than just a summary of the domain? Is it a place where we make intelligent recommendations based on what we've seen from a customer's behavior? These are really fun questions to work on. And I think it's an exciting time in our evolution to kind of go back to some of those initial assumptions and testing. Well, and part of what I love about what you guys have done with some of the onboarding and onboarding guys is, you know, to your point, like you open the door and there's a big spaceship and you can't you can't even figure out where the like the power button is for the spaceship. You're not going to get a lot of value out of the spaceship. And a big part of what you guys have done is really create kind of guided and curated onboarding flows that kind of steps users through the process of configuring and leveraging some of the core capabilities within Cloudflare to really get them started. Kind of how did you come up with that idea and what have we learned as a result of doing that? Yeah, great, great question. So I think we took a really micro approach to our experiments with this. I think I benefited from coming in late last year. So Cloudflare kind of reached the very robust period that it's in now and got to come in with fresh eyes and navigate the dashboard myself and try to figure it out. The big takeaway for me was there is a ton here, but I've got to go find it because we have a ton of teams working on a ton of products. And so those things are kind of living all over the place. And they're kind of organized by product. And so for me, customers must be going through that same experience. They've got to kind of hunt for the things that matter. And so we have this hypothesis. What if we take some of those things that are scattered across? I think it was 29 different tabs at the time. Was it really 29 tabs? 29 tabs and sub tabs and 71 controlled for three customers, just for three customers. Hell of a stat, Justin. Okay, good. This is my banner from back in December, how we got this project rolling to really rally around those numbers. So what if we take some important ones and we put them directly in front of the customer? Well, they've got that momentum of signing up. They're in the flow. They're bringing their site over to Cloudflare. And let's have them take a look at a small number of things that we think are important. Now, because we wanted to more test whether that was a good way to surface part of the UI for customers, we actually picked things that weren't that important. First, things that they could turn on and off without really damaging or causing much of a change to their site. Simply to see, hey, if we do put this here, is this useful to customers? Do they engage with it? It was a kind of obvious hypothesis, but the experiment bore out what we were expecting. We saw more than twice as many customers engaging with twice as many settings if we put them in that guided experience. And so that was, again, kind of unexpected, but I'd say overwhelming success. And so then we took a very simple implementation for that and kind of beefed up the design. And now we're on to what we call milestone three, which is now we're taking a look at that content. We said, hey, this guide thing works. Customers respond to this. They actually follow our guidance. Let's take a look at what we're showing them here and now move into the very important settings. So we partner with friends on the security side, for example. So, hey, what's a really bad thing the customer has punted out of compliance with the recommendation that we made? And so we've got a security focus section in the guide that says, hey, you're not doing HTTPS rewrites. We recommend that it helps fix mixed content errors and secure more of the traffic flowing through your site. And so I think it's an exciting next step for this guide, because, again, very, very simple experience, simple hypothesis. But now we're getting really important changes to customers' sites and their configurations as soon as they're signing up, rather than hoping that they make it to tab 27, you know, control 19 down at the bottom of the page. Well, especially because if we're successful at Cloudflare, a big portion of what ends up happening is a user sets it and forgets it, right? Like, it's like I set up Cloudflare, my site is faster and I'm not getting hacked, so I'm not going to bother to come back. And as a result, they may miss those opportunities. So trying to get them early in that conversion is important. Sorry, I interrupted you. No, not at all. I was just going to say, like, important lesson for everyone out there who is for a SaaS product. Like, notice Justin's story started with, I use the product myself. And like, there's nothing more powerful than that. Like, go use your own product and you'll learn so much. One thing that is often true at SaaS companies is that, or even just software architecture going all the way back, is that the UI, we even talk about it as the front end or the top end. But if you think about what Justin was just describing, how the UI is surfacing things like security settings and the certificate settings on an origin, that means that, Gurjundha, your team must have had to integrate with all kinds of back-end teams to figure out how to actually surface a detailed recommendation, like your Cypher suite or your security settings wrong, SSLs wrong. So, like, how does that work at Cloudflare? How does the dashboard team, you know, front-end JavaScript engineers get information about the customer settings of their origin to surface that into a nice friendly button that says, click here to secure origin? Like, how does those, what's the plumbing underneath the covers there? How does that even work? So, I would say, like, we leverage the work that has already been done by the teams. And because we use React, which is, again, a component-based library, you develop components, and those components are reusable pieces of work that can be used anywhere in your website. So, what we became very clever to actually reuse those components that have been developed by different teams by talking to them, like, figuring out what needs to be done to make them in this onboarding guide, like, use them in this onboarding guide. And it seems like it was pretty simplistic to move them into our onboarding guide. The real challenge was how to display that information to them. Kind of in what fashion, in what steps? Like, what do you do? What do you do first? What do you do second? How do you make sure you keep track of your progress? So, I think the real challenge was not really plumbing into the work back-end systems. That work had already been done by multiple teams across Cloudflare. And we reused the work that they had already done by reusing components. But if you look at our onboarding guide, like, you set up a new domain, you will be already, by default, be taken to onboarding guide. We show it like a stepper now. You scroll through the stepper and make sure, like, you have gone through the settings, and we persist those settings in your session so that when you go back to your overview, you can choose to skip it. Like, I'm halfway around. I don't have time right now. You know what? I want to come later to finish this off. You have an option to finish off later and go back to overview page, do whatever you want to do. We show you now a sweet little progress bar. And that progress bar is driven by the data that we store in the shared session about where the customer left off so that they can go back to the onboarding guide right where they left and make sure they have gone through all the settings, save them, and keep coming back later on if they want to. Great. Great software reuse story. So, it's literally the promise of React being able to do that. And then, you know, a lot of talent with design to make sure that we actually surface a story that makes sense. It's great. But I know that, like, when you guys built that, you spent a lot of time figuring out what the sequence of those things should be, and you really had to kind of lean into and leverage measurement. And I'm sure lots of folks are listening are like, oh, yeah, you just power up Google Analytics or you just power up, you know, some third-party, you know, embed your third-party JavaScript into the interface, and you're off to the races. But that wasn't quite the experience you guys ran into. Can you talk to us a little bit about your measurement journey? Genda, this is all you, my friend. So, from the measurement journey, by measurement, I'm assuming you're asking about our analytics pipeline, like how we get our data in there. So, you were right. We were using multiple third-party libraries to fetch our data from our dashboard to go to Google Analytics and keep. Again, first of all, that was not secure. So, I'm more talking about from the engineering side now. That was not a secure way to do the things because now we are downloading the third-party libraries. So, Cloudflare engineered its own library to capture user events. We call it Sparrow. It's a nice branding for the name, though. So, we capture user events now with their own piece of work, and we have funneled the whole backend system, which is an analytics pipeline kind of CDP, which is, again, the customer data platform. We get the data into our CDP and route the data to multiple types, like Google Analytics, where people are interested in our marketing side. People are interested more into analyzing how the data is flowing from marketing, and sometimes we have needs of making sure we look into certain trends and see what people are doing on our dashboard, and we write those measurements into a heap, for example. And Justin, when Justin came on board, our team, I think he did a lot of painstaking work and going through those dashboards, figuring out what makes sense of the data, what customers are doing, what settings make sense, what are the top five things that customers do, and sometimes we have a lot of tickets in our backend that comes through, like, how do we do that? How do we do that for our customer support representatives? So, we compiled all of that data to figure out what should be our top five things that we display on our onboarding guide and go from there, and I think that's where Justin worked with a lot of different teams to figure out, like, does it make sense to have the setting on there? And we basically, I believe we have all the data. We figured out how that success story is going, and it's pretty successful. Yeah, I can't stress enough how much of a boon it's been for us to have this technology in-house. I think there are two things that I love about it. So, first and foremost, to Gajendra's point, so we're a security company, right? And so, the fact that we have full control over the code and we know what's going on and nothing's going to sneak in a backdoor somewhere, and then, consequently, we subject all of our customers to that. So, that's obviously huge. And then the second is the control, right? So, again, Kepler, very security, privacy-focused, customer -centric. That allows us to embed that philosophy and decide the right way to monitor these analytics rather than Google or Segment making those decisions for us. We can listen to our customers or our customers listening to their customers and make sure that the way we're capturing these analytics are in line with our philosophies and our values as a company. And the third is performance, right? I think we leverage, like, why send on any of this stuff to the client when our whole point is to minimize bandwidths of the client. So, it made total sense for us to absolutely control at the bare minimum and then leverage workers and the other technology we have on our server side to broker it and to slice and dice it every way we need to and can do all the great analytics and analysis that Justin and the rest of the product team was working on. Yep. I think the best piece of this whole thing is that we are not putting our own product, which is workers. This whole thing is built on our own product, workers. And it is so efficient because we are not even sending the traffic back to our core data centers. Everything is computed and everything is routed to the edge. And I think that's our biggest advantage. It's very cost-effective. It's basically very fast performance, cost -effective, can lead to data going in there, like, instant. Yeah. One other thing that you worked on is deep linking. And, you know, so part of the dream of a marketer is in the same place that you're talking about and explaining and you're going through all this trouble to explain to a prospect, here's a piece of solution that might be interesting to you and you want to be able to just warp them directly into the part of the product where they can then play with that. Yeah. Instead of dropping them at 29 tabs and saying, go figure it out yourself. Like, why, you know, let's just, you know, if we're using Star Trek analogies, you know, like Scotty, beam me directly to the panel where I can turn this thing on or off. So talk about deep linking. What's deep linking? What was involved? What do we have to do? Actually, maybe Justin, you start. What was the vision? Yeah. So I think maybe I'll start with another personal anecdote because I always love customer empathy. Well, part of exploring, you know, products and the website before I'd actually joined the team, I signed up, I brought a personal site on and I found this really great thing our marketing team had put together called the Welcome Center. And it had a lot of great content in their instructions. But there was one thing that really stuck in my craw about it, because the marketing website and the product dashboard are different applications. And so we'd say, hey, here's this great thing to your point, go turn it on. And kind of the best we could do is put you at the top of that traffic page. But it's not actually traffic, it's traffic. And then this one over here and then tile number three down at the bottom. The concept for this Welcome Center is fantastic. It's comprehensive. It's informative. This is great. What a pain in the butt that I've got to click around and we're lacking the kind of those instructions aren't there. Right. It just drops you on the page and doesn't say, OK, now put here, now scroll down. And so that was really the use case here to the original point. That's a superpower for marketers to be able to talk about these things anywhere and send the customer directly to remove the friction for the adoption or for the discovery. So I think that was that we had a very clear vision for kind of what the product and the customer case was for this. And I think the tech underneath it is super cool. Yeah, exactly. Spot on. So, again, the motto of the onboarding is to make things easier for customers in the dashboard context, to make sure customers are able to get to where they want to go in the least amount of time and effort. And one thing which was lacking within the dashboard was the capability of easily deep linking, creating the deep linkable URLs that can be used from emails that can be used from, let's say, your support person sends you like, hey, can you try doing this and sends you the URL to go directly to do the stuff that you want to do. And also create a kind of platform where tomorrow, if we want to have these search capabilities within Cloudflare dashboard, which is another stuff that we are actively working on, figuring out how to make customer searches better, where customers can type something, keyword, and we can list you a setting. You can click on it, directly go into that in the dashboard. I think that that was a need of doing the deep linking, like having the deep linkable framework, which is resilient, kind of like performance is pretty good on that and making sure like we have a URL that makes sense. And end of the day, if you know like a URL does not end there, if you go to a particular set of particular page, there are 10 different settings on that page. Like, how do you go to the setting that you are interested in? So you have to have a framework that can help you do the scroll itself. You tell a URL where you want to go, and the URL takes you there, and the page automatically scrolls up to go where you want to go. I think that was the promise of the World Wide Web. It just got messed up by web applications, especially single page apps. But no reason why good engineering and good information architecture can't bring that promise back. And now look how many different use cases open up. Search, deep linking, marketing, you know, being able to jump people, help desk articles, like everywhere you look, it's like, why don't we just take the customer to the button they press? Let me talk about one of the, I think we passed over one of the cooler capabilities of this. We have a lot of customers who have tons and tons of sites running on the platform, right? And a lot of times where we want to send a customer specific to a site, maybe not a specific site, but one site, where we want to say, hey, go check out this great feature for a site. But this customer has 200 sites, so we don't want to launch 200. So what they did, they built this generalized router where we can say, hey, we don't know the account, and we don't know the site, but send the customer there anyway. And now that gets caught by our account selector in the dashboard. And once they pick that, it says, great, now you need to pick the site you want to go to as well. Then they pick the site, and then the flow continues. So all of that context is maintained, so we can go from ambiguous email link all the way down to 70% down on tab number 20. So there's a way now to create bookmarks that would really be only meaningful inside a company, inside one of our Cloudflare customers. It's never something we're going to see, but it will be customized in their internal docs to say, if you're the administrator of SiteFu, here are links to jump you to the settings that you might care about for that. Exactly. That's awesome, guys. I didn't even know you built that. Fantastic. It's a good thing we had this call. It's a good thing we had this call. It's a good thing we had this call. So Jen and I are apprised of some of the extraordinary work that our team does. So you guys, I mean, you have a really broad surface here. You've got onboarding. You've got the measurement framework. You've got all these sort of synergies and connections. You know, as I look at the dashboard now and I sort of squint at it, sort of what are some of the current challenges you're hearing from customers? What are some of the things you're sort of chewing on as you think about what might come next? This is a great question. So I think one of the biggest challenges we face right now, particularly with kind of the dashboard as a whole and how customers get around, is a really, really common pattern. I think you are more prone to it the faster you go. So the fact that we're here is kind of a hallmark to how quick we get new things out to customers. But if you're not paying close attention, you can kind of ship your org chart. You sort of hear people talk about writing. What it means is that the navigation of the UI really reflects more of the way we work on our products and not entirely the way customers think about using those products. And that's a really interesting challenge, not just because, again, we are a big platform, a lot of products, a lot of features, but because our customers are also very different amongst each other and between each other. And so there is not really one super crisp, clear way to take it from where we are today to where we're going to go next. But that's something we're working on right now. And there are a couple of ways to software. Gajinder mentioned one that I think we're really excited about, which is the search. You know, one great way to get people around where they want to go is just let them tell you where they want to go and work them over there real quick. Done. But also, we want to make sure that for folks who are more comfortable pointing, clicking around the dashboard, that the navigation experience, they have maps to the way that they think about using the product. And then again, I think the feature in product discovery is going to be an interesting and evolving challenge as we move forward, because we are certainly, I would say, based on my last year here, we are shipping things out much faster than we can test. And so we're really kind of more thinking about, you know, Gajinder mentioned about some recent experiments about just making simple product recommendations. We're thinking about frameworks. So less about how do we take this new cache analytics feature and put it in front of customers? And it's more, hey, where in dashboard is a good place to make a recommendation to a customer where they can learn, hey, this is a place to check where I can find out about new things that are relevant to me. And so we get that piece. And then we focus kind of separately on what goes in there. What are those actual recommendations? And so we think about kind of frameworks both for the product recommendations. And frankly, we think about the onboarding guide in the same way. We've got this guide and it does this thing. And we can kind of plug and play content because, for Gajinder's point, we have this great modularity in a lot of the dashboard. And so we need to bring new content in. It's pretty straightforward for us to do that. So those are, I think, kind of the two big challenges ahead of us. So great, guys. Thanks so much for joining us. As always, I'm amazed how quickly the 30 minutes goes. I can keep going. I've got like 12 more questions for you guys. We clearly need to invite Gajinder and Justin to come back. Thanks so much for joining us this afternoon. And thanks, everyone, for watching. Thanks, guys. Thank you. Thank you for having us. Bye. ♪♪♪