Delivery Management at Cloudflare

All right, we're live. Welcome everybody who's tuning in. It's me again, me again from yesterday's segment. I'm here again to talk about delivery management at Cloudflare but today I have Andre who is an engineering manager. So Andre, tell us about yourself and what team you're on. Howdy, howdy. Despite my accent I am based here in London which is a little bit grim and surprise, surprise. I am the engineering manager for the managed rules team. We are the team that owns the WAF, the web application firewall. So we are the team that owns the kind of rule sets around protecting our customers and protecting them from cross-site scripting and SQLI and a host of other malicious requests. Awesome and we work together on the same team. I feel like that should be a disclosure of we chat a lot and so if this if this conversation does go off the rails a little bit it's only because all of our conversations do. Exactly. So we've been working together since July, is that right? Yeah, so I moved over from Cloudflare San Francisco office in January this year, great timing, and we worked together a little bit. You were on the team and in stand-ups for a little bit at the beginning but I think you were working with another team here in London. So the answer to your question is yes. We started working together in earnest in June, July, sometime over the summer when you came back to us full-time. Yeah, so I'm interested to know what during January and July what was your experience like kind of working, you know, onboarding onto the Managed Rules team and then since July working with me what's it been like? Panic and abject terror is the answer to both honestly. So when I joined the team it was, so this is a first tangent, as an engineering manager it is very rare that you join a team that's going well for various reasons whether the engineering manager left or a new team is being spun up. It's always a time of chaos when you join a team and so that was what I stepped into. The, what was then known as the laugh team was at a time of transition and we're looking to shape things up, shake things up even, and so it came into a little bit of just disorganization. There wasn't a clear roadmap, there wasn't a clear objective, there was a bunch of desires and goals where we wanted to be in, you know, two, three years, but there wasn't a clear path on how to get there and so basically what I did when I started was kind of do a whole bunch of things from scratch and so it was honestly quite good that you weren't on team and, you know, with us to start because there wasn't a whole lot of organization or anything to even do. There needed to be started a bit before you could come in and then help us build the process on top of that and so the first, you know, even three or four months was getting the team together, figuring out how we worked together, and getting a direction before we could get a process. There was so much transition as far as technologies, you know, Richard and Alex, Richard is another engineering manager and Alex is another delivery manager here in London, they have talked previously, they were on Cloudflare TV last week with Jen and Usman and talked about some of the things that are being transitioned in the broader firewall organization and security organization and we're undergoing the same kinds of changes. I was on two, three months ago now with John Graham-Cumming and we talked about some of the changes that the WAF team is undergoing and that was really the start of it, it was early this year. Have I answered your question yet or should I keep rambling? What about the latter part? So since, you know, since kind of working with me, has it been better? Yeah, absolutely. So like I said, we needed to come up with a plan first and then we needed the process and so when you joined us we were ready and a little underwater with the process and so you joining the team and having a delivery manager on the team was fantastic for keeping us organized and helping us not slip back into that disarray and abject terror of, you know, not understanding what's going on, not having clear deliverables and so that was the kind of title shift then was getting, you know, we use JIRA internally for our ticket tracking and sprint planning and all that kind of stuff and so getting all of that in line and honestly helping me keep everyone honest with story points and with, you know, only putting things into the sprint that you're going to work on that sprint and not using it just as a hopper of like, I'll work on this sometime in the next couple weeks. Getting everyone more on the same page as far as workloads and organization and prioritization. Yeah, totally. It's also safe to say that we've created, you know, we've experimented in the team. We weren't always working to sprints before, you know, before you joined. I think we had a bit of a flexible scrum ban way going on. We had a camera on board, we were always putting things in. The team also has to be quite reactive because it has to deal with like customer escalations coming in or rule requests so there was always a bit of flexibility they needed to show in terms of their planning but we've experimented. We've tried things in the team to find out what works for us and now we're at a point where we have established sprints and we've realized actually we do want to do estimating in terms of story points but based on complexity. It's something I was talking to Michael about yesterday. It's not so much how much time we're going to spend on doing something, it's more about the complexity of the issue and some of the things you touched upon like, you know, we're part of the wider firewall security team so there's a bit of overlap with them and how do we, you know, how do we kind of create that collaboration with them and work together with them and not just in isolation as like just the WAF and managed roles team. Alex and Richard will be on later today, that's a little cheeky plug. They'll be on at 5.30 later today talking about the same thing but from a different perspective if you also are interested in tuning into that. So Audrey, if you had to pick three things that are going really well right now in the team, what would they be? Lists of things. Three things are going really well on team. Would you like me to constrain this to specifically like delivery management or like pie in the sky, what's going really well? Just delivery management. Just delivery management, all right. I think one of the things that we've really gotten better at and it kind of took me by surprise honestly, I wasn't paying attention to it because I didn't ask you, thank you, was that we're really good at making sure all of our tickets in JIRA, again this is going to be hyper specific to our kind of workflow here, but all of our tickets in JIRA have story points now and so you and I talked about this earlier in the week and surprise to anyone on the team watching this, we are in a place now where, you know, the better part of, you know, eight, nine months later where in our sprint planning meeting we can start paying attention to story points and, you know, follow the actual like kind of scrum sprint planning process where we say, okay, you've got 30 points this sprint, that's not going to happen. Try again. What needs to be kicked out? And so actually paying attention to, you know, the efforts and things that we are agreeing to. One of the things that I have found super helpful in the past and has really helped the team with estimation and, you know, communicating externally to the kinds of things we can commit to is getting good at estimation and it's taken us most of this year to become comfortable with estimation and one of the things that I've really talked to the team about when doing story points specifically but generally thinking about how long something is going to take is I always, always, always want the pessimistic timeline. Tell me how long you think it's going to take if everything goes wrong. And so that's that, you know, that's the thing that we should be communicating and likely we add another 10 to 20% on top of that just in case because estimating is hard. But we've flexed that muscle over this, these past few months and we've gotten to a point where, you know, low bar, we've got points on everything which is a good start and then we can continue to fine tune them and so the thing that is going really well is now we've gotten pretty good at that with, you know, you staying on top of people and making sure that everything has a point and when we're in a sprint planning meeting making sure that we don't start sprint without anything having, without anything having no points. And so that sets us up for being able to say, okay, you have however many points in this sprint, you only accomplished X amount of points last sprint, let's split the difference to see if we can accomplish this in the sprint and kind of iterating on that effort. And the other thing, the other part of that that I've tried to communicate, I'm going on a tangent here if you're going to wrangle me. The other thing that I've tried to communicate to the team is it's all relative. You know, the one thing that I will make a conscious effort to do and promise that I will never do is compare one person's, the amount of points that they are accomplishing the sprint to someone else's. What this is meant to do is help us in estimation and figuring out how long this particular project is going to take for you because everything is different. The whole points of story points is that they are vague. They are meant to convey a, like you say, complexity and not necessarily time. There is no direct mapping to story points to time. And so the advantage of all of this and the reason we do all of this is just to help us with estimation. If you see a weird white thing flipping around in the foreground here, it's a that I'm fidgeting with. Yeah. And one of the great things is that you mentioned, you know, it's not, story points aren't a judgment of what one person can do over another. You know, these are used to create transparency in the team of what we can accomplish together. The same thing we, you know, another one of the things that it's allowed us to have is informed conversations. What can be made possible and not be made possible and why? And that leads us on to, you know, the other element that's really important is outlining or rather surfacing dependencies and risks and issues that we are going to foresee in our projects. Really earlier on to that, it's quite clear in terms of all the things that can go wrong and really, you know, derail us. And another thing that's really helped us is delivery metrics. We've been having informed conversations with the team by putting delivery metrics at the end of each sprint. So we're looking at, you know, the team's velocity, you know, how fast are we and if we're not so fast, what took us so long this sprint and what were some of the risks and issues there? And usually the conversation around that is how did we misestimate whatever this thing was? Exactly. Yeah. And moving forward in the next sprint, how can we do better? So awesome. So on the flip sides, three things. Well, hang on. That was only one thing. Taking up time here. So I've got at least one more thing that I can talk about. And the other one is success metrics. And the thing that you have really, really helped, and it's corollary to this being able to measure things, is what kinds of things are we delivering? And so one of the most interesting things that you've put together the past couple of quarters is a breakdown of the categories of things that we work on, whether they are product related or internal or I've actually, I hadn't heard this term until Cloudflare, BAU, business as usual, as the everything else. In one of my former companies, we had a team called the reactor team. That was the, that was, they did everything else. And so that's been super helpful, not only internally to say, we don't want this reactive number to be this high. Again, how can we fix that? How can we pay attention to that? Or we haven't been tracking it all quarter. Accidentally, we worked on 50% of reactive things that we don't want to do anymore. How do we get that number down and paying attention to that? But the other important thing for that is the communicating upward and communicating to the executive team and to our managers about here's the breakdown of things we can point to a specific, we were interrupted on something. And this is why this particular project or series of projects were, took so long is because we were interrupted and here's the number. We had a whole bunch of things that came up for various reasons and we needed to prioritize them as something else. And so that has been ridiculously helpful in, it's going to sound like an excuse, but an excuse as to why either projects were delayed or something took so long, or we didn't hit deadlines that we agreed to three months ago. Yeah. Yeah. I mean, it's all, it's all fair while having, the way we do planning at Cloudbearers is through quarterly cycles. So every quarter we commit to the business, what we're going to do and achieve in the quarter. And it's all fair while kind of slipping dates and changing the dates. But if you, if you're, you know, what, in my experience, what I found was there was just no concrete evidence as to why dates were slipping or why we were unable to do something on time for a particular customer. And that, you know, we needed that data and information to be able to have an informed conversation with our stakeholders and the exec team to say, Hey, we didn't do this because of all of these distractions. And that's now allowed us to reflect as a team, say, okay, that number is too high for us. And we would like to decrease it the following quarter. We don't want to be that reactive, you know, and also identify what, what does good look like in terms of operational toil for us in terms of how much time do we actually want to spend on it? You know, typically in kind of software engineering terms, like you're looking at 20% time is considered BAU operational, keeping the lights on is what, what, what engineers should be spending on. But we don't know if that 20% is the right amount for us as a team. So, you know, as Andrew mentioned, we did, we did do a, I did do a report for like last quarter. We spent 50% of our time somehow, somewhere, and then we realized we, that, that was too high for us. So now we're, we're actively addressing that this quarter. Yeah. And that was one of the goals as a team that we talked about back early in this year is that as a team, we not only wanted to, but needed to become more proactive and, and less reactive when it comes to security, you know, WAF related projects. And so we, it was a balance because we still had to respond to those things, but we needed to get in that transition period. We needed to become more proactive. We needed to spend time building the systems and the platforms for us to be able to become more proactive while also dealing with these, these reactive things. And so the, the conscious decision to say, okay, we, we don't want this number to be this high really gave us a target of a budget basically of, okay, you know, we've, we've accomplished this many reactive things so far in the quarter, where should we be? We should be, be doing fewer, more. Can we, can we accomplish a few more things? And it, it, again, it allowed us to measure something and then we could improve it. Yeah. So last but not least, what's the third thing on your list? Yeah. So the, the other thing is less like project wise and delivering code and that kind of thing. It's more people wise. It's, it's been super helpful because of the, the context and the conversations that you have with people on the team. It's fantastic to just get another source of input about how people are doing on the team and how they feel about the projects that they are working on. And, you know, having another person, you know, confirming or denying the things that I'm seeing that that's happening on the team, whether people are happy with what they're working on or, you know, it's, it's, it's super useful to just have that other source of input. You know, you, you're by no means, I don't want to imply that you're like spying on or, you know, giving me any insider information about anyone on the team. That's definitely not the case. But what you are able to do is help me raise things that may be happening on the team of, you know, this, this person is, you know, they're not hitting their, their estimations consistently. And, you know, what is the reason for that? And, you know, then it's my job as, as the engineering manager to go and talk to them about why that is. Either they're really not enjoying what they're working on or they are finding getting up to speed on a particular thing difficult, whatever the case may be, just having that other signal. And, you know, again, it comes down to measuring, being able to measure these things and realizing that we're not hitting these, these targets that, that we wanted to and addressing them. Yeah, totally. That's a great dynamic to pick on. Actually, the way our relationship works is you're the line manager. That's your, you know, that's your responsibility as the engineering manager. My ask, I care about team health and team happiness. That's the most important thing because whatever we're doing as a partnership between engineering manager, product manager, you know, and myself as a delivery manager, if it's not working for the people who, you know, on the team, then we're not doing it right. And if they aren't happy, then, then, then something's clearly broken. So the, the team happiness and health element in kind of software delivery management is really, really important to me. And also agile delivery isn't something, it's not, it's not a tool you can't just chuck it at the team and say, this will fix all your problems. It's, it's, it's a, it's a mindset shift and it has to work for, for the team around it. So we figure out what works for us. It's not, it's not, it's not a process that's, you know, enforced on the team at all. It's, it's a collective effort. So it's really important to have them have their say so that we can make the right decisions on their behalf, you know, give them the right opportunities, let them work on the right projects that will upskill and grow them and, and, or pair them on the right set of, set of projects, but also have their input in terms of where they think the most problem areas are that we need to address. So that bit is like really, really important as I feed that back to you so that you can then, you know, enables you to be a line manager, but tailor it to what is actually required for, for an individual. Exactly. And it's, it's 100% in the spirit of agile development of continuing to iterate. And that means iterating on our process too. So if like you say, something isn't working, we, we recently updated. So at the end of every week, we do a team wrap up and we, we got together and decided that what we were doing wasn't working. The format that we had it in wasn't working. And so we decided to change it up and make it more interesting. And we got more engagement and it was more interesting partially because it was novel and something new and different on Friday. So we might have to do that more regularly. Yeah, that's, that's actually was quite good. What about, so what about challenges? Because not everything's, everything's perfect in the team and we do face some, some challenges. So if you have to, you know, pick three, again, the right theme of picking lists, what would be some of the challenges you think we're facing right now? Yeah. All right. So this is my time to do a on-camera live public performance review. Is that where we're going with this? One of the challenges that we are facing is intra-team coordination. And it's a challenge absolutely everywhere. And it's even more of a challenge because we're all working at home. And the, the exchanges, the more natural kind of, I just need to wander over, or we're actually sitting next to someone and you can kind of look over at them and say something feels more natural when you're in a, you know, shared space. And so I think that's one of the challenges. It's, it's, it's all kind of wrapped up in the communication, documentation, and coordination aspects. And I think that's one of the challenges that, you know, we're, we're going to be addressing. We, we, we have some plans to address that in the very near future. But that's, I think that's pretty universal to, to every team, to, you know, any, any team that has a delivery manager or not is the, how do you work with other teams and how can you facilitate that? And, you know, turning that kind of back on you, I'm interested to, hear about how you kind of view that challenge and, and, you know, as a delivery manager, how you think that, you know, that can be improved. Yeah, totally. I think you're right. It definitely is a universal challenge. It's probably, I, I would love to hear if anybody knows of any business that's probably not going through the same thing. There are probably many teams within businesses that, that experienced this, this kind of problem. I would, to add to that challenge, the teams, each individual team works very differently. So the way we work managers is very different to perhaps another team within Cloudflare. And that's totally dependent on whether they have delivery management support or not. So, you know, it really depends on the, the way they've got their workflow set up, the way they communicate externally with different teams. So there's a lot of, so where you do have dependencies is all farewell, kind of meeting somebody from another team and saying, Hey, I need this from you. But if that's, if their process or their way of working is different, you've got to break in communication there. And those are some of the, I would say that adds to the existing challenge of kind of, you know, cross collaboration across teams is that if a, if a team doesn't align with the way, you know, that you've kind of like set out in your team or it is not on board and they have a different way of working, then that can already cause like some, some friction in communication. So, which, you know, causes delays and there are various different kinds of project timelines or overlaps that are not clearly communicated or outlined. So that's something I would say, yes, we're definitely going to be actively addressing, but for the time being, you know, a lot of it is just perhaps to some of you is just basic things, but it is a lot of like meetings and having the right conversations and having the right people involved in the right conversations and decision-making. So yeah, definitely. I agree with you, Andrea, that's definitely an existing challenge that we, and to be honest, we'll probably continue to face it because we're huge. Teams are spinning up all the time. Yeah. And I was, I was going to say that the, the facilitated communications and the, you know, as you say, we are kind of expanding our delivery manager program here in London. One of the things that I think we will have to be careful of is that, that communication, particularly between like coordination and that coordination step doesn't become like a back channel. It needs to stay transparent and making sure that the people who are going to be involved in, you know, whatever projects or decision -making and stuff like that are involved. And it's, you know, we just have to be careful. I'm not worried about it necessarily. I don't think that we have a culture of, you know, things happening behind the scenes here at Cloudflare, but I think we need to just keep an eye on it, that it doesn't become, you know, the coordination and the prioritization doesn't happen behind the scenes. Yeah. Awesome. We are, we've got five minutes to go. I don't see any other questions coming through. So we're good. I can continue asking. Yes. Unless you have a question. I have a question. Yes. It's, it's actually about starting in delivery management. So when I moved here from the US, I'd never heard of delivery management. It's not a, it's not a position that any company that I know of in the US has. It's a, I think it might actually be a London specific thing, a British thing. And so my, my question is around kind of getting that started and the, what does, what does that look like, you know, from, from your perspective, like what was your kind of trajectory, but also, you know, if someone wanted to specifically go into this, what kind of, you know, degree would they get? What, what kind of schooling would they get? Where would they start their career to, to end up, you know, as a delivery manager? Great question. I can probably take up the rest of the time for that. Perfect, perfect ending. You know, you're not the first to say that you hadn't heard of delivery management. When I joined two years ago, people weren't familiar with the term at Cloudflare. They hadn't worked with a delivery manager, clearly at Cloudflare because their role didn't exist, but also even outside in the industry, like in their previous employment histories, they hadn't worked with a delivery manager before. So the term was very new to them and the term, and to be honest, it, it, for me, the experience was really, really different. I came from government where the role is established, you know, there's a community of practice for delivery managers. You know, agile is, is a, is a mandate within government at the government digital service. So everything's a mandate in government. Yeah, exactly. And so coming here, it was completely greenfield. I had the opportunity just to make it exactly what was perhaps what the business needed at the time. So working, working with the teams, it was, it's, it's been, it's been, it's been a real joy. And I've had a lot of flexibility to kind of shape it the way that teams need rather than it being a top down thing. It's almost been a bottom up approach, but, you know, leading by example, moving from team to team, solving smaller issues, demonstrating the value and having delivery management, and then kind of, you know, working a way up. And now we're at a point, you know, she has a, so the team's growing, we're hiring more delivery managers, you know, people across the business know more about it. So they, they're asking for help or they're showing curiosity in the sense of what's this? Can I find out more? And that's great that we've been, you know, I've been able to like help set up that space. What was the other part of your question? How do you, how would you recommend if someone would want to get into delivery management? What, what kind of background? So we're, you know, we're hiring for delivery managers now, when, when going through a resume, what kind of backgrounds are you looking for? Yeah, we're probably looking at somebody, you know, who's got a great ideal if you have certification and agile and safe methodologies, but it's not necessarily, I would say, an essential requirement in terms of background is probably more around the experience and the kind of companies that you've worked at probably around, you know, large scale software organizations or application products or services. And also just a very fast paced environment. I think a lot of, a lot of the candidates kind of we've been seeing this company coming from a banking environment, which probably reflects that, yes, you are, you can do the job, but it's probably not in the, in the right environment for cloud based on what we, what we do in our service offering. So, but I would encourage everybody to, to, to apply and it would be great to continue receiving applications and having conversations with more delivery managers, which is what we're doing now. Excellent. Well, I think we are seconds away from ending. So this has been fantastic. Thank you so much for, for having me. Great. All right. Thanks everyone for tuning in. Bye. Bye.