📊 Managing with Metrics

Hey, so I believe we are live. Well, the first thing we want to do is, you know, we want to thank everybody for taking the time to walk through this information with us, you know. And the second thing we want to do is preface it with the idea that this is not a formal Managing with Metrics Best Practices presentation. It's more of an amalgam of the knowledge my colleague Apoorv and I have put together over the course of our careers when it comes to the subject of Managing with Metrics. And, you know, before I give you guys a framework for how we're going to walk through this information, I'll give my colleague Apoorv an opportunity to introduce himself. Yeah, thanks, Uli. So, hey, everyone. My name is Apoorv. I'm a Data Analyst at Cloudflare in our Business Intelligence team. I've been here with Cloudflare for almost two years now, and it's been a great journey. And like Uli said, today's session is about our perspective on how you should, you know, work with metrics, how you define your success criteria, and if any of this information down the line helps you out, we are more than happy to share and let us know how it goes. Yeah, Uli? Yeah, so like Apoorv said, I'm also a Data Analyst on Cloudflare BI team. I've been with the organization for almost a year. You know, a lot of the work we've done, a lot of the very challenging work we've done has to do with cost, sales ops, finance, et cetera. And my background has to do with working with silos like FP&A, revenue ops, revenue management, sales ops, et cetera. So, a lot of challenging work in this particular subject matter. And, you know, we're really excited about talking about this. Managing with metrics has a nice ring to it, but metrics are often referred to as KPIs as well. And recently, I read them referred to as KSIs, Key Strategic Indicators, which I think more closely aligns with what we're going to talk about. So, basically, the first thing we're going to do is we're going to go into, you know, what are we solving for exactly? Who are we? Who is this information, you know, best prepared to serve? Why it's important? And then we'll go straight into our five principle framework. So, what are we solving for? Yeah. So, what are we solving for? And then, in order to answer this question, I think, ultimately, we need to ask another question is, how do you define your success criteria for your specific use case? What I mean by that is, when you choose any metric, not necessarily you have a definite answer to that outcome. Let's say today your sales projects that revenue could be 10% maybe next quarter, but is that growth sufficient to define it as a success? You may need to even account for internal expenses, operational cost, you know, your competitors' projection and other factors before defining this target. It may happen your operational cost may even be more than 10% of your growth that you were projecting. So, is that a success? Now, let's say if you are a new company, a startup that is initially growing, such kind of growth and maybe, you know, negative profit to say is acceptable, but you really need to have a pacing over the time to predict a time when you might become profitable. So, calculating that timeframe is very, very important in defining a success criteria in this kind of a scenario. So, I think, ultimately, you need to really ask this question in order to define the success criteria and estimating that figure that when you would be successful is very important is what we are trying to say here. Thanks, Apoorv. All right. So, and who are we, you know, who is this, who did we prepare this information to best serve? I think both Apoorv and I both agree that the information we prepared can serve anyone at any level of an organization, but the leaders we're really trying to help out here are those leaders that are trying to, you know, drive quantifiable performance improvement with metrics, whether their organization is data driven from the top down, because it's often much challenging in that type of environment. So, that's who we're trying to target with this information. And why is this important to anybody? Well, Apoorv and I chose three key, you know, key ideas why this managing with metrics is beneficial and why it's important to put into practice. The first one has to do with people. And the key word here is engagement. So, you know, Gallup prepared this research report called Q12 that had all to do with employee engagement. And, you know, the foundation of, in the hierarchy of employee needs at the foundation of it, the most basic and important need is do employees understand what's expected of them? And they found that one out of two employees in those companies are willing to say they understand what's expected of them. But if that ratio can be improved from one out of two to eight out of ten, then turnover rates can be reduced by up to 22%. So, those are some quantifiable benefits and arguably one of the most expensive line items for any organization. The second thing is execution. And what we mean here is that managing with metrics creates a culture of accountability. And when we talk about accountability, we don't mean whether people are doing something wrong or right. What we mean is that with a managing with metrics framework, we create, we solidify the criteria for success, which allows, you know, leaders, direct reports to orient themselves in the direction of, is what I'm doing pushing us in the direction of the outcomes that we're actually pursuing? Because we've quantified those outcomes. The final thing, and it kind of goes without saying, right, is performance. You know, the last two decades have shown a marked difference between organizations that drive decision -making, drive outcomes with metrics, and organizations that don't. You know, I won't mention any names here, but, you know, the companies that you're thinking about are probably the ones that we're thinking about. So, without further ado, we're going to go straight into our five principles framework. The first one is define your objective. From my perspective, the key words here are strategic alignment and precision. Strategic alignment has to do with when you set an objective, are you aligning this with your leadership strategic priorities? Is there an agreement in your organization that the outcomes that you're pursuing is aligned with the strategic priorities of, that your leadership has put into place? A good example of this would be, let's say that your organization's strategy for revenue expansion over the next five years is providing the most robust feature offering in the market, right? And let's say you're on a product engineering team, and this is an easy example, which is why we chose it. So, a very clear objective, quantifiable objective related to strategy might be, we want to increase the feature deliveries year over year by 10%. So, what we're trying to do there is make sure that the objective is very clearly related to the strategies your leadership has put into place. And we want to make sure that that objective is precise and quantifiable. Apoorv? Yeah. Yeah, I agree with you, Oli. The strategy and the position is definitely one of the key factors when you're defining an objective or defining any metrics. I just wanted to add another factor that I would say that there's experience, which is very, really necessary when it comes to defining an objective. And this is, as an example, when I joined Cloudflare, and it was a very early phase. Now, we were literally in the process of learning about our business, and there wasn't just enough data to come up with an objective. I think at that time, our senior leadership, that is the one key factor that helped us defining these objectives to define these success metrics. And ultimately, when I say experience, most of our senior leaderships, they have worked at other companies similar to which is a SaaS line -based business. And I think the experience that they bring it here in defining these metrics is definitely one of the key principles when it comes to take your company going forward. So having that experience will only help in defining that what could be your criteria to define that objective and what metrics you will choose down the line for the next two, three years in order to achieve that success. Thanks, Apoorv. The second principle is define your metric. And for me here, the keywords are tactical alignment and data. So there's a very distinct difference between your objective and your metric. You want both of them to be quantifiable, but your objective is much more closely related to the strategy. How closely is your objective related to the strategy of your organization? But your metric, from my perspective, has to be much more tactically related to the actions that your teams or team members can take on a day-to-day basis. Basically, what we want to do here is we want to make sure that there's a relationship between the metrics we use to measure the performance, the steps that we're taking in the direction of the objective. We want to make sure that those metrics measure the actions that our team and team members can actually perform on a day-to-day basis. The second keyword for me here is data. And Apoorv touched on this a little bit in the previous point. One of the biggest challenges when developing metrics is the deficiency, the availability or lack thereof of data. But the reason I mention this is because one of the mistakes that I've observed over the course of my experience is that teams start with the idea of what data is available and what metrics can we support with that data. And that sounds very intuitive to begin with, but from one of the experiences I've had in the past working on a consulting project, we were trying to create a predictive churn model. One of the ideas that the data science consultants introduced was that we want to start from the ideal state. We want to leverage the experience of our direct reports and our leadership to understand in an ideal state what metrics and what data points most closely relate to the actions our teams can take on a daily basis to reach this objective. And the reason you do that is because you don't want to sell yourself short by basing, by overestimating the deficiency of your data. You want to work backwards from an ideal state, an ideal state that's been based on the experience of the people that are available in the organization and work backwards from there to get as close to it as possible. Yeah, I mean, I totally agree. Like any metric that you chose has to be tactically aligned to the success parameters that you define. And what I'm trying to say here is more of an agile approach that when you define any of your metrics, you need to continuously even learn from it. Today, in our world, the market is constantly changing due to any innovations. And what you define today as a pillar of success may or may not be relevant down the line a year ago or something as you mature as an organization. Of course, there are some metrics that may remain constant, maybe like your overall revenue, your customer base. But as your industry and demands change, you also need to adapt to these changes and shift your success pillars accordingly. So whatever metrics that you have defined today may or may not be relevant a year after or later. One classic example I would like to bring up is of Nokia. We all know they were very successful as a firm, but they never tried to adapt to the changes in the market. Their CEO once said that they never did anything wrong, but in the market, they really fall out. And I think the main reason is that ultimately, the whole industry market and their customers may shift to the Android or maybe the iOS world, but they didn't adapt to that changes that quickly, and it ultimately led to their downfall. So you need to keep on adapting to the changes and keep an eye on it. Willie? Excellent point of proof. Thank you. Our third principle is get buy -in. And arguably, this is a roadblock, especially for those leaders that are in data-driven islands, in organizations that aren't data-driven from the top down. And this buy-in, the key word here is culture. And for me, culture is the biggest challenge when it comes to acquiring buy -in. But when we talk about buy-in, one of the key aspects of buy-in that I've observed that most leaders miss is that they acquire the buy-in from leadership in terms of their metrics, their objective, etc. But they miss a whole half of that equation when it comes to their direct reports. And the reason culture is so important here is because you might not have a data-driven culture from the top, but it's also important to have a data -driven culture from the bottom. And one of the most important ways, valuable ways to succeed at doing this is including your direct reports in the process of getting buy-in and developing these metrics. Especially for, like we mentioned, for those leaders that are in organizations that are not data-driven from the top down. My recommendation to overcome this cultural challenge when it comes to acquiring buy-in is to introduce your framework of managing with metrics as an open-ended question. If we were to manage with metrics, what would be the opinion of the direct reports in relation to the framework, etc., and leadership as well? Yeah, I agree. Everyone in your organizations needs to be aligned to that common goal, which is metrics in our case. As success comes from everyone, there is not just one person who will bring success to your organization. Your organization as a whole in a collaborative manner contributes to that effort. And of course, there will be some hesitations, some setbacks as you push forward and align everyone in your team to work on those metrics and define what could be the success criteria in that case. But in my opinion, the key here is the time and the results. It is always said number and insights plays a key role in gaining anyone's attention. Now, if you are able to prove the value based on your defined metrics, then everyone will definitely listen to you and align according to it eventually. No one will in the world say no to a million-dollar improvement if you try to bring those insights down the line. So it might take a little while to produce those results. But if you have defined your metrics and have defined your objectives clearly that we have discussed in the past manner, and if you are able to produce those results, then eventually everyone will listen to you and everyone will eventually align as well to those metrics. So that's very important. And we need to be patient in that. Awesome. Thanks, Aburviya. I think that point is very key. Oftentimes, especially for the audience we're trying to speak to here, you're working in a vacuum with your managing with metrics framework. And sometimes when you're working in a vacuum, acquiring those insights, acquiring those quantifiable results before introducing the subject to the organization at large goes a long way in improving trust and the influence capacity to instill that and inculcate that data-driven, metric-driven perspective into the culture of your organization. Our fourth principle is experiment. And this is the one, if I had to choose any of these principles, I'd find it very difficult to say which one is more important. I think it's necessary to implement them all in a very effective way in order to succeed. But experiment, I believe, is the one where most people fall short because it's not quite as clear. And what I mean here is when it comes to analytics, when it comes to dashboarding, when it comes to using metrics, one of the biggest challenges, not just for a leader trying to manage metrics, but including from a BI perspective, is how do we tie action to metric? And that's a struggle. From a BI perspective, that's a struggle that we experience on a day-to-day basis. How do we get leaders or as leaders, how do we understand what kind of recourses we have and how those relate to metric? And to overcome this, it's important to think of this principle, experiment. The idea here is if you've done your homework to bring home the three previous objectives, right? If you've defined your objective, it's quantifiable. If you have tactical metrics, if you have buy-in from your direct reports and your leadership, then the next step is in whatever time cadence that you have, overcome the cultural challenge of being willing to implement no more than three recommended tasks or projects for whatever time cadence you're working on. It could be a quarter, it could be a year, it could be a month. And when you tie, when you implement those projects, tie those back to what was the quantifiable effect on the metrics that you've elected. And I put culture here as a keyword as well, because I believe the most difficult part of this process is an organization's ability to absorb the idea that that's how leadership decisions will be related to your metrics. It'll be based on, you'll have a time cadence, you'll put into practice a certain number of projects, and then you'll reference back to that metric to understand if you had a quantifiable performance improvement upon it. And the biggest challenge to this is culture, right? Accepting that this is the framework that's going to be used, but this is arguably where the most value of managing these metrics comes from. Like we mentioned earlier, management metrics allows us to orient ourselves in terms of what projects work and what projects don't. More often than not, we'll find out what doesn't work, but finding out what doesn't work, it goes a long way into clearing up what kind of recourses we have to positively improve team performance. Yeah. I mean, culture bias and anything that will definitely help along the way in the experimentation. At the same time, I guess training is one thing that is very important in making sure your company and everyone in it understand those value of metrics. And this is, again, like coming from my experience, like in our organization, we define dollar net retention as one of our success criteria or metric. And it was hard initially for anyone and everyone to understand, including me, that how to get value out of it and how can we use it in order to improve on our business? So yes, there will be some learning curve. There will be some hesitation, but it is very, very important to coach, to train your teams in your organization as per the way they understand. Now, the sales or the finance or anyone else, maybe the infrastructure team or anything, they would look at this metric differently. So you need to really understand what is important to them and define the metric or the specific criteria or iteration of it in a way that they can understand it. So if it is a sales, you may need to create a different version of your metric in a way that they understand, then finance. And it's a, again, continuous process, more of an agile approach where you keep on learning to waiting. So today you presented them a version of it. You get their feedback. You try to understand their mindset and understand what they really need and what is important to them. And you iterate over it. Then you create another version and they might become more comfortable. And eventually you both will come to a consensus. And that the point where, you know, you both have aligned and you have understood like what is important and what is really matter to them. So it is a continuous learning process where you have to train and understand and keep on iterating over it. So that goes a long way and eventually it will work. Thanks, Apoorva. I really love this point that Apoorva makes here when it comes to experiment principle. You know, one of the challenges that, you know, I believe when we were working on BNR within our organization was that one, like a metric may not necessarily mean the same thing to one team that may mean to another. And, you know, managing the metrics framework, it's very important to make sure that all teams or anyone that's involved with the process is on the same page about what those metrics mean, how they're calculated, and how they're going to be used. So I really, really love this point that Apoorva speaks about. Our final principle is coach direct reports. And, you know, this principle is often more closely related to performance-based metrics, although it doesn't necessarily have to be. You know, my keywords for this principle are purposeful analysis and incentives. So what I mean by purposeful analysis is oftentimes when we create metrics to measure the performance of individual contributors, you know, it's very important that part of the process of doing that is that leaders take the time to observe and analyze purposely the differences between performers who succeed, individual contributors that succeed, and individual contributors that don't, and then use that analysis and observation to purposely make a curriculum that they can equip their middle or low performers with in order to succeed. And this is a really important point because, you know, in this principle, failing to succeed at this principle for the management metrics framework can really, you know, sour any of the good work that was done on the other principles. You know, if individual contributors don't feel like they're equipped with the information they need to succeed, a lot of the work that you've done on the other principles can be brought low. And so that purposeful analysis of what it looks like, what kind of behaviors drive successful metric performance and which kind don't, well-documented and then provided to the rest of the team as a resource is incredibly important to get this, you know, get the managing metrics framework across the finish line. The other keyword that I introduced here was incentives. And the reason I mentioned this is, you know, some of, you know, managing metrics can have some effects that some people have used to criticize the idea of managing metrics. And, you know, while typically the metrics are what's attacked, it's usually the incentives that are what need to be analyzed that lead to the consequences. A really good example of this was a recent, not too recent, but a pretty famous bank set some pretty high sales performance standards. And without any measure for quality that led to some, you know, the incentives were misaligned there with direct reports found any and all ways to reach their metric, right? So the idea here is once you create a metric driven framework, once you implement it, once direct reports, et cetera, have a clear understanding of what metrics they need to drive performance. There needs to be an analysis of how are these employees or teams being incentivized to reach these metrics. You know, my recommendation there would be, you know, with any metric driven framework that involves the performance of individual employees, it's important to add metrics that also account for quality. And that quality, those quality driven metrics should cover any potential misdirection with incentives, but it's still very important to pay attention to this potential pitfall as it can lead to some serious problems and completely undermine the effectiveness of a managing metrics framework. Yeah, it is. So this principle is basically the outcome of what we have defined in the principles so far, you have defined your metrics and you have mainly experimented with how it will go. And this principle is basically trying to analyze the outcome that you have produced so far. And basically what I'm trying to focus here is like, now, you know, you have your results and ultimately you need to focus on the entities that are underperforming based on those metric results. Now, it doesn't mean when I say underperforming that you need to really enforce some sort of controls over those entities. And ultimately you need to analyze what is going wrong. And sometimes things could be just okay. What I'm trying to say here is, let me take an example. Like, let's say if you are part of maybe a fashion industry and your target is to basically make expensive clothes. Now, you might be performing well in regions, maybe LA, Las Vegas, because there are more densely populated and there are more rich people in those areas who are willing to buy those expensive clothes, but it may not work in other regions. So that is the direct outcome that you need to understand that if some region is underperforming, do you need to strategically change your objective or your success criteria? You may need to involve marketing here in order to invest more in that area, or you may need to shift from your current criteria to maybe something that people understand in that particular area. So it is basically analyzing those results in a much deeper, granular way and all coming up with strategies that can improve upon over time. And the success won't be visible initially. It will take a time. Whatever you do today may give us outcome in next year or so. So again, here you need to be patient and you need to keep on iterating over again and again and see, are you able to reach close to your success criteria? And that's the key here. Awesome. Thanks, Apoorv. Yeah. So that's the five principle framework. Before we conclude, I definitely want to add one note when it comes to define your objective and strategic alignment. And the note I want to add is this framework is easier implemented for some departments than other. For example, administrative departments, like a billing department, they might have a more difficult time understanding how their work directly aligns to maybe the five strategies an organization is using at any given moment in order to succeed. But efficient and effective administration is a strategy that all organizations, whether they speak about it or they don't, they need to carry out. And so for general administrative tasks such as billing, which are incredibly important, that is a strategy, efficiency, quality. What's the age of our accounts? How efficiently are we processing billing? You know, what proportion of customers are being billed, their full amount, etc. So for administrative tasks, etc., your objectives can be strategically aligned if they prioritize the efficient performance of those tasks, etc. Awesome. Well, you know, I hope some of this information is useful to the people listening. You know, again, what we're really solving for is how do we know we are succeeding? That's what managing with metrics is all about. It's solidifying your criteria for success in order to be able to orient yourself in terms of the decisions and projects that are being implemented, right? Are they having a positive effect on the metrics that you, your leadership, your direct reports have agreed are most closely related to achieving the objective? So thanks, everybody. And if you have any questions, you can reach out to me or Apurv on LinkedIn, etc., or in person if we ever get the chance to meet. But thanks again, Apurv. Yeah, I totally agree. And this is just our perspective on how you measure success with your metrics based on our experience. So if you like something, we are more honored. And if there is anything that you would like to know further, please feel free to reach us too. So thank you so much, guys. Thank you for listening to us. Thank you. Transcribed by https://otter.ai