Decentralization is dangerous (if you’re not prepared)

Does that seem reasonable to you? That's bulls**t. It seemed crazy to me. It's interesting how you can have that misconception. I think the KPIs are misaligned. App developers want to ship, they want to build features, they want to see things out there, and security teams want to keep things safe. Hello, my name is Trey Guinn, Field CTO at Cloudflare. I'm here today with Peter from Redwood. We're going to talk about how decentralization can be dangerous or not. What organizations can do when you think about being in larger teams, decentralized teams, decentralized technology stacks? What are the patterns that lead to good outcomes and maybe those that don't lead to such positive outcomes? Thanks so much for joining us. It's a pleasure to be here. I'm happy to talk to you about decentralization. I guess maybe the first thing to think about is sort of in the world of talent and teams. So, you know, we now live in this world of remote work and, you know, I guess in the world of open source, we've had decentralized teams for a really long time. When you look at groups of developers or groups of technologists that are doing things successfully, when you look at these teams that are distributed, what are the things you look at to say this looks like a positive outcome or bad outcome? I don't know, how do you evaluate if a team is successful when it's working in a decentralized way? I think usually the number one thing you're looking for is velocity, the velocity to ship to production. So get some code in front of your users. That's probably the only thing or one of the major metrics that I look for. Are we shipping or are we not? And so if you're looking, yeah, at velocity, I guess time from a line of code being written to it actually going into production or being used, what are some of the patterns that you've seen in distributed teams that either, that lead them to improving that velocity? If I was gonna start my own little startup and had folks in a distributed way, and I said, how do I make them work really effectively together and drive velocity? What are some of the patterns that work for you? So I think the number one thing is context. So if I write some code and I hand it off to someone else so that they review it so we can ship it to production, what do they need to know in order to make the decisions whether that code is usable or not or valuable or not? If I just hand it off to you and I have absolutely no context, you have to go back and build it up. And it might be at a point in time when the stakeholders just aren't online. So you have to provide enough context so the person that you're handing off to can make an effective decision. And one of the things that we do in our startup, Redwood, is we provide videos that provide enough context or we give videos that provide enough context in the pull request. So we almost motivate the piece of code that we're reviewing. We offer insights into how we were thinking, why we did certain things, what we struggled with. And then the reviewer can say, all right, I get it. I have enough information here to make a decision about shipping this or not. Oh, that's super cool. So you actually attach like a, I don't know, a selfie video about like this pull request and what I was trying to accomplish. It's usually a screen recording with your face if you wanna, if you know, it's you're either presentable enough because people in remote teams often don't and where you're just running someone through the things that you're doing and why. I love that, this idea of how to take, to drive context between distributed teams and make them successful working in this distributed way. But if we like change gears a little bit and go from talent and making teams work well together, we also have to think about how to make the like distributed technology combination oftentimes between app developers and security. And when you think about folks that are not always completely in line. Interestingly, we did a survey recently and in the survey, 80% of the people that responded said that they thought that their security and app development teams were like totally in sync. Does that seem reasonable to you? That's, it seemed crazy to me. It's interesting how you can have that misconception, right? And I think the KPIs are misaligned. App developers wanna ship, they wanna build features, they wanna see things out there and security teams wanna keep things safe. And those two teams are just, they're not working together. Their goals are completely different. I mean, at the end of the day, the goal is to provide a good solution to your customers to make sure that their data is safe. So in that way it is, they are working together, but the leading indicators are way different. So that's super interesting to think about how these different goals can be aligned or misaligned. Oftentimes, like you said, everyone's trying to do right by the customer. Security teams wanna keep data safe, application teams wanna build the next capability and the next function. But oftentimes those goals can be misaligned. Have you seen patterns or examples where those teams really, really work better together? Oddly enough, I think the antidote to decentralization is somewhat centralization. And usually people will introduce a team, a platform team, or companies will introduce a platform team that's goal is to improve velocity and to make sure that all the different stakeholders are aligned and they own that. Velocity is the thing that they're trying to improve for the company. And I think that makes it much easier because they have final say and usually security team has final say. So if you've got this middleman, it's helpful. Sort of like Switzerland. Having Switzerland is super helpful. That's actually a really good point, this pattern of having platform teams and how that helps security teams and technology teams kind of get on the same page. Now, one of the big challenges organizations are oftentimes trying to accomplish is around compliance. Now, if you wanna go into a market, if you're in Europe, you need to have GDPR compliance, maybe PCI. I don't know if you've ever seen any sort of horror stories or patterns of what has helped people in their journey to either become compliant or maintain compliance or maybe anti -patterns they should avoid. I think one of the worst patterns that I've seen is people, developers wanna build against production realistic data. Production realistic data helps you understand why your product has certain bugs, helps you see performance issues. And a lot of developers actually will just download the entire production database and run that locally. So the end result is that they have all your customers' data on their own machine. It's one bad actor or one hacker away from being leaked. And obviously we shouldn't be doing that, but we don't have very good tools, or we didn't have very good tools to not have production data on our laptops. I guess that's like one more example of trying to drive alignment in a decentralized group. I'm sure if you ask the security team, say, hey, you gave me a safe way to test against production data, we'd be in a better position. And, you know, there are a lot of examples of like sort of intern developers that connect to the production database. They're trying to fix a bug and they drop tables or delete rows. And they don't even realize that they're actually connected to production. It's just part of the organization is just like, hey, you've got access. And developers need access, but you have to do it in a safe way. It's a really interesting point. You're talking about context, right? That's what we just described before. How do you maintain that context? Whether it's a junior developer or not, which database am I actually talking to, et cetera. Whether we're talking about security teams, application teams, et cetera, we're all oriented around the end result for the end user, right? The actual user experience. I'm curious, like, if you've seen patterns there where decentralization has actually either worked really well, or not on the technology side to drive good outcomes for your end customers, for the really positive end user experiences. Like where do the teams, does that experience need to be really integrated and sort of like brought together versus where can you use disparate systems to drive good outcomes? This all plays out on an organizational level. And it's gotta be about ownership. So if you have a design system, who owns it? And how do you give work to them? Like if you need a special kind of button for something, how do you ask them to do this? And can they do it in a timely fashion so that you can ship your feature to production quickly enough? Actually, a really great answer because you're talking about building design systems in a way that's sort of the context that other people need to have positive and like unified user experiences. So again, it's sort of that pattern of the context. Yeah, and when you have a design system or when you just do whatever you want, not only are you incurring technical debt because there might be several buttons that do the same thing or several inputs that do the same thing, but different code pieces of code. And if you wanna align them all, it's a ton of work, but it's also bad for the user. So the user would expect one text input to behave a certain way and it wouldn't in different parts of the app. And it just breaks down the trust that the user has for this piece of software that you're building or that they're using. I love these. These are really actually useful patterns to help when we're working in decentralized teams, both having a platform team to help get teams aligned, having a design system so that we're driving context, doing sort of screen recordings and videos with your pull requests. These are great. Just like AI, context really matters. In companies, turns out. Turns out context is important. And when we're thinking about the future, we're now five plus years into remote work and everyone's distributed, but we're also now introducing AI. I'm just kind of curious when you look into the future and think about the modern company, where do you think these, where are teams going, these distributed teams? Are we getting more centralized? Are we gonna keep doing remote work? What do you think the future of work looks like? I think teams might get smaller and people will be a little more productive. They'll be able to ship more in less time. I think the availability of context for why you're doing a particular task is gonna be easier to get. You wouldn't have to ask a person, why are all our buttons purple? That would be tribal knowledge that is accessible to you via an LLM or something like that. That's the future that I would like to see, where context is always relevant in the piece of work that you're busy doing. It's really interesting this idea how important context is in decentralized teams, decentralized technology, but that means that someone's gotta be making that decision, whether it's a design system or something else, they've got to decide that this is the way you make a blue button. How do you think about the right structure of identifying who's the owner of setting that context inside of a successful team? I think it comes down to the person who cares the most. In a startup, that's the founder. They care that this button acts this way or doesn't act this way, and they're gonna ask you to fix it if it doesn't. And in a large organization, a larger organization is people who have the expert knowledge to know why something should be in a certain way, and it's because they care. That's their job to do that. Now, there's someone who sort of owns the problem, maybe knows the user or the customer best, and is trying to make sure that the outcome is lined up. Yeah, and we spoke a lot about context here, but it would be really amazing if you could provide that to everyone in the organization when it was timely and it made sense for them to know that stuff. So it's like, all right, buttons should be like this, and here's why you should care. At the end of the day, that's why we're all doing work is because we care about the outcomes. That's a great point and also ties back to, maybe that's one of the things the future of the AI can help is, sharing that context at the right context at the right time across an organization. Thank you again, Peter. It's been a fantastic conversation learning about decentralized teams and technology and how important context is in making these teams successful and driving great outcomes for our customers. We've actually learned these sort of patterns and others through Cloudflare's App Innovation Report, where we surveyed thousands of companies. So look out for that. Also, come back and look at more episodes through the Beyond the App Stack episode here on Cloudflare TV, or listen to the podcast. We look forward to you joining on the next one.