🔒 Security Week Fireside Chat: Ken Breeman & Vijay Chauha

Hello and welcome. My name is Vijay Chauhan. I work in product marketing at Cloudflare and I'm thrilled to be holding these security week fireside chats where we get to talk to our customers about all things cyber security. We're about at the midpoint of security week so still a number of product and partner announcements still to come and today I'm really excited that we have here with us Kenneth Breeman, staff tech lead at HubSpot. Welcome Kenneth. Thank you very much. So to kick us off, Ken, if you could tell us a little bit more about yourself and what you do at HubSpot. Sure. So I'm a tech lead which means I have a little bit of management responsibilities and growing people on my team and I'm a member of the threat hunters team. So we're looking for the the biggest, baddest threats that are facing the company from a security perspective and trying to to get ahead of them, detect small problems before they become big ones, all of that. Awesome, awesome. And I know your background spans many different disciplines, software systems, security engineering, you got your CS degrees from Worcester Polytechnic. Love to learn about your route from there to now getting to HubSpot and especially any nuggets that you have related to your security journey along the way. Sure. So I started out in software engineering and I was really interested in security. I loved finding out how things worked and how to break them. But when I was going to school, they didn't really have cyber security as a major. So I took extra crypto classes and things like that to further that interest. But it wasn't until I really started full-time engineering at various companies that it was apparent just how much security is needed and all the challenges involved there. So I made the jump while I was at BrightCode and then started right on it when I joined HubSpot. Awesome. And I noticed on your profile you had something that said amateur radio operator license. You got to tell us more about that. Yeah. So WPI has an amateur radio club. It was a very active club, very inclusive, made some great friends there. And it was a great opportunity to volunteer. And much like security, you always want to be prepared. Ham radio is one of those things that works even if the cell phones are unavailable or any of that. So I'm a big fan of it. Definitely recommend it. Awesome. Now, switching to your current company, let us know a little bit about HubSpot for folks that maybe have not come across HubSpot before. Yeah. So HubSpot does marketing and sales automation. What that translates to is hosting websites and sending email. But we're trying to do things a little bit differently. We're no one likes spam. No one likes pushy sales or marketing or advertising. And so by following the inbound model, we make sales and advertising, all of that more effective and less annoying. Awesome. And so your customers are presumably span all kinds of industries, all kinds of organizations that are looking to have technology helping them with that inbound. Awesome. Well, what can you share about your software, your technology, your infrastructure stack at HubSpot? Sure. So we're primarily on AWS and we're using Cloudflare for our edge. We've built a tremendous amount of automation for actually running applications. We're running on a mix of Mesos and Kubernetes. There's a lot of fun security challenges when you're running thousands and thousands of microservices on that kind of infrastructure. Very cool. And maybe you can talk to us about some of the kind of challenges that you come across as a software vendor that's delivering these critical go-to-market capabilities to thousands of customers. Yeah. So one of the challenges is we have a freemium model. We give away a lot of our services for free with the intent of if you actually like them and you find value in them, then you'd be willing to upgrade and pay for them. The challenge there is there's plenty of bad actors out there looking to abuse them and try and squeeze malicious value out of any sort of free service. So always staying one step ahead of them and being proactive to find those sorts is an ongoing challenge. And I remember when we were speaking offline, you were talking about making security frictionless. Maybe you can double click a little bit on that. Yeah. So anytime you try and introduce some new layer of security, whether that's longer passwords or extra things you have to click through, you want to avoid that friction. If you put too much friction into the system, you're just going to annoy the people that are using it. If you make the password 64 characters, well, that's just a really long sticky note stuck to the monitor. Making it more usable makes it more effective and more secure. Great. If we can flip to talking a little bit about your journey with Cloudflare. I know you've a customer of Cloudflare for several years. We have a detailed case study that outlines your original use case. But maybe for folks that haven't read that, tell us a little bit about how did you come across Cloudflare? What was that original use case that you were looking to solve with Cloudflare? Sure. So the original use case, we had an automated SSL provisioning system that was struggling. And the most popular browser out there was Chrome. And Chrome was rolling out a change that would start marking any web page that took form data as insecure if it wasn't using SSL. And so we took it upon ourselves, because this was this is sort of the bread and butter of how inbound marketing works. We wanted to make that as secure and seamless as possible for our users. So we made the decision to basically give away SSL for all of our customers for free before Chrome turned on this feature. And well, provisioning 40 plus thousand certs has a tremendous number of challenges, especially on such a short deadline. So thankfully, Cloudflare rolled out that feature and had the APIs available and made it pretty much seamless. We were able to provision SSL for a huge number of customers with zero downtime. Excellent. Glad we were able to support in that. You spoke earlier about when there's a freemium model, malicious people try to exploit that. Maybe if you can share a little bit more about what adversaries, what attackers, what malicious folks are typically trying to do when such a freemium model exists? Sure. And with web pages, folks are trying to host phishing pages and other malicious content with email services. People love to send out spam and phishing and staying one step ahead of that and keeping the Internet a friendlier place is a worthwhile cause. And you were talking earlier as well about trying to find better ways of making sure you don't impact your legitimate users. Any insights on that side of things? Yeah. Anytime you implement any sort of restriction or control or thing that could block service, you really need to be careful with that. Any sort of broad stroke could easily deny service to legitimate customers. So it's a very, very fine line. You have to be very, very precise. One of the things we really like about Cloudflare's APIs and the offerings in terms of page rules and firewall rules, rate limits and workers is it gives us all of that flexibility to be really, really precise. Now for that, the SSL use case that you spoke about earlier with the change that was happening in Chrome, the time-sensitive change, what were other technologies or what were the other options that you were considering for that use case? Yeah. So we have a lot of engineers and we love tackling big problems. So we looked into what would it take if we wanted to become a certificate authority or what if we wanted to build our own provisioning system on top of Let's Encrypt or some other similar model. And going with Cloudflare was just a much easier choice. And if I'm not mistaken, you were saying that this was something that was maybe on roadmap already, but you helped us to push it out the door. Yeah. We were already looking at Cloudflare for all sorts of different features, but this really put the pedal to the metal. We had that deadline. Google actually bumped it up and decided they were going to ship it early. And so we just went for it. Great. Now, security week, I know you've been tracking closely the various announcements. Anything from the week so far that we've announced that's caught your attention? So I believe today you'd mentioned supply chain and client-side security. That's right. Yeah. And what in particular, obviously we announced the paid shield and then a little bit around BGP route leak detection. Any things there that you thought might be interesting to look at? Much like amateur radio, it's all very cooperative. There's not a lot of security built in. If you have a radio, you can pretty much broadcast. BGP is a similar way. It's amazing that the Internet even functions at all with BGP still in place. So it's really nice to see the extra attention there. Great. Great. And stay tuned because we do have other announcements later today, tomorrow, maybe even over the weekend. Outside of security week, what are some other top of mind security challenges, security issues that you're thinking about and anything you can share around your plans to address those? Yeah. So the sort of big event that recently happened with solar winds and supply chain attacks, the dependency confusion attack, all of that have really stirred up the security community in terms of how do we actually build trust into these systems? It's a really difficult problem. It's going to require cooperation from a lot of different groups, but ultimately it's a challenge that I'm excited for. There's a lot of opportunity here. And speaking of trust, for the last probably two or three years now, folks have been, vendors and customers alike have been speaking about Zero Trust. And it feels like we're now into, okay, moving from concept into implementation. Is Zero Trust something that your organization is thinking about? Yes. So Zero Trust is an initiative where we're currently working on. It's an iterative process. It's very difficult to change your security model entirely overnight. So finding effective ways to do it incrementally, roll things out, but still keep older systems working properly and securely is a very interesting challenge. And I remember you were saying that the traditional approach maybe is a little bit more brittle. Maybe you can double click on that and why the Zero Trust approach is needed. Yeah. So with a more traditional network, you have your firewall at the edge. And then once you're inside the network, you're in. Trying to build up trust just because some application is running on a particular IP address or someone happens to be logged into any given network, there's just so much more going on there. With any large company, the risk just keeps going up as the size of the company increases and infrastructure continues to build out. There's always the risk that something's going to go wrong. So the more layers that you can stack on internally, the better, the more you can lower your risk, the more you can keep things safe and secure. And anything you can share on it doing some of these things using, for example, crypto primitives. And I know we obviously have this access offering around remote access for users. What are your thoughts around that? Yeah. So Google introduced that concept of BeyondCorp and it's really, really taken off. Access is a wonderful implementation of that. Being able to sort of shed those traditional network boundaries of either you're inside or you're out and make that a little bit more transparent. Really, really effective. We're looking to do similar across large swaths of our infrastructure in terms of rolling out mutual TLS and lots of other factors like that. Awesome. Any advice that you have for other security leaders, security practitioners, as they are looking to shore up their defenses in 2021 and beyond? Definitely recommend a risk-based approach. No matter where you are in your security journey, the one thing a company that has like perfect security and no security have in common is that they're both probably out of business. No one's getting this perfect. Everyone is somewhere in their journey towards making things better. So by taking a risk-based approach and really looking at what are the most important things and what are the most likely things to go wrong and iterating on that and just continuing to iterate. It's going to be much more effective than designing the perfect system, rolling out the perfect system. By the time you ever actually finish that, you're out of business. So definitely iterating. It's exciting, right? The fact that this is a never-ending battle. Is that something that you take any comfort in? I love that. On the one hand, it's a little bit nerve-wracking. Things are always changing and you always have to be keeping up with the latest events, the latest vulnerabilities, all of that. On the other hand, it's a tremendous learning opportunity. Now for viewers who are maybe earlier in their careers in cybersecurity or maybe they have been experienced in an adjacent technology area, not necessarily cybersecurity, but they're looking at switching over. What advice would you give somebody that's looking at embarking on this kind of a career? Definitely start out with a breath-first kind of search. The security field is really, really broad. The kind of challenges you would run into as a front-end security engineer dealing with cross-site scripting and content security policies and all of those things are wildly different from a low -level back-end engineer dealing with C and C++ memory allocation and buffer management and all of that. So definitely seek out different areas and try and find your niche. Find the area that you thrive in, the one that you really enjoy working on, the one that you find most interesting, and then try to go deep. Read the source. Don't trust that it works. Verify it. Figure out how it works. Take it apart. See if you can break it. There's a chance that you could find a vulnerability. There's open source projects that have been out there 10 plus years and people are just now finding some of those vulnerabilities. It's a group effort. So start breath-first, look at those different areas, and then go deep. And don't be afraid to go deep because you might find something that nobody's noticed, like you said, for years, maybe even for decades. Good advice there. Now, it would be difficult to go through a conversation, and you touched on this earlier, it would be difficult to talk about something related to cybersecurity and not bring up probably the two big news items from the past few months, one of them being SolarWinds, the other one more recently being Microsoft Exchange. Obviously, they both made mainstream news. My parents know about it. As a security professional, what are the key takeaways? What should we learn from this? What should we try and do differently? What's the key takeaways from both of those incidents? So for SolarWinds, you definitely want to avoid any sort of single point of failure. People are always going to make mistakes. Things are always going to change. Employees will always come and go. Making your systems truly resilient and not just relying on any one person catching a bad password or anything like that. Building out automation so that you achieve your security goal, and then you have something that actually monitors that, hey, are we still good? Are we still enforcing a password policy? Are we still not exposing certain services to the Internet? By building that automation, you remove a lot of the sort of human failure element that's inevitable in any security system. How much do you think is, we often talk about people side of things, the technology side of things, the process and systems side of things. What mix of these do we need to think about if we want to learn from SolarWinds and from Microsoft Exchange? So every system starts with trust, and you have to trust in the people you're working with, but also building out that automation so that you can verify it and actually have the ability to quantify pieces of your security goes a really long way. Excellent. I had another question for you around, again, for folks that are in the industry or also folks that maybe are considering coming into cybersecurity. How do you keep up with news and trends related to cybersecurity? SolarWinds and Microsoft Exchange, I think everybody heard about those through mainstream media, but there's often a tier of news below that or several tiers below that. How do you keep up with those things? So one angle is vulnerability disclosure lists. There's a lot of email lists out there that just send out an email or a newsletter every time there's a new set of vulnerabilities. Some of it's really dry reading, and some of it will never apply to the technologies you're working with, but it's a great way to keep an eye on trends and see what sort of vulnerabilities are cropping up. There might be one particular technique that comes out that applies to one software, and suddenly people are exploiting it across wide swaths of software. So that's definitely one angle. Another, social media. Sites like Reddit and Twitter and all of those are constantly getting new content added. People love to talk about the things they're passionate about, and some of them really, really capitalize on the fact that if they think someone else is wrong on the Internet, they will gladly reply and give you hours of their time trying to convince you, and it's a great resource. It's one of the only places for such a unique field where you can find that just deluge of content. Lots of opportunities there. And any, I guess, any other things that you read or listen to? I know conferences these days have all turned virtual. Any other pro tips that you have for gathering news and just keeping up to date with what's going on in the industry? Yeah, I definitely miss the conference track with DEF CON and Black Hat and all of those similar conferences. It's a great way to meet people with similar interests, whether you're an expert or just getting started. I found it to be a very inclusive community. Awesome. Kenneth, with that, I think we can close out and just wanted to say a big thank you to you for joining us during Security Week to share your insights and experiences. And of course, thank you to our viewers as well for staying tuned. And please tune in for the other Cloudflare TV segments that we have and the other announcements that we still have to come during Security Week. So thanks again, Kenneth. Appreciate the time. Thank you as well. Thank you. We're betting on the technology for the future, not the technology for the past. So having a broad network, having global companies now running at full enterprise scale gives us great comfort. It's dead clear that no one is innovating in this space as fast as Cloudflare is. With the help of Cloudflare, we were able to add an extra layer of network security controlled by Allianz, including WAF, DDoS. Cloudflare uses CDN and so allows us to keep costs under control and caching and improve speed. Cloudflare has been an amazing partner in the privacy front. They've been willing to be extremely transparent about the data that they are collecting and why they're using it. And they've also been willing to throw those logs away. I think one of our favorite features of Cloudflare has been the worker technology. Our origins can go down and things will continue to operate perfectly. I think having that kind of a safety net provided by Cloudflare goes a long ways. We were able to leverage Cloudflare to save about $250,000 within about a day. The cost savings across the board is measurable, it's dramatic, and it's something that actually dwarfs the yearly cost of our service with Cloudflare. It's really amazing to partner with a vendor who's not just providing a great enterprise service, but also helping to move forward the security on the Internet. One of the things we didn't expect to happen is that the majority of traffic coming into our infrastructure would get faster response times, which is incredible. Zendesk just got 50% faster for all of these customers around the world because we migrated to Cloudflare. We chose Cloudflare over other existing technology vendors so we could provide a single standard for our global footprint, ensuring world-class capabilities in bot management and web application firewall to protect our large public-facing digital presence. We ended up building our own fleet of HAProxy servers such that we could easily lose one and then it wouldn't have a massive effect. But it was very hard to manage because we kept adding more and more machines as we grew. With Cloudflare we were able to just scrap all of that because Cloudflare now sits in front and does all the work for us. Cloudflare helped us to improve the customer satisfaction. It removed the friction with our customer engagement. It's very low maintenance and very cost effective and very easy to deploy and it improves the customer experiences big time. Cloudflare is amazing. Cloudflare is such a relief. Cloudflare is very easy to use. It's fast. Cloudflare really plays the first level of defense for us. Cloudflare has given us peace of mind. They've got our backs. Cloudflare has been fantastic. I would definitely recommend Cloudflare. Cloudflare is providing an incredible service to the world right now. Cloudflare has helped save lives through Project Fairshot. We will forever be grateful for your participation in getting the vaccine to those who need it most in an elegant, efficient, and ethical manner. Thank you. Thank you. Thank you. Thank you. Thank you. You run a successful business through your e -commerce platform. Sales are in an all -time high, costs are going down, and all your projection charts are moving up and to the right. One morning, you wake up and log into your science analytics platform to check on current sales and see that nothing has sold recently. You type in your URL only to find that it is unable to load. Unfortunately, your popularity may have made you a target of a DDoS or distributed denial of service attack, a malicious attempt to disrupt the normal functioning of your service. There are people out there with extensive computer knowledge whose intentions are to breach or bypass Internet security. They want nothing more than to disrupt the normal transactions of businesses like yours. They do this by infecting computers and other electronic hardware with malicious software or malware. Each infected device is called a bot. Each one of these infected bots works together with other bots in order to create a disruptive network called a botnet. Botnets are created for a lot of different reasons, but they all have the same objective, taking web resources like your website offline in order to deny your customers access. Luckily, with Cloudflare, DDoS attacks can be mitigated and your site can stay online no matter the size, duration, and complexity of the attack. When DDoS attacks are aimed at your Internet property, instead of your server becoming deluged with malicious traffic, Cloudflare stands in between you and any attack traffic like a buffer. Instead of allowing the attack to overwhelm your website, we filter and distribute the attack traffic across our global network of data centers using our Anycast network. No matter the size of the attack, Cloudflare Advanced DDoS Protection can guarantee that you stay up and run smoothly. Want to learn about DDoS attacks in more detail? Explore the Cloudflare Learning Center to learn more.