Cloudflare TV

Hacker Time

Presented by Evan Johnson
Originally aired on 

Join Evan Johnson as he speaks with security professionals about recent security news!

Original Airdate: November 20, 2020


Transcript (Beta)

Good morning and welcome to Hacker Time. I'll be your host Evan Johnson and Hacker Time is the number one security show anywhere.

I used to say just Cloudflare TV but definitely the number one security show anywhere in the world.

And I hope everybody is ready for, everybody located in the is ready for Thanksgiving next week.

And it's a weird Thanksgiving for sure with so many people having to make hard decisions about traveling and not traveling.

But I hope everybody's staying safe out there.

And for now, we're going to forget all of that and talk about computer security for about 23, 24 minutes.

And then we're going to play a blitz chess game, kind of becoming a staple of the show, playing a little chess on either or Lee chess.

And today, we're going to play a blitz, a rated blitz game or two.

So it should be a lot of fun. And today, talking about security, I'm walking through an old talk that I gave about two years ago, I believe, at AppSec Cali in Santa Monica.

And it was a great, great conference. It was one of my favorite conferences I've ever been to.

There's a lot of really cool people in the industry there.

It was on the beach. I mean, you can't beat a security conference on the beach.

Because if you needed a break, you could just go for a walk and sit on the beach and relax.

And then just the whole the whole vibe down there with the whole beach theme was just great.

And so I gave this talk there. And I thought it was really well received.

And I wanted to walk you through it today, because I still think all of it holds true.

All of it rings true still. And yeah, so I guess, I start the talk with kind of like, every talk starts like this.

Why should you listen to the speaker?

And I really said that I'm a security engineer first.

And most of my experience has been as a security engineer, working alongside software engineers building products at either security companies or, or I've just been a software engineer at at a security product company.

So I worked at LastPass as a software engineer, that was a really small team.

Besides the, the CEO and CTO, there's, I think, like, maybe six or seven other engineers.

So really, really small team.

And I was just working on the product there, learned, learned so much. I was a early, I was the first security hire at Segment, which recently got acquired by Twilio.

And they're, they're a neat company. And I was, I was also one of the early security employees here at Cloudflare.

And so I'm, I've mostly been around really small companies trying to do security.

And I think that makes really, really, there's really interesting challenges there.

Because the goal of a startup is to move fast, it's to build a product, it's to launch product quickly, and survive as a company.

And security isn't always everybody's top priority, but it sure is would be something that killed the company if there was an issue.

So it's both really important and not important at the same time.

And so there's, there's a lot of challenges that come with that.

And then I have a bunch of just, I guess the whole point of the talk was to tell you what works.

So because there's so many people who will be in this position at some point.

And also, I think that the things that make you successful as a security engineer at a, at a smaller company, are the same things that make your security team successful at a bigger company.

So the, the root and the base doesn't change. And, and so I have some funny slides here.

I have this one, I found this on

And I googled like, what does, what's the role of a CISO?

What does a security team do? And I really like this, because it's off the party, governance, leadership, management, it's very, it's almost like a, it's only missing the word synergy somewhere on here.

And then this is also another crazy slide I found around, around what the role of a CISO is.

And there's just so much, it's, it's so, so complicated. But to be successful, it's really about, it's really about making it simple.

And so the thing that I said, one of my central theses around this is to, is a lot of businesses are SaaS businesses now.

And that's mostly where, where my experience is. So your mileage may vary if you're outside of the software as a service world, B2B SaaS.

But they're, the biggest things to look out for is one, your production environment, two, your corporate environment.

I'll say four first, which is, isn't, is unexpected.

How do you handle things that come up that you didn't expect?

How do you be malleable and be able to handle situations that you either aren't prepared for as a business, or, or you, how do you handle situations you weren't prepared for, or need a lot more resources to handle the right way?

And then business operations is kind of a, is kind of, it's kind of a weird, weird one.

I don't remember what I was really getting at with that one.

But production, corporate, and then handling unexpected. I wonder why I have business operations.

I really can't remember. It was two years ago I gave this talk and I guess people change over time because I have no idea what I was talking about.

And so this is what security at a SaaS business is all about, securing production because you have hundreds of machines, potentially thousands of machines at Cloudflare.

We have machines running in all sorts of environments all over the world.

And, and that's different from other security startups where they might be just all in AWS and all in GCP.

So you really have to understand what your production environment looks like.

Corporate, I think most SaaS businesses look pretty similar on the corporate side.

Like just taking your Silicon Valley SaaS business, a lot of MacBooks, a lot of G Suite, a lot of Google Docs and Salesforce and SaaS tools.

And a lot of it looks very similar from one company to the next. And then incidents, almost all, I mean, no two incidents are exactly the same, but the whole handling of incidents, the same thing rings true for all of them.

So if you have like, no, obviously every company's production security incidents will be different, but you sure do see a lot of data breaches and you still sure do see a lot of the same type of compromises in the production world.

SSRF, if you're using AWS or something like that, there's all these similar playbooks that attackers are running on the production side.

And then the corporate side, it's pretty similar as well.

So a lot of phishing emails, lots of endpoint attacks to try to compromise endpoints.

So a lot of ransomware, a lot of phishing emails that say that they're the CEO of the company and to wire Amazon gift cards somewhere, you never know.

And then the unexpected is, I think, very dependent on the company you work at and what your business is doing.

It really depends on what your infrastructure looks like and just everything about the company.

So at Cloudflare, we have machines in all these different countries.

We have points of presence all around the world.

And that comes with our own unique set of challenges that other companies don't have.

In the AWS world, if you're all in on AWS, governance of different buckets and governance of identity and access management across everything is your biggest threat.

And your unexpected thing might be having a bucket policy set wrong where it's public and you don't have great logs about if it was accessed or anything.

So really, your job will be boiled down into these three things.

And then I have this funny slide, startups are about buzzwords.

And I just took buzzwords. But really, I thought I was really funny for the slide.

But looking back, wasn't a very good joke. Now I know why nobody laughed. When I showed the slide up, everybody was just confused.

So security at startups. To be successful, I think that these three things are super important at any company.

It doesn't matter if it's a startup.

It doesn't matter about if it's a fortune five.

The first thing that matters is relationships. When you are trying to get buy-in on why security is important, when you're trying to get buy-in and share what you want to do with others and make it happen, relationships are key.

Security culture is really key because you want the right mindset, not just in your security team, but outside the company.

One big thing that I'm a firm believer in in security culture is that everybody is an owner of security at the company.

So it doesn't matter if you work in accounting.

It doesn't matter if you're the CISO. Both have a very important role when it comes to the security of the company.

It's the CISO's full-time job, but every person in accounting, every person at every part of the company has access to sensitive things or things that they have to be careful with.

So drilling in that mindset is really important. And then last is compromise and continuous improvement.

You're never done with security, and that's really important to internalize.

Everything can always be more secure. Your business's challenge has changed.

The way the direction your business is going is constantly changing.

And so you've either completely stagnated and you're done with security and you're all just sitting around having a great old time or constant improvement needs to be happening.

And it's more like a limit function than a binary zero one.

You're constantly getting closer to this end state, which is more and more secure and robust where it's not like binary zero one.

You're done. It's almost never like that.

And it shouldn't be. If you're ever binary zero one when it comes to your security, saying that we feel good where we're at completely right now, that just means that your company might have completely stagnated.

And so I presented this kind of fictitious situation in my talk where I described your first day at a startup where there's like this dog running around and this amazing imported Italian brick wall that you see behind all the engineers.

So it's an old wooden building, but for some reason you go inside and you've got this fantastic exposed brick wall.

And this is like kind of all these people now. Now nobody's in offices now because of coronavirus.

And who knows if this will ever be what a startup office looks like.

But in 2008, it sure did. In 2018, it sure did.

And all of these people sitting hunched over their MacBooks typing away. Lots of natural light.

This is circa 2018, what every startup office looked like. And the CEO, I was actually, when I was describing this company, I was kind of talking about a friend of mine's company.

And that company ended up being very successful.

And so it's kind of funny now to think back about the situation.

But the whole crux of this fictitious situation is the co-founder of the company, it's your first day as a security engineer, and the co-founder of the company walks up to you and she says, what is it that you're going to do?

What is it that you do here? And you kind of have this moment where you're like, oh shoot, where do I get started?

And so this is the whole point of the talk.

It's like, where do you get started? You just landed on the moon a little bit, especially if you're coming from a more corporate environment to something like this, and how do you get started?

So one of the first things that I think that these three things are really important to internalize, where first, if you got hired because you're this to be the security expert, you should be the security expert.

And that doesn't mean that you have all the answers all the time, but it does mean that you're the person who wants to find the right answer.

Because like I've said a couple times, every company is different, every situation is different, and the details matter.

And having the right answer, you're never going to have the right answer immediately all the time, and it's important to find it.

And an expert can do that, but that doesn't mean you have to have every answer right offhand.

And you can also start being the first person at a startup.

You can kind of start working on whatever you want. You have the ability to prioritize whatever the biggest risk is.

And so because of that, you need a lot of internal guidance, where you kind of have your priorities set in your mind, even if it's not written down, even if you don't have much budget or much team supporting you.

And so I've talked a little bit about this, what informs your priority.

And every company will have a different set of priorities. If you're B2B versus B2C, company size, if you're really big and you're starting late, like you might want to prioritize hiring.

But if you're a seed round company where security is just really important for whatever reason, maybe hiring isn't so important.

Maybe there's one specific problem that's nagging your company.

And then customer base, are they really security conscious or not?

That also kind of means what the product's doing.

Engineering velocity, are you shipping every day or are you shipping quarterly?

More like shrink wrap software or are you shipping to production daily?

So there's all of these things to consider. But then I kind of had this...

I made this big grid for this talk about what you should probably be focusing on.

And it doesn't matter what your company is.

I think most of this stuff will be pretty true if you're in the B2B SaaS world, where this will probably be the first things that all SaaS companies will probably focus on.

And so I broke it into four sections, compliance, corporate security, detection response, incident response, and security engineering.

And starting with security engineering, because as somebody who is a software engineer, this one's near and dear to my heart.

So first, SDLC and security design reviews, starting to build at least, even if not everything is going through a software development life cycle, you want to start building this notion that people should be thinking before they act and measure twice, cut once with the systems that you build.

You should be understanding the tech stack by getting your hands on the keyboard.

That's one area that too many security teams overlook.

That in order... I'm a believer that in order to truly understand a system, you need to be contributing to it, and you need to be at least a little bit contributing to it, or also to truly empathize with the engineering teams you're supporting or any engineers you're working with.

You need to be doing the engineering and seeing what it's like. This is a big one, how you manage secrets, API keys, customer secrets, especially early stage.

This is a very specific problem, but especially in the world where all of these companies are using lots of SAS tools for their corporate environment, and especially when AWS and GCP are used by almost every company to some degree, it's really important to get a handle on those secrets early, because you never know when one that's buried in Git history somewhere is going to get exposed when a project gets open sourced, and then you're going to have a bad day.

So really important.

And then bug bounty. I say hold off if you can. I'm a big fan of bug bounties, and they're really important to build a relationship with researchers, but it's also a big time suck.

You have to dip your way into a pool.

If the water's cold and you slowly slide into the water, that's more how you want to be with a bug bounty than just diving in head first, because there's so much work to be done when it comes to bug bounty that it just never ends.

Detection response.

You want a basic plan. So even if it's a checklist of like, okay, we're going to get our PR team involved.

We're going to notify the CEO. We're going to form a team that gets in a conference room, just like any engineering set or incident, and and share it and make sure that people see it and know that when something hits the fan, that's what's going to get followed, that that checklist is what you're going to follow.

And then what are your top level signals for your org? So what are some important signals that indicate whether or not you've been compromised?

And starting from three, like are people logging into your identity provider from a country that you don't expect?

Like that's an easy one to look for.

Really important to pick a few and get started looking for things because if you don't, it can just, it can just drag on where it's better to start somewhere.

Otherwise, you're not going to be, you're going to sit in and wait for a long time.

Establish a communication channel with the rest of your company.

At Cloudflare, we did this really, really well early on where we have this alias cert SIR, probably shouldn't say it on Cloudflare TV, but we do have an alias where people email it.

And if you are, and it's just a way to say that something went wrong, and maybe you got a phishing email, maybe, maybe you noticed some, a bug in some software that you wanted to fix, but it it's just an easy way to, to start conversations about things where employees, if they see something, can say something.

Compliance is an important one that I know probably the least about on this list.

And we have a great compliance team at Cloudflare.

I did a Cloudflare TV episode with Rebecca Rogers from our compliance team, our security validations manager.

And it's really important, but one of the things that is really important for compliance is driving sales.

And that's one of the first reasons why it starts coming up in the company, in any company.

And I'm, I talked to some friends of mine who are starting small startups regularly, and I tell them you can get really far by having some, some documentation about your security practices, some knowledge bases for, to be able to fill out questionnaires that you receive from other teams, from other companies, security teams quickly.

And, and you can get really far without having to go through a really onerous compliance process.

You eventually probably will have to, but, but you can hold off for a while.

It shouldn't, it probably shouldn't be your first focus as a team.

And then understanding existing commitments.

Sometimes contracts get signed with promises and you want to understand those.

And then GDPR and current laws. This is a constantly changing regulatory landscape where this is really important.

And then last, corporate security, certainly not least, very important.

Identity and access management.

You need an identity provider. You need to figure out your strategy for identity and access management.

It is so critical because, especially in the cloud, everything is a hybrid where you, I am to your SaaS products in your corporate world, and I am to everything else, your production environment, super important.

Endpoint, you need to be able to manage your endpoints and make sure that you have full disk encryption on every device and the basics like that.

You, you've got to start early on this because the longer you wait, the, and the more the headcount of the company grows, the harder the problem gets to solve after that.

Onboarding and offboarding, really important to streamline and figure out a process for managing how people get their access to things at the company and also lose access to things when they leave.

And just critical first thing to do because there's nothing worse than realizing that you have a bunch of people who still have access to something after they've long gone.

And then last is workplace security. Very important.

People should feel, people should feel like they have a safe place to go to work.

And I mean, maybe that isn't such a priority with, with everybody, most people working from home, but it is still very important.

People should feel safe if they're going into an office.

People should feel, people should feel like, like they are, I think it's also a really visible way for, on the, to get people to buy into security.

One, one controversial one that you hear about is badges and if they help or hurt or what, but it is a visible thing that people see and, and know, and they know that this is for security, even if it, even if the benefits are arguable.

But it's, it's a good conversation to start having.

Do people feel safe at their workplace?

And that was my big checklist. And so I have some personal stories, but if I had to wrap it up, because I would like to do, I guess I can't play a blitz game.

I keep promising this and it doesn't happen. I'll do a puzzle though. But if I had to wrap it up, the things that I think are really important is ownership as a security team coming in.

Don't be a clipboard holding team where you're kind of off to the side and trying to direct other teams.

You want to get in the mix and, and, and take ownership and say, this is really important.

I'm happy to do this piece of it.

If, if you can help me do the, and do this piece of it. And so you, you don't want to just be telling people that there are problems.

You want to be, be prescribing some solutions and asking, does that work?

And when people say yes, then make it happen.

So ownership is a big one. It's thinking, doing things that aren't too much of a drain on your time, like bug bounty, like doing too much.

It's about continuous improvement, not about making everything perfect all the time and, and getting people to buy in.

So the establishing communication channels with the rest of your company, workplace security, making sure people feel safe and, and is there anything else here?

Identity and access management. These are all things where people kind of see that security is happening and that it's important.

So that was my talk. I hope you enjoyed it. I had a blast at this conference.

Like I said, it was great conference. And so I have about a minute and I think we can do a puzzle for the end of the show.

And as always, feel free to get in contact with me.

I'm EJCX underscore on Twitter, Cloudflare TV. There's a method to write in and talk to and send an email.

And I'm always happy to respond.

And this is going to be a tough puzzle that we have one minute to solve. So maybe this one isn't going to be tough.

Is this hard? So we have the white pieces are moving and we've got the strong attack on the H file here.

So the queen can, can't check here though, because this square is covered by the rook, can't check, check here though, because of this.

So what do you do? I think the thing to note here that's very nice is that if you play this move with the queen, the only move it would be mate, but the only move would be this Bishop can take.

But after Bishop takes on H8, the rook coming in is protected by this Bishop.

And with the King here, the King has nowhere to go because it'll be stuck.

So the correct move is going to be Queen H8 here.

And we deliver a checkmate. So that was the puzzle.

Thanks for joining the show and I'll see you in two weeks.

Thumbnail image for video "Hacker Time"

Hacker Time
Join Evan Johnson each week for the latest security news, programming tutorials, and more!
Watch more episodes