ℹ️ CIO Week: Fireside Chat with Adam Healy

All right. Hello and welcome to CROSSFIRE TV and also welcome to CIO Week here at Cloudflare.

Several weeks during the year, we take some time and celebrate a different theme and announce new products and new features that focus on a theme.

And this week's focus has been on all of the different problems that not just CEOs have, but really any organizations have keeping themselves safe, keeping their team members productive, and just focusing on the mission behind their organization.

And to that end, I'm really excited to have Adam Healy here on Cloudflare TV today.

Adam, welcome.

Thanks.

Thanks for having me. Great to.

Great to be here. And great to be doing this with you in Cloudflare.

Well.

We appreciate you having you on the air. We're doing this live today.

We originally going to pre-record this. So if we mumble or stumble on ourselves, forgive us.

But just to get started, why don't you tell our audience who you are, what you do, and what brings you to this conversation?

Yeah, absolutely.

So as you mentioned, my name is Adam Healey. I'm the chief security officer at Blockfi.

I've been in this role about 20 months now, and it's been quite a journey.

At Blockfi, we provide both retail space, consumer and institutional financial products across a wide array of different offerings, everything from our visa branded blockfi bitcoin rewards credit card where you can spend dollars and get back Bitcoin rewards instead of things like frequent flier miles or reward points, etc.

And on a number of different account structures yield earning accounts along those lines for retail.

And then we also have different products that we offer our institutional clients, things like different types of complex financing arrangements and loans and things along those lines.

And how long have you been there at Blockfi?

Well, like I said, I've been here about 20 months.

It's been quite a journey.

When I when I arrived, we were managing about $1,000,000,000 in assets on our platform.

And that sounds great until you think about today and 18, 20 months later, we're managing somewhere in the neighborhood of $15 billion in assets on our platform.

So in some measures, we have about 15 across the business in that year and a half period.

And as one can appreciate, and especially you and Cloudflare can appreciate, you don't hyper scale a business that quickly without accumulating some debt and having to make a lot of decisions really quickly.

So the journey has been great.

We really have invested heavily in partners like Cloudflare, but also building out a lot of our core engineering and security functions for me, which maybe is a bit unique for a head of security.

I and partly why I'm here speaking with you today is I actually get to oversee a pretty large swath of our of our engineering organization.

So have all of our traditional kind of security programs.

So red team, blue team, purple team, GRC, things along those lines, but also oversee our machine learning and data science team, oversee our cryptography and crypto engineering team, oversee our corporate I.T team and also oversee our kind of all things cloud engineering.

So DevOps, sorry, things along those lines. Wow, that is a huge portfolio within the security side, I guess.

What had you interested in a career in security did that were you working in security before you joined Blockfi or were you in engineering?

And this has been a new part of the role.

How did you find your way into this field?

So that's a long story.

We don't have a lot of time for that, but the short version of it is, I think instinctively I've always been focused on protecting events.

And I think my journey, my professional journey really started around 911 and joining the Marine Corps shortly after 911.

And that kind of put me on a trajectory to spend a number of years in the government space, both as a marine and then later as a contractor, and then even after that, as a civil servant, as a federal employee.

And during those three very different roles got to do a lot of things, got to work on some very cutting edge technology problems.

I got to work on some very interesting intelligence problems and also got to take all of my experiences and spend about three years deployed overseas in a couple of different countries.

So that time was very formative. And then from there I was able to transition out to the private sector and work at companies like Microsoft and work at companies like Palantir.

And then kind of as crypto is really heating up back in 2016, 2017 was able to pull on a lot of those different skills that I had built over about 15 years and kind of go full time into crypto.

And here I am today.

And those are some incredible organizations.

For the whole career.

What about Blockfi mission when you were looking into the crypto space, intrigued you to come join that organization?

So, so me joining Blockfi was actually a bit of a funny story in there by itself.

It was Blockfi was having some security challenges in the first half of 2020 and Zach Prince, our CEO, messaged me and I had known him previously.

He sent me an email.

Basically it was, Hey, what are you up to? And from there I had a couple of conversations with Zach and Flora and some others on the executive team and decided this was the right place for me.

And I think beyond the personal relationships and working with great people, which is obviously always very important, some of the technical problems that we're that they're trying to solve were very interesting to me.

How do you support a global client base?

I think right now we have somewhere between 1.6 and 1.7 million account profiles on our retail account, profiles on our platform.

How do you support that spread across 50 countries and how do you deliver content securely and how do you make sure that you're providing high assurance financial services to these individuals?

And on the other side, we have a very different archetype of clients.

We have institutions and we have multiple hundreds of institutions.

These are top tier institutions that probably all of the listeners here are very familiar with household brand names is certainly in the financial sector.

And it was a combination of those challenges and understanding the scale story and the hypergrowth story that Blockfi had been on even prior to my arrival, and then where it was going kind of brought all of those, all of those different things that I'm interested in together.

And then when you layer on crypto just in general, it has a lot of very unique security problems, something I always tell security engineers when I'm interviewing them.

And in the last 18 months I've hired about 65 people and a lot of interviewing, a lot of resumes, thousands of residents.

And one thing I always say is we have every problem from a security.

Perspective that a fast growing 1000 person fintech has.

We have PII data and PCI data and credit data and high value APIs with third parties that we're integrated with.

We have all these problems, but then we have $15 billion in Internet money that we also have to protect.

And it creates a very complex risk profile and requires a very sophisticated security program.

And when you're in those interviews and not building out that team, hiring 65 people, what are you what are you looking for?

For a person joining the security organization at Blockfi.

Really good means on Twitter usually.

I was actually speaking at Reinvent, I guess two weeks ago or something, and somebody asked if I was on Twitter.

I was like, Absolutely. I like crypto Twitter.

The means are amazing.

We actually have a team. We have a team internally that we call the meme team.

It's great.

The but no, honestly, I have actually hired two people that have DM'd on Twitter that were like, Hey, I submitted a resume, haven't heard back, here's kind of my profile.

What do you think? And I think what that to me, I mean, they've actually worked out really well and they're our top engineers.

But what, what I primarily look for is aptitude and hustle and not hustle in kind of the overused Silicon Valley use of the word hustle culture.

But really someone and how I would define that is an individual that will own the company's problems as their own, someone who's willing to put their hand in the air, someone who's willing to think in a non linear manner about complex technical and complex organizational problems and help provide solutions and not just surface needs.

And we've we've been really successful with that.

I've got to handpick the security organization really since the first day I got here.

When I arrived, there were about five folks on the security team and now we have a fairly large and sophisticated global team running 24 seven security operations.

So primarily, though aptitude and hustle, smart people that are willing to work hard is probably the two most important things that I look for.

It's a good formula.

And so once, once folks are on the team, what's the structure like?

Do you have people own different verticals, things horizontally? What's the organization look like?

Yeah, so we spent a lot of time working on this and I wish I could say we got it 100% right every time.

But when you're when you're growing an order this fast, we definitely have had to reshuffle some things and reevaluate where we're structuring people.

For me right now, I have a number of different DPS that report directly to me, one covering specifically cybersecurity, one covering specifically some of our cryptography services and crypto engineering services and custodial services that we leverage third parties for.

And a couple of others focused on things like cloud and DevOps.

And also then one that reports to me for things like internal services, GRC, project management.

So it's a fairly normal organizational structure I would say, and it seems to work really well for us.

And it's also something that is really easily explained to regulators, to insurance underwriters, to partners, as as we're looking to do more and more business across the spectrum, it really enables us to tell a pretty good story.

So I think we've kind of taken a fairly traditional security organization structure and then slightly tweaked it under the covers to fit our unique needs.

It makes a lot of sense to me within that security organization, if you are just kind of looking to the rest of the Blockfi team, what's the relationship like between security and product or security and I.T.

at Blockfi?

Well, the good news is I own it, too.

So it's a great it makes it pretty easy.

And I actually think it's a model that works.

And I've seen it work really well at a number of other companies.

So many of the problems that a security team deals with today do originate from external threats that are coming in through email.

We all see the reports from Verizon and Microsoft and everyone else.

Phishing is still a problem and we have a lot of controls around that.

But occasionally things still do get in and we have to respond to them accordingly.

And so I think putting i.t under security actually solves a lot of those problems and it also creates a culture within the security team that you can't just be a department of.

No.

And I think that's a very dangerous narrative for a security team to kind of fall victim to.

You have to be a security organization that enables the business.

You absolutely have to be and you have to be partnering with the business to make thoughtful decisions.

Something I say a lot is you have to be a little comfortable being uncomfortable.

And what I mean by that is risk management is a is a real thing.

And not everything is going to be 100% secure the way that the team would like it built.

So partnering with it is pretty easy for us and it's worked out. Our model of having it actually report into me, actually work really, really well, partnering with engineering and product.

I think every organization has a healthy tension or should have a healthy tension between those that are building product and want to ship it and those that are safeguarding the organization and in bad organizations.

It's a tension and good organizations, healthy tension.

And I think we have a fairly large engineering organization and our head of product, at least on the retail side or GM of product, has recently joined us over from PayPal, where he spent a number of years building product there.

So right now I think we have a very highly cohesive and very high functioning relationship across all those organizations.

But at the end of the day, that's to start at the top, right?

Zach can say and mandate and lead us to a certain direction.

But kind of that next layer down at kind of the head of security, the head of product and head of engineering level, that is where we have to have good relationships and then push down a culture of partnership.

And something I say a lot is we're not a we're not a security company.

We're not a IT company.

We're not in many ways, we're we're not really a tech company, although we could talk about fintech and things like that.

But what we need to be as security practitioners, as engineers, as product people, whatever that may be, as mechanisms to provide better financial services to our clients.

And we're a financial services company and in many ways, and that is the goal of what all of this money that we spend on building technology should be aimed at helping run the business and build the business.

Yes, we are.

I think has a very similar theory about how product and security should work together.

We build security products. It adds kind of an additional layer of, like you say, in a good way, healthy tension.

But frequently we're building products that solve a need that our own security team came to us and said, hey, there's no tool out there that does this, or we're actually doing engineering with them, which is really exciting, I guess kind of to that end, on the Cloudflare side, your organization uses Cloudflare.

How did that start?

Yeah.

So and I would actually just circle back to one point you made is when I say we're not a security.

We're not a security company, I do mean it. But at the same time, a core competency of ours and any company that is in the crypto space must be the safekeeping of client digital assets, client cryptocurrencies.

And we've invested incredible effort, money time into that as an organization, and we take it very seriously.

So when I say we're not a security company, what I really mean is we are that is, we're not selling security products.

Right.

But we are a one of our core competencies as an organization is really that the safekeeping of client digital assets.

So but to your question, how did the relationship with before come to be it was actually in the middle of an attack and it was I often say that every problem Blockfi has ever had is thoroughly documented on Reddit and Twitter because our clients are amazing people and have no worries about calling us out on social media.

So and I like it.

It keeps us accountable.

It keeps things transparent, just myself.

I've even engaged in social platforms directly with folks that have raised concerns, and I think it goes a lot to say, Hey, our executive team is accessible and we're willing to chime in and Zach and others across the C-suite all do the same thing.

So what happened, though, and we had. I had personally had a relationship with Cloudflare, my previous company, and had known many of the folks that actually have helped us get things off the ground here at Blockfi.

And we were having a denial of service attack and we recently published a case study with Cloudflare on this.

I think it's a it's a great case study.

So we were in the middle of both the denial of service attack on a series of our Internet facing infrastructure and also a bad actor was abusing a particular API that we use to facilitate our sign up workforce.

And these things were happening at the same time.

So presumably same threat actor, but threat actor attribution is difficult and and it was causing massive degradation of our platform and was also causing a very public but very public reputational incident because of the impact to our signup infrastructure.

So we had been struggling with another vendor for a while to get things like bot management or other technologies in place and with a phone call.

And I woke up someone on the Cloudflare team fairly senior in your organization pretty early.

I think it was like a6am email San Francisco time and the response was, What can we do to help?

And that's exactly what we're looking for.

And I've been thrilled with the partnership that we've gotten since we've since that relationship has started.

And what can we do to help turn it into.

Great, understood.

Give me 15 minutes and you're going to get a Zoom link.

And we join a Zoom call with some of the best engineers that Cloudflare could rally.

And it was first thing in the morning for them and they cleared their schedule.

And we spent about I want to say it was about 6 hours on the phone, on and off with them.

And between working, getting contracts in place and getting all of that stuff worked through, but then also doing all the technical migration.

And by the end of the day, about 6 hours later, we, which definitely wasn't the end of our day because we were still in the middle, I can imagine.

But by the but by the end of some days, we, we had Cloudflare deployed across our entire environment.

We had done a number of DNS migrations and we were behind an enterprise contract at Cloudflare, allowing us to access all the bells and whistles from bot management to WAAF to CDN and basically the onboarding process.

I don't think I've ever had as fast of an onboarding process with or with any partner that we've worked with.

But 6 hours I think is the record that.

I seem shorter, that we do pride ourselves in making that efficient and easy, especially in the event that every minute in your circumstance, every minute counts.

Right?

You're keeping an eye on that clock. Every minute.

Every minute costs money as well. Right.

And there's there's this every minute is every minute is incrementing our reputational damage and also incrementing our financial impact.

And those two things you can always recover conceptually.

You can always recover from the financial impact.

The reputational impact is very difficult.

Right.

That's a client trust problem. And those things are very, very hard to solve for once, once you've had an issue.

Absolutely.

Like you mentioned earlier about how core security is to everything that you do, the trust that your customers, your clients place in your organization.

That's something that's hard earned and hard capped. Right.

Absolutely.

Absolutely. I think especially in crypto, clients are clients and there's a huge spectrum across who is investing in crypto and we have lots and lots of research and stats on that as we think through marketing.

But the reality is a lot of the demographic that is invested in crypto are generally very tech savvy and very security aware.

You should see some of the messages I get on Twitter around like very, very, very thorough and detailed questions asking about how safe their assets will be.

And that also lends itself to a unique problem when you start to instrument reputational risk and how to prevent it or how to recover from it is these individuals are generally more tech savvy than just the than someone who is outside of crypto.

And it creates a difficult challenge, a difficult needle. And how do you kind of make sure that your organization is keeping that keep keeping your commitment to that core competency of security?

What do you use inside of your house to keep Blockfi safe?

I know Cloudflare is part of it.

What's kind of the tool suite look like?

So one of the.

Processes and programs?

Yeah.

Sure.

So one of the challenges that we had over the last year and a half was we were not only scaling our external infrastructure to support massive client growth, but we were also scaling all of our internal infrastructure and services, meaning we went from somewhere in the neighborhood of 250, 275 employees to about 1000 employees spread across 22 different countries, all working remote throughout the pandemic.

So it was a challenge to scale up that infrastructure.

But luckily, as the pandemic started, we were already pretty well positioned.

We weren't having to really make massive infrastructure changes at that point.

We were take your laptop, go home, and we'll ship you a monitor at the time it was and we'll see you in two weeks.

That, of course, has changed quite a bit and here we all are still working from home.

But well, at least largely, I will be going into the office in a little bit. But the I think the part for us that we really started to want to get serious on as we were using that as an opportunity was, okay, we're going to continue to hire at a pretty rapid pace.

And I think right now we I don't know off the top of my head, I think I have 17 open roles on my team.

So if anyone's listening and was wanting to work on hard data problems, hard cloud engineering problems, hard security problems, hard cryptography problems, feel free to check us out.

But there's probably somewhere in the neighborhood of 50 to 60 roles open across the entire company.

So we continue to hire at a very rapid pace and we still have a remote first work philosophy such that people are still working and probably more than 22.

But I think the last time I counted it was 22 countries and we needed a suite of tools that really allowed us to do that safely.

So as you're aware, we decided after evaluating a number of different products that Cloudflare teams is going to be a product that we were going to go all in on and a combination of teams and how that works with the tunnel infrastructure and how that works with the back end WAAF and CBN and how we're delivering content to our employees.

All of that has worked incredibly well, not only because the technology works well, but because it stepped in very seamlessly to things like Okta, which we have built our entire platform around, and then also Yubikey, which we have used to really lock down all of our internal services.

So to me it was kind of a no brainer.

We were already heavily invested on all of our Internet facing infrastructure.

With Cloudflare, we did a fairly thorough assessment and then really liked where we landed with teams and the fact that it just snapped into Okta and Yubikey work made a lot of sense for us.

So some customers ask me, Hey, from all the customers you talk to, what's the single thing I could do to make my organization more most secure or more secure with just one task?

And every time I tell them in our Zero Trust product, you can add it to the audience.

You can add a rule that requires people to use a hard key as their auth mechanism, not just if they fall back to an SMS code or something like that, they still get blocked.

And every time I tell a customer asking for this recommendation that that's the one rule that will change most, change your security footprint.

Everything else is fantastic layered on. But that, like you mentioned, the yubikey the importance of that is how we run Cloudflare internally.

Yeah.

And I think companies that take security seriously and have security core competencies, whether they're a security product company or they're a company that has a core competency around keeping client assets safe like us, you have to you have to invest in these things.

I do say and it's any organization that may be listening if you can't deploy Yubikey today, understand, that's not something that depending upon the legacy nature of your infrastructure, it's not thoroughly.

Ordered by all the mobile apps that you may use, that's fine.

But I would even like to zoom out a little bit from what you said and say the single biggest value add thing that an organization can deploy is two factor.

And if you can only get to SMS, it's better than nothing.

If you can only get to app based authenticator or totp authenticator apps, that's better than.

better.

And if you can, then if you get to Yubikey, that's even better.

And then if you can get to Yubikey layer on zero trust technologies, many of which we've built around the teams product for us from Cloudflare, if you can if you can do that, then it's even better.

And then you layer in things like device trust and all of these other things that kind of bolt it all together.

But you shouldn't be sacrificing something.

I actually was just talking yesterday with some folks about it.

Don't let perfection be the enemy of done.

So get some stuff done and then roll the version two of your multifactor strategy out whenever it makes sense.

And I'm sure in your role and just as security teams in general, it's never done.

But what's up next? What are you all focusing on as a team here?

Right now, we're kind of approaching the end of the year.

What is the beginning of next year look like?

Well, hopefully the beginning of next year doesn't start with a DDoS attack like it did this year.

But I think I think we're pretty well situated at this point. I think we're I think we're well taken care of.

But for me and I've been really lucky and I've had a lot of support from Jack, our CEO, from our board, from Tony, our CFO, but a lot of support from them.

And for me, I've been able to build out a team of direct reports to me that are really running a lot of the day to day program, and that's been really valuable for me.

So when I think about what am I thinking about next year, I'm kind of I kind of take a slightly different view than somebody who might be running a soc day to day or might be running an IR team day to day.

And from my viewpoint, it's better alignment with other teams.

So better and more alignment with the business better and more alignment with engineering better and more alignment with product.

I think we're great already, but we can always get better.

And I think it's funny, a lot of people that work for me sometimes can get frustrated because I never give them like a ten out of ten performance review.

At best it's a nine, and it's like, Well, why?

We can always get better.

So that's something I've carried with me even from days as a young marine doing fit reps on others.

So the.

So that's one area.

Another area is going to be mature. So we've spent a lot of time over the last year getting infrastructure and tooling and things in place.

I've only been here a year and a half, probably the first 90 days of my time here.

We're trying to just figure out where everything was and what was the scope of the problems that we were dealing with.

And now we're a year and a half in and we've got a really good handle on things.

We have a fairly sophisticated program, and now the focus will be how do we mature that up?

How do we get better at all of the things that we're good at? How do we get better from a capability, from a technical capability perspective?

Probably the third bucket would be probably tied for third, I would say is getting better at, I would say, the soft security things.

So we're already pretty good about with things like GRC and pretty good about how we partner with regulators and auditors.

And we do a lot of work on that front, but we can always we need to get a lot better telling those stories and getting a lot better around maturing our policies and procedures and the paperwork to catch up with the engineer.

And then tied for that, I would say, is professional development.

And I would say we've hired a lot of people and we've made some really good hiring decisions and we're really going to be investing next year in our people training conferences, sans black hat, all of the things that I think are really table stakes for a mature security program.

We're going to be investing a lot more people.

That's fantastic.

Hopefully a year from now we're celebrating another CIO Week and we're talking about how successful those programs were.

Is there any kind of in the couple of minutes we have left, if there's other CEOs or CSOs watching this program or people who want to go into that as part of their career?

Any one bit of advice you'd share with them from your career? Yeah.

So I would say there's probably two answers to that. The first one would be for folks that are maybe more junior in their career and wanting to head that direction, don't lose focus on the technology.

I think everyone can always kind of lose sight and want that next role and want the office, wants the title, wants the money, whatever it might be.

But what I have found is the best technical leaders come from an engineering background and are able to engage at a deep level with the engineers that work for them.

And that is a way that not only builds respect across the team, but also it will help the leader as they progress through their career to understand how to separate signal from noise and being able to know when to engage and when to dive deep on particular problems that they might be hearing about.

And then for folks that are maybe already in these roles and it's hard to be prescriptive, I never try to make any absolute statements.

We're all kind of in this fight together, I think something that I would like to see specifically around the crypto space as we become more and more material, part of the financial services sector at large would be a better focus on things like data sharing and threat intelligence sharing and leveraging, things like FSI, SAK and others that the traditional financial space has kind of done figured out pretty well.

Very cool.

Well, it's been a pleasure chatting with you today. And I know we love working with your team as a customer, so thanks for the trust you all put in us and helping out your organization.

And thank you as well for your time.

Absolutely.

Great to be here and look forward to doing this again sometime.