🎂 John Scott Railton & Joe Sullivan Fireside Chat

All right, we are on air, live on Cloudflare TV. Hi, everyone. My name is Joe Sullivan.

I'm the Chief Security Officer here at Cloudflare. I have a guest here with me today.

You want to introduce yourself to the audience? Hi, folks. My name is John Scott Railton.

I'm a Senior Researcher at the University of Toronto Citizen Lab.

Thank you for joining me today. As we've discussed, it's Cloudflare's birthday week.

It's a week where we like to, as a company, try and give back a little bit and also talk to other people and share experiences with people who are, I think, having a positive impact on the Internet and technology in general.

I'm really excited to have this conversation with you. We've gotten to know each other a little bit over the years through our work and I think, for me, as someone who works in the world of security and technology and Internet security in particular, one of the best things about working in this space is the people you get to meet because we all have a shared mission, right?

We all want the Internet to be better and safer.

The people we get to meet and work with are the people who are dedicated to doing that.

I put you on the short list of people that I'm most proud to have gotten to know a little bit and excited to share the work you're doing.

You currently are employed at Citizen Lab.

Will you tell our audience a little bit about Citizen Lab?

Citizen Lab kind of addresses this interesting problem space, which is sophisticated nation-state attackers are targeting governments, they're targeting industry, and they're targeting civil society.

Civil society is the odd group out because, although they have the same really sophisticated threat actors and pressure, they can't pay for security.

Often, their level of security is really low.

The logical result of this is an epidemic of breaches and harm done to journalists and reporters, truth -tellers and democratic processes around the world.

It's part of the resurgence of authoritarianism. What we do at the Citizen Lab and the work that I help direct is we try to track and understand what those threats look like and then use whatever tools are available to us, like publications, collaboration with big platforms to get patches out.

Do we have naming and shaming to try to write the balance a little bit between these very deserving but very vulnerable groups and everybody else?

Now, Citizen Lab, is it a non-profit?

Is it associated with a university? How does that work? Citizen Lab is a research laboratory at a university, which means that we sit under the university for all things ethics approval.

We have offices in the university.

You're actually looking at it in my Zoom background here. At the same time, we have a degree of autonomy.

A lot of the work that we do is funded by major philanthropy, not through the university.

That's actually pretty common for university research groups.

One of the things that I think I'm most proud of about the lab is that we have had a policy for a long time, and this is something that comes from our director, Ron Deibert, of not accepting direct operational funding from governments or corporations.

This means that we can honestly say that we have a degree of independence, especially when it comes to working with victims of hacking and other kinds of digital surveillance and harassment, who may be justifiably paranoid and suspicious about everybody's motives.

Nice. How did you come to find your place at Citizen Lab?

I think one of the things that I've discovered is that cybersecurity as a discipline is still, in some ways, young enough that everyone, for the most part, has their own winding, interesting story.

It's a fascinating thing often to ask people what their journey was.

Mine is that I was doing a PhD in something unrelated.

I was tracking the impact of rapid changes in climate on political violence in West Africa.

To do this, I was flying kites with robotic stabilized cameras to map flooding and to see how people in villages in West Africa modified their environments to respond to the flooding.

What I was trying to do was understand how it could be that people working together alone to respond to a climate disaster were, in some cases, making the situation worse.

Then the Arab Spring happened. I had lived in Egypt for a while and knew people who were in Cairo.

When the Egyptian government shut down the Internet, I wondered whether there was something I could do and ultimately built a collaborative project getting information out of Egypt during the Internet shutdown in the middle of the Arab Spring.

We're talking about 2011. The project worked to get information out.

We then franchised the project and did it again in Libya.

It was during that work that I began to see something that was really making me uncomfortable and really caught my attention, which is part of the project was using satellite phones and other sideways techniques to get around nation-scale Internet blackouts.

I began to notice something, which is it seemed like some of the people who were agreeing to talk to me and get information out, often anonymously, were telling me that they were having weird things happening on their computers.

It started dawning on me that maybe the Libyan government was hacking these opposition groups.

That led to a journey that involved me ultimately finding Citizen Lab as researchers who helped me understand what I was looking at.

I just found this problem set so interesting of people holding dictators accountable, using technology to try to change the world, and then getting hacked through it so compelling that I changed my career.

I've been working on this ever since.

That's awesome. I remember working on security issues during the Arab Spring as well.

I was the CSO at Facebook back then. We had to deal with our own challenges of nation-states in the region attempting to modify our services and capture our logins and things like that.

The other thing I remember about the Arab Spring was there was an optimism about the use of technology that has waned.

Remember that?

There was this excitement that technology could be a means for good, helping give voices to people.

Was that part of what attracted you to it, or was it more like, oh shoot, the governments are actually using technology to get even more aggressive?

I had a personal experience that I feel encapsulated those two realities because me and my collaborators had used Twitter and sat phones and all this other mixture of old -school tech and technical technology and then social media to get information out.

We had restored a voice for a lot of Egyptians who were prevented from telling the world what was going on.

This felt heady and really optimistic.

At the same time, I was already feeling the flip side of this, although technology reduced a historic asymmetry between people and their governments and the ability to get information out, it didn't get rid of the other historic asymmetries in power and risk between people and governments.

What's happened over the last decade is that governments have not only tried to right the balance, but in some cases, they've really pushed to turn the technologies that we all use into tools that allow them to exercise power over us.

I think people who have seen the Arab Spring from different perspectives remember the kind of evolution of narrative.

They were already skeptics at the time.

Benin Raza was one of them who was saying, look, this tech isn't going to fix everything.

But I think for a lot of us, it actually took experiencing both the promise and the threat and the flaw in that thinking.

I think it has led a lot of people, including myself, to think, okay, look, technology is great, but it is rarely of itself a solution to problems.

At the end of the day, everything's political.

If we don't think about the meaning of the technology that we're using and about the interplay of interests and power when we're doing things online, we're burying our heads in the sand.

Ultimately, we're going to get exploited. The things that we're trying to do are going to get eroded in ways that we didn't predict.

One of the things that's impressed me about Citizen Lab is how you get involved, not just on the public policy debate and narrative and discussion, a little bit like what we're talking about now about what should be allowed, but it seems like you really roll up your sleeves and dig into the technology.

I'm curious about, as an organization, what do you think are the priorities of Citizen Lab?

Where does this technical research fit into it? The lab is an interesting animal because our director is a political scientist.

On staff, we have people who are political scientists, computer science PhDs, and everything in between.

What I think that does is it means that when we look at problems that have a technological component, we don't stop our analysis with the indicators of compromise or the attribution.

That's often the jumping off point for the second half of what we do, which is trying to understand the realities of why the technology is being used in certain ways and how people are getting harm from it.

At the end of the day, for me, the most rewarding part about the work that we do is the work with victims.

Most of our work would not happen if people didn't share with us the fact that they were targeted or that they had suspicions.

A lot of our work is built off the bravery of people like journalists and others who will come to us and say, look, something weird is going on with my phone.

It happened more or less at the same time that I was photographing the super yachts that the prime minister's buddies were hanging out on in the Mediterranean, case in point.

That's a case that actually just published a couple of weeks ago with a Hungarian journalist who was being targeted with Pegasus while documenting the prime minister's buddies pounding around the Mediterranean.

What's interesting about our work is that while there's this huge technological component and there's some context, a huge part of what we do is actually direct engagement with brave people.

It's really trying to take their agency and expand it, trying to take their braveness and their cause and help move them forward.

We should dig in a little bit on this Pegasus project in particular.

I think a lot of the world started to pay attention to it when Amnesty did the disclosure of the really long list.

But for the benefit of people tuning in who don't know much about Pegasus, could you give a high-level explanation and then we can dig in?

Pegasus is the name for a piece of spyware or some would say a service provided by a company called NSO Group.

It's basically like a towel in a box, hacking in a box. It allows its government clients to remotely and often without even a click infect phones and turn them into real spies in the pocket of people who get targeted.

The technology is not new in the sense that a number of states have had this ability for a while to turn a smartphone into a surveillance device.

What differentiates Pegasus is that the company that makes it, NSO Group, has aggressively sought investment and growth.

As a result, has sold their technology to a huge number of countries around the world.

Predictably, many of those country users wind up doing bad things with it, targeting journalists, targeting human rights defenders, targeting human rights organizations, political opponents.

The president of Panama was monitoring his own mistress.

States abusing surveillance. When we talk about Pegasus, we're talking about that problem set.

You mentioned the Pegasus project.

For years, researchers, including myself, some of my colleagues, notably Bill Marsak, absolutely brilliant guy at Citizen Lab, have been tracking Pegasus.

At the same time, Amnesty International and their Amnesty Tech branch has been tracking Pegasus.

The things that we would find would really be hard ones. We might find like 30 cases in a given country and it'd be a big deal.

Then this year, something really interesting happened.

Amnesty, working as a partner with a news group called Forbidden Stories, which is actually a coalition of a lot of different major media organizations, published just a raft of stories highlighting the global scale of Pegasus hacking and targeting.

That was really a wake-up call to a lot of people because the targets were not just human rights defenders or journalists.

They were potentially the French prime minister and a bunch of members of the French cabinet and a number of other heads of state, officials, celebrities, and so on.

That, I think, helped to make the threat that this kind of software, which is deliberately designed to fly below the radar, helped to really highlight how not only problematic this stuff is and how spooky it is, but also how widely it gets used and abused.

If NSO Group went away, this problem wouldn't go away, would it? No.

NSO Group is like, in so many ways, it's just the most visible current version of a much larger ecosystem of companies that sell both mercenary spyware and other offensive capabilities to states.

We know a lot about Pegasus and NSO Group for some partly path-dependent reasons.

They're what both we and Amnesty have been looking at for a long time, also because they have a big market.

Ultimately, behind any player like NSO Group are just a host of other companies in a lot of different countries that are all contributing to that offensive ecosystem.

Part of the problem is that because that ecosystem is really under-regulated and under -scrutinized right now, a lot of harm is getting done from the proliferation of these capabilities, which are largely unchecked.

What do you think we should be doing to deal with this offensive spyware market?

It's a really tricky problem. I look at it like a stool.

Three legs, civil society, government, private sector. For years, civil society, and by this, I mean human rights defenders, journalists, and so on, and also research groups have been raising the alarm about this.

In recent years, interestingly enough, it became clear that big tech was pretty pissed off with NSO.

In 2019, WhatsApp and Facebook sued NSO Group. Then last year, a whole bunch of other big companies, Microsoft and Google, signed on to that lawsuit as amici, making it clear that the tech sector has had enough.

At the same time, they're also clearly leaning in, trying to detect the spyware and patch the exploits that it uses.

The third leg though, and that is government action, has been conspicuously absent.

It's a pretty tippy stool right now. I think where we need to go is for governments to recognize that while having some offensive marketplace may be useful to their interests, we're getting to a place where that marketplace is causing so much harm and so much potential blowback even to them, that it needs to get regulated and the harm needs to get dialed down.

Otherwise, we're going to get to a really troubling situation.

Why do you think governments are so silent on this topic?

The knowledge that I like to think about is the arms market. During the Cold War, both the great powers, Russia and the US, from my understanding, benefited from the existence of arms traffickers because there were lots of proxy conflicts in the world.

It was a way to get weapons into those proxy conflicts that you and your side were potentially supporting.

After the Cold War, a couple of things happened, including a massive influx of weapons from former Soviet states, which made a lot of the conflicts around the world more bloody.

A lot of that mediated by arms traffickers.

States really realized, we have to do something about arms trafficking.

It's causing harm everywhere and making conflicts a lot more bloody.

I think we need to get to the same place with spyware. For that to happen, officials need to fear it.

They need to feel that they and their political interests and their party and their prime ministers and presidents are just as likely to get targeted with this stuff as the problems that they'd like to look away from, like civil society and other things.

I think governments also have to recognize that big tech is crying out for assistance and help with this problem, having really recognized that tech and civil society alone can't solve it.

It does seem like the mainstream media is starting to cover this issue more.

There was the zero click Apple vulnerability, I guess, that was associated with this in the last couple of weeks that got a lot of attention.

Citizen Lab played a big part in that one as well, right?

That's right. That zero click was a particularly feisty one. Both we and MSD International had some awareness that there was something going on with iMessage.

Then a couple of weeks ago, my colleague Bill was going back through an old backup of a device that we had looked at months before and found some really interesting GIF files, which turned out to be the holiest of holies, a zero click iMessage zero day that NSO had been using to drop its implant onto people's phones.

We worked with Apple and Apple within a week had patches out for iOS, macOS, and even Apple watchOS.

That was a pretty quick turnaround. Obviously, this was a very severe exploited in the wild exploit, but what's interesting about this case is that you can have all this.

You can burn a really expensive, fancy exploit that's being used by bad actors in the wild, and they may just pivot and start using the next exploit that they have up the chain that they've been waiting for just such an eventuality.

It really shows us that we can all be concerned, like, oh, man, got to patch your device.

There's really a problem here. That doesn't necessarily solve the problem of the threat actor if the threat actor is well-resourced enough to have other exploits in waiting.

Right. Is it just a reality that there's going to be a never-ending stream of exploits?

I think it is. This gets to a really important problem set, which is there's an offensive industry, and there are people who would prefer to keep their head in the sand while finding interesting exploits and selling them.

That marketplace is absolutely fueling harm. I think that we need to get to a place as cybersecurity professionals and people in this industry of really having our norms and values catch up with the harm that we know that industry and that marketplace is causing.

We've got a long way to go because there's a lot of glorification on a lot of that stuff.

I think we have to get to a place where we recognize the role that it's playing in causing a lot of harm.

Yeah, it's fascinating because most of us who work in the world of security, we're not just in it for the neat technical things that we can discover through breaking or that we can invent through building.

We're in it because of the hope that we have a positive impact on people.

It seems like the people who are in this particular sub -industry must be looking the other way.

They've got to be looking in a certain direction.

I was just recently on a panel with Etienne Meunier of Amnesty International, formerly Citizen Lab.

He was making an interesting point. He pointed out that hacking team, which is in some ways a predecessor, part of the old DNA of NSO, and a hacking team was an Italian company that sold hacking capabilities, which sure enough was exposed targeting activists.

They also got massively breached.

One of the interesting things that came out of the breach, as Etienne pointed out, is you could see that the management was singing, telling a nice story to the technical staff, like, listen, everything's OK.

You're saving lives, and so on.

Meanwhile, the management knew what was really going on. There was a different reality there, and the motive was financial.

I'd like to believe that something similar is probably true with NSO, and companies like it.

There's a line that people are being fed, like, look, you're preventing serious crime and stopping terror.

But the reality is this kind of work causes the proliferation of espionage tools, and a lot of states are going to use those for really bad things that are going to make us net less secure.

I think we need to get to a place where not only do we glorify that kind of offensive work, but we also are honest with ourselves about what role it plays in geopolitics.

The answer is, in so many cases, it is fueling authoritarian regimes who would love to be technologically empowered, not only to hold the population in fear and to pry into their personal lives, but to jump across their borders and target people in other countries who have said things that are critical of them.

Wow, this is a topic I could enjoy talking with you about all day long, but we don't have all day.

We only have 30 minutes for this conversation.

I want to jump over to a different topic that I've been thinking about a little bit.

I saw your name in the news back on a little bit after January 6th.

January 6th is a date that we talk about a little bit here in the United States right now because that was the day that this organized group stormed our Capitol building in Washington, D.C.

In reaction to that, the technology community in particular seemed to be fascinated with a crowdsourced approach to identification of who was there that day and trying to hold them accountable.

How did you get pulled into that situation? I've been using some of the techniques that we actually use at the lab for things like infrastructure analysis to try to do some attribution around some of the targeting of potentially susceptible people to stop the steal messages back in the day.

In 2020, looking at some of these weird front groups that were pushing some of Trump's messaging and especially groups that seemed to be encouraging violence.

I've been tracking these groups, looking at domain registrations and other kinds of fun stuff as they moved towards January 6th.

I was getting really alarmed that something was going to happen on January 6th.

I had no idea what. That day, I was watching with concern, but then experienced what I think was the same gut punch that a lot of us felt when we saw images of people on the floor and in the observation gallery of the Senate holding zip ties, masked people wearing paramilitary style clothing.

That really caught my attention and my instinct, as is often the case for me when there's something that really bothers me, is like, I need some agency.

I need to feel like there's something I can do. I started using OSINT techniques to try to figure out who that person was, the guy holding zip ties, the first zip tie guy, using the just absolute torrent of imagery that had come out of that day.

I decided consciously that I was going to use Twitter to show my process in the hopes that others would join me.

In the back of my mind were two fears.

One, there might have been some kind of an organized conspiracy to potentially kidnap people and cause harm to our elected representatives.

I am an American, by the way. Secondarily, I felt, well, we have to make sure that there's some public accountability for these people because who knows what's going to happen between now and the election.

We have to do everything we can to scare these people away from ever doing this again.

After January 6th, a whole massive ecosystem, I'd like to call it an ecosystem of accountability, grew up to try to identify a lot of the people who had gone there, especially the masked people, the ones who were hard to figure out.

It was remarkable to watch because it was sort of like first probably tens of thousands and then thousands of people getting together and sort of pitching in with what knowledge they had to try to make these identifications.

Very quickly, started to get really careful about not identifying people publicly, about developing a set of methods and processes, forming groups so that they weren't exposing all the work they were doing publicly, which is an absolutely fascinating thing to watch.

I think it has partly resulted in making anybody who thinks, oh, maybe I should go visit violence at a state capital, think twice because there's now this possibility that a group of amateur sleuths and maybe some not so amateur are going to do their best to try to figure out who they are and make sure that they get held accountable.

Were you worried that people would be overzealous in their approach?

Absolutely. One of the things that I had in my mind and I think everybody else did was, man, we all remember what Reddit did with the Boston bomber.

One of the things I tried to do and a friend of mine, a guy named Art Toller at Bellingcat, I think says it best, he said, look, anytime there's a big event like this, there is going to be a crowdsourced effort to try to figure out what went on.

The important thing to do is to model careful ethical behavior and the right norms to try to guide that energy towards something productive.

Otherwise, there's always the possibility that it can certainly go south.

Right. What were some of the ways that you tried to do that in this situation?

One of the things that was most interesting to me was the fact that a fair number of the people there wore some kind of military paraphernalia.

I constructed a volunteer informal committee of former military folks who recognized insignia and who could also basically tell me whether these were wish.com soldiers or the real deal.

We actually went through hundreds of mostly men wearing different kinds of body armor and military kit and tried to triage who might be interested, who was wearing gear that was actually legit versus who was wearing airsoft stuff.

This was pictures of patches and gear, and in some cases, trying to identify weapons that people were carrying and other things like that.

One of the other approaches that we did was try to identify people based on who they were around and then try to work those things back towards social media profiles.

At the end of the day, what was interesting is how many tools are now available for people who want to do that kind of open source digging.

One of the big things that I decided on quite early was it would be important to move my process, which was ultimately a big volunteer collaboration, towards working with journalists so that when there was sort of a candidate guest for somebody's identity, the next step was for the journalist to pick up the phone and call that person and see if they would admit to being there or otherwise kind of give themselves away.

This felt like a way to try to make sure that the identification was pretty solid while at the same time ensuring that there would be a public conversation about who this person was and what they'd done.

It's interesting.

I think one of the other things that came out of this is we all have a heightened awareness that we can be identified out in the world.

If a collective group of people care enough about the situation, there's going to be an attribution effort.

That collective group can be volunteers, like in this situation, but it can also be a government organization itself.

To go back to the authoritarian conversation from earlier, technology in the Arab Spring.

You, as a researcher, you're quite a bit out there.

Like you said, you went and engaged on Twitter.

How do you think about that from a personal safety and security standpoint, putting yourself out there?

That's the last question. Yeah, Joe, this is a really good question.

I think one thing that I keep in mind is to try not to glorify the methods, but rather to focus on the meaning.

Because people can use OSINT for bad things.

I've seen people sort of say like, oh, well, it's just OSINT. It's like, well, but to what end?

For what purpose and by who? I think if there's one kind of common thread here, it's that the technology itself needs to be understood in terms of who's using it and what they're using it for.

Same for Pegasus, same for OSINT techniques.

I've certainly been concerned about the possibility of different kinds of digital targeting of me, both for some of the work I did on January 6th, but also for the work that me and my colleagues at Citizen Lab have been doing investigating players like Pegasus.

And we actually had some experience. So both myself and a colleague, Bahar Abdel -Razak at the Citizen Lab, were targeted by operatives for what looks like Black Q, if you believe Ronnie Farrell's book, a group of private spies who seemed to have been sent to try to discredit us and figure out some of the secrets behind our work.

And that, I think, gave me a kind of an interesting experience of being in a situation that had some parallels, although much less parallel to myself, to the people who we work with, to the high -risk journalists and activists who feel every day the tickle of surveillance and of state repression.

And I can tell you, ultimately, we ran a sting back against Black Q.

You can find out more about it if you Google my name, John Scott Relton, and bumbling spy, if you read the New York Times.