Disrupting scam and fraud networks with armies of AI agents on Cloudflare

So, my name's Pete Eckermann. I'm the co -founder and chief product and technology officer at Apartheid. What do we do? We waste the time, we break the business model of scammers and other threat actors globally. One thing about scam is it's a global problem. It's a $1 trillion industry. Pictures like that of scam and fraud compounds across the world, which are ran exactly like organized businesses, down to the fact that they've got script writing departments, finance teams, infrastructure teams, you name it. Where we come into this is we want to break their business model. And we do this in a couple of different ways. What we initially set out to do at Apartheid was change the economics of fraud. And ultimately, where our research began from the lab at Macquarie University in Sydney, Australia, was how do we waste their time and how do we take on and I suppose waste the human resource that scam centers operate? So, long conversations. I think our longest conversation to date has been just over two hours, where we're talking to the scammers, we're extracting intelligence on those voice calls, as well as engaging and then really taking out their ability for their call center operators to be able to make more calls. So, it completely demolishes their business model and starts to mean that there's less scam calls and there's less volume hitting people and hitting telcos and networks. But what we found very, very quickly in our journey is that voice was just one element of the entire equation. So, not long after we started with the voice technology and we deployed within the biggest bank in Australia and the biggest telco in Australia as our first clients, we started moving into Telegram, WhatsApp, SMS, web, email, which is surprisingly still one of the biggest threat vectors when it comes to scam and fraud and moved across, and those numbers are quite outdated, millions of conversations across each one of those channels and multiple artifacts extracted. So, our agents will go and extract URLs, the metadata, understanding infrastructure of the scammers and the fraudsters that we're talking to and be able to package all of that information up where we work with our partners, for instance, within law enforcement, intelligence agencies, telcos, and banks to take action and disrupt. But we might, just before we go too much further, I'll just play a little demo of what it sounds like on one of the calls that our bots have with scammers. I booked a tour to be delivered to an address in Glendale, California. Be for real, mate. I'm in Sydney, not bloody California. I mean, no. I definitely didn't buy anything like that. Okay, hang on a minute, mate. Like, are you saying I literally can't go buy on GitHub right now? That's kind of dodgy. Well, like I can get it and I just don't think that's the right shirt to wear. I still don't believe my shame over there. One thing to be told, like, this wasn't a big deal in Oregon. Like, here's COVID. Oh, it's been 21 years that we've faced. So, we've got hundreds of thousands of personas that we deploy in these environments and they take on wherever our victims and wherever the population we're protecting sits. So, for instance, across Australia, we mirror the demographics of the Australian population. Our deployments in the UK, very, very similar. We use personas which can be all sorts of age demographics and ranges, all sorts of cultural and different multicultural backgrounds as well to make sure our victim bots are the most convincing in what they do. One of my favorite things, though, is the frustration that we cause every day. Could you repeat that? Oh, dear. That sounds a bit intense. You are a... This is all day every day. So, we're probably one of the only companies in the world which has an OKR about the number of F-bombs that we get on our calls and text messages from scammers. It just means our product's doing the job. So, everything, though, from those conversations turns into intelligence. So, we understand from voice what the tactics, procedures, the way that they're operating these particular campaigns. And we can see these targeted campaigns spinning up against particular targets across all of our deployments. Say, for instance, we can understand when there is a scam trending within any country that we're deployed in and understand exactly what the tactics are so we can protect consumers. But beyond that, it's understanding the mule accounts, understanding what the payment rails are, and effectively all the infrastructure. So, we can understand the one side of it being the procedures and the tactics. But then, secondly, we've got actionable intelligence and insights that we can go and do something about. And so, we partner with some of the largest banks in the world and across...and telcos as well, I should say, to say, OK, we've identified that this particular bad actor is using this account. Let's take that account down. Let's debank them. Or let's start to do some offensive work against them to uncover their network, understand exactly what they're doing, and blot their attacks across multiple vectors. So, something about it, and I suppose this is the...I suppose a little brag sheet here. We actually reduced, for the biggest bank in Australia, their scam losses by 30%. The CEO of the bank at the AGM last week actually mentioned us by name. I think we're the only vendor where that's happened in recent memory, around what we've done to protect all of CBA's customers. And just to give everybody an idea of the scale of that bank, they probably bank around 60% of all transactions within Australia. In terms of consumer losses, we did $183,000 of losses prevented in one afternoon in one campaign. So, 15 live scams, a callback scam that our agency interacted with and immediately blunted that attack for one particular bank, Macquarie in Australia. Typically, and we work with a number of intelligence-sharing networks, we'll detect fraud events and detect campaigns, typically 12 to 13 days beforehand. And the last one, and I updated my slides but I don't think it's taken, is that we've done close to 17 years worth of engagement with scammers just on voice products with telcos alone to date. So, effectively wasting that much time that they've been able to really stop that impact that they've had across our customers. So, some of our clients, I'll just skip through this pretty quickly, but you can see some pretty big logos there and some of the partnerships that we're building now across the US as well to start to blunt the attacks here. So, one thing that I want to speak about is what we've built on Cloudflare. Now, initially when we constructed this, we're researchers and academics and a lot of our engineering was cobbled together in Python and having a little bit of fun and I suppose not really built for the efficiency or scale problem that we faced immediately. Ultimately, this meant a massive blow in costs, especially in the early days of our company. So, typically on a deployment we might be taking 30 or 40,000 calls, we may have to deal with 150,000 calls, this is daily numbers, at any one time when a campaign spins up against a particular telco or a particular bank. So, for us that meant cost immediately. So, we made the decision to work with Cloudflare to refactor and can reconsider around how we built our agent platform. So, initially it was one bot per container on our AWS setup, idle most of the time because if we've got a bot which is sitting engaging on WhatsApp, you're not going to get a WhatsApp message every second. So, we've got bots and compute there sitting there doing nothing and that linear cost scaling for us. So, why we chose Cloudflare? So, we've scaled our infrastructure close to infinitely. So, we're using a system of durable objects and every conversation being held by the durable object. So, sleep when idle, wakes in milliseconds and we don't have to spend the overhead in terms of managing servers and managing that infrastructure. It costs less, the cost stops growing then with every bot. We've got around on text close to 27,000 bots operating across countries and each one of these now we can scale even further and further and further to take on more conversations without that cost growing and it's much more faster and simpler to build on. Now, we've unified all of our platforms across text, voice, using AI gateway and workers AI as well to mean our infrastructure is much more simple to operate. What that's meant for us is that we've been able to quickly and rapidly scale our business, move across multiple jurisdictions and get more intelligence coverage much, much quicker than what we used to be able to do. So, today we're using Cloudflare Workers for ingress and routing of every single message that we're receiving, receiving hundreds of thousands of messaging messages per day, relaying via Twilio and where we have direct interconnection with telcos using workers. Durable objects, so one per bot to hold state. So, what we can do is have conversations which may start on a voice call and then move on to WhatsApp or move even into an email or different channels because typically what scammers try and do is move to the encrypted channels as the end point where they'll perpetrate their scam and release that sort of intelligence. That's typically where the payload of payment information or crypto wallets or anything like that really sits. We use Cloudflare containers to maintain our connections with WhatsApp and Telegram. We're working on Signal at the moment and a few other channels, including audio is there. So, we can pick up the phone when someone calls us on WhatsApp and be able to have that discussion and keep that persona alive. And we're using the AI gateway. So, for us, that's been a fantastic way to centralize all of our AI requests and the load that we have. As you can imagine, doing this sort of thing, generating responses is a heavy load on TTS, STT, and the models that we use for both analytics, response generation. So, it's allowed us to centralize and control that plane much, much easier than we've been able to in the past. Plus workflows, D1, R2, warp gateway, just about the whole stack of Cloudflare we're using in our business today. And for us, yeah, this has been the biggest thing for us. 50% lower cost than our previous infrastructure on AWS. 500 to 1,000 concurrent bots per container that connects to WhatsApp or to the Telco. And then the DO is running underneath that container. And one to three milliseconds to wake an agent. What used to happen for us was if we had a pod die or something like that, we'd be out of the conversation. And if we're out of the conversation, what it meant for our clients is that we're not actually extracting that intelligence. So, in the past, we'd had situations where we'd stop mid-conversation or the schema may be right at that point where we're getting bank account details or extracting out that intelligence and we'd miss that. And so, now we've got that confidence that every time we're in that conversation that we're able to extract that intelligence reliably. We know that our agents are going to wake up and engage with that traffic and be able to hold that context across each one of those conversations to maximize our intelligence and what we can do in terms of disruption. One thing as well, and a lot of the theory that we have around our engagement is based upon the concept of hidden Markov models and the Stackelberg equation. And ultimately, what this means is every time one of our agents goes into one of these discussions with a schema or other threat actor, we go into an adversarial environment. And the way that we consider this and the way that we deal with this when we're generating our responses is that we use a combination of LLMs in every conversation that we're having. So, ultimately, our technology and where our research was in modeling every step of a conversation. So, for instance, during this speech, I introduced myself. We did a little bit around setting the scene. Now, we're getting into the technology a little bit. We modeled that across scam and other threats. So, we can understand at what point is the hook, at what point is the part where the threat actor is going to provide those details. You know, how are they building trust with a victim? And we use a machine learning approach to influence our agents around what part of the conversation that they should be in. But ultimately, we also give the agency ability and that orchestration to pick the different LLM models so we can make sure that it can't be fingerprinted, to make sure that the way that we're engaging in each conversation, that we have the best chance of pulling out that intelligence. So, openly, what that means for us in terms of workers AI is utilizing open models on there, models like GLM and GPT -OSS. We do run a few other models out there, but it's allowed us that flexibility and then that visibility over what we're doing. Likewise with the gateway, and then ultimately, our tasks become cheaper and cheaper and cheaper. When you've got the quantum of data that we see, and we're typically sitting within around 17,000 groups every single day in terms of Facebook, WhatsApp, and Telegram, there's a lot of processing for us. And for us now, using things like Cleft, for instance, we can process every message in one of those groups at relatively low cost and high availability for us, which, well, it's meant for our business is that we've discovered all sorts of illicit activities where our bots were only looking for instance, typical investment scam. We've seen things like illicit tobacco, illegal gambling, people smuggling, all sorts of things. Now we can work with our clients to start to address those problems. So, ultimately, a lot of what we do as well is around the agent design and how we run, I suppose, each element of the conversation. And so, we have those faster models holding the conversation and keeping that sort of super low latency as we work with the scammer to extract intelligence. So, what that means for us, so one of our Telco clients is one of the largest in Middle East and across 18 different countries. We can run that on Cloudflare right in their environment, right next to their environment at super low latency and high concurrency, which is important for us in terms of keeping the conversation flowing. But whilst we're doing that, our agents are also operating in the background to provide additional context. So, we're doing tool calls, we're understanding, have we seen this person before? Have we seen this particular scam pattern before? When we're on WhatsApp, for instance, or some of the text channels, we may be generating images and doing things like that as well to provide, for instance, the scammer often will say, okay, can you show me proof that you've sent this payment? Whilst we're engaging, we'll send them a message on SMS or on WhatsApp or a different channel to say, here's the payment receipt that we've generated in the background, this is legit. In some circumstances, we're deploying as well synthetic credit cards, canary tokens, things like that to capture infrastructure and understand exactly what is going on there. One thing that I'm really proud of my engineers is, I suppose, the complexity as well that's been removed from our code. So, we've went from 90,000 lines down to 20,000 lines in terms of what we're doing. And it's meant for us, this refactor has made it a lot easier to maintain and then start building more features. So, a lot of the stuff that we can now focus on has been on our product roadmap for quite a long time, but now we've had that refactor and understand exactly where our product is sitting, we can really move a lot faster than where we have in the past. And ultimately, and I think I mentioned before, the products used on Cloudflare Workers, durable objects, containers, workflows, D1 and R2. And you can imagine that with a company like ours and the sort of threat actors that we're dealing with every day, we've got a pretty big target on our backs most of the time. So, securing everything behind Cloudflare as well. So, building fleets like this, what I'll tell anybody is really to match the compute to the job. That's probably one of the mistakes we made early on, but now I've been able to course correct on that very, very quickly. And now we have a much, much larger fleet at quite a significantly lower cost. Things like what we do in terms of believability and that control as well, we keep human hours. That's part of what we do in terms of our believability, but it's also another measure where it allows us to start to mimic and start to look more like humans when we're keeping, you know, dayline hours for the jurisdiction that we're in. We also do proxying, use a number of different ways to present as if we're in a particular country. We don't depend on the one model. That's been something that's been very, very important for us and really sizing the model to the jobs being one of the key learnings that we've had. In the past, you know, we'd use the highest quality model and believe that that was going to be the way that we push things ahead. But ultimately, we've continued to change models and refactored so we can understand how those different models can impact the conversations that we're having with threat actors, as well as seeing and being able to experiment between choice of model for the different workloads that we have. And one thing that we've done is really, rather than going big bang, we've migrated one of our fleets of bots first and then brought the rest across, which has allowed us to keep continuity for all of our clients, keep the intelligence flowing, but then build out on Cloudflare. Thanks so much for the opportunity.