Modern customer security engineering

So myself and Cristian are here to talk about modern customer security engineering. Can I get a quick show of hands for anybody that knows what customer security engineering is? Perfect, nobody can contradict us then, that works really well. Quick way of introductions, this is Cristian Cortez, senior software engineer. He works developing our software. He's built the stuff that we're going to show today. He's also Ireland's first AWS Golden Jacket recipient and a Coupser Opt. I am also here, so I am a tech lead within the customer security team and I have built and architected some of the previous iterations of the system as well. So customer security at eBay, so you now know who we are and maybe the next thing is why are you guys talking about this? Well eBay is one of few companies who have a customer security engineering team and to help define what that is, we're less focused on things like vulnerabilities and network intrusions and things of that nature. We're a security team who are orientated entirely around protecting eBay customers from becoming the victims of cybercrime or fraud on the platform. So we're dealing with stolen credentials and stolen information and trying to prevent that from being used on platform. Our team has been around for seven years and we've protected millions, it's actually tens of millions of eBay customers. As well as protecting tens of millions of eBay customers, we've also banned over six million illicit accounts as well. Because when you operate an e-commerce marketplace, trust is the number one primitive, right? You need to feel that you can trust the platform you're dealing with, which is why eBay have invested in this. There's not many companies who have customer security engineering teams, eBay were one of the first. So what we do is we turn insight from the landscape and that's what we have to be aware of, the landscape of this data is being shared and we try and turn that into actionable signals that our partner teams can use to protect customers on the platform. So we're going to talk about the future of customer security engineering and I think it's also the near-term future of customer security engineering because what we think about as a customer is being disrupted. So this number feels slightly low now, so it's in 393%, this is from an Adobe digital insights at 2026, year-on-year increase in AI-driven traffic to US retailers, okay? And by March they were 42% more likely to convert. So in the other sessions that we've heard so far, it's maybe more like a thousand percent, right? So, but it's not only customers who are using AI, it's also the cyber criminals. So there's that kind of duality of what we're trying to do. So agenda commerce is opening up new challenges to us, okay? So as I mentioned, not only are customers adopting it, but cyber criminals are, and AI-assisted cyber criminals can not only do more than they could before, but they do it faster. So Grant the Scissor was talking yesterday about the arms race, right? And how they're not encumbered with organizational constraints, right? So they are maybe have a bit of a head start in that race. I think the other thing about scale and speed is, and this is something that we've talked about in a few different places, I don't think it removes the need for partners. I just was in the last session on phishing, which was between Microsoft and Cloudflare partnering to bring down global phishing issues. And then the one before that was with Scissor, and that was again about partnerships. So I think the title of their talk is probably really key here, nobody defends alone. And that's still the case here. Even if we can move faster, having partnerships is so important to us. So customer security needs intelligence. So I think one of the key topics that I've seen in the other sessions I've been in has been around understanding agentic identity and intent. So that's a really important part. So yesterday, there was talk about trusted agent delegation and all things like that. So that's a really key thing that everybody in this industry is trying to work on. So even if we haven't, when the intent is deferred, how do we position trust going forward? So we're going to talk a little bit about why we decided to make a shift step in what we were doing, why we picked Cloudflare to do that, and what we're building going forward with an example of that. So this is a quote from our CEO. He's used it a lot. And it's, don't AI enable the past, but AI invent the future. And that's how we felt about our existing systems. They were working, but the landscape was shifting at an unprecedented pace. And trying to tinker with and update and augment our existing systems to manage that didn't feel right. So we thought we needed to take it from a new paradigm and basically combat agentic attacks with agentic defense. So we went to Cloudflare because it's security focused, not only because it's a very secure platform, but also from a security partnership that we have. We also were looking at a platform that develops at the speed of AI. I don't know if anybody else is struggling to keep up with all the updates and product releases that are going on. The developer platform that we were using now is completely different to the one at the start of the year in a positive way. It's built for agents. And that's what we mean when we say built for what's next. Cloudflare seemed targeted on the agentic web. It seemed to be dealing with those primitives. They seem to have an understanding of those and identifying agents and understanding them. It's not like the right place to build what we needed to build. And I'm going to pass it over to Cristian. Thank you, Patrick. So cyber criminals are moving at the speed of AI. What does it mean? Just a cliche, maybe. What is the speed of AI? So for us, I spend a lot of my time reading chain clocks. And this is because every new model that gets released represents an opportunity for us to be better, to be faster, to be more effective. But at the same time, if a cyber criminal detects a possibility to gain advantage, it won't hesitate. It will act immediately. The cyber criminal doesn't have a legal team to go to if they need to onboard a new model. They will just go for it. So that forces us to stay ahead of the emerging trends, I would say. I was thinking of an example to illustrate this problem. At some point, system 1 models appear. You know, these are fast classifiers. We've been using classical models to do classification. And of course, they are not good at that. At some point, they appear. There's an emerging trend there. And we spend some time identifying our system, where can we implement a system 1 model, which leads us to a few weeks of investigation from JEP. To give you an example, for us, it's important to know resources. As Patrick explained before, we have venues, underground venues, where we collect data. These venues have a life cycle. Basically, for us, it's very important to rank them before we decide how we are going to use the resources to extract the data from them. Some time ago, we have implemented a ranking. So we have a ranking that was determined before this. This involves a lot of data, grabbing samples, adding historical metrics we know from the venues, and then basically rank them. And we have a ranking with the classical model that took around, and we do this in batches, took around 10 seconds to complete. I think you know CLEF and CLEF Flash were launched by a few weeks ago, were launched by Cloudflare. They launched a system 1 model, which serves to classify for this purpose. They launched on a Thursday. On a Friday, we have it working, because we already have homework. We know where to swap it. The result is now we have the ranking of these batches under one second, consistently. We rank all the time. Every four hours we have a job to do this kind of ranking. So I think that shows very good how, you know, we are doing this. I think my colleague, Patrick, 72 hours later, found another use case for brand protection and phishing, because we knew, I think Chavez 0.042 per 1 million input tokens. CLEF is 0, CLEF Flash is 0 .09 per 1 million input tokens, but it has a larger context windows and support images, which allow Patrick to use that for phishing detection as well. Okay, so I'm here to talk to you about Project Starlord. Project Starlord is what is describing the year we have. Project Starlord has three objectives. The first one was to migrate the Kubernetes collection pipeline that we have in a different cloud into Cloudflare. The second objective was putting a harness for security engineers and full stack operators. If that is not ambitious enough for one year, we wanted to make it fully authentic. When I talk about fully authentic, it means once the pipeline is up and running, we need to have agents to monitor the different type of collections. We need to have agents to generate reports for metrics, and we need to have agents to basically do their own code. An example is starting workflows from Cloudflare. I think we went over this, but I will go very fast. I know your problem space will be different than us, but just to remind you, we collect for these underground venues, we do ETL jobs, we generate actionable insights. For some of them, we act directly, and for the other use cases, we generate signals so other departments between eBay can take action. So this was our objective. The other part is telling you probably what, which, the other question is like, what are you using from Cloudflare? And the quick response will be pretty much everything, right? We use pretty much everything from D1 to KBS Store, R2. So we're thinking, which are the critical parts that help us for the migration? I will say I'm pretty happy for the data catalog team. We store a lot of data in R2. I know if you're familiar, but you can run R2 SQL. They have Apache Iceberg, so you can have your own data catalog to help do SQL on top of the venues data. And we can lower that into the agents as well, for give us insights. As you can think, these venues are in different languages, so AI is there to help us translate, to detect intent, and to go for opportunities as well. Another services which I love, I think, is the other part of the corner store is the containers. Containers are super powerful. You can have container placement for different regions. We find cases where we need to spin out containers in a particular region to satisfy the need of finding cyber criminals that are targeting customers from Spain or from UK. So that's another feature we use a lot. On top of that, we use workflows. Workflows are super powerful. We use workflows to ingest not only our collection, but also to consume signals that come from third-party vendors that collect similar information that we lately aggregate. And I'm here to tell you now why we needed a harness. We needed a harness for security, for researchers. And the lesson learned is here we started back in January. And, you know, there was OpenCloud. There was not a name in January, December when we started. It was mobile and changed names. But we wanted to have a harness where a security researcher can go and start turning his job into something that is AI-assisted initially, which means they will develop the tools, the MCP tools, to share reports, to automate certain actions they do on top of these venues. And this was a great decision. We started earlier with this. We didn't give them the perfect harness from the beginning. We just gave them something on top of which they can start creating MCP tools and start evolving them and start making them more mature. This was fundamental for us later on. The other thing is, of course, having skilled jobs, which is something that OpenCloud supports. Let's see how it all aligns together. I think it was February. Cloudflare launches ModWorker. That is basically the combination of OpenCloud running in a container attached to a worker. Of course, we jumped immediately into this. And that was the other part. We didn't want to bring our own key and have external models. We want everything to be running in Cloudflare. So, we collaborated with them. So, they enabled for us using directly the worker's AI. So, that's the other part we use as well. Like, AI gateways have been for us super useful. We continue evolving. At some part, we have this idea to invent and simplify. And we say, we really don't need all this layer of OpenCloud. What's inside OpenCloud is a PI coordination. So, we started developing our own harness inspired in PI coordination. As you know, if you are PI users, such as myself, in the past, there's no direct support for MCP tools. So, we needed to either go for a plugin or develop that logic ourselves. And to prove we were going exactly in the same direction as Cloudflare, at some point Cloudflare announced the Think framework. The Think framework is something that was, as well, deeply inspired in the PI coordination, which solved for us all the needs we have for sessions, for organizing the sessions, loading the tools. So, when we see that, we say, like, there's no more need for us to continue maintaining our own extension on top of the PI coordination. So, we migrated to Think. At some point, our architecture looked like this. We have the worker. We have the Think harness. Think about a UI, like, similar to chat GPT UI that you have, or better than that, similar to Cloudflare OS, okay, where the researchers can contribute, have the chat, the sessions. We load the tools. We can assign them the models directly from Cloudflare. And then the initial versions, we needed this. We attach containers, containers with Docker file images, where the researchers can have their utilities and perform analysis directly on the cloud. And this is, as well, the power of the edge locations, because sometimes being near, putting it near to the edge location where the venue of the cybercrime is hosted is an incredible advantage, okay? And this let them start working their own MCP, so have MCP versions of the APIs and tools they were working on. We didn't mount everything on the five seasons, just a few folders, initially, that were mounted in R2 that we need to survive across container restarts. As well, we have to develop a browser for the agent. And then we have the workflows with the third party that I explained earlier. We consume not only our SQL data catalog, we use signals from third parties as well. If you need to do it today, this change, as well, this is a snapshot of the past. We don't have containers, except for rare cases where we need it. We try to avoid them because it's a surface area of, you know, like we need to maintain a golden Docker image and we don't want to get back there. So we use more. And for storage, we use artifacts. Right now, we solve the problem, as well, of agent orchestration. Right now, this is very simple. Our agents don't have access to Internet. We created as much tools to provide metrics and action that they live in a closed circuit. And all they know is what we provide to the MCP tools. So these artifacts is like a Git-like file system that the agents can use. And there's not only one of them. They work in a group. They have a hierarchy. We have an architect agent that is the one that sometimes gives the tasks to the other agents and leads. So they work in an org chart. If you are familiar with Gastown or Symfony, OpenAI, you will know what I'm talking about, or Paperclip. And the good part is like, now they have the tools to do the on-call. And they have other funny parts, like they generate reports. We don't have to go and have this assistive part of sharing the reports. Every morning, I get to the office, and I see that the agent has a stand-up. Right? So I can read what they identify, they are the action items, and how they correlate the metrics. What can I tell of results? One of the results is, compared with previous year, we increased five times the amount of protective actions. And now my colleague, Patrick, is going to tell you a bit more. Yeah, thanks, Cristian. So Star -Lord is probably more of an example than a blueprint for everyone else to follow, right? It was talking about a problem that we had, which was that it's hard to keep up with the shifts in the landscape and underground venues that can be due to the platforms themselves disrupting those venues. It could be law enforcement, it could be other things. The speed of that adaptation has increased dramatically. So we needed to increase as well. When you see these numbers that we have, you know, raw object collection of five times, I think if we'd stayed still, we wouldn't have got 1x of last year, right? Because it had been disrupted so much. We needed to change our pace of adaptation to be able to match it as well. So when we think about the next steps for us, there's a few different things that we want to do. We're looking to scale without linear operations. What does that mean? Well, it means that before it would have taken a significant time investment to be able to collect from different platforms, different types of places where data is being shared. That is no longer the case. We can adapt so much more quickly to get into some of these areas to access data. We also are continuously redrawing the boundary. So agentic engineering may have lowered the cost of building software. It hasn't quite had the same effect on the cost of owning software. It's happening, but not quite the same way. So we need partners. We still have partners. There's been a huge shift from all third-party collection to primarily first-party collection, which is what we're doing with Starlord. But we still have third parties to augment and support us with that. And then the next thing for us as well is that we're moving from agent identification to agent intelligence. We've talked about customer engineering. If an agent is a delegate of a customer, we need to think about how we can think about the security of that delegate. ATO used to mean account takeover. Maybe now it means agent takeover. So our mission is the same. We just think that customer security engineering can't be treated like any other security team. We have to treat it as a platform engineering move. And that's what we do and will continue to do. I think we're absolutely out of time. So I just wanted to thank everybody for listening to us today. Thank you very much.