Verifying bots and agents with cryptography in the age of AI

This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé ,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie This episode is presented by João Tomé,Thibault Meunier and Kevin Guthrie If I have a website, I own a website, I'm a content creator and I'm hearing this for the first time What should I know or even call to action? What should I do regarding this? Or maybe just be aware of it and not do anything What is the typical website owner behavior knowing this? I think already being aware of it is a big part and thanks for doing this podcast Probably the biggest thing at the moment is making sure that bots and crawlers would be okay to adopt it Make sure that content creators have a voice to say hey we would really like crawlers to adopt it Because it would ease the integration and the deals we can strike down the line If I can identify every crawler then it's much easier for me to say maybe this one we will discuss and have a specific rate limit For this one we will do a special type of payment system For this other one we will just allow it. I want to allow the Internet to archive, I will provide the content for free I know it's them because I have their public key So I think this is really the most important Make sure that as a content creator I have a voice to say I really want bots to be able to identify themselves And for bots, if you are a bot owner and you want to identify yourself, you can do so already This is valuable as we mentioned because IP addresses may be tricky Especially if you multiply them, want to localize your content And user agents are spoofable so make sure that your identity when you're making requests is actually preserved It creates something that maybe it's lacking actually Working with the radar team I've seen this over and over again Like user agent, someone can put like I'm a very stupid person user agent So user agents can definitely be changed in a way But this creates trust in this relation, the bots relation It does and it's like complementary to user agent User agent is like a kind of lightweight signal to allow Hey, I'm using Chrome, if you have specific content for Chrome, just serve it to me I'm using Firefox, if you have specific content for Firefox, serve it to me And this is because clients such as web browsers may have different implementation of APIs Should be like CSS, HTML, JavaScript and others And so as an origin sometimes I want to serve slightly different content To make sure that my content is served and faced differently based on the browser So user agents have a value and this is very distinct This is something that we call a signature agent Because a signature agent is more, hey I'm Google And in this case I'm using Chrome but maybe in this case I'm using Firefox And these are complementary signals and it's totally fine for the user agent not to be authenticated You don't need a signature to say hey I want Chrome content There's no value into that However, probably when you're sending a request as Google crawler for instance You want to sign your request because you want to show that it's actually you making the request And so these are two distinct concepts even though they have been kind of interleaved Because there was no solution to actually authenticate yourself We have the situation in Cloudflare that for example for post-quantum cryptography That was enabled by Cloudflare so the customers even if they don't know what post-quantum is Have that already, that standard being enabled in their websites For those who don't know post-quantum cryptography here is to make sure That when quantum computers arrive the cryptography that is old, that is behind us is protected Your data is still protected even if a quantum computer comes and can break the old encryption But how do you see this take the same path as post-quantum In terms of Cloudflare enabling it for its customers So if I'm a Cloudflare customer and Cloudflare does this with the new standard I'm already with this as a possibility, this signature relation as a possibility So this is much earlier than post-quantum and a different problem Post-quantum had years of academic research, of development And it's still a hard problem that Cloudflare definitely is pushing hard I think in here we're still a bit new but definitely Cloudflare as we will see And engage with various crawlers, with various content creators We will make sure that this is enabled smoothly throughout our stack We already provide this type of detection for verified bots on our platform And this is a new way to verify bots with more confidence So this will be provided as well One of the things is this is not a Cloudflare standard If you are on a multi-CDN setup, if you want to change platform You should be able to do so Once again that's why we're doing work at the ITF and in the open So you're not locked with Cloudflare If you want to go you should still have the ability to trust bots that you have partnership with So you should be able to do that on those CDNs So Cloudflare will enable that and push that because we think it's great But we're definitely not blocking you from exploring those solutions Or even have verification on your own premise One of the things I was also trying to understand a bit here Is how long can this really be in place? It depends potentially of the adoption, of the feedback that the protocol has But given the urgency can we put some timeline into it? Once again, I would not put a timeline I do think we can adopt it pretty fast But given this is something that needs a lot of coordination Sometimes it's good to not trust yourself too much I've written the standard I do think it's well written or in a well written shape That it can be put out, that people can access it We've got good feedback I do think bots can start using it Origin, when they're using Cloudflare for instance, can start using it We've put out examples on how can you have If you have your home server, can you validate these when you use CADI? But probably to have it at the web scale Probably we need some implementation in various languages We need implementation and plugin for Nginx, for Apache We need all the CDNs to start coming and say Hey, we will adopt this standard and implement it as well So we have actually a vibrant ecosystem And this can take time I do think we're on the right path In terms of we're seeing a lot of traction Can we keep the momentum? I do hope so But I won't put a strong date on it Is there a chance that this protocol also helps avoid abuses? Like someone is crawling and we don't want it to crawl your website So this trust relation that could be more highlighted with this Could also remove what you don't want to crawl your website It may, but this is kind of like a side effect This standard is really so that when you are an online bot You can sign your request and identify yourself This allows first to identify possible abuse or breach You can imagine you have a verified bot that is making too many requests Then you can identify it But this is very different from bots which may appear as malicious to content creators And in this case, this is like a side problem Here we already removed a lot of the content of the honest bots that are querying your website Then you still have the bots that you don't want This is a distinct problem But given that now we can somehow better distinguish bots which are trying to be honest From bots which are not We should be able to more aggressively pursue and challenge the bots that we think are not wanted Going a bit to your work in general You've worked in many different topics since you joined Callflare One close to heart, which is the end of CAPTCHAs A very cool blog post you wrote about that Let's end this madness, you said in the title Which was a great one Looking at the work you've been doing for six years now at Callflare What is the main highlight that you think is part of the research area of Callflare's culture? I think research culture is definitely interesting at Callflare There's this unique environment where Callflare has a massive network We have tried to be a bridge between content creators and users on the web Striking a good balance is really hard As you mentioned, Callflare is trying to end CAPTCHAs There's been a lot of work to that extent Still a lot of work ahead of us I do think it's been really, over the years A privilege to have been able to push somehow the Internet in various directions Hopefully in a good way You can see the work we did with WhatsApp To make sure that public keys are delivered securely It's really good The work the research team has done on private relay To make sure that users' IP and private information are not exposed It's really a marvellous effort and collaboration that Callflare has the opportunity to do The effort that we're trying to do This is a multi-year effort across Callflare To reduce the weight of CAPTCHA Make sure that content creators and bots have a way to engage in a meaningful manner It's definitely something that's great Is there one specific highlight? Not really, our topics But overall it's definitely been an exciting journey It's quite interesting to see that it's a research team It helps other teams It collaborates with the whole industry, with standards We've been talking about that It has this university feel about it But it also, because Callflare has a big network Many customers The work that is being done Actually has a very immediate application It has, and that's why I say The work of Callflare research is definitely Sometimes very exploratory Even when it is applied, it's still exploratory And that's one of the missions of Callflare Can we actually bring some cutting-edge research Actually in real-world use cases Let's do a quick fire of questions Maybe we'll find something interesting here What's one protocol or tech idea you think is underrated? I don't know I think TLS is definitely a nice layer That has seen a lot of progress And that could be used more TLS in terms of security? TLS in terms of protocol One area which I'm personally at the moment really interested in Is anonymous credentials Which allow people to have certain credentials Proof facts about them Without necessarily revealing the whole credential So you can imagine you have Let's say an identity card with your name With your place of birth, etc And you can reveal only certain information about it This is true for identity cards But you can imagine it's true for an account That you have at a company If you have a Callflare account You could prove that your number of zones Is between 10 and 20 And maybe that proves something about yourself This is, once again, very research But very interesting If you could redesign one part of the Internet What would that be? More encryption? More encryption from the get -go Definitely not having encryption from the beginning Trying to make sure that we add that layer of encryption Is something that I think is really important That could have been designed initially At the same time, it's simpler to design systems Which are not end-to-end encrypted Or not encrypted at all That was the first version of the web Based on trust And it worked It actually proved that you can use An interconnected network And it has some value You add encryption And we're trying to do it Last but not least What excites you the most About where AI and cryptography are headed? I think this is interesting Because AI and cryptography Overall tend to be in a lot of conflict At the moment And this is because In a lot of cases AI operates on raw data Being able to bridge that gap Would be interesting Can AI operate on encrypted data At a speed and a scale Which is as efficient and needed As AI requires right now And how does that play into User privacy, user trust And user right to encryption So it can help the path Of all of what AI is promising Right now, in a sense Cryptography may have a role here What type of role? I think it's rather broad What type of role should AI play? Rather broad as well And an open question There's many things changing In the industry MCP is something recent There's a lot of excitement In terms of developers Of what they can do With that specifically Is there a use case you see That could be really interesting In those areas? I do see Where I think about MCP Maybe half informed Is when S3 was launched For instance You had storage already S3 provided a convenient API So that you can store objects On a remote server Have a way to provide it And have a blob storage Very efficient In a way you can think of MCP Similar It's an API There's fundamental technology to it But the fact that it can be adopted By more people May help to build more tools And have something stronger That comes out of it Makes sense Thank you so much Thibault You're welcome This was great And that's a wrap I am Kevin Guthrie I am based out of Atlanta Working cross-functionally For the Pingur team And the Speed team I'm a principal software senior No Principal system software engineer At Cloudflare So I'm here to talk about A blog post I wrote a few I mean It only got published a couple days ago But I wrote it several months ago Called Performance measurements And the people who love them This is a blog Where I talk about The company Slash my experiences Trying to make Performance measurements More uniform across the company And outside of the company And make them more useful And approachable to Cloudflare employees Customers People People all over the place Performance measurements Is one of those things That I've been interested in For a long time And I got approached By some teams at Cloudflare That were helping customers Measure their own latency And understand The latency measurements That they were seeing And like how it reflected On Cloudflare Like was Cloudflare good Cloudflare bad How do you measure these things And communicate them In a concise And consistent way So the blog post Starts by Going through some definitions Saying okay If you're talking about latency You need to be able to Talk about things That have actual definitions Latency Has a sort of Vague definition of Something starts And then it ends And that's That's the latency When you're talking about Web latency It gets a little bit more Complicated because There are inherently Two different pictures Of latency The server side The client side So you have to say Okay I am interested In the latency From the client side Because the clients Are the people who Care about The latency That they're seeing I guess the real Bottom line is We have some tools That we are going to be Releasing eventually So the customers Can do this sort of Analysis and see These sort of results But that part Is a work in progress And that's why we say It's coming soon Once you have A handle on the statistics And the math To back it up You can get a lot of Powerful information Out of Out of a simple Data set like We are just using Individual request timing But with some Probability analysis And statistics You are able to get A much more Powerful set of tools From just this Simple data set This is the sort of Information you would Get from Like repeated Browser tests Or real user Measurement Those two things Are important But they are hard To do on a really Rapid basis Request testing You can do Very quickly And in a very Well automated way And being able to Get that sort of Aggregate data From individual request Is a powerful way To use statistics And software So you can Look forward to Seeing these tools And doing this sort of Analysis yourself In 2025