Workers AI, Email Routing, and ZX Spectrum retro computing

Hello everyone and welcome to This Week in Net. We're in Lisbon, Portugal at the WebSummit event. There's 70,000 attendees. We spoke with a few. One was a former chief decision scientist from Google. The other is the CEO of Internet Society. We spoke, for example, about cyber threats, threats online, but also generative AI. So, a lot to talk about. We're going also in this segment to talk with Celso Martinho, our very own Celso Martinho, about workers AI, our developer platforms related to AI in this case, but also about email routing. So, let's get started. I'm Cassie Kozarkov. I am CEO at Data Scientific, former chief decision scientist at Google, where I was for 10 years. And I have recently moved from New York to Miami, so now I'm Miami-based. So, we have habits of trust, and by we I mean people, all people, have habits of trust that are a little bit outdated. We are slightly too trusting. And the analogy I have for this is, in the early days of Photoshop, people would take a photograph much more seriously than they would take it today. When you look at those old photographs that were claiming to support the existence of fairies, those black and white pictures, or the Loch Ness Monster, some people actually took that at face value. They thought, wow, this means that surely fairies exist, because someone had found a way to put little wings on people and make that look miniature, that looked like evidence. Today, no person in their right mind would look at that image and think that means something about the outside world. So, what changed? It's not like our brains changed. We didn't evolve biologically. We got different habits of trust. And it takes a little while for habits of trust to propagate in society. Unfortunately, the technology is changing faster than the speed at which we culturally adapt how we interact with information. So, I don't see any of these problems being a problem in a hundred years' time. I do think that there will be some teething pains because we are slightly too gullible for what these systems can do. And our expectations of what something that is, say, machine generated versus human generated, our expectations for how to tell one from the other are outdated, which makes us the weakest link again. We are slightly vulnerable to manipulation. And the faster automation accelerates, the faster these technologies will change, the harder it will be for us individually to keep up with what we need to know in order to keep ourselves safe online. That said, I do expect that there will be large scale systems that do some of the intermediary layers and that protect us a little bit from the most egregious violations online. Again, whatever the cyber attackers have, the defenders will be able to build against eventually. There will just be a little interesting period where things shake out. Anyway, a lot of the tools that we have are just a stepping stone to better tools. And because we have automation accelerating, we will develop improved versions much faster now. So, we will get past that. These are the awkward teenage years. That's how I see it. This is where technology has braces on, you know, it's got all its pimples. And I think that we will grow into having tools that are so much more natural and useful. That said, what that will allow us to do is build much more complicated systems. And the way that education has worked, and my STEM, that is science, math, engineering, technology education, has worked in my generation is in order to be able to use something, you have to develop it from scratch. You have to know how it works from first principles. And there's this concept that one person can know the whole thing. That concept has to go away as we build more and more complex technology. So, then how do we think about collaborating with this complex technology? We will have individuals who only know a small piece. We've already got that. That's how software engineering is, but much more egregiously now. People will not need to be experts in building it from scratch just in using various pieces. And we'll need to know who we can trust to do their piece with their piece of technology as a useful collaborator to us. We don't have good systems and structures for that. We don't have good systems and structures for testing technology, for individually testing technology, for testing technology created by groups of people. And these are skills that as society, as humanity, we will need to build. So, we are being outclassed by the Lego-like potential of these tools. We can build things. We just don't know if they work when we build them. And we don't know the consequences. We don't know the at-scale consequences. So, when you mention things like cybersecurity and threats and holes in these systems, they will be full of them. And in order to plug those vulnerabilities, we'll need better systems of collaboration at scale. I'm Andrew Sullivan. I'm president and CEO of the Internet Society. I live in Toronto. We're a global non-profit. We have members all over the world, about more than 100,000 people who are members of the Internet Society, a number of organization members. We have chapters all over the world, and we have staff. We try to make sure that the Internet is for everyone to make sure that the Internet grows and is strong so that we can all benefit from it. I mean, anything that you can build, you can abuse. And so, we know that everything we build, somebody's going to say, oh, I got a way to break that and use it for my nefarious purposes. But what's interesting to me is the ways that we have countered this and the ways that we've been effective in addressing this have all come from innovators on the network, have looked around at, oh, how do we address this kind of thing? How do I seek that traffic? How do I address this massive botnet or whatever? And the place that really works is that kind of innovation rather than sort of saying, oh, well, we're going to make this illegal. Sure, you make it illegal, you track people down, you arrest them, you charge them, whatever. But the really big effects are, this attack just doesn't work because people have learned how to do that. And a whole lot of denial of service attacks that used to be really, really effective just don't work anymore because people got together and said, OK, what am I going to do about that? We got routing security. Routing security has massively improved over the last several years, partly because of an effort that the Internet Society has worked on called Manners. These efforts are the self-interest of the network operators. And so network operators get better at it. And that's the magic of the way the Internet works if we allow it to operate. Well, I think Cloudflare is doing marvelous work. I think that what we want to see is Cloudflare continuing this kind of effort that you've had for many years of having deep understanding of what's going on, of addressing these kinds of attacks and sort of figuring out, hey, I'm going to use this data in order to make the network performance better. So, you know, my view is you guys should keep doing what you're doing. Hello, Celso, and welcome to This Week in Net. Where are you? I'm in Lisbon at the office in Portugal. Thank you for inviting me. Thank you for doing this. For those who don't know, what do you do at Cloudflare? It's a good question I ask myself as well often, but I'm an engineering director at Cloudflare, been with the company for three something years, and it's been a lot of fun. And you worked throughout the years in different projects, right? More recently, Workers .ai, but different projects. Yeah, I've done quite a few things since I joined. Raider is obviously one of my first projects and teams, and I'm still very involved in that, mostly for emotional reasons now because the team is pretty solid and big. But I've been doing other things like email routing, D1, and now more recently, Workers.ai, which is a completely new but very interesting world that we're trying to build a platform for. In the past few weeks, there's two blog posts. You participated in email routing, as you were saying, and Workers.ai this week. Let's start possibly with email routing. The blog posting question is email routing, subdomain support, new APIs, and security protocols. What is that all about? We launched email routing, I don't know, about two years ago, something like that. And it quickly became one of our most popular products at Cloudflare. It's completely free, available to all customers, whether they are paying for Cloudflare products or not. They can just go and set up email routing and start using it. And so email routing is basically like an email forwarding solution where if you're hosting your domain at Cloudflare, you can just click a button, and we'll set up things for you. And you can start receiving emails for your domain through Cloudflare. You can set up some rules, and then what we do is we forward those emails to the upstreams you select. So suppose that you have a Gmail account, but you want to receive emails at joantomeia.com, you can just set up your domain and email routing with us, configure an email rule, and we'll receive your emails for you and send them to your selected Gmail account. We've been adding features to the product since we launched it, and it's become a pretty complete product in terms of features and security protocols that we support. But there's two things that we've been working on lately that customers have really, really been asking for for months. So one of them is subdomain support, which basically allows you not only to receive emails on your top domain, but suppose that you have like, I don't know, corporate.joantomeia.com subdomain, and then a personal.joantomeia .com subdomain, you can actually separate those two subdomains and start receiving email in each one of those independently. And you can actually set up different rules for each of those domains. This is especially important for bigger accounts that have complex email configurations, but it's still very important for overall all of our customers. And so we finally shipped it. There's already a couple thousand accounts using this feature since we announced it. So, you know, really proud that we were able to ship this. The other thing we've been investing in email routing is basically worker support for it. Cloudflare Workers is our compute platform. It allows you to deploy applications on top of our network. And we've just been adding email support to Cloudflare Workers through the last months. It was already possible to receive emails on Cloudflare Workers and treat them programmatically. What we did now is a couple of things. One is you can now reply to emails on a worker script. And you can also send emails from the worker without receiving any email in your domain. So suppose that you have an application that has like an online form where your customers can just put some values in the contact form. And suppose you want to send them an email when they complete the form. You can actually do all of that in a worker using email routing and using the new APIs that we just launched. And third... There's a few practical examples here of that working. Daily Digest with the news from your favorite publications, alert messages. There's a few here. Yeah, exactly. So you can just send emails whenever you want in your application. Obviously, we do have some security requirements. You cannot just start sending emails to anyone because that would be a security risk in terms of email reputation. But as long as you confirm that you're the owner of the destination email inbox, you can just send emails to that. Yeah. And the other third thing we've been doing is just trying to keep up with the latest security protocols out there. We already supported things like SPF, the Keen, DMARC, but we've added a few now. And those are ARC support, which basically tells the upstream email providers that the email went through Cloudflare email routing. And they can just trust us because we actually signed the messages proving that the email went through email routing. And the other one is MTS, which is kind of an improvement to using TLS with email. And it kind of solves an attack vector. And it just makes communications between email servers more secure. So this brings flexibility, security, more ways of exploration, even for those developers that are using workers. It's a very broad launch in a sense, right? Yeah. We basically aggregated a bunch of things we've been working over the last months, and we just shipped everything and announced in the same blog. You know, email was created in the 80s, actually. Pre-World Wide Web, in a sense. Yeah. It was never designed to be a secure protocol because the Internet was different back then. And the patterns that we have today were different. But email has become one of, if not the most important communication tool on the Internet. So the community has been adding features to the protocol and even new protocols on top of the original one to improve email as a whole and make it more secure and up to date with what we expect from services on the Internet today. Absolutely. Before we go to the Workers.ai perspective, let me just say to our audience that we did a very special episode about our postmortem on Cloudflare control plane and analytics outage that is here with Nitin Rao, that is already posted that people can see that episode in specific. Now we can go to specifically to the streaming and longer context lengths for LLMs on Workers.ai. Yeah. What is this really? What is this really? So AI is all the rage, as I think you know. It's hyped. Yeah. It's a generative AI, even though I was at Web Summit and that was in most people's mouths in a sense, speaking always about generative AI. Exactly. So we launched a constellation a few months ago that evolved into Workers.ai, which we announced on birthday week. And the goal of Workers.ai is to give our customers the capability of doing machine learning on top of our network with very competitive pricing and making it really easy for any developer, whether they have experience with AI and machine learning or not, to just start using any of our machine learning models in minutes, basically. And the adoption has been amazing. So typically to Cloudflare, what we've been doing is we launch what we think is a pretty good product, but we've been listening and we've been iterating and improving the platform as we learn more and as we listen to our customers. Two of the most requested features for Workers.ai lately have been improved LLM support. So currently we support LLM, which is one of the most popular LLMs by Meta on Workers .ai. It's open source in that case, right? It's open source, but you still need to apply to use it with Meta. I'd say simple process. We do have a partnership with them, so it's available through Workers.ai for all customers. And then there are different versions of LLM that are lower precision and higher precision, quantized or non-quantized versions of LLM. We've been supporting LLM 8-bit since we launched. And today we're introducing a full precision LLM in our catalog. So it's a higher precision LLM that will give you basically better results for the kind of questions you ask the LLM. And on top of that, we're also doing two things. One is we increased the context length and the sequence length of the LLM models that we support. So for those not familiar with these terms, context length is basically the size of the input sent to the LLM. And the sequence length is basically the maximum size of the output. In a nutshell, you'll be allowed to ask bigger questions and you can get longer answers from your questions from the LLM models that we support. And that's a good experience, right? Because you can interact with more detail if you want. It opens doors to more interesting, more complex applications. Let's put it this way. And then the last thing we shipped and announced is a streaming support. So even though we run LLMs pretty fast in our network, LLMs are still very expensive from a compute standpoint. And depending on the question you ask, you can actually need to wait a few seconds for an answer. And the way to make that... So waiting a few seconds for a response in an application can be bad developer experience, bad user experience, depending on the situation, but it can easily be bad user experience. And the way to solve that is to support streaming. So what happens is if you use streaming, you start getting results from the moment zero. And that's why you see that typewriter effect on things like chat GPT. They're basically using streaming behind the scenes. It will still take a few seconds to get the response, but you'll get a much better user experience because you start seeing the results the moment you ask the question. So this was basically a technical improvement that will give you a much better user experience, especially in the case where you ask complex questions that take a while to respond. That's really interesting. And it shows how these models work in terms of user experience, because when people get great results from these LLMs, but if they're waiting, they're not seeing nothing there, it's a worse experience. So people are now more... They want more from the application, in a sense. Yeah. We're also playing with other LLMs and there will be announcements in the near future. So expect our catalog to grow over the next few weeks. There are a few surprises coming. Let's wait for that. There's a few other two blog posts just to mention that came out this week, not related to workers AI, but other topics in this case, introducing hostname and ASN lists to simplify WAF rules creation. So it's all about the expanding of custom lists by enabling every user to create lists of hostnames and ASNs. Any comment on this one? Well, my comment would be the WAF is one of our most important products. It's one of our most complete products. And I think the challenge with Cloudflare is how you make such an important and complex product easy to use for our customers. And as everyone knows, we try to make that experience for all of our products a good one using the dashboard. And so what I think the WAF team is doing is just adding new features to the WAF that allow our customers to use the platform efficiently without having to go. So in this case, this is basically an improvement over a feature that they already had, which was using IP lists for their organizations, which could be a burden if you had lots of IPs and lots of prefixes. So what you can do now, besides using a list of IPs, you can actually use ASNs and hostnames in those lists, making maintenance of those lists much easier than they used to be. That's helpful. And the other one is introducing advanced session audit capabilities in Cloudflare One. For those who don't know, Cloudflare One has a bunch of products, really, right? Cloudflare Zero Trust is there, but there's more, right? Yeah. So auditing is really important in the software world. If you're an IT manager or if you manage anything at all, especially big organizations or big teams, having visibility about what's going on is really important, not only in the moments where you have to troubleshoot something, but also on the day-to-day to understand what's happening. And so everything we can do to increase that visibility is really important to our customers. And I think that's what's happening with this new feature of Zero Trust. It makes things much more visible in terms of what customers are doing with their Zero Trust configurations. Exactly. It gives more granular controls while maintaining an improved ability to troubleshoot and diagnose. So there's that. Before we go, I'm curious, the ZX Spectrum from the 80s is doing 40 years this year, and actually 40 years in Europe and the US. In the UK, it's actually 41, because it's from 1982. Do you have any stories related to ZX Spectrum? Well, not only I have a few stories, but I also have a few ZX Spectrums at home, and I think they're all working still. I only have one. My first computer, to be honest. You're kind of an amateur compared to me. I am completely. Yeah, but ZX Spectrum was my second computer. First one was actually the ZX81, also by Sinclair. I mean, everyone knows about the story of ZX Spectrum. There's still a huge community of ZX Spectrum users out there. Not just the nostalgia community, but even young people exploring retro computing in general. There are still people doing games for the ZX Spectrum. I've had a ZX Spectrum 48k for a few years. It was a lot of fun. I did a lot of hacking with that. It was actually one of the computers that taught me about programming, so it was really important for me. Good memories. Then I got a Commodore 64 next, but the ZX Spectrum was really, really the computer of a generation, and it was all over the place. You could see, I don't know if you remember, Jerome, but there was actually a TV show where you had people competing on TV. I don't remember that. It was really popular at the time. There's actually a ZX Spectrum museum in Portugal too. I'm guessing in a lot of other countries as well. It's still out there, not just the story, but the actual active communities celebrating this computer that initiated a lot of people in computers back then. I remember eight years, something like that, and playing Paperboy, the game, being really frustrated when that sound that you hear from the cassette tape didn't load the game correctly, because you had to wait a long time. I also remember my mother doing a curse about ZX Spectrum and doing a Christmas tree, and the code was blinking. The Christmas tree should blink, the lights blink, and that was, whoa, it's blinking. It brings some memories. I remember Manic Madison, Jet Set Wheelie, that was really fun. Chuckie Egg was really, really popular, I remember that. I still play Chuckie Egg sometimes. The Ares series, Ares Goes to Ski, Ares and the Spiders, those were really fun. Wasn't Street Fighter a game for that too? I don't remember if it was ZX Spectrum or other platform there, but I think so. Oh well, good memories for sure. Thank you so much, Celso, hope you enjoyed it. I did. Again, thanks for the invitation and have a good show. And that's a wrap. We end it for this week with our CDO, John Graham-Cumming, answering a few questions that were sent by our audience. Here's Ask the CDO. Someone asked about AI. How different is this AI moment we're living in now compared with the AI and machine learning you witnessed a few decades ago, and how can it impact the Internet in different aspects? Well, so given my age, I remember the AI winter. So when there was this great outpouring of research, money went into it, funnily enough, often using functional programming languages, which I love. There was this idea that the AI expert systems were going to be this incredible moment, and it's sort of in the late 1970s. And of course, that all died out because those systems reached a limit. And it was clear that they weren't going beyond that limit anytime soon. And of course, what we've seen recently is a real explosion in AI, in particular because of transformers. And so we're seeing things like chat GPT, we're seeing things like stable diffusion and DALI and all these kinds of things. And I think that the real question to ask is, does that actually reach a limit in the way that happened with the AI winter, or are we into a bright new horizon? My view is actually that we're not hitting the limit. We might be hitting the limit in particular in the models we have today, but this has become a huge area of research. And so if you look at what was happening in the 1970s with AI, it was still a relatively small group of people doing the research. But now we see, partly because of the Internet, partly because of open source, which has made a big difference, seeing a tremendous amount of work going into AI. So I suspect there are new innovations to come. And I suspect that we are really living in a different AI world than we were, than the situation we had before the AI winter occurred. This is an eternal question that comes up. Do you think Cloudflare will ever delve into affordable direct server hosting, like VPSs, dedicated servers, et cetera? Well, never say never, because you know what? Maybe. But I think that the way Cloudflare thinks about ourselves is as a connectivity cloud. We are the way to connect things together. So it could be people to their business. It could be computers talking to computers. It could be networks talking to networks. But fundamentally, it's about connectivity. And it's not really about storage. And it's not really about compute on dedicated machines and trying to be something like EC2, for example. However, never say never. Lots of things can change over time. We never had a storage service until we introduced Work as KV. We introduced Cloudflare Images. We introduced Cloudflare R2. That made sense for us. Maybe there's a day where containers made sense for us. Not sure yet, but maybe. We'll see.