Build low-latency pub/sub applications with MoQ

My name is Renan, and we're going to talk about MLQ. But before we start, I'm very curious how many people here have heard of MLQ? I hear great one. Oh, a few. Okay. How many of you have used MLQ? Great. Because I see some familiar faces that are involved in the specification and all these like deep topics. And I just want to say this is not a talk for you. We can talk after about the deep things, but this is about introducing a lot more other people to MLQ possibilities, different use cases, and how they work. So let's start with introducing the problem space that MLQ operates in, which is the real time problem space. Everything under a second, maybe there's a fan out component, but it could be live streaming, chats, webinars, clubhouse, if you remember that during the pandemic. Low latency information, stock tickers, betting, game score distribution, sharing, all these things are the problem space that we're working on when we say real time, low latency applications. And the landscape is dominant of tons of protocols. I arranged some of them from higher latency to lower latency. And as you can see, some of them deal with media distribution, and some of them deal with maybe custom protocols that you might have built because you work on the back end and the front end at the same time. It could be an app, it could be hardware. But there's a lot of protocols for different jobs to be done in the space. There's most notably, obviously, HTTP. You can pull new information if you don't care about latency that much, or if you don't care about latency distribution across different clients. You can also use something like WebRTC, where you care about this a lot, where if you, for example, are building a video conferencing application, your video conferencing application obviously needs to be below a certain latency so that a person can speak, the other person can hear, reply, how Zoom and Google Meet works. We need to use the right technology for the different problems. And there's a lot of, as I said, different technologies. I ordered them again. And each of them have different problems. Obviously, TCP-based protocols like WebSockets have head-of -line blocking. WebRTC is peer-to-peer, so you need something fan-out if you're doing fan-out, like live score distribution. Lack of signaling protocols. Some of them don't have standardized signaling protocols. When WebRTC was being standardized, for example, the problem that is being solved was so large, the video conferencing and the web problem, that people couldn't agree on a signaling protocol on what a phone on the Internet looks like. So now we have tons of applications like Zoom. We have applications like Google Meet and all these different walled gardens that have their own thing, but they all use WebRTC in the backend. And they don't interoperate. There's no reason why you can't join, from a technology perspective, a Google Meet from a Microsoft Teams meeting. And this should technically be possible if these were interoperable, but they're not. It's a signaling problem. Some of them are not supported in browsers. If you're writing a UDP-based custom protocol, the browsers can't do just arbitrary UDP. There's a lot more other problems that, depending on your use case, you need to choose the right protocol, and this is a hard job, especially when you're dealing with bad networks, bad last-mile latency. And the problem space we're dealing with is typically you optimize the backend or the frontend, but everything in between you need to optimize for as well. MLQ is another protocol that is meant to replace many or all of these use cases that is between TCP and UDP, but why are we inventing another protocol if we have many that already work? This is a good question, and there's good answers to this. The MLQ is getting a lot of attention and a lot of work, specifically from quick people and the IETF, but also many other people are putting work into it to make sure the protocol is worth it. To make sure that the gaps that we left in WebRTC, gaps that we left in WebSockets, gaps that we left in other protocols from the past, even the browser protocols, the protocols from the phone world, the protocols from the TV world, broadcasting world. We're working hard in the IETF, as Koffler, but also many other organizations, to make sure MLQ is worth it. It's a protocol that people can use. It's being developed, as I said, in the IETF. It's not a Cloudflare proprietary protocol. And MLQ stands for Media Over Quick, but it's not just for media, it's for arbitrary data. If you can carry any binary data, you can carry it, and it's PubSub. How does a PubSub model work? Each publisher connects to a relay, and then each subscriber connects to another relay, and then a connection is made so that, unlike the request-response model of HTTP, for example, there's a flow that's established, a low-latency flow that's established, that makes data from publisher to subscriber go. If you have a video conferencing application, you have maybe five publishers and five subscribers, and they all send data to each other through this system. If you have a video broadcasting live TV application, you have one-to-many fanout. Maybe you have one-to-one voice AI kind of situation. MLQ model works for all, where there's a publisher and a subscriber, and these are generic enough, and there's no negotiation, like WebRTC, that's needed between every single peer. You ask for a named track, and you get what that track consists of over time. There's a lot of technical benefits to other protocols to the previous state-of -the-art of MLQ. Number one that is most interesting to me is tunable latency. You can go between TCP's reliable communication and UDP's unreliable communication and have something that is tunable. A good example of this is, for example, say you're writing a podcasting application. It has a video conferencing component to it, it's a podcast application, but you also need to record this. You might have two subscribers to a publisher, one of them could be the human participant who's in the conversation, where if there's a little bit of data lost, maybe a quarter of a second, it's no big deal, let's skip over it. But you might have another subscriber that's doing the job of a recorder, that might say, okay, let's be more reliable, we have more room to, since we're recording, to recover the lost content from the wire. And you can have different use cases at the same time, running at the same time. You can't do this with WebRTC, because WebRTC will, if it can't keep up with the data, drop the data immediately. Same with WebSockets, you can't do this because WebSockets won't allow you to have the same latency guarantees that MLQ or WebRTC could provide for video conferencing. This is why you don't have video conferencing over WebSockets. It has native fanout for broadcasting, obviously, and the fanout system doesn't care about the contents. There's no head of line blocking, it's over quick streams. There's connection migration, again, through QUIC, there's also security, again, through QUIC, you know, TLS 1.3 is by default enabled for every QUIC session, so a lot of benefits from QUIC. Fast connection establishment is another one that is kind of important in MLQ. There's also a bunch of business benefits, you can do many cool things that you aren't able to do today, that now you're able to do with MLQ. So, getting into the more technical nitty -gritty, what do I need to know, what is the main data models, what's the API that I play with, what is MLQ? MLQ is a layer on top of QUIC, so in QUIC, you have the QUIC streams, you can set reliable, unreliable, et cetera, and there's a data stream. MLQ makes four new concepts available to you, and again, this is an ITF standard, so everyone else who talks MLQ understands these things and understands what the roles these things play. There's a track, which is a named stream of data that goes, and this could be video, this could be audio, this could be data stream, whatever it is, it's binary stream, and you can subscribe to a track. In a track, there's a group, and this is something that is independently decodable. For example, in video, it could be a keyframe interval, where no data, if you just jumped in between, makes sense, unless you have the beginning on the whole chunk together. Or, it could be like maybe a chat blob, where a certain part of the chat message doesn't make sense if you don't have the entire thing. So, it's independently managed group. A subgroup is a QUIC stream, and these you can prioritize. I'll give you an example in a second about how you can play with subgroups, and an object is an immutable payload. Everyone, and this is what powers the scaling of it, there's caching that the MLQ relays can do around an object. So, when you do your MLQ applications, you basically place what you decide in your application, like if it's a media application, if it's a chat application, whatever it is, where your data goes and where it falls into this, and MLQ takes care of the reliability properties. Koffler runs MLQ today in all these locations. Every Koffler location is a MLQ relay. What is a relay? Relays are for scaling MLQ, so you might have communication between two different server and two different machines, and you can do that over MLQ. But the benefit of MLQ comes from the fact that you can have something between, like CDN, that can scale. So, what you do is a publish and a subscriber each connect to their nearest relay, and then one of them publishes, one of them subscribes, and Koffler within the Koffler network makes the connection between these two so that the data flows reliably between each other. And the good thing is that once, say, for example, you make a connection from Sydney to London, that connection can be reused so that there doesn't need to be another connection happening for the scalability properties to work. Because object IDs are immutable, Koffler can reuse the object IDs and cache them, as it were, and have these data flows do efficient fanout. Internally, in our implementation of our relay, we use verbal objects to keep track of which server owns which data flow, which clients. Tunable latency is just an example of tunable latency, where using subgroups, you can have different versions, for example, 360, 720, 1080p, and depending on the congestion, that can be fastly dropped or brought back up very easily within the same connection. You're not doing another HTTP request. You're not doing another connection establishment. There's no other thing. It's in the same connection that exists. It can be dropped by the server and then brought back up and then dropped down again, depending on the fluctuation of the Internet. Another example is voice AI. A cool thing you can do in voice AI is, say you have a user speech-to-text model, the server does maybe inference, and then text-to-speech model. You have different reliability profiles, but in the same connection, you can have high reliability. Your voice needs to be very clear to AI for it to understand everything you're trying to say, but then the AI talks way faster to you because the tokens can flow way faster to you because AI, then you can hear. What you can do is you can, for example, make the first half a second of the data flowing back to you less reliable because you need to listen to that immediately, but you can have the rest of the 30-second message, 28 seconds of the 30-second message, have retransmission properties so that the message that comes in can be recovered while you listen to the first part of the message. This is not something that can be done by any other protocol, and it is interesting, creative use case of the tunable, even you can tune it during a message being sent, tunable reliability affordances that MLKey gives. You can do PubSub for data, match scores. You can align it however you want depending on what you want to see. There are certain things that are today, if you want to use MLKey, are not a good fit. For example, if you're doing pure cacheable file downloads, MLKey is not a good fit yet. There's a way to request the response, different objects in MLKey. It's called fetch. You can use it to fetch history, for example, in your stream, but this is not a crazy new thing that MLKey enables. It's like something that you can do with HTTP, and it's not differentiated that much. If you're using WebRTC's built -in echo cancellation, no expression features, bandwidth estimation features that you don't want to have control into at all, you can use WebRTC for that. Also, if you do DRM, there's no good browser -based DRM story like encrypted media extensions for MLKey, but these are all problems that are being solved with the ITF and other groups right now. Browser support. There is browser support on every major browser, including Safari iOS. You can use it today. MLKey is possible everywhere right now. We maintain MockRS, which also powers our server-side implementation for our relay, written in Rust. There's many libraries that we also support and you can play with. MOQ is not yet a ITF RFC yet, so there's different drafts that are ongoing. Currently, the current draft is, I think, draft 2022. Kafla supports draft 18, and we're upgrading our support as we go. But you can play with it, and soon, hopefully, it's going to be an RFC. If you control your front end and the back end, there's no reason to play with MLKey right now. MLKey is a serious business. There are cases where QUIC or UDP is not supported in certain networks. There's a solution to this. There's a standardized mapping of QUIC into TCP, TLS, WebSockets, H2, etc., called QMUX. That is the way to deal with older clients, clients that don't support these networking primitives like QUIC or networks and all these issues. QUIC is trying to be compatible with everything. MLKey is trying to be compatible with everything. In cases where it's not supported, there's an option. With Kafla's Relay, there's docs about them. Getting started is easy. Point your agent to the MLKey Alliance Getting Started page. Speaking of MLKey Alliance, there's a lot of companies working with MLKey. As I said, this is not a Cloudflare technology. Cisco, Intel, Amazon, all these people are going to support MLKey or have already done something with it. There was an MLKey-specific conference in San Francisco two weeks ago. The videos of that will be out soon, including Meta's use of MLKey on Instagram. If you go live on Instagram today, you will be able to use MLKey today. Their live-streaming pipeline on the app to the server is currently implemented with MLKey. Protocol development is at ATF. You can contribute. This is a secret email to all the engineers working on MLKey at Cloudflare if you want to get more information.