Episode #5: Rapid Side Projects with Cloudflare Workers and Workers Sites

Hello, hello, welcome, welcome to another episode of RapidSight with Cloudflare Workers on Cloudflare TV, the Internet and the world's hottest new channel. I'm your host, Trevor Noah. I mean, Steve Pack. Don't know why I always make that mistake. Must be something about, you know, both of us having rapid rises to fame in broadcasting. Thanks for coming back. As I said, this is episode five, where you watch me hack around with what we're here watching a lot, so probably better described as flailing around wildly, thinking I should prepare for these segments rather than just to worry. That's the spirit of Cloudflare TV, if you haven't figured that out yet. Okay, a little disclaimer. This is not official for workers or anything, actually. Look at the docs, contact support, contact your account team. This is a dude who spent some time on his own hacking around on occasion and wanting to share some of the lessons that I've learned just to get you up and running quickly. That's the spirit to take it in. I'll be accepting questions today. You can email livestudio at Cloudflare.tv. I'm going to share my, if I'm not already, actually I am, so you probably can already see here what we did previously with a worker's site. Within 20 minutes or so, we got a domain registered onto Cloudflare, a website up, and even a basic Vue.js site doing byte conversion, which is, there you go, all running locally. But then we wanted to get a bit more ambitious. We wanted to do that and have an API as well, and so we decided to use a foreign currency exchange converter as the example. We looked at the template and how you might test your sort of handler code using Mocha. We continued with that. We did some local tests. We deployed the real site, and then we got an end-to-end thing happening last week. You could select a currency pair. That's locally. Let's bring up the real thing. We got an end-to-end connectivity going where you could actually request a rate, and you would get it, and started to look at where else you want to do. Maybe you want to cache results, for example, to not have every single request when this becomes the world's most popular, to not sort of flood the origin API server that we're using, and we started to add caching. We didn't quite finish our call, so let's take a look at how far we got and go from there. One thing I know where we left off was we were trying to add caching logic. One thing I was doing, and there was a lot of data coming out. I think you might recall this was what a sort of typical wrangler was. I did just check before we started what a nicer command was, and that's here. If you really just want only the messages and not all the other metadata, type it through jq, and if we went back up to that structure, you can see there's some higher -level metadata, but then there's this log section and stuff. Sometimes it's useful, but often we just want those log messages, and so there's the command for you there. Type it through the jq logs array of messages, and the dash c just puts it all out in one line, so instead of that, you see that. That's just a tip if you're wanting to use this wrangler tail while you're doing this. Okay, so where we better wrangle tail, yeah, the cache implementation, which is my index. So we were flailing wildly. Use the royal we when it's flailing wildly rather than churning out beautiful code. A couple of things we've looked at here. This method was under test, and we're sort of happy that it was working like the day, and that the cache, we hadn't worked out yet how to do dependency injection for the cache in the same way we've done for fetch, so we kept this alone and just focused on putting the cache implementation up here. I think I went a couple of minutes. So if there's a couple of new lines here, I'll just walk through them so we know we're up to. So basically, the request comes in. We try and match the request to something that's already in the cache. If we get something, very simply, we return it. If we get something, either because it's not there or it's expired, then we go and use the get rate method that we implemented, which goes off to the origin. We create the response, which is a JSON string of the body. This line might have been off air. So, you know, he's saying to the browser, you should keep the response for 15 minutes. So the max age header is in seconds. The browser will look on disk first, if it has it, and return it. If it doesn't, it'll hit the Cloudflare edge here to see if that request is still in cache or not. If it's there and if it's not stale, we'll either return the cache response or, and we do get a real one, then we want to store it, and then we want to put it in cache. So I think this was off air as well. I was sort of just format we needed to store into cache, and so it looked like responses was what we wanted to store. You'll note that I'm doing a clone here, and that's because if you put the response in the cache and then also return it to the client, it has to be two separate instances. So that's the clone. And then one other thing I learned, reading the docs, is because this command, you know, won't be instant, it'll be very fast, putting something in the Cloudflare cache, it's better if we can do that asynchronously. And so there's a sort of convenience method that comes, actually I don't know if this is the worker's runtime or if this is in the, just the like web worker, but you can actually call wait and then provide a promise in here. And so processing will continue, the client will get their response quickly, keep running until the cache output is done. So we're gonna, that's what we want to do to sort of round out our cache implementation. The last thing though is the sort of interface we've been using. You know, this is the very start of our request handling, and then we're actually passing in event .request. So we don't have event in scope anymore. So if I had more time, I'd go and everything to just pass an event. But just in the interest of trying to get a bit further today, let's say handle request, take request and event. And we are going to pass the contact, both the request and that top level event object down. So here we're going to have event. And so hopefully we'll get some better intelligence sense this time of what wait until actually looks like. I think it's just a promise, but let's check. Okay, so it is on the, I've just done web worker, API, wait until promise. Okay, so cache .put is going to return a promise. So here we're just saying, like, put this in the cache, don't terminate the worker until that operation is finished, but carry on, turn the response to the user. And that's, I think that's about it for our cache implementation. Check if that looks okay. npm run deploy. Fail. Ah, okay. We didn't need a separate deploy. It's a lesson to be consistent, but wrangle publish should build and publish. That's probably enough. I did say this was failing just before, so let's see what's going on. We'll also, I wonder if wrangle with tail will have handled the deployment and kept tailing. We'll see. Cores, no access control, but this is... Okay, I'm only allowing requests for over HTTPS. It's interesting. I thought it was only the domain name. I didn't know it was the scheme. Anyway, we should handle it in a separate episode. There is enough curlies in that it's its own episode to itself. Okay, so I didn't see any messages there. Let's see if that actually went... Okay, so that actually was already filled from disk. So obviously, cache response. All right, so here we got a empty cache. Trying to do from fresh. Let's say this was the first request. Cache. Cool. Okay, so we made a request to our API asking for this pair. And we saw that because it wasn't in cache, it went off and queried our origin API, got back a result, stored it in cache. So if we were to get that rate again, you can see we didn't even get another hit here. We already had it. So let's try if this was coming from someone else on the Internet, so different cache on a new browser brave. So we shouldn't be able to fill this request from hoping it's their edge. And we'll return that. And so it is. And so you can see here, like, you know, we sort of went from an API that was deployed and was going to, like, hammer the origin server to one that's like, you know, with just a few lines of code is, you know, caching the response, both on the page and in the browser. And, you know, again, this is deployed 200 days worldwide. So, you know, straight out of the bat, got a super highly available API just there. A bit of background noise here at the live Berkeley studios. Welcome to COVID-19 living. Let's see what else we wanted to do today. Okay. I'm going to go the whole, we're about halfway through. We're not going to do all of this, but I wanted to show you something like, it's a pretty common use case, right? That you're building some sort of admin interface. And that's something where I've found in the past, like it goes from being a simple thing to complex thing. You've got to add auth and authorization. Do I use a, do I start building my own authorization mechanism? Hopefully not. Cloudflare can help with that. An awesome product access, which during COVID-19 is currently free. I think it's till the end of September. So great time to test it out. What I'm going to show you is how easily put an admin interface, like on the web and, you know, protect it and be confident that it's only me or whoever is to accessing it, like cyclically. And so like theme of like rapid site projects. Okay. So this is access. Oh yeah. Free until September 1. So I even mentioned, see a VPN, right? A common way you might do this is you deploy a whole separate admin app and then have some VPN to it. Incredibly painful, like not a good experience. Let's see if we can, let's see if we can do better. All right. I'm going to create an access policy. This is the sort of basis. And this is going to be the converter admin site and the, not the Cloudflare website, that's running Zoom on my MacBook Air at the same time. Okay. So what we're going to say is when anyone.converter.10.cf, where I'm going to put my admin page, that they're going to need to pass this policy and that policy is called Steve only. And so we're going to say that only this email address is the only one that's going to be allowed to access that. I'm thinking, okay, let's actually look though at one I created earlier. So you can see here, like I tried to access and I couldn't, and that's because I'd set up an access policy so that when the request hits the Cloudflare edge, it's a request to see if there's a cookie that has a valid token in it, basically a JWT token, that gives me access to the site. And if it's not there, you get redirected, you know, this page, if you've set up an identity provider, you know, you can use that. If not, you can just send yourself. I'm going to enter this one-time code. Yes, I'm aware I'm giving away something here, but that's okay. You can access the test website of no time for books. And I'm right, like as simple as that, just putting a rule to say, this is the pattern. Like in this case, I have a separate domain, but you can do test.com forward slash admin, whatever it is. It's Cloudflare edge. We check only if you pass the test by validating with an IDP or one-time code, do we forward you on. And, you know, that's just a really rapid way to get up the sort of admin or a test version of your site that you don't want to share. I think there'll be a lot of Cloudflare TV more dedicated to access, but I just wanted to like take that little detour to show you, you know, writing quickly on these. So some projects and you just hit this thing. It's like, ah, man, I need this admin page. For example, on the, this FX converter, I have a page where I like add all of the currency pairs that I want to support. For example, you know, that's the sort of way I would access for that. Okay. I've got about eight minutes left. And this may be the last episode. And I think to leave with this as the final UI. So although not strictly workers, I thought it would be at least try and make it look a little nicer. So let's at least add some style. This is not my strong suit, but anyway. I tend to go to Bootstrap. In my previous projects, I used Bootstrap Vue as well, which is, you know, integrated with Vue.js. I do remember it sort of forces you down a certain path. You have to like use components that aren't sort of vanilla. Like they have these like be like this sort of style. I don't know that, but it just doesn't, it just sort of feels like my sort of, I don't know, maybe my display logic and my app logic is starting to get mixed. I don't know. It's actually worked well for me. I wanted to try out just going back to vanilla and seeing if that was actually sort of as easy or easier. So we could just link to Vue.js to the other files. Surely there's a question here. I thought there was a question here. Install Bootstrap. Do that actually. Well, I said I was going to do this to sort of simplify things. So let's actually start with the easiest of easy. And we'll see if that needs Acro. Okay. So that's going to be on that top page. Base link before head for all other stylesheets. Okay. What if someone else says that though? Who do I trust? And then to see if we've got something working. Let's actually put something on the page. And I love my Jumbotrons. Remember that period where every web page on the Internet all looked exactly the same because it was Vue with a Jumbotron. Well, I've probably been part of that trend. Okay. So let's go like right up here. Jumbotron. And let's sort of think of this. What could this site be? Steve's tools. That would be something that we'd start. Add all of the tools that we build and that we share with the world on this very popular, very famous channel, Cloudflow TV. Okay. So it looks like I've got something. And that doesn't look right. I wonder if there's a, like an inner, you know, sort of div I'm meant to be inside here. I could do that inside. Anyway. So how long we got? I don't know if we're gonna get to our beautiful bootstrap in three minutes. So let's try to do one thing. I don't like how the rate is there on the side and is just sort of gray. That's a bit, it's a bit sort of, oh yeah, that's gonna be an amazing UI. So, all right, if we're gonna, if we, I'm gonna start wrapping this up. If we want to do a full nice site, then we'll do another episode. You'll have to look at the to see if that do that. We're on episode five. You know, what we've done is we've deployed, you know, this initial Vue .js to the Internet with a real domain. It was registered for free. We got it on Cloudflare. We then decided we wanted to add a new tool, a foreign exchange tool that would have an API that would actually fulfill them. We then added caching to make sure that, you know, we didn't overwhelm that origin, some time getting like a local environment set up. We got to a certain point. We didn't, we didn't do everything locally. We still had some other things we wanted to walk out. And then we looked at how to add an admin interface today. Didn't finish that, but we're able to see sort of the, you know, the basics. And then we're just starting now to look at putting a UI theming library and, you know, do something a bit nicer here in a UI, which I'm going to get to. So that might series happy to take requests, anything you'd like to see side projects related or anything else on Cloudflare. Happy to hear them. It's been a pleasure hacking, flailing wildly, even occasionally coding and testing and deploying with you around projects with Cloudflare Workers and worker sites. So when the server crashed and all hell was breaking loose, I woke up in the morning and said, Hey, what the hell's going on here? So I will begin from the top. So tell us about the COVID Symptom Study App. The COVID Symptom Study App is an easy to use app that people download and they self -report their symptoms every day, whether they're sick or not. It allows us to predict who is likely to have the virus, which areas of the country are going to get most affected. And it also allows us to predict which people are going to get sickest and need the most urgent care. It's been an amazing success story, really. The app went viral and within 24 hours, we had about a million downloads and we now have over 3.3 million users around the world. Julian, you must've been excited about the success of the app. How did you handle so many people using it at once? At 7.55 in the UK, which is still pretty early, the database crashed. Most of the team is actually still sleeping. And at that point, we're like, what's going on? How is it that we're crashing so early in the day? We launched the app really more in hope than anything else. And then it just took off. When the server crashed and surrounded by media, we said, hang on, this has kicked off. We can't go back now. How are you working with Cloudflare to secure the app? I already knew some people working at Cloudflare from previous experience and just told them, this is what's happening. We're having this great success. We could really benefit from not only your expertise, but your services. Can you do something for us? And by the end of the day, we were upgraded and part of the Galileo project. We're using it mostly for managing DNS, proxying traffic, but a lot of things at small scales, managing SSL certificates. We're also using some features to protect access to various parts of the website, basically the admin pages and making sure that people are authenticated coming from the company. Tim, what are you learning about this virus from all the data you're seeing? Data is power. Data is knowledge. And basically, there's been no data out there on what's happening to this virus in the population. We were able to use all the symptoms that people were giving us. And we've picked up through the app lots of symptoms that people weren't recording before. We were the first large group to pick out the loss of smell and taste was incredibly important. Pretty much every day we're finding out something new about this virus with this amazing data. How does it feel to be part of such a successful project? This project was never about having 1 million people. This project is about, you know, delivering value to the population and the research. So that's really where the satisfaction comes from. We just had the best team at the right place at the right moment. That's what I'm super proud about, that every one of them did their bit. And that's why it worked.