Hacker Time

Hello and welcome to Hacker Time, the number one show on Cloudflare TV, number one security show anywhere in the world, and I am here today to do some live programming with you. Normally we talk about security, we've done some live programming, we've played chess, we've done all sorts of things and today I am here to talk about and to build some software with you and talk about the software we're building with Cloudflare Workers and KV and I find the product really compelling and I hope you will too, especially after watching me program with it for about 20 minutes. I wanted to first start off with this web service that I built about a year ago called csprng.xyz and it is the craziest name ever, cryptographically secure pseudo random number generator dot xyz so csprng.xyz and it runs completely on workers and it was just a toy project that I built and it just returns randomness so you can see for yourself right here and I'll pipe this to jq and you run it every time you'll get different random bytes back and this is a really simple api just about 20-30 lines of code with workers and when I checked the analytics on this the Cloudflare analytics on this recently and it's getting 330,000 requests per day which is unbelievable to me it's it really I guess some people have used the api and some software somewhere and it's being hit from all over the world, United States, China, Brazil, Europe, it's the requests are truly coming from everywhere and over the last month 10 million requests which is pretty amazing. I'm not sure what it's being used for but what's really interesting is the website has really scaled not many other services you could just write 40 lines of code deploy it and have it handle 10 million requests a month and not even know that you're handling that amount of traffic, hit no production issues and just let it run so it's a pretty good case study of just workers being really compelling and scalable and today we're going to do some programming and I am going to build the first rebuild the very first web service that I ever built the very first thing that I ever built was a URL shortener and I was in school I was in the university at the university I went to and I was looking at bitly and I was like wow this is uh this seems so simple and it's uh they're a real business and they have something really cool why can't I build that and so I worked on a URL shortener I wrote it in PHP and I'll tell you about some of the design decisions I made throughout but and why it was so bad but I'm getting started here I'm gonna I'm gonna start with uh I'm using wrangler to build and deploy my worker uh and I'm gonna start with wrangler uh this is the second time I've used wrangler so uh bear with me while I read the docs in rtfm but we've got wrangler generate looks like generates a new workers project and we're gonna want to go fast so I might not spend a ton of time um explaining every little thing that I'm doing but okay so we've got a bunch of files in here wrangler.toml is the one that we want um let's take a look at this so account id I think I need to set up I'm using my Cloudflare account my personal Cloudflare account here that I have a bunch of websites in and we're going to use today ejcx.net um and we've got our account id and zone id right here so account id put that thing in there zone id put that thing in there and let's just try to get a hello world up and running so what is route wrangler.toml let's check out what route is route a list of routes that you'd like to use for your worker okay so uh that's very easy uh that's a that's a straightforward one so we're gonna do route https ejcx.net slash star we want our we want this worker to run everywhere um and then anything else um we have an index great let's get let's get this hello world worker running and see how it goes wrangler wrangler is it deploy build publish and then down here we'll curl ejcx.net slash asdf hello worker okay we've got a hello world worker running um and we want to now um what's next okay we want to get some routing set up basic routing so I have a router here that I built called the worker router it's very hacky I do not recommend using this in production it's about 30 lines of code and it's just a for loop really uh but the nice thing about it is it has some express like syntax that I'm familiar with so I'm going to use this thing uh I believe we have an official router but I'm not familiar with it and since I'm and since we're racing the clock here I'm going to uh opt to use something I'm more comfortable with okay npm install worker router we've got it now let me uh let me just copy and paste this thing this is my hello world worker here and it's not going to work unfortunately um because we are going to have to make wrangler work with um with our webpack that we want um great and if we do wrangler publish should just break require is not defined that doesn't look good something went wrong um um uncut reference error require is not defined why would require not be defined this is my hello world worker it should be working um I'll give myself 120 seconds to debug and if not I'm going to backtrack and uh and build and punt on routing this week and we'll uh we'll just do it uh it with request parameters but um let me start here webpack config um type we want webpack here uh so in the um in the worker toml you can specify that the type is webpack instead of javascript so um it will know that you should be webpacking things um I'm not sure why it doesn't see understand require there's another one though is it include javascript require oh import yes see how much programming I do these days um this should also break not quite work working I haven't installed a webpack I don't believe okay we're installing this wrangler jsnpm module and if we hit an error I'm backtracking I'm going backtrack and and uh and punt on webpack for the day and go uh full uh pure yeah yeah okay this looks like we got an error um cannot use new with import okay I'm going to punt on using uh using any of this today uh any of these dependencies I just got node installed maybe 20 minutes ago so um might not be the best thing today so I'm going to backtrack that is what programming is all about trial and error so we're going to go back to type javascript and we are going to worker hello world um let's try this playground okay perfect so what we want to do we want to add two routes we want to add a route for create and we want to add a route for um doing the url shortening so we will first um specify those here so how do you do a any routing Cloudflare Workers routing and you'll see how the routing works using this request object uh scripts how does the routing these days work all right here we go so so we actually don't even need a route we just need post and get um and then params so let's start here so if it is a get request get rid of auto indentation and then so if it is a post request we will want to return post let's just do return there's a post else we will do um return new response get since we made some since we did some open heart surgery here let's uh let's double check to make sure everything is working wrangler publish all right 18 minutes the uh the very first url shortener i built was in php all right x post what's that why did that break response that's why it broke this says response properly this is reponse that is not how you spell response all right this one will work uh we had an error down here as you'll see and now it works okay and if it's patch what happens based on the code we wrote if we send a patch it says get and uh it's really just post and otherwise we actually don't even need this so handle post post handle attempt to shorten okay so if it is a post we want to receive arguments store them in workers kb so let us do that um let me get rid of that player workers post arguments read post there's tons of helpful developer docs here tons of code that's already written to do the things that you want to do which is just fantastic um i do a lot of copying and pasting in my programming career and uh that's that's the way i like it um okay so we want to check for arguments how does this work read request body here we go uh this seems more complicated than it needs to be i think you can just do uh body.json i believe you can do let's try that uh request.body.json let b equals for body and then let us return the body will this actually work no it will not i i need to json stringify it before we return it okay so what should happen is we send json we get json back let's give it a go curl okay we want to send a post we actually don't need to specify that it's a post let me uh make this easier to read we can just send it a request with some json in it so first json can be tricky i i struggle with json because sometimes you need the right quotes and everything asdf colon one let's try this does jq like it jq does like it great that means for a good situation curl https djcx.net give it dash d dash d is data so it'll send us this data which curl will in infer is a post request so and we hit in error okay let's first just do this return new response stf wrangler publish there is a way to do dot json await request dot json okay so we can just do we don't need body it's just request dot json do another publish uh powered by coffee this morning all right look at that it works it works uh and now we just need to do json dot stringify the body and we should get back the data we put in so whatever data we send in this body we should get back in the request um there we go look at it works okay so now we can make a full api out of this we have plenty so we have our body what are the fields we want probably a url in a shortened term so uh the little short thing and uh uh maybe just the url how about let's just do a url you receive a url we generate some randomness and return um a shortened uh alias of that url so uh first we need to check if the body has a url field if body dot url this should work actually i prefer to program like this then we are going to return a new response invalid request send json containing url i believe we can do return a 400 like this let's give it a go so and we got an error why is that i believe we can return an error like that oh and it is all of these other things okay so response body is a little more complicated than i thought it was we actually want um um player workers so we want to give a good response to people who don't know what they're doing when they're curling our api so init and options that contains custom whatever valid options include status that's what we want okay so we want to return an object that says status 400 and i think this one will work right invalid request great we got our error message let's double check the status code and caps lock oh i'm not my keyboard is just going crazy um we got a 400 right here oh perfect okay so now this assumes that we do have a url so let's get the url and stuff it in workers kb so um we just want to store the url kb and uh ejcx net that's going to be my kb alias and player workers kb we need to look up how to do this with wrangler and um we uh how to how to mount our kb namespace to our worker using wrangler and then we need to uh just do a put key value great um await namespace put key value so this is going to be our namespace ejcx net uh dot put the key is going to be asdf placeholder and then value is going to be b.url our body dot url and then uh okay perfect then return new response smiley face it works return of 200. okay now how do we do this with wrangler wrangler.toml route uh route no worker kb so i hope this i hope uh me copying and pasting things is really um i hope a lot of you can relate to that way of programming um okay we they have a bunch of commands in wrangler for kb namespaces i did this one other time the other time i used wrangler in my life and it was a very positive experience so let's go wrangler kb namespace i have a lot of them create and we will call it ejcx net uh add this to my configuration file look how easy that was holy cow shout out to the to the uh folks who are working hard on wrangler that was pretty dope that it just worked all right so um now i think when i put this it should store the url and then here i should be able to get it wait get it should store anything that i want here um so the very first um url shortener i i built that i was telling you at the beginning shortened url yep that matches hit it in php and i didn't know i didn't have a database and i had no storage system so i used files on the file system and this was before i was really uh into application security and understood it well um so let's do a curl here and uh let's just say that that software um could have very easily been hacked that software that i wrote because the way i built it i would rewrite a php file i was doing one of those directory structure php programs where you have like uh back with apache where you have a like um i used to do my routing with like slash accounts and then in slash accounts in my www directory i'd put index.php and apache would default to loading index.php and you'd have a nice looking route that says slash accounts um because i never figured out routing so um you could it was rewriting a php file putting it in this directory structure and you could very easily just like put a single quotation mark or single quote and mess up the php direct php structure of the program and insert arbitrary commands um i know that today but i did not know that at the time okay now when we curl this and do a get request we should get back our asdf curl arguments and it broke why did it break cd worker so we're putting here the puts might work curl dash d url z okay we got our happy face look at that and then with git we should be getting a shortened url okay good is there anything i'm missing here with git i've used this before it's very simple oh duh this isn't defined here that's not how a kb works all right we've got four minutes left i'm actually pretty happy with the progress we made today we've got if we can get the uh this one should work now look at that we got our z's back okay so the last thing we would need to do to make this a fully operational um uh we have storage working so we can store a little randomly generated string and a uh in a url that gets submitted but there's a few things that we'd want to add and i'll flush that out here first we'd want to add url validation validate url so we want to make sure that whatever we're storing it's actually a url make sure we are actually storing a url and not just a url like http because uh i'll uh etsy password is also a url technically okay so that's one thing we'll want to do next time and then here handle attempt to shorten um we're going to want to add error checking here uh error checking and handling a case where we try to pull out a url that is invalid error checking and handling an invalid uh shortened shortened string so if you try to query for a shortened url that that hasn't been shortened um should should respond with um with something that makes sense so like a uh 400 or we'll have to check which error static which error code makes sense 400 doesn't make sense um and then lastly we need to before we do this put we also need to generate random shortened string so we can do this with base 64 and then stripping out special characters that might be an easy way or hexadecimal which is not the prettiest in a url and it's uh it could be shorter but um generate random shortened string what's the easiest way to do that instead could we allow it to um we could also make it user supplied but then there's a security aspect of it of if i send a user supplied um string and you send the same one we don't have authentication and authorization in this api to really enforce who uh who actually owns and can modify that shortened url so we'll need some so it's best not to allow users to specify their uh their url that's their shortened url uh we'll probably want to just um randomly generate that um and then what else do we need to do um besides this i would like to switch to a more routing based uh approach uh we need uh routing it should come from the path uh the key should come off the url path so routing url key should be specified specified that does not look right for some reason i'm not sure if it's right or not should be specified in uh the url path great all right well we have 48 seconds left and i really appreciate you joining me and hacking away at this we made pretty good progress for a half hour we got wrangler set up uh we copied and pasted a bunch of worker's code and we got things deployed um this is all just javascript and really easy to write i find it very very nice programming experience and it's really scalable so i hope you give it a try and i hope you don't make the same mistake i made which was uh uh with php just writing writing arbitrary php to a directory on your machine that's a great way to get hacked and so thank you for joining me on hacker time and i will see you next week where we'll continue working on this and finish it up cheers adios