Hacker Time

All right and welcome to this week's Hacker Time. I'm your host Evan Johnson from Cloudflare's product security team and this week we're continuing where we left off on the number one security show anywhere but definitely on Cloudflare TV number one security show and we're going to pick up where we left off and keep working on this Cloudflare worker. We're building a URL shortener and I was opining some of my sage wisdom about building very vulnerable URL shorteners in the past which I'll try to keep sprinkling in there and this is a programming blitz so we better get started. We've got 29 minutes 12 seconds and I want this thing done. Let's ship an MVP and so I'm just going to hop right into it. First let us connect to our tmux session right where we left off and just like that we're going to be off to the races. So we're using Wrangler. Okay I'm not in caps. My thing's broken with my tmux config but that's okay. It's workable. We're using Wrangler and that is how we are shipping our software. We're running Wrangler publish to take our software, batch it up, package it up and run it on Cloudflare's edge as a Cloudflare worker. So let's take a look at the code and I'll walk you through where we're at really quickly. So it starts with the regular worker event listener. This is something that you'll see in just about every single worker and then all of our handling of what's actually happening is right here. So when a request comes in, if it's a post request, we are making sure that a url was submitted. Then we are going to add some light validation here and then we are saving that url as kind of a shortener. We're going to generate some randomness still and then we're going to save the url that was submitted in workers .kb and then return a nice 200 with a smiley face. Then when it's a get request, we're going to have to otherwise pull out the tag at the end of the url. Try to read that from workers.kb and then issue a redirect. So we've got the post logic, it's not perfect. We should try to get something working with just a single hard -coded url tag first and then get it with a randomly generated url tag so this thing can shorten multiple urls. Then we'll add some error checking at the end and I think that will get us to the end. That'll get us to mvp and then maybe we can spruce up some do some front-end engineering whip out some html and css. So without further ado I'm just gonna get started and try to so I think actually we from where we are today right now I think right now if we submit a url it should get saved under a let's give it a try so curl https ujcx.net and we want to send a post request we actually don't need dash x we could just send it because a post is assumed when you use the dash d flag in curl unless you specify a different http verb and we want to submit you uh url I believe let me just check uh yes url oh it's supposed to be json oh boy um okay let's start with this url and then https i'm gonna forward to Cloudflare.com all right and let's see if this actually works okay we got a smiley face and so it seems to have worked and let's see if when we curl do a get request we get back okay great we got um oh this is a little funky could not resolve host um we got oh I have the stray letter t here okay so yeah it returns our url we just submitted when uh when we issue a get request okay so end to end this thing's working uh the the non -working mvp it's not quite working but all of the the um although all of the state writing and um most of the important apis that were um that we need work and so first things first let's actually get this redirect thing working so Cloudflare worker redirect okay this should be pretty easy it's just 301 you just need to response dot redirect destination url and status code perfect okay one line change so return instead of a response with the text of the uh instead of a response with the text we're going to respond with a redirect with short url and status code let's just hard code in 301 and then we'll run wrangler publish and this time we should see a redirect okay yep here we go so um we actually yeah we'll uh we'll keep it like that so we're running dash v here in our curl our curl to do a verbose but we're piping standard output to dev null so uh when you do verbose all of the http headers are printed to the standard error and so we're still able to see them and you'll see that now this location says uh there's a location header here and so when i go to it in my browser it should actually redirect me yep all right we're in business okay so we got that that's dead code now because it'll never be reached because we have a return statement on line above it okay now what i'd like to do is generate a random string and i have some code that i was showing last week csprng.xyz which will do exactly that so i'm going to do some copypasta here which is how all great programs are written copying and pasting great artists steal and um we can um okay so this is a way in workers to generate some random bytes and i would really rather not have it be base64 because base64 you get all those special characters and i think base64 is uh i like hex because i don't like looking at all the like pluses and hyphens and stuff that's in base64 so uh i want to google how to do hex um in javascript javascript bytes to hex let's see how this works and so one of the real beauties of workers that i love is um all right this looks safe to uh nothing too crazy here uh one of the real things that i love about workers is you can really channel your inner php or and and php gets a bad rap uh because there's there's a there's a lot of unsafe things you can do in php and i'm not talking about that in the workers context the thing that i love about php is that you can just sit there and hack something together with copy paste and get something working uh really really easily you can sit there with vim open hack on your website and um and see something get built in real time and i really feel like that's the same way uh things working with Cloudflare Workers okay so i'm going to copy pasta my random bytes function this is some open heart surgery here if i've ever seen it which i have not i've never witnessed open heart surgery so and we are going to instead of returning buff here we're going to get rid of this oh this function was always horrible so uh i generate buff here i fill it then i do this conversion and then i don't even return the converted value so this line has just always done nothing in my little function um instead we are going to return this and we're going to replace byte array with buff and i bet i bet when i call i bet this time when i call um return new response and we call random bytes let's call it uh 16 bytes every time i'm going to publish this and what should happen is every time we go to ejcx.net it should print out random uh hexadecimal bytes so let's let's do it all right random bytes 16 bytes every time uh that looks good to me um but we probably don't want 16 of it for it's supposed to be a url shortener let us instead uh let's do eight this is a toy example so it seems reasonable uh so it's resistant enough to collisions for a toy example but um probably definitely not uh perfectly collision resistant pretty easy to find one uh actually in that case i'm just going to pick 16 let's uh pick security over uh usability here okay so we want to use this random bytes we want to use random bytes to generate a token url so when somebody sends us a url to store in workers kv and to shorten we want to generate that random string uh what should i call this um url tag that seems reasonable and then we want to put this as url tag so we are using workers kv we're storing as the key this randomness and then the value as the url when we receive a request we're going to have um we're going to ask users to submit that url tag that we return uh when the url is saved and then uh and then issue the redirect if that uh key exists so um we now have we now instead of returning the smiley face unfortunately we have to return the url tag we can do like uh i don't know we can return a fancy string thing here and say uh just like uh shortened url and then we can make it here uh short url equals https jcx.net plus url tag okay here we go and then we want to do check this out this is going to be beautiful so we want to have a quote quote this is what happens when i don't have sprint f in a language i do horrible things like this okay so this should technically return valid json why am i doing this it should be just as easy to do um to do something like this um so json response this should be just as easy i think this will work and then we can just return json by json response response object how's that okay okay so let's publish this it might break i have a feeling it could break on this response object because i haven't written enough javascript recently okay we want to curl url Cloudflare.com okay look at that and it gave us a shortened url okay so if we go here it might technically work but that's not because it's actually working so uh we're just forwarding to the same url as we just submitted so uh we need to actually read this this uh random string so it gave us let me uh let me depict this a little better so when you send the request we're sending a post with url uh we want to forward to Cloudflare.com and it's a shortened url ejcx.net with this nice tag but when we go here we're not actually reading this yet in our worker from the path and we need to do that so i am going to to figure out how to do that in routing Cloudflare workers routing so what we need is some pathing i would like some example code i know it's somewhere here um okay we're at our community site i think i was just on this website yeah um i wrote my own worker router in the past and it's a npm module you can pull it up and this thing seems to work okay i would definitely would not use it in production solid 11 weekly downloads um but uh i hacked that thing together and maybe i can check that code real quick do some more copy pasta so so so i did it using regexes and so we what i did was you can get the url.path name here okay let's get the path um first things first let's read the path from the request so we have a path here we have parsed the request url into a url object which uh actually takes all the parts of the url the scheme the path the um the port um whatever whatever is in that url and it will put everything where it belongs so we can just reference the path name and then it looks like you can just check if it matches something so um with this api i would assume any path that's well let's let's first return the path just to show this is working and then i'll speak out loud and and uh think out loud around what i'm thinking about here all right so this i don't know what have i done did i reference request which doesn't exist no request is this here does url already exist shouldn't be a problem but could be uh could be something funky happened return a new response i need to create a new response object uh here and now this should work and it should just print out the path so it just printed out a slash here let me do a dash b so you can see it on its own line so if i do asdf asdf asdf you see it's slash asdf asdf asdf so by default the length is always going to be one the length is always um the path always starts with a slash so what i'm going to do here is assume that any path that i get if the length is greater than one then i want to remove that preceding slash and then try to try to read from kv uh what's in the remaining uh what's remaining after that slash so um in this case try to read asdf asdf asdf without the slash and um that seems very reasonable so i still can do a home page if it's slash i can uh print a home page if i wanted to and then if it's uh if you have the rest of this and then any other request that's not a post request will uh will try to do a url shorten thing okay so i think that makes sense if path name dot length is equal to one return new response this is the case where it's just a slash no path welcome to my url shortener and if we speed up i want to make a form to submit to this so uh i'll try to go fast otherwise we want to do uh we don't actually need this else here we can just continue here and say um half the name can you do a slice operation in uh javascript slice string i don't think you can do a slice with uh like python syntax so we can just do this okay so we can just do path name dot slice hello world will return with zero five should return just hello okay so we want one i think the second one is already is uh optional is that correct optional yes so we're going to start at the first character read every character in the string and uh omit the end so we want to read to the end of the string and it should return um the key then we can run it key and it should return the shortened url here and this should just all work i think this will work end to end so we'll be able to issue a request with a with a uh post so let's try my website now ejj.io so here's the shortened url and when i go here in the browser it should redirect me unless it broke nope it works voila uh end to end we've got a url shortener working that is fantastic uh so the last thing we need to turn this into a product uh we should do some validating urls in the uh if this if you're meant to be to be using this on the web there's probably no need to support file schemes there's probably no need to support all these other schemes that are possible my favorite old vulnerability uh my favorite php vulnerability is being able to slip in this file scheme into urls uh that people then curl um anybody i've i've it's a issue that i've seen a million times and so i'm very sensitive to anytime a url is submitted without the right scheme um uh and there's some other error checking we should can get to actually if uh if shortened we should probably handle the fact that if you just go to https with some anything after it uh yeah so there's a no error handling here when you fail to read from uh kb so um return okay this might be way too clever shortened url otherwise return um new response bad request i wonder if this will work and then i can uh then i can oh not in the right directory yep okay so now we've at least have some error handling um can also have it issue a redirect to um this might be a better way to do it shortened url if not shortened url shortened url equals okay it'll just redirect to the home page now if uh if there's an issue reading the value from kb all right we only have three minutes left i'm not sure if we'll get to uh finish the home page but i think we can i believe i think we can program really really fast now when we go here it should redirect us yes welcome to my url shortener and instead of this we're gonna just do real html in the javascript this is just like json in the database uh this is the just json raw just directly in the database this is the best design pattern that you can ask for uh so i've got a html body here and we are going to we're just going to make a form i always forget the the if you write method and then how you set the url action that's it i will forget this for the rest of my life um let's just copy some of this and actually let's copy this whole thing and this should just about work we just need one of these though to say url um this will submit the wrong thing though won't it so this is a uh this endeavor is doomed to fail because my api is expecting um a json post request and this is a form this is a regular old form submission so this won't work but um we could write a little json making a home page with a nice form i'll finish the form it should work uh you should you can at least see it to imagine oh no what have i done i wonder if it's angry about the syntax yeah it's probably angry about the syntax i'll try one more time i only have 30 seconds so this will be a down to the wire and i thank you all for watching this week i'm glad we actually got the work the uh this built on workers we built a url shortener and it's a bit of a toy but it does work and oh i forgot to set the content type so it's just text but i really thank you for joining me i appreciate you watching me and i hope you build something with workers it's a lot of fun and i'll see you next week