Developer Week: Bonus Challenge

Hi, Developer Week is finished and what a week. It was filled with so many announcements. I actually, even though we got to read them beforehand, still can't decide which is my favorite. So I'm Gretchen and this is Luke and we're here with the Cloudflare Workers team. So whilst we ran through Developer Week and had announcements left, right and center about the most amazing things, we also ran a whole series of developer challenges. So these were designed to get people on board with workers, to spend some time having fun and to help each other out in our Discord channel. The challenges got harder and harder as the days went on. And when we got to the weekend, we called it a bonus round, but really, it was a bit rough. I say rough. The challenge was super exciting. I didn't get to finish it. Luke gave it a solidly good effort and came out with some amazing results. So today, we're going to go through that challenge, which we called Remind Me. If you've got any questions as we go through this, you can email them and I'm just going to read the email address to make sure I get it right. It is livestudio at Cloudflare.tv. So if you flick an email through to that address, it comes to us and we can answer your questions as we go. So the code we're talking about today is shared on our Discord server. And if you've heard Luke arrive before, you'll know that probably about five times in the segment, I'll suggest you go and join our Discord community because it's actually the best place to be, I think. There's so much knowledge and learning and help and support there. And pretty much that's how I figured things out. Luke came to us via that channel as well. It is the place to be if you want to do anything with the workers platform. So the Remind Me challenge, the bonus challenge for the weekend, was kind of recognizing that life's a bit chaotic at the moment. There's a whole heap of stuff we keep forgetting. I need to-do lists or else nothing happens. But it was about building out an app, a reminder app, to tell us what to do. And the only asks were that you used Cron Triggers and Twilio, which sounds simple in and of itself, but I think it was quite a vague open-ended project. I did have a throwaway comment in there about, hey, what if you could email or sorry, text yourself your reminders as you went, you know, get up in the morning and your phone starts hassling you to do the things you need to do. Seemed like a good idea. I think if that happened to me every day, I'd probably get a bit angry. But getting all this up and built together and running over a weekend was a huge ask. We're so impressed with the submissions that came through. And I think one of the things I've forgotten to say so far is a big thank you to everyone who participated in the challenges, because time is precious and spending it with us and doing the silly fun things we had happening is, we don't take that lightly. We really appreciate it. So thank you. And I need to say that a bit more, I think. So Luke, when you read the challenge, were you like, oh, what do you want from me? What did you think? Basically, just because it was, it was more open-ended, right, than the other challenges. And so being a web developer, my first inclination was to just build a web app and then set up the cron trigger that would remind me at the start of each day to do those reminders, right. But then I took a step back and said, well, there's nothing in here that says it has to be a web app. So my approach and what we'll be going through today was to build an app that completely operated through texting. So that meant I would be adding reminders to my reminders list through a text message. And then I would be listing those reminders through a text message. And then I would still get the, you know, here's your list for the day, every single morning. So instead of wanting to throw my computer out the window, because it's hassling me to do stuff, I'll throw my phone. Right. Yeah. Yeah, you can smash that SMS reminder at the same time you turn off your alarm. So two birds, one stone. Was there any point you were going through this journey of building an app for your phone that you went, should have done a web app? Yeah, I'll call it out when we get there. But effectively, well, I can preface it a little bit. So effectively, 44, in a standard webhooks model there. So in this scenario, when Twilio is sending our endpoint a message, there has to be some authorization process, right, a verification that this message actually came from Twilio, right. And we saw this also in the Discord bot challenge, where we had to verify the incoming message, we looked at a signature header, a time header, and then the message contents, did a bunch of computation on it, made sure that they match. Same thing here. Except this time around, there wasn't actually a, I'm sure there, I could have found one, but there wasn't a solid recommendation within the Twilio documentation for a library to use for that. So I, given that my, you know, crypto algorithm knowledge isn't really up to par. I did bash my head a little bit against that and regretted it in that moment, but came to a crypto native solution, native to the workers platform, and I was pretty happy with it. I'm impressed. I probably would have pivoted straight to a web app. I can't do it. If I'm being honest, part of the development process for me was to just turn off that authorization step and skip the verification step and just make sure that the, you know, data passing back and forth worked, and then try to figure out the verification once again. Definitely not a production solution in the interim, but it was a nice way to make sure everything was piped correctly. You knew which bit was breaking then. I like that. Yeah. So I've seen your little video on the GitHub repo that shows how things work. Are you able to show us that and how you got to where you got to? Sure. So I guess we will, I think you can see. I can see a screen. Great. So we will, so this is the video at first, right? So I will go through the video once and then we'll go to the code, but basically there is a home screen component to the web app, right, where you enter your phone number, you hit the submit button, and then that kicks off the process on the worker side. Success. Please check your phone for a new SMS message. That then comes in. You know, this 605 number is my Twilio account. And right off the bat, so this can all be done with the Twilio trial, right? So during a trial, every single message is going to be pre-pended with this Twilio trial account header. So we'll just get used to it for now. But, you know, it gives you some instruction. Reply with new, with some text to add a reminder. So I'm saying new, hello world, send that off. And then Twilio replies, you know, added a new reminder. You now have one reminder. I think I added a second one here. New toolbar, one, two, three, standard message here. Same sort of thing, but there's also is a list command. So I can see the entire list coming down. I think this takes longer every time just because of the formatting side on Twilio's end, but it's really not that bad. Pause it here. So we see that the list has the two items we've added along with their IDs in front of them. So then it tells us we can either add a new reminder once again, or say done with a specific ID, right? So now I'm saying I'm done with the second reminder. So I have one reminder left. So this was all the Twilio trial. You got to set it up without having to pay any money to be on the platform. Right. I think each text message is less than seven tenths of a penny of a US penny. Right. So I don't know how to convert that. So I'm in Australia and my brain was like, it's very, very cheap. Right. So I think throughout the entire development process, I used a dollar and ten cents of my fifteen dollar credit. So, yeah, it was not too bad. Expander. Okay. So should we do the code now? Absolutely. I'd love to see how you got to that point because it's pretty impressive. Thanks. Well, so right off the bat, we do have a couple things at play here. Right. So within our Wrangler config, this during another important thing to note during a Twilio trial is you can only interact with the Twilio phone number from your your personal phone number. Right. And this is the one attached to the account. So that's called this test recipient here. And this is basically Twilio saying I'm only accepting SMS messages from this number. It's like a good safety feature whilst you're getting things up and running. Yeah. So you're not just giving away fifteen dollars to everyone. Right. And then, of course, your Twilio, the phone number that Twilio provides you, your Twilio account ID and an off token, which is must be added as a secret. And then also for this example, we use a KB namespace and I'm calling it reminders here very quickly just to get those those values inside your Twilio console. There is this account SID right here and then the off token. And so you just copy both of those and they plug them in as well as the phone number and see like that. A dollar seventeen total use for the trial. OK, so you may remember that the the only Web component of the entire app was the home screen. Right. And so that's where this get root comes from. And this is the silly utils render function, which is creating a response from a view template. So I won't go through all of this, but this is a just string concatenation where we have a full HTML page, very basic styling and configurable custom styles, but configurable title message and then body content. So the home page a lot of time here. Right. But you just took the docs and ran with the template. I went for I opted for string concatenation, these little string templates just because being a web developer, I didn't want to spend most of my time doing like a full front end application, like no react, no preact, spell any of that. Just like I only needed this one page and knew I needed the one page. So it was just get it over with. I'm sorry, that is so fitting for so many conversations. So the important part here is the actual form. So this is still within the welcome page. The form is a post which post to the login once it's actually valid. So I know it's hard to see because it isn't actually HTML syntax highlighting, but it's an input with a phone field, some HTML5 validation in here, but it is required. And that's the only thing that is required. So once this data comes through, we have this post login route. So by default, standard HTML form behavior, it will do the client side built in validation using our pattern rule here, which is completely optional, but I added it just because it will then post to login with a field called phone. So define that route here, this is still within our worker. So when it is a post slash login route, we're going to take the request, pull out the text from it. And again, by default, HTML forms are sending things as I can never remember, but X www form URL encoded content type, which means that we can pass it through a URL search parameters. So this is a string, we pass it through here and we can just pull out the phone. I was going to ask you, sorry, about line 40, because I have that phone format. What do you do around the world? But I see you solved that problem. So this is, I think this is how most applications actually solve global phone identifiers. And so that's called this E164 format. And it's what Julio uses too. So it's just a nice way to normalize it. And in this particular case, two E164 is just making sure it starts with a plus, and then there is a number, starts with the country code, the area code, and then the phone number itself. So this is creating that format, but is E164 is actually implying, is actually using a regular expression to test that. And so this is not a validation test, right? Like this is just a format test and we can do this locally before, where did that file go? Before, so we do that a couple of times. So before we, okay, so we check for country because the Twilio account only supports US country code because that's where I am. I thought you were going to talk about your dollar 17 and how it might've been more. There probably was a way around it, but it was, again, it's just my phone number. So I worked out here. Okay. So the local E164 format checks for all of these lines here, and we do that so we can throw away broken looking phone numbers before actually asking Twilio to say, hey, is this phone number valid for this country? Right. So the pre-check. Pre-check, right. And so it doesn't, it's faster because there's no latency involved there because there's no network requests. And we, if there's ever some sort of cost for this, not that there is from Twilio, but for other services, this is just like a pattern that I tend to do, right? So Twilio.lookup takes a phone in the country and Twilio.lookup has, make you dizzy. It has, well, documentation's here if you're interested, but it basically validates that the phone number is valid for the country that you've given it. Right. So that's the actual time that it, Twilio will decide, not us, but Twilio will decide that this is a real looking number and it will actually be valid to send something to. So. That's interesting. Sorry. I know that was a total tangent we hadn't discussed before. I just got distracted. That's okay. That's great. So in this case, there's no extra validation or error message that I can send on my end. So, you know, if it's not a valid match, Twilio is going to send like a 400 code and I'm just going to dump that to the client and say like, Hey, there's something wrong here. You know, the Twilio messaging would be pretty clear for, as to why it failed. But if it, if it is valid, it's going to send back a country code and a phone number. And in my case, that did match every single time because it is still US. Um, and it is, you know, I was giving it the, I was giving it the same format phone number, but it could come down with, you know, an extra hyphen sometimes or whatever. So the, the point of practice here is just to use and rely on the Twilio data as the normalized country and phone values here, even if they do match, but you know, this will be because our data is so reliant upon what Twilio has and what Twilio considers to be valid. We're just going to use that as our values. That makes sense. You don't need to reinvent this. Right. Um, so once we have that, those normalized values, uh, we then are looking to create a key, a key value or a KV key. Right. And so, so we take our phone, we take our country and then we're creating a key in this format. Right. I'm not sure if you can see, but you know, this could be a U S colon colon, and then we strip the, the plus out of it. That's this substring one. Um, so we have, uh, we have this key and again, it's in its own helper function to normalize it no matter where we're, where in the application we're asking for a key, take that key. We ask for a load, which is down here. So that's just interacting with our KV namespace and asking for a JSON array of our reminders, uh, that then gets passed through this display function. Um, which, you know, once KV runs, it gets parsed as JSON and we either have something or we have an empty array by default that gets passed through this, uh, this display function, which takes all those reminders and formats them into the message, the SMS message text. Right. So this is where we get, you have one reminder, you have two reminders, and then it's going through and listing, um, uh, it's doing this format, right. You have two reminders, colon, and then it does the ID and then the text. And then it, you know, as the footer message, it shows you the actions you can run. So this is just the logic to do that. It looks so simple once you've written it and I can read it. Uh, hopefully, but it's not always the case. Right. Um, so, so this produces a text string. This is our final message and it's, it's reactive to the number of reminders that we had. Um, and at that point we can ask Twilio to send an SMS message to that, to our normalized phone number with our display text. So, uh, sorry, here it is. The SMS function, um, is this. And so this is where I spent most of the time with, within the Twilio documentation. Luckily, most people I would say are using, um, sorry, I have the Zoom thing in my way, are using Twilio to send SMS messages. So there are plenty of, you know, client libraries are, there's plenty of documentation about this and there it's, it's very easy to find. Um, but basically I went through here and saw that there was, you know, of course the required destination endpoint that you had to send this to, and they need, they need the data to show up to, from, and body all capitalized. Um, and then they have these example snippets here on the right. So given that this is a worker's environment and I chose not to use a client library, um, I was looking at the curl example where they have this, you know, curl with a post to this, you know, messages.json endpoint, some body, some from, some to, and then with this dash you account auth token header. So this is the authentication layer, right? And this showed up as part of the lookup thing too, but this is, in my development process, this was the first time I actually stumbled across it because I built the SMS part before I built the lookup part. Um, but yeah, I just got right to the meat of it first. Did the fun bit and then went, how do I make it work now? Yeah. I thought I should probably use lookup because it's there and because it will guarantee that I'm always using the same data. So I'm saying all this as if like I knew ahead of time that these were the ways to go, but it was all, you know, you know, reiterating or iterations as I went along. Um, so in curl, this dash you, this is basic authentication, right? So this is, uh, short for a user or username doesn't matter. Uh, but basically that is a basic with, um, the username, which in Twilio's case is the account SID and then the password, which is the Twilio auth token. And remember one of these, this guy was, uh, a variable in our wrangler file. And then this guy was stored as a secret because it is a secret. Um, and just the way user, uh, basic authentication works is it's the basic type, this thing. And then you take the base 64 value of user plus a colon plus the password. And this isn't specific to Twilio. This is just how, you know, this is basically standard, um, life lessons from Luke. So we do that once and, and stored it in the top level because it was used for SMS, but it was also used for lookup, as I mentioned. So we just do it once and reuse that result in multiple places. Um, anyway, back to SMS, we have the phone, we have that display text. Um, this endpoint required, uh, required to be sent the URL encoded data, right. Um, just like before. Uh, so again, URL search parameters comes up. Once again, we are setting the, from, we are setting the two, we are setting the body. Um, the body is our display text. The two is our destination URL. This part is technically optional, right? Because we've had a normalized phone number, but it can't hurt, uh, just for the way this utils thing. Um, this function may be used by some other consumer later on within our application. So it doesn't hurt. Um, and then from is our Twilio phone number. So the number that Twilio reserved for, for our use. And at that point, we are just plugging in our account ID using that message and letting Twilio do its thing. So we send off the fetch request. We don't have to care about the response body. Uh, basically if it failed, it's going to throw and that's all we would have to worry about, but you know, we don't, we don't care about the result. It doesn't make a difference to what you're doing again. Yeah. If this fails, it stops here and we'll get a worker runtime error. And admittedly this should be handled in a better way, uh, for like for a final production example, but you know, it would just be a matter of a try catch around that. Um, and then finally for this post login route, we have, we have to render that success message. So we have utils that render, and then this, this view sent block, which is using our same HTML template function saying success, please check your phone for an SMS message with some custom styles. So that as a reminder somewhere is here, right? So this is the thing that gets sent. And before that little message came down, of course you can't see, but this would be, you know, the login slash login as the URL, right? Because that's just where the form sent the data. Oh yeah. So it's so simple when you show the solution, but when I started looking at this, there was moments I went, I don't even really know where I'm trying to get to. So all we've done at this point is kick off the kick off the request to say, you know, Twilio starts sending me stuff and then we have to actually interact with it. So at this point Twilio will send like, Hey, welcome to the app, whatever. But we have to hook up this messages, send response, which I'll show you really quickly. Cause this was a little hard to find. You go into your phone number, you go into your settings for it. And down here, when a message comes in, call this web hook, right? And so at this point, it's, it's my domain slash web hook. And it's a post. So if we define that route, you know, posts at a web hook and it looks like a lot here, but the flow is very simple, right? I did bang my head against this as I mentioned, but it is actually very simple. So there's extensive documentation for this, which did make it easier. But the, the flow is you have to take the request body coming in as text. Yeah. You have to take this Twilio signature header which again, feels like the discord bot challenge. There's that there's a header there, right? This case there, there is no timestamp value, but that's fine. Instead, what we have to do is we have to take that input that came from here. We have to take that input, grab the values from it, which we can do with URL search parameters, and then recompute a signature for this combination, right? So it's a combination of the URL and the parameters we were sent. So just very quickly, cause it's not that interesting, you know, it's requirements is you have to sort those parameters alphabetically, which URL search parameters does by default. And then this was the, these were this two, I guess, two lines of code that compute the signature natively. So it's a sign with an H Mac, the key that we derived from up here, and then we encode the value. And then you just have to cast it back to a string. If you really want to bash your head against it, I implore you to look at it after, but it does give you a signature string back and you are required to say for production apps, right? If, if those signatures don't match, it's been tampered with. So you have to reject this because it probably didn't come from Twilio, right? So yeah, definitely a must have with any webhook system. But once that's all clear, we can then actually look at the data. So we go back to the params thing and pull out a phone, do some phone validation. Once again, pull out a country. They are calling it different things at this point, but again, it's all documented. Pull out a body. And then really what we're doing is we're getting the data that Twilio said, that Twilio parsed for us saying, this sender has sent you something from this phone number. That phone number belongs to this country. And because those were Twilio values, we can use those to create our database key once again, right? So wrapping up here, but we, we look at the body, right? Because the, the display text was saying, send new with a text string or send done with an ID string to figure out what to do. There's also that, that list command. So we take the body, split it by, you know, a space or multiple spaces, and then assume, well, we know that the requirement was the first command had to be, the first word had to be the command. So we're just listing through all possible commands, doing some sort of KV interaction if necessary. Actually they all have a necessary step, but if it's new, we load the values, we push in a new value, we save it, send some sort of reminder text, same thing with done. Except in this case with done, we are taking all items from the database and just popping out the thing that is now done. But in all, in all cases, we can just return a response with our reply text, right? So when done runs, it's, it's mutated the list, saved the list. We create new display text, right? Where we say, you know, removed the reminder, you now have some number of reminders remaining. So if we return that response, that gets received by Twilio. And so Twilio, it's automatically sending that as a message back to me in this case, but. I've been remiss in my job. I've been so distracted by watching your code that we are down to 30 seconds. Oh, well, we finished this in time. Terrible. I really enjoyed this and seeing how you actually went through it. We've had conversations, but not as in-depth. As I said before, the code for this is on our Discord server. So come along and join that and have a look at what's happening on there. I was intrigued to see your to-do list was sensible stuff like hello world and fubar. When I was tinkering with these, I put book dentist appointment, still haven't done it. Anyway, we would love to see you back some more. There's a couple, there's a session next week. Bye.