🏗 Migrating to Cloudflare Pages: A look into git control, performance, and scalability

Hey everyone, my name is Christian Freeman. I'm an engineering manager here at Cloudflare for the developer advocacy team, and I'm joined by James Ross. We're talking today about his team's migration to Cloudflare Pages and kind of doing a little bit of storytelling, I guess you could say, of their journey so far through, basically, as far as I can tell, all of the Cloudflare stack. So really excited to have you here, James. You want to introduce yourself? Hey there. Yeah, thanks, Christian. As you said, I'm James, you know, Nodecraft go online by number, but as Christian said, we just kind of done a recent migration over to Pages, which we are kind of really excited to talk about and some of the great things that Pages brings to us coming from kind of a previous product where we had to kind of make it all ourselves. So we are all very excited for all of the kind of automation and CI and stuff around Pages. Yeah, so I imagine we'll kind of walk through the entire kind of story because I mean, like I said, I think you all have used like Workers a lot and like Worker Sites and stuff like that. A lot of things maybe that people don't even know exist. Frankly, you all, I think, are pretty like down in the depths of all the tools, like. Not afraid to do some plumbing, but a thing that I thought was kind of funny this morning we had our sort of kickoff for the Speaker Series, and I was talking a little bit about this call and this chat, and I guess it was I mean, maybe you remember it's probably six months ago, seven months ago that we talked about, just like your story so far is part of our shoot. What do we call it? I don't know, developer stories or something. I should remember what it's called, but at that point you all had not migrated to Pages yet, right? Or was that kind of being planned or where were you back then, maybe six or seven months ago? Yeah, exactly. So as you said, we were using Worker as a huge amount back like years and years ago and we pretty much started using Workers Sites, which was kind of the precursor to Pages, but they had kind of a whole other things you have to do yourself. You have to kind of bring your own CI all of your and previews, all of your routing, everything like that. It was all very hand baked, which wasn't really a problem for us. But like over time it became kind of this thing where like every project would be the same thing, copy and pasting all of the stuff over and over and over again. And like we've been kind of looking at Pages for a while and kind of evaluating moving over and there's been kind of a few things that were kind of kind of blocking us for a little bit, but especially with some of the improvements over Pages over the last few months. And I know today as well, there were some great announcements and Pages about Pages and things like that. We were very much able to kind of complete the migration a few weeks ago with some of the kind of improvements to functions and stuff like that, which is really exciting for us to finally get the majority of those very much will moved over the few sites we're currently working on, but I believe about half of our sites are now in Pages, which is really, really exciting and we're excited to move more over and then the future. Yeah. So I think we'll, we'll spend some time talking about, about Worker Sites because I think I mean, you are like I said, you're very like in the loop, helped contribute to like one of the core packages, a lot of Worker Sites users use and stuff like that. So maybe let's start from like kind of wherever you think makes sense in terms of like a beginning with your like usage of Cloudflare and that I think as far as I know, predates actually Workers, right. It was before Workers was even a thing or at least like you use Cloudflare before Workers was like part of the the toolkit, I guess that you guys are using, right? Yeah, exactly. We used Alphabet for I mean, as long as I can remember, it's been, I think nine or so years at this point, which is way before Workers and just using some of the other products. But official introduction to Workers themselves was, I believe, one of the early Birthday Weeks and we kind of built a few small little projects here and there, like image proxying and header rewriting and some of the kind of standard things that you do with Workers. But then once the kind of Workers Sites announcement kind of got released, and that was kind of a project where you could essentially like deploy actual sites onto Workers, like into the AD store and things like that to actually store, like all your assets. That was really, really exciting for us because it meant we very much no longer had to kind of maintain a bunch of infrastructure because previously a lot of these sites were have just been like on their random visas or random cloud hosts out there, which like over time gets a lot, a lot of maintenance. And if we can just throw it on to the edge with something as easy as KV, that was really exciting for us. And again, we had to very much bake a lot of the kind of CI stuff and the routing was all handbaked and things like that. But it was very, very functional. And of course being on the edge, it was extremely fast for us, which for our users is extremely important, like being gamers. Gamers especially want like. Can we take a sec? Actually, I don't even think we actually introduced the what your what your company is. You want to just give people a quick pitch as to what nodecraft is and what you all do. Yeah, yeah, sure. So nodecraft is this gaming cloud platform, which essentially means we can have gamers and people like that come to our website and very much spin up their own game servers for any of their favorite games for the friends and family to play on, they can bring their own kind of user generated content, things like mods and plug ins and stuff like that. We make it as easy as possible to spin up those games servers very much local to your users as much as possible, which is very important for gamers, as I was kind of saying in the sense of gamers really want that kind of low latency. So that's kind of kind of carries with them as they're kind of browsing the web and stuff like that. If your site takes a while to load, they very much get frustrated about that. So having those sites on the edge is really, really powerful. Gamer generation, I feel like, is not used to slow internet. Definitely. I had just a little bit of that when I was a kid, but by the time I was like playing, you know, like, I don't know, World of Warcraft or like Diablo 2 or whatever. My expectations are very high. The Internet needed to be fast, you know. So I think you're totally right. That's yeah, that latency is not something that people it's a no go, right? They'll use someone else. So let's let's take a sec. I think to one thing I really think I don't know if you thought about this way, but coming from my perspective, like I've been the developer advocate here since about 2019, the thing that I think is really cool about what you all like, the way you sort of adopted Workers, it's like very similar to, I think the way that we have like kind of our thinking around like how to tell people about Workers has evolved over time, which is really fun and interesting. Just to give you an example, like when I joined here and like the task that I first did as a developer advocate was like writing tutorials and stuff like that. Like my very original way of thinking about Workers was like what you're talking about, like proxying like URLs and images and stuff like that. Like the sort of like kind of glue code, I guess you could say, right? Of like, I have stuff here, I have stuff here. Like, how do I put them together? Kind of code, right? And it's like Workers is great at that, right? Like not to say that like we've evolved past that or whatever. Like, I still do that all the time with workers, but I think that our thinking has kind of evolved and the way that we talk about it has evolved too, when Worker Sites came out, we were like, Oh, like KV is really good at storing stuff on the edge. It's really fast, and Workers is really programable, like, let's put those together and let people host their sites on the edge, right? And like you said, eventually that did evolve into Pages, which we'll talk more about. But I think it's interesting and cool that like. The way that you all are thinking about like. How to use the product. It just follows very closely with like, I think, how the long term vision of Workers has changed as well, right? And to the point where like eventually you started writing. Maybe we'll talk about this next like services and like APIs and stuff like that on Workers. Like as we started to figure out more of the like Wrangler stuff and like the command line interface and it was a legitimate like developer like has a CLI, it has like analytics in the dashboard and stuff like that. Like it's, it's matured, it's like a platform. I think that like the being able to think about things like that becomes a lot more, a lot more reasonable. Like that kind of glue code is really easy to do in a UI editor on the dashboard. But when I'm like, Oh, this is my API service that like cannot go down like it's serious, right? It's production, whatever. Like, I want to have a CLI, I want to be able to configure things like using a config file and stuff like that. So it's just interesting how that has kind of evolved kind of in parallel to like what you all are doing. And like from my perspective at least as a developer advocate, like how the platform has changed, does that make sense? 100%, yeah, exactly. It's been really cool to kind of grow along with Workers that kind of just being there at the start and like as you said, all of the kind of things that Workers are great for, like all of the proxying and the rewriting and stuff like that. And then to kind of just have this kind of almost a mindset change over a while, just kind of in the sense of, okay, these can actually be hosting entire applications or we can actually host like an entire projects and a lot of our internal thinking now when kind of like deploying new projects and new sites and things like that is very much how can we get this on Cloudflare Workers in a way that's going to make sense. Just so that's kind of the first point of call rather than like let's migrate to them. Like anything that we build nowadays is very much our first thought is how can we run this on Cloudflare Workers in the way that we haven't got to worry about scaling or infrastructure or maintenance or any of those concerns? We just kind of write the code, hit publish, and we all pretty much get to go. So it's been very great to kind of very much, as you said, grow with the site's product and then with KV and year and then into now we have Pages and other cool stuff and it's been very exciting to watch it grow over time and kind of be part of that growth. So, tell me a little bit about like Worker Sites and, and maybe prior to the migration like. Where you were at, like struggling with like the kind of so many projects using that stuff. Like I think for us internally, Pages, like the lessons we learned from like doing Worker Sites is kind of a stepping stone to Pages, something more like comprehensive Pages. Like for instance, we use Worker Sites for like our developer docs and stuff like that. And we actually have a very similar, I think, story to tell that we're going to be publishing on the blog next month about how we moved our docs to Pages. It's probably very similar to it, what the stuff you were thinking about coming from Worker Sites, but from your perspective like what was that like and where are you struggling? And yeah, I just love to hear that perspective. Yeah, sure. So for us, using Worker Sites was kind of a huge win over what we were previously doing, which was again a bunch of like Cloud VPS and stuff like that, which are paying to maintain random PHP files or HTML files just floating around on CI on like GitHub repo. And it wasn't a whole lot of kind of baked in stuff to that and then kind of Worker Sites came along and it was a huge win for us, just kind of being able to have all of that published and then to be able to have kind of our own CI platform. You guys released an actual for GitHub, which that was kind of automates some of that process, which is really, really cool to see. But kind of over time, a lot of those projects were getting very repetitive. We had to constantly write the same code to handle things like routing, handle any kind of like preview deploys. And the CI was pretty much the exact same thing across like seven different projects that was just copy and paste, copy and paste and that is okay. But kind of as, as it scales it becomes a real problem. And once Pages was kind of announced and talked about with a lot of its own kind of CI and like preview builds and things like that. It was really exciting for us to be able to just kind of have our developers push to different branches and they would end up with a preview link straight there in the PL where they and the rest of the team can real quickly, just go have a look at what's change visually because it's one thing to look at a code review, but to actually be able to like very quickly click on the link and actually preview what this is going to look like is something that's really, really powerful and all of that's possible with Worker Sites, but you have to build it yourself, which again is a terrible experience and you end up with all of these problems and you have to keep doing the same thing over and over again, which is one of the huge reasons that we moved on to Pages and especially now with Pages Functions as well. All other kind of the interaction stuff, all of the kind of full-stack stuff unless you kind of do like templating and like HTML rewriting and stuff like that is now totally possible directly on Pages. So it kind of graduated from just being this Jamstack platform into an actual Full Stack platform, which was the point for us where it became a kind of, okay, this is actually something that's really, really cool now because we can actually write the same level of kind of control that we would get with something like a hand baked Worker Sites implementation, but being able to use all of the actual positive things that you get from Pages. Again, like all of the all the build stuff, all of the preview stuff and all of the gear integration and things like that. So it was really, really exciting to us to actually be able to finalize that move for a bunch of sites. And again, we are very, very hyped to finally get all the rest of the sites right. So, yeah. Yeah, that makes a ton of sense. And I think the functions thing in particular, like one of the reasons why people would stick with Worker Sites is that like really like even a couple of projects that we had built prior to Pages is like we were doing really kind of complex things with like taking static content, transforming it, like adding things from KV or even like a durable object or something like that. And so like having Pages and having pages functions to handle that. And generally like, let's say for most cases like that, routing based which people can go look over the docs like we kind of have like a route based or file name base routing is actually what I should say, for like your different functions, and running code like on specific routes and stuff like that. It's a pretty good fit for a lot of the things that I think Workers Sites did in the past. But yeah, you don't have to maintain it actually. Frankly, I'm shocked that you all even got pre deployments working because we tried to do that with a couple of more precise projects and really struggled. We were like, Wow, this is really hard. And we're also just redoing the work that the people who literally like are that we talk to every day have already done, right? Like we should just move to Pages or whatever, right? But that's yeah, I mean at a point it just it makes sense to switch. So maybe tell me a little bit about like maybe we pick one project that sounds like it's multiple sites, right? That you move to Pages like what does it look like? Kind of whether it's like a framework that you use or something like that. And then like how do you handle like what do the API routes look like? How do you write those and stuff like that? Like maybe that help people sort of understand like, hey, I want to see like a final production. Full stack pages application. Like what does that actually involve? Yeah, for sure. So there's a couple of examples I think things that we already have on Pages and then one that we're currently working to migrate to Pages, but the one that's currently on Pages that I think is a great example is MC versions, which is a website, the kind of correlates all of the previous Minecraft versions from the last like 15 or so years, however long it's been more than that now. Well, I'm old, but for a very long time and kind of it there's tons of different pages there, tons of different endpoints, and there's a lot of kind of CI behind the scenes that kind of goes and finds all these versions, kind of archives them and gets all the things and everything prepared. But one of the like core things for gamers is kind of user generated content and kind of very much for things like on PC. There's a lot of other things like game modding and stuff like that. So having kind of this archive of old Minecraft versions because not all mods work on this version and things like that, it's really, really useful to those people and I've been able to come back and actually grab all these things. And we're using a lot of just. The static HTML for the majority of that site. That's all built just via some custom tooling. There's no framework or anything behind that. We're kind of huge believers in because of announcement and just making things as static as possible. But there are kind of some functions running in there in the background, doing some kind of like optimized image serving and things like that, which was great for us to kind of very much transition over from Worker Sites as well. Again, from the whole web performance thing. Like we're huge believers in that and doing things like serving images in the most modern formats and things like that and like with functions and kind of asset function inside functions, we can accomplish essentially all the same things in Worker Sites, but now on Pages we get like all the benefits that we talked about before. Then not a really exciting one that we are currently looking at. Migrating in the next few weeks is PlayerDB , which is kind of this API that we've built, which we use very heavily internally and that coalesces a lot of player information data from a lot of platforms like Steam and Xbox and like Mojang's Minecraft and things like that, which allows people to freely and publicly query information about players and get like ID transformations and things like that. And that's really exciting for us just because that's a great example of a website that's going to be on Pages that uses both a lot of static assets. Like there is a couple of homepage assets, but then there's this like slash API which will be essentially just all functions. And that's kind of very much designed in a way that we're going to be able to essentially take all of those player APIs and put them into like their own files for that matter. As you said, firebase routing kind of like API slash steam API, slash Minecraft, that kind of thing. And they can kind of live in their own files and just run those functions, which we're really excited about because again, right now that's still running in Workers sites and we have this kind of complex router set up to handle different paths and things like that. So it's going to be really great to kind of get that into pages functions, right? Yeah. You in like a work of sites world. You would I mean, in the worst case, you would literally parse the URL yourself and do all of the routing that you would do like string matching and stuff like that. Or if you're a little bit more advanced, you like plug in a router and then you're doing kind of like express based routing sort of or something similar to where it's like, okay, if you see this path, like call this function, etc. And then the Pages version, which is great, is like you can right now, let's say make a folder. It's like slash API, slash steam, slash bracket ID dot ts and then you can do if you have like utilities and stuff, you can put that somewhere else and have like shared logic and stuff like that. But it's like very declarative and you're never going to be confused about like what route is which function and stuff like that. So that PlayerDB service does that use like, so there's a lot of sounds like a lot of like external services that's reaching out to, does it also have like a sort of storage mechanism built into it or like does it use KV or something or does it have like a database that it refers to or how does that all work? Yeah. So right now very much, which is it's a bunch of different services, like you said, Steam, Minecraft, I think Xbox as well right now. And currently all of that information is just using the cache APIs in Workers. It's not actually storing it permanently. So it's actually beneficial for us for like the more people that use it, the faster our platform itself is very much we use that internally and build through that. So when we get people externally using it actually maybe. Benefit from their populating the cache for you basically, right? Exactly. Yeah. And I think long term, we're going to definitely explore kind of storing that in a more kind of long term cache, like a KV or something like that, just so we can kind of reference that and kind of watch it historically and stuff like that, because that kind of data can change and it can be useful sometimes to kind of see how it's changed over time. So historical data, yeah, well, we may have something tomorrow that may be up your alley in that regard. You know, hashtag see this, as we say in the Discord. Yeah, that's super cool. So when you do that migration, what does that actually look like? I mean, how often are you actually changing code or are you just sort of like changing function signatures, really, or like what does that process look like for you all? So a lot of the process is essentially just like seeing a bunch of old stuff like we have to delete a bunch of GitHub action workflows. We can delete the entire Workers Sites, like code base. And then it's just a matter of taking all of those functions and just kind of reorganizing a little bit into like the functions API syntax for the Firebase routing. But it's primarily those PRs are like 95% removed code a lot of the time. Just because it's reducing all of that friction and all of that kind of bootstrapping that you have to do and Worker Sites, then just kind of transplanting it straight into a page project and there's no really new code we have to write or we may have to add a couple of like underscore headers files or something, which is a lot of great features of Pages. You haven't got to kind of manually type in headers. You can just create a file code on. Just go ahead and tell it just on slash API, assign all of these like JSON headers, for example, which is quite useful and things like that. But other than that, it's just a matter of taking code, copying it here and just pasting it over here just in the way that makes sense from the functions. Firebase routing. But the kind of like developer experience around all of that is, is so great now with Wrangler as well. We have Wrangler pages dev for example, which I know when version two just this week was announced that publicly. So that's really cool to see and all of that that's going as always getting better. So the page's dev tooling is great and it's it's very much an easy migration that I believe there's going to be a full migration gearing up on the docs if it's not already up for like Worker Sites projects. So, yeah. Yes. I mean, I think we I'm not in the in the driver's seat in regards to like when we know where time it's time to kind of officially tell people to move to Worker Sites, because I think there probably is always going to be some use cases, I think, right? Probably in some cases, if you have like an origin somewhere else that you want to like do stuff with and serve. I don't know. Honestly, I can't think of off the top of my head. But yeah, I think that that makes a ton of sense. I'm excited to hear that it's mostly deleting code. That's, that's always positive. Everyone must delete code, right. That's what I was feeling as a developer when you get the delete code and everything still works. Yeah, yeah. Maybe we because where you have gone, we only have 6 minutes left. Time has gone by. Very quickly, maybe we switch gears a little bit. I'd love to hear about and particularly Platform Week is going on right now and today was really a big day for Pages in particular. It's a lot of new Pages stuff announced. What things from today's announcement is there anything that you all are targeting is like, yeah, we need to integrate this or how are you feeling about those announcements today? Yeah, for sure. Some of the greatest announcements that I think are just kind of more around the whole build pipeline for Pages and some of the improvements there. I know that's a that's massively improved, it used to take five or so minutes to build stuff, and now it's 10, 15 seconds, which is always a huge, huge win for that kind of developer productivity. Improvements as well to things like logs and stuff like that, which is always helpful for us. A lot of that stuff is just improving the developer experience. I know there's been a couple of again, just a lot of really nice to have features like the ability to push code about triggering a build with like spy and stuff like that, which is always awesome and there's now branch config stuff. So you can like if you're just doing random stuff on a branch you don't really want to build yet, you can kind of just exclude that from being built. Like if you're in the middle of doing something and you just constantly, I don't know for a particular reason or you have some automated like image pipeline that's resizing and you don't want to do a pages build for every single thing like that. You can now do like actual builds being excluded from, from the entire build pipeline, which is really, really cool. So for us, a lot of the kind of improvements announced today are just a lot more around the whole like developer experience of how everything builds and kind of just kind of works together. And I know there was also direct upload today, which is really, really cool for just kind of showing up like, if you want to handle this yourself still, you can kind of just package up all your assets, all your functions. And Intergraph happen to actually connect to Git Provider, which is really, really quite cool if you have kind of your own control over that thing, if you want to just maintain all the actual code yourself and not have to pretty much hand over the CI like parts, that's really, really cool to see. But yeah. Yeah. All that stuff is super exciting. Generally outside of like Pages, what other things are you excited about for Platform Week? Because you're I mean, you're a Workers user too, right? So what things are exciting either the. For the exciting thing that was announced today. I think yesterday is service bindings now being generally available, which is really, really cool. There were a lot of kind of limitations before about how you could have Workers talking together and things like that. And now it's just so much more simple to have Workers kind of talking together and as well the kind of announcements to how that's build as well as changed so that you haven't got to worry about like paying for additional time that you aren't using and things like that. So we're very much excited about what these service bindings, that's really, really cool and especially on Monday as well. One of the things that really excites me is that whole kind of web interoperability standards announcement. Sure, yeah. Finally having kind of some standardization on these kind of edge platforms like Cloudflare and Deno and things like that is really, really exciting. And being able to kind of have some kind of standards around that is really cool. Yeah, I completely agree. And I think actually you like if you've been working with Workers enough, like probably compared to other people you very much understand like Workers for a while was kind of its own world of like, here's how you write JavaScript here. It's very different from other providers and stuff. So I think it's yeah, I'm really excited for that as well. It's nice to see going back even to the idea of like when I first started here, like it makes me reflect on just how, how the world has changed since like I wrote my first Worker and I was like, What in the world this is like so different from what I used to do? And it's cool to see that like kind of full circle, like contributing back into the wider ecosystem and stuff like that. So yeah, that's super cool. What I guess we're, we're kind of running out of time here like we only have about 2 minutes left. What would you recommend? Like, how do people get started with Pages if they want to do something similar to what you did? Like what's how did your team get up and running and what should they check out? I think generally now with the direct upload, if you've already got kind of a build process going and the kind of static site that you are already kind of creating or even a stack site, just try it, just upload it into direct upload and see what happens. Because if it's just static files, it's probably going to just work and you're going to get all the performance benefits of being on edge. I think functions are really, really cool. There isn't really anything else super like it. So it's definitely there's a learning curve there for sure. But I think once you can kind of get past that and you can kind of start integrating some more kind of. Dynamic pulse into those kind of applications that were previously static allows for a lot of really, really cool things. It's just a matter of kind of getting familiar with the platform. But I think direct upload and I know with Wrangler too with the CLI for that makes that really, really easy now. So I think that is going to be a really, really great way for people to kind of jump straight into Pages for the first time. And I'm excited to see kind of what everyone builds for that. Totally. Yeah. Yeah, that's a really good point. Yeah, it's I think it's a great time to between fast builds and stuff to like a lot of the performance stuff has gotten a lot better. So it's just it's a great time to really get into the platform. So believe it or not, we have been talking for 29 minutes, it went by so quickly in the last 30 seconds or so. Where could people find you online and what kind of stuff you want them to go check out or on nodecraft. Yeah, sure. If you can find me online, I'm sure pretty much everywhere. But for kind of what I'm working on, it's just nodecraft.com. And if there's any questions, obviously just reach out to me. I'm very happy to kind of follow up. Nice. I have a Minecraft server on nodecraft. I can highly recommend it. It's a lot of fun. Cool. Well, thank you so much. I appreciate you taking the time. Good luck with your migration, your next Pages project. And yeah, thank you, everyone. And enjoy the next talk. Thank you very much.