The Joy of Terraforming

Hi there. Thank you for joining us today. My name is Chris De La Garza and I'd like to welcome you to the joy of Terraforming. Since this is everyone's first time with us, thank you for your time. Today we're going to go through an initial setup of an existing Cloudflare zone. Right now it doesn't have too much in it, no DNS records or anything like that. Really all we've done is just pointed the main servers on over to it. The idea is we're going to be able to spin up some configuration and put some things in place using Terraform. Now what is Terraform, you may ask? Well, it's an infrastructure as code tool, meaning that you can write things such as server resources or backend resources and everything up from that all the way to your Cloudflare config all in one nice tool and in one nice location. We're going to use an existing repository to help kind of speed up the process and use some predefined things. Let's take a quick look at that and get an idea of what we're going to be working with today. So if I share my screen correctly. Fantastic. We should have in front of us here on the left hand side, we got a terminal where we're going to clone a GitHub repository, the one I just mentioned. And on the right hand side over here, we have that GitHub repository. Let's take a quick look at the readme here. So we're just going to run through these steps real fast and hopefully make some happy mistakes along the way and get those fixed up and then get ourselves with initial config. Just to kind of double confirm, there's nothing happening on that config. We're going to be working on a zone I have called ChrisDLG.com. You may not hear of a zone and think of it, well, isn't that a domain? Same thing. We just have different ways of saying it is all. Underneath the DNS tab, we'll notice right now. So it's got a couple of MX records here. Nothing going on, nothing to point to just yet. Within the repository we got here, we're going to go and clone this guy on down from the Argo Tunnel Examples repository and make usage of this Terraform wonder beneath it. After we copy it down, we're going to go ahead and change our directory or put our terminal into that directory that houses all this information we got right here. We'll copy this Terraform .tfbars.example file on over to Terraform .tfbars. We're going to fill in some information. Not to worry, I'll tell you where to get that information for your own instance, just in case you're following along at home. And then we'll go ahead and run it. After a few minutes, we'll have an application up and running that we can visit via browser and hopefully the ability to SSH into that server because that server is a bit locked down right now. So let's go ahead and get started. I'm going to go ahead and clone the repository. If you're curious as to how you can clone the repository, well, one, you just need to be authenticated against GitHub. I currently am. A couple of other things to note too, I already currently have Terraform installed on my system here. So I'm going to be using Terraform version 15 today. To go ahead and download and copy on over this repository, if you click on the code button up here, I'm using the HTTPS methodology. However, GitHub is pretty nifty, and they give you a few different options on how to pull that on down. So let's go ahead and pull it on down, folks, and get that going. Fantastic. Go ahead and change directory into the Argo tunnel examples directory. And if you notice, there's a few subdirectories underneath there. Some of these are better suited for other use cases such as Kubernetes, and we'll leave those outside of the discussion for today. So let's concentrate on the Terraform repository. Within this, we got a couple of different files, and as I mentioned earlier, I'm going to use the built-in copy command that my workstation has on it. I'm going to just copy this file called terraform.tfrs.example on over to terraform.tfrs. We're going to go ahead and clear out our terminal real quick, and the way I did that was with Control-L on my terminal. Alternatively, you could just run clear and get that going as well. Now, let's take a quick look at the contents of this file and get an idea of what we need to put in here to get this up and running because all Terraform is essentially doing for us is talking to various APIs on our behalf. In this particular example, it's going to be talking to the Cloudflare API to spin up some Cloudflare resources, and for our backend origin, we're going to be talking to the Google Cloud API to spin up some Google Cloud resources. Now, if you use a different hosting provider or have your applications in a different area, then not to worry. You may need to change, of course, some information in here for your specific provider, such as DigitalOcean or what have you, but the overall concept of how this all fits in together should be ultimately the same, and the Cloudflare configuration should definitely be the same as well. We're going to pull from the Cloudflare account some bits of information to fill in here within these quotes so that Terraform knows what to do when authenticating against the Cloudflare API. I'm already authenticated against my Google Cloud instance on the backend or my project, if you will, and you can check that out, I believe, with the gcloud auth command and enter thereafter. If we run that here, we should get some information. Made a little happy mistake right there. No worries. You run gcloud auth, you should get some additional information. It's going to tell us some ideas of what we got going on there. Turns out I wanted list instead. Let's go ahead and run that real quick, and we should be able to see here in just a moment that I'm already activated and up against our Google Cloud. The only thing I need to tell Google Cloud in relation to what we got going is which project I want to run this against. In my case, it's this particular project listed here. I also have a variable in here for where in Google I want to deploy my instance or server or VM, depending on your nomenclature, and what type of VM I want to deploy. In this case, it's going to be an F1 micro. Now, I'm going to go ahead, and I'm going to open up one of my preferred tools, which is Visual Studio Code. I'm going to do that offscreen real quick because I need to fill in this particular file with a couple of bits of information that we probably shouldn't share publicly. Not to worry, we'll see here in just a second that the GitHub repository is told to ignore this file, so we don't commit it to a versioning tool or anything like that, and we don't got to worry about leaking credentials or letting folks know or bad actors out there know. What our actual information is here, such as the account token. As I mentioned just a moment ago, too, we will go over here in just a moment how to get that information and where you want to pull that on down from. Right now, offscreen, what I'm doing over here is I'm grabbing down my credential information, and I'm saving that into the terraform.tflars file. That way, we have the information in here, and we can go ahead and get this up and running. Let's go ahead and bring up Visual Studio Code. By all means, too, you can follow along at home using the code editor of your preference. Most things shown in here, too, I'm not a Terraform expert or anything like that. You may have a better idea of how to go ahead and approach this. The idea is if you don't know how to approach this and you're thinking to yourself, man, I really want to use Cloudflare and I want to use Terraform, I just don't know where to start. Hopefully, this can give you some inspiration. Right now, I'm pulling up the built-in terminal within Visual Studio Code, and as you notice, it's going to put me within the project already, so we can go ahead and start spinning up some of these resources and getting this rocking and rolling. The first thing we want to do is within the directory or folder where you have your Terraform configuration, you want to initialize Terraform and tell it, hey, I'm working in here. We're going to go ahead and do that, and it's going to pull down some of the backend information that we need. If this is your first time running Terraform and the first time building out any sort of new resources or anything like that, the next command you probably want to run to make sure that everything is up and going, is Terraform plan. When you run Terraform plan, you can tell it to write what it's going to do, which is essentially what Terraform plan is doing underneath the hood, to a specific file. In this case, I'm writing it out to a file called super.plan. We'll see that pop up over here on the left-hand side, maybe your right-hand side, depending on how this broadcast is going, into the same directory here. What this is going to do is Terraform is going to look at the existing files that it's aware of, in other words, everything in .tf, as well as the server.tpl file, which Terraform has some awareness of. We'll go over that here in just a second. We'll see that it's going to go ahead and build out some different resources for us and get us rocking and rolling with our Terraform configuration, which is going to cover both, in this case, Google Cloud, and it will also cover Cloudflare's configuration as well. And that deployment is going to be quick, but I think it's going to take us a few moments. And this guy in my testing here takes about five minutes or so to get up and running, is the running of that server.tpl file we have there, which is essentially just a bash script. And that's going to go in and create all the resources we need on our backend, such as the Docker container that runs our web application, the configuration of the Cloudflare tunnel within that server, and telling it where to point to as its target. We'll notice here right now, Terraform, thanks to the terraform.tfr file, is going on through and applying the configuration as we have it defined within these various files here. If we take a look at the Terraform state list command, we can see that Terraform has spun up some resources for us. And if we jump on over to our Cloudflare dashboard, we should be able to take a look at a couple of these resources right now. Let's do that. I'm going to jump back on over to my browser here. Notice before, we didn't have any DNS information here, aside from a couple of MX records from my preferred mail hosting provider. If we go ahead and refresh our screen here, though, assuming I don't have to re-authenticate to the Cloudflare platform. Well, fantastic. We should see here that we now have two records that Terraform has created for us, one at the apex or root or parent domain, depending on your nomenclature, and another one for SSH. In this case, we're going to also have the Cloudflare tunnel that we're running on that local server, allow us to use it to SSH into that device. Pretty neat, huh? The way that the tunnels work is quite nifty, and we got a number of different blogs on that, which I'll highlight here in just a moment. It better explain how you can use a tunnel to talk different resources, such as HTTP applications, SSH, daemons, RDP, what have you. The idea is with the tunnels, when you first create it, you give it a name nowadays. That's going to give you a uuidtarget.cfrgotunnel.com that you can see name over various endpoints to. And we'll talk here in just a moment about how the origin knows what to do with these various endpoints being sent on over to it. If you wanted to add more endpoints, you can certainly do so. You would just create a name for them and point them on over to our tunnel instance. Right now, if we go ahead and look to see what our backend is doing, it is still going through and building all this out right now. As I mentioned, it takes about five minutes or so for Google Cloud to go on through and apply all the different configurations that we have in here. And while we're waiting for it to do that, let's talk about a couple of the files that we have already existing and how we can add on this to continue to build out our Cloudflare deployment on top of what we have going on. The first thing we're looking at here is the argo.tf file. When Terraform goes to build all this together, it's going to look to see within the directory you run it in. What does it have awareness of? In other words, that's going to be all these .tf files here. And the way Terraform goes through and builds these things out is by using resources. You can use Visual Studio Code, which I'm a big fan of, to give you a better description as to what a resource means or other Terraform utilities such as a data source or what have you. What we're doing here in this particular one I think is pretty nifty and a little bit straightforward, and hopefully you do too. The first thing going on here is we use the random resource and we give it a local name called tunnel underscore secret. And what it's going to do is it's going to generate a random string of characters for us that is 35 bytes in length. That's nifty because in order for a Cloudflare tunnel to get spun up, a.k.a. what used to be known as an Argo tunnel, it needs at least a 32-digit string in order to pull that together. And if we look here, one moment, I'll take a sip. I almost got ahead of myself talking too much. If we look here within our Cloudflare Argo tunnel resource, we'll see that it has a local name of auto underscore tunnel. It's going to pull in from that terraform.tf bars file my particular unique Cloudflare account ID. We'll look at where in just a moment we can get that information. It's going to give this tunnel a unique name called Zero Trust SSH HTTP with underscores underneath it. We can check that out because I have Cloudflare D running on my local machine. And because of that, I can do some pretty nifty stuff, such as type in the incorrect command as mentioned earlier. We don't make mistakes, as Mr. Bob Ross said. We just make happy accidents, and that was one of those right there. In this particular case, we actually wanted to run Cloudflare D tunnel list, and if we do so correctly this time, well, we can see we have the two tunnels here. My tunnel is still spawning up. Right now, we don't got any connections on that, but we will here in just a moment once that server goes through and gets upgraded. And it looks like the fantastic folks on the tunnel team already have a new version of that, and I can run Cloudflare D upgrade here in a little bit to get myself a new version of Cloudflare D. So now that we know why we're given a name, which gives us our UUID target that we're going to see name endpoints on over to, here's where we pull in the secret from this random ID resource right here, and we can do so within Terraform by calling the name of the resource, in this case, random underscore ID, dot its local name, and then any arguments that come afterwards. In this case, we want to use a base64 encoded string, and we're going to do that with base64 underscore STD. Now, as we mentioned a moment ago, you may be wondering, well, Chris, this is great, but where do I get my Cloudflare account ID? Where do I get all this other information to populate into the terraform.tfr file so I can follow along at home and run this with you? If we jump back on over to our Cloudflare dashboard, let's look at the overview tab for the zone we're working on. In my case, christlg.com. Of course, yours may differ. We come to the overview tab and come down the right-hand side a little bit. We'll notice under API, we have a couple of bits of information. Well, if you look at the terraform.tfr's example file, one of the first things it's going to ask you for is the Cloudflare zone. In my case, this will be christlg .com, and you want to put that inside of those brackets right there. Sorry, inside of those parentheses right there. The next thing it's going to ask you for is the Cloudflare zone ID, and we have that information for you right here on the right-hand side of the screen underneath API zone ID. Where you would find your Cloudflare account ID is also right underneath that as well. In my case, my particular account is called Production Sites. Yours may differ, but every zone within my particular account here should all have the same account ID, and each zone will have its own individual zone ID. Why use the terraform.tfr's file? Why use environment variables? Why not hard-code this information in so that we don't have to worry about plugging it into some sort of tooling like this or some sort of template like this? Because the idea with terraform is that we can use it like a recipe book, and if we like the configuration that we're deploying here for this one zone, if we want to deploy it in other zones, all we got to do is update our terraform.tfr's file or adjust your environment variables, depending on how you're going about and doing it. Now, let's talk about how, one, we now know that when we run this tooling here, what it's going to first do for us is create a Cloudflare tunnel at Cloudflare's edge with a UUID target that we can send in on over endpoints to, and it'll be ready and waiting for those targets or those host names or information to be sent down on that tunnel to the origin, and then at the origin, we'll be able to take that Cloudflare tunnel and point it to various local applications or things within the DMZ or local network of that server that it can reach. We take a look at instance .tfr. We're making usage of Google's provider, and one thing I do recommend, you may have seen it whenever we were looking here over at the browser real quick, is when you work with terraform, when you're building stuff out, it's always a good idea to have the terraform documentation for the particular provider you're working for up and ready. I oftentimes find myself directly copying from this information and then adjusting it to fit my specific use case. This is a good way to avoid any sort of what have you, mistypings, miscues, those happy little accidents that we've mentioned a few times ago. So within our Google project here, we're giving our machine a name. We're pulling in some of those variables I talked about. We're setting networking tags that are unique to my Google project that say don't allow SSH access in. So right now, we have to rely on that tunnel to get in there, and as soon as it spins up, we'll be able to SSH on in through that tunnel. And the way that we're passing information down to the bash script that gets configured or is being used to configure our resource or our backend VM, what have you, is through this metadata startup script argument we got here with Google Cloud. What this does, it says, hey, look for this file here, which is server.tpl. And then within that file, if you get any of these variables, such as web underscore zone, well, really, here's the actual variable from terraform that you can feed on in to that bash script. And we take a look at server.tpl, which, as mentioned earlier, I'm not a terraform expert. I like bash and I like Linux, but I'm by no means an expert. If you're looking at this and going, man, you could have done a much better job here, I challenge you to go through and do it. This is just to give you an idea of how to get it going, and it can definitely be improved, as can anything, and we can always make it a lot better. But essentially what we're doing when we're going through here is we're going to install Docker using the commands we see here. We make usage of some bash isms, like cure files, to populate a Docker compose file and get that spun up and running. And then we're going to go on through and we're going to install a Cloudflare D instance on this server. We're going to feed it some information and how to authenticate to our particular account, which in this case is using the certificate.json file with the information we see here. This is now available via our API endpoint to make this a little bit easier on yourselves. And then we're going to use this fantastic tooling that the tunnel has called ingress rules. I like to think of ingress rules as a web server's address book sort of thing. It's like if you're using Apache, these are kind of like virtual hosts in a way. When the tunnel receives a host name at its origin, so in this case, web underscore zone will become chrisdlg.com once this finishes up and running. If it gets a host header that says, hey, I'm looking for chrisdlg.com, it then knows to send that information on over to the service that's running on my local host of this machine at port 8080, which is that Docker container I previously mentioned. And here's how we're talking to it for SSH and how we're able to use a Cloudflare deep tunnel to proxy an SSH connection from my workstation on over to the server itself. Now, you may say to yourself, well, this is great, Chris, but I want to get some more information about what we saw and what we got going on on our various resources that Terraform has been configuring for us. And if you recall earlier, I ran a command called terraform state list. And within this command, I always like grabbing information such as we can grab this particular guy here and we can rerun this with terraform state show and then our resource. And once that goes on through, it'll tell us what Terraform is aware of when creating this altogether and give us a lot of bits of information about that particular resource so we can understand what it's getting spun up. Now, one thing we talked about earlier is sometimes we can take a random hosting that isn't explicitly defined here and we can send it on down the tunnel. And that's what this nice little wildcard provides here for us to the built-in hello world service that Terraform has. So let's go ahead and jump over to our instance real quick.tf file. And let's add in another DNS record. It'll do just that. I'm going to use resource. And as you can see here, this is the reason I like Visual Studio Code so much is as soon as I start typing, it goes, oh, hey, I see you're doing that. I know what's up. I see you've initialized Terraform. And because of that, I know what to do with it. Let's go ahead and build out a few things here. We're going to give it a zone ID. And by all means, you can definitely copy this on over from the top and just adjust things as necessary. But for the purposes of this particular walkthrough, let's all write it out by hand. And what that's going to become is supercool.chrisdlg.com. The value, in other words, where that record should point to, is going to be exactly what we see on top of here, which is our Cloudflare Argo Tunnels UUID resource .cfargotunnel .com. This is something I would definitely probably recommend copying and pasting on over just to avoid the risk of accidentally miscuing or mistyping one of those things. If I give it a quick eyeball look at all that, it looks good to me. Let's keep moving on and tell Cloudflare that this is supposed to be a CNAME DNS record. And we definitely want Cloudflare to go ahead and proxy this, which if you're curious about how Cloudflare proxies, what Cloudflare is up to, what we've got going on today, I encourage you to reach out to our sales folks and learn more about our fantastic products and what we've got going on over here. Once again, I'm going to go ahead and run Terraform plan to get an idea of what Terraform is going to do when I go ahead and rerun. We're going to give this another name here, new.plan. While it goes through and thinks we should see it, say, hey, everything here looks the same. You haven't changed anything except, whoa, you added one thing in here, which is this new record. As we can see here, the plan is to add one particular thing. If we come up here a little bit, well, fantastic. We see that the record we added with a local name of other endpoint is about to be applied. So now I can confidently say, let me go ahead and run Terraform apply, new.plan. Terraform is going to look at its existing state file, which big pro tip, never delete, remove, or alter your state file without Terraform being aware of it because all of a sudden you're in a bit of a pickle. But if you have an existing zone and you're saying to yourself, well, this is great for a new zone, Chris, but I want to take what I have existing and I want to use that and pull it into a Terraform configuration so I can make better usage of Cloud Player and my backend origins, I would highly encourage you to read our blog article called Cloud Player's Partnership with HashiCorp and Bootstrapping Terraform with CF Terraforming. Our very own Garrett Gallo has gone through an amazing walkthrough here explaining how to use a tool we have called CF Terraforming to pull in existing Cloud Player resources and make them into Terraform aware through resources and state. If you're curious about what we got going on today, I encourage you to read this blog called Automating Cloud Player Tunnel with Terraform. It's a bit of what we're going over today written by yours truly. And another one here about our new developments with Cloud Player D written by our own Adam Chalmers called Highly Available and Highly Scalable Cloud Player Tunnels. Fantastic resources, great information to better understand how we're deploying Terraform with Cloud Player and how you can use Terraform to augment your usage with us. So cool. We now see that we have our super cool resource. I guess it's been five minutes now. And as always with the live demo, let's give it a shot and see if everything's up and running. If I go to chrisdlg.com, well, fantastic. We are reaching our HD bin. Docker resource will be spun up using that server file. All we really have to do is pull down that repository, change some information that's applicable to us, and we're good to rock and roll. We even added in a new resource, which we should be able to reach at supercool.chrisdlg.com, which would take us on over. Awesome to our hello world instance. Now I mentioned earlier that there is a wide variety of things that we can do with Terraform and with Cloud Player tunnels, which I know this documentation is supposed to be about Terraform, but I like tunnels a whole lot. So we're talking about them a little bit just to give us some clues as to what we can do. If we take a looking at my existing SSH config, I have within there, which I can reach that by using the tilde symbol dot SSH slash config. If we take a look at the top portion of that file, our eyes should be concentrating on is this highlighted section here. I'm saying that if I reach out to the location of ssh .chrisdlg.com, which we'll recall earlier, is where that tunnel is expecting to accept host header request and send the line over to the local SSH statement. If I go ahead and try to initiate that, well, one, my server makes usage of SSH keys and does not use password authentication. So because of that, I tell it, here's where you can find my SSH key. And here's the reason I have Cloud Player de-installed my local workstation as well as the various endpoint. We need it in both directions so that we can proxy this SSH command. And the reason we want to proxy this SSH command is that we look at the access.tf file. We got an access policy in front of this, which accesses our Zero Trust product. That is a nice way to where you can say, Hey, I have my instance here reachable at an endpoint Cloud Player, but I want you to only be able to get to it. If you prove trust through this access policy. And if we go ahead and SSH on at that, we should see it open up a browser window, which it did for me off screen over here. The user that it's pulling in, if we take a look at the terraform.tf bars file, and if you come down and look at it here, it's the email address used to log in, which I haven't mentioned that just yet. Let's jump over to our Cloud Player documentation here. The top right hand corner, we can see my personal email here. If I click on my profile under API tokens, here's where I'm pulling in the token information. In addition to what we talked about, where you can get the zone ID and account information, that should give you every variable you need to follow along at home and get up and running. And I'm going to go ahead and type on in over here to the window that popped up, which let's pull that one over. The email address I use for my configuration, ask it to send me a code. And hopefully I didn't misspell that. If we take a look in over here and I'm jumping off screen to my particular inbox for them, I grab the code, or alternatively could use the link that's provided to me there. Drop that on in, hit sign in, say, yep, I want to go ahead and approve that. And we can see here that said it's successful. If we jump back on over to our Visual Studio Code, we're running our terminal. Ah, I would spin up these GCP instances a lot. Sometimes these GCP instances can make usage of a key on their side a couple of times over. And if you see this error, don't worry. All it's saying is, hey, I see a key in here that I've recognized in the past. I don't really know what to do with it. Can you change that? And we certainly can. I'm going to use my preferred editor called BIM. Alternatively, you could use the SSH key gen command to remove that for a particular known host. And if I jump on down here, I'm going to delete the offending key. I'm going to delete everything with the name PrisTLG on it. And if we go back in and try to SSH once more, we'll tell it, yep, we want to trust that key now and get on in there. And now we are in the remote VM and GOOP file that we spun up. And the way we got to it is via our Cloudflare tunnel. And we can check that out. This is an Ubuntu distribution, and Ubuntu records login and privilege information within var log auth .log. Well, fantastic. Look at this. We're not getting hammered by anybody in this particular instance, because one, we told Google block SSH connections to this particular instance. And two, we've said, if you do come in through SSH.prisTLG.com, you have to prove trust based on the access policy I've written in Terraform up here on the top. So now I can securely have this running and know that the only way folks can get into this particular VM, of course, we may want to have a backup plan, such as getting to it through console. But if we were looking at saying, how can folks get to it normally through a normal SSH flow? This would be our recommended way of approaching that and allowing you to leverage a Cloudflare tunnel on both your workstation and the remote instance with an access policy in front of it to further lock down and say, here's how you can and can't get to me. You can even go through and update the SSH config file or your preferred particular piece of tooling. Maybe you're using RDP to say only accept connections coming from the tunnel, which in this case is localhost. And we can also see here as well, I'm going to go ahead and use some privilege commands to run this. Here we have another little happy mistake as I made a quick misspelling here. We have our Cloudflare D tunnel running on it, doing stuff for us here. And the way it's doing that is as a system service. Fantastic. What a wonderful thing. Cool. So let's drop back out of this with the last few minutes we have left. If you wanted to keep adding onto this configuration to keep building things out, all you need to do is within the same directory you got here, is just add in a new file and call that new file something.tf. So that way you can continue to build this out. Let me see if I can get that done real quick for us here. We need an idea of what that would look like. If I can recall how to, oh, made a wrong and correct there. No problem at all. Fantastic. Here we go. I'm going to drop one in called firewall.tf. I'm going to cheat a little bit and I have pre-written over here to the side some various firewall configurations that we can use to deploy within our Terraform config. We'll get that up and running. Drop this information in here, save it. And if we run our Terraform plan command. We're getting a little bit close, but I just want to show you how you can keep adding on, keep building out your Cloudflare instances and continue to augment your plans with Cloudflare. I know Archie Dog here appreciates it. We appreciate you for joining us today. Thank you so much for using Cloudflare. Thank you so much for tuning in and we look forward to the next time we get to speak with you folks and best of luck to you.