Hacker Time

Hello and welcome to Hacker Time, the number one security show in the entire world. I am your host, Evan Johnson. Thank you for joining me. I'm from Cloudflare's product security team, proudly helping build a better Internet. And today, I wanted to talk a little bit about a project that I worked on a long time ago, about two years ago, and it's still holding up really well. It's still very useful in today's world, and not a ton of people know about it. So I wanted to share it with you. And here we go. So the project is a command line interface for Cloudflare. It's called CF. And as you can see, I worked on it three years ago. It's not quite three years ago. GitHub's rounding up just a smidge. It is about two years ago in six months, just passing six months. And so it's a command line interface for Cloudflare. And I've got all this fancy logos and everything, and live demo GIFs of how it works. And it was built before, if you're familiar with Wrangler or anything like that, that are command line interfaces to other tools that we have at Cloudflare, or Cloudflare D, it was built before anything like that. And it has really held up in terms of it still works today. It's still very useful. And so I wanted to talk about it and share it with you. And hopefully, it's something that you learn something from. And also, I think that there's some interesting design decisions to talk about, and security trade-offs to talk about. So here's our agenda today. I want to talk about what it is, why I built it, some of the design, how it works, how to get it set up, and then using it. So without further ado, I'm going to jump in. You can find the code at github.com, CF, and trying to stick to our agenda. Why I built it, I was actually between jobs, and I wanted a project that could be scoped to exactly one week. I wanted to not spend more than a week on it. So I wanted to build it really fast. And I wanted, and once I had the idea for this, I wanted it to be something that could stand the test of time. So something that would continue working for a long time, that I could continuously add to without much work, which I haven't really kept up with, but that's a different story. And I wanted to be opinionated about how to store the credentials, since there's so many there's so many different CLI tools that store credentials and do it really badly. And so when I set out to do it, I really wanted to use auto-generation, which I had no idea how to accomplish. And we'll talk about that. And I relied heavily on the underlying library Cloudflare Go, which is a Go library for anything Cloudflare v4 API. Cloudflare's API is affectionately known as the v4 API, because there's a v4 at the front of the string. And this is a big Go library with lots of support for all of the things in the Cloudflare v4 API. Not quite everything, but the team maintaining this does a pretty good job pushing new updates regularly and all of that stuff. And so I'll give you a tour of the code real quick and I'll start to show you some of the bits of it. So I'll open it up in Visual Studios code. Let us load VS code real quick. And I'll show you my Cloudflare account here. I've only got a few zones. This is my personal Cloudflare account. Got a few zones and I use a bunch of variety of features. So I've got, I talked about csprng.xyz before, which is very much workers, all workers serving tons of requests. I've got this, which is workers, this, which is email, this, which is, you know, just a bunch of random stuff. I've got some test zones here. And I can give you a tour of all of that with our CLI here. Okay. So I've got go source GitHub, ejcx and then cf. Okay. Now we're in, in the thralls of our project here. Okay. So like all good command line tools, go command line tools. I've got a readme. Let's start there. And the readme, Oh Lord, this is a little more than I wanted to see. Preview of the readme says getting set up with it is really easy. So this was back in the day. I actually looks like I don't have a setup command. And it's not quite as easy as go get anymore. So maybe that's a change that we can implement live on, on set. The way to get this set up is after you go get it, you really want to run depth. It requires depth. So today after I ran, go get it, downloaded it, it got an error. And I was sitting here thinking like, Oh shoot, is this not going to compile? And it was really easy after that I ran depth insure and, and it compiled. And then I ran go build. So it from the home directory. So it was actually really easy to, to get it set up, get it built. Depth is a little dated now. And now people are using go modules and all of that stuff. I should probably migrate it over at some point. But overall, it was pretty easy to go from uninstalled to working. And within the code base, it's structured into a bunch of auto generated code here. So auto generated dot go is kind of the brains of the entire program. And it's completely auto generated. I didn't write any of this software. It's auto generated from a very hacky program that I wrote that actually seems to work pretty well. So it's huge. This thing goes on for forever. It seems like a few thousand lines, 2,700 lines. And you auto generate it with this program, auto generate dot go, which is run from the make file. So from make, when you run make CF, it will, it will run this generate command, which will run go, run auto generate go, which the way auto generate works is it will run, it will, well, let's take a look. It uses this definitions file. And we'll use the, use a very hacky toml definition of each command that the command line supports and translate it into a call to that Cloudflare go library I showed you just a moment ago. And so it's pretty interesting how it all works. I mean, it's kind of hacky, but it actually seems to work pretty well. So the way you add an actual command is you go in here, you just write command, and then you copy, I usually copy and paste a lot of this. And then I, you have to get the right options. So these are arguments to the, I'll actually show you, I'll diagnose one completely. Let's do DNS, something related to DNS, create virtual DNS. Let's go to the top. I did DNS pretty early. Okay. So let's do list DNS records. No, edit DNS record. Let's go to edit DNS record. This is a command that is cf dns. So we have all of our commands here that we can run, and we want to do edit DNS records. So first let's list DNS records. Actually list is a good one to do. So list requires a few arguments. One of them that's required is zone ID. So you must have the zone listed for what zone ID. You'll see required flags, zone ID not set. And so we need the zone ID. So let's get cf zones list zones, plural. Plural is never the right way to do things. And then, oh wait, I have a special one, get zone, cf zone, get zone by name, get zone ID by name, I believe. I have a special, get ID by name. Okay. So I have a special command to get the zone ID by the name of the zone. And let's do ejj .io zone name. So here's my zone ID for ejj.io. And then we can list, we can do DNS list, DNS records, zone ID. I believe this will actually work. Okay. So here are all my DNS records. I've got a lot of junk in there. As you can see, I've been testing things and trying to break things. And let's, so here's the command cf DNS list DNS records. Let's actually dive in and show you how list DNS records works. List DNS records. So list DNS records is described up here as a sub command of DNS. And then list DNS records. It's pretty simple. There's a few options, but you basically just say the command name is list DNS records. Here's the description. This is the list DNS records v4 API name. This is the function within Cloudflare Go, the library that we're calling. So I'll show that to you. Go docs, Cloudflare Go. Let's trust Google search to help us find what we're needing here. And the reason why this is important and why it's a string is because we directly take this in the auto generation process and put it into Go code. So there's going to be Go code somewhere in autogenerated .go, which says in this giant switch statement, list DNS records, which calls within API, I believe, list DNS record right here. So just a little function I'm calling API DNS records. Oh, I'm sorry. I misspoke here. Within autogenerated code, within the toml here in definitions, list DNS records corresponds to this function in API.go that then I actually had to handwrite each function here that calls that calls into the Cloudflare Go library. So DNS records is handwritten. So the call is to DNS records right here. So end to end, let me walk through that again, because it's kind of confusing how auto generation works. You say list DNS records is your command. You give it a v4 API name list DNS records that is used for auto generating this giant file, which calls, which will automatically set up this list DNS records thing and call into this main function, which we don't actually, I'll show you it's a giant switch statement. Func main. Oh boy, it's going to be at the very bottom. This is main giant switch statement. Main is probably a little up here. Giant switch statement, which then calls into our API here, which we handle all the records and arguments that we need to handle and then calls the Cloudflare Go thing. So kind of confusing toml to auto generation to just a little stub that I have to write in order to call into Cloudflare Go. So all of the command line stuff is what's auto generated. All of the usage of Cobra, the Golang Cobra SPF 13, this command line argument framework and command line framework that's very, very useful for building powerful and rich command line, command line tools. They don't really have very nice looking docs here, but I can promise you this is a very powerful, very powerful tool that gives me all of the, it's what does all of this structure within CF. So there's a lot of auto generation. That part's kind of confusing. I don't expect many people to contribute to that part because it's kind of a Rube Goldberg machine of a make file, some toml, some of this, some of that. It was a weak sprint of programming. And so I don't expect it. I didn't expect it to be super clean, but it actually works very well. If you ever do contribute and you have trouble, please reach out. I'm happy to help, but let us show you. So list DNS records. This is an example. And one thing to note because I needed consistency across all of these commands and we have a lot and Cloudflare's API, there's a lot of the same types of arguments that are used for auto generating and to make things just consistent. I opted to only use long options, long command line options. It's a little verbose, but overall it's very clear what's happening. It's not so verbose. It's not so bad, but it can be annoying because a lot of our API calls require a zone ID or account ID. And it just means our account, our most, most commands you're writing are going to be pretty long because you have these big zone IDs that you need and big account IDs. So that's very interesting. One very important security design decision I made is with regard to the credentials. So all API, all command line tools that talk to an API need credentials. They need an API token in order to actually authenticate and authorize, well authenticate who you are and show that you're authorized to read and write certain resources. And so by default, in order to set up the API credential within Cloudflare Go, you run cf configure. And when you run cf configure, it'll prompt you for your email. It'll prompt you for an API key. And this is like a, whatever you type in, it doesn't show up on screen. And what it'll do by default is it'll try to save that within the key ring. It's kind of difficult logic to really show everywhere, but it will try to save. And with the logic within this creds.go file, what's happening is, well, I'll show you the configure command. We can walk through it all. That's not auto-generated. So command, it might be in here, configure. Actually, you can go to configure.go. Okay. Configure.go is how you, is how we fetch the credentials. And so it'll prompt for email and API key. We just showed that right here. And then it'll prompt actually for the origin CA API key. That's a specialized API key within Cloudflare's v4 API for only special functions. And then it will, it will attempt to read from, there's some environment variable usage. You can use environment variables if you so choose, but it'll attempt to load whatever credentials you set in the environment. Let's try to get rid of this, make it a little wider so you can see. And then it'll try to read from your machine's key chain. The key chain is, I'm using the 99 designs key chain that's from a few years ago that works very well, especially on Mac books and on Linux machines with their key ring. And it works on windows too. It's very cross-platform. And so in this day and age, most systems support the key ring API that they're using here. And each operating system, the specifics of their key ring is a little different. Every laptop you buy is going to have different security, hardware security technology within it. But by default, especially in a world where so many developers use Mac books and Linux, which will support the key ring without much difficulty, by default, you should try to save things within the key ring. You should try to put these credentials in the key ring. And the way most command line tools do it is they'll just take your credentials and put it in your home directory in some hidden file, like dot credentials or whatever. And that's, for example, what the AWO CLI did for a long time. I'm not sure if they still do. They probably do. And that's an unsafe default. And so having a safe default, even if it only works for 80% of people who try it, is better than an unsafe default that's unsafe for 100% of the people who try it. So I opted for this. And so when you run CF configure, I'll show you here, it will save it in the key ring. But if you get into problems with it, you can run CF CF, no key chain, and it will attempt to not store the credentials in the key chain. It'll just store it like the AWS CLI does within the hidden file in your home directory. But additionally, this tool does support reading from environment variables. So if you set your CF, um, whatever's in the key chain, if you, you can overwrite it by setting these two environment variables, CF API key and CF API email. And so I'll show you CF zone list zones. This will list all of my zones. But if I do CF API key equals SDF and CF API email equals EJJ .io, then it broke. It says that, um, that I couldn't authenticate. So this will overwrite whatever's in the key chain. It's a more specific thing that you've run. And so, um, by default, it'll, it'll, um, if these are set, it will try to, uh, it will just ignore whatever's in the key chain and it'll opt to use what's in the environment. So, um, if you're, I mean, if you take one thing from this 30 minute session, please, if you are making a CLI tool, do everybody a favor and do the extra work to opt in users into using the key ring. Um, and then if they get into trouble, it's not so difficult to have a dash dash no key ring. Um, I believe I even have that in the error message. So, um, um, if you have a command that fails, like if, let me, let me pull it up configure. Um, I guess I don't have it, but if you have a, uh, one improvement here is if the writing to the key ring fails, we could always have the error message saying, if you continually get an issue with the key chain and your system doesn't support it, um, you can try the unsafe, no key ring option. Um, so please opt for the key ring. It's a simple thing and goes a long way. Okay. Last, but not least, um, not last, but not least let's check the agenda. What else do we have to talk about? So key chain auto auto-generation I showed off Cloudflare go a little bit. This is really just a wrapper, a big, fancy command line wrapper around Cloudflare go. I didn't do any of the hard work integrating with the Cloudflare API. Um, this is just like a fancy auto -generating command line tool for, for calling into it. Um, getting it set up. I'll show you quickly how to get it set up from scratch. I'll just show you I'll RMRF the entire directory. Um, RM, RF, CF, goodbye. And then, um, we will run go get get hub.com slash EJCX slash CF. All great tons of errors though. And that's because, um, as some of these projects have evolved, some of the dependencies of the project have evolved. Um, things have broken, but within the go, uh, within the depth, uh, because I'm, it's set up with that within the go package lock and toml, it describes exactly what commits to go to, to make these things work. Um, on, in all transparency and honesty, this probably should be updated to both use go modules, to use the latest version of the key ring, um, libraries and all of that stuff. But it's something that I haven't opted to do and it only takes running. So depth. Okay. There's a bunch of commands to run, but the right one is depth ensure, which will make sure that we get all of the correct, um, all of the correct sub dependencies and, and, um, go packages that we need. It'll take a second. I hope I don't have to refill in my, my, my API token. That'd be a real shame because I'm not going to do that live on air. But I wonder if, uh, as this thing reinstalls the new binary, if, um, the old, if the new binary that I'm going to build won't have, um, access to the same key ring as the previous. So let's see. So we have the right, um, sub dependencies. Now it just takes a go build. There's some issues because the 99 key rings that I'm using is very old and it looks like, um, some things may have been deprecated since then, but CF, CF zone lists zones. Always allow. And great. Um, these are my zones. So let's, uh, let's kind of narrow it down. We must be a little more specific than that. EJJ.io. Great. So you see, this is my, um, this is all of the information returned from the list zone API. Um, you see my zone, my, uh, name servers, you see, uh, the, the organization it's in, you see the permissions that I have to it. Yeah. You see, it's a pro website, all of that. I pay $20 a month to have this website. Um, and, um, yeah, it's very interesting. So there's a lot of great stuff you can do with it. I'll just create a, since I'm logged in here, I'll just show you creating a DNS record by hand that it really, really works. So CF DNS creates DNS record. Great. Lot of stuff here. A lot of stuff here. Um, because there's a lot you can do with the create DNS API, everything from MX records to NS records, everything. So we want, um, content. Oh, well first type let's do a record content. Um, one, two, uh, how about two dot two dot two dot two. And then last name. We want two, two, two, two. What did I forget? I need zone ID. Ah, it, I need the zone ID. Luckily I had it on speed dial and zone ID. Boom. We have just created a DNS record. Here's the ID. And when we refresh this, um, we will see two, two, two, two should exist. Here it is. Two, two, two, two. It's a proxy DNS record. It's an a record and it works. It's a very powerful tool for just inspecting things, seeing everything the API has to show and, um, and interacting with the Cloudflare, um, API in a kind of structured and easy way. A lot of guardrails since there's required arguments. And thanks for joining me this week. I really appreciate it. Um, I will see you soon. Bye.