Hacker Time

Hello and welcome to Hacker Time, the number one security show anywhere in the world and the number one security show here on Cloudflare TV. I'm Evan Johnson. I work on Cloudflare's product security team. And today we are picking up where we started last week with a slight tweak. Last week, we started talking about building a suite of canaries or honeypots and trying to get those set up and built. And today I'm going to add a little scope creep. We're going to start doing this all over a new, all over the Cloudflare Zero Trust Terminal is what we'll be using for our development. So we're going to be building these in the browser live and I'll be showing you for a little bit the how the Zero Trust Terminal works and doing some exploration myself to make sure that I understand how it works. And then we'll set up our dev environment and we'll start building our honeypots. So it should be a lot of fun. And the show should look a little something like this. We're going to go over the Zero Trust Terminal. We're going to go over my development machine I've set up in the cloud. We're going to try to set up our development environment. And then if we get all of that, that way, if we get through all of that then I'll set up a little go. It should be a listener web server. That will be the basis of our very first, our very first honeypot. So that should be about 30 minutes. And so without further ado, let's get started. So the Zero Trust Terminal, if you have no idea what I'm talking about, you should read this blog post written by my esteemed colleague, Sam. He is the director of product for Cloudflare for Teams. And you can see it live in action in this blog post. There's a nice little video of Sam using this browser to type in commands and do all sorts of stuff. And there's an architecture diagram here where you can see that it's tough to really go to dive into everything on this architecture diagram. But basically the idea is you can connect from your device over browser, Cloudflare's edge and our servers all around the world will enforce access policies and rules, integrate with your identity provider, and then connect you over SSH to these servers over a secure tunnel. And I'll show you what this looks like in practice. And the blog post is fantastic. Give it a read. And what it functionally looks like is this browser in your terminal that Sam has. This terminal in your browser and not the browser in the terminal. Browser in the terminal, very cool though. You'll see that this is him in Chrome here at droplet.townlake.info. And I've taken the liberty to set this up myself. So I will be connecting over ssh .ejcx.dev and you'll see it's broken right now. So let's get it working. So I also have where I'll be doing my development, where we'll be connecting to is this instance in the cloud, this dev box, and it's currently stopped. So I need to start it. And once it starts, this whole thing should work. And I'll explain why. Let me make sure my terminal here is ready to go so I can securely copy and paste my key. Okay, this is running and we want to go to ssh.ejcx.dev and we're getting prompted. I'm gonna log in as root. You should probably not use your root account for regular development. Cat, shp, ppcopy, don't worry. I'm gonna expose just the very end of this SSH key here and then I'll rotate it after the episode. So I don't think any of you will be able to hack this server in the next 20 minutes. And if you can, more power to you. So yeah. That should have worked. This is not good though. So it looks like I've got a bunch of extra new lines here. All right. I'm gonna do some info here. I'm gonna flex and show my SSH key. No, this should actually be working. Okay. Not sure why that's pasting funny because the SSH key looked good. It just was pasting funny. No worries. We can sort it out. There's just all these funky new lines here for no reason. Yep. And probably should have tested that before the episode, but easy enough to debug live. I think, wow, this goes on for forever. I think this should work. Let's try it. Oh no. What did I do? Let me try one more time. I think there's new lines here. Yeah. No. This is so bizarre. I have no idea why this key isn't working. Let me try to paste it from my one password here where it's stored. This is very unexpected. That is excited as well. Truly no clue what's going on here. Why can't I log into... Why is this key in the wrong format? Wrong passphrase or invalid, unrecognized private key format. I promise this is a private key. It looks like a private key, but there's something wrong with how I'm copying and pasting this. What I'm going to do is instead, I'm going to regenerate a key. So this key is no good anyways. So what I'm going to do, let's do it this way. SSH key gen. Oh, you know what? This is the wrong key. I can't believe I just spent all that time on the wrong key. Make sure when you log in, you use the correct credentials. Let's try this again. Still not working. Nope, that was still the wrong key. This is the correct key. This is what you get for storing keys in one password. Okay, this should actually work. All right, nothing like some debugging in the morning. Okay, let me do that one more time so you can actually see it. You go to your website where you're hosting your terminal. And when you log in, type in your user, and it'll prompt you for a private key and a password. If you use the correct key, it will actually work. But you can also type in a password if you're doing password-based auth, and I'm going to have to rotate this key. All those other keys that you just saw, they've been dead for years. I don't use those anymore. But this key, I actually will need to rotate after this episode, and I will do that. So then you just type submit, and boom, you're in a browser, and you're in a terminal in a browser, and it looks like a regular terminal, which is kind of impressive. And it's pretty snappy. It doesn't lag very much. And if you look about what's going on in the, here, in the network tab, you can kind of see this whole handshake about how this works. So let's dive in one more time. I'm going to refresh the page. There is a WebSocket connection that got opened. And over that WebSocket connection, you can click on messages, and there's a bunch of messages. And this is actually the SSH protocol over a WebSocket directly getting piped to my server. And the way it works is the, on this server, we are running, my cat really wants to hang out with me, so, she's going to sit on my lap. On the server, we are running Cloudflare -D, system control status, Cloudflare-d .service, Cloudflare.service, Cloudflare-d .service. We're running Cloudflare-D and the Argo tunnels, and the Argo tunnels product from Cloudflare. And so Cloudflare -D, I'll try to explain this, but it's pretty complicated. Cloudflare-D runs on the server at the origin, reaches out to Cloudflare's network and opens a persistent connection. My browser is opening a WebSocket connection that's authenticated with Cloudflare access to Cloudflare's edge. And then sending SSH traffic from the browser directly over Cloudflare's edge. It's going all the way securely to the origin server to Cloudflare-D. And then Cloudflare-D is taking all of those bytes and just streaming the bytes over to the SSH daemon. And basically streaming bytes in one side, reading into the SSH daemon, responding with the bytes from the SSH daemon. And it doesn't have to do a lot of thinking. And it's really amazing because it's authenticated here at Cloudflare's edge. And then you can have server authentication of the key or the password or whatever you're doing. So even though I exposed my key here, you can't even access the server because it is behind Cloudflare's network. So the risk of you being able to actually use that key, very minimal. So it's really fantastic. And so the usability though of this browser is pretty awesome. So let's open up a new TMUX session. I'm a big TMUX and VIM person for my development environment and let's switch user to Mike. So the E user and let's go to... I already have the Go installed on this machine and everything. And it looks just like any other... It's pretty amazing. It looks like any other terminal session. And I don't notice any issues. So I am going to do two things. I'm gonna get my TMUX config. No, maybe I'll wait on the TMUX config. Next steps. I'll show you just a little bit about what the development, what the process for setting up Cloudflare V was. And it was pretty straightforward. So setting up this machine was pretty straightforward. I'll show you my bash history. I can basically did some curling, set up Go. You'll see that I had to get this dash L thing right for curl to follow the link. With Cloudflare V, I did a bunch of debugging because it's been a while since I've used Cloudflare V and figuring out the actual syntax of the command line but it was pretty straightforward. You can see in a matter of 10 commands, I figured it all out and got it working. And you'll see that there's a config file which I'll show you shortly as well. Okay, maybe there's a few more commands in 15 but it ended up working pretty well. And pretty quickly, I spent probably less than an hour on getting this working end to end. And then in Etsy Cloudflare V config, I installed a pretty simple config file here where I've got my credential file here which is allowing Cloudflare V to authenticate with Cloudflare as these host names. And then these host names, I've got ssh.ejcx.dev which will forward for the SSH connection to port 22 for the SSH daemon. That's what we're connected over right now. And then for web development, if we run a server on port 80, we'll be able to see it. So maybe we should do that and get that working end to end. But this whole thing is still pretty amazing to me because it just kind of worked right out of the box once I actually used the right SSH key. Okay, so let us start. We've got our go project. Let's RM these tests. Let us make a new directory. I'll do github.ejcx cftv.honeypot will be our project name. Is it dash L or dash P? Yep, that works, dash P. So make the entire directory tree that doesn't exist yet. And we will start with a, let's just start with main.go, package main. Let's get a little HTTP listener on port 80 running. And then we will make sure that we can connect to that and actually see the thing over. Since we'll be developing, this is a great Cloudflare D is a really great tool for your development environments, because I mean, when I'm developing, I introduce a lot of bugs into the software and you can make sure that access to that dev environment is authenticated. So if there's security issues or like just the whole thing's broken and you don't want people to see it, you can make sure that with Cloudflare D, being the way you access your dev environment, that it's all authenticated to get there. So I'm going to go lang HTTP listener example. This is going to be fantastically easy because I'm just going to copy and paste. Listen and serve. Good enough. The only thing missing here is to be in the main function. And we're going to need some import statements as well. Some of my spacing got a little messed up here. This looks pretty good though. I don't need foo and bar. I just need one really. So let's get rid of that. Let's just do this and let's do, hello, the path that looks great. And then we also need to import this stuff. Is it net HTTP? Yes. Import. And log. And I don't know what package HTML is. Is it just raw HTML? Is it like text HTML? Let's see. HTML. Go lang HTML package. I need to get go import set up here properly. Yep, just regular old HTML. Let's go with that. And actually we want port 80 because that's where our tunnel is. I could move the tunnel, but mind. Okay. Go run main.go. Let's try it. Exit status one. Oh yeah. We need to be root. So let me exit. Let me. So I'm root now. This is in home e source. And port 80 already in use. What's running on port 80? Oh, I have NGINX running on port 80. So you'll see, I just have the default NGINX page here. So let me spin that down. Now it should work. And voila, hello slash. All right, it works. That's basically exactly what we wanted to see. All right. Well. What's next? Let's check our agenda here. We went over the Zero Trust Terminal. If you have any questions, feel free to reach out to me about it. And it's something that's under active development. It's still really new, but I think it's really cool. Dev machine in the cloud. Oh, one final thing is I should probably show how all of this is set up in Cloudflare. I showed you the dev machine in the cloud. It's really nice having a dev machine in the cloud because like I've always wanted to use an iPad or something really minimal, but also I don't have to worry about switching laptops or anything. I just have to make sure that I have the right SSH key with me and it's really simple to log in. And you can have a really beefy, depending on what you're doing, it's really nice to have like better network connectivity from your dev machine. It's really nice to have way more RAM, not have all like Chrome running and eating up tons of RAM on your machine while you're also trying to do some really heavy lifting in your code. So I like to have a dev machine in the cloud, especially writing server and system software, be a little more reflective of that. So we covered that. And then let's go over the Cloudflare config quickly about what this actually looks like in Cloudflare. So I'm logged in, I'll probably be logged in to my Cloudflare.com email address. Nope, this is my Cloudflare account. And you guys can't see this, but you can see my six digit OTP code. Actually, I might have, I used my security key to log in. And I have an order of a bunch of accounts and we're using ejcx.dev here. So to get all of this set up in Cloudflare access was pretty straightforward. I have my identity provider and one-time pin set up where it can email me a one-time pin to access this server and prove that I'm, I have my email address control over my email address, which is granted access here. And here's my dev machine. That's really all that I've got here. An access policy over my dev machine. And I should probably go to the Teams dashboard. Let me look at this in the Teams, Cloudflare for Teams dashboard. And you will see my access application for SSH ejcx.dev here. All I had to do to enable this, to enable this was within settings, enable this browser rendering, which is a beta feature, which means that it will render SSH terminal sessions for this application in a web browser. And I will be honest, I'm not exactly sure how they know when to render SSH connections versus when not to. I wonder if something's happening behind the scenes with Cloudflare D since you register the protocol as SSH when you create the Argo tunnel, but it's very interesting. This was easy to set up, but literally a click of a button to be able to turn this on. And then a couple of cut and paste from the blog and from docs and I was ready to go. Okay, so the last thing that I want to do, maybe not this week, maybe not next week. I want to get my config file set up since if we're going to be doing a lot of Go programming, we've got to have our Go imports, our Go formats, our Go tools like that. And then also my Tmux, I have a very particular Tmux setup that I like, which I can get from this gist. So this is my Tmux conf. It's pretty simple just for resizing and really quickly and let us set that up. So I will do it and logged in as root. Is, I wonder if you can do it in Etsy like this. I don't know, honestly. Let's kill this session now. Yep, okay. So that's weird. Oh, yep. So my Tmux session, it's optimized for like quick resizing of panes and then I can like switch around quickly and switch between windows easily. So that's what, that's the way I kind of do my development. And then my Vimconf is in here as well. Don't have anything secret in here. Absolutely nothing sensitive. So I will probably get this set up at a later date just because one thing that I always get wrong is setting up the go tools with, I'll install this as well. Go, then go. So my VimRC is very, very, I'll show it next episode, but it's very, very empty. It's usually just VimGo for all of the Go programming I do, which handles Go format, Go imports. It handles all of the Go stuff that you want to do. It'll automatically format for you when you save the file, automatically do imports, remove imports you don't need, which is really nice. And it will also tell you when you have like syntax errors and stuff. So I always install that. And then besides that, I have in my VimRC two modes, which I'll show you kernel mode and regular mode. And it's for switching between code where people have strict tab requirements, where like you're committing a kernel patch or something. And then one where tabs are spaces. So where I press tab and it's automatically spaces. So that's really all the magic that's in my VimRC with some like basic settings, syntax on all of that. And so all the kernel, like the regular switching between the kernel and not in my VimRC really easy to set up. But I always struggle with VimGo and pathogen. And I don't use many Vim imports, but I'll get this set up and show that next episode and we'll get started. Hopefully next episode as well, we can implement. Next episode, we can show off my VimRC, which I'll be very proud of. We will be able to take this little net listener, start to abstract it out so it can handle more types of honeypots. So SSH, MySQL, whatever we choose, and also separate it out so that we can start doing notifications with Twilio via SMS or whatever we want. So thank you for joining me. Remember to use the right SSH keys and I will see you next week.