π SSH Command Logging
Presented by: Kenny Johnson
Originally aired on July 1, 2023 @ 11:00 AM - 11:30 AM EDT
Join Cloudflare's Product Management team to learn more about the products announced today during Security Week.
Read the blog posts:
- A bridge to Zero Trust
- Cloudflare partners with Microsoft to protect joint customers with a Global Zero Trust Network
- Introducing SSH command logging
- Zero Trust client sessions
- Managing Clouds - Cloudflare CASB and our not so secret plan for whatβs next
Tune in daily for more Security Week at Cloudflare!
SecurityWeek
English
Security Week
Transcript (Beta)
Hello. Good morning.
Good afternoon or good evening, depending on where you're joining from the world.
My name is Kenny Johnson.
I'm the product manager here at Cloudflare, My specific product is Cloudflare Access.
And today I'm excited to go through our a new feature that we just announced today as part of Security Week, SSH Command Logging.
So really the idea today is I want to do just a quick background on kind of the feature set and the product set that this this.
New idea falls under what products it belongs to within Cloudflare.
And then really I want to spend the majority of the time digging into the specifics of how this product works and showing a live demo.
So starting off really quickly, I always like to set the scene and show this quickly of where of all the different products that we offer at Cloudflare and then where this particular feature sets.
So the area that we're going to be focused on today is Cloudflare's Zero Trust Services, because SSH Command Logging is a key component of protecting and understanding what's going on with your resources that can be accessed via SSH.
So the majority of the time we're going to go ahead and dig into our Zero Trust Network Access solution and how SSH Command Logging works within that solution.
One piece I always like to dig into as well is that the reason that we're approaching and using Cloudflare's network to secure access to various applications is that we're seeing more and more companies need to move away from a perimeter based approach where they're creating either MPLS lines or VPN links across their branches, in between their data centers, and having their remote users use a VPN client to drop into their data center to then be able to access resources across their local network more to a software defined perimeter or SASE, is the analyst term that has become popular as in Secure Access Service Edge.
Basically what this allows you to do is wrap a perimeter around your hosted applications, your Internet based applications and protect your users from both inbound and outbound threats.
Part of this also allows us to inspect and protect traffic over search bound for specific resources within your data center or within your public clouds.
The last thing that I'll say is part of what makes Cloudflare so special is that we've built one of the largest networks in the world.
We had to build that to support our CDN and security products like our Web Application Firewall.
We have over 90% of the Internet connected population is within 50 milliseconds of our data centers across the world.
And everything that I'm about to show you today runs in every single Cloudflare data center.
So it's really fast in terms of latency as well as resiliency. Even if one data center goes down, we are able to route to the to the next closest one.
Awesome. So then diving into our command logging function a little bit more kind of where this sits is it's within our Zero Trust Network Access solution.
And what this allows us to do is we're able to hook into your cell, into your either your data center or your public clouds, GCP, AWS, Azure, any of those tools we're able to set up and proxy SSH commands coming from a user's machine with an agent or a client running on the device, we proxy that traffic to Cloudflare.
And then from Cloudflare, we're able to, as an SSH proxy, pass into that individual machine and using certificate based authentication, it really feels like magic to the user.
However, we're doing zero trust based authentication on whether or not that user should or shouldn't have access.
Excellent.
So what I'm going to go ahead and do is demonstrate how this feature actually works.
So what I've got is this probably looks familiar to folks who use GCP, but this works the same for something that you're hosting in your own infrastructure or any of the other public clouds.
I've got just a basic VM that I've spun up in GCP.
It really doesn't do anything other than the fact that we're going to be able to access it over SSH.
If I go to view network details.
I've got this particular VM up and open at a specific public IP address.
Alternatively, we can also make this available only over private the private IP as well.
I can talk a little bit about how that would work, but for today's demo, we're just going to go ahead and take a look at how this works within it with an external IP.
So the first step is to actually Proxy and make Cloudflare aware of this external IP.
So what I'm going to go ahead and do is I'll pull up my demo account here within the secure web gateway, which is basically the forward proxy from a user device.
I'm able to route all my traffic to Cloudflare and then out to the public internet.
And this is where I'm able to run various policies over that traffic. What I need to do is I need to go ahead and create an audit.
As a rule, in this case, I've already created one for GCP.
So what this looks like is I'm able to save for a specific destination IP.
I want to audit that.
S-h and a piece that's coming in the next few weeks is we're actually going to be able to enable full command logging to do a full session replay.
Right now, what this does is it supports event logging.
So it basically shows ongoing SSH connections as well as duration of the SSH session.
How long was it which and which specific user was logging into that particular sage section?
The piece that we're finalizing is will also capture all of the commands, run on a particular machine, and we'll dump that into an encrypted file that then you're able to download on your side.
The reason we encrypt that is because you can run really sensitive things on Sage and really only you should be able to see what's being run on an individual box.
We don't even want to see that information at Cloudflare and all that's required there.
Is that we that you provide us a public key that then we use to encrypt that data.
So it just drops it into an encrypted file.
And then if I go and look at the actual search logs, this is where I should I'm able to see either active sessions or ongoing sessions, which we'll look at in a moment or past sessions.
So this gives me a basic overview of, Hey, here's my email address.
I logged in to my destination IP and here's the overall session timing for that particular.
Ssa event.
And then in a few weeks we'll also have the ability to download the actual logs and to be able to play back exactly what was run on a given SSH machine.
Great. So now that we have the audit, as the policy stood up for GCP.
The other steps, I'll actually show you what's required from the box that we SSH to.
So the next step I'm going to do is actually SSH to this. And what I'm going to go ahead and check here is I don't yet have the Zero Trust client running on this call.
So if I pull up my command prompt here, let me pull this into the screen.
I've got my command.
Prompt up.
If I go ahead and try to switch to this machine.
It's going to go ahead and give me a permission tonight because it's not seeing the public key, because the public key actually lives at Cloudflare's Edge and I need to be able to proxy my traffic to Cloudflare and then through to the to the machine.
So what I'm going to go ahead and do is I'm going to switch on the Zero Trust client.
I might freeze for a second. Just bear with me here because I'm going to switch my Internet to being proxy to over to via Cloudflare.
So just one moment as I switch this on.
All right.
And I should be back now. All my traffic's being routed to Cloudflare.
So from my home office here up to Cloudflare Workers point of presence up in Dallas.
And then this is where we're actually going to be able to proxy that SSH to this GCP VM.
So I'm going to go ahead and rerun this SSH command now, and this is going to then drop me on to my GCP VM and I can do things like Run ls, CD now you can see I'm actually on this individual instance.
One other caveat that's worth pointing out.
That's a piece we're still working through.
As you saw, I had to add a couple security bypasses here.
Sorry about that.
Looks like I lost y'all for a second, but I am back now.
That was.
That was just a piece I didn't get just blip because I switched on the on the Warp client or on the Zero Trust client.
But here we are.
We're back.
And now I'm asked to fit into this box. Anything can and happens in a in a live demo.
Thanks for bearing with me there.
So you can see I'm on this, this individual machine.
And then if we actually go to the logs themselves, we should see an active ongoing connection to that particular machine.
So you can see that I've got an ongoing connection to that, to that machine, which is which is great.
It's something where I'm able to see across the world all the ongoing connections to individual machines, a piece we're hoping to add in the future, as well as the ability to actually terminate connections to specific machines if you see something anomalous.
Awesome.
So what I'm then going to go ahead and do now is just talk about the setup that's required on the individual VM in order to get this talk, because there's no setup required on the user device other than being logged into the Zero Trust client.
I don't have to mess around with keys. I don't have to put certs onto the user device because we're Proxying all of this traffic from Cloudflare directly to the server that's being SSH'd to.
Awesome.
So let's go ahead and pull up the individual things that need to be done on this server.
And I'll do it via proxy to SSH here. So I'm going to go ahead and jump into my ETC file and then my SSH directory.
So within SSH here, there's two things that I need to add and that's all.
And then this is set up and configured to work.
One is I need to add a root CA to this device and that root CA is generated using the Cloudflare API.
And then the other component is if I go into.
My SSH config.
The two things that I had to switch on to get this to work are public key authentication, which generally most associated boxes are already going to have that enabled.
That's kind of the best practice for managing SSH access. There's a lot of reasons why password based authentication is generally discouraged when it comes to S-H.
And then the only other piece is adding the trusted user keys to the to the root certificate that I added to this particular configuration.
So and that's it.
That's all I needed to add because that, that then tells the end machine what root certificate to trust.
And within the Zero Trust client running on the device.
I have an associated user account, so I've authenticated to my test account under my identity.
I've actually logged in using Okta credentials.
So on the Zero Trust client side, we know my identity and then I've added that or I already had that Unix username on my device, I've just had Tae Johnson.
So we look at it and see Kid Johnson at Cloudflare dot com is.
Is the identity associated with this end user machine.
So then what happens is I make the request.
It goes to Cloudflare's Edge.
Cloudflare recognizes that I am who I say I am.
I'm coming from the right device.
It then uses the public key that lives at Cloudflare's Edge for this account and uses that as a means of authentication into the SSH server using public key authentication with my particular associated username.
And this is a really big deal because it it means that you're not having to pass certificates or keys or anything like that out to your entire employee base.
You just have them enroll into the agent, which you might already be doing if you're using Cloudflare Secure web gateway and you get S-H secure, logged, audited s h to these individual devices.
And all you need to do is, is put a certificate and add it to its trusted key store for that particular device.
So we are very excited about this feature in terms of it's kind of the future of what we're going to be able to add and what it's capable of already today.
So then I'm going to go ahead and I can just log out of this guy and then that just takes me and ends my session and then this will update.
And show the log of the actual session.
That's that's been ended.
And I can take this a lot further.
I can actually in these network policies, I don't have to just configure network based checks.
I can also configure identity based checks as well as device posture checks.
So I can say things like, I need you to be. I need you to be running my instance of CrowdStrike, or I need to have you running carbon black for you to be able to ssh into an individual machine.
Or I need you to be in a specific user group, so I need you to be part of my Okta developers group in order for you to be able to access to my S-H.
So instead of having to mess around with your PAM style controls on your search box and limit down to specific usernames and keep all that stuff straight, you can do this at an IEP level now, just like your web applications, just like you're used to there.
This is kind of true. Zero trust for SSA where you can configure down to individual user emails or individual user groups with device posture checks before allowing access to a particular box set box.
And you can do this at a network overlay layer.
You don't have to go box to box to configure these security policies anymore.
You can do this at layer four and say for any search that I'm going to proxy through Cloudflare, I need to require specific device posture checks and specific users can only access them before they're able to access a given machine.
So there's a lot of power in that in terms of being able to control those specific users as opposed to having to do it either at a firewall level or an individual origin server level.
So we're very excited about this one.
We're going to continue to make it better.
We just announced it on the Cloudflare blog today.
I can actually pull that up and show you guys where to go.
Read more if you want.
I think it's our second or third blog we announced today for Zero Trust week.
So if you go to the blog, you'll see more information about Zero Trust command logging and definitely keep your eyes out.
Over the next few weeks, this is going to start getting rolled out as a beta product more broadly into zero trust accounts.
So we're very excited about this and I can't wait to get this out more broadly for folks to get their hands on it.
If there's specific things that you want to see with this, don't hesitate to reach out.
Don't hesitate to tweet at us to let us know on the on the blog post. We want to get this into folks hands and hear how we can do even better in terms of of SSH command logging.
So again, thank you all so much for joining.
Really looking forward to getting this out in the world and getting everybody's feedback on how we can make this better.
And thank you again.
Thanks for tuning in and thank you for riding along with us during security week at Cloudflare.
With that, I will go ahead and conclude for today. Thanks very much.
We're betting on the technology for the future, not the technology for the past.
So having a broad network, having global companies now running at full enterprise scale gives us great comfort.
It's dead clear that no one is innovating in this space as fast as Cloudflare is.
With the help of Cloudflare, we were able to add an extra layer of network security control by alliance, including WAF, DDoS Cloudflare users can also allow us to keep costs under control and caching and improve speed.
Cloudflare has been an amazing partner in the privacy front.
They've been willing to be extremely transparent about the data that they are collecting and why they're using it, and they've also been willing to throw those logs away.
I think one of our favorite features of Cloudflare has been the worker technology.
Our origins can go down and things will continue to operate perfectly.
I think having that kind of a safety net provided by Cloudflare goes a long ways.
We were able to leverage Cloudflare to save about $250,000 within about a day.
The cost savings across the board is measurable, it's dramatic, and it's something that actually dwarfs the yearly cost of our service.
With Cloudflare, it's really amazing to partner with a vendor who's not just providing a great enterprise service, but also helping to move forward the security on the Internet.
One of the things we didn't expect to happen is that the majority of traffic coming into our infrastructure would get faster response times, which is incredible.
Like Zendesk just got 50% faster for all of those customers around the world because we migrated to Cloudflare.
We chose Cloudflare over other existing technology vendors so we could provide a single standard for our global footprint, ensuring world class capabilities and bot management and web application firewall to protect our large public facing digital presence.
We ended up building our own fleet of proxy servers such that we could easily lose one and then it wouldn't have a massive effect.
But it was very hard to manage because we kept adding more and more machines.
As we grew with Cloudflare, we were able to just scrap all of that because Cloudflare now sits in front and does all the work for us.
Cloudflare helped us to improve the customer satisfaction.
It removed the friction with our customer engagement.
It's very low maintenance and are very cost effective and are very easy to deploy and it improves the customer experiences big time.
And Cloudflare is amazing.
The culture is such a relief.
It is very easy to use.
It's first.
Cloudflare plays the first level of defense for us.
Cloudflare has given us peace of mind.
They've got our backs.
Cloudflare has been fantastic.
I would definitely recommend Cloudflare.
Cloudflare is providing an incredible service to the world right now.
Cloudflare has helped save lives through Project Fair Shot.
We will forever be grateful for your participation in getting the vaccine to those who need it most in an elegant, efficient and ethical manner.
Thank you.
The real privilege of working at Mozilla is that we're a mission driven organization.
And what that means is that before we do things, we ask what's good for the users as opposed to what's going to make the most money.
Mozilla's values are similar to Cloudflare service.
They care about enabling the web for everybody in a way that is secure, in a way that is private, and in a way that is trustworthy.
We've been collaborating on improving the protocols that help secure connections between browsers and websites.
Mozilla and Cloudflare collaborate on a wide range of technologies.
The first place we really collaborated with the new TLS 1.3 protocol, and then we followed it up with quick and DNS server HTTPS and most recently the new Firefox private network.
Dns is core to the way that everything on the internet works.
It's a very old protocol and it's also in plain text, meaning that it's not encrypted.
And this is something that a lot of people don't realize. You can be using SSL and connecting securely to websites, but your DNS traffic may still be unencrypted.
When Mozilla was looking for a partner for providing encrypted DNS.
Cloudflare was a natural fit.
The idea was that Cloudflare would run the server piece of it and Mozilla won the client piece of it.
And the consequence would be that we'd protect DNS traffic. For anybody who used Firefox.
Cloudflare was a great partner with this because they were really willing early on to implement the protocol, stand up a trusted recursive resolver and create this experience for users.
They were strong supporters of it.
One of the great things about working with Cloudflare is their engineers are crazy fast.
So the time between we decided to do something and we write down the barest protocol sketch and they have it, running in their infrastructure is a matter of days to weeks, not a matter of months to years.
There's a difference between standing up a service that one person can use or ten people can use, and a service that everybody on the Internet can use.
When we talk about bringing new protocols to the Web, we're talking about bringing it not to millions, not to tens of millions.
We're talking about hundreds of millions to billions of people.
Cloudflare has been an amazing partner in the privacy front.
They've been willing to be extremely transparent about the data that they are collecting and why they're using it.
And they've also been willing to throw those logs away.
Really.
Users are getting two classes of benefits out of our partnership with Cloudflare.
The first is direct benefits. That is, we're offering services to the user that make them more secure and we're offering them via Cloudflare.
So that's like an immediate benefit that users are getting.
The indirect benefit that users are getting is that we're developing the next generation of security and privacy technology, and Cloudflare is helping us do it, and that will ultimately benefit every user, both Firefox users and every user, the Internet.
We're really excited to work with an organization like Mozilla that is aligned with the user's interests and in taking the Internet and moving it in a direction that is more private, more secure, and is aligned with what we think the Internet should be.
Hi.
We're Cloudflare. We're building one of the world's largest global cloud networks to help make the Internet more secure, faster and more reliable.
Meet our customer Neto.
- The thing that used to keep me up at night was security.
Cloudflare helps to mitigate a lot of those fears.
It actually is the front line for our platform and actually looks after pretty much all of the security as well as helping us on the cost side, as well.
As one of Australia's leading e-commerce platforms.
Neto powers the shopping experience for thousands of online retailers.
My name is Justin Hennessey.
I'm the VP of engineering at Neto.
Neto is one of the biggest ecommerce platforms in Australia.
Our platform receives between 85 and 90 million requests per day.
We have about 2800 merchants on our platform, single shop owners who are just trying to sell online all the way up to quite large organizations who do multi-warehouse sales.
In the landscape that we are now in, with cyber crime being as high as it is, the threats that hit our platform on a daily basis, it's really important to have both internal expertise and really good relationships with technology partners.
Neto first came to Cloudflare to streamline the process of securing its merchant sites.
Using Cloudflare SSL for SaaS, Neto automatically provisions and manages security certificates across thousands of its customers' vanity domains.
SSL for SaaS is essentially the primary driver of why we moved to Cloudflare.
We have a very complex onboarding process and part of that is issuing certificates to customers.
Cloudflare allowed us to make that a completely automated one-click process.
Anybody in the business could onboard and go live with the customer. Soon, Neto found additional opportunities to leverage Cloudflare platform for enhanced security, performance and reliability.
The two major things that we've really embarked on this year around Workers and AI bot management.
Cloudflare bot management is something that we've just recently turned on.
In its first day, we were able to block 2.4 million requests, and obviously that has a pretty significant cost effect over time.
Cloudflare Workers is actually quite an exciting piece of technology.
It's really allowed us to be quite creative about how we solve different problems.
I would definitely recommend Cloudflare as a as a technology vendor because I believe they offer the full gamut of products.
You can start very small and then you can grow into their feature sets.
With customers like Neto and over 25 million other Internet properties that trust Cloudflare with their security and performance.
we're making the Internet fast, secure and reliable for everyone.
Cloudflare.
Helping Build a Better Internet. Hi.
We're Cloudflare. We're building one of the world's largest global cloud networks to help make the Internet faster, more secure and more reliable.
Meet our customer, Falabella.
They're South America's largest department store chain with over 100 locations and operations in over six countries.
Well, I am a Development Manager for Falabella.com.
My name is Karen Tiwari.
I work as a lead architect in ecommerce at Falabella.
Like many other retailers in the industry, Falabella is in the midst of a digital transformation to evolve their business culture, to maintain their competitive advantage, and to better serve their customers.
We have a store legacy that we have to adapt to the digital culture a logistics legacy, an operations legacy, a legacy that works very well in-store and has performed splendidly but the challenge now is to transform it.
Cloudflare was an important step towards not only accelerating their website properties, but also increasing their organization's operational efficiencies and agility.
Then, the question of Cloudflare for example, it was not only an IT decision, it was also a business decision In other word s, how much faster can we, how much loading time can we improve for our site?
It means internalizing it as a business metric.
That is, to really understand later that performance, In other words, a loss in terms of time is a loss of confidence.
So I think we are looking at better agility, better response time in terms of support, better operational capabilities earlier for a cache.
It used to take around 2 hours.
Today, it takes around 20 milliseconds, 30 milliseconds to do a cache.
The homepage loads faster.
Your first view is much faster.
It's first.
Cloudflare plays an important role in safeguarding customer information and improving the efficiencies of all of their web properties.
Cloudflare.
For me, a perfect illustration of how to deliver value to clients quickly.
Yeah. That's it.
The big challenge now is to build the culture and lay the foundation to allow the teams or whomever do their job for the next 5 or 10 years.
With customers like Falabella and over 10 million other domains that trust Cloudflare with their security and performance, We're making the internet fast, secure, and reliable for everyone.
Cloudflare.
Helping Build a Better Internet. 80.