Ask a Solutions Engineer

All right, so welcome to another episode of Ask a Solutions Engineer. I'm Kabir Sikand, Solutions Engineer at Cloudflare joined by Jason Farber.

And Tim, I was like, I'll let them do some quick introductions.

Hey, I'm Jason Farber. I'm a Solutions Engineer Manager.

I work in the Austin mid market segment. And I live here in San Francisco.

Hey, everyone, I'm Tim. I'm also a Solutions Engineer and I work from the San Francisco office in America.

So the purpose of today's session is to take questions that Solutions Engineers receive from the public for this session.

And we we sort of paraphrase them and ask, we look at the collective group, and then we, we paraphrase them and put them together.

And we'll go through each of them and discuss our experiences and and what we think.

And we hope that everybody finds this and as an interesting and creative approach to to our role.

So here's the first question.

When did you first use Cloudflare? So maybe we start with you, Tim.

Yeah, I always like talking about my first time using Cloudflare, because it really is a story of my whole career, right?

When I one of my first jobs I had after university was working for a small web design agency back in Australia.

So if you can't tell from my accent, I am Australian and very excited to be representing us in America.

And in this first job for this web design agency, we were building websites for the music industry, lots of different bands, like the Cat Empire and Cage the Elephant.

And it's a small little agency. When I first started working there, the CEO of the company showed me this little CDN called Cloudflare, right?

I was a bit confused about what I was looking at. But this is fast, rewind way back to the early 2010s.

Very simple dashboard. But the I remember what the CEO said to me was, look at how much money we're saving, and how much bandwidth we're not spending by using Cloudflare.

And at the time, there wasn't even an Australian point of presence.

But indeed, we were saving terabytes and terabytes of images for these websites.

So it was a really good first experience, making these websites faster in the industry.

Yeah, it's like a pretty similar one, I suppose.

My first gig out of university was also at a smaller startup here in San Francisco Bay Area.

And I didn't really know the ideas of what a CDN did, or why it was useful, or why we needed to have someone else terminate TLS on our behalf.

And those are all things that came to light just working with our CTO at the time to implement Cloudflare.

See all the kind of tips and tricks that he used in his previous experience with it, and how we could host our DNS over there as well.

So both of you guys experienced Cloudflare's dashboard when it was really different from what it looks like now, right?

Yeah, definitely. Pretty funky back then. I remember some green, very round green buttons.

It's a completely different platform today. Yeah. Our experience it go ahead, just go ahead.

I recall, one of the greatest challenges we had at this company was serving the website during the announcements of a music festival events, where millions of people across Australia would go to the website to then go and audit their tickets.

And the company had done some amazing work to build extremely responsive servers to meet the demand.

But then one year, we reached out to Cloudflare and said, hey, can we please use you to service all of this traffic?

And Cloudflare was like, yeah, let's do that.

And for Cloudflare, they were like, yeah, it wasn't a problem.

We could just serve all of this traffic without really blinking an eye.

And that was a really hard problem for us to solve without Cloudflare, but a really easy problem for Cloudflare to solve.

Sweet.

My first experience was slightly different than you guys. I was at a competitor.

And Cloudflare would come up commonly as someone who we would compete against from a pre-sales perspective.

So I was one of the first solutions engineers at that last company, and we were selling bot mitigation, bot mitigation, bot detection.

And commonly, customers would say, well, you know, I'm at Cloudflare. Do you work with like to integrate well with Cloudflare?

Do you also do like what Cloudflare does?

And in many ways, Cloudflare, even though at the time they didn't have as robust of a bot management product, would still beat us in the pre -sales.

Yeah, I was just going to ask, I thought you joined us long before we had a formal bot management product.

Yeah, yeah, I did. And when I when I first joined, everyone would ask me about bot related questions.

But our WAF at the time, WAF, we didn't have rate limiting when I first joined Cloudflare either.

But the WAF and the IP blocking, we would pitch to customers ways of utilizing both of those things in conjunction to have a similar effect.

But now we have an actual robust bot management product, in addition to rate limiting and a lot of other neat DDoS features that that account for some of those gaps.

Yeah. It makes a lot of sense. WAFs are really great at mitigating obviously malicious bots.

But automated bot detection is great for things like scraping and not necessarily obviously malicious.

Yeah, one of the common bots that popped up often was bots that would fake Googlebot.

So that was something really common that I saw across almost every customer.

But Cloudflare had a rule in the WAF to protect against that. Because it's a pretty easy thing to spoof and a pretty easy thing to detect.

Cool.

So let's go to the next question. So what does product feedback look like at Cloudflare?

And I think this question kind of goes towards a few of the other ones we'll see today, really about how Cloudflare solutions engineering team works with other teams internally to represent our customers that and prospects that we're meeting with on a daily basis.

So to talk a little bit about this, I think Tim, this question might be good for you to kind of start answering.

Yeah, product feedback at Cloudflare is obviously very broad.

We have such a wide range of self-service customers and such a wide range of enterprise customers.

So there's lots of public feedback we get, people contributing to our community, and Twitter and things like that, public contributions in the open source space.

Where SEs get more involved is with helping bring our enterprise customer feedback and bubbling that up to our product teams.

Solutions engineers alongside the account teams are really the voice of our enterprise customers.

We do act as the technical voice of our enterprise customers to the rest of the business.

So something we work really hard to do is make sure every time we have one of these conversations with a customer, and we have hundreds, we have about 100 solutions engineers, and hundreds of account facing representatives working with about a few dozen products team members.

Every time we have one of these conversations, we want to make sure that we're capturing this feedback and making it easy for the product teams to see in aggregate, and also drill down if they need to see the details.

So we capture about a few dozen points of feedback from our customers every single week, which we bubble up throughout some internal systems and then ultimately feed it back to Jira, where the product managers use it.

Yeah, I think one of the useful things that I've noticed over the past few months is we've really bolstered how we share that feedback internally.

So Tim has been working and a lot of the other solutions engineers alongside him to start to summarize what those trends are within the product feedback that we get every week.

And maybe identify over the course of weeks, hey, we've started to see that there's a lot of feedback around this one product or this one area.

And how can we summarize that so the leadership on the product team and within the solutions engineering organizations know where some top line value is within the feedback that we're getting?

Yeah, one of the things I find really empowering and powerful about getting feedback from our customers is when customers give us feedback on our product, it's not just saying they want more capabilities, it's actually them asking and saying we want to trust Cloudflare with doing something new, supporting an additional components of the business, protecting another section of the business that we might not already be protecting.

That's really humbling to know that customers do want to trust us with protecting more of their organizations and more of their team members with things like our teams product.

And that's what they're asking us. So it's really humbling.

I think we also post our product managers utilize the community pages often to attract some of our current customers who are active there into the beta of some of the newest things that we offer.

And that that's also a pretty powerful way that we attract feedback.

Yeah, I think that's a good point. You know, there's not just enterprise customers at Cloudflare that are represented by account teams and solutions engineers, there's 1000s and 1000s of customers that are on our self service plans and the support lines that community pages.

There's a big community around Cloudflare as you kind of saw with how both Tim and I had first been introduced to the platform and the product.

The community page is a great way to get into touch with other members of the community.

And also the product managers are often out there lurking.

So, you know, your voice is not unheard if you don't have a solutions engineer and an account team to talk to.

Our PMs are really great about that.

If you have ever looked at a hacker news post, something related to Cloudflare, they're all over it, responding to as many comments as they can.

So I think we at Cloudflare take product feedback very seriously. Yeah, I think as a whole, the organization, you know, seems to be really in touch and engaged with the communities outside of just, you know, the internal feedback loops that we have.

Let's move on to the next question, which is, Cloudflare has so many products, how do solutions engineers know which ones are best fits for certain prospects?

So I think this question gets at something that's pretty interesting about the solutions engineering role here at Cloudflare.

You don't necessarily only need to know about the products that we build.

As a solutions engineer, it's very, very important to understand architecture that's tangential to Cloudflare.

So we sit in front of many cloud services, on prem services, maybe homegrown services, maybe even other competitors of ours, and how Cloudflare fits into the whole picture is something that you need to know at kind of a high level and be able to understand and dissect.

But even at a very, very in depth level, we have certain experts that are understanding of certain products that are tangential to us and also certain products, suites within Cloudflare and that feeds into what we call a subject matter expert program.

So Tim's been doing a lot of work around the SME program. And can maybe speak to kind of how that program started and where it's kind of going.

Yeah, and yes. So when you think of the sort of the solutions that people implement with Cloudflare, if you rewind a few years ago, and look at what Cloudflare was maybe back in 2017, most people saw, well, just a little CDN or WAF.

So people who are experts in performance and security tools were perfect.

However, as time has gone on, we're looking much more like a compute platform.

We have serverless compute with Cloudflare workers, we have storage with workers KV, video delivery with stream, we have direct network appliance solutions like Magic Transit and PYOIP.

So the skills and knowledge that we need are extremely broad for all of the different applications that our customers have.

So one of the valuable parts of having nominated subject matter experts is we can have experts in specific products that benefit things like our compute and workers applications or our network for Magic Transit or performance and security features so that the wider SE team can benefit from sharing knowledge and having trusted advisors within the team to assist with different solutions.

Yeah, and one thing I've noticed as a subject matter expert on the serverless side is you're really able to get deep into that particular area of expertise and then help folks out across our organization help prospects and customers out as they have more deep questions around that particular product area.

And kind of help them understand how it fits into the broader picture as well as go really deep on calls on, you know, how that product might work with their particular architecture.

I think, you know, it all starts with hiring also so that this being an SE at Cloudflare like you guys are saying is that we have we have lots of different products, lots of different features.

It's impossible to find a candidate that already possesses all of this knowledge when they come in and take over the role.

So the next best thing is to find people who have, you know, experience and expertise in one or maybe a few of the areas, and then that also have the potential to expand and learn the expertise and some of the other ones.

And there's, we don't like to put expectations or pressure on the SEs that they need to be an expert in every single thing right that's why we have the SME program.

What we do look for is SEs who are curious and have the desire to research and study these customer homegrown solutions or the tangential products like other customers DNS and how that works or serverless if they've never experienced serverless.

So yeah, we don't, we can't memorize everything but the SEs certainly are really great at research and knowledge sharing and helping each other along the way.

That's a really great point, Jason.

You never know what cloud you're going to be helping a customer with on any given day.

I think that's what makes the job so exciting for me, to be honest, I like the unknown.

We don't know what we're going to be talking about on the next call.

Could be anything. So let's see, we have 13 minutes left 12 minutes left.

I think we have time for a few more questions. So let's, let's hop into the next one, which is How do SEs keep track of all of the technical details of each product?

I think I bled into this one a little bit on my last answer.

But let's, what do you guys think? Yeah, I think we touched on some of this in the last question, but I think one thing to note or add is that kind of off of the idea of inherent curiosity.

Is how you keep track of all of that through Things like centralized wiki pages building out cheat sheets on various products and really starting to shadow each other's calls and see how certain folks might Might attack a certain architectural problem and talk through it with a customer.

Those things are all really useful to kind of keep track of how someone else on your team is being successful and how you might apply some of that to yourself.

And to the broader team and unlike the technical details side, the idea of keeping everything within centralized wiki spaces and Keeping up to date and working with folks outside of just the solutions engineering org is really important and definitely ties into how we we utilize things like our SME program.

Absolutely. One of the things I find as really helpful for myself, keeping on top of all of the technical details is One of our superpowers on the SE team is we're really good at sharing knowledge.

We did a number of different ways. With regular or at least bi-monthly or monthly syncs across not just the America's solutions engineering team, but also America's with Asia Pacific or America's with EMEA.

We share things we've learned and insights because we want to make sure that if one SE learns something, all of us can learn something from this thing.

So we only have to solve a problem and learn the solution once.

And beyond just talking to each other and sharing knowledge, we're sharing blog posts internally every day.

So if you thought the Cloudflare blog was kept up to date very regularly, the solutions internal wiki talking about different experiences our customers have had with products is even more regularly updated.

And finally, once we get to keeping track of specific solutions, we leverage a lot of infrastructure as code tools like Cloudflare's own Terraform provider to lock down if we see a solution that works really well, say combining Cloudflare with multiple clouds, multiple regions for highly available multi-region services.

We can bed that down into a Terraform template so that if one SE wants to present the solution to their customer that another SE in say Europe has presented, they can leverage it without actually doing all of the work of building it from scratch.

Yeah, I think that last one's a pretty good point to dig into.

You're not just looking at the requirements documents and understanding what's been built and how it's been integrated.

You're also building out demos and versions of the solutions and seeing what are the limitations of all these types of combinations of products.

How do I combine product A with product B or cloud A with cloud B and Cloudflare in front of it?

How can I use tools like Terraform? Are there changes to Terraform that have made certain things easier or harder on customers?

And there's no better way than talking to customers and actually building these things yourself.

Yeah, it's such a great feeling being able to, rather than just hypothetically talk to a customer about the idea of what problem you want to solve and how you want to solve it.

Being able to show them, this isn't working, here are the bots flying over the Internet.

It's doing the thing. It's really exciting. Yeah, and one other note that you mentioned, Tim, is how our internal blog is kept up to date even more so regularly than the external facing Cloudflare blog.

And that's because there's a lot of different communication channels within the solutions org and within Cloudflare in general that allow for very low cost and low stakes communication.

So one thing that might be an issue in many of the organizations I've worked at and folks that I've talked to is the idea that you might be asking a question that has already been asked before, or you might be asking a per se dumb question.

But we really kind of shy away from encouraging that type of thought.

It's instead, hey, I have a question. Here's a place I can reach out and talk about it.

Or hey, I have an idea. Here's a place I can share about that and get feedback on it.

And I think the low stakes communication is really important to keep knowledge kind of share alive through the organization.

Yeah, that's a really important point, Kabir.

I often say to newer SEs is don't be afraid to write, even if someone's already written about it, because you already, by experiencing a product yourself, you have a unique insight to it.

So it doesn't matter if someone's written about something five times before.

If you feel that you learned something new and you got value out of this thing you learned, it's worth writing about, because it's in itself keeping the documentation up to date with a new experience.

I know myself recently, I wrote a, I thought I was the smartest person in the world by writing a particular blog post, talking about a challenge with one of Cloudflare's products.

And someone responded to me saying everything old is new again.

And it's not an issue that was rewritten about.

In fact, it just makes it more important because we can point to multiple places and see this is something, a problem we need to solve.

And it empowers the business to know that we need to solve this problem.

That was a really great point.

Great points, guys. All right, so next question. How do we share knowledge across the SE team?

I think we bled into this one again, too.

Yeah, if I could distill it down to three points, it would be actual verbally communicating with each other, sharing travel knowledge, writing down that travel knowledge and also coding it, right, putting it into infrastructure as code so that problems we solved and solutions are repeatable and reusable across the whole team.

Yeah, for sure. And when you were talking earlier about having knowledge shares across different regions, all of those knowledge shares are recorded, documented and put in drives.

We have a regular line of communication weekly that we share with all of the SEs that have links to the blog posts and links to the video recordings.

So that way, if you happen to miss one of the knowledge shares, and then you also happen to miss the update of the newest blog posts in the wiki, and then you also happen to miss the post where somebody shares it in chat, at least we have an email.

There's so many different ways that that knowledge share is popping up in your workstation.

Yeah. And I think repeating yourself is actually a really powerful thing as well.

There are times where I've redelivered a particular thing I've learned to the SEs, not just because someone might have missed it, but also we're a growing team.

There are new SEs who have joined in the last six months who didn't see what I spoke about seven months ago.

So bringing things back up and talking about it again is a great way to ensure that there's a level playing field for new and tenured SEs.

Yeah, agreed. I think one other note that I would add is that Cloudflare is a global company.

And one of the unique things about that is that we have solutions engineers that are often working within their own geographic region.

So the EMEA region, the LATAM region, the Americas region. We have sinks within those regions all the time, of course.

You get to know those folks on those teams very well via Zoom or in the old days via coffee at the office.

And being a global Internet company now, we've started to do things like share knowledge between those regions.

So we have sinks that happen on a regular cadence between EMEA and America, EMEA and LATAM, LATAM and America.

And it's useful to see knowledge that often can get caught in a bubble within your own region.

Get shared externally and also hear things that are happening in other regions that might not be the same within the Americas.

That's a really good point. I want to add one more thing on top of that, that the engineering teams and the product teams are also distributed unevenly globally.

So the DDoS team is located in London and the Cloudflare for Teams is located in Seattle and Austin.

And so the SEs that are in those specific offices have more direct lines of communication with those product and engineering teams.

And back in the old days when we would interact with them in person, well now everything is virtual so it's a little bit different.

We should probably talk about that in the next session.

But when you're in the same office as some of these engineering product teams, you will obviously get to know a little bit more than say an SE that's in another office about that specific product.

Yeah. One of the things I found really interesting about being an SE both in Australia reporting into our Singapore office and also being in America into our San Francisco office is the culture.

It has been very consistent and it really is one global team rather than three separate teams.

And I felt like I was part of the same family, just located in a different space, which was amazing.

And also to add on to your point of having a global team, we have that follow the sun model.

So it's been great being able to trust that even if I turn my laptop off at the end of the day, someone is there to help during the night and into the morning.

So I think we're coming up to time here.

I think we can save some of the other questions for our next session.

Just one note, if anyone watching has any questions, feel free to email into the show.

You'll see that below the video on Cloudflare.tv.

You can also use our community lines, as we said, to get answers from the broader community or talk into our support team or your designated account team as well to get in touch with Cloudflare.