How We Train Technical Support Engineers

Intro Music Hello, thanks for tuning in to Cloudflare TV.

My name is Shane Ossa. I am the technical training program manager for the customer support team at Cloudflare and I'm really excited today to talk to you about how we train tech support engineers at Cloudflare and this is the third time we've done this segment.

It's really fun. We have conversations about how we train and the training program.

The past two times I've had a couple other technical trainers on the team to talk to and today I have another one, a relatively new technical trainer Rebecca Price.

Hi, I'm Rebecca.

I'm based in London and I've just switched being a technical trainer. I've just switched from being a technical support engineer so it's not that long that I've recently gone through the training program.

Now I'm in charge of training on new hires in the EMEA region.

Welcome to the team. I'm so excited to have you.

We're really lucky to have you join and it's fun to get to talk to you because you're a person who went through the program yourself and so as we get to the various topics around how the program works and all the different facets of the program, you can just jump in and say, hey, that worked well.

You know that we could have been better and I'm going to help us change that.

Yeah, I'm just really happy to have you on the team.

So welcome. So for those of you that have seen a segment before, there's going to be some review.

We're going to talk about how the whole training program works.

For those of you that haven't, you're in luck because we're going to talk about how the whole training program works.

So I'll just get us started off by discussing basically the training program is designed to take our new hires and train them to be technical support engineers that help Cloudflare customers with a variety of issues on dozens of products with hundreds of features.

We're constantly adding new products, new features and we do need to train all our tech support engineers.

Every time we release something new, it's hard to train our engineers.

Yeah, so the training program, I'd like to think of that all the training that we do sort of falls into these four categories.

So we're either training on a Cloudflare product and feature, how it works or how to troubleshoot it.

We're either training on how to use a troubleshooting tool. We have a lot of tools.

So we have product training, we have a tool training and then we have process training.

So we have a lot of policies and a lot of processes and this is the way you do this and this is the way you escalate when you need to escalate that.

So we train on processes. And then the fourth bucket, which is kind of a really big one and a really challenging one, is just concepts or technologies, right?

How does DNS work? How does SSL work? How does TCP work? How does HTTP work?

We train on all those things as well. Now we do hire really talented driven people and one of the things that we do look for when we hire is curiosity, right?

We definitely try to screen for people that want to learn and that's one thing that's been so impressive with you is you started I think a little over a year ago and you just learned so quickly and so well and you were interested in training and educating other people on the team and helping them grow and develop new skills.

Do you want to talk a little bit about your interest in that or just anything I mentioned?

Yes, I am quite a curious person and interested in lots and lots of things.

I remember turning up for my interview with kind of, I think my teammates called it the Bible of Cloudflare, which was everything I could learn from the learning section of Cloudflare and I had been studying and applying myself in order to get to the interview and through the interview.

And then that was just the start, so yeah.

I then hit the training program and the learning and studying and applying myself didn't stop.

That's great. Yeah, you've done so well and then you took an interest in helping other people learn so that's really great and as you continue on, you'll become a learning geek like the rest of the team, which is really fun.

So when Cloudflare tech support engineers start, we put them through a boot camp.

We call it CSUP boot camp. CSUP is short for customer support and the boot camp is sort of a two and a half week, roughly intensive training program that covers all of those things I mentioned before.

The Cloudflare core products, the teams processes, the tools we use and some of the underlying technologies beneath Cloudflare.

We put people through a two week boot camp and then we let them start working with free customers.

As you all, you know, viewers may or may not know, Cloudflare has different plans that you can subscribe to and one of them is actually free, which is great.

So we have millions and millions of free customers and they write to us all the time.

Can you help us with this?

Can you help us with that? And that's just a great place for the people who have gone through boot camp to work for a few weeks and just kind of learn about all the basic issues that you will encounter as a Cloudflare tech support engineer.

So the boot camp is sort of designed to give you, you know, just enough information to start going and helping free customers and we also really like to train people not just to memorize stuff but where they should go to look for information and how to find out stuff themselves and you know how to figure things out rather than just, you know, here's the answer, you memorize it, you get it, someone writes in, you get the answer.

We have to teach sort of a process for going to find the answer and researching and we have a lot of other mechanisms built in for our team to help each other with issues, you know, they ask questions of each other all the time.

Yes, that's true, it's really important to teach the new hires where to look for the answers maybe in our KBs, our internal documents, our internal wiki and to find the answer out for themselves as well as reach out to other team members.

Hey, I'm seeing this issue, it's not quite covered in this document, have you seen this issue before?

How do I approach it? Yeah, yeah, documentation is just such an important part of training in general, you know, we need that kind of source information to build our training upon and we rely on working with subject matter experts or SMEs in various areas and trainers are expected to be subject matter experts in everything, but that's really difficult and challenging when you have, you know, dozens of products and hundreds of features.

So naturally we do rely on, you know, engineers and product managers and other tech support engineers that have been at the company for a long time and our amazing team leads and now we have a product specialist team which were also tech support engineers that have now dedicated 100% of their time to helping us with things, you know, internal documentation, external documentation for the customer, communicating customer feature requests and support feedback on how to make things better to the product team.

It's great to have these resources for us because we need that content and we need someone to bounce these ideas off and verify, is this the way we should be doing this?

Is this expected behavior for this product?

And can we document this issue better so that when the next tech support engineer has that problem they can quickly find it?

So a lot of it is about, you know, being able to provide really good content that helps to solve the customer's issue and making it easy to find and easy to access for the tech support engineer.

Showing them where to go, giving them the kind of underlying core foundational knowledge they need on the tools, the processes, the products and the technologies and then providing these other kind of resources for them to really succeed.

So it's really amazing what the tech support engineers do every day. I'm just constantly blown away by the team and I just want to give a shout out kudos to any of the team that's watching that have gone through the amount of stuff we all need to learn and that you need to learn and retain and apply every day.

It's just super impressive.

So we work with an amazing group and that's a big part of the success of the program as well is just the learners that we bring in and the hiring.

Yeah, I think with the documentation since I started, the documentation has grown and grown and grown and I've added to it.

We all encourage other engineers to add to it as we go along and it's just a growing thing, a growing source of information.

Yep, so let's talk a little bit about the different modes of training that we have.

The program has e-learning, which are online trainings that we've recorded and we deliver these through an LMS, which stands for Learning Management System.

So we use an LMS to deliver, you know, we have over a hundred e-learnings in there on all sorts of topics, products, tools, processes, technologies and so we have all of these e-learnings.

They're great. We have an amazing e-learning developer, Scott, who I had on two weeks ago on a program and he is amazing at building these interactive e-learnings and he's on the team.

He's also our LMS administrator so he assigns all the courses to everyone and he makes sure eventually that they complete them, you know, with their due date.

So this is a typical function of a training program but it really helps because what we find as trainers we're doing a lot of time is giving presentations when we would really prefer to be doing hands-on activities, right?

We want to get people on the command line with their Cloudflare test account, you know, trying Cloudflare products and features, trying to break them so that they can unbreak them and that really is going to prepare them for the real thing of dealing with a customer issue.

Sometimes the customer issues are quite urgent as well.

So we have these e -learnings and as a trainer, as I was saying, you find yourself doing a lot of presentations and sometimes you find yourself doing the same presentation over and over again, pretty much the same.

And after you've done it a bunch and you've kind of teased out all the normal questions, you end up building like a very solid deck and presentation and what we end up doing is turning that into a recorded e-learning that we're just gonna be able to deliver at scale to our global team.

I should mention that we have, you know, over 90 people on our team, you know, in all over the globe.

We have people in Singapore, people in Munich, Lisbon, London, Austin, San Francisco.

Cloudflare has offices in other cities, but the support team, we do follow the Sun model 24 hours a day.

You can, as a Cloudflare customer, contact a Cloudflare tech support engineer and get help.

And we'll talk about the contact channels in a little bit.

So we have these e-learnings. It makes sense.

It's scalable. We can see when people have completed them. These are presentations that we'd be giving over and over again so that we can free up our time to do hands-on trainings.

So that's sort of the next mode we have. We have e -learnings you watch, and then we have hands-on trainings that the technical trainers like Rebecca and Jamie and Mark and the other people give all the time, almost every day.

So we do e-learnings. We do hands-on trainings. We do what we call shadow sessions, which is sort of on-the -job training, if you will.

So this is really just pair, sort of pair programming or side-by-side, I've heard it called, where a tech support engineer will sit next to another tech support engineer that's fairly new.

Rather, the new person will sit next to an experienced person and sort of watch them do, you know, watch them help customers.

I was gonna say watch them do tickets, but to give people watching the reference, when a customer contacts Cloudflare, it gets turned into what we call a ticket.

And we have a ticketing system that helps us process tickets, you know, in a timely fashion.

And so the new support engineer will sit next to the experienced support engineer and sort of watch what they do.

So this really gives them that real-life experience, right?

So we have, you're gonna watch the e-learning, you're gonna come to the hands-on training where we show you how to troubleshoot something, and then you're gonna do some on-the-job training, what we call shadow sessions.

And people really seem to like the shadow sessions.

Yeah, I think people really do, excuse me, people really do like the shadow sessions.

I know when I give them, I'll go through a couple of tickets and go, okay, this is the issue, this is how I'm going to approach it, and I'll run these tests, and then I'll convey this to the customer.

So after two or three tickets, I'll let the new hire look at the ticket and then just ask them, okay, what do you think we should do in this situation?

What tests are you gonna run? How are you gonna communicate with the customer?

And it helps to build up their confidence to know that somebody experienced is next to them and is guiding them through.

Yeah, so you know, at the start, the shadow sessions are mostly you watching somebody, and as you get more comfortable, and we run more and more shadow sessions, eventually we like to encourage people, the shadow, to actually do the tickets themselves with the person hosting the shadow session just kind of there to help them.

And so maybe a shadow session, if it's an hour and a half long, the first half roughly is them sort of watching someone do tickets, and the person who's doing tickets is narrating their workflow.

Now I check here, you know, now I'm running this test, and okay, I see this data, and I'm gonna give this to the customer, and it should help them, you know, fix something.

And then the second half is, okay, now you fly, and they share their screen, the learner shares their screen, and now the experienced person is there, and the learner tries, and they go, okay, I'm gonna try to do this ticket, and I'm thinking I should check here first, and they go, yep, you got it, and I'm thinking I should say this to the customer, yeah, that's perfect, and so that's just there to verify.

And we also, I should say, we do these things sort of during our boot camp, which we talked about earlier, we have each day is sort of about a subject.

So in the morning, you'll watch an e -learning on DNS.

What is DNS? What is Cloudflare DNS? How does it work? What are the common issues we might see as a tech support engineer?

Then you come to the hands-on training after that.

So the e -learning sort of primes you, and sort of lays the foundation for the hands-on training.

Hands-on training actually gets you using Digg, and other command-line tools that we have, and the other, you know, databases that we can pull off, and shows you how to do that.

And then the afternoon is a shadow session where we look at actual tickets related to DNS.

And so we try to do, at least in boot camp, we do these kind of day subject, you know, each day covers a various subject.

One of the core subjects that Cloudflare is relevant to the job.

And so what we found when we were doing shadow sessions was we were kind of looking for the perfect ticket to train someone on.

And sometimes it was there because, as we said earlier, we have a lot of free customers, and there's a lot of free tickets.

But sometimes it wasn't there, and so we went and found other tickets, and it was fine.

But we got the idea, or someone gave us the idea at one point, said, hey, y'all should make broken servers that are broken in a certain way, or let's say misconfigured, or misconfigured in a certain way so that, you know, the behavior, the interaction between the server with the website, and the Cloudflare is misconfigured in a certain way.

A way that is quite common for Cloudflare customers. So that when we do these shadow sessions, when we do these other trainings, and we want to train someone on a very common issue that they're going to encounter in real life, that we have something that's already there that's misconfigured in that way.

So that was a great idea.

And so we started building a suite of test servers to train people on, to debug.

So it's a massive project, and we're still building loads, and we're actually crowdsourcing this and trying to get other people in the company, there's a lot, you know, solutions engineers, you know, regular systems engineers and developers, anyone who wants to help us build a test server debug, we have a massive wishlist of servers that we want to build to train our people on.

And so that's another big facet of the program is having test servers ready with common issues for our tech support engineers to debug.

So, in addition to that, tech support engineers need to install and try out almost every Cloudflare product and feature.

We want all of the tech support engineers to kind of get that customer experience.

They should go through what the customer goes through when they set up Cloudflare.

So everyone is tasked with setting up a Cloudflare account, and we entitle it with all the features so that over time they can check out every feature and try it on their own web server.

We also, a lot of the people we hire already have some development experience, but we make sure that they have a web server.

We give them access to our web servers, they can spin one up, put it behind Cloudflare, and start playing with our features.

Not only do they kind of get the customer experience, but they get training on the product that way as well.

Yeah, I think that's a really important part for me when I was a technical support engineer.

I had several test zones, and I would always try to replicate the customer's issue, if at all possible, on my own test zone.

So yeah, my websites were constantly being broken and tested against, and then seeing how I could replicate the issue and how I could solve the issue, and then communicating this back to the customer.

Oh, I think maybe you've made a mistake here.

I was able to replicate it on my own domain.

This is what I advise my steps to be. Yeah, and that's a great point. And, you know, customers sometimes are reporting a bug, and we need to reproduce it.

We need to see if we can reproduce it. So not only do you get the customer experience, you go through what they go through in setting up a product, you get training on it that way too.

Then you get an issue reproduction environment. So when a customer does write in and says, I think we're encountering a bug here, you can replicate it, say, yeah, I can reproduce that error, and then we can escalate that to the engineering team with a real, you know, yes, we can reproduce this error, check it out.

Here's our test server. Here's their server. It's throwing these errors.

We were able to reproduce that. This really does seem like a bug. And so yeah, that's also a part of how to have a issue reproduction environment.

So we have test servers to debug setup that are misconfigured for people to practice on.

And then we give people their own test servers to let we just have an issue reproduction environment.

And then on top of that, top of the hands-on trainings, on top of the e-learnings and the shadow sessions, we also do what we might call micro learning opportunities.

And a micro learning opportunity is sort of a less structured, more casual way for us to share knowledge.

As you can imagine, we have 90 plus people on our team, everyone has their own experiences, career experiences and skills to bring to the table.

As they do tickets over time, they learn different things.

And we are trying to facilitate ways to share that knowledge amongst the team with each other.

So we we do knowledge shares monthly and sometimes even more regularly.

Yeah, I arrange our knowledge share for the EMEA region, and we get a variety of the engineers from the most experienced down to, you know, maybe even the new hires who just want to share something that they've learned.

Maybe not everybody knows this tiny little, this bug or this little tool that they've found or this website that they can use to test DNS or, or something like that.

And people will give a presentation a few minutes long to the whole team.

And in that way, we share the knowledge.

That's right.

So we do monthly knowledge shares regionally. And we also facilitate Lunch and Learns every now and then.

And we do things like coding clubs. A lot of these were sort of grown from people on the team themselves.

And they organize them since themselves, which is which is great.

Sometimes these are outside of the hours.

And sometimes these are within working hours. And so these are just kind of designed to kind of additionally just share all of our wealth of knowledge and skills with each other.

That new browser extension you've found that script that you wrote, that alias that I'm using to save you time, third party website, you've been using, you know, anything really anything that someone has would like to share, we want to make sure it gets shared with the whole team.

So part of that is just across the board, sharing each other's knowledge to increase efficiency.

So that's one another function of the training program.

Another really important thing for the training program is what we call case studies.

So these are tickets that, you know, are really illustrative of how to to solve a specific problem, a common problem, or maybe it's an edge case, it's really interesting, that was just a really great example of how to use this tool, a really great example of how you should go about fixing this type of issue.

And so we catalog case studies, and we categorize them, and we put them in different places, we put them in our documentation, and we just have a big database of case studies.

And that's another thing that we kind of review, kind of ad hoc occasionally as a team is interesting cases, where we found, you know, we had this customer write in about this one issue, but it sort of took us down this whole other avenue, and we found out that there's this, there's this other, you know, service that's really interesting, we didn't know about.

So we have case studies, we're constantly building more.

And, you know, we're getting a lot of tickets, a lot of interesting tickets, and tickets on all sorts of topics.

And so we have case studies to train people on services. Another thing that we do a lot of is ticket QA, quality assurance.

And this really helps us make sure that people are following our standards, that we're all being consistent in the way that we answer customers.

And the team leads that we work with do a lot of the QA, the training team does the QA for the first while we definitely do their first 10 tickets, but we do a certain amount of tickets for the new hires first three and six months.

And then we'll drop in and do them every now and then.

And this also gives us a quantitative number to sort of measure and benchmark against.

And so that we can look at some data on, hey, when you started, you were getting a 76 in this one area.

And then over time, after three months, after six months, now you're up at 94.

So really, this is just about, you know, helping the team get more consistent and provide quality customer experience.

That's what it's all about.

But it also sort of helps us to design trainings and say, hey, you know, look at this one, you know, metric from this one, you know, type of ticket, we need to do a training on that.

Yeah, I think QA is a valuable source of information to see where new hires are up to, how they're coping, maybe gaps in their knowledge that need to be addressed by the training team, or maybe gaps in the documentation, or maybe a new training is needed for a particular subject.

Yeah, so it's very valuable. Yeah, definitely. So, yeah, in addition to that, we also have a program called Training Opportunities, where fellow tech support engineers, or anyone, even, you know, success managers, solutions engineers, anyone in the company, can sort of mark a ticket as a training opportunity, and say, hey, we, you know, we looked at this ticket between, you know, tech support engineer and the customer.

And did you know that there's actually this tool you can use to get at the answer?

Oh, great. Or do you know that, you know, you, you call this instead of that really anything, or maybe we could have handled the situation a little better if we had said something like this.

So this is really important for tech support engineers and anyone in the company to be able to say, hey, you know, we could have done this a little better, or there's another tool you could have used or something like that, right.

And we take those and we review everyone.

And we sort of look for a couple different things. We sometimes it's just one person didn't know about one tool, and that's fine.

But sometimes we notice trends and we say, hey, maybe we should do a bigger training, a bigger refresher on this tool.

Or sometimes it surfaces like to go back to what we were talking earlier about documentation.

Sometimes these training opportunities surface that hey, there's no documentation that explains how to solve this problem.

How can we expect this person to have done this when there's this, you know, they have nowhere that says what they should go and do.

And so this is another way for us to surface those needs.

Yeah, or, or maybe there's a gap in the in the documentation in this particular issue wasn't in that particular runbook.

And it highlights the fact that maybe we should add it to the runbook so that she comes up again, an engineer knows what to do.

Yeah.

So the runbooks are our internal documentation that explain how to solve specific problems, process documents or troubleshooting guides, if you will.

And these, you know, step one, do this step two, step three, and versus the public knowledge base articles, which we call the KB knowledge base KB.

And the customers use the KB articles and the KB articles are great.

And the content is amazing. They explain how Cloudflare products work, they explain, you know, common issues you might see or misconfigurations with your, your web server that that might occur.

And that content is amazing.

And we use that a lot too, to be honest, a lot of the time we use the KB, we, we try to make sure that the customers have all the same information that we're working with.

The only reason that we have our own internal documentation is, of course, it references internal tools that are not available to customers.

So when we're pulling up a customer account, you know, and looking at some data to see what happened or what they did, you know, we're running internal tools that aren't available, but there is a side project and initiative at Cloudflare with the support operations team and some other engineering teams to, to really expose all of the tools that we're using internally, safely to the customer.

If we're using a certain tool to debug something, we want the customer to be able to do that safely as well.

So we're trying to package those and put those in the dashboard in an appropriate way.

When we have internal documentation that explains how to do something, we make sure that it's public as you know, that it gets into the public KB as well.

So we read the KB articles all the time. We help to curate them, we help to keep them up to date.

Yeah, as a TSC, I used to refer to the public KB articles quite a lot, actually, because it's quite hard as a new hire to memorize everything immediately.

So yeah, I would refer to public KBs quite often, just to help solve customers' issues.

Yeah. That's right.

I mean, sometimes we're dropping and pasting content out of the KB because the KB is so well written, you know, we're going to explain it in a different way if we need to explain it in a different way in order to help someone understand.

But otherwise, a lot of time the answers are there. And we're really just helping customers find those and you know, I'm guilty of copy and pasting the amazing content we have in the KB.

So training ops will also help us kind of find that out.

You know, someone in a training op might say something like, hey, this isn't documented anywhere, we should really have this documented.

And we follow up on that, you know, file it with the appropriate team and they should be exposed to customers.

Or it's an intern that we need to do some documentation on. So that's training ops.

We are constantly asking for feedback on our train program and each training that we give and each e -learning that we've developed, the whole program as a whole, the knowledge shares that we run, the shadow session someone hosted.

It's all about asking for feedback and continuous improvement for us. Of course, we have bigger strategic initiatives and we have a roadmap.

But at the same time, we're also trying to just collect constant feedback.

So what could we have done to make this better?

Was this relevant to your job? Did you apply this, the skills that we went over in the training to your job?

And this is one way for us to just sort of kind of make sure that we're improving everything, we're not staying stagnant, and we get people's ideas.

You know, we get people, great people who have, of course, gone through trainings and other companies, and they've, you know, or they've been tech supportive engineers, companies that do different things.

And we really want to know what everyone's doing, what we could do better.

So we administer a post training survey, this is natural, most people who have done a training afterwards, you get a survey from, from the trainer that says, you know, how did the trainer do?

Did they, did they communicate the material well?

And how could we improve the training? And so this is sort of considered level one in terms of the training evaluation.

Did you enjoy it?

Did you find it relevant? Was it good? How could we improve? And that's just really important to do on every training.

Yeah, that's, that's valuable feedback. And I think in my time, going through the boot camp and ramping up and sending feedback, I've seen the trainings evolve and get better and more useful to our, to our new hire.

Yeah, really, you know, I've been at Cloudflare for about three and a half years.

And when we started, we did have trainings we gave during boot camp. We gave certain amounts, they weren't designed per se, they were kind of just like, hey, we're going to train you on this product.

And we're going to put you with the person who knows the most about that.

And they'll talk about it for an hour.

And there were some slides. And there were a couple of slides that were some rough outlines of, you know, the major key points that need to be hit.

And so we kind of took that and built upon that, based on the feedback that we got from people going through the program.

So really, the program is built on off the feedback that we get, hey, next time, it'd be really cool to do it in this order, not the order you did it, because it makes more sense to do that order instead of this, in terms of the sequence of instruction.

Or, hey, you really need a training on this one tool.

We found this later on, and we wish that we had been told about it sooner.

Anything, anything like that. The whole thing is built on the feedback that we got from everyone going through it.

Each time a new person goes through it, it gets better.

Yeah. So what I've noticed with feedback is it's good to ask for it in multiple ways, for multiple people, anonymously, non-anonymously.

We ask for it in a survey form.

We ask for them just to tell us. We have to provide a lot of different channels, and we have to ask for it at different times.

Sometimes it's not until later that someone has to contact us.

Oh, you know what, now that I've been doing this for two months, I think it would have been good to have this included in that training.

And so we're just kind of constantly asking for it in different ways, anonymously.

We ask the managers to make sure that they're checking.

So it's coming from another angle, so that we make sure we really get the feedback out of people that we need to keep improving.

In terms of measuring training, you have your surveys, and your surveys are very useful and highly valuable.

But we also want to look at how well did we train someone on that material?

How well did they learn it? So a lot of the trainings have a post-training evaluation or an assessment.

And this helps us sort of see how well we did on that training as trainers.

And it also sort of helps to reinforce the knowledge that we were sort of trying to impart.

So definitely all of the e-learnings, they all have a post-training assessment.

And the assessment is fairly short. This isn't a 100-question comprehensive thing.

It's mostly to kind of hammer in the key points.

And it also kind of gives us a data point later on. And we can kind of see where we're not doing that great in terms of the training.

We didn't cover that for long enough, or we didn't explain that in enough detail, because everyone seems to be getting this one question wrong.

And then we go back, and we look at our source material.

We look at, can we deliver this better? So that's sort of what we consider level two in terms of training evaluation.

There's four levels, but kind of the standard process, level one, the survey, level two, the evaluation.

We have a long-term goal of getting the evaluations even better.

What we would love is for the evaluations to show that someone can actually debug or solve a specific problem.

So we have it on our roadmap to take the test servers that we built and make it so that we can tell when someone has debugged them.

We'll give them SSH access to the actual server.

We'll task them with fixing a problem that we know has a solution, a certain solution or a certain set of solutions.

And once they have solved that puzzle and fixed that server, it checks a box.

And this sort of helps us understand are they qualified.

And so that's a big part of training is verifying that someone is qualified, not just imparting the minimum you need to know about something, but testing to see now if they can actually go do the thing that you were setting out to show them how to do.

So it's one thing to present information and to give a great lecture, and someone memorizes something, but is that knowledge you're giving them going to allow them to solve a real life problem that they're going to encounter?

And we want to test that not just on multiple choice questions or fill in, you know, write in answer questions, which we do, of course, but that doesn't necessarily show that they're going to be able to fix the thing.

So that kind of takes us into like, level three, almost of, of, can we actually see that someone is fixing the thing that we were, you know, the training was intended and designed to help them learn how to fix.

And so we also do that via QA, we mentioned QA earlier.

And that's how we can tell if we train someone to use a certain tool.

We do a training, we give them the survey, they fill out the survey, the training seems good, they have an idea of how to make it better, but it seems pretty good.

Level two, we asked them questions, you know, multiple choice and write in your own answer.

And okay, they seem to do well on the evaluation. And so that's kind of showing us they were able to retain this knowledge.

There's a lot of variables there.

And then the third one is when we do QA, did they actually use that tool in the situation that they were supposed to use it?

And did they use it correctly?

And so that's really interesting kind of part of it as well. And yeah, just looking at the QA, a QA ticket so you can say to the technical support engineer, if you remember your training, maybe you could have considered this tool and tested for this thing that was mentioned in this training or in this e-learning.

Yeah, so we look at metrics as well as a training team. You know, we have goals of making the team, you know, more efficient.

We want to make sure that the team is giving high quality responses.

We are trying to meet the organizational needs and the business goals as well of reducing costs by, you know, really increasing the speed to proficiency or time to proficiency or time to productivity of the tech support engineers.

The faster we can get people up to speed and being a fully productive member of the team.

That's one of our big goals.

But time to proficiency as well. How long does it take a person to be proficient with a specific tool or on a specific type of issue?

And so we're constantly looking at metrics ourselves.

QA, we've mentioned a bunch of time. Time to productivity is something, time to proficiency is something we're measuring as well.

And so when we build this suite of test servers that you're tasked to go and fix, we'll have a metric on, you know, how long it took for you to be able to go and fix those.

And it sort of can be a challenge for the learner, and it can sort of help us benchmark as to how we can get better and faster and more efficient as trainers.

We also look at customer satisfaction, of course. In the support industry, customer satisfaction, also known as CSAT is a standard metric that everyone looks at.

After a ticket is closed, the customer gets sent a survey. Was it a good or bad experience and why?

And we look at that and we're trying to improve that as well.

I think as a technical support engineer, you look at your CSAT and your feedback from the customers.

And if they've said that you've given clear steps to solve the issue, that means you feel a bit more confident about your abilities and your communication with the customer and how you get something across, which is very important.

Yep. Yeah, and we look at a lot of other metrics across the board.

So we work with the team leads there who are looking at metrics, and we're trying to help improve them.

And if it surfaces something on an individual level or on a group level that we should do a refresher on or that we should add a training on, that's just another way for us to know.

Something that's been really interesting for us over the last year, over this year, is moving to entirely remote work.

And the challenges for trainers on this front, training tends to favor hands-on.

So like we mentioned, we do e-learnings, of course, and these are recorded trainings.

Some of these are just presentations that we actually just record ourselves giving, but some of these are actually interactive e-learnings that our e-learning developer has built with some really cool content authoring software.

And we have those. We've always had those and we always will. But what we loved was hands-on training.

People like to be sitting in the same room as you, their laptop is open, you're next to them, they're next to you, you're showing them this command, you're having them try the command, you're pulling logs, you know, you're parsing logs.

And now that's just more challenging, right? We're doing that via screen share.

It's just like this. I mean, we're just doing Zoom meetings all the time where we're sharing our screening, we're asking people to share their screen.

We're using the little whiteboard feature to draw stuff. And we onboard groups of people if possible.

And so these have been really interesting challenges for us to solve and go through as a training team to move to 100% remote.

And I have to say, I was really concerned about it at the start, but I think it's actually going quite well.

Yeah, I'm quite surprised it's going quite well. Whilst, of course, I do miss that reading somebody's body language and looking for that aha moment when something suddenly clicks.

You still get some of that through remote learning.

Yeah. And asking them to share their screen and looking at the screen and going, okay, maybe if you change this and do this, and you still get those aha moments, but I do miss the...

Yeah, that's what we live for as trainers and teachers and educators in general.

It's when you're talking to someone about a subject and it clicks, like you said, and they get it.

Oh my God, I get it now. That was great.

We live for that. But that's been harder, but I think we've been able to do it.

We have been able to do it. We've actually hired people now that have come up 100% remote.

We went through the recruiting process remotely, when they went through the training process remotely, and now they're fully contributing members of the team.

And that's really hats off to the hard work of all our trainers and really the learners themselves into picking up this material and becoming successful at their jobs.

But I think the biggest thing from a training and really learning and development standpoint is we're missing that sitting in the room with the team, sitting in the room with other people that have been doing this for years or months more than you have, and you're sitting next to someone and you're, have you ever seen this problem?

And they just go, yeah, you got to do this.

Or, oh, I've seen this crazy, how do I use this one tool? Oh, let me just show you that really quickly.

And it's just five minutes. And it's one of our experienced people that is really good at explaining stuff, or one of our experienced people that's really good at helping people learn how to fish for themselves.

It's not just going to give you the answer. They're going to just, so we're missing all of that.

And so that's, I think, even bigger than delivering the training and making sure people can do the job.

We can do that. It's this intangible interactions that we get all day when you're sitting in the room with people, not just trainers, we're talking about the whole team, because that's really what was happening a lot of the time.

We have a training program and it's great, but there's just a lot of just ad hoc learning that occurs and sharing of knowledge that occurs all the time.

So, we're trying to build in, it's all got to be Zoom meetings and video chats and video conferences now, right?

So, we're trying to replicate that by having office hours or Q&A sessions or open hangouts that we staff with trainers, mostly team leads really, and experienced and senior members of the team that we say, okay, it's your turn to go into the New Hire chat room, video chat room for two hours and let the New Hires just ad hoc, you know, they're working on a ticket, hey, have you seen this issue?

And then you go like, oh yeah, I've seen that issue looks like this.

And it doesn't have to be formal. It's not a structured training.

It's not like we go, you know, we design trainings, you know, but this is not that, this is casual.

Yeah, I mean, remembering back to when I started, I was in the office and I had my team all around me.

So, it was a chance to build personal relationships.

And I had somebody to my left or to my right or behind me or across from me.

Yeah. Quickly ask questions too. So, you build up this relationship. I think it's very difficult for New Hires to interview remotely and then start remotely and be on board remotely to build up those relationships.

And so, that's why we've put these things in place to try and address them.

Yeah. And we do after work hangouts.

There's game, you know, we do like games hangouts. There's like a pub quiz one that the London team, the European team does.

It's the dig and trace route.

Yeah. The dig and trace route is their virtual pub where they do a pub quiz and that's really fun.

But, you know, so yeah, I mean, I'm glad you mentioned that, it's not just the learning from each other.

It's the camaraderie and it's the culture and it's the friendships and relationships.

We spend a lot of time at work in general.

You spend more time with the people you work with often than your actual friends and family outside of that.

So, we're missing that component of it. And I think that's the biggest challenge.

We're able to train people. People are able to successfully do the job and it's great.

And we're doing more meetings that try to facilitate this sort of casual hanging out and sharing of knowledge.

And so, that's a big piece and that's something that's just been really interesting and challenging for us to try to figure out and help with through these strange and challenging times.

So, yeah, this has been a really good discussion. We're still hiring as a team across all of our, you know, in every city that we have an office that has a support team, you know, especially multilingual people.

One of the big goals for Cloudflare and for the support team is to provide support in other languages other than English.

Spanish, French, German, Portuguese, Chinese, and more.

We want to provide all languages. So, people, especially if they're multilingual, that's great.

We're hiring people who speak English too. Just English, that's fine.

But multilingual people out there, if you're listening or if you have a friend that likes, you know, troubleshooting computers and speaks languages, apply for the job and be curious and want to learn.

And like Rebecca did, read everything on our website.

And we have a training program for you that's going to teach you how to do this stuff.

Ah, yes. So, you know, the goal...

Go ahead. No, please do. Yeah, so, you know. You go ahead. Yeah, so I'm just appealing, appealing to the masses, you know, we're hiring in every office.

Please apply.

You know, be curious. We have a training program for we'll show you how to do it.

We'd love if you've already installed a web server. So, if you're listening out there and you're thinking about applying, some good things that would be good to go and do right now would be to sign up for a Cloudflare account, a free Cloudflare account, and start messing around in the dashboard and, you know, get a website, build a website yourself from scratch, even better.

Or you can go to one of the, you know, website building tools that are out there and put it behind Cloudflare.

But even better would be to get a remote server, a VPS, virtual private server, and install a web server on it, a Linux web server, go through that experience, and then put it behind Cloudflare.

If you have installed a web server and have got a free Cloudflare account and your web server, your website is behind the Cloudflare account, that's a great sign.

We're looking for that. And we're looking for people who have done that already.

Another thing you want, you know, even more bonus, go into the Cloudflare community forums.

Interact in the Cloudflare community forums.

We love that. When we see people that have done those three things, I have a web server, it's behind Cloudflare, I've been a Cloudflare customer before, and I'm active on the communities, and I'm a very curious person, that's a great sign.

We're looking for all those things. And that, those are really the foundations.

If you can do those things, the program is going to work perfectly for you, and you're going to be able to be really successful as a tech support engineer.

Yes, please do. I mean, I've learned so much through web servers that I've built a Raspberry Pi at home, and played around with load balancing, and played with all the features.

So, yeah, if you're curious like me, and you want to learn, then please do apply.

Oh, I'm glad you mentioned that, the Raspberry Pi. We use those for training as well.

We have that built into our program to supply every support engineer with a Raspberry Pi, and they can expense it.

And we do labs where we install a web server and put it behind Cloudflare with the Pi.

Some people are actually using it for fun for lots of cool different things at their house, monitoring the temperature, or even just a web server that they can use that they have a Cloudflare service running on, so that they could see if there's any latency or anything like that due to any issues.

So, that's another fun kind of part.

Sometimes we do that outside of work, but it's also, you know, sometimes we run the lab during work as well, where we get people using a Pi.

Well, cool.

It's been really fun to talk to you, Rebecca. Thanks for joining me on how we train Cloudflare support engineers.

Would you do it again? And yes, thank you for having me.

And yes, I'll do it again. That's great. Well, we're going to sign off now for the week.

We'll see you again in a couple weeks for the next installment of how we train Cloudflare tech support engineers.

My name is Shane. Thanks for tuning in to Cloudflare TV.

See you next week.

Cloudflare Access allows you to securely expose your internal applications and services, enforce user access policies, and log per-application activity, all without a VPN.

This video will show you how to enable Cloudflare Access, configure an identity provider, build access policies, and enable Access App Launch.

Before enabling Access, you need to create an account and add a domain to Cloudflare.

If you have a Cloudflare account, sign in, navigate to the Access app, and then click Enable Access.

For this demo, Cloudflare Access is already enabled, so let's move on to the next step, configuring an identity provider.

Depending on your subscription plan, Access supports integration with all major identity providers, or IDPs, that support OIDC or SAML.

To configure an IDP, click the Add button in the Login Methods card, then select an identity provider.

For the purposes of this demo, we're going to choose Azure AD.

Follow the provider- specific setup instructions to retrieve the application ID and application secret, along with the directory ID.

Toggle Support Groups to On if you want to give Cloudflare Access to read specific SAML attributes about the users in your tenant of Azure AD.

Enter the required fields, then click Save. If you'd like to test the configuration after saving, click the Test button.

Cloudflare Access policies allow you to protect an entire website or resource by defining specific users or groups to deny, allow, or ignore.

For the purposes of this demo, we're going to create a policy to protect a generic internal resource, resource on intra.net.

To set up your policy, click Create Access Policy.

Let's call this application Internal Wiki.

As you can see here, policies can apply to an entire site, a specific path, apex domain, subdomain, or all subdomains using a wildcard policy.

Session duration determines the length of time an authenticated user can access your application without having to log in again.

This can range from 30 minutes to one month.

Let's choose 24 hours. For the purposes of this demo, let's call the policy Just Me.

You can choose to allow, deny, bypass, or choose non-identity. Non -identity policies enforce authentication flows that don't require an identity provider IDP login, such as service tokens.

You can choose to include users by an email address, emails ending in a certain domain, access groups, which are policies defined within the access app in the Cloudflare dashboard, IP ranges, so you can lock down a resource to a specific location or whitelist a location, or your existing Azure groups.

Large businesses with complex Azure groupings tend to choose this option.

For this demo, let's use an email address. After finalizing the policy parameters, click Save.

To test this policy, let's open an incognito window and navigate to the resource, resource on intra.net.

Cloudflare has inserted a login screen that forces me to authenticate.

Let's choose Azure AD, log in with the Microsoft username and password, and click Sign In.

After a successful authentication, I'm directed to the resource.

This process works well for an individual resource or application, but what if you have a large number of resources or applications?

That's where Access App Launch comes in handy.

Access App Launch serves as a single dashboard for your users to view and launch their allowed applications.

Our test domain already has Access App Launch enabled, but to enable this feature, click the Create App Launch Portal button, which usually shows here.

In the Edit Access App Launch dialog that appears, select a rule type from the Include drop-down list.

You have the option to include the same types of users or groups that you do when creating policies.

You also have the option to exclude or require certain users or groups by clicking these buttons.

After configuring your rule, click Save.

After saving the policy, users can access the App Launch Portal at the URL listed on the Access App Launch card.

If you or your users navigate to that portal and authenticate, you'll see every application that you or your user is allowed to view based on the Cloudflare access policies you've configured.

Now, you're ready to get started with Cloudflare Access.

In this demo, you've seen how to configure an identity provider, build access policies, and enable Access App Launch.

To learn more about how Cloudflare can help you protect your users and network, visit teams .Cloudflare.com backslash access.

Thanks for watching.