π Security Week Fireside Chat: Marco Boniardi & Vijay Chauhan
In this Cloudflare TV Security Week segment, Vijay Chauhan will host a fireside chat with Marco Boniardi, Head of Security Engineering, LastMinute Group.
Transcript (Beta)
Well, hello and welcome. My name is Vijay Chauhan. I work in product marketing at Cloudflare and I'm super excited to be holding these Security Week fireside chats where we get to talk to our customers about all things cybersecurity.
We're just over the midpoint of Security Week, still very many product and partner announcements to come.
And today I'm excited that we're being joined by Marco Boniardi, Head of Security Engineering at Lastminute.com Group.
Welcome Marco. Hi. And Marco, why don't we, to get us started off, we'd love to learn a little bit more about you and what you do at Lastminute .com Group.
Thank you Vijay for your invitation.
As you told before, in Lastminute.com Group, I manage the team of security engineering.
We work together with the Data Protection Office and the internal audit department to guarantee compliance, intercept and prevent vulnerabilities, define rules and procedures, and govern the security control metrics, the huge list of control we are supposed to perform during the year because we are a public listed company and we need to be ready to pass any regular audit.
Now Marco, I know your background spans software development, IT, security. You got your engineering degree from Politecnico di Milano.
I'd love to learn about your route from there to Lastminute .com Group and especially the things that you worked on related to security along the way.
Well, my professional career started in 1999 as web developer during the initial golden age of Internet.
Initially in Emerville, San Francisco, and then immediately after in Barcelona, where the adventure of E-Dreams started.
E -Dreams was the first large online travel agency in Europe.
They started in US and then they moved to Europe. In 2003, after a short period as a project manager in 3D mobile company, I became the developer director in Fiori Milano, one of the world's largest trade fair and exposition organizer, to drive its project to move trade fairs from the physical world to the Internet.
In 2007, I joined again the former Italian E-Dreams management team to found BrowFly, which was to be renamed Lastminute.com years later.
And so in BrowFly, I started focusing on infrastructure with all the challenges related to performance, scalability, reliability, and security.
I've been the chief system and infrastructure officer for a long time, introducing the use of CDN as the main web security protection.
In that period, I increased my interest and knowledge in security.
So in 2018, pushed by the incoming GDPR regulation, we decided security needed a specific multi -department focus.
So we created a security engineering department in the form of a guild.
So we have a few people full-time and the network of expertise are spread in many different company departments all around the company.
Got it. Got it. And that brings us to your current role at Lastminute.com group.
Tell us a little bit about your business. Well, so Lastminute.com is an online travel agency among the top three travel player in Europe.
We have multiple brands covering 17 languages. We provide a complete offer of travel products and services for customer in more than 40 countries.
This means flight tickets, hotel, car rental, combination of hotel plus flight, classical holiday packages, holiday breaks, whatever.
So in our company, security engineering team is the technical arm of security.
We provide rules to increase security and ensure compliance in particular for PCI DSS and GDPR.
And we are a principal actor in design phases to prevent security issues since the beginning of the development phases.
As a European company, most of our effort for security is drained by the GDPR regulation.
We had to redesign most of our internal processes to increase the level of security in all department, both regarding external and insider threats.
Well, having a solid solution to protect the infrastructure from external standard attacks allow us to focus on internal process and procedures to reducing our risks.
And so it sounds like GDPR has had a kind of a big, or regulations in general, have had a big impact on your security and your infrastructure.
And I'd love if you could, Marco, if you could share a little bit about what does your infrastructure stack, your technology stack look like, your software stack look like within Lastminute.com group?
Well, first point is that we develop everything we use.
So any piece of software used by in our company, except for administration tools, administration system or so on, they are all developed in-house.
Our infrastructure has evolved in recent years from a classical on-premises data center with a built-in machine, now to an hybrid solution based on container and the Kubernetes, in part in cloud and in part in the on-premises data center.
This provides us with the flexibility and the scalability we needed.
And then to reach the time to market requirement from business, we pushed the DevOps model, giving developers additional degree of freedom.
This new environment required a new security paradigm. A way for continuity, we can rely on the fail fast, fix fast, because anyway, there is a payback.
For security, we cannot fail because at the moment you realize you failed, it could be too late.
So we have always to deal with developer, you will always to find a temptation of security shortcut to get things done faster.
And keeping the control of all the development phases, with all this degree of freedom, this is a real challenge.
Yeah, I can see that. And in terms of, are there challenges, security challenges that you face?
Obviously, you're an online retailer, but you're specifically an online travel and leisure retailer.
What are some of the challenges, security challenges that come with that?
Well, I think the model and the challenges are very similar for any online retailer.
Yes, we have some peculiarities.
The first point, the principal point is we publish content.
I mean, our content is the price and availability of flight, hotel, anything else.
And this data are coming from a huge range of external sources. We are integrated with really thousands of providers, thousands of islands, thousands of single hotel, a chain of hotel, or a huge provider of hotels.
This combination with segregation of data is a high value content, very complex to collect.
For this reason, we are continually, let's say, under attack from a screen scraper, from people that want to store and use this content.
So, we collect this content, they want to get it and for a different use.
The problem of screen scraper is not a problem of performance in our case.
In normal days, not now during the pandemic period, but in normal days, we arrive to have 25 million or search in our environment.
They are not all from people because alpha from people, alpha comes from our partners.
But with this huge amount of searches, no scraper can impact our performance.
We have a different problem. We have an analytic environment where any single decision from the color of the button, where to locate the ranking of the result, everything is based on metrics, traffic, the user behavior, the final conversion.
So, these scrapers change the statistics.
So, our decision model are impacted by this traffic. So, we need to reduce the impact of scrapers, not for a technical performance reason, but for business reasons.
Got it. Got it. That makes sense. And I think we'll talk more about the scrapers a little bit later.
I wanted to ask you, over the years, lastminute.com group, you've acquired a number of companies.
What are the security risks that any kind of M&A introduces?
And how do you think about mitigating that risk when you have a new acquisition?
Well, it was a long path during this year with lots of merger acquisition.
Some of these were coming from a structured company, as when we bought lastminute.com, and some were a smaller startup or a business that was not structured enough.
So, we tend, whenever possible, to enforce our standards.
It's not easy. As I told you, we have seen multiple M&A.
We have different technologies, different maturity models, different approaches.
So, we try, as much as possible, to respect the peculiarity of each of this business.
For this reason, we need to move away from the opinion, let's call it in this way, related to the best practices.
Because we can, especially when the maturity level is quite low, we can go to have a fighting because we give for granted some rules, some way of thinking, while other smaller companies, not structured, has a different vision.
So, we need to move from this opinion about best practice to analytic numbers, security score, scan result, and standard solution.
So, it's easy for us to define clear targets to reach because when you show a security scan with all the security score that give you the clear picture that you need to patch this machine because they were not patches in the last year, then they have lots of problems, people can accept, even the people that has no maturity.
When you start speaking about methodology, when you speak about procedure, sometimes they think of this as an overhead that can damage, especially for startup, they can damage their creativity.
Now, switching subjects, Marco, to a little bit about your journey with Cloudflare.
I know you've been a customer of Cloudflare for a few years.
We have a detailed case study that outlines your use case. But for folks who maybe haven't read that, tell us a little bit about how did you find Cloudflare?
What was that original use case that you were looking to solve? Well, it was a long journey to arrive to Cloudflare.
As I told you before, as a chief system infrastructure officer in 2012, 2013, I introduced the reverse proxy model as a security protection layer.
It happens after the first DDoS attack we received.
Initially, the solution was designed in-house with some reverse proxy installed in multiple zones on Amazon, on AWS.
Then we moved shortly to a commercial CDN solution.
Just the fact that just the presence of this CDN was able to reduce the DDoS risk.
In following years, we added the web application firewall and other specific security features provided by the CDN.
Then around 2018, we decided to review this solution because we needed more flexibility and automation to manage the huge number of our properties.
We have a very big problem with our marketing that invent a new domain or a new application very quickly and very often.
Cloudflare was a well -known product. Everybody in our SRE department has a previous experience with the free version.
This is the important point.
It was quite a natural choice. We started investigating the capability of this new platform and we decided to move from the previous CDN to Cloudflare.
The focus has always been the on-website protection.
Preventing known attacks, blocking malicious user agent, limiting the anomalous peak of traffic, we rate control and this kind of stuff.
Got it. What can you share in terms of what the attackers, what the adversaries were trying to do?
Well, before introducing a CDN, we were exposed to the risk of very simple DDoS attack such as a malformat request, slow loris, reflection.
Obviously, by design, this attack disappeared with a CDN because the CDN forward you only well-formatted the request.
Anything that is out of a well -formatted request, they crash against the edge of the CDN.
So, in our case, after 2013, all of this attack was only bad memories.
We received before 2013, we received, as I remember, a couple of attack of this kind.
Always in the lower part of the stack. So, quite always at network level or with very basic malformed request.
And we still have a bot sending correct request in abnormal quantities.
And okay, we...
I know impartially we are able to block it with a CDN, but we know that it will be forever a method to find the right balancing in blocking anomalous traffic without impacting low fuller requests.
And at the time, were there other options, other technologies that you were considering?
Well, as I told you before, one of the reasons why we started in 2018 discussing the existing platform was due to the huge number of our properties.
So, our domains. We needed orchestration and automation.
Before, we were managing any configuration through a web console.
So, people that open a console, enter, click, save, and so on.
This was not very practical. So, we were, as in the rest of the infrastructure, we wanted to move from the classical model to infrastructure as code.
So, as we now deploy everything through scripts, we decided to use the similar technology even for managing the CDN.
So, Cloudflare offered us the possibility to use Terraform, that is an orchestrator.
And so, now we are managing the entire environment with Terraform.
So, with a more detailed control of what is done. So, basically, we change our code.
So, the platform is applying the delta for the new changes, or in case we can even say, okay, drop everything and rebuild from scratch because we have everything we need inside our script.
So, okay, there is an agreement that nobody is going to change things manually.
But we can rebuild.
So, everything is versioned. So, you know what's happening exactly the same day.
So, everything is under control. And everything is considered as a piece of code that changes the configuration in the infrastructure.
And this was the element, the principle element that guided our decision to move from the other CDN to Cloudflare.
Got it. Now, I know you've been following our announcements this week for security week.
Is there anything that you've heard about that you've seen that has caught your attention?
Well, to be sincere, many interesting products. We need to enter more in detail, obviously.
I think about the effect of the COVID-19 will change some processes forever.
Even the most conservative enterprises, they were forced to deal with the remote working.
And at the end, they will never come back as they were before.
But the new working approach needs new tools. For sure, we are reviewing detail of the opportunity to change our classical VPN model to use some alternative solution, like I see or MagiWanna or MagiFirewall.
And the same is for some product to increase the security of our data managed by remote colleagues.
Once, when everybody was in the office, it was easy to control the box.
And so, to protect all the people inside and protect the work they were doing and the data they were managing.
Now, they are remote and with a large number of people, we have some rules not to create problem in the network.
So, we need to control because they are not full -time attached to our VPN.
So, they work even standalone on, I don't know, Google document or things like this.
So, we need to be sure that they are under protection in every moment of their working activity.
So, for this reason, we need some product to increase the security as required by GDPR with data loss prevention tools or ransomware or malware filters.
So, probably we will enter more in detail even on this kind of product.
Great. Glad to hear that. What are some other top of mind security challenges and anything you can share in terms of how you plan to address?
Well, as I told you before, GDPR is for sure one of our main priorities with all the possible impact that GDPR can create.
As a travel company, we deal with personal and sensitive data on a large scale.
We, in the last two years, we had a huge investment, internal investment, to introduce pseudonymization.
So, we offer PIA data managed by our platform. This means that now in all our databases, in all our storage, you will not find any personal data.
You will find a token and you need to deal with an external platform to return from this token to the original personal data.
This will protect for any kind of direct attack to our databases or any data sources.
So, now that we made this huge effort, the focus can be moved from the risk prevention of direct data leak, steep injection, unauthorized access to the database, whatever, to the risk prevention of application behavior.
Because obviously, our application provides data in a clear mode.
So, our API provides data in clear mode. They deal with the database, they transform the token into clear test data, but they provide.
So, we need to be sure that all the and all the, our application has the correct behavior.
As I told you before, we develop all application in-house and we know how hard it is to verify custom application.
Because, okay, if you use a standard application, you run your scanner and they say, okay, your application needs to be updated.
But when you have a custom application, you need to enter in detail to understand if everything was done in the right way.
So, what we can do is continuous training and we follow the team with an architectural review.
So, the guys on my team are expert or a system or in development.
And so, they sit down with the guys when they develop something and they review together with them what they are doing before they write any piece, any line of code.
Just understand, be sure that at least at the design level, everything is done in the right way.
Then there is implementation and a week, which is another point that is hard to follow.
But in general, at least we are sure that they do not shortcut some security rules or they decide to, they take the wrong choice.
Now, looking forward to the rest of 2021 and maybe even beyond, what are some of the other areas of investment to help reduce your attack surface?
So, we can divide our attack surface into main area.
The one is the external one and one is the internal one.
From the external one, we need to increase the security of our exposed services.
So, we are planning to introduce a stronger governance for API with all the related tools, such as a modern API gateway to control the access and the behavior of this API.
Internally, we need to increase the control of our processes and introducing a stronger data loss prevention tools to keep the processes and data under control.
And any advice for other security leaders or practitioners as they look to shore up their defenses?
Well, especially for European companies, GDPR is a really critical issue.
A mistake can lead to the end of the company due to the huge fines that they can receive.
So, the main effort should be on protecting data in the larger meaning.
PA data is everywhere. DBXL file, email, logs.
It's really mandatory to keep all this source of data under control during each step of the lifecycle of these files.
It's common to make an audit on a file server and find the tons of Excel full of personal data that was used five years ago to make an analysis and nobody deleted them.
So, we need to keep under control of all this content.
So, we need to define a retention and then the procedure to destroy whatever is not needed anymore.
In this moment, I mean, especially for European companies, I think this is one of the most important points because the breaches, one of the most critical breaches was done not with Hacker that were able to do some strange stuff about YouTube people that lost the PC full of data or they lost a USB key that was full of data or things like this.
Excellent. We have about a minute left. Maybe one last bonus question for folks that are early in their career in cybersecurity or maybe looking to switch over to cybersecurity.
Any advice that you can give for them?
Well, today security needs multiple different profiles from the technical one to the functional one.
Probably governance skills are the more important than the technical ones.
In my experience, we never missed people able to implement a security solution or security control.
We had a lot of people able to suggest the right solution and the right control to implement this prevention.
Even if you're not interested to get a certification, a training such as CISSP or similar is a good point to start to building a basic knowledge needed in this area.
Awesome. Well, with that, Marco, I wanted to just close out, say a big thank you for joining us today during security week to share your insights and your experiences.
Of course, thank you to our viewers for staying tuned and please do stay tuned.
We have other Cloudflare TV segments and the rest of the announcements during what we expect to continue to be an exceptional security week.
Thank you again, Marco.
I really enjoyed the conversation today. You're welcome. Thank you.