Immerse Sessions: How airBaltic & Ridango protect against attacks with enterprise services

Welcome everybody. Thank you for joining us for this session. This session is going to be a little bit more relaxed. We're going to have a conversation here with a couple of customers. We're going to talk about their example, how they've came to CloudFare, the challenges they had, and then ask some questions about those challenges. more back and forth, a bit more natural. So I'm going to leave here now, sit down, and have a bit more of a casual conversation. So hi Ziga, hi Kaspers. Good to meet you again after we spoke last week and previously as well before. So let's first start Ziga with yourself. You're with Redango. I think both of these companies are based in the transport space, but can you tell me a little bit about Redango the company and then a little bit about your challenges that you had and how you ended up going with it? Cloudflare? So we're actually a system solution provider for public transport. We basically provide global IoT solutions for public transport authorities or operators. What does it mean? We basically provide systems for managing buses, trains, trams, for payments in public transport, meaning that you know the credit card payments, mobile app payments, physical card payments and so on, right? So whenever you see something like this there is some magic that happens in there. end this is actually our systems that are managing this and also the systems that are making sure that your buses are running on time and of course passenger web applications for all of those systems right so that you can buy your tickets online or see where the buses are or trains and so on right so how we actually came to Cloudflare right we were passively using it for the last three years we actually started with the free plan where we were kind of adding in new systems and services we were making relatively passively right but then we kind of received a wake-up call right because most of our customers are usually some form of some sort of like a government usually they're like city authorities like for example our customers like our city of Tallinn city of Vilnius multiple customers in Sweden and all around the globe and our systems which were actually like their their public websites were starting to get like massive DDoS attacks right which we couldn't handle purely by scaling up the systems because they were like very rapid very massive and so on so we needed to find some very quick solutions on how how to fix them right as we were already using the cloud there this was kind of the most obvious choice and we continued with it but as we were continuing with the implementation we actually we saw that our system is a bit more complex for just like a pure lift and shift to be fully behind cloudware. So we needed to make some small adjustments and especially we required some very special services in order to achieve that. And in the end, right, I guess later we'll go in a bit more details, but in the end, we basically provided like an end-to-end flow that everything goes to cloudware, everything is fully protected. since then we received a couple of DDoS attacks that were either fully mitigated by Cloudflare or by WAF and so far so good and let's let's hope it continues like this. All right well we'll come back to you in a little bit so Kaspers do you want to tell me a little bit about AirBaltic again obviously a transport organization but tell me a little bit about AirBaltic and how you ended up using Cloudflare and we actually had a conversation earlier on to come back to as well okay yeah so air baltic is an airline actually has been quite popular here in czech republic lately especially our latest campaign and and we it was actually when we started our journey with cloudflare it was i think 2015 it was a time when bitcoin was 300 dollars so it has quite a long time it was very different times actually So, and how it started, we had an, it was on Friday evening, or Thursday evening probably, and we just, suddenly our website went down, and basically we lost internet and we started what's happening, and we got a massive DDoS attack on our website, but it was so massive that our internet service providers black-holed our network to protect our other customers, because they were starting having issues because of our attack, so we were cut off basically. So it shows that whatever we would have done on our end wouldn't have helped. So and then we got an email asking please pay 10 bitcoin or we just will continue that tomorrow and then basically we will not stop it until you pay. So it runs for one hour then everything stopped we got our connectivity back everything good and then we got an email so 24 hours countdown starts now. So it was a fun evening for me I was just thinking what to do. Googling around, Cloudflare wasn't a big name at the time, And I was somewhere in some forum, somebody said, yeah, we just had similar case, we just switched to Cloudflare and that helped. I said, okay, there's nothing to lose, basically. I just went to Cloudflare website, checked their best DDoS protection plan, which they had, which was $200 per month, paid with credit card. I mean, normally such change would require a lot of QA testing stuff. We just did it. like snap it took me like one hour to to to move whole domain to Cloudflare change DNS on our registries and it worked kind of we switched on all protections kind of fine did smoke testing didn't break anything everything fine so next day came we got more emails from the group telling that yeah you're not paying we didn't get our Bitcoin said that I didn't buy them way for precaution just to be on safe side and and then time 24 hours passed and just nothing like I don't know if they tried didn't try but basically that's it we were safe and and since that moment we were on cloudflare and and we also yeah there were more more key moments like when we switched to bot management which we very much had similar experience like the previous talker so and then these also we can maybe talk but but this was how we got there great thanks very much just going back to you Zigo you were one of the other things you mentioned to me was going a bit to the point there about speed I think was when you were merging you had another company that you needed to merge in with Rudango it was an acquisition I believe yeah I actually came from a Slovenian company that was merged together with Estonia and Rudango right and on that side of the company where we were very already using cloudflare for most of our services we were pushing everything there but this was going quite passively right and we already started this process on the remaining of redango services right but usually on all of those like passive projects it works until it doesn't right when you encounter some blockers or maybe some small redesigns needed and so on and as you probably a lot of times know that everything security can sometimes have like a secondary priority right then some of those blockers they were quite hard to overcome at the time right until you basically get like a kick in the butt and they need to be done ASAP right the interesting part about that was as our systems at the moment at the beginning were quite different and they weren't as consolidated and so on right it also required some structural changes to optimize the whole flow right also optimize the the endpoints the rules the the basically inside network rules right because as the as it was mentioned also on the previous previous discussion right once you move your core systems there there is also like a very very large part of like internal services they need to run there. Web check, CICD, or some internal users that are doing, let's say technical actions that maybe Cloudflare would detect them as like malicious. For example, testing some Postman API calls and so on, right? But in reality, these are services that need to be done, So in the end, there is also this internal kind of like a process flow that you need to consolidate over. In the end, I must say that this wasn't that difficult. because it's like quite easy to do this with like, you know, WAF has a very easy graphical user interface. It's very easy to define the rules. You can set up this list, integrate it and so on. So in the end, it went like quite smoothly everything. Great. Good stuff. Kaspers, just coming back to you. We were talking earlier on about one of the key features that you found with Cloudflow was the ability to abstract from what is front facing and customer facing to how you support the business from an infrastructure perspective at the back end so it gives you the ability to use multiple cloud services or infrastructure to support that and then be fairly fluid with that can you just explain for the purposes of so yeah for us we are we are mostly our strategy is based on best of breed approach so for every business need we actually are trying to find the solution which fits which fits best and this is also driven because of industry we in many cases we need to use there are very limited number of suppliers which offer aviation specific services so we actually are forced occasionally to use one or other technology one or other cloud platform or anything so we need something to bring this whole mess together as there and and we have actually found that Cloudflare is one which gives us this very much enable system have this whole flexibility because we are still having very very sizable on premise operation we have our own data centers we have or also some stuff which we run on cloud ourselves we just launched a mobile app where we have all the middle we're running on AWS well we have some API's running on premise and and so with that and we also some include you need to include data from customer loyalty program, which is SaaS service and so on. And to bring this whole together, we have Cloudflare where we can bring all the things together. And we have a single pane of glass, which is, as I told, even if Cloudflare didn't have any other services, like completely just logging, it would already give so much benefit, like logging and reporting. Reporting is great on Cloudflare itself. Nevertheless, it has limited in the time, how much they store. So we are actually pushing those. logs to our own elastic and then we can do magic there we have colleagues which do our own machine learning stuff on top we can run machine learning on all blogs to identify some threats which we might have had a year ago we had like just we had a major sales campaign we wanted to understand how bots were acting last year during the same campaign we just take cloud for logs run through it find some patterns apply the protections to this year's campaign things like that which wouldn't be possible without Cloudflare because then the things would be in different places and everything so we can just and we had also load balance we can quickly shift that we have not enough capacity on our own premise we just run some stuff on Amazon on Cloudflare we just either load balance or just shift some parts of website somewhere else as we are doing the micro services architecture that's like really the missing part which which can bring that stuff together yeah Brilliant. Coming back to you, Ziga, one of the other areas that you talked about was application performance as well, because you have a lot of portals either for internal agencies, but also there's some end customer facing. Yeah. Yeah. So kind of the main reason that even like in the initially with the free plans and so on, before we moved to the enterprise, we started to use the cloud fair was basically security. It was basically first point DDoS protection, second point WAF. third point everything else right um but as we were onboarding more and more portals and more and more systems behind it right we also started to to use a lot of those like nice to have features that you get with there right um application performance caching we also very start actively started to use the ssl management and so on right um the application performance itself it's it's i mean objectively speaking it's not a feature that we would pay for because use case right um because we are mostly like b2b focused right um it maybe has like a smaller impact but because it is there it is actually like a huge uh huge boost right um what we did notice was right and this i must say like a kudos to some of the cloudware engineers that were there in a couple of webinars where they also helped us to kind of like operation wise explain or give some tips how to better manage security especially related to ddos caching right they also explained like a what a crucial part of the DDoS preventive strategy is caching right even like if something does go through the WAV through DDoS protection and so on right kind of the third layer that can potentially mitigate this is also like a static content caching on the cloud there which in the end reduces the amount of requests that come to the that come to the back-end services to your audit yeah absolutely I'm coming back to Kaspers, I'm going to keep doing this back and forth. I'm quite enjoying this. You've also been looking at things like latency reduction, also doing server-side rendering as well to take latency out of applications. Do you want to tell me a little bit about why you would do that and the impact it's had on the front end? Yeah, of course. I mean, we just again, we mentioned just a campaign which we had a couple of weeks. Actually, it's still running. still get the ticket prices but especially when campaign starts and people and and we had issues technical issues last year in the same campaign when people just brought down our website which is kind of good thing a lot of people willing to buy tickets actually and we already uh get rid of bots because of qualified bot protection so it so we made sure that it's only people who who come during this campaign and then this year we prepared we really uh worked on on cash We worked together with developers, we very much adjusted what we can cache, how long we can cache it, and then we could, we figured out that we could even cache API responses, not only web, so I mean, we were already caching all the static content, but now we worked on dynamic content caching, HTML, and also API request caching. And with that, we could bring down load on the backend system significantly, and of course reduce latency for for customers so this just gives gain like more revenue for business and and of course yeah so more also more happy customers and you're also using our workers platform for yes augmenting bot management I guess that that's based on what you're doing from the logging analysis that you talked about previously exactly that so basically when we started our bot management journey talked with developers and they said that we don't want cloudflare to block the bots actually we want to know that these are bots and then on the back end we decide what to do with them so we used workers to first add bot scores which cannot be done by cloudflare itself but we also did some additional logic in the workers which basically decided what back-end should do and then back-end it can either like present some stale pricing if the bot is scraping for pricing we could just give the cheaper endpoint to the bot so it still gets something but just not as expensive for us in in terms of back-end capacity or we can just show captcha for like if it's api request then if cloudflare would put up the page that would break the apis but we can send some uh some different api responses or we can integrate the captcha better in our UI than Cloudflare would do. So with the help of workers we basically amend our our bot management and make a decision is made on worker's side but then origin is acting on behalf. We just had a great example with our famous city voting campaign where we had very very it was much higher demand and more people came to vote and also lots and lots of bots were written to vote and we did exactly that. Cloudflare bot management was our saviour again in this campaign. We gained, especially from Czech Republic, the Czechs were the most active voters. I don't know why but it was a complete surprise for us. Nevertheless and then also there are hundreds of thousands of bot requests coming in and we could with a Cloudflare amend it with our own uh what management uh with our own logic on on top of elastic we could very successfully filter out all the bots and still get only human votes counted and we didn't get anyone complaining that we had something wrong there yeah that's a really interesting um story there particularly where not only you're using it for scoring bots but then serving them at a lower cost so you mitigate them make them happy enough that they think they're getting something, but then it's still saving you money and also mitigating that bot at the same time. This is an important part, which I think we have learned, is that bots, while you observe them, they're actually quite visible. You will see them, you will see, but if they're not harming you, maybe it's even better let them go, because as soon as you start actively blocking them, they become more stealthy. And our approach is that first thing, either we block them just before in sensitive moments, like one hour before the campaign starts, we... just blocked the bots. We saw them actually before but we didn't touch them and then dumps they are and then they don't have time to react. Another thing like with a voting campaign we don't give them immediate response if they were detected or not. So basically we were updating results just once per day we saw the bots but we allowed them to vote. They got the same response as humans and then we just filtered them out in the backend so they can't adjust their algorithms because they don't know. if the word was counted or not it's just we who know that and this doesn't tell because they also use AI they use machine learning bot operators are not dumb they're actually quite smart people and and and that way we don't give them data which is what they need yeah thank you going back to yourself again Ziga so one of the things that you mentioned to me when we were talking earlier in the last week was some of the client client communication that you have because obviously you have a lot of like we said earlier agencies that use your services and how you're using cloudflare to secure that um do you want to tell me a little bit about how that was uh beneficial for you yeah um so one uh or just understand like some some uh how some of our customers actually the the systems and so on are structured right is that basically we provide the back end or even the front end of the systems like also for the passenger facing right but of course they don't want their portals to be like ridango.com right they just want it to be like their city right so and what we are also able to achieve through this is that that we are able to route customer domains through our cloudflare so meaning that for example if you go if you go to the to the web pages of of some of the cities in estonia for example right that traffic actually even though it's not our domain that traffic does go through cloudware so we can control the data flow we can control the security we can control anything that happens there right and then in the back end basically goes to our back end our cloudware and so on right so this was this is something that has been especially useful for our use case where in most of the cases you don't directly control those domains and sometimes let's say customers that have their own domains for you know they're usually public sector they sometimes they have their own rules right maybe they wouldn't be that keen that they would directly switch to Cloudflare, right? But this way it's like extremely convenient to them, you know, all it takes is like one DNS change to go instead of directly to us, then it goes to Cloudflare and then on Cloudflare we direct it to us, right? Yeah, and also you've got quite a wide region as well, you've got multiple countries. Yeah, maybe I was in beginning more focusing on Europe, right? But we do have a lot of customers here in the Baltics, in the Nordic Sea. in the Eastern Europe, but we also have a lot of customers in the Middle East, also some in the Africa and especially in the Asia and Australia. Right. Yeah. So, you know, let's not underplay caching in that regard. Right. So that has been a big impact. Yeah, definitely. So the let's say caching and maybe the edge computing. Right. It's one of those features that we are actively using, but maybe in our particular case, it wasn't like. of the main reasons we started to use it right usually for those customers that are let's say in middle east or in australia we still host their systems and data somewhere locally right so usually like within a few hundred kilometers right so in that sense it's maybe like it's not that huge of an issue on the latency and so on but again because this is something that is bundled in right we are very happy to use it we use uh also like edge computing for their side. In the end, the customers, they get like a better response time, they get a better experience, right? And everyone is happy, especially in, because in some of those cases, right? The cloud hosting regions, for example, they are in, I don't know, United Arab Emirates, right? But our customers are in Oman. So it's not directly local, but this way you can still provide them like a better, better customer experience. And in the end, business customers are happy when their passengers are happy and everyone wins right yeah tell me a little bit more about the how you're using workers edge compute right now and and are you looking to expand that usage um we are we are passively expanding it with new customers and so on right but at the moment we don't have any major any major plans uh related to the to the edge computing itself but we we do have some extra expansion plan plan for our IoT platform so because we do manage thousands of devices across the globe there needs to be like a special mechanism how you can actually protect them right because they are they use different SIM providers right so it's not something that you can directly do there and so on and one of the main reasons we actually came to the to the enterprise plan was because simply functionally wise the the business plan was not sufficient for us in order to protect our iot devices we needed to go further than the classic tls authentication right we needed to we need to switch to the mutual tls so this way that instead of just saying that you you trust the server it actually goes the other way around and you can say we trust the client so meaning that when devices are connecting to the back end you can you can see which are legit which are not because especially during this transition you know, until you kind of have all the endpoints behind Cloudflare or behind the DDoS protection in that sense, you know, there, you still have some holes and you're not really protected in that sense, right? Tell me a little bit about these IoT devices. What are they that interests me from the point of view of your business, I'm wondering what these devices are? Yeah, so the ones that are kind of the most obvious are, for example, validators, ticket vending machines, point of sales devices, so where you go, you can buy tickets or when you go to the train that... you scan your QR code right but the ones that are a bit more hidden are actually like small computers that are hidden inside the trains or buses which basically control everything that is happening there right both from the point of data collection so on second to second log intervals but both in the sense of managing items there right so for example when you get like audio announcements when you get like some ads shown or some LCD portals right And in the end, also managing those validators, ticket sales and everything that goes there, right? It is a bit different customer to customer, but this is kind of the magic that happens. Yeah, does some of those, does some of that compute inside buses and etc. do things like schedule timing and timing to arrival and that sort of thing as well? No, this is scheduled on the backend, but it is then synced to the buses itself, right? On the scheduling... so the schedules itself are usually defined as like a fixed times because they have to be shared uh but we actually do have like a ai platform that automatically calculates in real time all the time estimated time of arrivals so in the cities that that we provide our services to right passengers can know like uh half an hour hour or two before like if their bus will be late early how late it will be right and if you know in advance that your bus will be like 15 minutes late it's like okay i can grab a quick coffee I can go there I can still take it and everyone is happy very useful are you considering using workers AI maybe to offload that very honestly no the reason for this is because our systems are built in an agnostic way that they also sometimes have to work in on-premise data right very honestly I would love it if we could be like cloud only but we do have certain countries or regions which are not that keen on cloud technologies right and it is like a relatively sizable part of our business and it's simply a business strategy that that we can't lose those customers or you know that we can't lose that part of the market to be competitive. No understood. Casco is coming back to AirBaltic again being an airline you also have a wide audience around the world so obviously I expect that caching and so that basic reach of that you get with Cloudflare has been very important for all of us. as well. Sure and we can actually also amend a bit of what's just Zig at all. Basically we are now in the migration process. We currently still run quite an old school content management system which runs on our on-premise servers which we then publish through Cloudflare but we are in the migration process to headless cloud-based CMS and also our development teams want to do a search. server-based rendering for content rendering so they have chosen Nuxt as a platform which could do that and we are currently very much on the process we want to test if we could move the server based rendering to the Cloudflare to run on Cloudflare pages, Cloudflare workers, maybe even amended with Zara's for tag management which is also a major thing so we are very much looking forward that that could both increase our security but also reduce latency because then the code would be actually rendered next really to the i mean for customers anywhere in the world they could get it much faster but but this is like for our project which we are really looking towards because yeah this is our new website which we are now building great and how does cloudflare help you when it comes to scaling because typically you'd need to scale you know manually and of course if you're developing server-side rendering or you've got workers in front of other services that the scaling is to an extent taken care of. I mean it's essential of course and especially what we are doing now we have built our Kubernetes platform where we can do of course for backend we can do auto scaling and we can scale on our own data centers we can scale also to the cloud. actually could do it on any cloud, and then Cloudflare is then basically steering the traffic where we need it. So if we need to scale out, we just configure in Cloudflare, I mean, we just have it pre-configured, so it will send our customers to where the computers, yeah, so where we want it, basically, exactly, and together with the tires caching. which i suddenly discovered sm for some time it was like separate paid service but some time ago i figured out oh it's suddenly included in our plan without additional costs okay great thing so so that's actually good to go through the web page every few months yeah like some new features this is actually one one thing which i have learned i mean occasionally i need to onboard new people on the cloudflare or some people ask hey please show me the cloudflare and they're showing like yeah and here you can do this thing and thing and this oh what is this button here oh and this thing oh they have this new feature i mean every time i mean even if i did it month ago after a month again i am always like wow what did this uh i mean my colleague christopher sitting he can remember that when i was showing him it was many times like oh this is a new thing i didn't see it like before and so on and then immediately like ideas oh and this we could use that for this purpose and so on and some of these things that oh i was so much waiting for its future to to arrive so that's great to hear so that leads me on to one of my last questions for you and that's what are the sort of future things that you see doing with Cloudflow or the projects that you're planning in the near term to expand your use you know and improve the services that you provide Thank you. Yeah, so you start moving more of the actual AirBaltic platform to workers and then have less reliance on the third parties. Or workers, or pages, or yeah, or Zara's is a nice example as well, AI, Web3, yeah. also actually quite active in blockchain sphere so we also could so we have currently looked a bit on those services and for sure in the future we could do much more. One question that's not Cloudflare related but it's something that always I find fascinating a lot of airlines still at the back end somewhere have mainframes. Do you have one? No, we are fortunately new enough so we didn't have left to hear it. We are really moving to microservices and everything new we build, we build already with modern architectures, but as I said, we are in the aviation sector, it has a lot of legacy. We are still using telegrams for data exchange, I think x25 are things which we use, maybe others have already forgotten such existed or maybe learned in school, we still have to deal with those technologies and integrate. them in in modern with modern so it's it's it's really exciting yeah there's quite a challenge isn't it being being very front-facing by using edge compute but at the same time then having things like telex and x25 on the opposites that's sublime to the ridiculous yeah yeah but you can't easily get rid of this because it's so well baked in it's it's in in the in the environment that's probably why we still have eight digit codes for our bookings Yeah, exactly. I mean, this is actually changing now. There's a new NDC and things, I mean, environment, acronyms which are probably only known to our industry, but this whole thing, how tickets are bought, this is actually changing from the old GDS type of approach and all those booking codes to completely new approach, which is very much based on direct networking between airlines and it also could use Cloudware for sure. this great okay it's good to hear um come back to you ziga so um same question to you um what's your what are your near plans for for expanding the offering you have and where do you see cloudflare fitting in um good question um in in general right um we are building some new solutions at the moment and what we have done is that basically from the day one we were considering architecture is how we can establish this end-to-end connectivity especially related to security so that you can provide this like airtight airtight solution right so how to consider everything from from basically traffic flows from VAV DDoS protection right and even here in the initial architecture even for example caching is being considered and performance and so on but again with limitations that at the end of the day this also to work without cloud fair right so in that sense we are trying to do everything in a way that we wouldn't get locked in right even though that i must say that i never actually had like a feeling that we would be locked in right um but uh you know this way this way you can kind of utilize most of the features you it's quite easy to use right and it's kind of like a mental exercise when we were kind of considering moving to the enterprise plan you know because the cost also increases and so on uh were also considering okay what if the worst happens what if like our price increases too much and so on right and basically we figured that in order to migrate the whole stack away would take that take us less than a week right so this way we also have this kind of like a feeling of security that like i don't know if we wouldn't get the value we feel that we would need it would be relatively easy to move away i'm not saying that at the moment we're not getting it at the moment we are quite happy but obviously it is important for anyone to feel like they do have that mobility yeah yeah because for example you know there are services which in reality unless you recode half of the solution it's almost impossible to move away right but here it's like uh at the moment you know there is this like a sense that you know it's not kind of like a lock-in that you have to be here right but you kind of have this feeling uh we're using this because we want to and if we if we ever see that i don't know the value wouldn't be there which i doubt it honestly but, uh, but, uh, but yeah, then, then you can also consider the alternatives. But at the end of the day, it's kind of like, if you look what you get here, right? Because as I mentioned before, a couple of times, it's like what we came because of security, right? Um, but on the path there, we also encountered a lot of new features, which are actively using. So kind of, if you draw the line, if you look at the bundle, it's like a lot of services, if you would kind of want to have all of those, like one. one it is like a you know in any kind of like hyperscale that you would have to pay for. I think like Casco said it's important to go back to the console every so often refresh yourself yeah yeah exactly see what's new see what's come along and see if you can utilize it straight away exactly absolutely all right so we're at the end of our time now thank you so much to my guests here it's great to have real customer stories and success and understand exactly what they're doing and learn a little bit so we can all helpfully replicate some of that success in our business as well. So I'd like to thank both of my guests here. Thank you very much.