People Behind the Packets

Hello and welcome everyone to the third episode of People Behind the Packets. This is the Cloudflare TV show where we have conversations with leaders around the world who are building the underlying infrastructure that the Internet relies on. My name is Nitin Rao. I lead our global infrastructure team at Cloudflare and I'm honored to have as my guest Michele McCann who's the head of interconnection at Teraco in South Africa. Thank you so much for joining us, Michele. I really appreciate you joining us. When you think of the underlying layers of infrastructure that we don't spend a lot of time thinking about but are so critical, one of them is hardware, data centers. One of them is, of course, connectivity. I think you're one of the leaders who's had just an outsized impact on connectivity in so many countries in the world. But for folks who aren't on the sort of peering conference circuit, would you mind just taking a moment to just explain for the average Internet user, what is peering and why should I as an Internet user care about this? Sure. So I think the key part of it, and I'll use Cloudflare as a great example as well as how you've invested in Africa. So as you know, Cloudflare obviously hosts a whole bunch of very key, let's say, websites. We as NAP Africa run what's called an Internet exchange, simple forms. It's a whole bunch of switches located in a really big data center. And those switches basically connect network operators to someone key like ourselves like Cloudflare. So if I'm a mobile user and I want to access website X, Y, Z, it will choose the quickest route to be able to get to the Cloudflare infrastructure. So thereby actually making content available to you and me on the street. Okay. And can you give us a sense for how large the... You run NAP Africa, which is now the largest interconnection point by long distance in Africa. And the three of them across South Africa in Durban, in Johannesburg, in Cape Town. Can you give us a sense for just the number of countries and Internet users that are impacted by it? The scale is staggering. So right now at NAP Africa, we connect to over 450 networks doing 1.5 terabits of peering capacity. And that's servicing 16 different African countries. So if you have a look at the map of Africa and you kind of go Angola all the way through a straight line, all the way across, all those countries below there access their Internet and their content from NAP Africa. So you can imagine that South Africa, Botswana, Swaziland, Malawi, Zimbabwe, Mozambique. And so the islands, Mauritius, which we all want to go on holiday to. So while everyone's relaxing on the beach and watching their Netflix or ordering things online, that's all coming across NAP Africa. That's amazing. And really critical Internet infrastructure. Can you talk a little bit about how the relationships came together over time? So that's not an easy feat getting so many networks to come together in this sort of coordinated manner. One of the things I've always respected is that no matter which part of how you're associated with the peering community, you have been a great sort of partner and advisor to all the companies and networks. Can you talk about how you built those relationships over time? I think that the key is that Africa was always this untouched market. And there was not a lot of knowledge and still isn't a lot of knowledge. People generally look at Africa from what they see on the news, which could be quite scary. So it was generally around my team going out there and explaining Africa and that, you know what, we really actually do have stable infrastructure. We really, I mean, you see all the stuff on the news with the power outages and all these things, you know, but there's ways to get around that and ways to implement great data centers that keep that up and running. So I think it was around actually being an advocate for Africa. And our team has spent probably the last eight years really hard going out there and being advocates for Africa. So it's not such a bad place. It's not all what you see on the news and it's okay to invest in Africa. For so many Cloudflare employees, we have a hard time saying which ones are our favorite deployment, but the one answer that comes up several times is Johannesburg, because it was so exciting to see what was at the time the first of now many deployments in the region. How about the relationships with the ISPs themselves? So if you're a network in Malawi, for example, like why should you participate in an Internet exchange like this? Previously, before guys like Cloudflare invested in Africa, the shortest route for you to be able to get content and think about it, it's only eight years ago, was Europe. So that adds, you're sitting at like 200 milliseconds of latency. So, you know, Africans were very used to in those days, that little buffering circle, because we didn't have anything online. I remember Facebook when it started, it was kind of like you would upload your photos and your friends would see it the next day. It's not this instantaneous. We didn't have Netflix. We didn't have Google. We didn't have any of these applications that today are readily available. So someone like, let's take a Malawi for them, what would make sense instead of them and think of it kind of like driving on a road, you know, you want to go to the shortest way to get to your destination. So there's no way someone in Malawi is going to drive all the way to Europe and then come all the way back down to South Africa. So it's easier for them to get in there four by four and go through a whole bunch of borders and then end up in Johannesburg. And that's pretty much what we've seen around a lot of the African countries. There's still a lot of work to be done, and I think it's more knowledge and understanding within the African countries, but we're getting there. And so what's your vision for what's next? Like, what do you have planned next for NAD Africa? So what we've done, as I mentioned now, is there's a lot of skills that need to be developed. You can imagine we've only, if you look at Africa, only been part of this world, the Internet game for a very short time. I mean, eight years is not much. So we spend a lot of our time on skills development. So making sure the engineers really understand what it means. We look at the youth and see how we can incorporate the youth and bring them in to be the new advocates of the Internet and how to work on the Internet. So we run a lot of events, we do something called Tariqa Tech Days, where we bring knowledgeable people like yourselves to come and have a talk to Africans. Because normally, you know, us being able to talk to someone like a cloud player is like dealing with a rock star. So for the African guys, it's generally around how do we actually upskill, and then through upskilling means more and more connections that are actually happening to NAD Africa. The other side and the other what's next is there's a huge amount of undersea cable capacity starting to come into Africa. And it's been happening over years, but there's large cables like Equano coming in, the 2Africa cable is coming in. And what that does is start driving down pricing. And with pricing coming down, someone like a cloud player can release more content. And with more content, it means the user on the street has got access to more information. And these cables quite literally like land like by like at your facility. So what better place to interconnect than literally where the cables land. So that's perfect. And it's the closest point to meet all the network operators. So the guys who service the mobile operators, the fixed line operators, they're all in our data centers. So it's easy, you just run one cable and you're suddenly literally on the networks and accessing the users. So switching around a little bit, would you have imagined yourself like doing this role of like how did you end up doing what you're doing right now? How does somebody become, you know, get to run a massive Internet exchange? Well, it was more by chance. No, I didn't. I would never imagine that I would be doing this. So probably a little bit of history was I ran my, before joining Terrica and NAP Africa, I ran my own ISP and then sold that off. And at the time, my previous boss before that, he called me and said, hey, come do this NAP Africa thing. So I said, you know, what is this NAP Africa thing? He said, oh no, it's this Internet exchange, it's for free. And at that time running an ISP was really draining. And I thought, fantastic, you know, it's a free product out into the market. How easy can that be? And here we're sitting pretty much nearly nine years later, and it hasn't been easy. I can be honest about that, but it's been an amazing journey. It's been, and will continue to be an amazing journey. When you start seeing people go online, you know, things like this pandemic that we're sitting in, if we didn't have a NAP Africa, and if we didn't have investment like someone like Cloudflare, you and I wouldn't have been able to have this call. You know, so actually seeing people get livelihoods because of the investment from multiple parties happening. And when the pandemic happened, I was like, wow, okay, now you're really seeing this thing in action and it's been amazing. How have the last six months been? It feels like everyone's lives have changed so much. What's it been like for the NAP Africa team? Not much sleep, but the growth has been amazing. So to give you an idea, it took us eight years to get the exchange to a terabit of capacity. And in the last five months, we've grown it by 500 gig. So it just, it shows you how people have moved online. And it's amazing from a South Africa perspective, because I mean, I'm going back, we've now moved to a more open level. So I'm going back to the data center just because I like going there. It's no other reason. I don't have to go there. I can work from home. But the traffic, there's nothing. So everyone has realized that online world, what used to take me two hours to drive to the data center and traffic now takes me 30 minutes. And everyone, you can see everyone has realized, even though the country has opened up, people have realized, let's carry on with this online world. It's a better place to be. Yeah, it's taking some adjustment for all of us, but I'm not sure we'll ever sort of go back to the world as it was just before this. It feels like it's hard to predict what's going to happen, but it feels like the world has changed pretty permanently, because we're used to having conversations like this. Absolutely. And I think that's actually, I mean, I'll use TerraCode Tech Days as an example is we used to host it at the data center. So it was very limited to only people could from Johannesburg, unless they were going to pay for a ticket to fly in and see whoever presenting whatever skill. And as COVID happened, we kind of said as a team, we don't want to stop Tech Days because one of our goals is upskilling people. So we said, well, let's move it online and let's give it a go. And I mean, now we are training people all the way from Congo, Malawi, et cetera. So the opportunity has grown from, let's say, 40 people from Johannesburg attending and getting these skills to now multiple African countries dialing in and learning great things around how the Internet operates. That's just terrific. That's such a terrific example of, that's a terrific story that you actually be able to reach more people. That's wonderful. For folks listening, and if you have any questions, feel free to send us an email at livestudio.tv and we'll try to get to them through the show. Mishal, can you tell us a little bit about the other sort of Internet exchanges? So if you're trying to set up an ISP and considering joining an Internet exchange, or you're setting up an Internet exchange and you aspire to be like NAP Africa, what advice would you provide? Because there are a number of other sort of nascent, promising Internet exchanges. So what can we learn from NAP Africa success? I think the step one is have a good team. So, I mean, we've got a fantastic, we don't have a big team. There's literally like four of us, but we're a very close team. And each person brings their own elements of success for the exchange. So I think first look internal and make sure you've got the right team driving the exchange for the right values, because an Internet exchange is more around making the world a better place and being able to have Internet access for everyone. So it's around that first, before having to look at how much revenue you could possibly generate from it. And in NAP Africa's case, we do it for free. So absolutely no charge. And when we started it, a lot of people thought we were absolutely out of our minds. And today we've got, you know, one of the largest instant exchanges in the world. And behind that is one of the largest data center companies. So, you know, it's actually just a culminated into a massive amount of success. So that's the first part of it. And the second side is around having a look at your community. So stay really close to your community. It is a community. And which is the best part about an Internet exchange is your members are your friends. And keeping that in mind, help them grow their businesses. So don't just sit there and get them connected and say, you know, I mean, NAP Africa, we've got a saying where it's connected, never forget. So we never forget our members, we make sure that everyone is constantly growing, especially members who are sitting far away from South Africa, you know, let them know what's happening. You guys see the news, but what is really happening on the ground and constantly communicating that and be that trusted source for your members. That's great. That's great. Tell us a little more about Terraco, the data center that hosts it and how Terraco has changed over the years. Terraco, we were the first neutral co-location and still funny enough are the only neutral co-location operator in South Africa. And so pretty much it's 11 years old now and we started out of a storeroom. So in Cape Town, so we bought over a little storeroom and with the idea to make it easy for people to connect to each other. So back in the day, you know, 11 years ago, you could only buy really expensive fiber cables and you would run from service provider data center service provider, which would cost hundreds and thousands of dollars to be able to do that. So what we said is let's get the storeroom and we'll convert it into a data center and allow people to easily connect to each other and not have any politics around it either at a low cost. So we launched that and we allowed people to connect to each other for less than $50. So people went, wow, this is amazing. So we had a rush of ISPs coming in. I know even my old ISP, I raced in there because suddenly I could save thousands of dollars on operating costs and started to do this. And then we launched another data center in Johannesburg and around the same time we launched NAP Africa. And then we started seeing, hang on, but Johannesburg is a great place to be able to service multiple other African countries because of its proximity. And then we started to kind of grow Johannesburg and then we launched Durban. And Durban has been fantastic because, I mean, it's taken a while, but we've now got the Metis cable, which is the cable connecting up Mauritius, Reunion, Seychelles landing in the data center. So that now starts opening up all those islands for the holiday makers to get easy access to content. So today we've got the three facilities and sitting at over a hundred megawatts of power that we are providing out of those facilities and growing. So we're building more facilities and more facilities and it's all been around making content and easily available to everyone and allowing people to connect. That's great. That's great. You mentioned Mauritius. I remember really being amazed. I think, hopefully I'm not the only one, but I think folks underestimate just how much traffic is in Mauritius because it's a lot higher than folks might realize. I think what's happening with Mauritius is because it's from a tax perspective, it's very attractive and a lot of South Africans are actually moving to Mauritius. And why not? You live on a beautiful island, you're saving on your tax, you're a four hour flight away. So we're getting a lot of South Africans actually moving to Mauritius. And who knows, maybe what's happening this year might make people more likely to... Absolutely. If you can stay online and live on an island, why not? That makes sense. Would you mind telling me more about your team and how the team came together? You said the team's really important and how did the team come together? How do you divvy up who does what? It initially actually started, Andrew Owens was there even before I was, and he heads up all the technical. So he is incredibly knowledgeable on a technical basis and just an amazing person who's willing to share his knowledge with a number of folk. So Andrew and I were just the two of us for actually funny enough, for years. So I was the one trying to hit it up and make sure Andrew got all the budget he needed and do all of those games. And then Andrew was the one who physically built the exchange from a technical perspective. And so it was the two of us for about four years together. And then we got Mark Brunt to help on the engineering side and also fantastic. And he came from an ISP background and running voice over IP networks. So for him, it was a total new world that he was entering. And he's grown with it and very charismatic. If any of you have met Mark Brunt, you'll definitely know he's quite a humorous individual. And then two years ago, in the name of growing the youth, we got Yolandi, she's 22, and fantastic from a peering coordinator point of view. So really eager to learn and grow and be part of the team. So that's kind of the team at the moment. And what we are doing, because it sounds really small, running 450 or supporting 450 networks, we spend a lot of our time around automation. So making sure we look at the day-to-day tasks and how do we automate that so that we can spend more time with our members. That's great. And I love how there's a sort of like a spirit of sharing amongst Internet exchanges. So just like, I think it was yesterday or the day before, I saw the post from INEX, for example, in Dublin, and how the peering manager tools they built are now used by, I think, well north of 100 exchanges around the world. Including ourselves. So it's great. We can sort of like, help each other that way. Well, I mean, I think that's the spirit of the Internet is, you know, it's an open network, it's about sharing, it's about learning from each other, it's about making things work better for the man on the street. What has surprised you along the way? I think you're somewhat matter of fact about like, you know, these really, really impressive things that have come together. What has been surprising? Can you just share any stories from along the way that surprised you? The recent one was the pandemic and how much traffic that was. So that did surprise me. But luckily, our European fellows phoned us and said, hey, just watch out, there's this huge amount of traffic coming your way, because South Africa only hit the pandemic about two or three months later after Europe. And then I think around surprises has been how the world has come together in order to be able to support Africa. So, you know, you wouldn't think when you look, as I've mentioned before, there's a lot of people watch Africa on the news and they go on a scary place, let's stay away. And the amount of people have actually gotten together and said, hang on, we can actually invest in here, we can make it successful, ignore the news, etc. And let's just see it grow and how they have grown. So it's not been a kind of a guess and it's gave it a go and it didn't work. You know, everyone has grown and everyone has been really successful out of it over the last eight years. Yeah, some of the fastest growth we've seen. Some of our fastest growth is has really been in South Africa and across Africa. So Louis, one of our on our network engineering team actually wrote a really fun blog, I think it may have been about a year ago, really like detailing all the growth we're seeing and IPv6 adoption and it's been it's been very, very, very exciting. What's been especially hard? And what is a particularly hard challenge along the way? I think changing people's mindsets. And that's a continuous, I mean, even now as we enter into the enterprise market, I mean, I had a call today and this enterprise guy said, why must that change? I don't see. And I'm like, but it's for free, you know, and it will improve your network and you'll be closer to the content. And so I think it's a mindset thing, you know, where people kind of get they get stuck in their ways and they kind of say, this is the way it should be. And it's because it's always worked that way. And when you go along and you say, but, hey, guys, you know, here's a better way. Here's a more exciting way. Here's some way to grow your teams, to do all these things. I think that's going to be an ongoing challenge that we need to face. So traditionally, when you think of intranet exchanges, you think of sort of, you know, the sort of the distributed networks that serve, sort of serve applications at the edge, you think of hosting companies and ISPs. But you mentioned enterprises, it increasingly makes sense, even if you're a local enterprise, it makes sense for you to connect to an intranet exchange. Why is that? And what has changed? Like, does it make more sense now than it did a few years ago? Why is that? So I think, I mean, in Africa, our bandwidth and transfer fees, they're still pretty high. So especially for the enterprises. So for them to be able to save on that has been key. And then the other side of it, from an enterprise perspective, is being able to control how they route things. As the cloud becomes a reality, especially in Africa, which it's now only recently become a reality, with Microsoft and Amazon, and these guys starting to really invest in Africa with big cloud regions. They're starting to say, okay, instead of me looking after the server infrastructure, you know, let me outsource that. How do I start outsourcing those things? And if I'm outsourcing my website, for example, to Cloudflare, and all of these things start coming together. And then the IT guys kind of sitting there and saying, but how do I control where this traffic must move to what? And that's where the big question mark is starting to come in. And that's where the peering opportunities coming in. And they're kind of saying, actually, I want to be able to, instead of somebody owning my IP addresses, somebody doing this for me. How do I know if I can trust them? Let me do this myself. The other side is they're starting to peer off their own content. So they're starting to see themselves as content owners, like banks and insurance companies. They're starting to say, how do I get my content closer to the user? And so they're kind of morphing in terms of that. Well, I think the NAB Africa team really has a lot to be very proud of. And I think as a community, we're very grateful, not only for the work that your team does for NAB Africa, but also how it's been a model for everybody else who wants to either set up an Internet exchange or learn about peering. So I know you have many demands on your time. So thank you for joining us and sharing your story. We really appreciate it. You said you might get a little bit of time off. Yes. So I'm planning to do a trip to the Safari in the beginning of October. And it's been fantastic because I got an email today actually saying you're getting a free upgrade. It's for my husband's birthday. So I was like, wow, yay. We're super excited about that, to go on a little bit of a Safari tour and take a break. Wonderful. Well, thanks so much for joining us, Michelle. Really appreciate it. It's been great chatting with you. Thank you. Thanks. Thank you, Nitin, for your time.