People Behind the Packets

Hello and welcome everyone to the third episode of People Behind the Packets. This is the Cloudflare TV show where we have conversations with leaders around the world who are building the underlying infrastructure that the Internet relies on.

My name is Nitin Rao. I lead our global infrastructure team at Cloudflare and I'm honored to have as my guest Michele McCann who's the head of interconnection at Teraco in South Africa.

Thank you so much for joining us Michele. So I really really appreciate your joining us and when you think of sort of the underlying layers of infrastructure that we don't spend a lot of time thinking about but are so critical.

One of them is hardware data centers. One of them is of course connectivity and I think you're one of the leaders who's had just an outsized impact on connectivity in so many countries in the world.

But for folks who aren't on the sort of you know peering conference circuit would you mind just taking a moment to just explain for the average Internet user what is peering and why should I as an Internet user care about this?

So I think the key part of it and I'll use Cloudflare as a great example as well as how you've invested in Africa.

So as you know Cloudflare obviously hosts a whole bunch of very key let's say websites.

We as NAP Africa run in what's called an Internet exchange simple form. So it's a whole bunch of switches located in a really big data center and that's those switches basically connect network operators to someone key like yourself like Cloudflare.

So if I'm a mobile user and I want to access website xyz it will choose the quickest route to be able to get to the Cloudflare infrastructure.

So thereby actually making content available to you and me on the street.

And can you give us a sense for like how large the you run NAP Africa which is like now the largest interconnection point by a long distance in Africa and the three of them across South Africa in Durban, in Johannesburg, in Cape Town.

Can you give us a sense for just the like the number of countries and Internet users that are impacted by it?

The scale is staggering. So right now at NAP Africa we connect over 450 networks doing 1.5 terabits of peering capacity and that's servicing 16 different African countries.

So if you have a look at the map of Africa and you kind of go Angola all the way through a straight line and go all the way across all those countries below they access their Internet and their content from NAP Africa.

So you can imagine that's South Africa, Botswana, Swaziland, Malawi, Zimbabwe, Mozambique and so the islands Mauritius which we all want to go on holiday to.

So while everyone's relaxing on the beach and watching their Netflix or ordering things online that's all coming across NAP Africa.

That's amazing and really critical Internet infrastructure.

Can you talk a little bit about how the relationships came together over time?

So that's not an easy feat getting sort of so many networks to come together in this sort of coordinated manner.

One of the things I've always respected is that no matter how you're associated with the peering community you've been a great sort of partner and advisor to all the companies and networks.

Can you talk about how you built those relationships over time?

I think that the key is that Africa was always this untouched market and there was not a lot of knowledge and still isn't a lot of knowledge.

People generally look at Africa from what they see on the news which could be quite scary.

So it was generally around my team going out there and explaining Africa and that you know what we really actually do have stable infrastructure.

We really I mean you see all the stuff on the news with the power outages and all these things you know but there's ways to get around that and ways to implement great data centers that keep that up and running.

So I think it was around actually being an advocate for Africa and our team has spent probably the last eight years really hard going out there and being advocates for Africa so it's not such a bad place.

It's not all what you see on the news and it's okay to invest in Africa.

For so many California employees they have a hard time saying which ones are our favorite deployment but the one answer that comes up several times is Johannesburg because it was so exciting to see what was at the time the first of now many deployments in the region.

How about the relationships with the ISPs themselves?

So if you're a network in Malawi for example like why should you participate in an Internet exchange like this?

Previously before guys like Cloudflare invested in Africa the shortest route for you to be able to get content and think about it's only eight years ago was Europe.

So that adds you're sitting at like 200 milliseconds of latency. So Africans were very used to in those days that little buffering circle because we didn't have anything online.

I remember Facebook when it started it was kind of like you would upload your photos and your friends would see it the next day.

It's not this instantaneous.

We didn't have Netflix. We didn't have Google. We didn't have any of these applications that today are readily available.

So someone like let's take a Malawi for them what would make sense instead of them and think of it kind of like driving on a road you know you want to go to the shortest way to get to your destination.

So there's no way someone in Malawi is going to drive all the way to Europe and then come all the way back down to South Africa.

So it's easier for them to get in there four by four and go through a whole bunch of borders and end up in Johannesburg.

That's pretty much what we've seen around a lot of the African countries.

There's still a lot of work to be done and I think it's more knowledge and understanding within the African countries but we're getting there.

So what's your vision for what's next?

What do you have planned next for NAD Africa?

So what we've done as I mentioned now is there's a lot of skills that need to be developed.

You can imagine we've only if you look at Africa only been part of this world the Internet game for a very short time.

I mean eight years is not much.

So we spend a lot of our time on skills development. So making sure the engineers really understand what it means.

We look at the youth and see how we can incorporate the youth and bring them in to be the new advocates of the Internet and how to work on the Internet.

So we run a lot of events.

We do something called Terika Tech Days where we bring knowledgeable people like yourselves to come and have a talk to Africans because normally you know us being able to talk to someone like a cloud player is like dealing with a rock star.

So for the African guys it's generally around how do we actually upskill and then through upskilling means more and more connections that are actually happening to NAD Africa.

The other side and the other what's next is there's a huge amount of undersea cable capacity starting to come into Africa and it's been happening over years but there's large cables like Equano coming in.

The 2Africa cable is coming in and what that does is start driving down pricing and with pricing coming down someone like a cloud player can release more content and with more content it means the user on the street has got access to more information.

And these cables quite literally like land like by like at your facility like they're so what better place to interconnect than literally where the cables land.

That's perfect.

And it's the closest point to meet all the network operators. So the guys who service the mobile operators, the fixed line operators, they're all in our data center so it's easy you just run one cable and you're suddenly literally on the networks and accessing the users.

So switching around a little bit, would you have imagined sort of imagine yourself like doing this role of like how did you end up doing what you're doing right now?

How does somebody get to run a massive Internet exchange?

Well it was more by chance. No I didn't. I would never imagine that I would be doing this.

So probably a little bit of history was I ran my, before joining Terrica and NAP Africa, I ran my own ISP and then sold that off.

And at the time my previous boss before that he called me and said hey come do this NAP Africa thing.

So I said you know what is this NAP Africa thing? And he said oh no it's this Internet exchange, it's for free.

And at that time running an ISP was really draining.

And I thought fantastic you know it's a free product out into the market.

How easy can that be? And here we're sitting pretty much nearly nine years later and it hasn't been easy.

I can be honest about that. But it's been an amazing journey.

It's been and will continue to be an amazing journey when you start seeing people go online.

You know things like this pandemic that we're sitting in.

If we didn't have a NAP Africa and if we didn't have investments like someone like Cloudflare, you and I wouldn't have been able to have this call.

You know so actually seeing people get livelihoods because of the investment from multiple parties happening.

And when the pandemic happened I was like wow okay now you're really seeing this thing in action and it's been amazing.

How have the last six months been?

Okay it feels like everyone's lives have changed so much. What's it been like for the NAP Africa team?

Not much sleep but the growth has been amazing.

So to give you an idea it took us eight years to get the exchange to a terabit of capacity and in the last five months we've grown it by 500 gig.

So it just it shows you how people have moved online and and it's amazing from a South Africa perspective because I mean I'm going back we've now moved to a more a more open level.

So I'm going back to the data center just because I like going there.

It's no other reason I don't have to go there I can work from home. But the traffic there's nothing so everyone has realized that this online world what used to take me two hours to drive to the data center in traffic now takes me 30 minutes.

And everyone you can see everyone has realized even though the country has opened up you know people have realized let's carry on with this online world it's a better place to be.

Yeah it's taking some adjustment for all of us but I'm not sure we'll ever sort of go back to the world as it was just before this.

It feels like it's hard to predict what's going to happen but it feels like the world has changed pretty permanently because we're used to having conversations like this.

Absolutely I think it's actually I mean I'll use TerraCode TechDes as an example is we used to host it at the data center so it was very limited to only people could from Johannesburg unless they were going to pay for a ticket to fly in and see you know whoever presenting whatever skill.

And as COVID happened we kind of said as a team we don't want to stop TechDes because one of our goals is upskilling people.

So we said well let's move it online and let's give it a go. And I mean now we are training people all the way from Congo, Malawi etc.

So the opportunity has grown from let's say 40 people from Johannesburg you know attending and getting these skills to now multiple African countries dialing in and learning great things around how the Internet operates.

That's just terrific like that's just that's just that's such a terrific example of that's such a terrific story that you actually be able to reach more people.

That's just wonderful. For folks listening and if you have any questions feel free to send us an email at livestudio at livestudio.tv and we'll try to get to them through the show.

Mishal can you tell us a little bit about the other sort of Internet exchanges?

So for if you're trying to set up an ISP and considering joining an Internet exchange or you're setting up an Internet exchange and you aspire to be like NAP Africa what advice would you would you provide?

Because there are a number of other sort of nascent promising Internet exchanges.

So what can they learn from NAP Africa success?

I think the step one is have a good team. So I mean we've got a fantastic we don't have a big team you know there's literally like four of us but we're a very close team and each person brings their own elements of success for the exchange.

So I think first look internal and make sure you've got the right team driving the exchange for the right values because an Internet exchange is more around making the world a better place and being able to have Internet access for everyone.

So it's around that first before having to look at how much revenue you could possibly generate from it and in NAP Africa's case we do it for free so absolutely no charge and when we started it a lot of people thought we were absolutely out of our minds and today we've got you know one of the largest instant exchanges in the world and behind that is one of the largest data center companies.

So you know it's actually just a culminated into a massive amount of success.

So that's the first part and the second side is around having a look at your community.

So stay really close to your community.

It is a community and which is the best part about an Internet exchange is your members are your friends and keeping that in mind help them grow their businesses.

So don't just sit there and get them connected and say you know I mean NAP Africa we've got a saying where it's connect and never forget.

So we never forget our members.

We make sure that everyone is constantly going especially members who are sitting far away from South Africa you know let them know what's happening.

You guys see the news but what is really happening on the ground and constantly communicating that and be that trusted source to your members.

That's great, that's great. Tell us a little more about Terraco, the data center that hosts it and how Terraco has changed over the years.

Terraco, we were the first neutral co-location and still funny enough are the only neutral co -location operator in South Africa and so pretty much it's 11 years old now.

We started out of a storeroom so in Cape Town.

So we bought over a little storeroom and with the idea to make it easy for people to connect to each other.

So back in the day you know 11 years ago you could only buy really expensive fiber cables and you would run from service provider data center service provider which would cost hundreds and thousands of dollars to be able to do that.

So what we said is let's get the storeroom and we'll convert it into a data center and allow people to easily connect to each other and not have any politics around it either at a low cost.

So we launched that and we allowed people to connect to each other for less than 50 dollars.

So people went wow this is amazing. So we had a rush of ISPs coming in.

I know even my old ISP I raced in there because suddenly I could save thousands of dollars on operating costs and started to do this and then we launched another data center in Johannesburg and around the same time we launched NAP Africa and then we started seeing hang on but Johannesburg is a great place to be able to service multiple other African countries because of its proximity and then we started to kind of grow Johannesburg and then we launched Durban and Durban has been fantastic because I mean it's taken a while but we've now got the Metis cable which is the cable connecting up Mauritius, Reunion, Seychelles landing in the data center.

So that now starts opening up all those islands for for the holiday makers to get easy access to content.

So today we've got the three facilities and sitting at over 100 megawatts of power that we are providing out of those facilities and growing.

So we're building more facilities and more facilities and it's all been around making content and easily available to everyone and allowing people to connect.

That's great, that's great. You mentioned Mauritius.

I remember really being amazed. I think hopefully I'm not the only one but I think folks underestimate just how much traffic is in Mauritius because it's a lot higher than folks might realize.

I think what's happening with Mauritius is because it's from a tax perspective, it's very attractive and a lot of South Africans are actually moving to Mauritius and why not?

You live on a beautiful island, you're saving on your tax, you're a four hour flight away.

So we're getting a lot of South Africans actually moving to Mauritius.

And who knows, maybe what's happening this year might make people more likely to.

Absolutely, if you can stay online and live on an island, why not? That makes sense.

Would you mind telling me more about your team and how the team came together?

You said the team's really important and how did the team come together?

How do you divvy up who does what? Andrew Owens was there even before I was and he heads up all the technical.

So he is incredibly knowledgeable on a technical basis and just an amazing person who's willing to share his knowledge with a number of folk.

So Andrew and I were just the two of us who were actually funny enough for years.

So I was the one trying to hit it up and make sure Andrew got all the budget he needed and do all of those games.

And then Andrew was the one who physically built the exchange from a technical perspective.

And so it was the two of us for about four years together and then we got Mark Brunt to help on the engineering side and also fantastic.

And he came from an ISP background and running voice over IP networks.

So for him it was a total new world that he was entering and he's grown with it and very charismatic.

If any of you have met Mark Brunt, you'll definitely know he's quite a humorous individual.

And then two years ago in the name of growing the youth, we got Yolande who is, she's 22 and fantastic from a peering coordinator point of view.

So really eager to learn and grow and be part of the team.

So that's kind of the team at the moment and what we are doing, because it sounds really small running 450 or supporting 450 networks, we spend a lot of our time around automation.

So making sure we look at the day-to-day tasks and how do we automate that so that we can spend more time with our members.

That's great.

And I love how there's a sort of like a spirit of sharing amongst Internet exchanges.

So just like, I think it was yesterday or day before I saw the post from INEX, for example, in Dublin and how the peering manager tools they built are now used by, I think well north of a hundred exchanges around the world.

So it's great.

We can sort of like, you know, help each other that way. Well, I mean, I think that's the spirit of the Internet is, you know, it's an open network.

It's about sharing.

It's about learning from each other. It's about making things work better for the man on the street.

What has surprised you along the way? I think you're somewhat matter of fact about like, you know, these really, really impressive things that have come together.

What has been surprising? Can you just share any stories from along the way that were, that surprised you?

The recent one was the pandemic and how much traffic that was.

So that did surprise me, but luckily our European fellows phoned us and said, hey, just watch out.

There's this huge amount of traffic coming your way because South Africa only hit the pandemic about two or three months later after Europe.

And then I think around surprises has been how the world has come together in order to be able to support Africa.

So, you know, you wouldn't think when you look, as I've mentioned before, there's a lot of people watch Africa on the news and they go on a scary place, let's stay away.

And the amount of people have actually gotten together and said, hang on, we can actually invest in here.

We can make it successful, ignore the news, et cetera. And let's just see it grow and how they have grown.

So it's not been a kind of a, oh, let's guess and let's give it a go.

And then it didn't work. You know, everyone has grown and everyone has been really successful out of it over the last eight years.

Yeah. Some of the fastest growth we've seen, some of our fastest growth has really been in South Africa and across Africa.

So Louis, one of our, on our network engineering team actually wrote a really fun blog.

I think it may have been about a year ago, really like detailing all the growth we're seeing and IPv6 adoption.

And it's been very, very, very exciting. What's been especially hard and what is a particularly hard challenge along the way?

I think changing people's mindsets.

So, and that's a continuous, I mean, even now as we enter into the enterprise market, I mean, I had a call today and this enterprise guy said, why must that change?

I don't see. And I'm like, but it's for free, you know, and it will improve your network and you'll be closer to the content.

And so I think it's a mindset thing, you know, where people kind of get, they get stuck in their ways and they kind of say, this is the way it should be.

And it's because it's always worked that way.

And when you go along and you say, but, hey guys, you know, here's a better way.

Here's a more exciting way. Here's some way to grow your teams, to do all these things.

I think that's going to be an ongoing challenge that we need to face.

So traditionally, when you think of intranet exchanges, you think of sort of, you know, the sort of the distributed networks that serve applications at the edge.

You think of hosting companies and ISPs, but you mentioned enterprises.

It increasingly makes sense, even if you're a local enterprise, it makes sense for you to connect to an intranet exchange.

Why is that?

And what has changed? Like, does it make more sense now than it did a few years ago?

Why is that? So I think, I mean, in Africa, our bandwidth and transfer fees are still pretty high.

So especially for the enterprises. So for them to be able to save on that has been key.

And then the other side of it from an enterprise perspective is being able to control how they read things as the cloud becomes a reality, especially in Africa, which it's now only recently become a reality with Microsoft and Amazon and these guys starting to really invest in Africa with big cloud regions.

They're starting to say, okay, instead of me looking after the server infrastructure, you know, let me outsource that.

And how do I start outsourcing those things?

And if I'm outsourcing my website, for example, to Cloudflare, and all of these things start coming together.

And then the IT guys kind of sitting there and saying, but how do I control where this traffic must move to what?

And that's where the big question mark is starting to come in. And that's where the peering opportunity is coming in.

And they're kind of saying, actually, I want to be able to instead of somebody owning my IP addresses, somebody doing this for me.

How do I know if I can trust them? Let me do this myself. The other side is they're starting to peer off their own content.

So they're starting to see themselves as content owners, like banks and insurance companies.

They're starting to say, how do I get my content closer to the user?

And so they're kind of morphing in terms of that.

Well, I think the NAP Africa team really has a lot to be very proud of.

And I think as a community, we're very grateful, not only for the work that your team does for NAP Africa, but also how it's been a model for everybody else who wants to either set up an Internet exchange or learn about peering.

So I know you have many demands on your time.

So thank you for joining us and sharing your story.

We really appreciate it. You said you might get a little bit of time off.

Yes.

So I'm planning to do a trip to the safari in the beginning of October. And it's been fantastic because I got an email today actually saying you're getting a free upgrade.

It's for my husband's birthday. So I was like, wow, yay. We're super excited about that to go on a little bit of a safari tour and take a break.

Wonderful. Well, thanks so much for joining us, Michelle. Really appreciate it.

It's been great chatting with you. Thank you. Thanks. Thank you, Nitin, for your time.