Project Galileo Spotlight: ZOE

Hi everyone, my name is Jocelyn. I work on the public policy team at Cloudflare and we're celebrating Project Galileo's 6th anniversary. So I'm excited to have Julien from Zoe, who is under Project Galileo, speak with us about the organization and the COVID app, tracking app that you guys created. So Julien, thanks for taking the time to chat today. So can you give us an overview of what Zoe does and really the mission of the organization? Sure. Hi everyone. It's great to be here. So Zoe is a startup based in London. We also have an office in Boston and we are a nutritional science company. So for the last two years, we've been acquiring quite a lot of data on how people respond to food. And our core belief is that actually the food you eat is the best medicine you can take. But we also believe that everybody actually do respond to food a bit differently, which is something on which actually there's not a lot of data. Some nutritional study tend to be small scale. So we've run two studies called Predict 1 and 2 with a bit more than a thousand people each, which is actually almost word of magnitude bigger than what tends to happen in this field. And we were in the process of building or launching a product when we got a derail like everybody else with COVID-19. And interestingly, we've diverted quite a lot of resources towards building what's now called the COVID Symptom Study App, which is an app in the UK, in the US and in Sweden that you can use to report your symptoms, whether you believe that you have COVID-19 or not. So we can ask you questions such as, do you have any kind of medical antecedents? And then we're going to ask you to report daily whether you think you've got cough, fever, loss of smell, plenty of other symptoms. There's actually a lot more symptoms related to COVID-19 that people realize. But that flows back to scientists. And in the UK, it also flows back to authorities, hospital, and basically helps both the research side of things, but also coordinating the response to the pandemic. So that's been something we've been working for the last three months, totally pro bono. And we've basically been driven to do that because for the nutritional study that we're running, we're already working with quite a few universities. So for example, King's College in London, but also Massachusetts General Hospital in the US and a few others. And basically, those people reach out to us and they say, hey, can you do something for us? Sure. What do you need? You know, we're closing down because of COVID-19. So it would be awesome to be able to do some research. But to do some research, we need some data. Can you build a mobile app for us? And we say, yeah, why not? When do you need it? Could you do tomorrow evening? Let's give it a try. So that's actually started this project started as a hackathon. And it's been extremely exciting and interesting because about six days later, we were not only launched, but we had actually 1 million people in the UK and we're about 3.5 million people using the app those days. Wow. So we've actually seen a lot of organizations such as yours on Galileo that really have like spearheaded these types of initiatives to provide some type relief during COVID-19. So can you kind of explain the steps of the COVID symptom tractor and how it kind of works in practice? Yeah, sure. So it's pretty simple. And in some ways, it's also pretty directional. So, you know, basically you would install the app and the app, you know, the first step is registration. And as I mentioned before, you would have to fill your medical antecedents or, you know, it would ask you, you know, what is your gender? You know, what is your age? Do you have, you know, are you a part of ethnic minorities? Do you have, you know, diabetes? Do you have any kind of cardiovascular disease? Plenty of other questions like that. And then, you know, you move towards, you know, did you have any COVID-19 tests already? Do you have any symptoms, which is, you know, something we would ask you daily. And basically, you kind of feel that that on every day you get a reminder to feel your symptoms. And at the end, you just have basically some kind of new thank you screen that say, you know, we're publishing some data back to the users, but mostly in some kind of aggregated way. So if you go on covid .joinzoe.com, you can see all the numbers we're publishing, whether this is for the US or the UK, Sweden is a different website. So you could see maps of where COVID-19 is currently in the kind of infection levels and their provision in a weekly basis. But again, more interestingly, the real reason why we did that was not to push back data to people, was actually just to build a huge data set about COVID. So on the back of that, there's a lot of research that has been done by various teams around the world. And for example, one of the things we've discovered quite early by using the COVID Symptom Study app, is that one of the best predictor on whether you have COVID-19 is not cough or fever, like most people and most government thought at the beginning, it's actually anosmia, which is the loss of taste and smell. So basically, by installing the app and contributing to what we call citizen science, and you're helping researchers around the world, just doing very interesting discoveries and understanding better the disease basically. So it's more, as I said, it's more like something you do for others and something that has like a direct benefit for you as a person. Yeah, definitely. So how did you originally get started at ZOE? Because you're an engineer. So was it just a natural progression to work for a nutrition type of company? So yeah, it's an interesting question. Because for me, so I joined ZOE about six months ago. So a lot later after the company was funded. I'm very far from being the first engineer here. But basically, I already knew one of the founders. So obviously, didn't know that we've been working together before. So I wasn't in touch with them. And for me, I was just at a point in my career before that, where I felt like I was just starting to get a bit bored. I wasn't sure what was next in my previous place. And my previous place to be clear, I had like great experience, love working with people, but this was not necessarily a company that was doing anything particularly interesting or good for the world. So I was interested in finding a place where I could, you know, say to my kids, you know, I've done that thing. It's not just, you know, to make someone richer or to, you know, provide a service that is just useful to some very, very specific people. But I was really keen to be able to tell them, you know, this is how I contributed to, you know, in a very small fashion, make the world a better place. And I think ZOE is we're not there yet, obviously. But that's one of the core beliefs that what we can do around nutrition could help a lot of people because nutrition is a field where there's not a lot of clarity, you can read everything. And then, you know, the next day, the exact contrary. So we're trying to bring about a bit more science in that. And that's really what got me extremely interested in ZOE and leading the engineering team there. And you say that ZOE is a small company, but with the Symptom Studies app, you had more than a million users in your first couple days. So I don't, I don't know if that's necessarily small, but that's, that's really awesome. So the company started mainly with nutrition, and they decided to make, create this tracker. Can you kind of explain the connection between the nutrition part and the COVID symptoms part? Sure. So there are really two parts to this discussion, which is initially when we started working on the COVID Symptom Study app, you know, there was no plan associated with it. It was, you know, we were in London, I think it was week two of lockdown. So everybody was really down in terms of morale. And, you know, we were talking internally, some of us at least on whether there could be something we could do to help on COVID-19, because it was like various initiatives in various countries around that. And at the same time, so one of our other founder, Tim Spector, which is also actually working at KCL, kind of started to make the link and say, well, actually, there is that thing we can do for the research team at KCL. So it really started without, you know, any second thought about whether it was good or not for the company and about whether there was any link with nutrition. It was just, you know, trying to help, but at that time, it's very easy now to say, oh, that's a great thing. But at that time, it's just like, we're going to spend two days, you know, trying building that thing, and then most likely nothing will happen, right? Now, it's a big thing, but in many ways, totally by accident. But there is actually an interesting link with nutrition, which is, you know, since COVID -19 started, there's been quite a lot of studies showing that, for example, people that are diabetic or people with cardiovascular disease, are a lot more at risk, or people maybe with some deficiencies in some specific vitamins, the research is really in flux, but there is definitely like a stream of, you know, if you have these specific conditions, you are at risk. And one of the things that Zoe believes in is that we can help people with diabetes or cardiovascular disease by having better long term health and by changing their diet. So there could actually be a link in the sense, in some ways, you know, maybe one of the best ways to help people is just to tell them to eat better food, better diets. You know, obviously, we're all hoping that there will be a vaccine and treatments. But, you know, this is a vaccine or treatment that will happen for COVID-19, then there will be, you know, another disease sometime in the future, hopefully not too soon. And we generally believe that, you know, food is a very good way to, you know, protect yourself against those, or at least not into specifically one virus, but at least just have a better long term health and be a bit less at risk. Yeah, I definitely think the part of just overall health, it seems pretty simple how Zoe would move to be able to create this symptom tracker. And you mentioned that you work with a lot of different partners, so the King's College of London, and then the Massachusetts General Hospital. So whenever individuals fill out this specific app and the symptom tracker, how is that information distributed to these different partners and used? Yeah, so indeed, as you say, the main goal is to give those various entities access to the data. But, you know, obviously, the very important thing is that they don't have access to the road that everything is anonymized. So if I feel my symptoms, they will never know that this is Julian symptoms, they will just see, you know, an ID in some regions of the UK. And, you know, they will obviously have, you know, some of my properties. But for example, with age, in some cases, at least with some partners is anonymized, like we don't even give the age, we would put people in the bucket. Or, you know, in the UK, postcodes are extremely specific, they can actually identify the building. Same thing, we don't give people like the exact postcode, we just give them like a general location. So the data basically is kind of uploaded to those various entities daily. And there are various mechanism depending with where we work, basically, we've got kind of three mechanism. We send the data directly to Massachusetts General Hospital for the team of researchers there. We do the same thing in Sweden with Lund University. And in the UK, the UK data is actually spread a lot more widely because we do send it to researcher, but it's actually also sent to the government and specific reports are sent daily to the government about the state of things. And the NHS, which is basically our health system. So all the kind of local authorities actually have access to the data if they want to. And that's a way they can, they can coordinate the response. And, you know, we've got less cases now in the UK. So that's a bit less of a worry. But what we do know is that our numbers kind of lead the hospital admission by I think a week or two weeks. So it actually gives, you know, if people want to look at them, they actually have a better chance to prepare and to know which specific, you know, area is going to be at risk to be kind of overloaded. Yeah. And you, and you mentioned that you've launched, you've launched the app, you first launched the app in the UK, and then you also launched it in the US and Sweden. Can you kind of give some of the big differences that you've noticed in launching in these places? Because I know a lot of different countries have different views on the virus. So I'm curious what some of the biggest differences you found when launching it? Yeah, so there's indeed quite a few differences. I mean, the obvious one from a perspective is that the kind of, you know, we had more reach in the UK and Sweden versus the total side of the population than in the US. It's not entirely clear why, you know, we have some hypothesis, but I think it's it's not unfair to say that there might be bigger privacy concern in the US than in the UK. I think people in the UK and Sweden also have a lot of privacy concern. But some of the feedback we had in the US was actually people don't want their data to be shared at all with the government, which if you live in the UK was something that you're not necessarily used to. I think people are a bit more afraid that you would share with the government or with insurance that I would have like a detrimental impact on their personal life, which by the way, we don't do like, you know, as I said, we do shut that up, but only in aggregated way, we can't go back to the specific individual. And so that's one of the key thing that we had. And the second thing is that to launch an app like that, you need to have like an ethic approval, you can't just create an app like that on Google Play or App Store and get it accepted. You actually need to get a lot of work. And that's why we're able to launch because actually, you need basically those partners and universities to, you know, put their stamp and say, yes, this is legitimate research. This is not just a massive data collection for just, you know, for doing whatever. It's actually going to be some research has been approved by kind of the local authorities. And, you know, different ethics approval would need, you know, different things. So, you know, depending on the laws of the country, you know, you have to make some slight change, you say no in that country, for example, we want to ask for in the UK, we're asking for the postcode and quite detailed level, because we can, if we want to, we could link with the NHS record. But for example, you may not want to do that at the same level in Sweden, because they may have less appetite for you being able to really, pinpoint where people are living. So everything, there's a lot of slight difference like that, like, for example, the way you ask ethnicity is very different from one country to another, just because different habits. And then obviously, you know, when we launched in Sweden, that was like a totally new translation, which was quite interesting, quite challenging, because in the team, we have no one speaking Swedish. So you've got a team of engineers that, you know, try the app in Swedish, but they have no idea if it's right. It's just, you know, it's gibberish for them. So that was also an interesting challenge. Yeah. And you mentioned the part with a lot of your partners, it's like the building of the trust part. So having these like NHS, for example, or like King's College London, back you up, I think is also a really important aspect of, of the app, because people tend to think, um, maybe initiatives like this, they don't have these large partners that are backing them, it's hard to create this trust. So I think that's definitely one of the most interesting part of the app. So, um, have you noticed any significant differences in the data when you've in the United States or in Sweden, and how you generally talk about the virus? Um, there are some differences, but being just a simple engineer, I'm not the right person to, to really comment. I mean, Sweden is very interesting, because they did not impose any lockdown at all. They basically just relied on population to, you know, distance themselves. Whereas UK had a very stricter lockdown mechanism. On the US, sadly, you know, it's also it's, again, a different story about lockdown, and especially about kind of, you know, releasing it a lot faster than maybe the UK. But sadly, we also have less data. So it may be a bit less interesting to, to compare. But there are definitely some, some, you know, some differences between the country, like we see in Sweden, that infection rates are, are generally higher than in the UK, just because there was, there was no lockdown. But the thing I would say to people basically, all the research we're doing is available on covid .joinsoe.com. And there's quite a lot of various research paper have been published in the last few weeks, in various scientific journals. So it's, it's, it's all available. And they can find out about the details if they want to. Of course, and you say, previously, you said that a lot of these people would use every day, they would go back and kind of track their symptoms. So are you seeing certain spikes of traffic, maybe like in the mornings with people putting in their symptoms? So it's all the time having these influxes of traffic? Or are you saying, primarily just a steady traffic trends? So there was basically, there were basically three different scenarios. We do some notifications daily. So whenever we send notification, we see a traffic spike. And that's, that's immediate. And we actually have to be a bit careful, like we ramp up the notification, we don't send a notification to 1 million people at once. So we actually spread it over a few hours. And when we start, we start kind of slowly slowly and kind of accelerate just so that it gives more time for the infrastructure to scale to, to the right size, because we're using auto scaling mechanism. So that's obviously one of the main way we some, some, some additional traffic. The two other ones are interesting. So the first, you know, the biggest traffic spike that we had actually was probably on day two in the UK, basically, day one was actually quite good. We launched at 9am in the morning. And basically, we had Tim Spector, which is kind of one of the founder working at KCL started tweeting about it. And, you know, there was almost instantly some traffic, some people picking up and retweeting and etc. But I think possibly what happened is that the press started learning about it during the day. But actually prepared kind of, you know, articles and news, mostly the next day, and the next day, actually, at 755, the whole system crash on the load, there was just so many people trying to install the app. And so it was like, absolutely amazing. So at the end of the first day, we had 200,000 people at 755. The next day, we were already having more traffic than any point at the day before. And by the end of the day, we had 1 million people subscribe. So we handle that spike, you know, not too well, but we cover it actually in like 30 minutes, and the system was online. And one of the thing we kind of always are careful about or kind of struggle with is that there's been quite a few times where Tim or other people were on TV, or for example, there was one time where a journalist asked the government about our app during live questions. Whenever that happens, you basically have a 20x traffic spike. And that literally goes from, you know, some flatline to the next second 20x. Like, it's not like scaling over the next, you know, 10 minutes or something like that. It's just totally instantaneous. Like people are on TV, they see it, it's like, oh, wonderful. Let's, you know, let's check our symptom, let's download the app, etc. And that's actually quite challenging, because, you know, auto scaling is great. But at least, you know, the auto scaling that we have just doesn't adapt absolutely instantly to traffic spike, it needs actually a couple of minutes to, to find the right response. So when that happened, we tend to have a bit of a latency, latency goes from 150 millisecond to interact with any screen on the app to, you know, basically, maybe a couple seconds or more. And that lasts for two minutes, and then everything is back to normal. Yeah, exactly. So, so were you more worried whenever you launched the app about types of cyber attacks? Or was it just trying to figure out how to handle these huge influxes of traffic? So I think when we launched the app, there was one thing on which we're really, really clear, which is even if the app was small scale, we're still worried about security. And that's something on which, you know, we never, we never took any shortcut. So, you know, security, you're never 100% safe. You know, it's like, you know, you could be in a perfect condition. Like, you know, you have the perfect health, you could still cross the roads and, you know, be hit by a bus. So, you know, or you could have still, you know, cardiac arrest or whatever, like it could happen. I think, you know, security, people have to understand that to a large degree, you know, you can do everything right, that doesn't mean there is zero risk, that just means it's a lot harder than, you know, almost everybody else to, you know, to be attacked. So we didn't take any shortcuts. But we're more worried about that, for sure, than kind of scalability and traffic spike, because at the beginning, we had no idea if anyone would install our app. So we just, you know, we didn't want to have any data breach. But scalability was, you know, not something we thought we would necessarily have any trouble with. Well, it turns out that we did have some trouble with that, but kind of, you know, reasonably quickly and stabilized and security, I think we've, for the next kind of months after the launch, we basically spend a lot of time still kind of, you know, improving it. And that's, you know, that's why we also contacted Cloudflare in the first place to try to get, you know, some additional support, some, you know, protection against various possible attack vectors, or just, you know, denial of service, and, you know, various things like that. So yeah, definitely was something we're a bit worried about from the very beginning. So what do you think are the biggest, looking back when you launched the app, and to where you are now, what do you think the biggest difference is in terms of what you've seen on the app in terms of traffic, or any attacks that you've had, or just where you've progressed from there to now? So, I mean, the traffic patterns have been reasonably stable in the sense we had like huge spike of traffic the first few days, a lot of people, then there was some kind of, um, reasonably linear growth. And now we had a point where it's kind of more more stable, not necessarily a lot of user growth, which makes sense, because our biggest market, market's probably not the right word, but our biggest presence is in the UK. And with the lockdown, people are a bit more relaxed. So it will be interesting to see actually what happens if numbers of case actually start increasing again, because it may actually lead to this kind of not only a second wave of cases of COVID, I hope not, but also a second wave of people, you know, starting using the app, because they realize that, hey, actually, it's not, you know, it's not a done deal. So I think traffic patterns has not been too much of a worry. I think in terms of security, as I said, we've improved a lot. We were also very lucky, because we had a lot of partners from the very beginning, so Cloudflare on one side, also, you know, some reach out to say, hey, we'd like to do some penetration testing, probably not for you, which we did, we're very happy that they didn't find any, you know, major issues. There was a couple of small things such as, you know, upgrading SSL that they've suggested that we did, which was actually just clicking a button in Cloudflare, if I'm honest. So we did the things we've improved. We've added, you know, additional layers of security in various places. And that was definitely, you know, a good thing. We spend more time automating our infrastructure, because it's been built extremely rapidly in the span of a few days. And that was definitely one point where we were a bit weak, kind of being able to reproduce the infrastructure, even if it's on the cloud, having everything automated. And I guess the biggest change has really been kind of the evolution of the app. Because the way we've built the app, since there was this kind of massive time pressure to release something, and we just had no idea if it would work, you know, the actual app on your phone, we took every single shortcut we could, like literally, when we launched the app, for example, you had a registration, but you could not reset your password, which turns out is actually something that people really, really, really do want. So we had a lot of people and a lot of bad reviews, because people are like, what do you mean I can't reset our password? You know, totally fair question. Yeah, we do want them to reset their password. It's just, you know, when you build an app over four days, that's not necessarily the first feature you need, because you know, you can totally live without reset password, but you cannot live without, you know, the core feature of the app. And I think some of the things that took us a lot of bandwidth in the next few weeks is just, you know, improving the app in general, the UX, but most importantly, adding a lot of questions and features. So, you know, initially, we had a reasonably small set of questions and given to us by the scientists. When they realized, you know, after two days of the app running that had so many users, it was like, oh, this is wonderful. Now we've got so many questions we want to ask people. And what has been very interesting, it's, you know, the question I wanted to ask was also quite linked to the various research going, you know, around the world, but in many case, at very small scale, I think, you know, I was in touch with some researcher recently in France that were, you know, what they are trying to do is very interesting. They realized that they've got great data about hospital admission, but they've got absolutely no data whatsoever in France, apparently about what happens outside of hospital. And it turns out there is actually, you know, a lot that happened outside of hospital because the, you know, the hospital are basically the worst case. Those are the people that are very, very, very sick. So, you know, you actually have only a very, very small subset. And I think there is a lot of research has been done, you know, kind of, you know, small scale data sets. And I think a lot of what our scientists or scientists we're working with have been trying to get is, you know, how we realize that, you know, those category of people are at risk. Okay, let's ask our people if, you Because for quite a few of those, we already know if they had COVID or not, they've been tested, we've got the data recorded. Based on that, and their symptoms, we actually have some reasonably good prediction models on whether we do think they've got COVID. Now we can add that additional dimension, which is, you know, do they fit into category X, Y, Z, and then almost suddenly that gives you, you know, data set with, you know, several million people that, you know, tell you if that factor is actually a risk factor or not. So they are super excited because this is, they've never done research at that speed, basically, and with that amount of data. So a lot of what we've done in the last couple of months, basically just adding questions after questions and just try to expand the data set in various dimensions. Yeah, of course. And I want to make the most of your time. So really, my last question is, what do you want more people to know about the symptom tracker? Well, I would say, you know, just, you know, use it, not for your own benefit, but for helping the science. So, you know, go on COVID.joinzui.com, download the app. You know, we're not doing anything bad with your data. You know, you can look at the various research article. And yeah, if you think, you know, that's great, just, you know, advertise it on one side. If you are an engineer working on mobile engineering, the code is actually open source. So, you know, you know, go on GitHub, give us a hand and, you know, help us make the app better. Yeah, of course. Well, thanks so much, Julien. I definitely am going to download the app and take and track my symptoms. I'm definitely going to do that. I hope everybody at Cloudflare and everybody who's watching will do it. So thanks so much for your time. Thanks to you and have a great day. You too. Bye -bye.