Project Galileo Spotlight: ZOE

Hi everyone, my name is Jocelyn. I work on the public policy team at Cloudflare and we're celebrating Project Galileo's 6th anniversary. So I'm excited to have Julien from Zoe, who is under Project Galileo, speak with us about the organization and the COVID app, tracking app that you guys created. So Julien, thanks for taking the time to chat today. So can you give us an overview of what Zoe does and really the mission of the organization? Sure. Hi everyone. It's great to be here. So Zoe is a startup based in London. We also have an office in Boston and we are a nutritional science company. So for the last two years, we've been acquiring quite a lot of data on how people respond to food. And our core belief is that actually the food you eat is the best medicine you can take. But we also believe that everybody actually do respond to food a bit differently, which is something on which actually there's not a lot of data. Some nutritional study tend to be small scale. So we've run two studies called Predict 1 and 2 with a bit more than a thousand people each, which is actually almost word of magnitude bigger than what tends to happen in this field. And we were in the process of building or launching a product when we got a derail like everybody else with COVID-19. And interestingly, we've diverted quite a lot of resources towards building what's now called the COVID Symptom Study App, which is an app in the UK, in the US and in Sweden that you can use to report your symptoms, whether you believe that you have COVID-19 or not. So we can ask you questions such as, do you have any kind of medical antecedents? And then we're going to ask you to report daily whether you think you've got cough, fever, loss of smell, plenty of other symptoms. There's actually a lot more symptoms related to COVID-19 that people realize. But that flows back to scientists. And in the UK, it also flows back to authorities, hospital, and basically helps both the research side of things, but also coordinating the response to the pandemic. So that's been something we've been working for the last three months, totally pro bono. And we've basically been driven to do that because for the nutritional study that we're running, we're already working with quite a few universities. So for example, King's College in London, but also Massachusetts General Hospital in the US and a few others. And basically, those people reach out to us and they say, hey, can you do something for us? Sure. What do you need? You know, we're closing down because of COVID-19. So it would be awesome to be able to do some research. But to do some research, we need some data. Can you build a mobile app for us? And we say, yeah, why not? When do you need it? Could you do tomorrow evening? Let's give it a try. So that's actually started this project started as a hackathon. And it's been extremely exciting and interesting because about six days later, we were not only launched, but we had actually 1 million people in the UK and we're about 3.5 million people using the app those days. Wow. So we've actually seen a lot of organizations such as yours on Galileo that really have like spearheaded these types of initiatives to provide some type relief during COVID-19. So can you kind of explain the steps of the COVID symptom tractor and how it kind of works in practice? Yeah, sure. So it's pretty simple. And in some ways, it's also pretty directional. So, you know, basically you would install the app and the app, you know, the first step is registration. And as I mentioned before, you would have to fill your medical antecedents. So, you know, it would ask you, you know, what is your gender? You know, what is your age? Do you have, you know, are you a part of ethnic minorities? Do you have, you know, diabetes? Do you have any kind of cardiovascular disease? Plenty of other questions like that. And then, you know, you move towards, you know, did you have any COVID-19 tests already? Do you have any symptoms, which is, you know, something we would ask you daily. And basically, you kind of feel that that on every day you get a reminder to feel your symptoms. And at the end, you just have basically some kind of new thank you screen that say, you know, we're publishing some data back to the users, but mostly in some kind of aggregated way. So if you go on COVID .joinzoe.com, you can see all the numbers we're publishing, um, whether this is for the US or the UK, Sweden is a different website. So you could see maps of where COVID-19 is currently in the kind of infection levels, um, and their provision and weekly basis. Um, but again, more, more interestingly, the real reason why we did that was not to, um, push back that out to, uh, to people was actually just to build a huge data set about COVID. So on, on the back of that, there's a lot of research has been done by, by various teams around the world. Um, and for example, one of the things we've discovered quite early by using, um, the COVID symptom study app is that one of the best predictor on whether you have COVID -19 is not cough or fever, uh, like most people and most government thought at the beginning, uh, it's actually an osmia, which is the loss of, uh, taste and smell. So basically, you know, by, by installing the app and contributing, you are contributing to what we call citizen science. Um, and you're, you're helping, you know, researchers around the world, um, just, just doing very interesting discoveries and, and, you know, understanding better the disease basically. Uh, so it's more, you know, uh, as I said, it's more like, you know, something you do for others and something that has like a direct benefit for you, um, as a person. Yeah, definitely. How did, so how did you originally get started at Zoe? Cause you, uh, you're a, uh, an engineer. So was it just a natural progression for, to work for a nutrition type of company? So, yeah, it's an interesting question because for me, so I joined Zoe about six months ago. So, um, a lot later after, um, the company was, uh, was, was funded. I'm very far from being the first engineer here. Um, but, um, basically I already knew, um, one of the founders. So obviously didn't know that, that helps we've been working together before. So I wasn't in touch with them. And, and for me, I was just at a point in my career before that, where I felt like I was just starting to get, um, a bit bored. I wasn't sure what's was next in my previous place. Um, and my previous place to be clear, I mean, it was, I had like great experience and love working with people, but this was not necessarily, you know, a company that was doing, um, you know, anything particularly interesting or good for the world. So, um, I was interested in, in finding a place where I could, uh, you know, say to my kids, you know, I've, I've done that thing. It's, it's not just, you know, to make someone richer or to, you know, provide a service that is just useful to some very, very specific people. Um, but I was really keen to be able to tell them, you know, um, this is how I contributed to, you know, in a very small fashion, make the world a better place. And I think Zoe has, um, um, you know, we're not there yet, uh, obviously, uh, but that's, um, one of the core beliefs that what we can do around nutrition could help a lot of people because nutrition is, um, a field where there is not a lot of clarity. You can read everything. And then, you know, on the next day, the exact contrary. So we're trying to bring about a bit more science, um, in that. And that's, that's really what, um, got me extremely interested in, in, in Zoe, um, and leading the engineering team there. And you say, you say that Zoe is a small company, but with the symptom studies app, you had more than a million users in your first couple of days. So I don't, I don't know if that's small, but that's, that's really awesome. So the company started, uh, mainly with nutrition and they decided to make, uh, create this tracker. Can you kind of explain the connection between the nutrition part and the COVID symptoms part? Sure. So there are really two parts of this discussion, which is initially when we started working on the COVID symptom study app, um, you know, there was no plan, uh, seated with it. It was, you know, we were in London. I think it was week two of lockdown. So everybody was really down in terms of morale. Um, and you know, we were talking internally, uh, some of us, at least on the, whether there could be something we could do to help on COVID-19 because it was like various initiative in various countries around that. Um, and at the same time, so one of our other funder, Tim Spector, which is also actually working at KCL, um, kind of started to make the link and say, well, actually, you know, there is that thing we can do for the research team at KCL. So it really started without, um, you know, any second thought about whether it was, uh, good or not for the, for the company and about whether there was any link with, uh, with nutrition. Um, it was just, you know, trying to, to help, but, you know, at that time it's, you know, it's very easy, um, now to say, oh, that's a great thing. But at that time it's just like, we're going to spend two days, you know, and then most likely nothing will happen. Right. Um, now it's a big thing, but you know, in many ways totally by accident. Um, but there is actually an interesting link with nutrition, which is, you know, since, since COVID-19 started, um, there's been quite a lot of studies showing that, uh, for example, people that are diabetic or people with cardiovascular disease, uh, or a lot more at risk, uh, or people maybe with some deficiencies in some specific vitamins, the research is really in flux, but there is definitely like a stream of, um, you know, if you have these specific, um, conditions, you, you are, you're at risk. Um, and one of the things that Zoe, um, you know, believes in is that we can help people with diabetes or cardiovascular disease by having better longterm health and by changing their diet. Um, so that could actually be a link in the sense in some ways, um, you know, maybe one of the best way to help people is just to tell them to hit better food, uh, better diets. Um, you know, obviously we're all hoping that there will be a vaccine and, and, and treatments. Um, but you know, this is a vaccine or a treatment that will happen for COVID-19. Then there will be, you know, another disease sometime in the future, hopefully not too soon. And, and we generally believe that, um, you know, food is a, is a very good way to protect yourself against, against those, or at least not specifically one virus, but at least just have a better longterm health and, and be a bit less at risk. Yeah, I definitely think the part of just overall health, it seems, it seems pretty simple how Zoe would move to be able to create the symptom tracker. And you mentioned that, uh, you work with a lot of different partners. So the King's College of London and then the Massachusetts General Hospital. So whenever individuals fill out this, uh, the specific app and the symptom tracker, how is that information, um, distributed to these different partners and used? Yeah. So, um, indeed, as you say, that the main goal is to give those various entities access to the data. But, you know, obviously the very important thing is that they don't have access to the road that everything is anonymized. Um, so if, you know, if I feel by symptoms, they will never know that this is, uh, you know, Julian symptoms, they will just see, you know, an ID in some regions of, uh, of the UK and, you know, they will obviously have seen some of my, um, properties. Uh, but for example, with an age, in some cases, uh, at least with some partners is anonymized. Like we don't even give the edge. We would put people in the bucket, um, or, you know, in the UK postcodes are extremely specific. They can actually identify the building. Uh, same thing. We don't give people like the exact postcode. We just give them like a general location. So the data basically is kind of applauded to those, um, various entities, um, daily. Um, and there are various mechanism depending with, uh, where we work. Basically we've got kind of three mechanism. Um, we send the data directly to Massachusetts general hospital, um, for the team of researchers there. Um, we do the same thing in Sweden with London university. Um, in the UK, the UK data is actually spread a lot more widely because we do send it to researcher, but it's actually also, um, send to the government, um, and specific reports or send daily to the government about the state of things. Um, and the NHS, which is basically our health system. Um, so all the kind of local authorities actually have access to the data if they want to, and that's the way they can, um, they can coordinate the response. Um, and you know, we've got less cases now in the UK, so that's a bit less of a worry, but what we do know is that our, our numbers kind of lead, uh, the hospital admission by I think a week or two weeks. Um, so it actually gives, um, you know, if people want to look at them, they actually have a better chance to prepare and to know which specific, you know, um, area is going to be at risk, uh, to be kind of overloaded. Yeah. And you, and you mentioned that you've launched, you've launched the app, you first launched the app in the UK, and then you also launched it in the U S and Sweden. Can you kind of give some of the big differences that you've noticed in launching in these places? Because I know a lot of, uh, different countries have different views on the virus. So I'm curious what some of the biggest differences you, you found when launching it. Yeah. So there's indeed quite a few differences. I mean, the, the, the obvious one from a perspective is that, um, the kind of, you know, we had more reach in the UK and Sweden, um, versus the total side of the population than in the U S. Um, it's not entirely clear why we have some hypothesis, but, um, I think it's, it's not unfair to say that there might be bigger privacy concern in the U S than in the UK. I think people in the UK and Sweden also have a lot of privacy concern, but some of the feedback we had in the U S was actually people don't want their data to be shared at all with the government, which if you live in the UK was something that you're not necessarily used to. I think people are a bit more afraid that you would share with the government or with insurance that they would have like a detrimental impact on their personal life, which by the way, we don't do like, you know, as I said, we do shut that up, but only in aggregated way, we can't go back to the specific individual. Um, so that's one of the key thing that we had. Um, the second thing is that to launch an app like that, you need to have like an ethic approval. You can't just, um, create an app like that on Google play or app store and get it accepted. Uh, you actually need to get a lot of paperwork. Um, and that's why we're able to launch because actually you need basically those partners and universities to, you know, put their stamp and say, yes, this is legitimate research. This is not just a massive data collection for just, you know, for doing whatever, uh, it's actually going to be some, some research has been approved, um, by kind of the, the local authorities. Um, and you know, different ethics approval would need, you know, different things. So, you know, depending on the laws of the country, uh, you know, you have to make some slight change. You say, you know, in that country, for example, we want to ask for in the UK, we're asking for the postcode, um, and quite detailed level because we can, if we want to, we could link with the NHS, uh, record. But, uh, for example, you may not want to do that, um, at the same level in, in Sweden, um, because they may have less appetite for, um, for you being able to really, really pinpoint, um, you know, where people are living. So everything, there's a lot of slight difference like that. Like for example, the way you ask ethnic, uh, ethnicity is very different from one country to another, um, just because just different habits. Um, um, and then obviously, you know, when we launched in Sweden, that was like a totally new translation, which was quite interesting and quite challenging because in the team, we have no one, uh, speaking Swedish. So you've got, you know, a team of engineers, uh, that, you know, try the app in Swedish, but they have no idea if it's right. It's just, you know, it's gibberish for them. Um, so that was also an interesting challenge. Yeah. And you mentioned, uh, the part with a lot of your partners, it's like the building of the trust part. So having these like NHS, for example, or like King's College London back you up, I think is also a really important aspect of, of the app because people tend to think, um, maybe initiatives like this, they don't have, uh, these large partners that are backing them. It's hard to create this trust. So I think that's definitely one of the most interesting part, um, of the app. So, um, have you noticed any significant differences in the data when you've, uh, in the United States or in Sweden, um, and how you generally talk about the virus? Um, there are some differences, but being just a simple engineer, I'm not the right person to, um, to really comment. Um, I mean, Sweden is very interesting because they did not impose any lockdown, um, at all. Uh, they basically just, um, relied on population to, uh, you know, distance themselves. Uh, whereas UK had a very stricter, um, lockdown mechanism, um, on the U S sadly, you know, it's also, it's again, a different story about lockdown and especially about kind of, um, you know, releasing it a lot faster than maybe the UK. Um, but sadly we also have less data. So it may be a bit less interesting to, um, to compare. Um, but there are definitely some, some, you know, some differences between the country. Like we see in Sweden that infection rates are, um, are generally higher than, than, than in the UK, just because there was, um, there was no lockdown. But the thing I would say to people is basically all the research we're doing is available on covid .joinsoe.com. And there's quite a lot of, um, you know, various research paper have been published in the last few weeks, um, in various scientific journals. So it's, it's, it's all available and they can, um, you know, find out about the details if they want to. Um, of course. And you say, um, previously you said that a lot of these people would use every day, they would go back and kind of track their symptoms. So are you seeing certain spikes of traffic, maybe like in the mornings with people putting in their symptoms? So it's all the time having these influxes of traffic, or are you seeing, um, primarily just a steady, uh, traffic trends? So there was basically, there were basically three different scenarios. Um, we do some notifications daily. So whenever we send notification, we see traffic spike, um, that's, that's, uh, immediate. And then we actually have to be a bit careful. Like we ramp up the notification. We don't send a notification to 1 million people at once. So we'd actually spread it over a few hours. And when we start, we start kind of, you know, um, semi slowly and kind of accelerate just so that it gives more time for the infrastructure to scale, uh, to, um, to the right size because we're using auto scaling mechanism. So that's obviously, um, one of the main, um, way we, some, some, some additional traffic. Um, the two other ones are interesting. So the first, you know, the biggest traffic spike that we had actually was probably on, on day two, uh, in the UK, basically day one was actually quite good. We launched at 9.00 AM in the morning. And basically we had Tim Spector, which is kind of one of the founder working at KCL started tweeting about it. And, you know, there was almost instantly some traffic, some people picking up and retweeting and et cetera. Um, but I think possibly what happened is that the press started learning about it during the day, but actually prepared kind of, you know, uh, articles and news mostly the next day. And the next day actually at 7.55, um, the whole system crash on the load. Uh, there was just so many people trying to, to install the app. Um, so it was like absolutely amazing. So at the end of the first day we had 200,000 people at 7.55, the next day we were already having more traffic than any point at the day before. And by the end of the day, we had 1 million people, um, subscribe. So we handled that spike, you know, not too well, but we cover it actually in like 30 minutes and the system was online. And one of the thing we kind of, um, always are careful about or kind of struggled with is that there's been quite a few times where Tim or other people were on TV, or for example, there was one time where a journalist asked, uh, the government about our app, um, on, on, you know, during live questions, uh, whenever that happens, you basically have a 20 X traffic spike. And that literally goes from, you know, some flat line to the next second 20 X. Like it's not like scaling over the next, you know, 10 minutes or something like that. It's just totally instantaneous. Like people are on TV. They see it. It's like, Oh, wonderful. Let's, you know, uh, let's, uh, check our symptom. Let's download the app, et cetera. Um, and that's actually quite challenging because, um, you know, auto -scaling is great, but at least, you know, the auto-scaling that we have just doesn't, um, adapt absolutely instantly to traffic spike. It needs actually a couple of minutes to, um, to find kind of the right response. Um, so when that happened, we tend to have a bit of a latency. Latencies goes from, you know, 150 millisecond to interact with any screen on the app to, you know, basically maybe a couple of seconds or more, uh, and that lasts for two minutes and then everything is back to normal. Yeah, exactly. So, um, so were you more worried whenever you launched the app about, uh, types of cyber attacks or was it just trying to figure out how to handle all these huge, uh, influxes of traffic? So I think when we launched the app, there was one thing on which we're really, really clear, which is even if the app was small scale, we're still worried about security. And that's something on which, you know, we, we never, um, we never took any shortcut. Um, so, you know, security, you're never 100% safe. Uh, you know, it's like, you know, you could be in a perfect condition, uh, like, you know, you have the perfect health. You could still cross the roads and, you know, uh, be it by a bus. Um, so, you know, or you could have still, you know, cardiac arrest or whatever, like it, it could happen. And I think, you know, security people have to understand that to a large degree, um, you know, you can do everything right. That doesn't mean there is zero risk. That just mean it's a lot harder than, you know, almost everybody else, um, to, you know, to, to be attacked. Um, so, um, we didn't take any shortcuts. Um, but we're more worried about that, uh, for sure, uh, than kind of scalability and traffic spike, because at the beginning, um, we had no idea if anyone would install our app. So we just, you know, we didn't want to have any data breach, uh, but, um, scalability was, was, you know, not something we thought we would necessarily have any trouble with. Um, well, it turns out that we did have some trouble with that, but kind of, you know, reasonably quickly, um, stabilize and security. I think we've, um, you know, for the next kind of months after the launch, we basically spend a lot of time still kind of, you know, improving it. And that's, you know, that's why we also contacted Cloudflare in the first place to try to get, um, to get, you know, some, some additional support, some, you know, protection against, uh, various possible effect vectors or just, you know, denial of service, um, and, and, you know, various things like that. So yeah, definitely was something we're a bit worried on about from the very beginning. So what do you think are the biggest, um, looking back when you launched the app and to where you are now, what do you think the biggest difference is in terms of what you've seen, uh, on the app in terms of traffic or any attacks that you've had, or just where you've progressed from, uh, there to now? So, I mean, the, the, the traffic patterns have been reasonably stable in the sense we had like huge spike of traffic the first few days, a lot of people, then there was some kind of, um, reasonably linear growth. And now we had a point where it's kind of more, more stable, not necessarily a lot of, uh, user growth, um, which makes sense because our biggest market, market's probably not the right word, but, uh, our biggest presence is in the UK. Um, and with the lockdown, people are a bit more relaxed. Uh, so it will be interesting to see actually what happens, um, if numbers of case actually start increasing again, because it may actually lead to this kind of, um, not only a second wave of cases of COVID, I hope not, but also a second wave of people, you know, starting using the app because they realized that, Hey, actually it's not, you know, uh, it's not a done deal. Um, so I think traffic patterns has not been too much of a worry. Um, I think in terms of security, as I said, we've improved a lot. Um, we were also very lucky because we had a lot of partners from the very beginnings, a Cloudflare on one side, also, you know, some company reach out to say, Hey, we'd like to do some penetration testing, probably not for you, which we did. We're very happy that they didn't find any, you know, major issues. Um, it was a couple of small things such as, you know, upgrading, um, SSL that they've suggested that we did, which was actually just, you know, clicking a button in Cloudflare if I'm, if I'm honest. Um, so we, we, we did the things we've improved. Uh, we've added, you know, additional layers of security in various places. Um, and that was definitely, you know, a good thing. Um, we spend more time automating our infrastructure because, um, it's been built, um, extremely rapidly in the span of a few days. And that was definitely one point where we were, um, a bit weak kind of being able to reproduce the infrastructure. And if it's on the cloud, um, having everything automated, um, I guess the biggest change has really been kind of the evolution of the app. Um, because the way we've built the app, since it was this kind of massive time pressure to release something, and we just had no idea if it would work. Um, you know, the, the actual app on your phone, we took every single shortcut we could like literally, um, when we launched the app, for example, you had a registration, but you could not reset your password. Um, which turns out is actually something that people really, really, really do want. So we had a lot of people and a lot of bad reviews because people are like, what do you mean? I can't reset our password. You know, it's totally fair question. Uh, yeah, we do want them to reset the password. It's just, you know, when you build an app over four days, uh, that's not necessarily the first feature you need because, you know, you can totally leave without reset password, but you cannot, uh, leave without, you know, the core, um, feature of, of the app. And I think some of the thing that took us a lot of bandwidth in the next few weeks is just, you know, improving the app in general, the UX, but most importantly, adding a lot of questions and features. So, you know, initially we had a reasonably small set of questions and given to us by the scientists when they realize, you know, after, um, two days of the app running that had so many users, it was like, oh, this is wonderful. Now we've got so many questions we want to ask people. And what has been very interesting, it's, you know, the question I wanted to ask was also, um, quite linked to the various research going around the world, but in many case at very small scale, I think, you know, I was in touch with some researcher recently in France that were, you know, what, what they are trying to do is very interesting. They realized that they've got great, uh, data about hospital admission, but they've got absolutely no data whatsoever in France, apparently about what happens outside of hospital. Um, and it turns out there is actually, you know, a lot that happened outside of hospital because the, you know, the hospital are basically the worst case. Those are the people that are very, very, very sick. So, you know, you actually have only a very, very small subset. Um, and I think there is a lot of research has been done, um, you know, kind of, you know, small scale, uh, data sets. And I think a lot of what our scientists or scientists we're working with have been trying to, uh, get is, you know, how we realize that, uh, you know, those category of people are at risk. Okay. Let's ask our people if, uh, you know, if they fit into that category, because for quite a few of those, we already know if they had COVID or not, they've been tested, we've got the data recorded, um, based on that and their symptoms, we actually have some reasonably good prediction models on whether we do think they've got COVID. Now we can add that additional dimension, which is, you know, um, do they fit into category X, Y, Z, and then almost suddenly that gives you, you know, data set with, you know, several million people, uh, that, you know, tell you if, if that factor is actually a risk factor or not. Um, so they are super excited because this is, um, they've never done research at that speed basically. Uh, and we've got that amount of data. So a lot of what we've done in the last, in the last couple of months, basically just adding questions after questions and just try to expand the data set in various dimensions. Yeah, of course. And I want to make the most of your time. So really my last question is, what do you want more people to know about the symptom tracker? Well, I would say, you know, just, uh, um, you know, use it, uh, not for your own benefit, but, um, for, for, for helping the science. So, you know, go on the, on COVID.joinzui.com, uh, download the app. Uh, you know, we're not doing anything bad with your data. Um, you know, you can look at the various research article. Um, and, uh, yeah, if you think, you know, that's, that's great. Just, you know, advertise it on one side. Um, if you are an engineer working on, on mobile engineering, uh, the code is actually open source. So, you know, uh, you know, go on GitHub, give us a hand, uh, and, you know, um, help us make the app better. Yeah, of course. Well, thanks so much, Julien. I definitely am going to download the app and take and track my symptoms. I'm definitely going to do that. And I hope everybody at Cloudflare and everybody who's watching will do it. So thanks so much for your time. Thanks to you and have a great day. You too. Bye-bye.