Lessons from building the UK’s test and trace app – Randeep Sidhu

Lily Smith: 0:00Randy, you’ve done a lot of different kinds of roles. So what’s the most intense environment you’ve ever been in? Geez,

Randy Silver: 0:08I worked really hard launching Amazon’s music store in the US in the UK way back when, you know, we did night shifts in the warehouse. And I don’t think I took a proper weekend break for like six months. I was pretty burnt out at the end of that.

Lily Smith: 0:24Well, yeah, that sounds pretty full on. But I’m afraid it’s not on the same scale as what Randeep sedu. Experienced recently, he came into lead product for UK his COVID test and trace app last year, launching an app to literally save lives in the middle of a global pandemic is probably up there as one of the most intense product launches.

Randy Silver: 0:44I think you might have set me up there really, I wasn’t trying to compare. But this was a really great chat. I learned a lot from ram deep. And it’s a format of interview that I want to more of, you know, we looked back at the project almost like a bit of a retro.

Lily Smith: 1:01Also, he told us something after we stopped recording that if you have questions for him based on his conversation, you could just reach out to him on Twitter and tag us to at MTP pod, and he’ll be really happy to get back to you.

Randy Silver: 1:13And we’ve got a link to his Twitter handle in the show notes to the episode. But you know, you can’t tweet at him with questions from the episode until you actually hear the chat. So let’s not waste any more time with this intro.

Lily Smith: 1:33The product experience is brought to you by mind the product.

Randy Silver: 1:37Every week, we talk to the best product people from around the globe about how we can improve our practice and build products that

Lily Smith: 1:43people love. Because it mind the product calm to catch up on past episodes, and to discover an extensive library of great content and videos,

Randy Silver: 1:52browse for free, or become a mind the product member to unlock premium articles, unseen videos, ama’s roundtables, discounts to our conferences around the world training opportunities.

Lily Smith: 2:05Mind the product also offers free product tank meetups in more than 200 cities. And there’s probably one way. Randy, welcome to the product experience. It’s really great to be chatting to you tonight. Thanks so much for your time. And so before we kick off with our topic for the evening, it’d be really great if you gave us your origin story and kind of how you got into product.

Randeep Sidhu: 2:32Yeah, so it’s a bit I suppose people very non standard I, I studied evolutionary biology, like means no, no, it’s completely relevant for most things. And I started the world in kind of the NGO space because I kind of was lucky enough and worked hard enough to get to Cambridge and thought education was important. So I did teach first helped set this charity up. And then kind of got into consulting, like innovation, product innovation, kind of just data science degree. And then I became really good at user behaviour, user insight, and then it kind of fell into my first product role in gaming, and then just grew from there. So then I spent the last, what, six, seven years in health tech. So yeah, it’s non standard, based all about human insight, which is what the best products are.

Lily Smith: 3:14Awesome. And you ended up leading the development from the product point of view for the test and trace app in the UK. So tell us a little bit about how you ended up being the lead on that project?

Randeep Sidhu: 3:28Oh, yeah. So actually, I think it’s quite pertinent because there’s pretty much I think, almost exactly a year ago today, when I was called up out of the blue. So I have been working in health tech for quite a while. And I was at that point at a company called Babylon health, which is in the UK in America. And I was leading a lot of their products globally, as well as building health tech in parts of the world like Rwanda. And I got called up out of the blue by someone saying, Well, you know, the NHS are looking for someone to help them. And at that point, they pitched it like they just need a bit of advice for those kind of non UK listeners. In July last year or two end of June last year, the government had spent three months the National Health Service NHS had spent three months building a COVID app. And is this very similar to the ones globally but that COVID app was to detect if you’ve been exposed to someone who had COVID. And to notify you. That app kind of never launched in is quite catastrophic failure. So in my head, I thought, okay, they just want a bit of advice, or someone who worked in healthcare to give them some thoughts. unbeknownst to me, they were kind of sniffing me out, and then I think I spoke to him over the weekend. They called me up two days later and said, we really need you national emergency. That was Tuesday night, Wednesday morning, I spoke to my employer, it said, like this urgent need for me to try and do this thing. Lots of people at that time will say, don’t do it. It’s going to be career suicide. Like why would you do this? Failed what is in government? Like, it’s not going to work? You know, COVID is too unpredictable. But you know, I think at that point, the risk i thought was is a bigger risk to people in kind of those underrepresented communities like brown and black people were dying a horrible rate. So I thought, I have to do this. And so I took the chance and actually left on leave, or you know, the temporary leave from Babylon three days after. So I spoke on Wednesday morning, left Friday night, started a test and trace immediately. And that was 11 months of my life. So yeah, it was extreme. It was it was very extreme.

Randy Silver: 5:36So before we go into any specifics of it, I’m curious. You get this call, you have a perception of what government worker might be like you did, as you said, Everyone told you don’t do it. It’s already failed once. What was that perception? And there was a different when you actually went there?

Randeep Sidhu: 5:55So I suppose Actually, I mean, I have a perception I suppose I’d asked you, what do you think was your perception of government? I suppose it’s probably a good, good question. What do you think?

Lily Smith: 6:06Yeah, behind the times, kind of stuffy, lots of red tape waterful. All those things, although Actually, I kind of thing this slightly tongue in cheek, because I know, our government Digital Services team are amazing. And I’ve done some work

Randy Silver: 6:21with them. So I’m not going to answer this one,

Randeep Sidhu: 6:24I would say, is the best and the worst of what you’ve described. So there are some genuinely truly amazing people. And let’s not forget, government salaries are not high. So anybody who does product or tech, could be earning multiples of those salaries outside of government. So we should always remember that they are key workers in the same way as many other people are, like, we kind of forget that we like them, you know, this, there’s actually some people for life choice to do this. And actually, which I maybe wouldn’t have done. So you know, there are some amazing people, the challenges are as you go more senior, because people are quite tenured and have lived and worked there for quite a long time. Whereas in our worlds, often people move for a year or two years, you know, it’s quite amorphous and quite fluid. That’s not always the case in government. So there are some people who are as you get more more senior, you get more and more people who have less experience that kind of parts of technology. The other challenges, it’s, I think, someone described Americans Randy as a nation divided by a common language. And that sort of, you know, the same words mean different things in America in the UK. And I think that’s often the case in government, where they use the words of Scrum, agile product, but they mean very, very different things. And so there was a little bit of education to make sure people use the terms in the way that maybe I and the rest of the team would understand. But the biggest challenges, like many large organisations, with some of the listeners will will, will be part of is it’s kind of an amorphous blob. And everyone’s got a point of view. And whereas in some companies, you go, the CEO says, No, or the head of product can say, No, we have to establish that process in government, because there’s this really tedious thing where someone goes, my minister needs this. That’s like, I don’t care what your minister needs, because your minister doesn’t understand what we’re trying to do. So there was a problem before success, then when the app got successful, there’s a different problem. But everyone tried to change it for their own purposes, like transport, immigration, you know. So it’s just about kind of making sure you focus on what the product is doing, and ignore all the noise, which is what all product managers have to do anyway.

Lily Smith: 8:38So you were conscripted into this role? To try and salvage the test and trace app, there’d been another version of the app that that had failed for whatever reason. So like, what did you walk into one on day one? Like, where did you start? How How did you approach your first? Oh, my

Randeep Sidhu: 9:04God, I walked in. I should mention at this point, I mentioned what the COVID app is for people who aren’t necessarily sure of it. So the first step, like some other apps, like India, and I think France was built with as an engineering project. To see can we use Apple and Google phones or going to Android and iOS phones to work out if you’re exposed to someone who tested positive for COVID, and how long you are close to them for. And so they were trying to do it, which basically meant your phone, even when it was asleep will be broadcasting and receiving signals. Now, the problem with that is phones, particularly, you know, especially with that battery saving technology and other stuff, they just switch themselves off, they switch off Bluetooth, if the phone doesn’t think it needs to be on it won’t be on. And so the efficiency of this programme that was being built was really, really poor. So it was great engineering, but it didn’t work a fish efficiently. So I came in, and that code that was built, wasn’t deployed. And it was also tested in a really limited way. So it’s great technology, great engineering, but essentially all of that team got disbanded. So I came in to kind of scorched earth. It was originally built in a department called NHS x, who are one division of government, it got moved into a completely separate division of government. So I joined that division, still part of the NHS in the Department of Health. So I had no product people, because it turns out actually, just before they tried hiring someone, Junior, so I was a director, which is quite senior. Why you found out six months in is they actually hired someone who lasted five days and left. So because I walked in, I was like, why is there no one here? I’m just, it’s just me, with you. Where is everyone? And I didn’t realise that someone actually tried bringing some they didn’t dare tell me that they that this lady started on a Monday left by Friday. And I turned up on Monday and was like, Don’t let him leave, don’t let him leave. So I kind of came in with no historical code, really, and no product definition of what it was trying to do. So that’s why I had to do so my job to be done. Number one was, what is the product for? And what’s the strategy to get it bill? So I can simply that’s what I tried

Randy Silver: 11:14doing. We’re in something so big, and you know, an international pandemic and crisis. That definition could have been as expansive as anything. You don’t want to fail fast in this, you do want to iterate, but your MVP has to actually work. So where did you start? How did you define success and create a definition of something that was good enough, but not so ambitious, that you weren’t going to get there quickly?

Randeep Sidhu: 11:44So there was the first thing is, I mean, the what the product was trying to is actually really basic, it was trying to stop COVID, which sounds that is a huge problem. So it’s trying to reduce the how do you do that? So I looked at the lessons of what maybe didn’t work in the first app. The first app didn’t necessarily have clarity of its purpose. It was good, but it was a bit of a scientific project. Can you measure this with an app, reverse engineering process? It was centralised data. There’s lots of things which didn’t work for the general public. So you know, a centralised database tracking where you are would not work for many, many people. So what we focused on is, what is it this app should do? So it should stop COVID? How do we stop COVID. And what we ended up thinking about was, the product itself can be engineered. And we ended up using a Google and Apple piece of technology, an API that they built. So we were using that to do the broadcasting and listening. But what is it that would make it work? And that’s where psychology came in? Because what is it the people wanted at that point, so let’s put your put your mind back to July last year. Literally, no one knew what was going on. COVID is completely unpredictable. It was scary. So we needed to think about the psychology of the reassurance that public wanted, like when you’re in a pandemic, people want to feel like they can take action, they want to do something. So we thought, What’s the psychology of what people need, so they need to be able to use the app, but to use your house have some functional basis and needs to do something. So beyond just pinging you when you get exposed. So we thought about, can we have a check in feature, we can see, you know, without having to test and learn lots of third party kind of testing and checking in systems have been built, I thought we’d build one ourselves, similar to what we liked in New Zealand really well, that would give a user ability to do something active to protect themselves. So we basically quickly defined six features, three that helped me selfishly as an individual, because if I helped myself, I would use the app. And those three kind of features, you know, we’re notifying if you’ve been exposed the basic function of the app, letting you check in somewhere, if there’s an outbreak in that venue, you’ll get alerted and telling you about the local area risk in your postcode region. But as well, things that I knew were more public benefits like booking a test, symptom checking, you know, stuff, which actually you might need if you’re exposed and that account downtime, and people didn’t know how long they should isolate for. So by focusing on those two areas, we thought, we need to appeal to selfish needs and protection. And then so that’s kind of we thought about the features. And we started testing that with users to see if they wanted it.

Lily Smith: 14:23And I guess one of the things that you definitely didn’t have was the luxury of time. And so how did you ensure that you were kind of moving at speed, especially when you were having to set up a team as well as you know, basically start from scratch?

Randy Silver: 14:42You also, if you don’t want Let’s start with the with the team specifically, you said you walked in and you were there alone? How did you get back going?

Randeep Sidhu: 14:51So the two things about team is we needed a team and we needed a structure for the team to be successful. So as we spoke earlier about government I mean, the team I basically just stole people I looked across because I would not have found people. There were there were lots of consultants and people in testing trace. And I found a few key allies and grew from there. And so that’s, and I honestly, hand on heart had an amazing team without a family. And so we managed to get the first person explained the mission of what we were trying to do, which was, you know, stop COVID. Look at being inclusive in doing that, because actually, there’s something I learned when I used to be a teacher, which is a lesson that works for a low ability child is just a good lesson. So if you build an app that works for the bottom of the pyramid, it is just a good app, I said, that vision of not just building an app and making sure it helps those who are most impacted by COVID, like COVID scoring, I think, if you’re in the least in the UK, if you’re poor and the bottom quartile, you’ve got twice the death rate of COVID. If you’ll be any black Asian minority, I think you’ve got twice the death rate, if you’re black is four times the death rate. COVID itself isn’t a disease of inequality. Society made this become a disease of inequality. So when I started explaining that to people, we got a lot of people who are really passionate about trying to not just stop COVID, but fix that mission of trying to do it in the right way. So actually, I picked two people, they then recruited others, and we kind of went through a vetting process, but I ended up with 10, product managers, non standard, like different bits of project and delivery. But we changed the process of how, how we bought a product in government. So we kind of had product, first of all, because in government, they didn’t fully understand how a product works. So we were saying that, you know, I would help to find what we build. And that kind of foot soldier, product managers would kind of define the requirements underneath, and then went to a dev agency to build and then other bits of government like policy provi assurance that this product is a medical app. So I needed medical insurance, we just kind of built a funnel where I was at the beginning. And I just kept pushing things through. So we had a T file quickly from consultants and a process to make sure it worked.

Randy Silver: 17:01So you didn’t follow the GDS approach of let’s go through the all the different assessments of alpha and public beta and private, private beta, public beta, etc. You did it a totally different bespoke approach for this.

Randeep Sidhu: 17:15There was I bill app was launched six weeks after I turned up. An app that worked and we focused on was doing it in pilot, because the reality was we did some user testing, we did three different kinds of testing for users. We did kind of small group, user focus, you know, testing, as most people do, we used to research and focus groups. And then we actually did a pilot, and we the community outreach. And so we launched that within about six weeks. And then we had it in pilot for six weeks. And in those six weeks, we did a lot of engagement to kind of keep refining and testing at mass scale, instead of just three or four people in a room. So yeah, we kind of did it slightly differently, but it worked. And that community outreach was really important.

Randy Silver: 18:05Usually makes it easy to build an embed micro surveys into your product, to learn about your customers in real time. product teams at companies such as square, Adobe and Dropbox, use user leap to gather qualitative insights as easily as they get quantitative once and automatically analyse the results. So teams can take quick action,

Lily Smith: 18:26if you’re part of an agile product team that believes in building better products. And I sure hope you are by obtaining insights from users, then give us a leap a try for free by visiting us elite.com. That is user leap.com. So I’m curious were you doing? Wait, how much time you’re spending, thinking about the process, versus just getting it all done? Because you know, when we when we talk about product development, and it’s like, you know, retrospectives and and it’s good to like, look at your process and stuff. But I can, I can imagine that you just don’t even have time for that either. It’s just a case of like constantly, just asking yourself, how can we move faster? Pretty much.

Randeep Sidhu: 19:19I mean, I was saying seven days a week, 15 hours a day. I mean, just to get it done, and the rest of the team are the same. And so we did have little iterations of you know, we kind of did in house retrospectives, where we’d say, okay, we need to kind of build a feature. Let’s test this concept with users. Let’s get someone else who has lots of parallelization. So we would kind of start thinking about how to build something while someone was testing it. At the same time as someone else. We’re trying to engage with users. Someone else was designing it. But you’re right, we couldn’t follow a more formal structure of iterative loops. But we did focus on what we thought were the core product ideas that those six features I mentioned earlier. And there was The complication was we were always having to think about the direction COVID were moving. So we actually focused much more on checking and user notification if they’ve been exposed as the two primary use cases. And we were thinking, what would happen if COVID goes in an x direction? Are there bets we’re going to start thinking about or testing in advance of that happening? So you’re right, we weren’t able to move in a more standard way. But I think the pilot really did help in us understanding how it worked. And I’d highlight that the pilot we did was very different from the one that was done in the first step. So the first step piloted in an environment which is very atypical for the UK, it’s a little island called the Isle of Wight, which, if you know about diversity in the UK, it lives up to its name. It’s very atypical, even for white people in terms of how it works. And so we made sure we tested an urban dense environment with multi ethnicities, multiple backgrounds, multiple levels of poverty, to get a real representation of what the UK is like. So the app, if it works for this demographic will work for everyone. So that was something I think we focused on more than because we couldn’t even do a B testing often because of limitations in how we could gather data. One of

Randy Silver: 21:10the things that impressed me about this is you did it this quickly. But you also made it accessible straightaway. So you know, so many product, people say accessibility is something that will come after the MVP. We’ll build that in later. And how did you manage to bake it in from the start if you launch something that quickly?

Randeep Sidhu: 21:31So what’s interesting, actually, that gun to your head actually changes how you think about a product. So if I said to you, I’m equally guilty for this. A lot of the products I’ve built are for people who have smartphones or people who understand technology, games, healthcare. But if I said to you think about this universal, what can you think of this as a universal app? facebook, facebook, facebook zero exists in India, but WhatsApp, the zero Chrome’s zero UI, basically, the most universal app is Sunday isn’t an app is basically a text messaging. So you basically how to go back to complete brass tacks. I will mention a podcast that you guys actually did a few weeks ago, Dan Olson, he talked about a product pyramid product market fit pyramid. And when he mentioned that, I said, Oh my God, that’s kind of what I did. So I did was I said, This healthcare is universal. Everyone needs to be able to access this, of course, is digital exclusion. Not everyone has a smartphone that could use it. But let’s assume if you have a smartphone, how can I work out what I should build? So I know that I need to build these six features, how do I build them. So then basically, I built I went back to basics, I will use Maslow’s hierarchy of needs, if you’ve ever come across that. So it’s like a triangular pyramid, if you don’t know, and it’s the basic, the ground level is wide. And I think it’s safety, I need to be, you know, I need to have a safe, I need to have shelter. The next level up is I think, security, then it goes up to you know, wherever, wherever, and most people know it.

Randy Silver: 23:02So I base there’s the there’s the amended version that has Wi Fi as the bottom level now. Which probably

Randeep Sidhu: 23:10will act as a serious consideration, actually, because not everyone even has internet connection. But I actually adopted Maslow’s Hierarchy when let me go to the bottom. Who is this app going to be dangerous for? and genuinely, it was a really interesting challenge. Because the spot level is there are people for who having this app will be dangerous LGBT people and part of the community refugees, who will be of unstated immigration status, domestic violence sufferers who may be if the app has information in it, that could be detrimental to them, you know, so we basically started with making sure it didn’t have any information or was safe. And that’s where anonymity came in as a ground level assumption. If it was anonymous, it would work for LGBT refugees, communities, then the next is can I use it? Is it in my language? Is it accessible is so you know, does it work with a screen reader? So if you build up that way, and it was a really useful model, because actually, I built it, I kind of put criteria of things that we needed to do all the way up to the top, which in Maslow is self actualization, and in product parlance would be, do I love it? Do I want to reuse it? Well, I recommend it. But most of us spend our product lives thinking about the top of the pyramid of getting people to reuse it, love it feel secure. Actually, the bottom is where you start, because if you build bottom up, you’ll get a lot of people who can and will use it. So back to your original question. That’s how we did it. And we made sure that the schema UX was you know, accessible, was translatable. You know, we built it in 12 languages, which represent a programme 97% of the spoken language in the UK. So it’s, it was just a rod that we had to, you know, build for ourselves. And I said to the team, because I couldn’t be across every decision if any problem comes up. If this is a bottom of the pyramid, it gets fixed before it’s the top of the pyramid. So we always prioritise things that were about safety function access, before we got into love, reuse, recommend,

Lily Smith: 25:08and when you were working through the pilots, and, you know, often we’re kind of looking at success metrics, and you kind of mentioned that the purpose of the app was to stop COVID. But that presumably, and, you know, within the six week timeframe, and that would have been quite hard to, you know, to see whether you were stopping COVID in there. And also, you know, if it’s contained within a certain area, that doesn’t necessarily mean that it’s completely contained. So, like, how did you know that the pilot, how did you measure the pilot, in terms of its success, and know that you were ready to roll it out to that next group of people.

Randeep Sidhu: 25:51So what was really challenging is, the app was anonymous by design deliberately, because of that thing I mentioned about the bottom of the pyramid. As such, we published the code. And we signed a privacy statement, which means we could unlike any other I’ve ever worked in, we just measured data, you’ve just got a whole set of Google Analytics, which is constantly things, you could only measure something, if we convinced the Ico the Information Commissioner’s Office in the UK that we can measure it. And there’s a separate board in the UK called the Office of the National Data Guardian, who are kind of a group that protects civilian data rights in the UK. So we have to convince both of those before we can even measure anything. And because it was anonymous, we didn’t even know where you were. So we had to use quite blunt metrics about how many downloads, we’ve got, how many check ins we were getting, because we couldn’t have any location based data. And of course, the app is better if we know there’s a concentration of users. So we did lots of surveys, field research, finding out whether people using it, and actually just feedback. So a lot of the stuff we were focusing on was true metrics around trust, because we believed if users trusted it, they would download it and then use it. And unfortunately, yeah, that we focused on trust metrics.

Randy Silver: 27:05When you’re working with other bodies like that, like the Office of the data Guardian, and the Ico. They have their own processes, they have their own challenges and things like this. And that can be you know, sometimes you’re at the mercy of other teams and departments in the company, within this year, you know, it’s hold different departments, government departments, we How were you able to work with them, were you able to get them to dedicate people to this project and have them become part of the part of the team in a way without losing their objectivity?

Randeep Sidhu: 27:38I think that’s the point. Everyone wants to be helpful. Everyone wants to help. And in COVID, even more, so you just make sure that you can communicate clearly enough your need, and make it clear if they’re not helpful, what the impact of that will be, because it’s quite is it does focus the mind when you say, if this isn’t approved, people will die. Or if people don’t take if you can’t help us work through this data problem, there is going to be issues. So we of course, worked with them more closely. And I think there was some great people we did work with there. But yeah, it was back to product, you have to be able to clearly articulate why you need something and the benefit it provides to end users. So I found the most compelling thing was to basically use that and focus on end users, because that focus everyone’s mind, but it wasn’t easy.

Lily Smith: 28:31You mentioned earlier about, you know, seven days a week, 15 hours a day. I’m not sure. Many people would be able to last that long, working that many hours, let alone on such a critical project for the country. How was the morale of the team during that time? And how did you how did you survive?

Randeep Sidhu: 28:57So honestly, I think I think is quite personal in terms of what you were doing. So in terms of why I was doing it. I mean, like my mum work worked in a factory before COVID, someone that she used to get a lift with into the factory, died at the lunch table. He just and so I was speaking to my mom, and she ended up living with me for four months while I was doing this because I took her out of where she was because I’ve got key workers in my family in the hospital and the police. So I was like, You can’t stay there you will be exposed. So it was very acute because of who was suffering and why. And so I think for me, that’s something that motivates me. But what’s interesting is, the team we ended up creating not because of me was incredibly diverse and incredibly passionate about the same thing. So actually, it was very energising It was incredibly frustrating when things didn’t work. But there was definitely that kind of we’re all in a fight and all we can do is do I fight our little battalion in the front. No one thought was going to work when we first started. So we were kind of the underdog. Because we never thought it was going to work. And then we started feeling success. And we’ve got to launch a new one. Because you know, even piloting a new one was challenging because of big p politics. It was not a conservative run area, which is the different that was run by different government pide and the ones in power. So they didn’t want us to launch there. And those See there was so much so much stuff holding us against against us. You were kind of like David against Goliath. Yeah, we can do that we can do this. And so I think that kept us going. And then we saw the number of downloads, I think when we launched it, it was the fastest downloaded app in UK in the UK history, I think that thing made us realise that what we had done had made an impact. So I think just making sure we focus on the successes when we got them. And I did. It was something a bit odd, which is how I kept morale up is by making people understand that like someone says to me, when I work in healthcare, Randy, you will kill someone in your product, someone will die because of a decision you’ve made. And you just have to be comfortable with that happening and know that you did the best you could before that happened. So I kind of gave them a weird pep talk about this going, you know, you can’t assume that, no, someone’s going to die, which means that all you have to do is focus on your job and not worrying about the terror of you doing something wrong, which would make it worse. It’s it’s an inevitable consequence of some of the decisions you make. There is no perfect outcome here. So I think the toxifying some of the risks there also helped. Not sure if that answered your question was a bit of a strange answer.

Lily Smith: 31:35It just sounds even more stressful than self care was in Poland. Yeah.

Randy Silver: 31:42So given all that, I severely hope you never have another job quite like this. Because I severely hope that we never need you to have another job quite like this. But looking into the future, the next thing that you start new, what’s the something you’re going to take with you into that? what’s what’s the lesson that you learned that you’re going to carry through into your career?

Randeep Sidhu: 32:06I think there’s two things I returned relevant to the product community. I think one never forget passion and clarity of purpose can really motivate people around you. So you asked Randy, how did you get everyone to kind of do what you wanted? I just by sheer force of will, able in government, outside of government, different departments, Apple, Google, whoever, we just made it really clear why we’re doing it, why we’re passionate about doing it, and what benefits it would provide, let people help you make sure that they can feel that they’re going to be helpful. And that’s something I didn’t realise I could do until this job. I think the second thing I’d say, is kind of your gut instinct sticking to your guns, like the one mistake I may have made, I made loads, I’m sure. But we have a tendency to just build digital products as a red facsimile of the offline product like oh, teaching works in this way. Let’s copy it and make it digital. Healthcare works in this way, let’s just digitise it. And I think probably a proxy of the time we had. So there was a feature, which was a check in feature, which I copied how a sign in book was written up at a venue just write your name, what time you turned up and who you were. And in an effort to make it easy for others to use the non digital people who worked in other government departments, I copied the exact same system. Even though I could have you automatically check out I could do lots of other technological stuff to make it more accurate. I dumbed it down to make it more like an offline process copying that, which basically that meant it didn’t work as effectively as it could do. So there’s something there about always sticking to your product guns and trying to make sure that you kind of always think New when you’re doing something. So those are the two lessons I’ll take from it.

Lily Smith: 33:48Randy, it’s been so interesting hearing this story. And thank you so much for joining us, I hope you have had a very well deserved rest, having gone through and developed an amazing app that’s helped millions of lives. So I think we should also give you a little chair, you and your team.

Randeep Sidhu: 34:13Just to be clear, and the people who did this? Well, I mean, like the thing that never gets reported is this was a global success. And we will help Apple and Google improve their algorithms globally. And it’s a UK success story. I work closely as my team with the Alan Turing Institute and others. So what people don’t realise is the lead data that our UK population gathered, improve the algorithms for Apple and Google globally. So that’s something I think we as a nation should be proud of.

Lily Smith: 34:38Amazing, really, really impressive and fantastic. And thank you so much for joining us. Thanks, guys. Wow, I feel kind of emotional after that. And I want to give that whole team a massive hug and Maybe some donuts,

Randy Silver: 35:01donuts. Did you learn that one from Ken Norton?

Lily Smith: 35:05I think he actually stole it from me. I was the original donut giver.

Randy Silver: 35:12Ken was one of our first guests. We may have to bring him back on to settle that to date. But in the meantime, like, subscribe, make fun of Ken if you want to, and then we’ll get him back on. Have a great night.

Lily Smith: 35:32hates me, Lily Smith

Randy Silver: 35:34and me Randy silver.

Lily Smith: 35:37Emily Tate is our producer. And Luke Smith is our editor.

Randy Silver: 35:41Our theme music is from Humbard baseband power. That’s p au. Thanks to Ana kittler, who runs product tank and MTP engage in Hamburg and plays bass in the band for letting us use their music. Connect with your local product community via product tag or regular free meetups in over 200 cities worldwide.

Lily Smith: 36:00If there’s not one Nagy you can consider starting one yourself. To find out more go to mind the product.com forward slash product tank. Product tech

Randy Silver: 36:10is a global community of meetups driven by infor product people. We offer expert talks group discussion and a safe environment for product people to come together and share greetings and tips.