In the latest Prioritised AMA, we were joined by Teresa Torres who took questions from the audience covering a long list of juicy discovery topics. From how to keep track of your team’s discovery efforts and why the process of discovery is just so damn hard, to effective ways to share discovery results and tips on taking your stakeholders along on the discovery journey with you. Watch the session in full above or read on for answers to the questions covered and further resources.
About Teresa
Teresa is a Product Discovery Coach who helps teams gain valuable insights from customer interviews, run effective product experiments, and drive product outcomes that create value for their customers and their businesses. She teaches teams how to connect the dots between their research activities and their product decisions, inspiring confidence that they are on the right track. Recent clients include CarMax, Snagajob, Spotify, and Tesco. Right now, she’s in the process of writing a book: Continuous Discovery Habits and hope to be able to get this out into the world early next year. Stay tuned!
Questions Covered
- Why is building great discovery practices so hard?
- Can outcome-driven innovation – Jobs to be Done – coexist well with continuous discovery?
- How do you get teams thinking about opportunities instead of features or solutions?
- How do you make the case for continuous discovery when your stakeholders have an output-driven mindset?
- How should teams approach solution discovery versus product discovery? Something that was recently written about by Marty Cagan
- How did you come to come up with the Opportunity Solution Tree?
- How do you effectively share discovery results, especially those that disrupt existing plans?
- How do we make a shift from project research to continuous research?
- What can you do when your discovery track trail is leading your delivery track?
- How do you include external stakeholders from product discovery?
- How much does technical risk play a role in discovery, especially when you have a lot of technical debt?
- What discovery synthesis challenges have you seen for companies going fully remote?
- What are some regular recurring questions I can ask my product managers to see whether or not they’re performing continuous discovery?
For even more, check out the list of resources mentioned during the session. AND, if you have a question about discovery that’s not covered here, let us know! We’ll factor it into upcoming discovery content – share your question.
Why is building great discovery practices so hard?
I think it’s as simple as the urgent versus important matrix – the culture of our organisations is that everything about delivery is urgent but discovery tends to fall on that important but not urgent box.
So, if we work in a culture where, every day we’re being asked when we’ll ship and there’s this bug and there’s this missing feature, it’s all about outputs which makes it really hard to take a step back and say ‘ok let’s start talking about impact’ and let’s start thinking about whether we’re building the right things or just pushing things through the sausage factory.
In the last five years, though, we have seen a pretty huge culture shift in the industry, I’ve even seen it in old stodgy industries like insurance, banking, and even airlines and typically, I think business changes really slowly, but competition forces business to change quicker, and crisis forces companies to change even quicker.
I’m hoping one of the silver linings of a year like 2020, is that we see a lot more companies realise that what’s worked for them in the past isn’t going to work from them in the future and that they need a better way to figure it out.
Can outcome-driven innovation – Jobs to be Done – coexist well with continuous discovery?
I think continuous discovery supports outcome-driven innovation.
I try to define discovery in a way that’s framework agnostic because I’ve seen that teams need to use the frameworks that work for them and that every team is going to find that different tools and frameworks resonate with them.
So, as I look at it, we have three things we’ve got to get right.
- We’ve got to focus on outcomes.
- We can’t jump straight to solutions – we have to have some understanding of the opportunities such as, what are the customer needs, pain points and desires?
- We have to develop and ship solutions that address those needs in a way that drives the outcome.
Those are the three buckets that I’d say simplify the structure underlying all of discovery and that’s generally what someone in the organisation needs to be doing.
Jobs to be Done, I think, is actually a really effective framework for uncovering customer needs, pain points and desires. When I work with companies that use Jobs to be Done, usually their jobs end up being the top-level opportunities that they’d have to break down to get to the more specific, smaller needs to be addressed.
I know some people that are really dogmatic about Jobs to be Done, I actually think it’s a fantastic framework, but it’s definitely not the only way to do opportunity finding. I know for some companies the language is really tough and for other companies, the language works really well. I’ve just seen that too often people say this is the one framework everybody should always use.
How do you get teams thinking about opportunities instead of features or solutions?
It’s hard because for a lot of companies that culture is all about solutions and I actually think it’s human nature to think in solutions. I think our brains are wired in such a way that we hear about a problem, a need or a pain point, and we really want to address it.
We experience this in our personal lives too. Imagine you’re talking to a friend while having a drink at a bar (when we used to go to bars!) and they tell you about a problem they’re experiencing. You want to fix it but what’s funny is that that’s usually not what the friend wants. A friend just wants empathy and an ear.
And so I think that the more that we can cultivate this ability to hear a problem, and just pause, it will not only help us in our personal life but also in our work life as product people. It’s really a mindfulness thing.
The other point is that in a lot of companies, a lot of cultures, and a lot of teams, the conflict and disagreement that’s happening in the solution space is, oftentimes, because we haven’t agreed on what problem we’re solving. And again we need to take a pause, take a step back and recognise ‘Oh, I think you’re framing the problem differently from me’
How do you make the case for continuous discovery when your stakeholders have an output-driven mindset?
There are two parts to this question.
Part One: Don’t underestimate how much you can do individually, regardless of how your organisation works. I say this because I’ve spent almost all of my full-time employee experience at companies where we didn’t have strong product leadership. We didn’t have a CEO that supported a strong product culture and really, even my first job out of college, I just went and did it. I was really lucky in that as a college student I was introduced to human-centred design, and I entered the business world thinking this was how business worked. Why was I naive? So I really just had to be stubborn. I just thought I don’t know how to do my job without talking to customers so I’m going to do it.
I know that in big companies there will be policies in place about who’s allowed to talk to customers, and how you talk to them, and you obviously need to work within the constraints that exist, but there’s a lot that an individual product team can do to start to adopt some of these habits.
For example, you can go find a sales buddy and say, ‘can I just come along on your calls?’, you can go find an account manager who’s managing your active customers and say ‘hey can I come to join some of your conversations?’. You can read your support tickets – there are lots of ways to just jump in and get involved.
As far as how to influence and change the culture, the biggest mistake that I see people make is they get dogmatic about it. The reality is that your leaders are in the position they are in, because the way they’ve worked has been successful. So if you’re below them in the hierarchy and you say, ‘hey, there’s a better way to do this you’re doing this wrong’, they’re just going to get defensive and dig their heels in.
The really important thing to do as an influencer is to do two things:
- Bring new information to the conversation to share what you’re learning from customers and share what you’re learning because those leaders may not already have access to that information
- Highlight the gaps in what you’re doing today. That might mean instrumenting your product in the little bits and pieces where you can and start to say ‘hey we thought this was a great idea and it kind of fell short of our expectations’. Start to look for opportunities to expose the pain of not doing discovery well.
Then, to the degree that you can be a maverick, just find a way to do some of it on your own. The best way to influence is to be really successful because you did good discovery.
How should teams approach solution discovery versus product discovery? Something that was recently written about by Marty Cagan
I read that article (Discovery – Problem vs. Solution by Marty Cagan), and when I first started reading the article I was like, ‘Oh no, is he telling people not to define opportunities?’. But then, by the time I got to the middle of the article I was like, ‘Oh no, actually, Marty is saying that we need to take a cross-functional approach to both defining the problem and the solution, and that it’s not enough to define the problem – we don’t create value unless we ship something right’ and all of that is true.
I actually have an article that is titled Product Managers Don’t Own the Problem (And Designers Don’t Own the Solution), which I think is really Marty’s primary point in that article – that we’re trying to create these artificial boundaries between roles when we know that the best digital products come from the trio of roles collaborating together.
I think there was a second part of that article, that was maybe less explicit, which is that some teams spend a tonne of time doing research to find problems and they never ship anything. And actually I see this at companies that are really well known for expertise in design thinking. I have a design thinking background so I’m not criticising design thinking, but it’s really easy to slip into a mindset of ‘hey let’s just get to know our users and have empathy and do research’ and forget that our job is to deliver value to the customer, not just get to know them.
How did you come to come up with the Opportunity Solution Tree?
It was probably several years of just racking my brain trying to solve a couple of specific problems but it started when I was a product manager.
This was right around the time that The Lean Startup came out, which is really nice because I feel like The Lean Startup put language to what a lot of us were already doing. It gave this idea of assumptions and testing assumptions, which a lot of us were intuitively doing, we just didn’t have language around it.
One of the problems that I was experiencing with the team that I was working with was that we had what we thought was a good solution idea in mind. We ran an experiment and got some feedback that didn’t support the idea. I’d be sitting with a team of engineers trying to figure out how to fix it and people would want to throw the whole idea out.
I’d be like, ‘ok, no, we don’t have to throw the baby out with the bathwater, this is still a good idea we just have this one piece that isn’t quite working and we’ve got to figure out how to evolve the idea to make it better’.
The problem was that I had all this context in my head – all the customer needs in the opportunity space – and we were having a conversation 100% in the solution space. This made it really hard for us to talk about what pieces were working and what pieces weren’t.
As a result, I started trying to map it out using hypothesis one, hypothesis two hypothesis three of which one and two were fine but three needed fixing. But nobody on my team was thinking about hypotheses and so I failed.
I then went and I did a Master’s programme at Northwestern, and I had a class where I had to work with a coach. I had to pick a problem to work on and I said ‘I don’t know how to visualise this in my head and bring a team along’. I probably spent six months, playing with how to do this and I got closer, but iI hadn’t fully described the problem, so I wasn’t really solving it, I was dancing around it.
Two years later I was coaching a team and they came to me one day, really frustrated, and they said ‘Teresa, we’re learning a lot of tactics from you but we have no idea what to do when you always have to tell us what to do next’. That was a little bit heartbreaking because part of the reason why I work as a coach and not as a consultant is that I want to leave a team better off than how I found them. I don’t want them to be dependent on me to do their job. I want to level them up so that I can walk away and move on to the next set of teams to work with.
I then spent a lot of time just thinking about that. Thinking about how to give structure to the discovery work. It was very serendipitous because, at the same time, I was reading Peak by Anders Ericsson which looked at what experts do in comparison to novices. One of the things that he hit on was how experts work from more mature mental representations, internally in their heads. The example he gives is that if you’re a grandmaster chess player, you look at a chessboard and you don’t see a whole bunch of individual pieces, you see a pattern that represents a moment in a game, while a novice is just going to see the individual pieces. That was actually the spark for me to start to think about the mental representation in my head and that’s where The Opportunity Solution Tree emerged.
How do you effectively share discovery results, especially those that disrupt existing plans?
One resource I’ll share right off the bat is a talk that I did at Mind the Product San Francisco, called Show Your Work. It’s all about how to use tools like The Opportunity Solution Tree to bring your stakeholders along.
If your team is using this tool as part of their daily life, it’s going to be messy and before you show it to stakeholders, you’re going to have to clean it up and simplify it. So, the thing to think about when you’re communicating out is who’s your audience, and how much detail do they need?
I think I’m finding a way to bring people along continuously – it’s funny how much everything comes back to ‘continuously’, and not ‘we’re going to give you an update once a quarter, where you’re going to be overwhelmed by all the things I tell you’. Instead, finding an easy way to continuously keep people in the loop makes a big difference because, if you do it once a quarter, you might get derailed on your second bullet point of 50 and the conversation goes astray. Whereas if they’re along with you on the journey, it’s a lot easier.
Also, I work with some big companies where there may be 20-50 plus teams doing discovery, and there’s a question of ‘how do we coordinate our work across each team?’ and ‘how do we share what we’re learning?’.
There’s are two parts to this:
1: A lot of teams do discovery share outs. Not every team is going to share what they’re learning in discovery every week, because that’s a nightmarish meeting, but having a regular cadence of discovery is a really good practice to build internally.
2: Now, everyone uses Slack and there’s this concept of working out loud that I think product teams really should adopt. It’s how you make your work visible continuously so that anybody who wants to can just follow along. There’s a really good book that shows what this looks like in practice, written by Scott Berkun, it’s called The Year Without Pants. He wrote this because he spent a year working for Automattic which is a 100% remote company and he wrote about his experience. They may not even know about this concept of working out loud, but they have an amazing culture of working out loud and it’s a great way to bring people along, to manage stakeholders, and just to make all of your work visible
Then there’s this piece ‘ok, we stumbled on something that’s going to collapse the house of cards that our company is based on’ and I think the theory is to be really careful here. Because again, most of us are working lower in the hierarchy. We have to be respectful of our leaders and why they are, where they are. I think I’m crazy when I hear myself saying this because my 22-year-old self would be like F that. But that is the reality of organisations. You have to think about how you learned something new that your leaders may not have. And it’s going to be hard for them to hear that. The key is, how do you show them what you learned, rather than tell them what you learned, because you’re in a position where you have less power whether we like it or not, and so we have to be diligent about how we show them. I would recommend that you recreate the experience you had when you had that insight, so they can have that insight too.
How do we make a shift from project research to continuous research?
This is actually a really common question and it’s because, if we think about UX and we think about user research, how did it grow up? Mostly in external design agencies – so Adaptive Path, Frog Design, so looking at how those companies work, if I’m a consultant, I’m not going to sell a one week project because the contracting and invoicing is way too much work. I’m going to sell the three-month project. A lot of the reasons why our UX and user research practices work on long horizons is because those practices grew up in agencies and that’s how agencies need to make a living.
Now that we’ve brought those practices into our companies. They don’t need to work that way anymore. Now, what is great about having a centralised user research team is that we do have long horizon questions that deserve research. I work in music streaming entertaining and, as an example, if I work in a streaming entertainment company, somebody in the company better be doing research about cable cutters, because that is a current trend that dramatically impacts your business. And that’s probably not a research question that an individual product team is going to tackle because it’s not it doesn’t affect right now and what they’re doing right now.
So first of all, research needs to happen on different cadences which means your design thinkers and your centralised user research team should be tackling those long horizon and strategic questions. But that shouldn’t be in lieu of what an individual product team needs to get fast answers to questions like ‘what do we call this?’, ‘how should this work?’ and ‘where do we integrate it in the interface?’ – all those day-to-day product questions need fast answers. The first thing I would do is start to have a conversation around types of research questions. We have big research questions. We have medium research questions and we have small research questions. How do we get them all answered and who should be doing what? I don’t threaten the model if you have great resources around long-horizon research questions.
What you’re probably going to run into (because I’ve seen this in every company that has this model) is that you’re going to want the product teams to do their own small research questions, which is what they should be doing, and your centralised user research team is going to say ‘but you don’t know how to research, you’re not going to get reliable feedback’ – they’re the experts, they want to control it.
This is where you say, ‘ok, partner with us, be a subject matter expert for us, give us feedback on our research plans’. But frame the problem as: ‘we have small questions we need answers to – what’s the best way we can do that?’ because there’s no way your centralised research team can support small question research from all your product teams, it’s impossible. That’s the challenge, that you co-create the solution together.
What can you do when your discovery track trail is leading your delivery track?
This is pretty common and this is normal. Some of it can be a symptom of what Marty Cagan calls ‘mini waterfall’, where we actually are still taking a waterfall mindset to our work and just have trains running through on two-week cycles. But some of this is also just normal. Let’s talk a little bit about the difference.
This is where I think that outcome opportunity solution distinction is helpful because often it takes some time to learn about an opportunity before we’re ready to solution. Then, when we’re solutioning, we have additional discovery questions. So if your discovery is around an opportunity while building a solution for a different opportunity, I don’t actually have a challenge for that, that’s part of a normal discovery cadence. If you’re discovering a solution while building another solution, I think that’s where you’re getting a little bit more into this mini waterfall mindset.
Here’s what I would do to fix that: a lot of teams are still taking on too big of a chunk of work as their first iteration. There’s a good example of this in The Lean Startup where Eric Ries talks about his work at IBM – there they built this whole app when really they should have just built download the app because nobody downloaded it.
So, your very first step of understanding if people can see the feature, if they understand it, if they want to use it, and that might be a smoke screen test. Start with that smokescreen test, get some feedback and move on to the next little bit and then the next little bit. That’s what we typically define as an MVP as a v1 – in short, if you’re finding that you’re discovering one solution and building another solution, think about how you can build in smaller increments
You might like: How Uniregistry Used Smoke Testing in Product Validation: A Case Study
How do you include external stakeholders from product discovery?
As you’re doing discovery you want to be as inclusive as you can, but inclusion slows you down, so that’s the trade-off you’re trying to balance – how do I make sure I’m leveraging all the expertise in the building, but not making decisions that take forever.
I think this is where the idea of a trio or triad came from – so we know that for most decisions we want a product manager, a designer and an engineer involved. That’s going to be the core of our discovery team. Depending on the types of decisions you’re making in discovery, you might expand that trio temporarily to include a salesperson, or marketing person, or a business development person. That’s where you’re making a judgement call, but I would think about it from a ‘what type of decision do we need to make this week?’, and ‘what expertise do we want in the room to make the best decision that we can?’ perspective.
What that means is that if you’re integrating with a partner, and you’re doing discovery of how the integration might work, that biz dev person is probably there on the team. If you’re doing discovery around your go-to-market launch strategy that product marketer is probably involved in those decisions.
There’s also the bigger piece of updating everybody else in the organisation about what you’ve been doing.
That was really the subject of my Mind the Product San Francisco talk called Show Your Work, where I think the key is in how you use visuals to show not just your conclusion, but all the thinking that went into it. That means you’re bringing them on a journey to make similar decisions rather than just saying ‘here’s the conclusion’. And I think the key to doing that well is to visualise it and do a little bit of work to synthesise what is the critical piece for this person to know.
How much does technical risk play a role in discovery, especially when you have a lot of technical debt?
This is a meaty topic and I think a lot of factors that come into play here. First of all, I think technical debt is a symptom of a feature factory mindset. That mindset that we’re just going to ship features hope they work. We don’t really build them in a way that makes them easy to maintain and some of them (probably the majority) don’t actually work the way we hope. We’re just going to let them die on the vine and our codebase becomes worse and worse over time. Then we find ourselves in this awful situation of paying down debt.
I think we could avoid this problem altogether if we start to adopt two principles:
1. Build iteratively
By truly building iteratively, I don’t mean this mini waterfall Scrum two week, nonsense – I’m not criticising Scrum, because that’s not the nature of Scrum, but that is what Scrum looks like in practice at a lot of companies. We’ve got to break that model to get better at experimenting within a sprint, and iterating within a sprint.
2. Throw away code
We need to build a really strong mindset and practice of throwing away code. This idea that like every line of code we write is valuable is not true. It’s only valuable if it delivers value to the customer, and so we need to remove features from our platforms and from our interface. We need to refactor so that if we iterate and learn that something’s going to work a different way, and it wasn’t designed to work that way, we need to take the time to refactor it. And frankly – I’m going to take a swing at engineering leadership on this one – I actually think that, just as we need really strong product leaders, we need really strong engineering leaders and, as an industry, we need to develop a practice and a culture around what it means to be a good engineering leader. This question is the number one responsibility of an engineering leader – how do we build a culture of maintaining good code in a way that allows us to support the business? Because when we build tech debt, we’re losing the ability to support the business over time.
What discovery synthesis challenges have you seen for companies going fully remote?
I’ve worked fully remote for seven years, and I’ve worked with a number of companies that have worked fully remote. So, when the whole world went remote I was like ‘yeah, welcome to my world’. I actually think we’re at a point where we have really good tools for this.
Here’s what I will say – if your business culture is to be in meetings all day long, then remote is not going to work, because Zoom fatigue is a real thing. But here’s the deal, if your business culture is to be in meetings all day long, your company wasn’t working in person. Nobody can get work done. Period.
The first thing to tackle, when fully remote, is dramatically reducing the meetings that you do because if your product teams are in meetings all day they’re not talking to customers they’re not doing discovery. I actually think there are some advantages to discovery when fully remote.
Because I coach remotely I have worked with teams using these digital tools while during the ones that were in person, we used stickies on the wall. We had to create photos and then we were working from photos which you can’t edit. We now all work out of a Mural, and people tell me regularly that those tools are better than being in front of a whiteboard because when you’re in front of a whiteboard, not everybody has a pen, not everybody can edit and collaborate at the same time, and not everybody can throw out ideas. You have to take turns. So, in some ways, there are some huge advantages to remote work.
I’m also seeing a lot of industries have better access to remote customers. Instead of just talking to customers that are local, they’re talking to customers around the world.
What are some regular recurring questions I can ask my product managers to see whether or not they’re performing continuous discovery?
I love this question because I actually think there are two questions that every product team should ask in their retrospectives (they’re also great questions that a leader can ask their teams):
- When was the last time you talked to a customer?
- What did we learn this week that surprised us?
If the answer to the second question is nothing, they’re not doing good discovery. Then, for all the things that they did learn / that surprised them ask: How could you have learned that sooner? This is one of the best ways for a team to iterate on their own discovery process.
Recommended Resources
- Show Your Work by Teresa Torres
- The Lean Startup by Eric Ries
- Discovery – Problem vs. Solution by Marty Cagan
- Peak by Anders Ericsson
- The Opportunity Solution Tree
- Product Managers Don’t Own the Problem (And Designers Don’t Own the Solution) by Teresa Torres
- The Year Without Pants by Scott Berkun
- MTP Prioritised: AMA with Scott Berkun
