post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Guest Post

Creating metrics that matter

A case study: Launching a product process in a project-based company

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

How much are product managers getting paid in 2024?

Leading product ops at Delivery Hero with Koen Dubelaar

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

“You’re not as competent as you think” 5 lessons from Ken Chin on navigating a product career (CPO, Spark Networks)

Everything wrong with the product management job market. And how to fix it

Rob Boyett is Product Design Director at Idean UK (formerly Adaptive Lab). He's been working in digital product for over a decade, predominantly in the mobile space pre and post the smartphone revolution. His work spans across a wide set of sectors, including technology, fitness, cultural institutions, hospitality, home delivery and finance. Rob helped Sony Ericsson, The British Museum, Fitness First, Magic Leap design, build and ship great digital products. He also led the team that won the British Journalism Innovation of the year award for the iOS app Guardian Witness.

Rob Boyett

Hello product designers, this is for you. I want to talk to you about product goals, metrics, and how they get muddled in the product design process, leading to some less than humane outcomes.
But first a little story.
In the early 1970s, two behavioural scientists working at Princeton University set out to investigate the psychology of prosocial behaviour. Their names were John Darley and Daniel Batson, and they created the <a href="https://www.psychologytoday.com/gb/blog/darwins-subterranean-world/201703/my-favorite-psychology-study" target="_blank" rel="noopener noreferrer">Good Samaritan study</a> to ask a seemingly simple question: why do people do good things for others?
The method was to test two groups. One that defined themselves as having Christian beliefs, and another with no religious beliefs. Each group was read the parable of the Good Samaritan – the story of a Samaritan coming to the aid of a distressed traveller after others have ignored their plight. The participants were then asked to move to another building to complete the test. Along the way, an obviously in-distress person was strategically placed slumped in an alleyway. This was the real test: would they stop to assist this person or keep on walking?
Interestingly, the only factor that affected their chance of stopping was if they perceived themselves to be in a rush or not. Those in a rush kept going, irrespective of how religious they were. Whereas those with time to spare were highly likely to stop. Again, how religious they reported being before the test had no bearing on the outcome.
What does that have to do with goals, metrics, and even design? What can we learn from this? Well, this study implies that when the heat is on, we&#8217;re highly likely to ignore issues in the periphery – things that are not directly related to the task at hand – even if they’re in direct conflict with our moral code. It seems we don’t have the bandwidth for it, and this somehow galvanises a belief that the mission we’re on gives us the right to ignore.
So let&#8217;s talk about how this issue can manifest in the product design process, and how we might become more robust in the face of pressures &#8211; like the ones that make you walk past people who are clearly in distress.
So, product goals and metrics – the tools that allow you to build design foundations and shape a strategy. These often-confused bedfellows can set you up for problems later on, because metrics are not goals. You know that, right? Tech entrepreneur Avichal Garg puts this very well in his aptly named piece <a href="https://avichal.com/2017/07/06/metrics/" target="_blank" rel="noopener noreferrer">Metrics</a>: &#8220;The biggest risk in creating a metrics-informed culture is that over time, people conflate metrics and goals.” Avichal goes on to say: “If you lose sight of the value you are ultimately creating, you can move metrics for the sake of moving metrics.”
For clarity, goals can be summarised as an aspiration to create real value for customers. A metric is a proxy for that value, an abstraction that allows you to track progress toward a goal.
If you lose sight of the goal and instead focus on the metrics, you’ll very quickly forget about people. Goals are the guiding force design needs to stay true to design’s purpose of solving for real human needs.
If a goal is Boolean, it promotes an &#8220;at all costs&#8221; way of thinking. This means that if a product goal is actually a metric in disguise, for example to “increase time on site”, the goal is seemingly clear. As the pressure to improve the goal mounts, any and all tactics become justified. The ends justify the means. But we all know this is wrong, and in the end, it can sink a good product.
Today&#8217;s product design process is just too good at doing whatever it sets out to do. Currently, products have a massively unfair advantage over customers. We’ve become very good at applying a scientific method at the front, middle, and back of product design, revealing predictable human patterns in detail. These lakes of data have been filtered for insights and fed into learning algorithmic decision machines. It&#8217;s bonkers. We have the ability to redeploy updated versions of our design at lightning speed, and we can test variants of our design in the wild at massive scales. It’s evolution on rocket fuel. Shooting fish in a barrel falls short as an adequate analogy. Designers have laser pistols while users have pointy sticks.
No doubt you’ll be aware how this has played out as features on social media platforms like Facebook and Twitter. How their metrics-driven view has created obscene advertising growth, and as a result, an attention economy littered with shady practices that can be traced back to what I assume are some reasonable humans making these decisions, but who were just following metrics instead of goals. If you want more on this, then read <a href="https://www.amazon.co.uk/Ruined-Design-Designers-Destroyed-World-ebook/dp/B07PS16XY9" target="_blank" rel="noopener noreferrer">Mike Monteiro’s Ruined By Design</a>.
<h3>Positive Examples of Product Goals</h3>
How does design stay true to real product goals? By finding a system of checks and balances that encourage better behaviour. How that happens can be as unique as the teams building the product.
As a rule, companies don’t tend to share their product goals publicly – perhaps due to worries about loss of competitive advantage, or negative customer reactions. But there are lots of upsides from being transparent with customers. And there’s one very valuable side effect: your customers can use your goals to hold you accountable. That’s a very powerful team motivator.
One company that <a href="https://monzo.com/blog/2018/07/10/making-quarterly-goals-public/" target="_blank" rel="noopener noreferrer">shares its product goals is Monzo</a>. It uses extreme transparency with customers to drive standards up and hold itself in check: “By sharing our quarterly goals, we can be totally transparent and hold ourselves accountable to you.”
When Monzo started this experiment of sharing product goals back in 2018, it had three major goals:
<ol>
<li>Grow!</li>
<li>Increase revenue and decrease costs</li>
<li>Make Monzo the best account to use as your primary account</li>
</ol>
Goals one and two are not surprising. Growth and money are the lifeblood of startups. But, these are exactly the kinds of goals that run the risk of hard metrics sneaking in, and in turn, lead to bad design choices. Monzo counters that with the third goal, acting as the balance, a focused customer-experience goal.
Another organisation that&#8217;s operating at this level of transparency is the <a href="https://insidegovuk.blog.gov.uk/2017/02/13/the-2017-to-2018-gov-uk-roadmap/" target="_blank" rel="noopener noreferrer">GDS (Government Digital Service)</a> inside the UK government. It&#8217;s been publishing a blog post for the last few years that details its goals. It started with a set of big hitters, like “make it easier for the public to hold government to account” (I love that one). This small set of bigger goals has an accompanying breakdown that’s descriptive while being open enough to allow teams to explore and not fall into the &#8220;increase X by Y%&#8221; problem when metrics masquerade as goals.
<h3>What can you do?</h3>
You might not be ready to splash your goals on the company blog, so I have a couple of light tips that can be done today. The hope is that doing the work today will keep you on target later:
<ol>
<li>Do the work to unpick any metrics out of goals. Use your goals and metrics as intended; goals for setting a destination, metrics to show you are heading the right way.</li>
<li>Make sure the goals are understood and easily accessed by everyone working on the product. While you’re talking to them, let them know whistleblowers are welcome. If people in your team feel something is wrong they should feel safe to raise a hand.</li>
</ol>
How we set ourselves up for those high-pressure times is hugely important. Being conscious that the blinkers will descend as the pressure rises is a great start, and hopefully affords some protection from the risk of misguided motivation.

core/freeform

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

How to fix Your Product Goals for Better Human Outcomes

Rob Boyett

Read more

Comments