post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

Sunday Rewind: Whose job is it anyway? The rise of Product Ops and why it deserves to be an independent function

Guest Post

Creating metrics that matter

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

How much are product managers getting paid in 2024?

Leading product ops at Delivery Hero with Koen Dubelaar

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

“You’re not as competent as you think” 5 lessons from Ken Chin on navigating a product career (CPO, Spark Networks)

Everything wrong with the product management job market. And how to fix it

Jock Busuttil (jockbusuttil.com) is a product management and leadership coach, product leader and author. He has spent over two decades working with technology companies to improve their product management practices, from startups to multinationals. In 2012 Jock founded Product People Limited (productpeo.pl), which provides product management consultancy, coaching and training. Its clients include BBC, University of Cambridge, Ometria, Prolific and the UK’s Ministry of Justice and Government Digital Service (GDS). Jock holds a master’s degree in Classics from the University of Cambridge. He is the author of the popular book The Practitioner’s Guide To Product Management (jockbusuttil.com/#book), which was published in January 2015 by Grand Central Publishing in the US and Piatkus in the UK. He writes the blog I Manage Products (imanageproducts.com) and weekly product management newsletter PRODUCTHEAD (producthead.news). You can find him on Twitter (@jockbu) and LinkedIn.

Freelance Head of Product, Coach, Author & Speaker

Jock Busuttil

If you’d have told me that August’s <a title="ProductTank" href="http://www.producttank.com/">ProductTank</a> London on analytics and decision-making would be one of the most risqué to-date, I’d have struggled to believe you. The talks had everything &#8211; sex, drugs and racy pictures of hot plumbing action. Read on for an 18-rated recap.

<h2>Making Data Actionable</h2>
<a href="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry.jpg"><img loading="lazy" decoding="async" class="alignright size-medium wp-image-5796" src="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-300x200.jpg" alt="Matty Curry" width="300" height="200" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-300x200.jpg 300w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-1024x683.jpg 1024w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry.jpg 2048w" sizes="(max-width: 300px) 100vw, 300px" /></a>Matty Curry (<a title="Matty Curry on Twitter" href="https://twitter.com/mattycurry">@mattycurry</a>) opened with stark warnings (or promises?) of rude words and ruder pictures. As the director of e-commerce for Lovehoney, one of the largest online purveyors of sex toys, we were obliged to believe him. But hey, it was a warm Friday night in August and everyone had a drink in hand, so the audience was in the mood.
From the slightly incongruous and isolated setting of Bath, Lovehoney’s small and agile team tends to do things for themselves. (Team size isn’t everything. It’s what you do with it that counts. &#8211; Ed.) When there’s a problem with the plumbing, they fix it themselves. Their philosophy of self-reliance means they make the occasional mistake &#8211; Matty once accidentally set every product’s price to £0.01 prompting what can only be described as ‘a bit of a rush’. But when they make mistakes, they learn from them.
<figure id="attachment_5792" aria-describedby="caption-attachment-5792" style="width: 1024px" class="wp-caption aligncenter"><a href="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Drinks-at-ProductTank-August-2014.jpg"><img loading="lazy" decoding="async" class="size-large wp-image-5792" src="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Drinks-at-ProductTank-August-2014-1024x683.jpg" alt="A drink or two was needed to prepare for Matty's talk" width="1024" height="683" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Drinks-at-ProductTank-August-2014-1024x683.jpg 1024w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Drinks-at-ProductTank-August-2014-300x200.jpg 300w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Drinks-at-ProductTank-August-2014.jpg 2048w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><figcaption id="caption-attachment-5792" class="wp-caption-text">We needed a drink or two to prepare for Matty&#8217;s talk</figcaption></figure>
The first thing they’d learned was that attempting to report on absolutely every possible metric was a lesson in futility. They had the vain hope that they might discover the magic bullet for their business, but in their search they were paralysing themselves.
The second thing they learned was that real-time analytics was essentially statistical pornography &#8211; and not even good quality porn at that. It goes on forever and doesn’t really lead anywhere.
After each metrics review meeting, they would return to their desks and would continue to do exactly what they’d been doing before. They were not learning anything. Instead, they needed to find the data that would cause them to take action and try something different.
The next thing they learned was that there was no point in reporting on high-level KPIs such as conversion rate. If they wanted to up the conversion rate, they could have a sale. The rate would rise, but it wouldn’t necessarily make any more money &#8211; see Matty’s accidental £0.01 fire sale for details.
So instead they needed to drill down into the detail, to find the human behaviour and emotions driving each metric.
When examining view-to-buy rate, they needed to strip out the misleading information that was clouding the real story. Out-of-stock products or fractured products (where 80 percent of the product varieties are out of stock) will have a zero or low view-to-buy rate because people can’t buy them. They also cleared out from the report in-stock but unviewed items. The merchandising team could easily generate some interest in these by sending out samples for review.
With all this dross data out of the way, they could concentrate on products with a low view-to-buy rate. Was it a problem with the content? Do people look at the pictures or the videos showing how to use the sex toy in the first place?
<figure id="attachment_5793" aria-describedby="caption-attachment-5793" style="width: 1024px" class="wp-caption aligncenter"><a href="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-Presenting.jpg"><img loading="lazy" decoding="async" class="size-large wp-image-5793" src="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-Presenting-1024x683.jpg" alt="Gosh" width="1024" height="683" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-Presenting-1024x683.jpg 1024w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-Presenting-300x200.jpg 300w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Matty-Curry-Presenting.jpg 2048w" sizes="(max-width: 1024px) 100vw, 1024px" /></a><figcaption id="caption-attachment-5793" class="wp-caption-text">Gosh</figcaption></figure>
They were also able to identify a segment they euphemistically call ‘tyre-kickers’, namely people who are only there for the provocative pictures and videos, and will never actually buy anything. They segmented these out based on their search terms, such as those specifically looking for “sexy lingerie pictures”.
Examining their on-site search also helped to reveal intention. If a customer searches for something, then refines the keywords, chances are they’re not sure what they’re looking for. Could the search algorithm be improved to help them?
Similarly searches that return no results can reveal valuable trends. One such insight led Lovehoney to become the first UK shop other than Amazon to sell E.L. James’s 50 Shades of Grey, and from that won the deal to be the first store to sell 50 Shades-branded products.
People come to Lovehoney’s site for different reasons, some to learn as well as to buy. As content has different purposes, content KPIs should reflect that also. Has the content prompted the customer to fulfil the desired action on that page?
<ul>
<li>Don’t bother trying to report on all of the things</li>
<li>Avoid metrics porn &#8211; focus on what will prompt you to do things differently as a result</li>
<li>Learn the human behaviours behind your metrics</li>
<li>Clear out the dross results hiding the real story</li>
<li>Search behaviour can reveal customer’s intent</li>
<li>Don’t treat all your content the same</li>
</ul>
<h2>Analytics for Non-Retail Environments</h2>
<a href="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Alec-Cochrane.jpg"><img loading="lazy" decoding="async" class="alignright size-medium wp-image-5791" src="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Alec-Cochrane-300x200.jpg" alt="Alec Cochrane" width="300" height="200" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Alec-Cochrane-300x200.jpg 300w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Alec-Cochrane-1024x683.jpg 1024w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Alec-Cochrane.jpg 2048w" sizes="(max-width: 300px) 100vw, 300px" /></a>Digital analytics consultant Alec Cochrane (<a title="Alec Cochrane on Twitter" href="https://twitter.com/whencanistop">@whencanistop</a>) had the unenviable task of following Matty’s talk. Through some case studies of clients he’s worked with, he spoke about how to measure the outcome of transactions when they occur somewhere other than your own site.
He started by recapping the analytics framework he uses with clients.
<ul>
<li>Strategic objective
<ul>
<li>Business objectives
<ul>
<li>Critical success factors (CSFs)
<ul>
<li>Key performance indicators (KPIs)
<ul>
<li>KPI targets
<ul>
<li>Segments</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
</li>
</ul>
One pharmaceutical company had hired an agency to create content to achieve two objectives:
<ol>
<li>to tell everyone about how to diagnose and prescribe for a particular condition;</li>
<li>to distinguish this condition from a similar one.</li>
</ol>
However, they’d only see tangible success if prescriptions for their drug to treat that condition rose. Using this as a basis, they were able to define KPIs that would show whether the agency was writing effective content.
Alec mentioned other case studies:
<ul>
<li>a company moving away from online advertising to lead generation, for which it hosts branded content on its site as a means of driving leads to the companies providing the branded content;</li>
<li>a pharmaceutical company that answers questions from medical professional such as whether it’s safe to prescribe the company’s drug with another; and</li>
<li>another pharmaceutical company that needed to provide support for patients taking its drug because initial unpleasant side-effects were discouraging patients from continuing their course of medication before they saw the benefits.</li>
</ul>
In order to try and measure outcomes when they happen elsewhere, Alec suggested a few approaches:
<ol>
<li>Can you connect different data sources? For example sale conversions from one system and financial transactions from another system.</li>
<li>Intent surveys. If the desired outcome for some website content was to encourage the patient to see a doctor, then ask half the visitors how likely they would be to go to the doctor at the beginning of the journey, and the other half at the end. You could then measure whether there was an increase in intent from those who had seen the content over those who hadn’t, although this still wouldn’t tell you whether they had in fact gone to see the doctor in the end.</li>
<li>Econometric studies. These are expensive but look at minute changes over time, and can highlight where to spend money and what effect it has.</li>
</ol>
<h2>Thanks</h2>
<a href="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Peter-ONeill.jpg"><img loading="lazy" decoding="async" class="alignright size-medium wp-image-5794" src="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Peter-ONeill-300x200.jpg" alt="Peter O'Neill" width="300" height="200" srcset="https://www.mindtheproduct.com/wp-content/uploads/2014/10/Peter-ONeill-300x200.jpg 300w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Peter-ONeill-1024x683.jpg 1024w, https://www.mindtheproduct.com/wp-content/uploads/2014/10/Peter-ONeill.jpg 2048w" sizes="(max-width: 300px) 100vw, 300px" /></a>August’s talks were curated by Peter O’Neill (<a title="Peter O'Neill on Twitter" href="https://twitter.com/peter_oneill">@peter_oneill</a>) who also gave a brief <del>off-the-wrist</del> off-the-cuff (That’s enough now &#8211; Ed.) summary of analytics essentials. Drinks were kindly sponsored by Tesco.
Do join us for our next <a title="ProductTank" href="http://www.producttank.com/">ProductTank London</a>. <a title="ProductTank on Meetup" href="http://www.meetup.com/ProductTank/" target="_blank">Tickets</a> will be available shortly (join the waitlist if none are currently available).
You can also get in touch with us if you’re interested in <a href="http://www.meetup.com/ProductTank/suggestion/" target="_blank">speaking at or curating a ProductTank</a>, <a title="Guest Post Guidelines" href="https://www.mindtheproduct.com/guest-post-guidelines/" target="_blank">writing for our blog</a> on Mind The Product, or <a title="Sponsor a Product Managers Event" href="https://www.mindtheproduct.com/sponsor-a-product-managers-event/" target="_blank">sponsoring our events</a>. See you soon!

core/freeform

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Analytics and Decision-Making – Recap of August’s ProductTank London

Jock Busuttil

Read more

Comments