post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

AI governance

Perplexity lawsuit sends warning to product managers

Guest Post

The Irony of Product-led Growth: How Adding More Can Slow You Down

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

Why teams often struggle with OKRs: A conversation with Jeff Gothelf

Lessons from my first year as a Prod Ops Manager

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

5 ways to invent and simplify as an AI/ML product manager

Kieran Kilbride-Singh

In my experience, to succeed at data-informed product management you need to master &#8220;double-think&#8221; and be comfortable with two conflicting statements: &#8220;Be data-driven&#8221; and &#8220;don&#8217;t trust your data&#8221;.
Blindly following data can lead to some bad things. Analytics products aren&#8217;t perfect &#8211; sometimes data capture gets messed up, and slight changes to code can cause all sorts of anomalies. It pays to be skeptical about your data points.
But data is important. We (<a href="http://weared4.com">D4</a>) prefer the term &#8220;<a href="https://en.wikipedia.org/wiki/Data-informed_decision-making">data informed</a>&#8220;, which, for product management, we see as:
“Being data driven” + “a deep mistrust of data” = “data-informed decision making”
For us, there are three key aspects of data-informed decision making:
<ul>
<li>Mining the raw data in front of you for patterns and trends, anomalies and fluctuations, and anything just plain weird </li>
<li>Never trust what you&#8217;ve mined &#8211; data doesn&#8217;t live in a bubble so compare it to some more data and apply the context of what&#8217;s been happening with your product, customers, the weather&#8230;</li>
<li>Based on the contextualized mined data, read between the lines (or data points) to validate your findings and develop a testable hypothesis to further validate</li>
</ul>
Doing just this with one of our products, we raised prices and won new customers.
<h2>Mine the raw Data (let the Data Guide you)</h2>
Our product SQLizer helps developers to convert files from various formats into SQL. It&#8217;s free up to a certain file size. If someone tries to convert a file that is too big, then they hit a paywall.
Looking through our analytics, we noticed a large number of people hitting the paywall but a relatively low conversion rate. People clearly had a need for the product because they were trying to use it. Maybe the landing page wasn&#8217;t right for the audience? Perhaps they were put off by the price? Could they just be testing some data?
Despite the supposed evidence to the contrary (never trust your data), we were fairly confident that the landing page was hitting the mark and that our prices were already on the lower end of the spectrum. So before overhauling the design and slashing prices we decided to look into some more data, this time on our current customers. It turned out we were getting a lot of new users each month for our monthly subscription but the churn rate was very high. What was going on?
<h2>Contextualize Your Data and a Testable Hypothesis (Know Your Customer&#8217;s Problem)</h2>
The problem our product solves is uniform for our audience: they all need to convert a file to SQL. But the problem differs in its regularity. Some developers work with databases all the time and have a regular need to migrate data. Other developers only need to transfer data every so often.
Based on data in our analytics we hypothesized that conversion rates were low and churn so high because not everyone needed to convert files throughout the month, they might only need to do this once every couple of months.
The data seemed to tell us that people were willing to pay the current price, even for short-term use. Maybe the inconvenience of signing up and then needing to cancel was one reason for such a low conversion rate?
We decided to test a new pricing structure:
<ul>
<li>24-hour pass (same price as monthly subscription fee)</li>
<li>Monthly subscription (50% price increase on previous fee but grandfathering in existing users)</li>
<li>Annual subscription (save 20% on month-on-month subscription)</li>
</ul>
An added benefit of a 24-hour pass is that it acts as a relatively cost-effective trial for people who may go on to sign up for a monthly subscription.
Just a couple of hours after the changes went live someone purchased a 24-hour pass. Later that day, someone else purchased a monthly subscription.
The first month of the new pricing structure was the best we&#8217;d ever had for new payments. Every month since has held this record.
Over the next three months our 24-hour pass usage continued to grow. This cannibalized monthly subscriptions somewhat (which is what we expected) but, despite overall lower numbers, month-on-month monthly subscriptions continued to grow and the churn rate fell dramatically.
Success!
<h2>Data as a Primer for Customer Emotion</h2>
Reading between the data points in our analytics helped us discover a new way people derived value from our product. While this new segment of users isn&#8217;t one that boosts overall retention, the product was attracting a group of people with this need. Sifting through our data allowed us to capitalize on this. This was a data-informed decision based on the three points of:
<ul>
<li>Mining raw data</li>
<li>Being skeptical</li>
<li>Developing a testable hypothesis</li>
</ul>
While this was a data-informed decision, data can only get you so far. We needed to drill down into the human emotions behind the data. All purchases (even B2B) are made on an emotional level (thanks <a href="https://en.wikipedia.org/wiki/Basal_ganglia">lizard brain</a>!). Data can only guide you so far &#8211; you need to <a href="https://medium.com/microsoft-design/if-you-want-to-be-creative-dont-be-data-driven-55db74078eda">get creative</a> if you want to find out (or at least hypothesize) about true intentions behind data points.
Unfortunately, analytics tools are limited in their ability to tell us exactly what people want. Even with great tech like video playback, as product managers we still need to practise the dark arts of delving into the minds of our customers.
Until some potentially-questionable analytics product is developed that can read the emotional intention of humans, you&#8217;ll need to rely on reading between the data points.
<ul>
<li>Mine raw analytics data for patterns and trends, anomalies and fluctuations, and anything just plain weird </li>
<li>But be skeptical and don&#8217;t trust what the initial data tells you. Cross-reference with more data and try to apply the context of what&#8217;s been happening with your product, customers, the weather&#8230;</li>
<li>Let the data guide your thought process but be creative &#8211; just because it sounds farfetched doesn&#8217;t mean it&#8217;s not right</li>
<li>Come up with a hypothesis and test that hypothesis &#8211; but be careful to only test one hypothesis per test so you can safely attribute results to the right hypothesis</li>
</ul>
If your tests are wrong, at least you now know that that particular angle is wrong and it was guided by data. You are now closer to 100% product insight that you were before. Congratulations!

core/freeform

How to lead data-centric product teams: Jon Mora (Chief AI Officer, Zefr)

How to be a better data-driven product manager

User insights for product decisions by Cris Valerio

Data scaling for startups by Crystal Widjaja

Make Better Product Decisions by Using Quantitative and Qualitative Data, by Brian Tran

Using Analytics to Raise Prices and win new Customers

Kieran Kilbride-Singh

Read more

Comments