Measuring the right North Star metric

“What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so.”
(V. F. Ridgway 1956)

Ridgway clearly was onto something in 1956. Not everything that matters can be measured. Not everything that we can measure matters.

This article focuses on measuring the right things in order to build successful products. Throughout the piece, I’ll discuss different types of metrics and provide an actionable framework that can be used by Product Managers to measure the path from the customer problem to reaching the product vision. I’ll also provide actionable insights into how the execution of the product strategy is performing. It is important to focus on the most relevant metrics in order to be efficient and not get lost in overwhelming data.

First, we examine different types of metrics. ProductPlan defines three tiers of product success metrics, from least to most actionable metrics:

Tier 3: Vanity metrics: These are the vanity metrics that can boost a product team’s ego (e.g. number of views of a promotional banner). Be careful with those and do not put too much emphasis on those ones.

Tier 2: Proxy metrics: These are proxy metrics that can suggest something about your product’s potential success (e.g. number of prospects who take action showing interest in your product, such as downloading a document about it). But because they do not themselves represent direct evidence of how your product will resonate with users, you need to be careful about these metrics.

Tier 1: Business and customer-oriented metrics: These are the business and customer-oriented metrics you should focus on. They capture concrete data about revenue, customer retention, acquisition, the size of your user base, etc.

What is a North Star Metric?

The North Star metric should provide evidence of a clear and direct relationship between your product’s problem and the degree to which it is solving it for your target market.

It is the single metric your team or company decides will be the one that best represents how well your product is performing in the market.

A north star metric has usually the following characteristics:

• Expresses value
• Represents vision & strategy
• A leading indicator
• Actionable
• Quantifiable

Usually, the north star metric is measured by certain leading indicators which might be tier 2 and tier 3 metrics. Listen to this podcast to find out when to not use the North Star metric. 

The North Star framework

So how do I actually define the right north star metric? You have to focus on the customer problem you want to solve and the product vision, so to say the envisioned solution for the customer problem. The product strategy describes the path to reach the product vision. The north star metric measures the degree to which the customer’s problem is solved by your product. This metric is the core metric showing your progress on how successful your product performs.

Your north star metric must be derived from a true understanding of what action provides realized value to the customer. This means that “Daily Active Users” or “Registered Users” are not good metrics as they belong to the tier 3 metrics (vanity metrics). They do not give you any meaningful information about what your customers value about your product. When product teams fail to establish a direct connection between customer value and the north star metric, they risk leading their business along the wrong alley.

Watch this MTP Panel on defining what metrics matter. 

The north star metric is usually measured as a multiplication of several leading indicators, most times a combination of reach, engagement and frequency:

Reach: Number of relevant users

Engagement: A certain action

Frequency: Number of repetitions of the action taken

The North Star framework applied to Netflix

Let’s have a look at what the North Star Framework could look like for Netflix. The product vision of Netflix might be something like: “Entertain the world”. The product strategy might be summarized and simplified as: “Netflix is a movie subscription service that delivers fast, easy entertainment in a friendly, straightforward way.”

So how can I measure if Netflix is “entertaining the world”? One possible north star metric might be to focus on how much users engage with the product, the streaming service. So the north star metric can be “Time spent watching streaming content”. This metric measures the customer value and is aligned with the product strategy.

The north star metric example could be measured by:

< “The number of users”> x <”hours watched per session”> x <”sessions per week>”

With this north star metric in mind and some leading indicators defined I can start to run different experiments. For instance, I could run an A/B test with different personalized TV show recommendation algorithms to examine if the one algorithm increases <”sessions per week>”. If there is a positive significant effect it directly influences the north star metric. Thus, this framework provides actionable insights to test hypotheses by experimenting and focussing on the most relevant metrics.

What makes a good North Star Metric

Amplitude states that a good north star has three attributes:

1. it measures the moment that a customer finds value from your product,
2. it represents the core of your current product strategy and
3. is a leading (not lagging) indicator of a future business outcome that your company cares about.

Here you can see some exemplary potential north star metrics for known products:

Wrapping up

To sum up, it is important to have a good product vision defined in order to build your specific North Star Framework. The north star metric needs to be selected in a way that it provides evidence of a clear and direct relationship between your product’s problem and the degree to which it is solving it for your target market, representing the core of your product strategy..

There’s more where that came from! Access more great content on KPIs

• 8 KPIs that every product team should measure
• Metrics and KPIs every SaaS product manager should track
• Essential KPIs for data-driven product managers