In this ProductTank New York talk, Tomer Sharon, Managing Director, Head of User Research & Metrics at Goldman Sachs, introduces us to Key Experience Indicators (KEIs).
The key points of his talk include:
- Framing
- Principles
- The Google HEART framework
- Actions to take
Watch the video to see Tomer’s talk in full. Or read on for an overview of his key points.
Framing
KEIs are quantitative scores for specific, important, and actionable phenomena related to using products or services. They are a type of user research method used after a product has launched and can be useful in helping teams move from launching products to landing products. In landing products, teams can find out if the product was successful, if users liked it and how they can go about getting repeat users.
Principles
The principles of using KEIs are as follows:
- Specific vs overall. Measure very specific things vs the overall satisfaction or engagement of a product. Try to hone in on things related to the critical user journey, especially when starting out.
- In context vs out of context. Focus on finding the right people at the right time and in the right place.
- Core vs perimeter. Measure only your core experiences and not everything. In many cases, this would be three but for other companies, there could be a lot more.
Google HEART Framework
The Google HEART framework focuses on the areas of happiness, engagement, adoption, retention and task success. Tomer uses this framework to provide some key metrics that product people should focus on and some mistakes that are made in measuring them.
Happiness
Metrics to measure: the average satisfaction score and actual net promoter score. Learn how satisfied customers are with a product or if they recommended the product to a person in the last week, as opposed to at all like the NPS metric typically asks.
Some of the mistakes made: are not being specific, being out of context or overcomplicating things. Happiness cannot be explained using data, it only gives some guidelines. To get further clarification, product people need to go out and talk to customers.
Engagement
Metrics to measure: percentage of people to use the product at least once in a particular time frame, actions per user and time between action. Product people need to choose the right event to determine how engaged people are with a product.
Some mistakes made are: measuring everything and not being specific enough or assuming that engagement is different for B2B compared to B2C customers.
Adoption
Metrics to measure: adoption rate and time to first customer. Adoption can tell you how fast people are adopting a product or feature. It can provide insight into marketing efforts.
Mistakes typically made: are only measuring an overall adoption rate or confusing it with engagement.
Retention
Metrics to measure: retention rate, churn rate. Understand how many users are being retained.
Mistakes made include: choosing the wrong time frame to measure or the wrong event.
Task Success
Metrics to measure: success rate, time on task, lostness. Anything within a product that has a clear start and endpoint you can measure task success.
Mistakes made include: ignoring this measurement completely or asking customers about the success rate rather than measuring it.
Actions
Actions that product teams can take are to set goals for KEIs and use them to help evaluate planned changes. The key takeaway from this talk is that product managers need to measure experience to truly understand how a product is doing.
