Prioritisation for product managers: are we doing it right?

November 13, 2020

23 min read

Share on

Imogen Schels

Director of Content Strategy at Mind the Product

Imogen brings more than 10 years of experience in print and digital to her role as Director of Content Strategy for Mind the Product. The core of her experience comes from 10 years working as an editor for Bauer Consumer Media, a division of Europe’s largest privately-owned publishing group, and as Head of Content for Benivo, an award-winning startup.

Frameworks – rated and slated

No product manager will ever be short of options for prioritisation frameworks. However, the list is long and this can leave product managers wondering which ones to try. What’s right for you and your team?

Unfortunately, as much as we might like to, we can’t answer this question for you because frameworks are simply tools in your product management toolbox. The key is knowing when to use which tool and when to tailor a framework to suit – you do not always have to follow them to the letter.

“It’s important to remember that while there are many different options, not all of them are going to be relevant in every stage of your company,” says Mind the Product’s Martin Eriksson. So you should use this article to give you the information you need to make an informed decision

To get an idea of attitudes we asked 55 product managers around the globe to tell us about the frameworks they’ve used. Here are the 5 frameworks most used, along with some useful resources on each:

1. Weighted Scoring Prioritisation: used by 38%

In Weighted Scoring Prioritisation, features or initiatives are scored using a benefit-versus-cost framework based on a number of criteria. You then rank your features/initiatives based on those scores.

Useful resources:

2. Value vs. Complexity Quadrant: used by 34%

This framework comes in the form of a matrix – a simple 2 x 2 grid with value plotted against complexity and is designed to categorise product features by their expected value and their implementation complexity.

Useful resources:

Value vs Complexity – A Prioritization Framework
Enter The Matrix – Lean Prioritisation by Andy Wicks

3. The RICE Framework: used by 29%

In this framework (which is a specific type of Weighted Scoring) RICE stands for four factors – Reach, Impact, Confidence and Effort, that are used to evaluate your ideas (features, new products etc). Each factor receives a numerical score based on the following:

Reach: How many people your idea will impact in a given timeframe
Impact: How much will it impact each individual person (people score this differently. Intercom, for example, uses a tiered scoring system for estimating a project’s impact where 3 = massive impact, 2 = high impact, 1 = medium impact, 0.5 = low impact and 0.25 = minimal impact)
Confidence: How confident you about your estimations where (again from Intercom) 100% = high confidence, 80% = medium confidence and 50% = low confidence
Effort: How much work will the idea will require. This is estimated as a number of ‘person-months’ (the work one team member can do in a month)

Your RICE score is then calculated like this:

Reach x Impact x Confidence = Score

Effort

Useful resources:

RICE: Simple prioritization for product managers by Sean McBride
Management: RICE Scoring Model for Prioritization by Lazaro Ibanez

4. The MoSCoW Method: used by 27%

The MoSCoW method, also known as MoSCoW analysis, stands for must have, should have, could have and won’t have and is designed to help you figure out which requirements to complete first, which can come later and which to exclude entirely. It’s broken down like this:

Must-haves: Non-negotiable requirements that are critical for delivery
Should-haves: Important requirements that are not necessary for delivery
Could-haves: Desirable requirements, also not necessary for delivery
Will not have at this time: The least critical, non-essentials – these are typically dropped or reconsidered at a later date

Useful resources:

5. The Kano Model 18%

The Kano model is a simple two-axis grid, comparing product investment with customer satisfaction.

On the X-axis – feature implementation
On the Y-axis – user satisfaction

The power of the Kano model comes in mapping different features against this simple grid in an attempt to identify which features are most likely to satisfy your customers.

Useful resources:

Choosing a framework

In his article, How to choose your Product Prioritization Framework. Head of Product at Littlepay, Andrew Quan, shares his own framework for choosing frameworks – one he created to help fellow product managers narrow down the list. The article provides an index of the most popular frameworks ordered by complexity and the level of data required for each. Andrew was inspired to write the article when he realised the product managers on his team were all using different frameworks.

“There were various frameworks floating around and we needed to decide which to use collectively as a team. This led us to compare each framework and the pros and cons of each. I ended up with a bunch of notes that I wanted to share with a larger audience, so I simply published them via blog”.”

Andrew’s guide provides a helpful overview of 12 frameworks and we’d urge you to check it out. But Andrew, like us, wants access to more qualitative data around prioritisation frameworks. “During my research, I didn’t see anyone perform a qualitative study with product managers and leaders as to what their experiences were with each framework and why they chose them.”

So, let’s take a look at what our product managers had to say.

Least effective prioritisation framework

The least effective framework (36%) was The MoSCoW method, which was described as being too ‘vague’ and ‘subjective’. “It’s based on business requests without any evidence of customers needs,” says senior Product Manager Afruz Besharati while CPO Mario Lenz explained that The MoSCoW method really didn’t feel like a prioritisation framework at all but rather “a tool for jotting down results”. For senior Product Manager Adrian Lester, the feeling is much the same. “Must, should, etc are subjective depending on which stakeholder or persona you ask! It gathers their priorities but doesn’t help me find up with overall priority.”

Martin Eriksson agrees. “For me, MoSCoW is not a prioritisation tool, but more of a scoping tool,” he says. “It’s not something to use to prioritise your backlog of ideas but it can be great when you get to a point where you know you want to build your MVP or V1 that have already been prioritised.” At this stage, MoSCoW can be used to hone in on the must-haves needed for the build to work. In fact, says Emily Tate, Mind the Product’s Chief of Staff, the “Will not have” is a particularly valuable part of this framework. “The ‘Will not have” states upfront what this ISN’T which in many cases is just as important as what you will be doing.”

The other thing to keep in mind, says Martin, is that to use it correctly, MoSCoW should be used with timeboxing. “This in and of itself is why MoSCoW isn’t a tool to prioritise among things. It can be a tool to prioritise the scope of something, given that you’ve decided to invest, a month in feature X based on a different prioritisation method. Then you can use MoSCow to figure out what to do in that month in order to achieve that.” Ultimately, MoSCoW won’t help you to decide what to prioritise but it can help to force conversations around the things you must have and can let go of.

Most effective prioritisation framework

The framework considered to be the most effective was Weighted Scoring Prioritisation. For 36%, this framework was number one. It was described as the most balanced approach, one that’s easy to understand and apply, one that factors in goals, effort and value and, for many, it was the framework deemed the easiest to communicate to stakeholders and leadership.

“It gives everyone a say and produces a very clear and easy to understand output,” says senior Product Manager Gary Finnigan, while another senior Product Manager says: “It allows for adjustment for what the organisation values through weighting. It provides better visibility on assumptions and deeper conversations.”

For Emily, scoring methods certainly can be very helpful but, she warns, it’s important to be mindful that they can also turn bias into numbers. “If you’re going to go down the scorecard route, it has to be balanced with evidence. You have to be able to answer why you put a number on any of those given things, use the method as a way to provide the basis for conversation and to surface any assumptions and biases,” she says. “Prioritisation – it’s an art, and you can’t math your way into art. Scoring methodologies, although they may appear clear cut, will not magically tell you which one thing to do when.”

For many of the people who voted for Weighted Scoring, the main benefit of the method was how easy it is to communicate to stakeholders and leadership. Says Martin: “I have seen scoring used as a proxy for doing proper stakeholder management, and this is something to be wary of because that’s really abdicating the stakeholder management of that decision and blaming the score, as opposed to having the tough conversations with stakeholders about it.”

Most effective prioritisation framework – Weight Scoring

What about the rest?

Both RICE and Value vs. Complexity Quadrant were described by most who’ve used them as simple, straightforward, and easy to visualise.

“Value vs. Complexity Quadrant is a fairly simple model that is pretty intuitive and effective in determining what will have the most impact and how much it might cost,” says product manager Alex. “It also helps distinguish low-hanging fruit that can provide quick wins for our team and stakeholders.”

Director of Product at XING, Arne Kittler, made the point that really, when it comes to which prioritisation framework to use, it just depends! “I think it’s helpful if product managers have a clear understanding of the context in which they are prioritising,” he says. “Depending on which stakeholders have an interest in the priority and on how controversial it is it can be helpful to apply frameworks to “rationalise” the process and structure reasoning and discussion. In other cases, this may not be necessary.”

There’s no one size fits all

While our results above show a clear preference for certain frameworks in some areas, as Arne suggests, they don’t tell the whole story. Our respondents work in different industries, different countries, have varying levels of experience, and in companies of different sizes and maturity. So, it’s important to keep in mind that what works for one Product Manager and team, might not work for another. Although it would seem that this is an answer some are looking for. In fact, when asked what one question they wanted to answer for them, a number of respondents wanted to know which exact framework would be best for their particular industry.

Says Emily: “The problem is not that we don’t have the right frameworks. The problem is that there’s not a one size fits all framework, and there’s not a framework that’s going to do it for you and so the alternative is for your team to have the hard discussions, and to go through the trial and error of what works best for you.”

Katerina Suchkova, Group Product Manager at 15Five, agrees. “I think of the biggest mistakes people make is reading articles about things like RICE or effort to value, and thinking that just because it worked for someone else, that it will work for you,”

“It’s a mistake to blindly accept everything that’s out there without evaluating if this is the right thing for your organisation.” Katerina admits to having made this mistake early on in her career. “There are so many resources and so many smart people who’ve been there and done it and so you take their opinion, or their perspective, at face value. But, it’s a mistake to apply that to your situation without adding that critical questioning in to see if it’s really the right thing. It’s like we say ‘hey, just give me a quick fix, give me a pill, give me something that’s just going to work so I don’t have to think about anything’ but that’s just not how it works. In fact, that’s not even prioritisation.”

Customisation and updates

Instead, Katerina suggests that we should take advantage of the beauty of frameworks – that we can customise them, assess them, evaluate them, and even build our own. We can consider the frameworks available to us, borrow concepts from them or use those concepts as the foundations of something you can build on.

In addition, she recommends regularly questioning the effectiveness of the framework you use because it might not work for you, your team, your organisation or your product forever. In fact, in light of COVID, she believes it’s more important than ever to keep revisiting how we prioritise, even if it seems like things are working. She suggests you revisit your approach every time there is a change in business strategy and goals, for whatever reason. “Whether this is an external or internal factor, the prioritization framework will need to be re-visited and re-assessed at a high level of a product organisation.”

But Katerina hasn’t always taken this approach herself. “If you’d asked me about this last year, or even a couple of years ago, I would say yes, choose what works for you and stick to it, because it’s a proven model and if it works, that’s great,” she says. “But fast forward to where we are today and I’ve changed my mind. Prioritisation is in a constant state of flux because of the changing environment, and changing priorities and the world is changing so much right now. The impact of COVID has shown us all, especially product people, how flexible we need to be not only in how we build products but also in what goes into making decisions.”

When asked, our product managers revealed that almost half of them prioritise monthly (49%). The other half of the group prioritise weekly (22%) or quarterly (20%), with a very small percentage saying their prioritisation activities take place daily (5%) or even biannually (2%).

“Product managers need to retain the flexibility to reprioritise whenever they learn something new that invalidates earlier assumptions/priorities,” says Martin. “Sometimes that’s daily, sometimes that’s quarterly.” But, he suggests, “if prioritisation only happens annually, or even less frequently, it’s likely you’re not doing enough discovery to invalidate your own assumptions”.

Build Your Own Framework

If like Katerina, you don’t believe in applying a framework out of the box or have tried a number of frameworks without success, there is the option to create your own. In an additional case study, she shares how her team took to build a framework that best suited their organisation, team and products.

Read: Building a Bespoke Prioritisation Framework: A Case Study

Size, scale, and maturity

Other factors that affect how you might prioritise include the size and scale of your company, and the maturity of your products.

Typically, the bigger your organisation and the greater your scale, the more challenging prioritisation will become. Says Emily: “The process becomes more complex as you get multiple products, as you get multiple teams, the more complex the business itself is and, where you have greater dependencies across teams. All of that makes prioritisation harder.” It’s for this reason that communication around prioritisation is all the more vital. At scale, you will need greater clarity in communication around what you’re doing, why you’re doing it, as well as what your different teams are working on and why.

Then there’s the maturity of your product to consider. “Even in a bigger organisation, if you’re trying a completely new product or feature, you will have to invest a little bit more time in figuring out the right direction to go in versus optimising a product that’s been in the market for five or 10 years,” says Martin. “Ultimately, the bigger the bet, the more time you’ll need to have those conversations around prioritisation.”

For tips on how to create a prioritisation process that is appropriate to your business (whatever its stage) and the size of your team, check out Martin’s Product Prioritisation 101. This article suggests how you might:

Match your process to your scale
Know your KPIs before prioritising
Use simple scoring to focus on relative priorities over absolute priorities
Split bugs and small tweaks priorities from the main roadmap
Communicate not just the roadmap but the process behind it

Your biggest prioritisation challenge

When asking product managers about the main challenges of prioritisation, our results showed one crystal-clear theme – stakeholder management and leadership buy-in. It seems that even when we’ve found our framework, the problem of communicating priorities remains a challenge. But, our experts all agree that it’s important to remember that prioritisation is not prioritisation by committee.

As Gabi Buffrem explained in her #mtpcon Digital session Prioritisation for Impact in July 2020, you should not be trying to make stakeholders happy by finding features that work for them. “They are not your users. They are your stakeholders. So prioritisation is not about making all of them happy,” she says. “It’s also not organising a list of features that was given to you by someone else. Prioritisation is a big complex topic that when done correctly, is super impactful.”

Lai-Ping Lai, CPO of tails.com agrees. “The results of the Mind the Product survey suggest to me that people are prioritising so that stakeholders, or the leadership group, feel comfortable enough or buy-in and this way of prioritising is what I’m trying to move against.”

For Lai-Ping, not letting stakeholders rule your process is just the start. She also believes we need to stop focusing on the ‘process’ of prioritisation as we know it. “Focusing too much on the process becomes very ‘other’, rather than ‘me’ and ‘myself’, but I’m the product person here. I make choices for the customers, for the business for X, Y, and Z.”

As for how stakeholders can be convinced? “Well, this is not actually your job,” she says. “Your job as a product person is not to convince other people. Your job is to do the right thing for the business, for the company, for the customers. Asking questions like ‘how do I avoid undue stakeholder influence on priorities?’, that’s us and them again.”

In her opinion, ‘prioritisation’ is now so ingrained in product people, that the process itself becomes the thing we too often focus on. By moving away from our traditional approach, Lai-Ping believes we will be able to do more product, to be closer to our customers, and that we’ll stop getting sucked into prioritisation. “I believe it will prevent us from spending too much time trying to work out whether we’ve prioritised right, whether we’ve used the right framework and from explaining our prioritisation process to person X, Y and Z. I see it taking up a lot of mind space and of literal time and effort.”

In general, she says, doing it better is faster too. “In product, we talk a lot about continuous feedback loops and if you’re spending your time trying to prioritise, you’re not actually spending your time talking to customers, building things, showing your customers things or putting those things out there, getting results and iterating fast.”

Take the impact of COVID, for example. “During the early months of lockdown, we saw how a lot of companies, companies that weren’t purely digital or that didn’t have a direct consumer were not prioritising. A lot of them were like ‘OK I believe I can take advantage of this, right now’,” she says.

“At the start of the pandemic I saw the restaurant, Côte, quickly launched Côte at Home and I wonder, did they actually go through a prioritisation exercise? Did they sit there trying to prioritise whether to enable these meal kits? I imagine they believed there was an opportunity to get them out to the customers and they went for it.”

Discover more content on product prioritisation.

Moving away from frameworks

So how can you put aside the approach you’ve always known? Lai-Ping suggests it’s not as difficult as it might seem. She says:

First, stop leading with your prioritisation framework. Lead with the goals, outcomes and metrics.
Reframe the way you talk about consequences. Instead of saying we need to do this, not that, say that ‘by doing this I will get X and by doing that I will get Y’. Talk about the consequences of doing it and not doing it, rather than the trade-off.

She argues that this is a way of simplifying why you have prioritised something, versus trying to explain why you made a change because you’ve used a prioritisation framework that meant you looked at reach over cost and effort etc. “That in itself takes another 10-15 minutes to explain rather than saying I believe X based on our research, qualitative or quantitative data.”

This approach, says Martin, is certainly one that can work, but it’s one that requires a high trust environment and experience. “One of the outcomes of choosing a prioritisation framework is building trust in the process and perhaps there’s a corollary of how much trust there is within an organisation versus how detailed your prioritisation process needs to be. For example, the less trust there is in your team, and the bigger the organisation, the more likely you are to need a framework to have clarity of why decisions are being made, versus a small super nimble startup that can go more on gut, because they’re trusted to do that and they’ve shown that their gut is right, that it’s based on evidence and research and they’ve been talking to customers, dealing with problems and looking at the data – it’s a more informed gut feel.”

Top tips

It’s only one part of the Product process

Prioritisation is not a single activity. For it to be successful, and for you to reach sound, prioritised decisions, you need to do discovery, conduct user research, analyse data, and have clear business goals too.

It’s good to talk

One of the most valuable things that comes out of the prioritisation processes is the discussions we have with stakeholders. Use them to understand what it is your stakeholders know that you don’t that makes them think the priorities should be different. Also, use those discussions to communicate what you know that they don’t – this helps you to explain why you’ve shaped the priorities as they are.

Beware the silos

Prioritising in a silo is risky. Failure to get people involved, or at the very least, explain your process, is a quick way to ensure you’ll be met with resistance, even if you have a great framework. Get people involved early and often.

Stay Flexible

2020 aside, we live in a world where things change at a rapid pace which means your prioritisation process will benefit from a regular assessment. Whether it’s annually, quarterly, monthly or weekly, take time to consider your process and if it’s working as time moves on.

Give yourself a break

When you’re a product manager, (especially one who’s new to the role), you’re learning how to influence designers and engineers, but perhaps not those above you in leadership roles. The process is hard and as Emily Tate says, “it’s an art”. It takes time to learn, so don’t put too much pressure on yourself to know it all. Ask for help, seek advice, and give things a try.

Now share your ideas

Now that we’ve given you some food for thought we’d love to give you the chance to share your own ideas, opinions, and provide you with an outlet for collaboration. Head to our Prioritisation Discussion Board where you’ll be able to share resources, ideas, and your feedback on prioritisation frameworks. If you’ve not used Mural before, you’ll see tip tools when you enter and here are some quick pointers from us:

You will be asked for an email and name as you enter but you don’t need to provide this, you can simply click ‘Enter as a Visitor’
Once in, right-click to add sticky notes and comments in the different sections
Zoom in and out to see more or less

Go to Discussion Board