post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Product Management

Sunday Rewind: Whose job is it anyway? The rise of Product Ops and why it deserves to be an independent function

Guest Post

Creating metrics that matter

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

How much are product managers getting paid in 2024?

Leading product ops at Delivery Hero with Koen Dubelaar

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

“You’re not as competent as you think” 5 lessons from Ken Chin on navigating a product career (CPO, Spark Networks)

Everything wrong with the product management job market. And how to fix it

Alexey Kutsenko is CMO at <a href="http://ddi-dev.com">DDI Development</a> company, that provides custom web &amp; development services to clients worldwide. Alexey has vast experience in building marketing strategies for various clients, defining the client’s needs, and leading digital marketing strategies in the company.

Alexey Kutsenko

Currently, there are thousands of products, apps, and services driven by machine learning (ML) that we use every day. As was reported by Crunchbase, in 2019 there were <a href="https://www.forbes.com/sites/louiscolumbus/2019/05/27/25-machine-learning-startups-to-watch-in-2019/#2af22a7b3c0b">8,705 companies and startups</a> that rely on this technology. According to PWC’s research, it’s predicted that ML and AI technologies will contribute about <a href="https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html">$15.7 trillion to global GDP</a> by 2030.
It&#8217;s obvious that ML is reshaping the software industry, along with digital product management, but it’s not clear what product managers need to know about ML. What skills do you need, and what do you need to understand to become a successful ML product manager? Let’s take a look at what ML actually involves, and then we can highlight six key lessons you should bear in mind to succeed.
<h2>What is Machine Learning?</h2>
First of all, let’s be clear that Machine Learning is distinct from Artificial Intelligence (AI). Although the two terms are occasionally &#8211; incorrectly &#8211; used interchangeably, AI is a very broad term, covering most efforts to make computers “smarter”. <a href="https://medium.com/towards-artificial-intelligence/differences-between-ai-and-machine-learning-and-why-it-matters-1255b182fc6">ML is a subset of AI, and specifically refers to enabling machines to adapt (“learn”) for themselves</a>.
It&#8217;s the study behind the structuring and classifying of data, and the design of dynamic feedback loops as a way to make predictions and/or decisions without being explicitly programmed to do so.  At its core, it’s designing computer algorithms that can evolve automatically through “experience” (new, incoming data) just like humans.
Thus, understanding ML requires understanding the nuances of statistics and datasets &#8211; in particular, the “training data” that is used to set up the initial systems &#8211; as well as the feedback processes that software uses to adjust it’s decision models.
<h2>The Role of a Machine Learning Product Manager</h2>
Being a Machine Learning Product Manager means to have the same responsibilities and skills as a traditional product manager, but with a very good understanding of maths and statistics. And, crucially, the limits of what adaptive software systems can achieve.
ML means expanding your responsibilities and including the following aspects:
<ul>
<li>Problem mapping is a key skill for any product management to develop, but it’s especially relevant with ML. Although the potential of the technology is fascinating, certain problems are still incredibly hard to solve with ML (for example, because of the structure or structure of data) , and it&#8217;s relatively easy to fall into the trap of trying to solve all data problems with ML, when you have the technology available. Thus ML PMs should remain user-centric and keep the focus on customer needs, and what ML is actually well-placed to help with.</li>
<li>Data literacy. Besides the common stakeholders and teammates, data engineers as well as data scientists join the team. Machine learning product managers must provide ML-literate specifications, ask the right questions about data, and understand what is and isn’t feasible with the available data. If your aim is to be an ML expert in product, you need to be an expert in this skillset.</li>
<li>Communication is always key to product manager success. This skill is not unique for ML products or teams, but it does take on a new importance. Not only should product managers be fantastic communicators in general, but it’s also imperative that they be able to bridge the gap between product development and data science.</li>
<li>Explainability is vital for building customers’ and stakeholders’ trust in the product and in overall business success. Given that an ML product is inherently technical, abstract, and possibly hard to grasp, it’s the responsibility of the ML product managers to make the product clear, coherent and understandable to less technical stakeholders.</li>
<li>Acceptance criteria are something that machine learning product managers should pay particular attention to. By its nature, the inner workings of an ML system are incredibly hard to dig into and understand. Thus, the only reliable way to determine if an ML system is working well is to define rigorous acceptance criteria for the outputs. This involves defining quality control for the data process and results. In addition, product managers should perform regular checks on open bugs, output precision issues, result incompleteness and inconsistency, missing data, etc. It’s not exactly a new skill, but here it takes on new significance, and requires more attention.</li>
</ul>
<h2>6 Machine Learning Lessons for Product Managers</h2>
Machine Learning evolves rapidly, and it’s going to continue to upend the way we build products. It seems that almost every new software product seems to aim to implement ML to meet users’ demands more efficiently, regardless of whether it’s a good use of the technology. There is ample reason to learn the technology and understand its basics, so that you’re well-placed to understand it’s impact in your own products, or just your market in general. Here are six lessons for product managers to consider as they start the journey.
1. Develop a Nuanced Understanding of ML
This shouldn’t be a surprise, but when working with ML, product managers need to develop additional skills and competencies alongside their basic product practices:
<ul>
<li>An intuitive understanding of how ML works, and the datasets it requires.</li>
<li>The ability to see which tasks can and &#8211; more importantly &#8211; can’t be efficiently performed with ML.</li>
<li>The ability to understand which tasks are feasible for ML, but at a disproportionate / unjustifiable amount of effort.</li>
<li>An additional layer of understanding about your company, focusing on what data you have access to, what weaknesses could benefit from an automatic / ML solution, and what strengths will allow you to differentiate your ML implementation.</li>
</ul>
This combination of skills and insights will help product managers to be confident that they are working on what really matters to the business, and doing it in a way that is feasible and effective.
<h3>2. Define the Problem You’re Trying to Solve With ML</h3>
Every product aims to solve a user problem or pain, or to somehow improve their situation. So the very first thing to do is to define the value you’re aiming to create with your product, so that you can start to work out how you should create that value. Before developing an ML-driven product, you should ask yourself:
<ul>
<li>Will machine learning enhance the customer experience and make it more personalized? Combine data analysis with observation to identify the problem and decide if you’re targeting the right part of the customer experience.</li>
<li>Is it possible to make the customer experience safer and more secure by implementing ML into the product? Do a very thorough risk analysis to identify the potential issues with an ML implementation, and whether you can reduce any risk via an ML implementation.</li>
<li>Will it make the product more useful to the user? Can it help the customer to accomplish their objective faster and easier? Explore and analyze the user journey to understand how you can simplify or shorten it with an ML implementation.</li>
<li>Will the ML-driven product bring a fresh and/or totally new experience to your customers? Your product should be unique with distinctive feature/s on offer. Only the extensive review and study of your audience will help you answer this question.</li>
</ul>
<h3>3. Assess Whether ML is the Best way to Solve the Problem</h3>
Before you start developing an ML-equipped product, you should obviously understand whether it solves the users’ problem, and then decide whether an ML approach is worth investing in. The implementation of machine learning to your project requires time, effort, and budget, and some problems are better suited to an ML solution than others. Once you’ve decided if the problem is suitable for machine learning, don’t forget that you will also need:
<ul>
<li>Good professionals &#8211; ML experts are needed in the team, so if you don&#8217;t have them, you need to hire them.</li>
<li>Good quality data &#8211; Gathering and sifting data is a time-consuming task. However, without good training data, all further efforts are meaningless and will result in biases and mistakes, or meaningless outputs. It’s still a good idea to launch a Minimum Viable ML Product (<a href="https://ddi-dev.com/blog/programming/how-to-build-minimum-viable-product/">MVP</a>), but the ML product manager should still keep in mind that 80% of the work on the product takes place after the first release, and after analyzing and interpreting both incoming data sources and product analytics. There is a huge difference between carefully tracking data that will be used to predict or model the layers in the machine learning system, and collecting data that is brought by real user experience.</li>
<li>Lots of iterations &#8211; Don’t expect that everything will work smoothly at launch! ML products require more adjustments and iterations than non-ML products, so get ready to interact with the model of the product regularly as you make progress.</li>
</ul>
<h3>4. Identify Mistakes and Biases</h3>
According to Murphy&#8217;s law, whatever can go wrong, will go wrong and, when working with ML, things can go wrong very quickly in ways that might be hard to spot. That said, it’s essential to brainstorm how the ML model might fail (another area where your own ML expertise and understanding are critical to being effective). At the very least, you should understand that your own biases or biases in your training data, can have a dramatic impact on your ML model and customer outcomes.
Ask yourselves how a semi-autonomous adaptive system might start to behave in ways that are counter to your business or your customers, how you might spot those trends early, and how you might intervene. Ideally, you should have a plan for how you and your team could mitigate issues early. Bear in mind that fixing biases and mistakes in an ML system later can be a much more expensive and difficult task.
<h3>5. Get Used to Managing Uncertainty</h3>
ML products are probabilistic in themselves, and their development can often lead to meaningless depletion of the budget with no clear positive outcomes &#8211; not a great experience for either the product manager, or the organisation. While product managers are no strangers to uncertainty, some features and pitfalls of working with ML need to be taken into account:
<ol>
<li>Manage your own doubts &#8211; Product managers are usually reluctant to work on products or with technologies that are expensive to develop, when the chances of success are uncertain. Be ready to embrace a certain amount of constructive doubt, and work to remove ambiguity as you move forward. This is part of every product manager’s role, but ML can be particularly ambiguous.</li>
<li>Manage high stakeholder expectations &#8211; Many businesses tend to justify the costs of ML products that require large research investments by setting impossibly high expectations for what can be achieved by ML. It’s important to remind your stakeholders that machine learning isn’t magic, and ensure that they understand the limitations and strengths of the technology.</li>
<li>Manage timeframes &#8211; Building highly effective machine learning can take a long time! Sometimes this is because it takes a long time to get access to good quality training data, and sometimes it’s because the algorithms and software are just hard to build. Just like any other launch, some ML products have to be split into separate projects and launched to the market faster than you’d like to validate your vision and start delivering value.</li>
<li>Always be looking ahead &#8211; Remember that you’re trying to release a semi-autonomous, self-learning software system. It’s going to progress in ways that you can’t necessarily anticipate, and it’s going to be creating future challenges and opportunities that are radically new. It is crucial to understand what is being created now and why, and what will need to be developed later to either capitalise or mitigate today’s work.</li>
<li>Always have a plan B &#8211;  Always &#8211; and I mean always &#8211; have a backup plan for what to do if the ML implementation cannot deliver the outcomes needed by your customers or your business. Because ML is a high-uncertainly field, there’s a higher-than-average chance that you’ll struggle to deliver on the product vision.</li>
</ol>
<h3>6. Don’t Forget: Little Changes, big Consequences</h3>
With everything we’ve covered, hopefully you can see that ML has the potential to be immensely powerful, but it is also not as well understood as other technologies, and there are still large degrees of uncertainty in developing machine
Building a machine learning product has as much in common with science as engineering, meaning you are dealign with more unknowns, you are constantly experimenting with the fundamentals of your product, and &#8211; crucially &#8211; you need to control for variables.
We all feel the desire to constantly improve the UX or the product copy or the user flow. However, these impulses must be avoided because even small changes, if not properly factored in, can lead to startling changes in your ML system. Which can then lead to more serious business problems, if not fully understood. Even just &#8220;reformulating the questions you ask your users&#8221; can significantly change the training data you’re using to build your ML model, or break a model that’s been built using historical data.
This is not to say that you should feel paralysed or prevented from adapting your product &#8211; of course not &#8211; you should just be very aware of the potential scope of seemingly small changes, and ensure that you’re collaborating with your data scientists, engineers and designers to fully understand what you’re trying to achieve, and how best to get there.

core/freeform

AI ethics advice from former White House technologist – Kasia Chmielinski (Co-founder, The Data Nutrition Project)

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

LLM workflows for product managers: 3 key takeaways (Niloufar Salehi, Assistant Professor at UC Berkeley) – ProductTank SF

The future of product management: Insights from ProductTank San Francisco

Product lessons learned making early moves with AI in media: Lindsey Jayne (CPO, Financial Times)

A year with ChatGPT and product innovation: Navigating the AI landscape

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

Machine Learning for Product Managers – A Quick Primer

Alexey Kutsenko

Read more

Comments