post

Footer About

Footer Contribute

Footer Get Involved

Footer Links

Mind the Product

Guest Post

Creating metrics that matter

A case study: Launching a product process in a project-based company

B2B product management

Artificial Intelligence (AI) & Machine Learning (ML)

Building Product Teams

Business Strategy

Data Driven Product Management

Digital Transformation

Diversity and Inclusion

Minimum Viable Product

OKRs

Prioritisation

Product Design

Product Research

Product Development Process

Product Ethics

Product Discovery

Product Leadership

Product Management Career

Product Strategy

Stakeholder Management

🌎 <a href="https://www.mindtheproduct.com/world-product-day/" style="color:#FFFFFF;text-decoration:underline;">Happy World Product Day!</a> Be sure to celebrate product, yourself, and your peers today! 🎉

How much are product managers getting paid in 2024?

Leading product ops at Delivery Hero with Koen Dubelaar

Connecting fans to creators at Spotify (Dariusz Dziuk, Product Lead, Spotify)

“You’re not as competent as you think” 5 lessons from Ken Chin on navigating a product career (CPO, Spark Networks)

Everything wrong with the product management job market. And how to fix it

Eira is an editor for Mind the Product. She's been a business journalist, editor, and copywriter for longer than she cares to think about.

Eira Hayward

<p>Whether it’s in taxi and food-ordering apps, Spotify and Netflix recommendations, spam email filters, chatbots and virtual assistants, machine learning is all around us. It’s hailed as a technology with many advantages, with its ability to identify trends and patterns and continuously improve without the need for human intervention.</p>
<p>Little wonder that interest in the technology is soaring.<a href="https://www.idc.com/getdoc.jsp?containerId=prUS48881422" target="_blank" rel="noopener"> IDC</a> forecasts that the worldwide market for artificial intelligence, including software, hardware and services, will break the $500 billion mark in 2023.</p>
<p><strong>TL;DR</strong></p>
<ul>
<li><strong>Machine learning (ML) is a broad church &#8211; it encompasses everything from applied research through to plug and play third-party APIs</strong></li>
<li><strong>There’s a lot of excitement about machine learning but it requires specialist skills and knowledge</strong></li>
<li><strong>It’s all about the quality of the data</strong></li>
<li><strong>Don’t forget the usual rules of product management when you’re working on ML products</strong></li>
<li><strong>Developing ML products can be expensive and time consuming</strong></li>
<li><strong>Did we say? It’s all about the quality of the data</strong></li>
</ul>
<p>While you may be excited about machine learning, it does require specialist skills and knowledge. As <a href="https://www.linkedin.com/in/christophercwade/" target="_blank" rel="noopener">Chris Wade</a>, Senior Product Manager at Smartnumbers says: “In a lot of cases, people think they should be using machine learning, because everyone else is and machine learning is the big, new, cool thing.” So if you aren&#8217;t already, then the chances are that, as a product manager, you will be tasked with looking into the whys and wherefores of introducing machine learning into your company’s products.</p>
<p>Help is at hand. In the first of a two-part series, we speak to some senior product people who live and breathe machine learning to find out what you need to know and where you should start. Here goes.</p>
<h3>What is machine learning?</h3>
<p>The terms artificial intelligence and machine learning are often used interchangeably, and that’s fine. But if we’re being perfectionists, then machine learning is a subset of artificial intelligence.</p>
<p>A good definition of machine learning comes from IBM (which has a long history of work in developing artificial intelligence): “Machine learning is a branch of artificial intelligence (AI) and computer science which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its accuracy.”</p>
<p>There&#8217;s another term you may also have come across &#8211; deep learning. It’s a type of machine learning based on neural networks and designed to imitate how humans think and learn. Examples include things like virtual assistants and driverless cars.</p>
<h2>What does a product manager need to know about ML tech?</h2>
<p>Product managers can call on designers for their design problems, and on engineers for coding. Is machine learning any different? The answer is more nuanced.</p>
<h3>Develop some intuition</h3>
<p>“The first thing people think about machine learning is that it&#8217;s magic. It solves all known problems,” says <a href="https://www.linkedin.com/in/jamessmith7/" target="_blank" rel="noopener">James Smith</a>, Group Product Manager at Google. “And the big answer, the big secret, is that it really, really doesn&#8217;t. In fact, it can add a bunch of problems.”</p>
<p>You need at least to get to grips with some of the high-end concepts in ML. As <a href="https://www.linkedin.com/in/graham-paterson-938b861b/" target="_blank" rel="noopener">Graham Paterson</a>, Product Partner at Connect Ventures and a former senior product manager at Deliveroo, comments: “You use websites, you use apps, you have been a customer, so you know the end outcome that you&#8217;re looking for. You probably haven’t knowingly used machine learning models &#8211; do you know how they work, what they produce, what they do well, what they don&#8217;t do well. It helps to build up a basic intuition of what machine learning does.”</p>
<p><a href="https://www.linkedin.com/in/clairewoodcock/" target="_blank" rel="noopener">Claire Woodcock</a>, Staff Product Manager at Twitter research lab Cortex Applied AI Research also points out that machine learning isn’t just “one thing”. It can be anything from plugging third-party products into a problem through to defining new mathematics, or new machine learnings. She recommends the online publication <a href="https://towardsdatascience.com/" target="_blank" rel="noopener">Towards Data Science</a> for any product manager who wants to learn a bit more.</p>
<p>We’ve also outlined some useful courses and resources at the end of this post that can help you to understand a bit more about ML technology.</p>
<p>The exciting thing about ML, Claire adds, is that it&#8217;s starting to become a tool that any software company can use, thanks to the falling cost of computing power and a growing understanding of what it can do. “But it’s not plug and play. Generally, there’s a gap in knowledge about how you set yourself up for success and make machine learning work for your particular use case.”</p>
<h3>Apply the rules of product discovery</h3>
<p>Not every problem is appropriate for ML. It’s a product manager’s job to remain disciplined and focused on the product, and not to get carried away by the possibilities of exciting new technology. So start by asking what user problem you’re solving, as you would with any product management problem.</p>
<p>Chris Wade comments that when technology started moving to the cloud &#8211; a big, new, cool thing from a decade ago &#8211; it was easy to see the benefits: “But with machine learning, people misunderstand the time, the cost, the suitability, and even the feasibility.”  Chris wrote a post, previously published on Mind the Product, <a href="https://www.mindtheproduct.com/should-you-really-be-using-machine-learning/" target="_blank" rel="noopener"> Should you really be using machine learning?</a>, that emphasises the need to assess the five product risks  &#8211; is it valuable, usable, feasible, viable and ethical &#8211; when doing machine learning discovery.</p>
<p>Make no mistake, machine learning can be very time-consuming and expensive. Chris adds: “Feasibility is almost underselling it. The level of uncertainty involved with machine learning plus the time it can take to figure something out is often not appreciated.”</p>
<p>“Machine learning can do some things really well. It&#8217;s really good at classifying, and at automating boring, repetitive tasks,” says Claire. “What it really struggles with is anything which you would attribute to human problems or creative challenges. Anything which is trying to codify culture or social interaction, you just don&#8217;t want to go there.</p>
<p>Many problems that we think we can solve with machine learning can be solved more simply, adds James Smith. Once you introduce machine learning, you also introduce a huge burden in terms of maintenance cost and opaqueness, because a lot of these machine learning systems are not explainable. “You probably also start to introduce a lot of problems around unexpected consequences &#8211; machine learning systems frequently do go wrong,” he says.</p>
<h3>Do you have the right data?</h3>
<p>If you’ve established that machine learning can solve your user problem you need the right data. As <a href="https://www.linkedin.com/in/jackyma/" target="_blank" rel="noopener">Jacky Ma</a>, Director of Data Science at Remerge, comments: “A tech project’s truth is in the code, but machine learning’s truth is in data.”</p>
<p>Machine learning can only be as good as the data you have. Here are some ways for product managers to think about the suitability of their data for machine learning:-</p>
<ul>
<li>How fresh and reliable is the data? Is it up to date? Do you have data arriving at the same time every day or are there gaps in the data?</li>
<li>How good is the data quality? Is it complete, is it biased? Product managers can measure data quality by setting up metrics to measure what’s important about their data.</li>
<li>Is the data easy to understand? If you produce high quantities of good-quality data points and you need to make decisions with them very quickly, then machine learning can really help you. If your decisions are slower and you have fewer data points then a human being will be more suitable.</li>
<li>How secure is the data? If it’s not secure then you are vulnerable to injection attacks and the introduction of unintended consequences.</li>
<li>Is the data available in a form that can be consumed by a machine learning algorithm? If your data is, say, full of free text boxes where anyone can type what they like, then it’s probably not suitable.</li>
<li>Do you have user consent to use their data in the way that you want to? In some territories, there is legislation that requires explicit user consent to use their personal information to make an algorithmic decision about them.</li>
</ul>
<p>A further thought on bias &#8211;  as James Smith comments, product managers play a vital role in organisational culture. They should  articulate what’s important to the organisation about bias, and monitor for statistical anomalies that indicate whether accurate information on all groups is collected.</p>
<p>The importance of the right data can’t be overestimated, and making sure you have the right data is an expensive process. “If you don&#8217;t feel like you have the budget to do all the data work, then you probably don&#8217;t have the budget to do a machine learning solution,” says James.</p>
<h3>Make sure you’ve got the people</h3>
<p>As Claire Woodcock says, there are talent challenges at every stage of machine learning. There aren&#8217;t enough data scientists for the kind of applied research labs where she works, so they&#8217;re in high demand and command huge salaries. These businesses, says Claire, can hire new postgraduates from PhD programmes, but attracting experienced people is hard.</p>
<p>Such top talent is beyond the reach of startups and scaleups, but it’s a problem you can approach in other ways. There are consultancies and the like who can get you started and lots of third-party machine learning APIs available to buy through the likes of Amazon and Google. As Chris Wade says: “I’ve been running ML teams for the last three years and I didn’t have a background in machine learning. The actual engineering isn&#8217;t necessarily the hardest part. It&#8217;s making sure you’ve got the data.”</p>
<h3>Manage expectations</h3>
<p>Machine learning is time-consuming and can be expensive. Your business stakeholders may have the impression that the technology can do much more than it’s able to and have very high expectations of how well it can perform.</p>
<p>Chris Wade comments that machine learning is different from anything you’ve done before as a product manager and that uncertainty can be quite difficult to deal with. His advice is to start small &#8211; find a project that might be suitable for machine learning, and see if you can get some budget and some engineers together and play around with it for a few months.</p>
<p>You should also bear in mind that machine learning is not infallible and there will always be edge cases. So think about tolerance levels. Says Claire Woodcock: “Think about where you want the algorithm to index &#8211; for instance, if you’re working on something medical you would over-index on caution. If it’s a shopping application then precision is less of a concern.”</p>
<p>With machine learning, as with product management generally, the most important thing before you start building is to understand what success looks like.</p>
<h3>Next time</h3>
<p>Next time we’ll look at setting yourself up for success &#8211; at some of these issues in more detail and at the skills you need to hone to run an ML team. We’ll also look at some examples and case studies of machine learning in action.</p>
<h3>Further resources</h3>
<p>US data scientist Andrew Ng is something of an AI visionary and online learning pioneer. He hosts a number of online educational courses on ML through his company Coursera including <a href="https://www.coursera.org/learn/ai-for-everyone" target="_blank" rel="noopener">AI for everyone</a> and <a href="https://www.coursera.org/specializations/machine-learning-introduction" target="_blank" rel="noopener">Machine learning specialization</a>.</p>
<p>In this blog post, <a href="https://medium.com/@gpat_uk/behind-the-machine-product-managing-an-algorithmic-product-team-d48cd5f9bf6d" target="_blank" rel="noopener">Behind the machine: product managing an algorithmic product team</a>, Graham Paterson relates some of the things he learned while running an algorithmic product team at Deliveroo.</p>
<p>This talk by James Smith <a href="https://www.youtube.com/watch?v=WAZUyDhgv4U" target="_blank" rel="noopener">Building AI Products</a> runs through some basics and offers a simple decision tree to help you decide if your product is suitable for ML</p>
<p><a href="https://www.pendo.io/resources/the-analytics-led-path-to-digital-adoption/?&amp;utm_source=mtp&amp;utm_medium=display&amp;utm_campaign=mtp-resources-analytics-led-q3" target="_blank" rel="noopener"><span data-sheets-value="{&quot;1&quot;:2,&quot;2&quot;:&quot;The analytics-led path to digital adoption&quot;}" data-sheets-userformat="{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}">The analytics-led path to digital adoption</span></a></p>
<p><a href="https://nexocode.com/blog/posts/ai-products-you-already-use/" target="_blank" rel="noopener">AI Products You (Most Probably) Already Use</a></p>

core/freeform

AI ethics advice from former White House technologist – Kasia Chmielinski (Co-founder, The Data Nutrition Project)

How Google makes AI work at Enterprise scale – Miku Jha (Director, AI/ML and Generative AI, Google Cloud)

LLM workflows for product managers: 3 key takeaways (Niloufar Salehi, Assistant Professor at UC Berkeley) – ProductTank SF

The future of product management: Insights from ProductTank San Francisco

Product lessons learned making early moves with AI in media: Lindsey Jayne (CPO, Financial Times)

A year with ChatGPT and product innovation: Navigating the AI landscape

How to keep your head about generative AI (when everyone is losing theirs) by Claire Woodcock

A guide to machine learning Part 1: What’s important

Eira Hayward

Read more

Comments