Large IT firms aren’t the only ones employing artificial intelligence in production and distribution operations. Whether companies take a cautious approach or leap headfirst, AI programs are gaining popularity across many industries, from the automobile to retail, education, and healthcare.
In this ProductTank Tokyo talk, Hiroki Nakamura examines an AI Product Manager’s tasks and how AI is transforming the product management industry. According to Hiroki Nakamura, here are the important roles that an AI product manager plays in managing the quality of the products:
Perceiving the issues in AI model quality
The three essential elements of product management are user experience, business, and technology, and they should remain the same when using generative AI. However, the technology field is full of uncertainties. Maintaining the AI model in addition to the software technology is crucial, just like with conventional apps and services.
You might be able to focus on accuracy without giving cost much concern if the AI models are lightweight. On the other hand, generative AI makes correct content that meets user expectations using huge AI models. If the AI model is vast, the hosting server requirements and supply costs are not trivial and often hard to manage – unless the company is very well endowed.
Looking for flaws in the product and providing measures
Even if you optimize for Precision/Recall, it will never be flawless. Therefore, you must consider how to deal with that flaw throughout the procedure. This holds true for cognitive AI models as well as generative AI models.
In generative AI models like conversation generation, erroneous speech is very hard to prohibit. Therefore, the ideal course of action is integrating new methodologies while striving for perfection in the AI model. In the case of a conversation-generating AI model, a filtering system, such as a rule-based system, can be employed to multiplex the input from the user and the output created by the AI model.
Ensuring faster iteration of the AI models
In many circumstances, it could be difficult to accurately anticipate how good generative AI will be. It might be possible to forecast the result if recognition-based AI has a history. The evaluation method itself changes depending on the service’s direction, and relatively few examples of generative AI are applied to actual goods, making it difficult to set a quality target and predict the point of achievement based on past performance.
To quickly iterate, improvements are the best approach for making predictions. There are two critical measures to take in order to iterate quickly. The first step is maintaining the separation between the models-improvement and service-implementation processes.
Considering the costs for pre-generating methods
If neither of these problems exists, there is no good reason not to create the traffic in real-time, especially considering the cost of real-time production for the anticipated traffic and the effect of the generation time on UX or user experience.
However, it should be discontinued if real-time manufacturing is too costly to run continually or takes too long to make a single instance. In such cases, one option is to create a substantial portion of the content offline beforehand and select from the pre-made content for use in the service before deciding not to utilize the developed model.
Key takeaways
Some critical notes that Hiroki Nakamura gave during his interview about the role of an AI product manager were:
- Consider the costs for pre-generating methods
- Perceive the issues in AI model quality
- Look for flaws in the product and providing measures
- Ensure faster iteration of the AI Models
Even if it were made available as a tool as it is, Hiroki thinks generative AI is still too difficult to handle to be included in and utilized as a part of a product. However, with the right application, it is possible to create new services unrelated to anything that has come before. Furthermore, because technology is evolving quickly, you can leverage it to your advantage by deploying and updating it as new developments occur.
Learn more about ProductTank – find your local meetup, explore more ProductTank content, see the latest ProductTank news, and discover ways to get involved!
