Why ML-based products require an agile approach to road mapping

Product innovation has always been a key factor in the success of any business. With the rise of machine learning (ML) based products, this innovation has taken on a whole new level. These products constantly evolve and require a different planning and road mapping approach. Unlike traditional products, Machine Learning-based products often rely on experimentation and iteration to achieve the desired outcome. This means that the product discovery phase can take longer, and the go-to-market strategy must be constantly adapted. To effectively manage the expectations of both engineering and Machine Learning, an agile approach to road mapping is crucial. In this article, we will discuss why Machine Learning-based products require an agile approach to road-mapping and how it can help businesses successfully navigate the ever-changing landscape of product innovation.

Embarking on the journey of product discovery for Machine Learning-based products unveils a set of distinct challenges, setting them apart from their traditional counterparts. This phase is not just about identifying potential features or market needs; it involves a deep dive into data analysis, algorithm development, and continuous validation to ensure the components are not only feasible but optimized for real-world applications. Unlike conventional products where the path from concept to prototype might be more straightforward, Machine Learning-based innovations demand an iterative cycle of hypothesis, data preparation, experimentation, and learning to refine the algorithms that underpin them. This process is inherently time-consuming and complex, as it requires collecting and processing vast amounts of data, selecting the appropriate learning style, and continuously iterating on these models to enhance performance. 

The nature of Machine Learning experimentation means that product discovery cannot be rushed or compressed into neat timelines without risking the integrity and effectiveness of the final product. Each iteration uncovers new insights about user behaviors, potential biases in the data, or unforeseen challenges in model scalability and performance. Thus, planning for Machine Learning-based products must incorporate this uncertainty, expecting and embracing shifts in direction as part of the discovery phase. A rushed product discovery is what led to the downfall of IBM Watson for Healthcare, where an immature technology and a lack of continued customer focus, fueled with data challenges resulted in a struggling product with adoption challenges.  

This iterative exploration in the discovery phase is critical not only for refining the product but also for uncovering innovative applications of this Machine Learning technology that could provide a competitive edge in the market. Consequently, teams venturing into developing Machine Learning-based products need to arm themselves with patience, flexibility, and readiness to navigate the unpredictable waters of machine learning discovery.

Launching Machine Learning-based products into the market demands a flexible and responsive approach, distinguishing it from the traditional linear pathways to market entry. An agile go-to-market strategy is indispensable, allowing for the assimilation of feedback and insights directly from early users via offline experimenting. This adaptive approach facilitates the identification of product refinements and potential pivots in the product’s direction that may be necessary to meet evolving customer needs and expectations more effectively. A general and very classic example of agile Go-To-Market strategy is Slack Technologies, which was originally founded with a focus of developing online video games. During the game’s development, the team created an internal communication tool to streamline collaboration within their team. While the game faced challenges, the company made a strategic pivot and repurposed the communication tool into Slack, the enterprise messaging platform we all know today.

Incorporating user feedback early into the development cycle often enables the team to make informed decisions about feature adjustments, interface improvements, and even significant changes in the product offering. This loop of feedback and iteration helps in fine-tuning the product to better fit the market demand, ensuring that when the product is officially launched, it resonates well with its intended audience. Instagram’s evolution from Burbn is a testament to how user feedback-led pivots in Go-To-Market strategy can become massive successes. 

Moreover, an agile go-to-market strategy supports the dynamic nature of Machine Learning models, which may evolve as they are exposed to more data and diverse use cases. This flexibility in strategy ensures that the promotional tactics and product positioning can be quickly adjusted to reflect the product's evolving capabilities and add-on value proposition. It's almost a proactive measure against the backdrop of fast-moving technology sectors where customer preferences and competitive landscapes shift rapidly. In my personal experience, creating a foundation with your core value proposition and maintaining a consistent message will not only support the initial product launch but will also sustain ongoing growth and adaptation in the lifecycle as the product evolves. 

Iteration stands as the cornerstone of any successful Machine Learning product development process. Unlike traditional software where a linear development trajectory may suffice, Machine Learning initiatives thrive on a cycle of build, measure, learn, and iterate. As these products are exposed to new data and real-world interactions, the underlying models will need to be continuously refined. This iterative refinement is not a sign of setbacks but a marker of progress, indicating a deepening understanding of both the business problem and the potential of Machine Learning solutions to address it. 

The need for iteration arises from the complex and dynamic nature of machine learning, where initial assumptions about data and user behavior often evolve. For one of our projects, we encountered significant delays, initially due to the unavailability of training data, and subsequently because of the quality mismatch between the training and production data. This led to several model re-training and testing iterations and impacted the overall development timeline. Essentially in this context, iteration transcended from being merely a development phase; it became an integral philosophy guiding the journey of our Machine Learning product creation.

As general guidance, roadmaps for Machine Learning products need to be elastic enough to allow for such re-evaluation of existing strategies within the uncertainty of experimentation. While the elasticity creates an inclination towards adding buffers in timelines, often it is quite difficult to benchmark the research phase timelines and set the right expectations. What I have learned from working on such projects is often expanding your product discovery phase and focusing on developing a basic proof of concept (POC), to validate the idea before a full-scale development, can help in disambiguating the situation.  

The POC will not only help you to test the waters, identify future problems, and get timeline estimates but it will also help you set realistic leadership expectations by providing tangible evidence of a proposed product's feasibility. This builds confidence and can help increase their buy-in for the project overall. 

Navigating the intersection between engineering disciplines and the nuances of machine learning is a pivotal aspect of developing these Machine Learning-based products. Bridging the gap between these two perspectives hinges on clear, open communication channels and a mutual understanding of the iterative nature inherent to Machine Learning projects. Engineering teams tend to lean towards a more deterministic and structured development cycle, accustomed to predefined specifications and a clear endpoint. In contrast, Machine Learning scientists often operate within a probabilistic framework, where outcomes are not always certain, adjustments are made continuously based on new data and insights and meeting timelines can become a best-case scenario. 

Early and frequent efforts to establish a common language and shared objectives can be instrumental in harmonizing the perspectives of Machine Learning and engineering teams. In my own experience, while shared objectives fostered a sense of collective ownership, the real turning point was implementing joint planning sessions.

During bi-weekly planning meetings, we didn't just focus on team goals; we consistently emphasized the interdependencies between each team's milestones. These sessions allowed both teams to gain a deep understanding of each other's workloads and dependencies. This transparency had a significant impact on the project. For instance, during one such session, the engineering team proactively identified potential technical feasibility issues. This early communication not only managed expectations with the Machine Learning team but also resulted in far more informed estimates for the project roadmap.

In essence, all team members must appreciate the value of flexibility and the potential for change as the project progresses. Cultivating an environment where iterative development is recognized as a pathway to innovation, rather than a series of setbacks, can significantly streamline the journey from concept to deployment. This collaborative approach ensures that expectations are managed effectively, paving the way for the successful realization of Machine Learning-based products.