Artificial Intelligence (AI) & Machine Learning (ML)

How to obtain Product- Market fit with AI

Darian Chavira, senior product manager at Rockwell Automation, outlines the power of data science in achieving product-market fit and how it can transform your product development strategy.

March 11, 2024

8 min read

Share on

Darian Chavira

Darian has a diverse background ranging from air & space defense, AWS, and OT cybersecurity. He has a passion for sharing my experiences in product management, development, and leadership. Currently, he is a senior product development manger at Rockwell Automation with a focus on B2B cybersecurity services and solutions

Data, data, everywhere

Data science is an integral part of product management and culture. Data science allows product teams to move from focusing on time-to-market to time-to-outcome.

In a time-to-market model product teams rush to market to find out how their product sells. In time-to-outcome model product teams work towards an outcome based on customer data and input.

“Data is the new electricity of our economy, and just like electricity, it is touching every part of our society. AI is one of the most important technologies of our time, but it’s really customer data that’s the fuel that powers AI.” Satya Nadella, CEO of Microsoft

Machine learning (ML) is dependent on high-quality data for generating accurate predictions. The integrity and relevance of data are crucial in determining the effectiveness and dependability of ML models. Here’s how to ensure the collection of quality data:

Design targeted surveys tailored to specific analytical goals.
Establish clear data collection objectives in early-stage product development.
Iteratively refine data collection methods and analytic tools.

Selecting the right model

Assuming we collected quality data, we can select our ML model. The model is dependent on the data and the business outcome.

To be most effective, we should have our outcome in mind when we develop our data collection plan. Working backwards flow:

Thought Process

Business Outcomes → ML Model → Data Prep → Data Collection → Data Planning

Execution

Data planning → Data Collection → Data Prep → ML Model → Business Outcomes

ML model examples

There are a ton of AI/ML/Deep Learning models that could drive outcomes. Some typical business use cases: Customer Churn Analysis, Market Basket Analysis, Marketing Analysis, Operations Analysis, Prediction, Automation, Sentiment Analysis, Recommendations, so forth and so on.

I outline three models: Market Basket Analysis, Customer Churn, and Marketing Analysis.

All pictures are from RapidMiner on synthetic data created on a local repository.

Market basket analysis

My customer bought widget X from me, and I wish I could know what other widgets they would buy? Or do our current customers have this install base what should we offer next?

Okay, let’s break this down.

1. Data:

a. Involves loading transaction data that contains a transaction id, a product id, and a quantifier. The quantifier indicates the number of times a product has been purchased in a transaction.

2. Prepare data through exploration and preparation.

b. Data needs to be cleaned and balanced.

3. FP-Growth:

a. This is a step where the FP-Growth algorithm is used to determine frequent item sets. FP-Growth is an efficient and scalable method for mining the complete set of frequent patterns by pattern fragment growth, using an extended prefix-tree structure for storing compressed, crucial information about frequent patterns called FP-tree.

b. Frequent item sets: These are sets of items that appear together in transactions more often than a defined threshold (support). Decision Tree and/or Random Forest outcomes

4. Create Association:

a. This step involves using the frequent item sets to generate association rules. These rules can suggest that if a customer buys a certain set of items, they are likely to buy another item (or set of items). The strength of these rules is determined by metrics like confidence and lift.

5. Outcomes:

a. Association Rules: Rules that indicate a strong relationship between the purchase of one set of items and another.

b. Frequent Item Set: The commonly purchased sets of items.

Image 1 shows all associations for potential products my customers would buy based on their current purchasing habits.

Focusing on the highest confidence level I narrow my product associations to four products with one association and one product with two associations. Product 20 would be the most likely candidate based on our customers previous purchasing history.

This product selection can be validated through surveys and interviews either externally with customers or internally with sales/commercial leaders.

Customer Churn

Predicting when your customers will churn and why through decision trees and random forests.

Okay, let’s break this down.

1. Start with historical data of customer retention and churn.

a. Hopefully, you have a repository of customer interactions.

b. No customers, stop. Churn modeling is a later problem, not a now problem.

2. Prepare data through exploration and preparation.

a. Data needs to be cleaned and balanced.

3. Select model: Decision Tree and/or Random Forest

a. Key callouts: configuring model for accuracy, recall, and prediction is important to get desired outcomes.

4. Decision Tree and/or Random Forest outcomes

a. Predict the likelihood of future customer churn.

Based on the decision tree, the factors contributing to customer churn can be prioritized as follows:

Number of support calls last year: This appears to be the most significant predictor of churn. Customers making more than 9.5 support calls are most likely to churn, indicating that a high number of support calls is a strong indicator of dissatisfaction.
Average bill amount: For customers with fewer than 9.5 support calls, the average bill is the next most significant factor. An average bill amounts greater than 14.5 leads to churn, suggesting that higher bills may be a churn factor among customers who don’t have extreme numbers of support calls.
Customer since (Date): For those with fewer support calls and a lower average bill, the duration of their relationship with the company plays a role. Newer customers (since after April 18, 2021) are more likely to churn than those who have been customers for a longer time.
Support calls last year (a lower threshold): At the lower end of support calls and average bill, if the customer made more than 4 support calls, they are predicted to churn. This indicates that even at lower levels, the number of support calls is still a predictor of churn, albeit less significant than the higher threshold of 9.5 calls.

Product and direct marketing

Create a customer response model based on past responses to targeted marketing campaigns to predict those customers that are likely to respond to new campaigns and increase the conversion rate.

Okay, let’s break this down.

1. Start with historical marketing data.

2. Based on the historical calculate relevant weights for attributes.

a. Heavier weight means the attribute has more significance.

3. Apply a model to create ideal new customers to market to based historical results.

4. Select visualizations to represent models.

This model created a breakdown of the probability of sales representatives getting responses from their target list based on customer communication preferences, purchasing preferences, and recent communications in the data set.

Humphrey (salesperson) has a 70% probability of getting call backs from his list. From this analysis, we can further analyze the customers on the list and target a customer segmentation based off this list.