Lessons from building data products 

We’ve all heard it, “Data is the new oil of the 21st century”. This couldn't be truer today, especially with the rise of Gen AI. With advancements in computing power and AI models, companies now have cheaper and easier ways to tap into their data goldmines. 

The reality, though, is that many organizations are swimming in data but barely staying afloat when it comes to extracting meaningful insights and taking strategic advantage of their data. They’re data-rich but insights-poor. 

To reap benefits from data that are not just incremental but can advance business goals in an accelerated manner, I’ve found that applying a product management mindset to data products is key. Over the years, with the evolution of product management as a discipline and now increasing importance of data products, I have seen the difference applying a product management mindset to building and managing data products can make. 

But before exploring that, it is important to clarify what data products are, given the definition varies depending on who you ask. When talking about data products, I fall back to the original definition provided by Dr. DJ Patil (former Chief Data Scientist, US government) as data products being those that leverage data to fulfill a specific objective. These can be internal facing products that help organizations with business decisions, or external facing products that are monetized. 

Some examples of external facing products are Walmart Luminate that provides insights to its merchants and suppliers on top of sales data, Foursquare’s location insights products, recommendation products that power Amazon, Netflix and Mastercard and Visa payment insights products, among many others. 

In my experience building and managing data products at Mastercard and other companies, I’ve found that there are some key considerations that can make a world of difference between success or failure:

Start by clearly outlining what it is that you want to achieve with your data. Regardless of whether you’re building for internal or external use, understanding the business goal is crucial as it will guide multiple aspects such as guardrails around standards for data quality and completeness, capabilities to be established, and expectations around timelines for product delivery.

For example, if the end goal is to build a data product with intent to monetize, this would require building up several key capabilities, especially if there is no precedence of commercialization of data products within the org, and establishing an ecosystem that serves this goal. 

As an example, you would need a strong understanding of the customer base to be served - which segments, existing or adjacent, data assets - are they strategic and differentiated enough or is there a need for data partnerships to augment existing assets, business model setup - is it complementary to existing business or stand-alone as a new business model plus also evaluation of other capabilities and resource skills needed to successfully implement (Go-To-Market, delivery, sales). 

Versus, if the goal is to enable internal business users to efficiently make decisions, the focus may be more on looking at datasets internally and seeing how those could be brought together in a quicker manner to enable decisions, leveraging existing tech stack and data applications to enable insights. In this case, solutions could potentially be enabled faster with less need for sophistication and refinement of an end deliverable. Also, such a product does not require an extensive ecosystem to support its launch. 

Data is the foundational factor for success of your data product. Which is why, before diving into building any data product it's important to analyze and understand your data. I’ve found that this means knowing your data sources, understanding how the data is being transformed, assessing if data is complete and evaluating the data quality.

This step is crucial because it helps you see what's possible with your data and where you might face limitations. It also helps manage stakeholder expectations and sets a realistic roadmap for what can be achieved now versus later.

During my time building global data products at large financial services organizations, whenever a business need to add new insights modules was identified, an initial step when evaluating the product idea was to assess completeness percentage across markets, attributes and regions. This was done using a global data quality dashboard which helped identify where we had data that was complete and whether the quality was reliable. 

Doing this helped ensure we were not jumping the gun in agreeing on a potential product idea and could set business expectations appropriately. In addition, we could also proactively work with the data quality team to remediate data that was not complete or inaccurate, especially for use cases where a strong business and customer need was emerging. 

When building a data product, the user experience and end user need can sometimes end up being an afterthought. As data products can sometimes be a step or two removed from the end user, product managers can overfocus on applicability of data, with user need becoming secondary. 

When working on a payments insights product, my team and I discovered that the needs of users varied quite a lot based on their role, function, interactions with data, and the kind of decisions they needed to make.

Business users preferred insights on topline portfolio KPIs with the ability to get a quick explanation of the trends to help them make decisions at a glance. Versus, operational users who wanted detailed insights into specific areas of their interest (e.g. Fraud, Authorization) and desired frequent updates to the data to help them unblock day-to-day decisions. 

In addition to the type of insights, the form factor varied as well based on region, size and technical maturity of the customer. Certain customer segments preferred to consume insights via APIs vs.others showed a preference towards visual insights. This nuanced understanding of the customer helped the team critically evaluate the product portfolio to come up with tailored products that served specific use cases and personas.

This is a crucial step as not only can it help build a differentiated solution but it also helps inform technology and investment decisions based on the product desired. A solution needing real-time data versus updates that are monthly in nature will have very different investment and architectural needs.

Trust is super important for the success and adoption of data products. If users don't trust the insights or find it hard to understand how insights are generated, or if they feel their privacy is compromised, the data product is likely to fail.

Partnering with privacy and compliance teams is important as you’re building data products. In my experience, that has meant having a continuous two way communication, helping privacy teams understand the ‘why’ and ‘what’ of the products and making sure they understand the customer challenges so that product and privacy can collaborate in creative ways and make sure that privacy requirements are held of utmost regard but also don’t hinder product value and innovation.

In addition to the close collaboration with privacy teams, I’ve found working with design to provide transparency to customers and embed an understanding of how data is being collected and what it is being used for, is also helpful to establish a sense of trust with customers. Essentially giving customers more clarity and not leaving them with more questions is helpful to address any questions proactively and remove any concerns upfront.

Building data products, especially in highly regulated environment such as financial services, has taken the form of explaining in layman terms why data is being collected or consent is required and how this is needed to in turn deliver value back to the customer and also having a simplified consent collection process which if applicable for multiple products relevant for the same audience, may make sense to collect across all products in the portfolio.

By embedding privacy, trust, and transparency from the get-go, you'll build a data product that's not only compliant but also trusted and easily adopted by users.

Over time, data products can start to drift away from their original goals if they're not carefully monitored. You might end up with a mess of insights that don't align with your objectives, leading to tech debt and higher costs. Just like when building any other product, setting up an end-to-end evaluation process, from source to delivery, for data products is crucial.

Some critical metrics to think about are in the areas of business outcomes being enabled (e.g. revenue, cost reduction, retention, experience), customer outcomes (e.g. engagement metrics such as measuring specific actions taken, CSAT). These I’ve found are typically the more intuitive ones (especially in a B2B data product setting).

However, apart from these, it is especially important to measure other metrics that can adversely impact the business and customer metrics if they start regressing. These metrics are important to understand the health of your data product, such as:

Data quality: are you seeing any emerging challenges with the quality of your data either through automated quality checks or by monitoring tickets and bugs proactively to remediate any emergence of challenges in this area.,

Data latency: Is the data being refreshed and kept up to date in accordance with the SLAs agreed upon with customers? Are there any frequent delays in the data pipeline that may indicate a bigger issue with architecture?

System performance: Is data being returned to users as per SLA, are there any emerging hot spots of slow query performance, increasing usage that should be proactively addressed

Additionally, I’ve found that for established data products, it’s vital to keep an eye on engagement with the different insight domains. If there are insights that are not being used for a certain period of time, it may be worthwhile evaluating if a different model of serving them up to users might be beneficial versus keeping them on a product that is meant for more frequent usage. 

This can help with reducing unnecessary bloating of the data pipeline which can also come at the expense of performance and overall maintenance costs.

Last but probably not least, data products and the initiatives surrounding them can be pricey and, depending on the organization's maturity, could require significant changes in its capabilities. This is why getting leadership buy-in is essential. Without top-level support and alignment on strategic goals, data products and initiatives are likely to fail. 

It's crucial to mobilize various teams and ensure they move in the same direction. Therefore, it's vital to achieve buy-in from senior leaders, key business teams, technology, privacy, and compliance stakeholders. 

As data products gain momentum across organizations, companies are increasingly focusing on evaluating how they can extract value and not get left behind in the age of AI and data. This requires organizations to apply product thinking and hire people who understand data and AI and are fluent in the product discipline. 

For product managers, this is a really exciting time as it will lead to immense opportunities for growth and new opportunities to apply product skills in the space of data and AI to help organizations build innovative products.