Data product

What is a data product?

Essentially a data product is a product built around data, containing everything someone needs to complete a specific task or objective using that underlying data.

In his book Data Jujitsu: The Art of Turning Data into Product, DJ Patil, former United States Chief Data Scientist, defined it as “a product that facilitates an end goal through the use of data.”

Data products can be grouped into three main types:

• Those that output data as a service, such as an interactive dashboard, a stock ticker or a weather app on your phone.
• This that enhance existing products with data, such as a self-driving car or a smart watch/smart clothing
• Those that analyze data and use it to create data insights, such as Google Analytics or recommendation engines

The key difference between a data product and raw data, is usability – it should be able to be used by a non-specialist without data skills or knowledge of the specific raw data within it to achieve a specific purpose. It can be used standalone, rather than as a subset of another program or product.

A data product therefore contains:

• The raw datasets
• Metadata to provide context, governance and documentation
• A straightforward interface that allows the data product to be accessed and used
• Code to manage the data pipeline, access control and compliance
• Infrastructure to host the product and run the code

Rather than running individual projects to use and re-use data, adopting a data product strategy creates a more standardized, efficient approach that does not reinvent the wheel for each and every project.

How does a data product relate to data mesh?

Data products are one of the four foundational pillars of data mesh, enabling distributed data to be successfully consumed across an organization.

In fact, Gartner describes data mesh as an architecture created with “the specific goal of building business-focused data products”.

However, data products can be developed and deployed without requiring a data mesh implementation or architecture.

What are the benefits of adopting data products?

Data products deliver major benefits for organizations and data users:

• They are faster to develop than project-based approaches, as they reuse infrastructure, methods and previous work
• They are more efficient to develop, run and maintain, providing economies of scale
• They enable organizations to better meet user/business needs, solving specific problems through the use of data
• They enable organizations to create new revenue streams, supplying data products alongside existing products and services
• They democratize data use, by making it easy for non-specialists to use data to solve their specific business problems. This helps build a data-driven organization that makes better, more transparent and faster decisions and can automate and digitize operations
• They incorporate security and access control features as standard, reducing risk and ensuring compliance.

According to Accenture, “Data products typically offer far greater ROI and lower cost-per-use than data projects, because they evolve to support multiple outcomes over time.”

How do you create data products?

For data products to be successful, they need to follow a similar process to creating any new product – physical or virtual:

• Appoint a data product owner, with responsibility for the product’s development and ongoing usage
• Define the need by talking to users and listening to their requirements
• Use these requirements to develop and test a prototype, iterating with end users until it meets their needs
• Create and document the full data product, ensuring it is independent and contains everything required for its operation and maintenance
• Deploy the data product, sharing it with users
• Monitor usage and improve the product in response to feedback and external developments

What are the challenges to creating data products?

As with any shift to become more data-driven, the process of creating data products requires businesses to overcome cultural, organizational and technical obstacles.

• Data culture. The company and its employees have to be open and receptive to using data as part of their working lives, with an open data culture in place
• Data quality. Data has to be available, high quality and trustworthy, rather than being trapped in departmental silos or fragmented in sources across the organization
• Skills and approaches. The organization has to have the skills in place to create and run data products, including employees with the understanding and expertise to act as data product managers. The shift from project to product-based development requires new competencies and ways of working
• Focus on needs. Data products have to be built on the specific needs of data customers (whether internal employees or external to the organization), requiring close collaboration with these audiences
• Prioritization. There has to be a clear roadmap for the creation of data products, prioritizing the most critical products to deliver an effective strategy and clear ROI.