Ebook: Democratizing data access and use

Download
Glossary

Data Mesh

Data mesh is a decentralized, federated approach to data management that enables data sharing and data democratization across the organization.

What is data mesh?

Data mesh is an enterprise data architecture based on a distributed, decentralized approach to managing and sharing data.

It is designed to increase the use of data across the organization, enabling companies to become more data-driven by making it faster to scale, share and create data products. Data mesh therefore supports strategies that aim to deliver data democratization.

The concept was originally proposed by Zhamak Dehghani of consultancy Thoughtworks in 2019, and has since been developed and adopted by multiple organizations.

Unlike a lot of previous data architectures it focuses on the organization itself, rather than technology. It looks to decentralize responsibilities for particular data to those that are closest to them, but backed up by agreed, company-wide governance and metadata standards to ensure interoperability, with the architecture enabled by a shared self-service data infrastructure.

Essentially, it is a federated model, a bit like the United States of America, with some central government but with power and responsibility in the hands of the states, those closest to citizens (data owners).

What are the principles of data mesh?

Unlike concepts such as data warehouse or data lake, data mesh is not a specific tool or technology. Instead it is a set of principles that define how companies govern, work with, and share data within the organization.

It is based on four key principles:

  1. Domain ownership – rather than a central team, data is owned by those that are closest to it, such as those that create it. These independent, distributed teams are responsible for ensuring data is available, discoverable, addressable, trustworthy, reliable, interoperable, and understandable by all.
  2. Data as a product – data is seen as a product supplied by teams to their customers across the rest of the business, meaning it must be designed to meet their specific needs.
  3. Self-serve data platform – data has to be available to all through self-service, so it can be accessed easily without requiring additional support.
  4. Federated computational governance data governance and metadata standards are agreed and managed centrally to ensure interoperability, consistency and security.

What are the benefits of data mesh?

Organizations want to be able to share data internally to improve decision-making, increase transparency, and drive innovation. Data mesh underpins this through:

  • Faster access to understandable data products by the whole business, making data part of everyone’s role.
  • Simpler and quicker development of data products as independent data teams are responsible for their own data, rather than everything having to be controlled by a central data team.
  • Greater reusability – teams share processes and learn from each other, delivering faster results with less resources.
  • A common language and vocabulary around data that is shared by the whole business, ensuring consistency and common understanding.
  • Centralized governance ensures common standards around security and metadata, meaning that regulatory compliance needs are met.
  • Flexibility in terms of tools. Teams can use whichever tool is the best fit for their needs, giving them independence and increasing their buy-in to the program.
  • Empowered teams. Individuals within independent data teams are trusted as domain experts, with their expertise valued, further driving engagement and maximizing use of resources.

How is data mesh different from other data management methodologies?

There are two main differences between data mesh and other data management methodologies:

  • Data is not centralized (as in a data warehouse or a data lake). The federated nature of data mesh means that data owners are distributed across the organization, supported by centralized governance.
  • Data mesh is not a specific technology or tool. Essentially all existing tools are data mesh compatible, meaning companies can start with their strategy and aims, and then deploy the right tools to meet their needs. This reduces the risk of projects failing, taking too long to implement or not delivering enough results.

What are the challenges to successfully implementing data mesh?

Unlike traditional data projects, data mesh relies less on technology and more on implementing a data-driven approach across the organization. This can lead to three main challenges:

  • Adopting data mesh requires internal transformation to build a common data culture. This means breaking down silos and extensive change management across the organization. This takes investment of time and resources.
  • Common rules around data governance need to be agreed, put in place and then followed across departments. This requires engagement and buy-in across teams.
  • Organizations need to be able to take a strategic approach that identifies problems to solve first, rather than simply adopting technology. This requires data maturity, built on a data culture.

Want to learn more about our data democratization platform? Contact one of our experts!

Learn more

Putting in place an effective data governance strategy Data Trends
Putting in place an effective data governance strategy

What are the practical steps that organizations need to take when it comes to data governance? How can they ensure their programs deliver real business benefits? To learn the secrets of data governanc...

How do you structure your data team for success? Digital transformation
How do you structure your data team for success?

Ensuring data flows effectively around your organization means adopting the right structure for your data team. What are the options and when should you pick a centralized, rather than decentralized a...

How to convince your boss to back your data project Data Trends
How to convince your boss to back your data project

While data use and sharing is growing, it doesn’t mean that your data project will automatically get the senior management backing it needs to be successful. What are the challenges to having your p...

Putting in place an effective data governance strategy Data Trends
Putting in place an effective data governance strategy

What are the practical steps that organizations need to take when it comes to data governance? How can they ensure their programs deliver real business benefits? To learn the secrets of data governanc...

How do you structure your data team for success? Digital transformation
How do you structure your data team for success?

Ensuring data flows effectively around your organization means adopting the right structure for your data team. What are the options and when should you pick a centralized, rather than decentralized a...

How to convince your boss to back your data project Data Trends
How to convince your boss to back your data project

While data use and sharing is growing, it doesn’t mean that your data project will automatically get the senior management backing it needs to be successful. What are the challenges to having your p...

Start creating the best data experiences

Request a demo