[NEW REPORT] The State of European Energy Data Maturity - In-depth research with E.DSO and GEODE

Download here
Glossary

Data Mesh

Data mesh is a decentralized, federated approach to data management that enables data sharing and data democratization across the organization.

What is data mesh?

Data mesh is an enterprise data architecture based on a distributed, decentralized approach to managing and sharing data.

It is designed to increase the use of data across the organization, enabling companies to become more data-driven by making it faster to scale, share and create data products. Data mesh therefore supports strategies that aim to deliver data democratization.

The concept was originally proposed by Zhamak Dehghani of consultancy Thoughtworks in 2019, and has since been developed and adopted by multiple organizations.

Unlike a lot of previous data architectures it focuses on the organization itself, rather than technology. It looks to decentralize responsibilities for particular data to those that are closest to them, but backed up by agreed, company-wide governance and metadata standards to ensure interoperability, with the architecture enabled by a shared self-service data infrastructure.

Essentially, it is a federated model, a bit like the United States of America, with some central government but with power and responsibility in the hands of the states, those closest to citizens (data owners).

What are the principles of data mesh?

Unlike concepts such as data warehouse or data lake, data mesh is not a specific tool or technology. Instead it is a set of principles that define how companies govern, work with, and share data within the organization.

It is based on four key principles:

  1. Domain ownership – rather than a central team, data is owned by those that are closest to it, such as those that create it. These independent, distributed teams are responsible for ensuring data is available, discoverable, addressable, trustworthy, reliable, interoperable, and understandable by all.
  2. Data as a product – data is seen as a product supplied by teams to their customers across the rest of the business, meaning it must be designed to meet their specific needs.
  3. Self-serve data platform – data has to be available to all through self-service, so it can be accessed easily without requiring additional support.
  4. Federated computational governance data governance and metadata standards are agreed and managed centrally to ensure interoperability, consistency and security.

What are the benefits of data mesh?

Organizations want to be able to share data internally to improve decision-making, increase transparency, and drive innovation. Data mesh underpins this through:

  • Faster access to understandable data products by the whole business, making data part of everyone’s role.
  • Simpler and quicker development of data products as independent data teams are responsible for their own data, rather than everything having to be controlled by a central data team.
  • Greater reusability – teams share processes and learn from each other, delivering faster results with less resources.
  • A common language and vocabulary around data that is shared by the whole business, ensuring consistency and common understanding.
  • Centralized governance ensures common standards around security and metadata, meaning that regulatory compliance needs are met.
  • Flexibility in terms of tools. Teams can use whichever tool is the best fit for their needs, giving them independence and increasing their buy-in to the program.
  • Empowered teams. Individuals within independent data teams are trusted as domain experts, with their expertise valued, further driving engagement and maximizing use of resources.

How is data mesh different from other data management methodologies?

There are two main differences between data mesh and other data management methodologies:

  • Data is not centralized (as in a data warehouse or a data lake). The federated nature of data mesh means that data owners are distributed across the organization, supported by centralized governance.
  • Data mesh is not a specific technology or tool. Essentially all existing tools are data mesh compatible, meaning companies can start with their strategy and aims, and then deploy the right tools to meet their needs. This reduces the risk of projects failing, taking too long to implement or not delivering enough results.

What are the challenges to successfully implementing data mesh?

Unlike traditional data projects, data mesh relies less on technology and more on implementing a data-driven approach across the organization. This can lead to three main challenges:

  • Adopting data mesh requires internal transformation to build a common data culture. This means breaking down silos and extensive change management across the organization. This takes investment of time and resources.
  • Common rules around data governance need to be agreed, put in place and then followed across departments. This requires engagement and buy-in across teams.
  • Organizations need to be able to take a strategic approach that identifies problems to solve first, rather than simply adopting technology. This requires data maturity, built on a data culture.

Want to learn more about our data democratization platform? Contact one of our experts!

Learn more
Scaling smart city projects beyond the pilot phase Public Sector
Scaling smart city projects beyond the pilot phase

Delivering smart city success starts with pilot projects to prove that concepts benefit the community. However, often projects fail to scale beyond their initial rollouts, meaning their benefits are lost. We explain the importance of data portals to maximize the chances of project success by seamlessly sharing information with stakeholders.

Taking the next steps with data portals in the Middle East Open data & transparency
Taking the next steps with data portals in the Middle East

Even more than in other areas data portals have a key role to play in delivering innovation, transparency and new services to citizens, businesses and governments across the Middle East. Based on best practice examples, we explain where should organizations focus when it comes to transforming their portals.

Data portals: a major asset for players in the energy sector Energy & Utilities
Data portals: a major asset for players in the energy sector

An in-depth exploration of the impact of data portals in the energy sector. From decarbonization to collaborative innovation, discover how data drives innovation and strengthens regulatory compliance.

Scaling smart city projects beyond the pilot phase Public Sector
Scaling smart city projects beyond the pilot phase

Delivering smart city success starts with pilot projects to prove that concepts benefit the community. However, often projects fail to scale beyond their initial rollouts, meaning their benefits are lost. We explain the importance of data portals to maximize the chances of project success by seamlessly sharing information with stakeholders.

Taking the next steps with data portals in the Middle East Open data & transparency
Taking the next steps with data portals in the Middle East

Even more than in other areas data portals have a key role to play in delivering innovation, transparency and new services to citizens, businesses and governments across the Middle East. Based on best practice examples, we explain where should organizations focus when it comes to transforming their portals.

Data portals: a major asset for players in the energy sector Energy & Utilities
Data portals: a major asset for players in the energy sector

An in-depth exploration of the impact of data portals in the energy sector. From decarbonization to collaborative innovation, discover how data drives innovation and strengthens regulatory compliance.

Start creating the best data experiences
Request a demo