Glossary
Data Mapping
Data mapping is the process of matching data fields from one database or information source, to related fields in a destination source or system.
What is data mapping?
Data mapping is the process of matching data fields or elements from one database or information source to related fields in a destination source or system. It provides a set of instructions that help standardize data, and therefore is a vital part of ensuring quality, following data governance processes, and making data useful when shared between the different databases and tools used within a tech stack.
For example, your corporate standard for US states may be to refer to them by their two-letter abbreviation (e.g. TX for Texas). In this case, data mapping identifies and links the “state” field in a data source you are connecting, transforming it to the appropriate format.
Why is data mapping important?
Data mapping provides a visual view of how data moves around an organization. It is a fundamental step in good data management and enables organizations to:
- Ensure that information is consistent and follows corporate data governance standards
- Prevent errors and duplications that impact quality by merging related records (such as records of the same customer held on different systems). The process can flag potential issues (such as missing data) before it becomes a problem
- Enable the integration of data between different systems
- Make it easy to share data seamlessly internally and externally
Data mapping is central to these key processes:
- Data migration – moving data from one system to another on a permanent, one-off basis
- Data integration – integrating two systems so that data flows between them on an ongoing basis
- Data transformation – changing data as it enters a system, by applying rules to enable consistency or to add value
- Data warehousing – centralizing data in a single data warehouse, ensuring it is consistent and accurate
- Business intelligence – analyzing data for reporting, to uncover insights and to better understand business performance
- Data portal – sharing data internally and externally via a single, centralized data portal that delivers consistent, integrated and reliable data
What is the data mapping process?
Data mapping can be carried out in one of three ways:
Manual data mapping
This requires every data connection to be created individually by a data professional writing specific code to map databases and fields. While this delivers a customized, tailored approach it is extremely time-consuming and requires data teams to have specific skills to write and maintain code.
Semi-manual data mapping
This approach relies on graphical tools that enable data professionals to create schema maps by manually linking fields via a visual interface. This then generates an output script used for coding. Semi-manual data mapping delivers flexibility but still requires specialist coding knowledge and significant amounts of time.
Automated data mapping
Most modern data mapping tools now enable full automation of the process, without requiring any coding at all. This lowers the barrier to entry for users, meaning they do not need to have programming skills. Automated data mapping can be scaled quickly, with regular refreshes to capture any changes. The only downside is that companies need to pay for tools and training.
What are the benefits and challenges of data mapping?
The benefits of data mapping
By providing the foundation for data management, data mapping delivers significant benefits around:
- Trust – employees trust that data is correct and reliable, and are therefore more likely to use it in their jobs
- Reduces silos – it brings together data from different departments and standardizes it, removing silos and barriers to sharing
- Security/GDPR compliance – it provides a full record of how data is mapped across the organization, including its uses. This helps with security and demonstrating regulatory compliance
- Faster, more comprehensive reporting – data is available for analysis much more quickly, with much fuller access to provide greater depth
The challenges of data mapping
The challenges to successful data mapping mirror those around data management more widely, as companies deal with a greater variety and volume of data sources, generated at a higher velocity than ever before. Challenges revolve around:
- The complexity of maps, particularly if manual methods are used to model extremely complex data structures
- The volume of data can increase the risk of human error leading to data quality issues
- The evolving nature of data maps, means they need to be regularly updated and checked
- The need to be able to cover all major data formats to provide a comprehensive solution
Opendatasoft’s data portal platform includes powerful data preparation tools, including 50 processors designed to map and standardize formats and fields, all without having to write a single line of code.
Learn more
How to break down organizational silos to engage everyone in your data project
Organizational silos prevent data sharing and collaboration, increasing risk and reducing efficiency and innovation. How can companies remove them and ensure that data flows seamlessly around the organization so that it can be used by every employee?
What the American Privacy Rights Act means for data governance
The American Privacy Rights Act (APRA) promises to increase the protection, privacy and security of consumer data, across the United States through federal legislation. We look at what it covers, next steps on its implementation and the benefits it brings to organizations as well as consumers.
How to break down organizational silos to engage everyone in your data project
Organizational silos prevent data sharing and collaboration, increasing risk and reducing efficiency and innovation. How can companies remove them and ensure that data flows seamlessly around the organization so that it can be used by every employee?
What the American Privacy Rights Act means for data governance
The American Privacy Rights Act (APRA) promises to increase the protection, privacy and security of consumer data, across the United States through federal legislation. We look at what it covers, next steps on its implementation and the benefits it brings to organizations as well as consumers.