Glossary
Data Mapping
Data mapping is the process of matching data fields from one database or information source, to related fields in a destination source or system.
What is data mapping?
Data mapping is the process of matching data fields or elements from one database or information source, to related fields in a destination source or system. It provides a set of instructions that help standardize data, and therefore is a vital part of ensuring quality, following data governance processes and making data useful when shared between the different databases and tools used within a tech stack.
For example, your corporate standard for US states may be to refer to them by their two-letter abbreviation (e.g. TX for Texas). In this case data mapping identifies and links the “state” field in a data source that you are connecting, and transforms it to the appropriate format.
Why is data mapping important?
Data mapping provides a visual view of how data moves around an organization. It is a fundamental step in good data management and enables organizations to:
- Ensure that information is consistent and follows corporate data governance standards
- Prevent errors and duplications which impact quality by merging related records (such as records of the same customer held on different systems). The process can flag potential issues (such as missing data) before it becomes a problem
- Enable the integration of data between different systems
- Make it easy to share data seamlessly internally and externally
Data mapping is central to these key processes:
- Data migration – moving data from one system to another on a permanent, one-off basis
- Data integration – integrating two systems so that data flows between them on an ongoing basis
- Data transformation – changing data as it enters a system, by applying rules to enable consistency or to add value
- Data warehousing – centralizing data in a single data warehouse, ensuring it is consistent and accurate
- Business intelligence – analyzing data for reporting, to uncover insights and to better understand business performance
- Data portal – sharing data internally and externally via a single, centralized data portal that delivers consistent, integrated and reliable data
What is the data mapping process?Data mapping can be carried out in one of three ways
Manual data mapping
This requires every data connection to be created individually by a data professional writing specific code to map databases and fields. While this delivers a customized, tailored approach it is extremely time-consuming and requires data teams to have specific skills to write and maintain code.
Semi-manual data mapping
This approach relies on graphical tools that enable data professionals to create schema maps by manually linking fields via a visual interface. This then generates an output script used for coding. Semi-manual data mapping delivers flexibility, but still requires specialist coding knowledge and significant amounts of time.
Automated data mapping
Most modern data mapping tools now enable full automation of the process, without requiring any coding at all. This lowers the barrier to entry for users, meaning they do not need to have programming skills. Automated data mapping can be scaled quickly, with regular refreshes to capture any changes. The only downside is that companies need to pay for tools and training.
What are the benefits and challenges of data mapping?
The benefits of data mapping
By providing the foundation for data management, data mapping delivers significant benefits around:
- Trust – employees trust that data is correct and reliable, and are therefore more likely to use it in their jobs
- Reduces silos – it brings together data from different departments and standardizes it, removing silos and barriers to sharing
- Security/GDPR compliance – it provides a full record of how data is mapped across the organization, including its uses. This helps with security and demonstrating regulatory compliance
- Faster, more comprehensive reporting – data is available for analysis much more quickly, with much fuller access to provide greater depth
The challenges of data mapping
The challenges to successful data mapping mirror those around data management more widely, as companies deal with a greater variety and volume of data sources, generated at a higher velocity than ever before. Challenges revolve around:
- The complexity of maps, particularly if manual methods are used to model extremely complex data structures
- The volume of data which can increase the risk of human error leading to data quality issues
- The evolving nature of data maps, which means they need to be regularly updated and checked
- The need to be able to cover all major data formats to provide a comprehensive solution
Opendatasoft’s data portal platform includes powerful data preparation tools, including 50 processors designed to map and standardize formats and fields, all without having to write a single line of code.
Learn more
The importance of data governance to municipal data portal success
What are the challenges municipalities face when it comes to effective data governance. We look at the importance of data portals and robust data governance programs to enable municipalities to securely share reliable, compliant data internally and externally.
Data Trends
New Opendatasoft research finds data democratization still in its infancy
How are organizations embracing greater data sharing and reuse? The latest Opendatasoft/Odoxa Data Democratization Study highlights that while organizations are becoming more mature in their use of data, there’s still a way to go to enable data-centricity.
What is cloud-based data governance and why is it crucial for companies?
Data governance is critical to ensuring that data is reliable, trustworthy and accessible by the right users, enabling organizations to become truly data-centric. Ensuring that cloud-based data is well-governed brings new challenges around control, security and compliance - this blog explains how to overcome them.
The importance of data governance to municipal data portal success
What are the challenges municipalities face when it comes to effective data governance. We look at the importance of data portals and robust data governance programs to enable municipalities to securely share reliable, compliant data internally and externally.
Data Trends
New Opendatasoft research finds data democratization still in its infancy
How are organizations embracing greater data sharing and reuse? The latest Opendatasoft/Odoxa Data Democratization Study highlights that while organizations are becoming more mature in their use of data, there’s still a way to go to enable data-centricity.
What is cloud-based data governance and why is it crucial for companies?
Data governance is critical to ensuring that data is reliable, trustworthy and accessible by the right users, enabling organizations to become truly data-centric. Ensuring that cloud-based data is well-governed brings new challenges around control, security and compliance - this blog explains how to overcome them.