[REPLAY] Product Talk: Using AI to enhance the data marketplace search experience

Watch the replay
Glossary

Reference Data

Reference data is static, or semi-static, data used to classify or categorize other data, ensuring consistency and standardization in data management.

What is reference data?

Reference data is static, or semi-static, data used to classify or categorize other data. It is an important part of data standardization, ensuring that data is described consistently across an organization and beyond.

Reference data can come from external sources (such as regulators or standards organizations such as ISO), or be created internally.

Examples of external reference data includes:

  • Country codes
  • Measurement units
  • Currencies
  • Calendar and time structures
  • Postal/Zip codes
  • Airport codes

Examples of internal reference data includes:

  • Financial hierarchies
  • Products numbers/references
  • Names/descriptors of offices or factories

What is the difference between reference data and master data?

While similar, there are major differences between reference data and master data as their definitions and use demonstrate:

  • Master data is data shared across the organization to drive business processes. It could be customer or product names for example and can change quickly, such as when a new customer comes on board.
  • Reference data is more stable, standards-based data used for classification and categorization of all data within an organization.

What is financial reference data?

Financial reference data is a specialized subset of reference data. It describes the specific information used to categorize financial transactions, such as the sale of a security or bond. This data is vital to the seamless operation of financial markets, as it provides a method of automating transactions between different entities. Examples include:

  • Instrument identifiers (the type of instrument being traded)
  • Market identifiers (where the trade took place)
  • Counterparty identifiers (which legal entities were involved in the trade)

While much financial reference data started at a local country or market level, there are increasing attempts by regulators to standardize it globally to aid reporting, monitoring and compliance and to prevent fraud and financial irregularities.

Why is reference data important?

Applying reference data enables organizations to classify their data, ensuring consistency across all data sources. This delivers multiple key benefits, including:

  • Higher data quality, as all data is classified correctly and consistently
    A shared understanding across the organization. As all data follows the same formats and classification, staff in different departments can easily understand exactly what it means, driving greater reuse
  • Stronger governance. Reference data is a key element in ensuring successful data governance across the organization
  • Higher levels of trust. All staff have trust that the data they are working with is consistent and can easily be compared between datasets
  • Greater efficiency. Data is automatically standardized through the application of reference data, avoiding the need for time-consuming manual changes. It also means that transactions can be automated, and will not be queried due to discrepancies, particularly in financial markets
  • Reduced risk. Standardization using reference data reduces risk, as it provides a clear audit trail around data that can be used with auditors and regulators
  • Easier reporting. As data is consistent it is straightforward to compare and report using business intelligence and analytics with greater confidence

What components do you need to successfully manage reference data?

Reference data is a key element in a successful data management strategy. Managing and applying reference data requires the following components to be in place:

  • Data governance standards and processes to ensure that data is accurate, complete, valid, timely, consistent, available and compliant with regulations.
  • Data lineage processes to understand where reference data has been created, how it moves through systems and to manage any changes that are made
  • Workflows to manage the application of reference data and to flag any inconsistencies
  • Set rules and responsibilities, with data stewards in place for individual datasets and clear processes for the application of reference data
  • Management, monitoring and compliance policies and frameworks

What are the steps in the process of managing reference data?

Successfully applying reference data requires a multi-step process:

  • Firstly, formalize which reference data you will use and where. This involves auditing data to see which classifications are needed across your organization
  • Decide which external reference data you will apply (such as ISO standards). Using external standards provides greater interoperability with your ecosystem and removes the need to create your own reference data
  • For the areas that external reference data is not available or relevant, create your own internal reference data sets
  • Educate and train everyone using data on the importance of reference data and where definitions are located
  • Apply reference data to existing data, as well as all new data being created. Monitor to ensure compliance and manage any inconsistencies
  • Keep reference data updated. This is particularly important for external standards as changes are made regularly and need to be applied to your own reference data models. Version control your reference data, so that you can see which changes have been made and when.

 

Download the ebook making data widely accessible and usable

Learn more
Accelerating public sector data sharing – best practice from Australia Public Sector
Accelerating public sector data sharing – best practice from Australia

Data sharing enables public sector organizations to increase accountability, boost efficiency and meet changing stakeholder needs. Our blog shares use cases from Australia to inspire cities and municipalities around the world

Opendatasoft integrates Mistral AI’s LLM models to provide a multi-model AI approach tailored to client needs Product
Opendatasoft integrates Mistral AI’s LLM models to provide a multi-model AI approach tailored to client needs

To give customers choice when it comes to AI, the Opendatasoft data portal solution now includes Mistral AI's generative AI, alongside its existing deployment of OpenAI's model. As we explain in this blog, this multi-model approach delivers significant advantages for clients, their users, our R&D teams and future innovation.

Using data to drive innovation across the Middle East Data Trends
Using data to drive innovation across the Middle East

The recent GITEX event in Dubai provided the perfect opportunity to understand how data sharing is changing across the Middle East. Based on our discussions, this blog highlights 5 key themes driving data use in the region.

Start creating the best data experiences