Transform your data catalog into an internal data portal to create greater value

As the volume and variety of data produced by organizations continues to grow, data catalogs are increasingly vital tools, providing organizations with the ability to document and organize their data assets. However, data catalogs alone are not enough to democratize usage and create value.

The creation of a data portal is the key to exploiting data and facilitating its reuse beyond experts and specialists within organizations. In this blog find out more about the limitations of data catalog solutions, and what you can do to generate greater value from your data at scale.

Faced with ever-increasing quantities of data, data analysts and data scientists need tools to sort and document their data assets. This is what a data catalog does, facilitating data collection, quality control, enrichment and analysis.

A data catalog is an inventory of all an organization’s data. In addition to bringing data together in a single location, the catalog contains metadata that describes these assets and enables users to determine whether or not it is relevant to their needs.

Implementing a data catalog delivers a number of benefits to organizations including:

• Documentation: all data is cataloged in one place, providing centralized information on every data asset.
• Search: thanks to filtering, sorting and search functions, users can easily locate information relevant to their needs.
• Time savings: as data is easier to find, technical teams save time when managing, using and exploiting data.

While the implementation of data catalogs is becoming a requirement for data-driven organizations, the technology itself has several limitations.

The majority of data catalog tools are relatively complex, codified solutions. To use these tools, specific technical skills are required, both for implementation and for data retrieval. This limits their use to trained experts.

However, in order to unlock real value from data across the organization, everyone needs easy access to data, not just experts with advanced technical skills. Data has to be available to all – for example to staff in HR, marketing, finance and operations, without requiring them to master complex new tools.

In addition to their lack of simplicity, data catalogs have a limited use. Essentially they provide a searchable metadata catalog. This makes it easier to identify data,
but this doesn’t go far enough. They don’t help to create new services or optimize decision-making using this data. Instead, organizations need to make data easily accessible, comprehensible and reusable, and, importantly, available to everyone on a self-service basis.

While a data catalog is vital for the development of a data strategy, it is insufficient on its own to deliver value at scale from your data.

To democratize data in your ecosystem, it must be:

• quality-enhanced, enriched and formatted. Data must be cleansed before being made accessible to all on a data portal. This creates trust with users and increases data sharing across your organization.
• accessible by all audiences that you choose via a single access point. Through powerful, intuitive search and filter functions, users with no technical expertise should be able to find ready-to-use data in just a few clicks.
• presented in formats that meet user needs through data visualizations, dashboards, graphs, or raw formats. Data must also be easily reusable with the organization’s business tools.
• documented with metadata (DCAT, DCAT-AP, INSPIRE) so that it can be understood by all and reused with confidence.
• available in formats that encourage reuse through the ability to export in common formats including Excel, CSV, APIs and as GIS data, such as GPX for smartphone mapping.

The Rouen Normandie metropolitan area decided to adopt a data catalog to manage its geographic data in 2017. A data directory already existed, but it was not updated, making access to the data relatively complex.

To take data sharing a step further, it decided to implement an open data approach to deliver two main objectives:

• Break down silos between departments and automate processes through connectors, web services and data harvesting. The aim was to make publishing data as easy as possible for data producers, and to strengthen data governance processes.
• Promote an open data approach in the region, thanks to a single access point to all of the area’s data. This supports greater public transparency and strengthens citizen engagement and participation in government. It also encourages data reuse and innovation: by improving knowledge of the area’s data assets, it’s easier to reuse the available information and therefore to innovate.

Schneider Electric’s technology stack includes multiple tools dedicated to data management, including data catalogs. But these did not allow the organization to truly democratize data and add value to its usage.

Schneider Electric therefore turned to Opendatasoft to set up an internal portal accessible throughout the company: the Data Library. Thanks to this internal data portal, everyone has full access to data. It means that data scientists can concentrate on analyzing data rather than preparing and processing it, while employees now have access to a series of customizable dashboards, and can share datasets internally.

However, this is not the only benefit of moving beyond a data catalog. Schneider Electric has also been able to create Exchange, a data services marketplace which brings together a wide range of data, resources and tools for sharing and monetizing its data with partners. This would have been impossible with a simple data catalog.

Data catalogs are indispensable tools for inventorying data and supporting data governance policies. However, they are not enough to make data easily accessible and consumable internally by as many people as possible. They don’t help accelerate the wider use of data or release its value. Creating a data portal enables organizations to take this next step, and unlock the full potential of data, helping to democratize its use and build truly data-centric organizations.