[NEW REPORT] The State of European Energy Data Maturity - In-depth research with E.DSO and GEODE

Download here
Product

Open Data as Terraces

Open Terraces - 0

Open data is nothing more than a way to spread data. As there are plenty of different kinds of distribution networks, we can imagine several ways to grant easy access to data that are ready to be reused. We’ve been working on such a distribution architecture, and it looks like terraces.

Head Of Engineering, Opendatasoft
More articles
Copy to clipboard

“A terrace is a piece of sloped plane that has been cut into a series of successively receding flat surfaces or platforms, which resemble steps, for the purposes of more effective farming”. Wikipedia

Among all the pros of terraces, there are three main characteristics that may, in my opinion, inspire us. These three points echo the issues that someone in charge of a data portal can face.

  • Better exposure. And when it comes to data, reaching its public and its re-users may be quite difficult. Google is a natural gateway but its algorithms will favor data coming from organizations good with SEO but not necessarily with releasing data.
  • Better irrigation. Basically when you manage data, you know that everything has a cost, every step requires work. So anything that allows you to better manage the resources and automatize work is interesting.
  • Simplifies a cultivator’s work, well at least the ground is flat. If we want more open data, we have to make the lives of the people in charge easier.
Copy to clipboard

Classical data distribution is pretty simple, generally coming in the form of a catalog of licensed datasets. Problems begin when you have a large number of datasets, multiple formats, and even more when you have various organizations sharing their data in the same place. It looks like a lake on top of a mountain or of a volcano. It’s a real passion for a lot of people: the trek is enjoyable, and what one sees at the top can be really interesting. But still, reaching the top might be inaccessible for most people.

Imagine someone who wants to compare his or her city with another one:

  • Step1: Find and go to the portal
  • Step2: search for the data
  • Step3: What format?!
  • Step4: OK, I know that Excel logo
  • Step5: open that file
  • Step6: How do I filter the data?
  • Step7: what’s my area ID??
  • Step20: reuse the data…
  • ಠ_ಠ

Others have tried to develop more advanced/user friendly open data portals. When the data are indexed before the distribution, filtering and searching becomes far more effective and the data are easier to find. Since they are indexed you can allow a user to try building maps or charts. They can truly see what the data look like before downloading them. That’s what we do at Opendatasoft but we are not the only ones.

How we like to see the data distribution on our portals

Copy to clipboard

The open data movement is still very young, and we may not have found the perfect way to distribute data in every cases.

So we decided to craft a new architecture for open data portals.

The whole architecture is based on the concept of sub-domains. Our customers are now able to generate sub-domains from their open data portals’ back office. These sub domains can be perfectly shaped to a company’s departments, mirror administrative divisions or to allow an industry to share data with various actors within its territory.

The sub domains are real open data portals by themselves: they are full-featured, they have the same graphic charter than the principal portal, they have their own administrators, their own data and their own ways to advertise the data.

All of this is quite interesting, but I think the key feature here is the ability to federate data in both ways. The principal portal can distribute datasets or parts of datasets (think ‘at the good granularity’) to each of the sub portals. And if any of sub portals load a new piece of data, the principal one can unify it. This is also a new way to decentralize and distribute the data collection!

Architecture of the sub-domain way to deal with data distribution

Copy to clipboard

Success follows a Power Law

A lot of thing in life follow a Power Law, including the distribution of downloads among open datasets. This become even more true when we speak about information technologies: in an economy where anything can be commoditized in a few years, network effects and data-network effects are stronger and exacerbate the distribution.

That is one of the key factors in open data’s strength. As you don’t know who will have the best design to better serve your citizen or reusers, you create the infrastructure so that anyone can use the data and develop it’s own solution.

Both by creating new distribution channels and decentralizing the ability to leverage the data, and you increase your chances to see new services created around your data and you strengthen your position in the ecosystem.

 

Copy to clipboard

open data has always been about empowering people: citizens, the people administering the portal, and gradually more and more people within an organization consuming the data.
Releasing data empower people. Giving certain people the opportunity to handle a data portal all by themselves with access to all the necessary tools for data cleaning, ETL processes, APIs, char and map building, is a new way to build some very strong links with your community or into your organization.

Make data even easier to discover

At Opendatasoft, our goal is to make the data move faster and faster. By developing a new way to distribute data, depending on the granularity of the data and involving more and more people, we do believe more data will find their re-users.

This article was first published on Medium

Articles on the same topic : Open data

Read more about Open Data
Scaling smart city projects beyond the pilot phase Public Sector
Scaling smart city projects beyond the pilot phase

Delivering smart city success starts with pilot projects to prove that concepts benefit the community. However, often projects fail to scale beyond their initial rollouts, meaning their benefits are lost. We explain the importance of data portals to maximize the chances of project success by seamlessly sharing information with stakeholders.

Taking the next steps with data portals in the Middle East Open data & transparency
Taking the next steps with data portals in the Middle East

Even more than in other areas data portals have a key role to play in delivering innovation, transparency and new services to citizens, businesses and governments across the Middle East. Based on best practice examples, we explain where should organizations focus when it comes to transforming their portals.

The importance of data governance to municipal data portal success Open data & transparency
The importance of data governance to municipal data portal success

What are the challenges municipalities face when it comes to effective data governance. We look at the importance of data portals and robust data governance programs to enable municipalities to securely share reliable, compliant data internally and externally.

Scaling smart city projects beyond the pilot phase Public Sector
Scaling smart city projects beyond the pilot phase

Delivering smart city success starts with pilot projects to prove that concepts benefit the community. However, often projects fail to scale beyond their initial rollouts, meaning their benefits are lost. We explain the importance of data portals to maximize the chances of project success by seamlessly sharing information with stakeholders.

Taking the next steps with data portals in the Middle East Open data & transparency
Taking the next steps with data portals in the Middle East

Even more than in other areas data portals have a key role to play in delivering innovation, transparency and new services to citizens, businesses and governments across the Middle East. Based on best practice examples, we explain where should organizations focus when it comes to transforming their portals.

The importance of data governance to municipal data portal success Open data & transparency
The importance of data governance to municipal data portal success

What are the challenges municipalities face when it comes to effective data governance. We look at the importance of data portals and robust data governance programs to enable municipalities to securely share reliable, compliant data internally and externally.