Today, we want to talk about some of the common challenges that face Open Data program managers. In our experience working with Open Data administrators, we’ve noticed several Open Data pain points emerge: data complexity, minimal Open Data releases, and data that nobody understands. A successful Open Data program needs to avoid a lot of pitfalls to stay afloat.
Pain Point #1: Data are too complicated (of the importance of user interface)
The first pain point we see is that Open Data are often too complicated. It can be very much far too tech-y sometimes. True, data are complex in general. But not all data need to be presented as such.
If we – Open Data solution providers – don’t make it easy for our users to deal with data, who will?
Facing the reality of data complexity should be a prime objective in an Open Data program because this can alienate a potentially large group of users. They may not have the patience to work with CSV files, or complicated interfaces.
You can turn these processors on and off as you wish without affecting the data. If you make a mistake, you can correct it in seconds. It is as easy as it sounds.
Remember, data acquisition can be a direct roadblock to an Open Data program, because your collaborators will be far less willing to open their data in the first place when it’s too difficult. We know that trying to build up a team, and build up a momentum around Open Data is one of the first steps in really creating a successful Open Data program. So just in removing this barrier of data complexity through a user-friendly interface can be a great first step to overcoming some of that first pain point.
Open Data Pain Point #2: Weak open data
In addition to that we’ve seen that Open Data can often be pretty weak. And what we mean by that is that some of the most common reuses for Open Data outside of a city government’s internal sharing can come from applications. These applications rely on a very handy piece of technology: APIs.
APIs are – amongst other things – a designated point of communication between data provided by a city (or a company) and applications. Instead of gathering the data herself, a developer can connect application to an API. She will use third-party data and focus on the service to be built on top of them.
A lot of Open Data solutions generate APIs from your data. However, APIs that come from these other Open Data solutions can be overly simple.
Often we’ll see that they’re only capable of searching for data sets, but not within data. What’s the point of knowing that a dataset exists when you can’t look at the data it contains?
Developers can often find ways around these roadblocks. But this just takes extra time that they could be spending making other reuses and bringing even further value to your data. In addition to that we see that a lot of APIs, in a lot of Open Data solutions, cannot process data in real-time. And yes, real-time data are important.
When we talk about a city service like transportation, this can be a real roadblock. Especially when you’re trying to add event-related data to your data portal, for example. When we’re talking about increasing accessibility to your city’s transportation network, these event-related real-time data are extremely valuable important for them to be somewhere in an accessible open manner for your community.
Pain Point #3: Nobody gets your Open Data
If you’re using a software that has a very complicated user interface, your end users (the public or developers) will have a much harder time knowing what they’re looking for, or looking at in that matter, as well.Thus finding that value that’s there in your data once they are even open.
We believe that data portals should also please the eye and allow non-techy users to have fun with the data at their disposal. Because, there’s more to Open Data than just machine readable data. There’s also the human or communicative side of Open Data.
Once your data are open, how do we transform all these really cool data sets into visualizations, and how do we weave all those into a narrative about something that our citizens are having a conversation about?
We put together a page for the City and County of Denver to illustrate how we could tell a narrative about something that is very topical in the city and county Denver. They passed an anti-camping ordinance somewhere early 2015 and they started enforcing it. And there was a lot of pundits, there were a lot of people talking about whether this was good or bad. But when started diving deeper into what people were saying, we noticed that they weren’t talking with data. They were talking from an emotive, emotional point of view. So pretty much opinion versus opinion, rather than fact. What we decided to do was create something that was based more on data, but also show the human side of how this actually affects a single individual.
We have worked on a lot of Open Data projects, and we’ve seen where they have succeeded and where they have failed. And where they have succeeded is when there is communication between the Open Data program and the community that it serves. We chose the Denver story because it was part of the city and counties’ conversation. Despite the fact that Denver might not like the story we were telling, we figured they seemed to want to be transparent. They seemed to want to use data to discuss policy issues. And what we think that this story tells, is the unintended consequences of the camping ban in the City and County of Denver.
But what we believe is that this story illustrates a very good point on powerful Open Data programs. And that powerful Open Data programs sometimes tell stories that are not entirely positive for the community in which the story was just published, like this one. But what it does do, when you have those kind of stories though, it could be it engenders trust. A trust that might lead to some policy changes down the road.
In the bigger picture, a lot of these data stories that come out of community conversations could be addressed with Open Data. That’s really where touch on the humanity of Open Data as well as the utility of it. It creates a base layer of fact on which to have a perhaps unpleasant discussion, but a discussion that can create real benefits for the community and the administration alike.
Obstacles (and pain points) are the way
At OpenDataSoft, we’ve listened closely to these problems that our customers’ face in trying to make the most of their data. We’ve decided that looking the other way was not an appropriate philosophy. For us, data should be accessible to the largest number of non-technical people, Open Data programs should be powerful, and Open Data should speak to all.
Want to know about how our philosophy as developed into a product? Check the full Open Data Pain Points webinar video recap!