Open data in the Wallonia Region of Belgium is not as young as it may seem. There are still only a…
Open Data and Logic Models: The Pillars of Performance Management
There is currently a weak link between open data and performance management. During the Performance Management Conference at Suffolk University, hosted by Rutgers, I quoted Andrew Ballard, “Open Data is the Beauty Pageant of Performance Management.” It got a laugh, but the implications are serious. If open data efforts are to be sustainable, open data needs to be seen as a first class data type and used to manage performance. If open data were treated with the rigor that data are treated, there would be more confidence in using it to develop an ecosystem of applications. We are at that point where public sector agencies are assessing whether open data has a return on investment. This post will look at the performance management aspect of open data re-use, and what is needed to make open data suitable for performance management.
Performance management is a way of assessing whether a public sector agency is acting effectively, and access to data is only a piece of the puzzle. Only carefully governed data is actually useable, and hence, valuable. Here’s how a structured logic model strengthened with data allows for effective performance management.
It all Starts with a Logic Model and an Organization
The Anatomy of a Simple Logic Model
Below is a simple logic model that could be employed by a public sector agency.
Image from Ihttps://www.innonet.org/. Do-It-Yourself Logic Models: Examples, Templates, and Checklists Webinar with GrantStation. February, 2014.
Logic Models Have Goals
An effective logic model requires a problem, a solution, and how to get there. Or in Logic Model jargon: a problem statement, objectives (complete with KPIs), and a goal.
Data does not have to be tied to technology. Several years ago I worked as the web services lead for Durham Public Schools. We combined carefully governed data with focused logic models to solve a complicated problem. This was data curated by hand. Imagine how much easier it would have been had the technology been there to curate data on the fly.
The problem was that the schools were over student capacity and needed capital improvements. The plan was to hold a bond referendum to fund the improvements and new construction including several new schools. To calculate how much was needed, we began collecting data and rigorously updating KPIs. This process organically created a logic model.
The Durham Public Schools Bond Referendum Logic Model
Worked with stakeholders including residents, county board members, employees;
To pinpoint a problem statement of not enough room for all the students in the schools and capital improvements needed;
Then rigorously monitored the quality of the data, the data visualizations, and the engagement and re-use of the data by the stakeholders;
To ultimately reach a goal building out as necessary and improving the facilities;
By a bond referendum to raise the funds needed
We then turned to the data. The data supported the plan, and the referendum passed. But this is not the end of the story.
Fastforward to 2011. This is when something unexpected happened. While rigorously updating the KPIs, we realized there had been a miscalculation of capacity in one of the neighborhoods. The construction of the new high school was scheduled to begin soon, but the accumulated data over the past four years told us that the construction of a new high school was no longer necessary. Suddenly there was a surplus of $1.9 million.
So now there was a new problem statement: what do we do with this extra money? A Bond referendum is specific and prescriptive. Repurposing Bond dollars is not allowed, by law.
The proposed use of this $1.9 million was to update every student computing lab in the district to a state of parity. In order to inform the public on the new “problem” and gauge stakeholder support on our proposed goal, we built a special website where citizens could “tour” all the computer labs. We also published data detailing the state of all the computer labs. The measure to improve these facilities passed with overwhelming support. We had done all the background work with our logic models and data governance/accessibility. It was now possible for the rest of the stakeholders to easily analyze the data and come to the same conclusion.
Most importantly: The data was rigorously managed and the stakeholders were met with on a regular, timely basis.
Logic models and open data need to be used together for effective performance management. Both require active attention and updating. Simply opening data will not be useful without goals, and logic models won’t be accurate without the proper data and KPIs. So here’s to open data: a very useful tool, but not the magic cure for all your performance management woes. But who knows, you could end up with an extra $1.9 million!
The Link Between Open Data and Performance Management
These KPIs as described above in the logic model are great. What powers a key performance indicator? Data. How do we know this KPI is valid? We did our due diligence on managing the data. In the Summer of 2016 someone quipped aloud, “I would not measure my bungee cord with open data”. The event was an Open Data Round Table held at the White House. Ironically, the Round Table was about data quality. The suspicions are justified, and maintaining the quality of data should always be a top priority.
Why do we Bother to Manage or Measure Performance?
Joseph Wholey of the University of Southern California and Kathryn Newcomer of George Washington University observe that “the current focus on performance measurement at all levels of government and in nonprofit organizations reflects citizen demands for evidence of program effectiveness that have been made around the world”.
By using data to measure performance, we can begin to manage outcomes. This idea has been around for about 100 years in the US. By using KPIs and measurements, we can manage business units more effectively. To get those measures we need quality data, and open data is a great vehicle to encourage reuse by supporting outcome measurement and management.
Historically, performance management in the US comes at the end of the 19th century and a series of “spoils of war” forms of government. Political patronage and other types of graft and incompetence were hallmarks of 19th century US governance. Regardless of one’s political views, public sector in the US is under a constant microscope through accountability and performance programs. In spite of these accountability and performance programs, today there is a weak link between open data, data governance and performance management.
Technological advances continue to take hold in data publishing and management systems, including open data and data sharing platforms. Municipal leaders are seeing the important role of the spread and exchange of data and are changing their very decision-making processes. Public sector stakeholders are driving data publishing companies to build better, more intuitive products. The Civil Analytics Network published an open letter to the open data community requesting a greater emphasis on accessibility and usability features within data publishing platforms.
Are we Witnessing an Evolution in this Notion of Data Governance?
Yes and no.
Yes, in that more public sector organizations recognize the importance of measuring and managing government performance. New programs and certificates from universities are promoting the idea of data governance as a necessary, but not completely sufficient, condition for performance management.
No, in that open data programs are sometimes measured by the press release and the “wow” factor of complex and nicely shaded graphs and maps. Open data programs are not measured by their utility to the public (reuse). Open data programs are not usually linked to strategic plans or performance plans. Why is that? Probably because people that work in planning offices, budget offices, and police departments have more rigorous standards for their data governance than the open data program in their city.
So how can we reconcile the need for open data, visualizations and the need for sound data governance?
Data, data analytics, open data people, and data managers really boil down to one of two types:
- Descriptive: historical, data quality and data reporting activities.
- This is what performance management does in part
- Performance management relies on sound historical data to tell a story about the state of government now and how it is trending.
- Prescriptive: predictive, analytics, insight, data storytelling and data visualizations.
- From sound data governance, we can then choose targets and start to develop predictive models
- These models close the gap between where we are now and where we want to be
Effective links between open data and government performance have some common characteristics:
- Uses current data to analyze specific, previously defined aspects of recent performance
- Provides feedback on performance vs. targets
- Follows up on previous decisions and commitments to produce results and learn from efforts to improve
- Identifies and solves performance-deficit problems, and sets the next performance targets
We see that data ambassadors are an essential player in advocating and building support for a comprehensive data sharing strategy. In addition, we will explore the expected impact and benefits that result from a larger conception of data governance.
How do we define data governance and performance management? A city is on the right track with its performance management strategy if:
- It holds an ongoing series of regular, periodic meetings;
- These meetings should involve high powered stakeholders including:
- The city leader
- The principal members of the leadership team plus the individual director (and the top managers) of different city agencies
- The city leader
- Stakeholders must use data to analyze a given agency’s past performance
- This performance measurement then establishes the next performance objectives
- Finally, stakeholders must examine a given agency’s overall performance strategies.
There is More to Discuss
As the open data grows and matures, researchers and thought leaders are still experiencing difficulty in answering the question of overall societal benefits outside of individual and often isolated case studies.
This discussion will seek to further the idea of performance management by insisting that a data governance structure must be established in order to fully realize the potential of open data and data sharing.
As always I welcome my colleagues in the open data space, the performance management space and the public sector space to comment and continue the dialog.
Want to learn more about open data?
Check out our Open Data America initiative. We may have already created a portal showcasing how your city and OpenDataSoft can be partners, to re-use public information and co-create innovative solutions to improve your quality of life and your city.