(Photo Credit: istockphoto)
A handful of cities have invested considerable time, energy, and tax-dollars into building an exciting new kind of public infrastructure only for it to be abandoned and fall into disuse shortly after completion. Like the ghost-town villages and crumbling stadiums of Olympic host cities, open data portals were built and celebrated only to be forgotten after the thrill of newness faded away. Local governments are reacting and responding to demands from their constituencies to be more open, accountable, and effective.
Today, many portals have become a cache of unorganized and aging government data. Far from the heralded hub of modern local government, open data portals are now yet another layer of government complexity purposeful only to a small and specialized set of people.
When portals are used, it’s by journalists whose job it is to sort through dozens of datasets before giving up and inevitably submitting an open-records request. Or by watchdogs, searching for scandals in pay schedules and personnel data. Not exactly the utopic front door to local government that was promised.
Of course, there are exceptions. New York City, Philadelphia, and New Orleans have leveraged their data to analyze crime patterns, create open APIs for property tax data, and streamline public transit through consumer-facing apps.
But, how’s open data doing in a broader sense? The answer isn’t so clear.
The State of Open Data
First, let’s start with the good news: 12 of the top 15 largest cities in America have an open data policy. That’s not a small feat. Socrata, the most popular open data platform vendor on the market, serves over 300 cities including New York City, Seattle, and San Francisco. From a pure saturation perspective, open data advocates have certainly found an audience ready to adopt its ideas.
Local governments are reacting and responding to demands from their constituencies to be more open, accountable, and effective (Lake Research Partners, 2009). It’s easy to see how open data could be the perfect panacea to those demands to city leaders everywhere. It’s a relatively intuitive band-aid to systemic and institutional challenges—ambitious city politicians loved it, of course.
What have the actual results been? The National League of Cities released a report on open data in 2014 finding that when it came to measuring success and outcomes, things got really fuzzy for many places. Municipalities had a hard time choosing the right metrics to even determine what success actually looked like.
“An important limitation found in most of the existing open data policies is the lack of formal tools or methods of evaluation to measure the benefits and outcomes achieved,” the National League of Cities report found. “Developing evaluation tools should be an integral part of any future open data policies.”
Most use the obvious metrics: how many datasets are hosted on the portal, how many datasets have been downloaded, and total number of active users. Essentially, cities determined that raw usage on the portal itself (uploads, downloads, and users) makes for a successful open data policy. And therein lies the core problem; we’ve missed what success ought to look like and thus ended up with flawed metrics.
César A. Hidalgo is an associate professor of Media Arts and Sciences at the Massachusetts Institute of Technology, and he tackled why this is problematic from a design perspective. Writing for the Scientific American, Hidalgo compared usage-driven portals to a nightmarish supermarket where everything is stored in boxes that look exactly the same.
“Most open data sites are badly designed, and here I am not talking about their aesthetics —which are also subpar—but about the conceptual model used to organize and deliver data to users. The design of most open data sites follows a throwing-spaghetti-against-the-wall strategy, where opening more data, instead of opening data better, has been the driving force.”
Just as one wouldn’t determine if a public works department were effective by the total volume of concrete it poured, an open data policy should not be labeled successful based on its data output alone.
The Right Metrics
If not uploads, downloads, and users, what should we be measuring for? Or put another way, what should a successful open data policy look like?
Outcomes need to replace outputs. Effective open data policies should be designed to help cities solve public problems, improve government services, and empower residents. By working backwards from those outcomes, we can arrive at measurable and meaningful goals. At its best, open data should strive to further our understanding of how our governments operate and inspire them to work even better—more efficiently, effectively, and equitably.
A recent report issued by Govlab recommends creating a “Metrics Bank,” with input from stakeholders, researchers, and experts. It’s easy to imagine this Metrics Bank having several categories built around key themes, such as economic issues (return on investment, private sector revenue generated) or public services (increases of efficiency in services delivered, hours of work saved).
The authors suggested that the Metrics Bank should be periodically reviewed and updated by a citizen group or panel dedicated to the purpose. This keeps open data policies agile and thus ensures its longevity by recognizing that the demands of government can and do change.
Open data can be far more than a simple inventory of unorganized public data, and its success stories demonstrate just that. But local leaders have to go beyond designing policies that look good on paper, deploying a relatively inexpensive portal and popping the champagne. This approach to open data ensures it will become yet another clunky government service that is underutilized and difficult to navigate—the exact problem it hoped to solve.
Avoiding that fate means thinking beyond just portal and policy. A holistic and inherently more successful approach means treating data as a vital form of public infrastructure in the 21st century; that requires as much of a cultural change as a policy change.
At its best, open data should strive to further our understanding of how our governments operate and inspire them to work even better—more efficiently, effectively, and equitably. To reach that potential, however, we have to push past the outputs and rethink what success can look like.
Joda Thongnopnua is executive director of the Metro Ideas Project. He was a participant in the 2016 Socrates Winter Seminar “Who Are Cities For? Opportunity, Access and Hope in the American City.” Socrates Seminars are held annually as part of the Aspen Institute Socrates Program. Visit the Socrates Program page to learn more about upcoming seminars.
