The Open Data Charter: A breakthrough for joined-up data standards

The Open Data movement is roughly ten years old. Starting in California in 2007 a number of attempts have been made to enshrine the principles of open data. I think it is fair comment that most of these charters have been drafted by and for constituencies in North America and Europe and have focused primarily on the relationship between western liberal governments and economies and the technically literate sections of their citizenry. Until now.

The international Open Data Charter launched in New York last month is the first globally inclusive and comprehensive manifesto of its type. It recognizes the north-south digital divide. It recognizes the challenges and needs facing developing countries. It embraces the data revolution for sustainable development data. As Jose Alonso from the Web Foundation has argued, it provides a canonical open data standard that can be embedded, adopted, and implemented by other related initiatives without duplication or fragmentation.

Of equal importance is the inclusion of the principle of “Comparability and Interoperability” –  the first time that a charter like this has moved beyond transparency to properly consider the whole point of open data: usage. Most datasets are pretty meaningless on their own. It is only when they are combined and contextualized do they generally make sense. Joined-up data creates the information we need.

You can only join up datasets if they are comparable and “speak the same language”. The Charter recognizes “that in order to be most effective and useful, data should be easy to compare within and between sectors, across geographic locations, and over time”.

And if you wish machines to do the joining up for you, the systems, formats, and definitions used by each dataset need to be interoperable: “data should be presented in structured and standardized formats to support interoperability, traceability, and effective reuse”.

Most of the useful data that we need today – even that which is already open – resides in a massive Tower of Babel. Global institutions – the UN, OECD, World Bank, and IMF among them – maintain countless incompatible and competing standards covering financial, administrative, geographic, and socio-economic data. So do national governments. Most open data portals are silos in which thousands of datasets sit in isolation from each other, each speaking their own distinct language or dialect. Continuing this language metaphor, there are two solutions to the problem: either agree to speak the same language or get an interpreter.

Principle four of the Charter first calls for “increased interoperability between existing international standards” – making sure we can translate accurately between standards. It then demands that we “support the creation of common, global data standards where they do not already exist and ensure that any new data standards we create are, to the greatest extent possible, interoperable with existing standards” – in other words: standards that speak the same language.

The importance of the Charter is that it provides us with a platform from which to launch a two-pronged attack on data silos.

We need to engage with all global and regional standard-setting institutions

• to persuade them of the importance of joined-up standards;
• to get them to cooperate with each other on resolving incompatibilities; and
• to get them to agree that all future standards will be properly joined-up.

And we need to stimulate the market to build more translation tools that will enable us to generate more usable and accessible information from existing data.

This blog was written as a part of the Joined-up Data Standards project, a joint initiative between Development Initiatives and Publish What You Fund.