Lost in translation: Why joined up data matters at a national level

This is a guest blog written by Danny Walker. Danny is a Data Analyst at Development Gateway (DG), a non-profit that uses technology and applied research to support data-driven decision-making. At DG, Danny has led the quantitative component of the Results Data Initiative, which includes advising development partner organizations on how to improve the transparency and accessibility of results information.

Despite the immense potential in – and amount of – data in international development, collecting and analysing this information in a meaningful and holistic way remains a challenge. As the Joined-Up Data Standards project acknowledged in its recent consultation paper, as countries seek to localise the Sustainable Development Goals, joining up information will only become more important. This this will require joining up data for the development community, as well as joining up the technical standards that they engage with and use.

If you search for population growth rates worldwide, you may find something interesting: three distinct data sources. Furthermore, these sources – the World Bank’s World Development Indicators (WDIs), the Central Intelligence Agency’s World Factbook, and the United Nations’ World Population Prospects – all have slightly disparate numbers.

Dig a little deeper, and you’ll find something even more interesting – even surprising, depending on your statistical experience. These three data sources are themselves derived from a range of distinct yet, at times, overlapping sources. Digging into WDI metadata, for example, shows that the top-line national population numbers are based on:

1. United Nations Population Division. World Population Prospects
2. Census reports and other statistical publications from national statistical offices
3. Eurostat: Demographic Statistics
4. United Nations Statistical Division. Population and Vital Statistics Report (various years)
5. U.S. Census Bureau: International Database
6. Secretariat of the Pacific Community: Statistics and Demography Programme.

So what does this mean? At Development Gateway (DG), we’ve explored before how differences in data compilation can cause challenges. When we’re talking about joining up data – in this case, surveys – varying sources often means varying methodologies, and limited comparability. Barriers to joining up survey data can include:

• varying sample sizes
• different topical focus areas
• survey time intervals
• ranges of data-gathering techniques and technologies. (This interoperability barrier has significant implications, particularly at the national level.)

For national statistical offices and other data-gathering agencies, the myriad local, bilateral and multilateral organisations collecting data – often in parallel – can lead to additional interoperability challenges. Incentives for coordination across data producers are weak; it’s often more expedient to create your own fit-for-purpose system than try to build consensus around interoperability. This expediency often leads to investments in standalone data-gathering mechanisms and platforms, rather than a government-led joint system.

DG’s research has also shown that, too often, development actors prioritise bottom-up reporting, instead of front-line data relevance. This leads to heavy local level data gathering burdens and information with limited value for service delivery. An example from our work in Tanzania follows:

Our friend … founded and operates an HIV clinic that serves hundreds of patients. [He] gets funding from multiple donors, and as a result, has to fulfil a number of indicator reporting requirements. This often means his staff must fill out multiple (yet similar) reporting templates.

To give an example of these donor requirements, the doctor spoke at some length about the HIV prevalence indicators that he must report to donors, and about how he instead needs data about HIV incidence to ensure his clinic’s effectiveness.

[The] doctor’s point is pretty clear. In addition to collecting and reporting donor reporting requirements … his staff also must collect and record separate data to ensure the clinic’s effectiveness.

So what’s to be done? As part of DG’s Results Data Initiative we have engaged qualitatively and quantitatively with these issues to better understand data flows and use, and to discover feasible solutions to challenges expressed by local-level data producers and users.

For the challenge of unsynchronised data gathering, our key recommendations are threefold:

• First, we recommend that the development community works to streamline indicators. This would include assessing the essentiality of each indicator, collaborating with other agencies to reduce duplication and, perhaps most importantly, basing indicators on national government priorities.
• Second, we would recommend that the development community takes a more proactive approach to making data available to government agencies. Without regular and consistent access to data and metadata, it is nearly impossible for governments to aggregate information across data-gathering entities, let alone to compare information across geographical or administrative areas where data-gathering methods may have differed. Only by systematic data and metadata sharing can indicators be both streamlined and compared.
• Finally, for the data revolution to move forward there is a clear need for targeted investments in national data systems and staff. These investments, combined with more strategic coordination and alignment of statistical and results data collection, could represent a significant step forward in having harmonised, actionable data to inform policy and improve lives.

This blog was written as a part of the Joined-up Data Standards project, a joint initiative between Development Initiatives and Publish What You Fund.