Missing baselines: have recent increases in climate finance been exaggerated?
To what extent might increases in climate finance since 2009 be due to changes in reporting practices? We used a natural language processing model and backcast the results to find out.
The Independent Commission for Aid Impact (ICAI) has recently published a review of the UK’s progress towards its international climate finance (ICF) target of spending £11.6 billion between 2021/22 and 2025/26. The review highlights that one way the UK will look to meet this target is by “scrubbing” existing expenditure: reevaluating projects to see if any climate-relevant spending has been missed.
Unsurprisingly, this exercise didn’t yield much (around £215 million) relative to the overall target: the combination of an ambitious target and severe cuts to the official development assistance (ODA) budget in recent years has already provided a strong incentive to squeeze as much climate finance as possible out of existing programmes. But this raises an important question: how much of the increase in climate finance from wealthy nations since 2009 (the date that the US$100 billion climate finance target was agreed) is genuine, and how much has come from a change in reporting practices? To find out how significant this might be, DI trained a natural-language processing (NLP) model to identify which projects the UK’s Foreign, Commonwealth & Development Office (FCDO) projects were classed as climate finance in 2022, and applied this model to previous years.
How much of the increase in climate finance is genuine?
There has been a huge increase in reported climate finance since 2009. The value of ODA across bilateral donors with a Rio-marker applied increased from around US$6 billion in 2009 to US$30 billion, and in percentage terms, this was an increase from 7% to 26%. Clearly, this is partly a shift in spending patterns as climate finance became a higher priority. At the same time, increased political pressure to be seen to be taking action on climate change also gives providers the incentive to scrutinise projects more carefully to ensure everything relevant to climate is being counted, as in the case of the UK’s recent “scrubbing”.
This is something that DI has heard from multiple officials, as part of an ongoing programme of work trying to better understand how different countries’ climate finance figures are constructed. In the past, project managers may not have bothered to screen projects for their climate impact (especially if there are additional reporting requirements specific to climate finance projects), but over time, awareness of the importance of climate finance reporting has increased. While this suggests that accuracy has improved in some cases, it also means the increase in climate finance is exaggerated.
Take renewable energy. In 2022, around 91% of ODA to renewable energy generation was marked as climate mitigation, but in 2009, the figure was only 58%, and 27% wasn’t screened at all for its mitigation impact. If we apply the 2022 shares to previous years, as shown by the lighter line in Figure 1, then the increase in total disbursements marked as climate mitigation falls from 268% to 127% over this period. However, for most sectors it is not clear whether a change in the share of Rio-marked ODA denotes a change in reporting, or a real shift towards climate relevant spending.
Figure 1: Greater care in marking renewable energy projects as climate finance has exaggerated trend
Mitigation-marked ODA in the renewable energy sector, actual, and 2022 shares held constant
Line chart showing how greater care in marking renewable energy projects as climate finance has exaggerated trends
Notes: In 2022 there was a sharp increase in the share of bilateral ODA which had not been screened for mitigation impact. It is possible that this will be revised in subsequent editions of the CRS data.
Source: DI analysis of OECD CRS
Using machine learning to reassess old projects
We used a natural language processing model (NLP) to explore how reporting has changed over time, with the FCDO as a test case. Such a model can be trained to identify what sort of text in project descriptions distinguishes projects that are tagged as climate finance in the latest data. Then, it can predict what projects would have been classed as climate finance in previous years by comparing the similarity of previous project descriptions to those text patterns. (DI has previously used a similar model to compare the internal consistency of FCDO and World Bank climate finance reporting, using methodology developed by DI’s senior data scientist Alex Miller).
If reporting practices have changed over time, then we would expect a model trained on the most recent data to be less accurate in predicting what projects were classified as climate finance in the past. Specifically, the percentage of projects that the model classifies as climate finance that were actually labelled as such would be expected to decline as we go back in time. We used a NLP model to test this proposition in the case of FCDO. The model was trained on 4,000 unique project descriptions in 2022, taken from the International Aid Transparency Initiative (IATI) (split equally between climate and non-climate projects), and achieved an accuracy of over 98%, indicating a strong ability to accurately classify projects according to 2022 reporting practices.
Applying this model to projects from previous years indicates that reporting practices have indeed changed. Of the projects that the model has classified as climate finance, the share which the FCDO classified as such steadily declines as we go back in time. Figure 2 shows that in 2022, 85% of finance classified as climate finance by the model was tagged by FCDO as such, but in 2011-2014, this figure was around 51%.
Figure 2: By today’s standards, lots of FCDO’s previous climate finance was not counted
Percentage of FCDO individual projects identified as climate finance by NLP which were also classed as ICF by FCDO
Line chart showing that by today’s standards, lots of FCDO’s previous climate finance was not counted
Source: DI analysis of International Aid Transparency Initiative data
Assessing the exact impact that this has on total changes in the FCDO’s climate finance is difficult, as IATI data is not complete for previous years (IATI is a voluntary reporting platform). Figure 3 uses the available data to demonstrate how FCDO’s climate finance would look if the projects predicted as climate finance by the model had been included.
Figure 3: Potential impact of reporting changes on FCDO climate finance trend
FCDO climate finance reported to IATI, and alternative series adjusted for projects identified as climate finance by NLP model
Line chart, showing the potential impact of reporting changes on FCDO climate finance trends
Notes: Adjusted International Climate Finance (ICF) is the combination of actual ICF, and projects that were also identified as climate finance by the model. It is not the predictions from the model, which would also exclude false negatives (ICF projects not identified as such by the model).
Source: DI analysis of International Aid Transparency Initiative data
According to IATI data, climate finance that the FCDO has tagged as climate finance has increased from £703 million to £946 million between 2016 and 2022, an increase of 34%. However, if climate finance is adjusted to include those projects identified by the model, then the increase is only 7%.
The importance of accurate baselines for measuring progress
In a sense, this is good news, as these results suggest that the UK has spent more on climate-relevant projects than we thought: between 2016 and 2021, the UK was estimated to have spent around £5.8 billion on climate finance. This model suggests the true amount could be a third higher. In addition, the estimate of the impact that UK climate finance has had might be too low, as it only reviews projects counted as ICF.
On the other hand, the results also suggest that the upwards trend in climate finance has been exaggerated. If this finding applies across all countries then it has big implications for the extent to which climate finance is additional. If the same types of projects were happening in 2009 but just not being counted, then not only is climate finance not additional to the ODA budget, it isn’t even additional to past climate finance. When countries come together to agree on a New Collective Quantified Goal for climate finance at COP29, much greater attention needs to be paid to setting an appropriate baseline, and understanding how countries currently perform against it.
Our next steps
This study is a part of a larger body of work that DI is undertaking to analyse contributor reporting; identifying best practices and developing recommendations that can inform a more robust reporting framework for future targets. To hear more about our work, join our mailing list, follow us on Twitter or LinkedIn, or get directly in touch with questions or comments about the approach set out in this blog.
The author is very grateful to Alex Miller for sharing his code and expertise
Related content
How can AI be used to improve predictions of humanitarian need?
We used machine learning to project the possible impacts of different climate change scenarios on humanitarian need. What might the future hold?
Is climate finance wrongly reported by over a billion dollars per year?
We used AI to assess how the World Bank and the UK are recording climate funds. Our model identifies one in five of the Bank's projects as appearing suspicious and warranting further investigation, compared with the UK's one in 50.
Climate vulnerability, climate finance (ODA) and protracted crisis
Use our interactive chart to reveal vulnerability to the impacts of climate change, volumes of climate finance (ODA) by country, and how specific risk and geographic groups fare.