INSIDER: Lessons from Developing a Monitoring and Evaluation Framework to Measure Local Adaptation Grants Impact in 14 Developing Countries

As global temperatures steadily rise and extreme weather events wreak havoc on communities around the world, climate change has begun to take center stage in international diplomacy. Much progress on resilience has been made in this arena – the Paris Agreement set the first-ever global goal on adaptation, spurring countries to develop and begin implementing resilience actions within their Nationally Determined Contributions while also moving ahead with their National Adaptation Plans (NAPs).

Yet progress at the local level, where successful adaptation actions must occur to drive concrete, lasting change, remains uneven. Many subnational decision-makers around the world grapple with a series of constraints – from insufficient resources and limited capacity to top-down public administration structures and restrictive funding priorities – that hinder their ability to understand and effectively manage climate risks. In least developed countries, these challenges are especially acute.

Understanding the critical role that local authorities play in building resilience to climate change, the United Nations Capital Development Fund's (UNCDF) innovative LoCAL program enables these decision-makers to overcome many of the obstacles that stand in the way of effective adaptation strategies. LoCAL channels performance-based climate resilience grants to local officials across 14 least developed and developing countries and provides much-needed capacity-building support. These performance-based climate resilience grants provide funding to local authorities who address adaptation needs in their communities, for instance, by climate-proofing a road that heavy rainfall routinely washes away. But still, available climate finance comes in short supply, and ensuring that these limited funds catalyze effective action is critical.

Robust monitoring and evaluation (M&E) frameworks that measure the impact of resilience programs can help by providing clear, systematic steps and processes to gather and analyze evidence. But new research from WRI conducted while developing the Assessing Climate Change Adaptation Framework (ACCAF), an M&E framework for LoCAL, reveals challenges that are characteristic of the broader thorny bundle of obstacles associated with climate change adaptation, such as the contextual nature of resilience and a lack of standard indicators to measure adaptation progress. This new WRI working paper, Assessing the Effectiveness of Climate Resilience Grants to Local Governments in Least Developed Countries, outlines practical approaches to some of these methodological challenges. It identifies three broader lessons learned from creating an M&E framework for resilience:

1. M&E systems for adaptation must also be adaptable. LoCAL's existing assessment system exemplified good practice for performance-based grants, but it did not adequately measure adaptation gains. Designing the ACCAF required building a framework that incorporated good practice from adaptation M&E, complemented the program's existing performance-based grants assessment system and fit well with the LoCAL program approach. Integrating good adaptation M&E practice into a complex, evolving global-yet-local program was not always straightforward. The process of developing the ACCAF for LoCAL required continuous engagement with UNCDF staff managing the program as well as pilot testing with country staff implementing the initiative to ensure that the final system worked well across different contexts. For instance, good adaptation M&E practice suggests that, when formulating indicators that assess grant-funded interventions, practitioners should tailor these indicators to reflect the specifics of the context and intervention.

However, because LoCAL is also a global portfolio of local initiatives, this process required creating some standardized indicators that, when aggregated, could help describe the facility's broader impact. Thus, the ACCAF ultimately offered a combination of intervention-specific adaptation outcome indicators that feed into periodic country-level adaptation evaluations alongside a set of standardized output indicators that the global team can use to describe the portfolio of interventions.

2. To measure the effectiveness of mainstreaming adaptation into development programming, it is important to demonstrate how interventions contribute to climate change priorities. We recommend that practitioners start by articulating a clear adaptation rationale for each intervention, substantiating why and how it will reduce vulnerability or increase adaptive capacity. While adaptation interventions can also support broader development aims, it is nevertheless important to justify why and how an intervention decreases climate risk when using funds intended specifically for adaptation. Having this clear adaptation rationale is especially important for mainstreamed adaptation, as such interventions are so closely linked with the development. It also enables the rest of the M&E system to function well, because it explains how an intervention is intended to reduce vulnerability or increase resilience to climate change.

Without such a rationale, it is difficult for an M&E system to demonstrate how effectively a specific grant contributes to adaptation aims. Yet developing clear adaptation rationales rests on stakeholders' abilities to confidently distinguish adaptation from business-as-usual development – a skill that requires training and capacity-building. Strong leadership and management at the global level within the program are essential to ensuring that such capacity-building initiatives are on-going rather than one-off trainings.

3. Accounting for a wide range of stakeholders' needs in adaptation M&E systems can be challenging, but it is also critical to success. In the LoCAL program, interventions occur at the local level across 14 countries, comprising a global portfolio. At each of the three levels – local, national and global – there are different stakeholders with specific information needs. The original ACCAF methodology was heavily grounded in country program experiences and perspectives, because LoCAL's tangible outputs occur at the local level. However, M&E focused at this level could not sufficiently represent the program's efforts as a whole. Revisions and additions were necessary to meet the needs of the global team.

To balance these requirements, the ACCAF included a methodology and data tracker to aggregate data across any intervention receiving grant funding in any locality in any country. Ideally, M&E practitioners should account for and weight different audiences' diverse needs at the beginning of the process to ensure clarity for the resulting system. This echoes research that highlights contrasting values, goals and judgment by different actors in adaptation evaluation.

Sociopolitical, ecological and economic dynamics shape effective resilience programs, and thus, the process of developing an M&E framework for adaptation initiatives will vary widely. However, these lessons are broadly applicable to donors developing or funding resilience programming, implementers and M&E specialists who collaborate with local authorities.