Monitoring and Evaluation

Guiding questions

• What role do stakeholders play in your M&E (e.g. also in the light of a policy outcome?) What is their feedback on the CRA?

• Is learning ensured? How?

• Is new data available regarding the risks and/or the system? What else is needed (e.g., data, resources, competencies, research) to understand the risks better?

• How do you want to communicate your final outcome?

Monitoring and evaluation (M&E) is both a departure and ending point, as the goal of the CRA is to iteratively go through its process while aiming at continuous improvement. This step therefore also brings in another momentum of opportunities. The M&E step is essential for ensuring climate-related risks (key and less urgent risks) are effectively monitored and managed and that climate risk management efforts can have appropriate impact. It may include participatory processes and has a special focus on the learning component. Finally, communicating the final results of the CRA is an important outcome of the CRA.

The M&E step unites two separate but also connected sub-steps. While Monitoring has its focus on the process, Evaluation puts emphasis on the outcome. Monitoring therefore refers to the surveillance of climate hazards and their respective risks, whereas Evaluation aims at (re-)evaluating the Climate Risk Assessment itself.

Guiding questions - Monitoring

• Which indicators for M&E fit your case best?

• What is your envisaged reassessment period for monitoring climate risks?

• Do you want to focus on the M&E of the risks or also include respective adaptation measures?

• Are both, key and less urgent risks, included?

• How are key risks dealt with? Do necessary steps contribute to alleviation of the identified risk?

• What are key indicators, metrics, methods and means to track the risks, as well as the progress on adaptation and resilience-building?

In Monitoring, regions should consider Less Urgent Risks as well as Key Risks and observe them over time. This translates into putting them into context in relation to specific objectives, principles or needs that were set earlier (e.g., during Scoping or Risk Exploration). It also allows to benchmark the observations against a potential recurring CRA to understand de- or increases of risk as well as potential patterns over time.

On the other hand, Evaluation consists of an iterative reflection on the Climate Risk Assessment cycle and its robustness. The frequency of Evaluation is defined by the region and may vary according to Risk Outcomes, objectives or policy interest. Learning is of key significance in this step as evaluation may continuously generate new knowledge and data and can thus influence processes about, e.g., governance, policy and decision-making or relevant ministries and institutions. Depending on the issues and needs of a region, it can be beneficial to collect quantitative/qualitative data that is needed for improving future iterations of the CRA. It is crucial to understand what works well and what does not and act upon it– both in the CRA process, and in dealing with risks.

Guiding questions - Evaluation: Efficacy, impact and usefulness

• What worked well? What did not?

• How efficiently did you use your resources (e.g. time, staff, cost)?

• How might your efficiency have impacted the CRA process positively/negatively?

• How would you value the impact of the CRA regarding an improved understanding of risk (e.g. public awareness and engagement, institutional capacity, funding and investment)?

• How do you evaluate the applicability of the Risk Assessment for CRM practices (e.g. number of times used, acceptance by experts and stakeholder, translation of results into CRM)?

Selecting indicators for the M&E is crucial to summarize the output of the CRA and make it more valuable. Indicators should therefore be linked to specific objectives, sectors as well as stakeholder’s needs and can be assessed quantitatively or qualitatively. While it is difficult to provide a full set of indicators due to the heterogeneity of regions and risk situations, regions can define their individual indicators that are more relevant for their context and purposes.

Further indicators may be taken from the Sendai Framework.

Climate-related risks require indicators that embrace periodical reassessment and should refer to a baseline to track changes over time effectively. Depending on the focus of the M&E, indicators may change. Table 8 provides a selection of potential M&E indicators: Here, the process of Climate Risk Monitoring refers to both, workflow specific and unspecific indicators, whereas the evaluation of the CRA itself provides an initial set of performance indicators, i.e., efficacy, usefulness and impact.

Table 8 Non-exhaustive selection of indicators for the M&E process with a) workflow specific and workflow unspecific indicators for Climate Risk Monitoring and b) performance indicators for the evaluation of the Climate Risk Assessment itself.

a) Indicators for Climate Risk Monitoring Workflow nonspecific indicators Frequency, intensity, spatial distribution, and duration of climate events; public awareness and engagement; institutional capacity. Workflow specific indicators Workflow Indicators/impacts Flood Risk Number of overflows and related probabilities (100 year event, 500 year event) Losses and damages (injuries, hospitalisations, fatalities, livestock, crops, business, tourism, buildings, etc.) Disruption of public infrastructure Heavy Rainfall Risk Losses and damages (injuries, hospitalisations, fatalities, livestock, crops, business, tourism, buildings, etc.) Disruption of public infrastructure Overflow of sewage systems Flooding of low-land points (e.g., under bridges, underground parking lots) and surface water accumulation across an urban area Drought Risk Losses and damages to agriculture (crops, livestock) Availability of tap water Extent of agricultural areas and people affected Heatwave Risk Health impacts (hospitalisations and fatalities) Economic productivity Wildfire Risk Number of fires; extent of affected areas and people; loss and damages (injuries, hospitalisations, fatalities, agricultural and forest areas, farm and wild animals, buildings, tourism, etc.) Disruption of public infrastructure Snow Risk Disruption of public infrastructure Loss and damages (injuries, hospitalisations, fatalities, livestock, business, buildings, etc.) Windstorm Risk Disruption of public infrastructure Loss and damages (injuries, hospitalisations, fatalities, livestock, business, buildings, forests, etc.) b) Indicators for the Evaluation of Climate Risk Assessment Process Performance Indicator Examples Efficacy Efficient use of resources, e.g. time, staff, cost Usefulness Raised awareness, improved understanding of risk (Public awareness and engagement, institutional capacity, funding and investment) Impact Applicability for Risk Assessment and CRM (number of times used, acceptance by experts and stakeholder, translation of results into CRM)

Communication is part of the participatory processes and should thus be enforced throughout the whole CRA, to decrease the risk of generating plans and policies which are not relevant. However, at this stage it is especially important to communicate results to the public and/or key audiences with a focus on policy- and decision-makers. Regular reporting (e.g., interim reports about decision on workflows or relevant priority groups) or a continuous dialogue with policy- and decision-makers may enhance relevance and accountability of the CRA and the process.

Tip

The output of the M&E step can inform a report or plan to document and communicate results. This should also include marginalized and disproportionately affected groups (social justice/just resilience) as well as gender, educational or socio-economic aspects.

Tip

At this stage it can be relevant to reassess targets, objectives and responsibilities in national/sectoral adaptation plans or strategies and include them in the M&E process.