The evaluation aims at identifying feasible policy options to improve risk reduction management in future drought episodes
RATIONALE
• Drought is a natural phenomenon that can evolve into a disaster with high economic and social costs
• Drought event occurrence is likely to increase in frequency and severity according to climate change predictions
• Drought management through risk reduction contributes to impacts prevention and mitigation
OBJECTIVES
• To assess past drought management strategies
• To understand current opportunities and pitfalls in addressing vulnerability and drought risk reduction
• To determine policy gaps and integration needs in present public drought policies
• To identify good practices for drought management
APPROACH
• Integrative approach, sensitive to the socio-economic, institutional and environmental context
• Ex-post evaluation: it recapitulates and judges an intervention when it is over
• Analysis from a Case Study perspective
• Analysis of two past drought events per Case Study
• Evaluation timeframe: Oct 2011-Oct 2012
Drought is a recurring natural phenomenon that can evolve into a disaster. It is about water, humans, environmental and socio-economic systems. During the past decade, there has been a shift from facing droughts using an emergency approach to framing them into disaster risk reduction policies. Has this shift improved our policy and institutional responses to droughts?
Existing drought policy assessment initiatives mainly focus on descriptive analysis of responses or on modeling of specific drought impacts. They rarely consider ex-post evaluation or look at drought responses from a systemic perspective. An integrative evaluation approach based on a Theory-driven Evaluation Model (TEM) is being developed to deal with these evaluative gaps and will be tested in six case studies in Europe.
Theory-driven Evaluation Models have been rarely applied to nature-related policies. They allow for the systematic analysis of the socio-economic, policy/institutional and environmental context and give valuable insights into what and why measures have functioned or not.
CASE STUDY
Case Studies were selected to include a range of different
• Scales (macro-micro levels): national- river basin – local context (island)
• Drought typologies and hydrological characteristics
• Socio-economic, environmental and management characteristics
STEP 1 RECONSTRUCTION OF THE PROGRAM
THEORY
TEM EVALUATION PROCESS
STEP 2 DIAGRAM THE LOGIC MODEL
.
. . . .
.
Netherland Switzerland Portugal Jucar River Basin Po River Basin Syros Island
.
.
. .
.
.
ESA; Photo: ESA/Envisat; Image: ESA/NASA - Europe
POLICY AND INSTITUTIONAL RESPONSES TO DROUGHT: DO THEY WORK AND WHY
J. Urquijo & L. De Stefano Departamento de Geodinámica, Facultad de Ciencias Geológicas
Universidad Complutense de Madrid (UCM) Madrid, Spain
Poster presented at International Water Security Conference WATER SECURITY, RISK & SOCIETY, Oxford, UK, April 2012
ACKNOWLEDGEMENT Research supported by the European project DROUGHT R&PSI: Fostering Drought Research and Science-Policy Interface www. http://www.eu-drought.org
CONTACT INFORMATION Julia Urquijo –PhD candidate: [email protected] Lucia De Stefano – Associate Professor: [email protected] Facultad de Ciencias Geológicas – Depto. Geodinámica Universidad Complutense de Madrid - SPAIN
The logic model summarizes the theory of how the intervention works
It also helps and guides the emergence of the evaluation questions, that must correspond to a real need of information by the stakeholders and should cover all the dimensions of the evaluated policy
A Theory-driven Evaluation Model (TEM) refers to a variety of ways of developing a causal modal linking programme inputs and activities to a chain of intended or observed outcomes, and then using this model to guide the evaluation (Rogers et al., 2000)
STEP 3 EVALUATION
MATRIX
STEP 4 DATA COLLECTION,
ANALYSIS & REPORTING
• Primary and secondary data
• Questionnaires
• Qualitative data from interviews and stakeholders group discussion
• SWOT analysis results
CASE STUDIES DATASET
• 120 Surveys
• 60 Interviews
• 6 SWOT analysis
• 12 Best practices
Need for a common framework of analysis: establishment of a comprehensive and integrative framework, structured in two levels:
• General: Policy – Institutional – Organisational framework
• Operative: DROUGHT MEASURES as the unit of analysis
STAKEHOLDERS INVOLVEMENT
A diagram or visual scheme of the intervention theory or model organizes such information into different dimensions or components of analysis:
Mixed methods that include quantitative and qualitative techniques and triangulation will be used to analyse the results.
• Outputs: effects that are under the entire responsibility of officials/program
• Outcomes: the responses of target groups to these outputs
• Impacts: the ultimate effects of the intervention
INPUTS
ACTIVITIES
(Measures)
OUTCOMES - IMPACTS OUTPUTS NEEDS
CONTEXT
ACTORS
CAUSAL RELATIONS
1. Structural elements: All what is necessary for the
program to start and function
2. Process: Sequence of activities which represent the manner in which
the program is implemented
3. Results:
STAKEHOLDERS INVOLVEMENT
STAKEHOLDERS INVOLVEMENT
Evaluation matrix
Toward a risk management approach?
EVALUATION PROGRESS
The evaluation framework and tools have been designed and pre-validated in one Case Study (Jucar basin, March 2012)
Case Study data collection in progress
The evaluation process finalizes in October 2012
LESSONS LEARNT SO FAR
The logic model is a simplification of a complex reality but it helps making the intervention logics explicit
The evaluation process leads to an improved understanding of the mechanisms by which the program produce effects
Tailoring the approach to the circumstances is a critical step when applying a TEM approach to a risk-related policy
Working at different scales (Case Study) requires a flexible evaluation framework
The establishment of a common understanding of drought-related concepts, drought measures and effects typology is key
Drought management responses are mainly considered as a water management issue only
Systematic analysis and ex-post evaluation is lacking even in advanced risk-management settings (Júcar River Basin CS)
INTRODUCTION
METHODOLOGY
CONCLUSIONS
REFERENCE Rogers, P.J. 2008 Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions. Evaluation 14, 29-48
• Literature review
• Stakeholders perception survey
• Semi-structured interviews
• Stakeholders group discussion
• Specific for each CS
• Based on: Review of previous
research/literature Expert opinion Stakeholders
perception
• 17 evaluation questions
7 on structural dimension
5 on process
5 on results
Group discussion with Jucar river stakeholders. Valencia, March 28th, 2012
• Ad hoc criteria Participation and
coordination Integration and policy
coherence Resources availability Effectiveness &
relevance Contribution to
impact reduction
Fight-fighter who tries to
estinguish a drought...
EVALUATION QUESTIONS
CRITERIA INDICATORS TECHNIQUES SOURCES OF
INFORMATION