1 external relations evaluation unit result and impact indicators for the road sector

1

External Relations Evaluation Unit

Result and Impact Indicators for the Road

Sector

2

Why are Result & Impact Indicators Needed?

To better understand the positive/negative results of EC aid.

The main questions are:1. What change is needed?2. For whom?3. How to measure this change?

3

Why are Result & Impact Indicators Needed?

1. Assist EC country teams to develop a set of indicators for the programming level (e.g. to guide development and monitoring of a Country Strategy Paper)

2. To fill, as much as possible, the 'missing middle' between implementation indicators and global impact indicators

4

The Missing Middle

Outputs Results(Outcomes)

Specific Impacts

Intermediate Impacts

Global Impacts

Implementation Programming

RoadsRoads Poverty Poverty ReductionReduction

5

Common Indicator of Success

6

Examples - Real Indicators of Success

Time

Cost

Education

Environment

Health

Trade

Jobs/Profits

7

Intervention Logic

Institutional Reform

Social

Environment

Infrastructure(construction/ maintenance)

Economic

Improved Institutional

Management of Road System

Lower Road Transport Time

Lower Road Transport Costs

Increased Employment

Enhanced Safety & Health Linked

to Roads

Environmental Impacts of Roads

Minimised

Greater Accessibility to Social Services

Greater Economic

Accessibility Improved

Conditions for Economic Growth

Economic Growth

Social Development

Poverty Reduction

OutputClusters

Results(Outcomes)

SpecificImpacts

IntermediateImpacts

GlobalImpacts

Enhanced Human Capacity

MDG 7

MDG 6

MDG 1

8

Result (Outcome) Indicators

Institutional Reform

Infrastructure(construction/ maintenance)

Economic

Improved Institutional

Management of Road System

Lower Road Transport Time

Lower Road Transport Costs

Increased Employment

3. Implementation and enforcement of laws related to roads

1. Travel Times

2. Transportation costs

4. Employment resulting from construction and maintenance of roads

OutputClusters

Results(Outcomes)

Indicators

Environment

Social

9

Specific Impact Indicators

Improved Institutional

Management of Road System

Lower Road Transport Time

Lower Road Transport Costs

Social

Environment

Enhanced Safety & Health Linked

to Roads

Environmental Impacts of Roads

Minimised

Greater Accessibility to Social Services

Greater Economic

Accessibility

OutputClusters

SpecificImpacts

10. Road deaths and injuries

11. Disease transmissions influenced by improved mobility

9. Environmental effects of road system

5. Access to roads

6. Use of public transportation

7. Business productivity

8. Access to social services

Indicators

Economic

Institutional Reform

Infrastructure(construction/maintenance)

Results(Outcomes)

10

Intermediate Impact Indicators

Increased Employment

Enhanced Safety & Health Linked

to Roads

Environmental Impacts of Roads

Minimised

Greater Accessibility to Social Services

Greater Economic Accessibility

SpecificImpacts

IntermediateImpacts

Improved Conditions for

Economic Growth

Indicators

Enhanced Human Capacity

14. Employment

15. Trade

16. School attendance

17. Health attendance

Outputs

Environment

Social

Infrastructure (construction/maintenance)

Institutional Reform

Economic

Lower Road Transport Time

Lower Road Transport Costs

Improved Institutional

Management of Road System

Results(Outcomes)

11

The Missing Middle

Outputs Results(Outcomes)

Specific Impacts

Intermediate Impacts

Global Impacts

Implementation Programming

RoadsRoads Poverty Poverty ReductionReduction

12

Attention

1. This is a simplified tool, not a template. Intervention logic must be developed for each country based on its context and existing government strategies

2. Indicators are not fixed but provide ideas on how to set, monitor and evaluate CSP objectives

3. Crosscutting issues should be mainstreamed into relevant indicators (e.g. disaggregation of data by gender)

4. Outcomes/Impact are not fully attributable to the EC Outputs are 100% attributable to the EC but attribution falls as you move up the chain of results

13

List of Key Indicators (with examples)

Result (Outcome) Indicators

1.Travel times Public/Commercial transportation time

2. Transportation costsCost of public transport fares; commercial operating costs

3. Implementation and enforcement of laws related to roads

Number/type of prosecutions related to road offences; number/type of approved laws related to roads

4. Employment resulting from road construction and maintenance

Number of people directly and indirectly employed in road construction and maintenance projects

Specific Impact Indicators

5. Access to roadsPopulation living within x kilometres of a road network

6. Use of public transportationNumber of people using public transportation

7. Businesses productivity Market return for traded goods; transfer to higher-value goods

8. Access to social servicesNumber of people who live within x time of a school, health

centre, or local government office; number of visits by government officials to towns/villages

9. Environmental effects of road systemSoil erosion directly related to roads; lead and carbon monoxide

pollution; market share for unleaded petrol; number of vehicles meeting x environmental quality standard

10. Road deaths and injuresNumber of road deaths; number of road injuries

11. Disease transmissions influenced by improved mobility

HIV/AIDS infection rate; TB infection rate

Intermediate Impact Indicators

12. EmploymentEmployment opportunities within x travel time; increase in

employment within x area

13. TradeVolume of trade between y and z regions; value of trade

between y and z regions

14. School attendanceSchool enrolment rates; school completion rates

15. Health attendanceNumber of visits to health centres; number of supervised births;

child immunisation rates

14

Intervention Logic

Institutional Reform

Social

Environment

Infrastructure(construction/ maintenance)

Economic

Improved Institutional

Management of Road System

Lower Road Transport Time

Lower Road Transport Costs

Increased Employment

Enhanced Safety & Health Linked

to Roads

Environmental Impacts of Roads

Minimised

Greater Accessibility to Social Services

Greater Economic

Accessibility Improved

Conditions for Economic Growth

Economic Growth

Social Development

Poverty Reduction

OutputClusters

SpecificImpacts

IntermediateImpacts

GlobalImpacts

Enhanced Human Capacity

Results(Outcomes)