Regional PolicyRegional Policy

Result Orientation of future ETC Programes

Veronica GaffeyHead of Evaluation & European Semester

23 April 2013

Learning from the Past

Ex Post Evaluation of 2000-2006: We could quantify the activity (outputs) of Interreg, but

not the impact

Policy objectives and programme strategies not specific or focused

Indicators and target setting not aligned with objectives

Trying to capture the outcomes of 2007-2013 programmes:

Pilot exercise run by Interact

Useful reflection process, BUT

Impossible to build a result orientation into a programme ex post…

2

Future Result Orientation

3

o An intervention logic that incorporates: Selection of investment priorities and corresponding

specific objectives – which relate to the needs of the territory to be addressed

Result indicator which captures the intended change for the territory/sector – baseline and target (qualitative or quantitative)

Output indicators which reflect the direct activities of the priority (common and programme specific), cumulative target values

Allocation of resources

A Performance Framework

4

Monitor that implementation is as planned

A subset of programme indicators

Financial & Output indicators & Key Implementation Steps

Milestones for 2018 and Targets for 2022

Must be unambiguous and milestones and targets realistic

Possible suspensions and corrections for non-achievement

Will NOT answer the impact question (what have been the effects of the policy?)

Regional PolicyRegional Policy

Difference: Results Orientation and Performance Framework?

• Interlinked but distinct:

Results orientation is wider and locates the programme in its context

Performance Framework is about efficient implementation of the programme

5

Regional PolicyRegional Policy

6

Results Focus in Programme Design

• Guiding questions: What do you want to change?

What indicator can capture this change?

What is the baseline (the situation before the programme)?

How will the outputs of the programme contribute to change?

• Results relate to change in the region/sector – not just for supported entities

Regional Policy

Regional PolicyRegional Policy

7

Role of Result Indicators Capture what you want to change

Should be close to policy – so that the policy may be reflected in the evolution of the result indicator

Targets quantitative or qualitative

Regular monitoring to prompt policy debate (not sanctions)

Selected by programmes – not imposed top down – recognising the different "journeys" to EU2020

Evaluation to disentangle the contribution of the policy to change from the influence of other factors (impact)

Regional Policy

Regional PolicyRegional Policy

8

Role of Output Indicators

• Output Indicators: Capture what the resources are spent on

Common & Programme Specific indicators

Baselines zero, Quantified cumulative targets

Rooted in an intervention logic - how should this amount of resources spent on these outputs contribute to change in result indicator? – to be assessed in ex ante evaluation

Regional Policy

Regional PolicyRegional Policy

9

Common Output Indicators

Relate to the most frequently implemented actions

Provide aggregate information for communication purposes

46 indicators for ETC – 40 general and 6 ETC-specific

Should be complemented with programme specific output indicators when necessary

Avoid numbers of projects, or numbers of solutions/mechanisms/agreements – as in aggregate these are not meaningful

Regional Policy

Regional PolicyRegional Policy

10

Challenges

Defining the specific objectives and associated result indicators – this requires choices to be made

Establishing the baseline and targets – qualitative or quantitative(the ex ante evaluation can help)

Understanding the intervention logic

Translating the result indicators into selection criteria to ensure projects will contribute to the results

Making sure that performance frameworks represent the priorities and that milestones and targets are fixed at an appropriate level

Regional Policy

Regional PolicyRegional Policy

Reference documents - GuidanceMonitoring and Evaluation for ERDF/CF

Performance Framework & Reserve – being consulted

Pilot Tests on Result Indicators

Ex ante evaluation (ERDF, ESF, CF)http://ec.europa.eu/regional_policy/information/evaluations/guidance_en.cfm#1 11

RegionalPolicy

Regional PolicyRegional Policy

Thank you for your attention!