evaluation of eu structural interventions ex-ante evaluation mid-term / updated mid-term evaluation...
TRANSCRIPT
![Page 1: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/1.jpg)
Evaluation of EU Structural
Interventions
Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation
Ex-post Evaluation Project Assessments
![Page 2: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/2.jpg)
Königsteiner Strasse 116D-65812 Bad SodenGermany
Phone: +49-(0)6196-654168Fax: +49-(0)6196-654178
Internet: www.prac.de
P L A N U N G & F O R S C H U N G POLI CY RESEARCH & CONSULTANCY
![Page 3: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/3.jpg)
Purpose
![Page 4: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/4.jpg)
Explanation on the Purpose
The quality and adequacy of projects and fanding activities go hand in hand with the quality and adequacy of the programme in which the project is part of. A regional or sectoral development programme (as e.g. the EU Structural Funds) aims at fostering inter-regional convergence and/or stabilising important regional growth poles for the national economy. The purpose is not to simply subsidise institutions, business and individuals, Funding is the means and not the end.
Programme evaluation has to consider the respective quality, adequacy and implementation of a programme. Therefore the criteria applied in programme evaluation need also be translated into the quality assessment of the projects materialising the programme. Hence, as a truism, a project can only be as good as the programme and the programme can only be as good as the projects.
![Page 5: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/5.jpg)
1. Ex-ante Evaluation
![Page 6: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/6.jpg)
Structural Funds interventions are implemented in accordance to Operational Programmes (or Single Programming Documents) which must be (1) based on a reliable qualitative and quantitative analysis of the socio-economic situation of the region including its SWOTs and previous experiences, (2) the determination of a target-oriented development strategy, (3) the choice of appropriate measures to materialise the strategy, (4) suitable projects and (5) sound provisions governing financing, responsibilities, evaluation etc..
The assessment of the consistency of the above listed contents of an OP (or SPD respectively) is the core rationale of the Ex-ante evaluation. Further tasks to be carried out in the context of the Ex-ante Evaluation are (7) the quantification of the objectives and (8) the estimation of the overall socio-economic impact.
![Page 7: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/7.jpg)
Objectives of the Ex-ante Evaluation
• Assessment of the appropriateness of the plan in addressing the regional development problems;
• Assessment of the consistency of analysis and concept;
• Definition of suitable indicators and quantification of objectives;
![Page 8: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/8.jpg)
Objectives ...
• Analysis of the adequacy of the implementation and monitoring arrangements; and
• Impact assessment
![Page 9: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/9.jpg)
Assessment of Appropriateness
• Lessons learned from past experiences;
• Difficulties and shortcomings experienced and to be tackled;
• Assessment of the validity of data used in previous programmes;
• Consideration of thematical evaluations;
• Consideration of experiences in other programmes; and
• Verification of the socio-economic analysis and the consistency of SWOTs
![Page 10: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/10.jpg)
Consistency of SWOTs
• Strengths are static features to be supported
• Weaknesses are static features to be reduced
• Opportunities are dynamic features to be reaped
• Threats are dynamic features to be forestalled
![Page 11: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/11.jpg)
Consistency of Analysis and Strategic Concept
• Are the strategic objectives well specified ?• Do the strategic objectives sufficiently respond to the
needs identified in the socio-economic analysis?• Are the proposed measures justified by the strategy?• Is the strategy sufficiently coordinated with other
related supra-national or national policies/programmes (nat. regional policy, NAP, CIs, Objective 3 etc.) ?
![Page 12: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/12.jpg)
Suitable Indicators (examples)
• Basic indicators:
• Programme indicators:• Measure related
indicators:
• Performance indicators:
• Gross regional value added
• Supported investment
• No. of supported SMEs
• Disbursement/allocated funds
![Page 13: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/13.jpg)
Quantification of Objectives
• Related to INPUTS-OUTPUTS-RESULTS-IMPACTS
• Determination of a realistic quantification of indicators
• Comparison with results achieved in former programme periods
• Proxy-indicators to monitor intangible effects
![Page 14: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/14.jpg)
Appropriateness of the Implementation System
• Management and implementation responsibilities;
• Transparency;
• Efficiency and effectiveness of inter-institutional co-ordination;
• Competitiveness in project choice;
• Accountability in line with national and community regulations (competition policy, additionality, horizontal priorities, innovation, NAP etc.); and
• Partnership in programming and monitoring.
![Page 15: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/15.jpg)
Impact assessment
• Micro-economic net effects (notably employment, productivity, regional GDP);
• Macro-economic net effects for large programmes (macro-econometric estimations and simulations on short-term demand, long-term supply, labour market and government sector)
![Page 16: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/16.jpg)
2. Mid-term Evaluation and its Update
![Page 17: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/17.jpg)
Mid-term Evaluation
• Update of the analysis of relevance;• Update of the analysis of consistency of priorities;• Analysis of the validity of the objectives;• Comparison of plan and achievements (vertical
and horizontal);• Assessment of the implementation performance in
terms of financial and administrative management;• Contribution to the decision on the allocation of
the Performance Reserve;
![Page 18: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/18.jpg)
Mid-term Evaluation ...
• The responsibility of the mid-term evaluation is with the member state;
• The evaluation is to be carried out by an independent assessor;
• An update of the mid-term evaluation is to be elaborated one year before termination of the CSF/SPD; and
• The evaluations are accompanied and approved by the monitoring committees.
![Page 19: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/19.jpg)
Analysis at Programme Level
• Were recommendations from former evaluations sufficiently considered?
• Have the socio-economic problems (justifying the intervention) changed?
• Is the SWOT analysis adequate?• Is the Programme still consistent and coherent?• Has the programme been implemented
professionally?
![Page 20: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/20.jpg)
Analysis at Priority/Measure Level
• Financial implementation analysis
• Effectiveness analysis (plan vs. achievement)
• Efficiency analysis (unit cost comparisons)
• Impact analysis (GDP, net employment)
Cf. EU Commission/DG Regio: Working Paper 8
![Page 21: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/21.jpg)
Update Mid-term Evaluation
The role of the updated mid-term evaluation is to confirm the implementation of recommendations put forward in the mid-term evaluation and to cover fields of analysis which could not be evaluated earlier.
Cf. EU Commission/DG Regio: Working Paper 9
![Page 22: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/22.jpg)
3. Ex-post Evaluation
![Page 23: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/23.jpg)
The Ex-post evaluation is carried out under the responsibility of the EU Commission. It is elaborated as a global evaluation for each of the regional(Objective 1 and 2) and sectoral (Objective 3) programmes covering all targeted member states.
The main purpose is to assess the quality, effectiveness and impacts of the programmes. It should clearly reveal the strengths and weaknesses of the planning process, the strategic approach and implementation. Lessons learned from this exercise should contribute to better policy formulation in current and future programming periods.
![Page 24: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/24.jpg)
Scope of the Ex-post Evaluation
• Analysis of the relevance of strategy and the expenditure at EU, national and regional levels;
• Effectiveness analysis;• Efficiency and impact assessment;• Assessment of the delivery system;• Analysis of the Community value added; and• Lessons learned for the future and policy
recommendations.
![Page 25: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/25.jpg)
Relevance of Strategy and Expenditure
• Are the strategies adopted by the regions sufficiently relevant?
• Are the strategies appropriately implemented in practice?
• Has there been any deviation of the interpretation of the strategy (new direction)?
• Is the philosophy of the strategy still in line with that of the prior evaluations?
• Were the funds accordingly allocated (dispersion vs. Concentration?
• Were the changes of the strategy relevant or arbitrary?
• Was the aid intensity relevant?
• The degree of coherence in the approaches across all CSFs/SPDs of Objective 1 or Objective 2 respectively?
![Page 26: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/26.jpg)
Effectiveness Analysis
• Comparison of plan and realisation in terms of policy priorities (e.g. private sector support, innovation, business related infrastructure, HRD etc.);
• Assessment of the formerly expected and realised performance of the policy instruments (e.g. financial engineering)
![Page 27: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/27.jpg)
Efficiency and impacts
• Value for money assessment;
• Assessment of the net-effects (employment, productivity, regional GDP etc.); and
• Macro-econometric model estimations for major programmes (notably Objective 1)
![Page 28: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/28.jpg)
Delivery System
• Assessment of the institutional management and implementation capacity;
• Functioning of partnership;• Sufficiency of transparency;• Functioning of the monitoring system;• Functioning of the project selection procedures;• Appropriateness of the practical provisions to
ensure the legal underpinnings; and• Functioning of the quality control mechanisms
![Page 29: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/29.jpg)
Community Value Added
• Economic and social convergence;
• Internal and external Cohesion; and
• Institutional improvements (e.g. better coordination)
![Page 30: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/30.jpg)
`Lessons Learned` and Policy Recommendations
• Conceptual level and supra-national regulations (EU regional policy formulation);
• Justification of the financial contributions and/or the level of additionality respectively; and
• Implementation level (management, M&E partnership etc.)
![Page 31: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/31.jpg)
Programming
Ex-ante Evaluation
Implementation
Mid-term Evaluation
Adjustments
Mid-term Update
Programme results
Ex-post Evaluation
THE EVALUATION CYCLE
![Page 32: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/32.jpg)
Project assessments
![Page 33: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/33.jpg)
Project Assessment within Structural Funds Programmes
• Rules and funding regulations laid down in the Programme Complement
• Supra-national, national and regional funding regulations
• Assessment of the scope and aim of the application
• Control/audit• Funding approval … Proof of fund utilisation
![Page 34: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/34.jpg)
Criteria listed the Programme Complement
The PC includes the specific funding regulations and criteria for all measures of the programme. Any institution/individual applying for funding needs to comply to the provisions made in the PC. The PC defines
•The valid funding provisions
•The thematical/sectoral scope of funding
•The pre-conditions of funding (e.g. specific target groups)
•Eligible expenditure
•Ceiling of the subsidy
•Period of validity
•Geographical coverage
![Page 35: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/35.jpg)
Funding regulations
• Regulations of the EC (e.g. Council Regulation 1260/1999)
• European competition law (e.g. Article 92)
• National competition law
• Regional funding provisions
Note: all regulations need to be co-ordinated
![Page 36: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/36.jpg)
Project assessment grid (I)
• Is the project thematically suitable for funding under the respective measure?
• Does the project contribute to the quantified objectives of the measure?
• Is the project idea sufficiently original?• Does the project suggest synergy effects for
the programme?• Is the required co-financing adequate?
![Page 37: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/37.jpg)
Project assessment grid (II)
• Is the applicant (or the group of applicants) part of the target group?
• Is the applicant (or the group of applicants) trustworthy
• …
![Page 38: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/38.jpg)
Controlling/auditing
• Internal controlling of the managing authority notifying fraud and irregularities
• Monitoring of the project progress by the Managing Authority
• Controlling and issuing of expenditure notes by the Paying Authority
• 5% random audits carried out by the Independent Auditing Unit
![Page 39: Evaluation of EU Structural Interventions Ex-ante Evaluation Mid-term / Updated Mid-term Evaluation Ex-post Evaluation Project Assessments](https://reader035.vdocument.in/reader035/viewer/2022081503/56649cc55503460f9498e145/html5/thumbnails/39.jpg)
Author: Rolf Bergs, 4 November 2004