lecture 8. evaluation in agriculture and rural development msc economic policy studies ec8011:...
TRANSCRIPT
![Page 1: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/1.jpg)
LECTURE 8.EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT
MSc Economic Policy StudiesEC8011: Welfare Economics, Project and Programme Appraisal and EvaluationAlan [email protected] Nov 2011
![Page 2: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/2.jpg)
Outline
The policy context for EU and Irish agricultural policy
Evaluation experiences Evaluation specificities Case studies
Arterial drainage, agricultural research, rural development
Forestry
![Page 3: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/3.jpg)
THE POLICY CONTEXT FOR AGRICULTURE AND RURAL DEVELOPMENT
![Page 4: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/4.jpg)
Agricultural and rural development policy
Multiple objectives Ensuring food security and food at
reasonable prices Regulating food safety Supporting farm incomes Stabilising prices Promoting sustainable agriculture Achieving greater territorial cohesion Encouraging growth in agricultural output
through provision of public goods
![Page 5: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/5.jpg)
The policy framework
Market measures and direct payments provided through Pillar 1 of the CAP
Rural development measures (improving agricultural competitiveness, sustainable land management and territorial cohesion) promoted through Pillar 2 of the CAP
Significant state expenditure through co-financing EU schemes as well as independent schemes and regulatory agencies
![Page 6: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/6.jpg)
Specificities of evaluation in the agricultural and rural development context Covers both policies and programmes Address both socio-economic and
environmental objectives Complexity
Objectives are often very general in nature Difficulties in identifying impacts
attributable to specific measures in the context of multiple intervening factors
The challenges of establishing the counterfactual of what would have happened in the absence of the programme
![Page 7: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/7.jpg)
EVALUATION EXPERIENCIES
![Page 8: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/8.jpg)
Role of the Office of Public Works
Began CB analysis of arterial drainage schemes in 1970
Methodology evaluated by Bruton and Convery (1982) “The Office of Public Works stands alone among
public agencies in Ireland in its willingness to publish its investment appraisal procedures. We have accepted the invitation implicit in this act and have critically analysed the procedures so described. In doing so we are reviewing techniques of analysis which in many – perhaps almost all – cases are no doubt better than those employment elsewhere in government in Ireland. We salute the members of the Office of Public Works who have had the courage and confidence to subject their work to public scrutiny: this volume is dedicated to them.”
![Page 9: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/9.jpg)
Department of Agriculture and Food
Analysis and Evaluation Unit in the 1990s Supported by EU Structural Funds
![Page 10: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/10.jpg)
Value for Money Reviews
Department of Agriculture, Fisheries and Food Beef Classification Scheme, 2005 Compensatory Allowance Scheme, 2006 BSE eradication, 2006 Laboratory Testing for Plant and Animal Diseases,
2007 Farm Waste Management Scheme, 2007 Dairy Hygiene Scheme, 2008 Food Industry Research Measure, 2008 Processing investment aids, 2008 Bovine TB eradication, 2008 Young Farmers Installation Scheme, 2009 Forest Roads Scheme, 2010
![Page 11: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/11.jpg)
Agricultural policy covered by EU standard guidelines for evaluation and IAs Evaluation plan Impact assessment of proposed policy
changes e.g. CAP reform post-2013 Public quality statement on the evaluation
produced ex-post by DG Agri DG Agriculture and Rural Development
web page In addition, the European Court of Auditors
(ECA) periodically undertakes evaluations of particular programmes
![Page 12: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/12.jpg)
![Page 13: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/13.jpg)
EU rural development programmes
In the EU, a formal system of evaluation is in place.
EU member states are obliged by EU legislation to assess the performance of their Rural Development Programmes (RDP).
The EU Rural Development Regulations of 1999 and 2005 require that each RDP undergoes ex ante, mid-term and ex post evaluations according to a specified timetable.
![Page 14: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/14.jpg)
Innovations in evaluation 2007-2013 A more strategic approach concerning the definition
of rural development programmes; A common framework for the monitoring and
evaluation of all measures funded through the European Agricultural Fund for Rural Development;
The establishment of an “ongoing evaluation system” in order to: i) better linking monitoring activities with evaluation needs in terms of data collection; ii) establishing and quantifying baseline indicators and target levels in a timely manner; and regularly assessing the progress of the programmes in achieving quantified goals against baselines;
Setting up of a European Evaluation Network to help establishing good practice and capacity building in the evaluation of Rural Development Programmes.
![Page 15: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/15.jpg)
EU rural development evaluation requirements
Annual MS progress report containing summary of on-going evaluation activities.
In 2010, ongoing evaluation will be in the form of a separate mid-term evaluation report.
In 2015 it will be in the form of a separate ex-post evaluation report.
Also, in 2010 and each year thereafter, each member state will be required to submit to the European Commission a summary progress report implementing its national strategy plan and objectives, and strategic guidelines.
A summary of the midterm and ex post evaluation reports prepared by the member states will be made, under the responsibility of the European Commission, by end-2016.
![Page 16: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/16.jpg)
The evaluation system16
Period 2007-2013
“Ongoing evaluation”
Programming
Ex-ante evaluation
Mid-term evaluation
Policy implementation
Ex-post evaluation
Evaluation network for RD HelpdeskEvaluation expert committee
CMEF
![Page 17: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/17.jpg)
17
1. Community strategic guidelines define EU priorities of each of the axes
2. National Strategy Plans translate EU priorities into strategies in light of national specificities
3. National or regional programmes of measures on the basis of needs assessments and the national strategies
4. Implementation of programmes: monitoring and ongoing evaluation based on a Common Monitoring and Evaluation Framework
EU strategic approach for rural development programming 2007-2013
![Page 18: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/18.jpg)
![Page 19: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/19.jpg)
![Page 20: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/20.jpg)
![Page 21: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/21.jpg)
![Page 22: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/22.jpg)
![Page 23: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/23.jpg)
CMEF indicators
Common baseline indicators (59)
Objective-related indicators + targets Context-related indicators
Common monitoring indicators (95)
Financial indicators Output indicators Result indicators
Common impact indicators (7)
Additional programme-specific indicators
Handbook on the CMEF (Guidance document with 4 annexes consisting of 15 chapters)http://ec.europa.eu/agriculture/rurdev/eval/index_en.htm
![Page 24: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/24.jpg)
ROLE OF MODELS IN AGRICULTURAL POLICY EVALUATION
![Page 25: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/25.jpg)
Model simulations - applications
Implications of moving to decoupled payments
Implications of removing milk quotas Implications of concluding a Doha Round
agreement Implications of meeting climate change
targets
![Page 26: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/26.jpg)
Models as experimental laboratories
Types of economic models Partial equilibrium commodity models
Single-commodity models (food safety regulation, labelling..)
Multi-commodity models (FAPRI-Ireland….) Calibrated or econometric
Partial equilibrium programming models Computable general equilibrium models Microsimulation models (tax-benefit..)
![Page 27: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/27.jpg)
Model characteristics
Time series econometric vs. market equilibrium models
Static vs. dynamic Level of aggregation Establishing the baseline Defining scenarios
![Page 28: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/28.jpg)
Pros and cons of models
Validation Credibility gained through repeated use
(HERMES model…) Expensive to construct and maintain More complex models can become ‘black
boxes’ – difficult to communicate to policy makers (SEAMLESS…)
![Page 29: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/29.jpg)
EXAMPLEARTERIAL DRAINAGE INVESTMENTBRUTON AND CONVERY 1982
![Page 30: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/30.jpg)
Arterial drainage case study
Background Arterial drainage – artificial widening and
deepening of main rivers to increase their effectiveness at draining their catchment areas
Field drainage – the activities necessary to remove surplus water from fields
Interdependent Long history in Ireland Undertaken by the Office of Public Works
![Page 31: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/31.jpg)
Arterial drainage case study
Ireland suffers from drainage problem due to its topography
By preventing flooding, main benefit is to increase the productivity of affected lands
Potential environmental implications for: Water supply Fisheries Wildlife (drainage of wetlands) Aesthetic and amenity considerations
Value of investment ensured only through regular maintainance
![Page 32: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/32.jpg)
Why public involvement?
Arterial drainage is a public good Free-rider problem Transactions costs Role of externalities Income redistribution (using tax revenues to
enhance the productivity of land belonging to more deprived members of the community)
Rationale for involvement in field drainage Less clear cut? Higher discount rates of
elderly farmers
![Page 33: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/33.jpg)
Analytical framework
To assess the net impacts of drainage, the prospective impacts if the land were not drained should be deducted from the prospective impacts with drainage (with-without approach)
Analysis of alternatives should proceed marginally Analysis should examine alternative means of
achieving the same objective (expanding net farm income)
Test validity of assumptions using ex post analysis Adjust flows of funds over time using appropriate
rate of interest
![Page 34: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/34.jpg)
Select project with highest NPV(for mutually exclusive alternatives)
Year A B C
0 -10 -40 -30
10 40 100 60
Results
Present net worth 14.6 21.4 6.8
Internal rate of return (%)
14.9 9.6 7.2
Benefit/cost ratio 2.46 1.53 1.23Choosing the alternative which maximises either the IRR or the B/C ratio can mean foregoing increments at the margin which return more than they cost
![Page 35: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/35.jpg)
Analytical framework
Shadow pricing Estimating the net additional income
accruing to the landholders Cost of labour (if otherwise unemployed)?
Valuation of environmental effects Assigning market values not practical Marginal analysis can help identify threshold
effects Secondary impacts
Multiplier effects – not valid
![Page 36: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/36.jpg)
1945 Arterial Drainage Act
Based on recommendations of a Commission set up to study how the government might best intervene on arterial drainage Should focus on draining entire catchments Govt should bear all the cost of construction Beneficiaries should pay 70% of the assessed
improved annual value of the lands affected towards the cost of maintenance
Significant government subsidy justified on grounds of spin-off effects of greater agricultural prosperity, the positive impacts in making field drainage possible, avoiding remedial costs of deterioriating drainage conditions, indirect benefits to transport, public health, urban areas, sewage outfalls
Arterial draining is an “essential service” and a matter of “national pride”
![Page 37: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/37.jpg)
The brave Mr Hanna (DoF, minority report) Did not accept that the (unestimated) benefits were
sufficient to justify a programme whose cost per acre was almost twice the prevailing price of land
Better agricultural returns could be had by improving existing dry land
The programme offered a “palliative of very limited efficacy for unemployment”
Dismissed the argument of indirect benefits, pointing out that investments in housing and primary education would be far more valuable
Concluded that adoption of the programme would involve “a dissipation rather than a creation of national wealth”.
Catchments should only be drained where returns commensurate with the cost of construction and maintenance could be obtained, and state should never contribute more than 60% of construction cost.
![Page 38: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/38.jpg)
Government decision
Government adopted an arterial drainage programme covering twice the area recommended by the Commission (1.2 million acres rather than 600,000 acres)
28 major catchments, ranked by severity of flooding
Thirty years later, Department of Finance Appraisal Team (1968) noted that, often, cost of drainage exceeded full post-drainage value of affected lands and recommended halt to programme until full cost-benefit appraisal of future schemes
Hence introduction of CB analysis for arterial drainage in 1970
![Page 39: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/39.jpg)
The OPW methodology
Primary benefit: increase in landholder income Secondary benefits: spin-off employment and
incidental benefits to non-farm community Direct costs of arterial drainage Cost of follow-up field drainage and investment in
livestock and buildings associated with the output targets
Environmental impacts not costed but reported in EIA
Fifty year time horizon and discount rate of 3.5% used
![Page 40: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/40.jpg)
Difficulties with OPW methodology
Alternative measures of gains to landholder income Improved vs unimproved value of land NPV of additional income Requires assumption (90%) re proportion of
farmers that will adopt the follow up field drainage
Either estimate very sensitive to base year given volatility in prices/incomes
Treatment of secondary benefits Implications of cost overruns
![Page 41: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/41.jpg)
Bruton and Convery recommendations Found evidence of diminishing returns – the
more cost-effective schemes already undertaken
Undertake more marginal analysis of schemes Introduce some cost-sharing to reduce demand
for what is perceived as a free public good Drop estimation of secondary benefits
(assuming zero shadow price of labour) Do more ex-post analysis Equalise rate of grant among competing
Department schemes
![Page 42: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/42.jpg)
EXAMPLEEVALUATION OF AGRICULTURAL RESEARCH EXPENDITURE
![Page 43: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/43.jpg)
Benefits from public sector R&D
Investment in agricultural research => changes in technology =>increases in agricultural productivity
Research as a public good Measuring the benefits from research
Estimating consumers and producer benefits using a supply and demand model of a commodity market
Using regression analysis to estimate parameters of an aggregate production function model
Both ex-ante and ex-post applications
![Page 44: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/44.jpg)
Issues in measuring R&D benefits
Assessing the relationship between the size of investment in R&D and output or productivity
Identifying the relationships between increased productivity flows and economic benefits
Accounting for the timing of streams of benefits and costs, given lengthy lag times
![Page 45: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/45.jpg)
The basic supply-and-demand model of research benefits
S0
S1
D
P
Q
P0
P1
A B
C
F
GH
![Page 46: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/46.jpg)
Issues
Applies equally to demand-enhancing as to supply-increasing technological change
Clearly identifies distributional implications Measurement issues (slopes of curves…) Taking account of market distortions (price
support, environmental externalities) Potential for understating costs (cost of public
funds…) Potential for overstating benefits (ignoring other
sources of productivity gains, only evaluating ‘winners’,…)
![Page 47: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/47.jpg)
EXAMPLEINDECON MID-TERM EVALUATION OF IRISH RURAL DEVELOPMENT PROGRAMME 2010
![Page 48: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/48.jpg)
Pillar 2 Rural development
As set out in the Rural Development National Strategy, the programme is designed to contribute to the following overarching objectives:
Improving the competitiveness of agriculture by supporting restructuring, development and innovation (Axis 1);
Improving the environment and the countryside by supporting land management (Axis 2);
Improving the quality of life in rural areas and encouraging diversification of economic activity (Axis 3).
LEADER Action Groups (Axis 4)
![Page 49: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/49.jpg)
Terms of reference
Assess the degree of utilisation of resources, the effectiveness and efficiency of the programming of the EAFRD, its socio-economic impact and its impact on the Community priorities;
Answer the Common Evaluation Questions (including the horizontal evaluation questions) required as part of the CMEF guidelines set out by the European Commission;
Assess measures with respect to their balance within the programme;
Judge on the degree to which the Rural Development Programme Ireland contributes to achieving the objectives set out in the national and Community strategy;
Identify the factors that contribute to the success or failure of the programme’s implementation, including as regards sustainability, and the identification of best practice;
Propose measures to improve the quality of the programme and its implementation;
Review the goals of the programme and aim to draw lessons concerning rural development policy;
![Page 50: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/50.jpg)
![Page 51: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/51.jpg)
![Page 52: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/52.jpg)
Economic context required changes in RDP At EU level, the CAP reforms (including the CAP Health Check and the
European Economic Recovery Plan (EERP)) have combined with national budgetary decisions to require significant adaptation of Ireland’s RDP.
Closure of the Farm Improvement Scheme (Measure 121) to new applicants from October 2007;
Suspension of the Young Farmers Installation Scheme (Measure 112) and the related Early Retirement Scheme (Measure 113) to new applicants from October 2008;
Reduction in the maximum hectarage limit supported under the Less Favoured Area scheme (Measure 212) in October 2008; and
Reduction in the rate of the Rural Environment Protection Scheme (REPS) (Measure 214) in 2009 and the subsequent closure of the scheme to new entrants from July 2009.
In 2009, additional funds of €146.3 million were made available under the CAP Health Check modulation agreement and under the EERP. New monies allocated to a number of new schemes including a new investment scheme (the Targeted Agricultural Modernisation Scheme (TAMS)), a new Agri-Environment Options Scheme (AEOS), a new Natura 2000 scheme and a Rural Broadband scheme.
These modifications to the programme were finally agreed in March 2010 while adhering to the requirement to maintain a 10:80:10 ratio of EAFRD funding between Axes 1, 2 and 3-4.
![Page 53: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/53.jpg)
Primary survey research
Survey of farmers supported under the Young Farmers Installation Scheme and under the Early Retirement Scheme – beneficiaries under these two schemes were surveyed together owing to the relationship between the two schemes;
Survey of farmers supported under the Farm Improvement Scheme;
Survey of farmers supported under the Less Favoured Area scheme, the Rural Environment Protection Scheme (REPS), Natura 2000 payments and Vocational Training for REPS;
Survey of Leader Local Action Groups (LAGs) – 36 groups in total.
![Page 54: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/54.jpg)
Overview of expenditure progression
![Page 55: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/55.jpg)
![Page 56: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/56.jpg)
Axis 1 of the RDP aims to improve the competitiveness of agriculture in Ireland through support for farm restructuring, development and innovation
Measures Achieving the transfer of land to young trained
farmers better able to meet the new challenges facing Irish agriculture;
Providing training to beneficiaries under the NATURA 2000 and Agri-environment measures under Axis 2 to optimise delivery of these measures;
Encouraging the transfer of holdings from older farmers to young farmers (setting up or already established);
Supporting capital improvements in farm structures to ensure that primary agriculture becomes competitive and market-oriented.
![Page 57: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/57.jpg)
Axis 1 monitoring indicators
![Page 58: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/58.jpg)
Measure 111 indicators
![Page 59: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/59.jpg)
Measure 111 – Views of participants
![Page 60: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/60.jpg)
Measure 112 – Young Farmers’ Installation Scheme
The objectives of this measure are to: Increase the amount of land transferred to
young, trained farmers better able to meet the new challenges facing Irish agriculture,
Off-set the costs faced by young people when setting up their farm, and
Provide assistance for the investments required on such holdings.
![Page 61: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/61.jpg)
Indicators
![Page 62: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/62.jpg)
Evaluation
While 92.1% indicated that the Scheme was very significant or significant in its contribution to them setting up as a young farmer, only 30% of respondents indicated that they would not have embarked on a career in farming in the absence of YFIS financial supports.
DAFF VFM review of scheme Cost of scheme amounted to €1 for every €3
distributed to young farmers Limited impact in encouraging agricultural
qualifications (other schemes more important) Scheme was discontinued
![Page 63: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/63.jpg)
Measure 113 Early retirement scheme
The objectives of this measure are to complement the YFIS and encourage land mobility. Older farmers are provided with an early pension to encourage the transfer of their land to younger trained farmers.
![Page 64: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/64.jpg)
Indicators
![Page 65: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/65.jpg)
Evaluation
Survey results suggest low deadweight
![Page 66: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/66.jpg)
Axis 2
Axis 2 of the Rural Development Programme 2007-2013 is designed to ensure continued agricultural land use, promotion of environmentally friendly farming practices and preservation of the farmed landscape.
80% of expenditure under the RDP comes under Axis 2.
![Page 67: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/67.jpg)
![Page 68: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/68.jpg)
Measure 212 – Payments to Farmers in Areas with Handicaps: Less Favoured Areas This scheme aims to avoid land
abandonment and the disintegration of farming communities in less advantaged areas by ensuring the continued use of the agricultural land in an environmentally sustainable manner.
![Page 69: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/69.jpg)
![Page 70: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/70.jpg)
Environmental impacts
Although little data exists to prove the level of contribution of this scheme to maintaining the countryside and improving the environment, there are clear benefits to the environment from a farmer retaining their holding, farming in compliance with environmental protection requirements consistent with their statutory obligations and complying with a minimum stocking level.
Due to this measure 3.28 million hectares are now deemed to be successfully contributing to: Biodiversity Water Quality Climate change Improvement in soil quality Avoidance of marginalisation and land abandonment.
![Page 71: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/71.jpg)
Evaluation of LFA scheme
Land abandonment - what is happening to land prices?
![Page 72: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/72.jpg)
Measure 214 – Agri-environmental Payments: REPS scheme
The REPS 4 scheme was designed to promote the following: Techniques for using agricultural land which protect and
improve the environment, biodiversity, the landscape and its features, climate change, natural resources, water quality and the soil;
Environmentally friendly farming processes; The conservation of farmed environments which are of
high nature value; The upkeep of historical features on agricultural land; The use of environmental planning in farming practice; The continued use of agricultural lands (even in
disadvantaged areas); The continued vitality and viability of rural
communities; The conversion to organic production standards.
![Page 73: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/73.jpg)
Evidence of REPS success?
![Page 74: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/74.jpg)
Difference-in-difference approach
Hynes et al (2007) examined six variables over a decade (1995-2005).
This study assumed that input use on REPS farms would develop in a similar pattern to input use on extensive Non-REPS farms in the absence of REPS.
An extensive Non-REPS farm index was developed for the reference periods for the six variables. The extensive Non-REPS farm index was then applied to the base year figures for REPS farms to establish a benchmark for activities in a subsequent year.
The proportionate difference between the benchmark figure in the chosen year and the actual figure was then attributed to the presence of the REPS programme.
![Page 75: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/75.jpg)
Hynes et al (2007) results
![Page 76: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/76.jpg)
REPS and biodiversity
The REPS scheme also aims to improve and promote farming practices that protect and increase biodiversity (wildlife habitats and endangered species of flora and fauna).
Copland and O’ Halloran study aimed to determine whether REPS farms had greater numbers and more diversity of birds than Non-REPS farms. Bird and habitat data were collected from 61 REPS farms and 61 Non-REPS farms from three regions in Ireland over three bird breeding seasons.
The study found that overall there was no increased diversity among REPS farms compared to Non-REPS farms and concluded that REPS had little or no impact on bird diversity and numbers.
“The lack of any significant differences in species diversity between REPS and non-REPS farms is perhaps not too surprising given the “shallow” nature of the scheme. The basic eleven measures within REPS are likely to have limited impacts on bird diversity, with many of the measures requiring habitats to be “retained” or “maintained”, rather than managed, increased, created or improved.”
![Page 77: LECTURE 8. EVALUATION IN AGRICULTURE AND RURAL DEVELOPMENT MSc Economic Policy Studies EC8011: Welfare Economics, Project and Programme Appraisal and Evaluation](https://reader037.vdocument.in/reader037/viewer/2022102819/56649e205503460f94b0be55/html5/thumbnails/77.jpg)
Further references
DAFF Value for Money Reviews Forestry
Bacon Report 2004, A Review and Appraisal of Ireland’s Forestry Development Strategy
Clinch, P. 1999, Economics of Irish forestry: evaluating the returns to economy and society, COFORD