the world bankdocuments.worldbank.org/curated/en/...uchuva. exports of uchuva over the project...

Document of The World Bank

Report No: ICR2299

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73130)

ON A

LOAN

IN THE AMOUNT OF US$ 30.0 MILLION EQUIVALENT

TO THE

REPUBLIC OF COLOMBIA

FOR AN

AGRICULTURE TRANSITION PROJECT

May 1, 2012

Sustainable Development Department Colombia and Mexico Country Management Unit Latin America and Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 28, 2012)

Currency Unit = Colombian Peso (COP) 1,785 COP = US$ 1.00

US$ 1.00 = 0.00038 COP

FISCAL YEAR

January 1 – December 30

ABBREVIATIONS AND ACRONYMS

AGRONET Red de Información y Comunicación Estratégica del Sector Agropecuario (Strategic information and Communication Network of Agriculture Sector)

BIRF (IBRD) Banco Internacional de Reconstrucción y Fomento (International Bank for Reconstruction and Development)

CAS Country Assistance Strategy CENIS Centros nacionales de investigación (National Research Centres) CGR Contraloría General de la República COLCIENCIAS Instituto Colombiano para el Desarrollo de la Ciencia y la Tecnología (Colombian

Institute for the Development of Science and Technology) CORPOICA Corporación Colombiana de Investigación Agropecuaria (Colombian Corporation

for Agricultural Research) CPNCTA Consejo del programa nacional de Ciencias y tecnologías agropecuarias (National

Agricultural Science and Technology Program Council) CTIA Ciencia, Tecnología e Innovación en el Sector Agropecuario (Science, Technology

and Innovation in the Agricultural Sector) DANE Departamento Nacional de Estadística (National Department of Statistic) DCP Dirección de Cadenas Productivas (Production Chains Directorship) DDTPS Dirección de Desarrollo Tecnológico y Protección Sanitaria (Technological

Development and Sanitary Protection Directorship) DNP Departamento Nacional de Planeación (National Department of Planning) DREAM Dynamic Research Evaluation Management model EAP NAS

Environmental Action Plan National Agricultural Survey

FAO Food and Agriculture Organization of the United Nations FEDEGAN Federación Colombiana de Ganaderos FEDESARROLLO Federación para la Educación Superior y el Desarrollo (Secundary Education and

Devepeloment Federation) FIDA (IFAD) Fondo Internacional para el Desarrollo Agrícola (International Fund for

Agricultural Development) FMD Foot and Mouth Disease FM Financial Management FMA Financial Management Assessment FMR Financial Management Report FTA Free Trade Agreement GAP Good Agriculture Practices GMP Good Manufacturing Practices GDP Gross Domestic Product GOC Government of Colombia HACCP Hazard Analysis and Critical Control Points

ICA Instituto Colombiano Agropecuario (Colombian Agricultural Institute) ICR Implementation Completion Report IDB Inter-American Development Bank IERR Internal Economic Rate of Return IFAD International Fund for Agricultural Development IFC International Finance Corporation IICA Instituto Interamericano de Cooperación para la Agricultura (Inter American

Institute for Agricultural Cooperation) INVIMA Instituto Nacional de Vigilancia de Medicamentos y Alimentos (National Institute for

Drug and Food Surveillance) IP Indigenous Peoples IRR Internal Rate of Return ISDS Integrated Safeguard Data Sheet LAC Latin America and the Caribbean Region LMR Límites Mínimos de Residuos MADR Ministerio de Agricultura y Desarrollo Rural (Ministry of Agriculture and Rural

Development) MCIT Ministerio de Comercio, Industria y Turismo (Ministry of Commerce, Industry and

Tourism) MDGs Millennium Development Goals MHCP Ministerio de Hacienda y Crédito Público (Ministry of Finance) MTR Mid-term Review NBL3AG National Biosafety Laboratory Level 3 Agriculture NGOs Non Governmental Organizations OMC Organización Mundial del Comercio OP Operational Policy PAD Project Appraisal Document PANAFTOSA Centro Panamericano de Fiebre Aftosa (Pan-American Food and Mouth Disease

Center) PCN Project Concept Note PCU Project Coordination Unit PDO Project Development Objective PAHO Pan-American Health Organization PHEFA Hemispheric Plan of Eradication of Foot and Mouth Disease PIC Public Information Center PID Project Information Document PRA Pest Risk Analyses PSE Producer Subsidy Equivalent PTA Proyecto de Transición Agrícola (Agriculture Transition Project) PRONATTA Programa Nacional de Transferencia de Tecnología Agropecuaria (Agriculture

Technology Development Project) QCBS Quality and Cost Based Selection Method R&D Research and Development RACCP Risk Analysis and Critical Control Points RVP Regional Vice President M&E (S&E) Monitoring and Evaluation (Seguimiento y Evaluación) S&T Science and Technology SAC Sociedad de Agricultores de Colombia SENA Servicio Nacional de Aprendizaje (National Apprenticeship Service) SGS Accredited Auditor Training Organization SIL Sector Investment Loan SMEs Small and Medium Enterprises SNCTA Sistema Nacional de Ciencia y Tecnología Agropecuaria (National System for

Agricultural Science and Technology) SOEs Statement of Expenditures SPS National Sanitary and Phytosanitary Measures System

S&T Science and Technology ST+I Science, Technology and Innovation UCP-S&E Unidad de Coordinación del Proyecto – Seguimiento y Evaluación USAID United States Agency for International Development USAID-APHIS United States Agency for International Development – Animal and Plant Health

Inspection Service USDA United States Department of Agriculture WB World Bank WTO World Trade Organization YR Year

Vice President: Hassan Tuluy Country Director: Gloria M. Grandolini

Sector Manager: Laurent Msellati Project Team Leader: Diego Arias

ICR Team Leader: Diego Arias

Colombia Agriculture Transition Project

CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ................................................... 1

1.1 Context at Appraisal ................................................................................................. 1 1.1.1 Country Background. ..................................................................................... 1

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) .................................................................................................................. 2 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification ................................................................................................. 2 1.4 Main Beneficiaries ................................................................................................... 2 1.5 Original Components (as approved) ......................................................................... 3 1.6 Revised Components ................................................................................................ 3 1.7 Other significant changes ......................................................................................... 3

2. Key Factors Affecting Implementation and Outcomes .................................................. 4

2.1 Project Preparation, Design and Quality at Entry .................................................... 4 2.2 Implementation ......................................................................................................... 7 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization ......... 9 2.4 Safeguard and Fiduciary Compliance ...................................................................... 9 2.5 Post-completion Operation/Next Phase .................................................................. 10

3. Assessment of Outcomes .............................................................................................. 11

3.1 Relevance of Objectives, Design and Implementation .......................................... 11 3.2 Achievement of Project Development Objectives ................................................. 11 3.3 Efficiency ............................................................................................................... 16 3.4 Justification of Overall Outcome Rating ................................................................ 17 3.5 Overarching Themes, Other Outcomes and Impacts .............................................. 17 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ...... 18

4. Assessment of Risk to Development Outcome ............................................................. 19 5. Assessment of Bank and Borrower Performance ......................................................... 19

5.1 Bank Performance .................................................................................................. 19 5.2 Borrower Performance ........................................................................................... 21

6. Lessons Learned............................................................................................................ 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ............... 25 Annex 1. Project Costs and Financing .............................................................................. 26 Annex 2. Outputs by Component...................................................................................... 27 Annex 3. Economic and Financial Analysis ..................................................................... 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes ................. 43 Annex 5. Beneficiary and Stakeholder Survey and Impact Evaluation Results ............... 45 Annex 6. Stakeholder Workshop Report and Results ....................................................... 49 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ......................... 50 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 51 Annex 9. List of Supporting Documents .......................................................................... 52

DATA SHEET A. Basic Information

Country: Colombia Project Name: Agricultural Transition Project ID: P082167 L/C/TF Number(s): IBRD-73130 ICR Date: 05/01/2012 ICR Type: Core ICR

Lending Instrument: SIL Borrower: GOVERNMENT OF COLOMBIA

Original Total Commitment:

USD 30.00M Disbursed Amount: USD 29.93M

Revised Amount: USD 30.00M Environmental Category: B Implementing Agencies: Ministry of Agriculture Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 12/04/2003 Effectiveness: 01/10/2006 01/10/2006 Appraisal: 04/25/2005 Restructuring(s): Approval: 06/28/2005 Mid-term Review: 11/03/2008 Closing: 03/31/2010 12/30/2011 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Agricultural extension and research 69 69 Animal production 13 13 Central government administration 12 12 Crops 6 6

Theme Code (as % of total Bank financing) Export development and competitiveness 28 28 Infrastructure services for private sector development 29 29 Rural policies and institutions 14 14 Rural services and infrastructure 29 29 E. Bank Staff

Positions At ICR At Approval Vice President: Hasan A. Tuluy Pamela Cox Country Director: Gloria M. Grandolini Isabel M. Guerrero Sector Manager: Ethel Sennhauser John Redwood Project Team Leader: Diego Arias Carballo Matthew A. McMahon ICR Team Leader: Diego Arias Carballo ICR Primary Author: Diego Arias Carballo F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is to strengthen the National Agricultural Science and Technology and Sanitary and Phytosanitary Systems by supporting the joint participation of both the public and private sectors, through the mechanism of production

chains, thereby contributing to the competitiveness of Colombian agriculture and improving the accessibility of export-potential products to international markets. Revised Project Development Objectives (as approved by original approving authority) No changes. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Selected production chains have improved their participation in nat. and internat. markets: increase in market share indicator, increase in export related SPS indicators for beef & uchuva production

Value quantitative or Qualitative)

Not measured at initiation.

Increase in Market share and export related SPS indicators.

Total agriculture exports increased by 13% annually during the life of the project. Increase in 25% in beef production. No change in the production (volume/area) of uchuva. Exports of uchuva over the project period were maintained at approximately 4,300 MT

Date achieved 01/10/2006 03/31/2010 12/31/2011 Comments (incl. % achievement)

Given the nature of the indicators, it has been difficult to attribute the impact of the project on these variables. Beef production increased, but exports decreased due to problems at the border with Venezuela.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Number of MADR staff with improved capacity to support the delivery of new institutional paradigms such as cofinancing, decentralization, and response to demand-driven changes.

Value (quantitative or Qualitative)

0 11 11

Date achieved 01/10/2006 03/31/2010 12/31/2011

Comments (incl. % achievement)

Indicator 2 : Measures of administrative efficiency such as timely delivery of reports (both technical and financial).


0 100% 80%


There were delays in presenting annual reports, as well as some financial reports and audits.

Indicator 3 : Level of implementation of Annual operation plan in each year of the project. Value (quantitative or Qualitative)

0 100% 90%


Activities in the MSF component, in particular related to the National Biosafety Laboratory, faced problems that produced that some of the Annual Operation Plans fell short of their implementation expectations.

Indicator 4 : No. of officials and extensionists trained in GAPs and GMPs knowledge. Value (quantitative or Qualitative)

0 150 460


This large amount of training was due to the amount of resources put by the Government in addition to the resources from the Loan.

Indicator 5 : Strengthening of laboratories for the fruit and vegetable chain. Value (quantitative or Qualitative)

0 100% 100%


Indicator 6 : Submission to the USA of RA (risk assessment) for establishment of MLR (max residue limit) for active ingredients for its approval.


0 100% 20%


Technical proposals exist for the methodology for determining MLRs to be submitted to the USA.

Indicator 7 : Submission of protocol for recognition of fruit fly (C. capitata) low-prevalence area and risk mitigation protocol for C. capitata as alternative to cold treatment for its approval.


0 1 1


Protocol has been accepted and validated by APHIS

Indicator 8 : No. of R&D subprojects being co-financed by the Competitive Fund. Value (quantitative or Qualitative)

29 225 591


The MADR co-financed a large amount of subprojects in addition to those financed by the Loan.

Indicator 9 : Number of prospective R&D agendas developed in a participatory and decentralized manner among production chain actors.


0 20 25


Indicator 10 : Percentage increase in the co-financing of R&D agenda by the production chain participants


0 20-30% 25%


G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 12/22/2005 Moderately Unsatisfactory Unsatisfactory 0.00

2 06/01/2006 Satisfactory Satisfactory 3.00 3 12/29/2006 Satisfactory Satisfactory 3.67 4 02/22/2007 Satisfactory Satisfactory 3.67 5 07/30/2007 Satisfactory Satisfactory 5.93 6 01/25/2008 Satisfactory Satisfactory 18.08 7 06/02/2008 Satisfactory Satisfactory 18.08

8 11/17/2008 Satisfactory Satisfactory 24.27 9 05/28/2009 Satisfactory Satisfactory 26.29

10 06/30/2009 Satisfactory Satisfactory 26.29 11 12/29/2009 Satisfactory Satisfactory 29.08 12 05/07/2010 Satisfactory Satisfactory 29.08 13 06/29/2010 Moderately Satisfactory Satisfactory 29.08 14 01/15/2011 Moderately Satisfactory Moderately Satisfactory 29.72 15 07/27/2011 Moderately Satisfactory Moderately Satisfactory 29.72 16 12/28/2011 Moderately Satisfactory Moderately Satisfactory 29.93

H. Restructuring (if any) Not Applicable

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1.1.1 Country Background. In the years previous to project appraisal, in 2005, the Government of Colombia (GOC) was developing several strategies to attain the Millennium Development Goals (MDGs) by introducing a package of macro- and micro-economic reforms aimed at promoting sustainable economic growth, supplemented by specific measures aimed at reducing poverty. For this, the country had approved the Colombian National Development Plan (2002-2006) with five pillars: (i) foster sustainable economic growth and employment generation by implementing labor and pension reform, promoting competitiveness, innovation and technology, fostering trade integration and increasing private participation in the delivery of public services and infrastructure; (ii) bring about security by seeking a negotiated solution, fighting narcotics through eradication, alternative development and prevention of drug consumption, controlling arms traffic and money laundering, strengthening the army, and improving the judicial system; (iii) building a more equitable society through broadening the quality and coverage of education, widening and improving social protection, strengthening small and medium enterprises (SMEs) and cooperatives, implementing a social and development strategy for rural areas - Manejo Social del Campo- and improving the quality of life in urban areas; (iv) increasing the efficiency of the State through implementing political and public administration reform, furthering decentralization and territorial development, and strengthening local democracies and civil society participation in public administration; and (v) reduce the fiscal deficit by implementing a program that includes expenditure reductions and a comprehensive overhaul of the public sector and of the taxation system. 1.1.2 Sector Context. Within the context of this National Development Plan and the reforms that the GOC was looking to introduced, included the support to the rural economy by improving the competitiveness of the agriculture sector, transitioning towards a context of trade liberalization. At the time of appraisal, the Colombian agricultural and agro-entrepreneurial sector accounted for 21 percent of aggregate GDP, 25 percent of export revenues, and 30 percent of job creation in the country, employing more than 4.5 million. However, Colombia was ranking lower in terms of sector competitiveness when compared with countries at similar level of development. By the time of project appraisal, the GOC had embarked, along with its partner countries in the Andean Region, on the negotiation of a Free Trade Agreement (FTA) with the United States. In this context, improving agricultural competitiveness, developing the rural areas, and improving the living standards of the rural population, was identified as a key to Colombia becoming a peaceful and productive country, enabling the country to consolidate the competitive advantages of the rural sector, and to take advantage of the opportunities brought about by the commercial opening envisioned in the FTA. The agriculture sector needed to improve to key public services to achieve this competitive edge: (a) undertaking focused sector-responsive research and development work to improve productivity; and (b) establishing SPS standards to enable access to international markets (US and other potential new markets). 1.1.3 Rationale for Bank involvement. While there were other financiers supporting the agriculture sector of Colombia at the time (the Inter-American Development Bank, the United States Agency for International Development, and International Fund for Agricultural Development), the Bank was the principal player in the rural sector of Colombia, bringing much needed international knowledge and expertise in terms of public-private partnerships in agriculture research and development (R&D), and guidance in establishing SPS standards. The Bank had a long history of support to the rural sector in Colombia, and Bank-assisted projects

2

have managed to execute decentralized projects very effectively, such as the Agriculture Technology Development Project (Proyecto de Desarrollo Tecnológico Agropecuario); Programa Nacional de Transferencia de Tecnologia Agropecuaria (PRONATTA), Support to Productive Partnerships (Apoyo a alianzas productivas); Magdalena Medio I & II; Zonas de Reserva Campesina and Earthquake Recovery). These investments provided important lessons learned for the project design of the PTA. In addition, a series of earlier country-specific Bank knowledge services and studies provided important inputs for project design, such as the sector study on: (i) Agricultural and Rural Competitiveness (Report No. 27523), (ii) Land Policy in Transition (Report No. 27942), and (iii) Rural Finance - access issues, challenges, and opportunities (Report No. 27269). Finally, the Bank had experience dealing with other institutions in Colombia’s rural sector, and this experience, coupled with the lessons learned from the long-term involvement of the Bank in the rural sector of Colombia, especially in decentralized projects, and the Bank’s international experience, provided the basis for the Bank’s involvement and project design.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 1.2.1 The Project Development Objective as stated in the PAD and in the Loan Agreement was to strengthen the National Agricultural Science and Technology and Sanitary and Phytosanitary Systems (SPS) by supporting the joint participation of both the public and private sectors, through the mechanism of production chains, thereby contributing to the competitiveness of Colombian agriculture and improving the accessibility of export-potential products to international markets. 1.2.2 Project Development Indicators were tied closely with the activities related to the SPS investments, and to a lesser extent to agriculture research and development (R&D) activities. A series of proxy supply chains were selected to measure the improvement in competitiveness by evaluating indicators such as the change in national and international market participation (see 1.5.1, 2.1.4 and Annex 2 for more details). 1.2.3 The higher level objective of the Project is agriculture competitiveness, which depends on adoption of the agriculture R&D outcomes. The indicators for the PDO therefore should have been grouped around the following two broad sub-PDOs: (i) the focus on participatory process to determine the agriculture R&D agenda (co-financing of the research proposals, shared implementation of research project between public and private sector institutions, to ensure that the research agenda reflects the priorities of well-defined value chains to provide a critical research and development input for competitiveness); and (ii) the development of SPS protocols (this activity focused on beef and the uchuva fruit). The indicators for the PDO therefore should be grouped around these two broad sub-PDOs. 1.2.4 For the agriculture R&D outcome, the key indicators would relate to the process - extent of participation by value chain actors, co-financing levels, number of sub-projects shared for implementation, and to provide an idea of the critical mass, the number (and value of) sub-projects focused on specific value chains. A second indicator would relate to the contribution of the R&D sub-projects to competitiveness, for which the surrogate indicator would be increased incomes at the producer level. For the SPS outcome, the key indicators would be the development of the protocol and its relevance across the beef and fruits and vegetables industry, and, since the protocol is critical to accessing new markets with stringent SPS requirements, increased exports and diversification of export markets (in beef and fruit and vegetables sector).

Indicators in PAD Indicators inferred by ICR author Agriculture R&D sub-

None Number of R&D agendas developed in a participatory and decentralized manner among

3

PDO production chain actors. Increase income (competitiveness) of farmers benefiting from R&D investments.

SPS sub-PDO

Selected production chains have improved their participation in national and international markets: increase in market share indicator, increase in export related SPS indicators for beef & uchuva production.

Fulfillment of conditions required for international recognition of fruits and vegetables (uchuva) and meat chain-specific product admissibility (beef).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification N/A

1.4 Main Beneficiaries 1.4.1 The Project had national coverage and was intended to benefit those agents linked to the production chains targeted by the instruments financed through this operation. Specific provisions were to be adopted to make sure that small producers involved in those chains would not be excluded from the benefits derived from the project, particularly by putting in place dissemination mechanisms that assure their proper participation in the relevant activities proposed. In addition to farmers benefiting from the R&D and SPS activities at the national level, other institutions that would benefit from the project included: (i) institutions carrying out R&D activities, such as universities, research centers, and technological development centers (who may decide to compete for resources in the R&D Competitive Fund ); (ii) institutions responsible for the implementation and monitoring of sanitary and phytosanitary (SPS) measures, such as ICA and INVIMA; and (iii) companies and organizations involved in the traditional Export Sector.

1.5 Original Components (as approved) 1.5.1 The project planned to finance the following components: (a) Knowledge Generation and Innovation (US$20.7 million, about 69% of total project costs), through the strengthening of agricultural production chain actors in the definition, co-financing and implementation of R&D, as well as the preparation of participatory and demand driven R&D agendas around certain production chains and the implementation and co-financing of these agendas through a Competitive Fund. Chain research agendas were not identified as of appraisal and were expected to be jointly defined with the Agricultural Sector-related National Science and Technology System actors, under the leadership of the MADR and the National Agricultural Science and Technology Program Council (CPNCTA) attached to COLCIENCIAS, which was expected to also provide financial support. Since R&D subprojects financed under the Competitive Fund of this component were expected to take up to three years to be implemented, financing of subprojects chosen in the Call for Proposals of years 3 and 4 were already expected to go beyond the deadline set for implementation of this operation with World Bank loan resources. (b) Strengthening of the National Sanitary and Phytosanitary Measures System (US$7.2 million, about 24% of total project costs), enabling it to respond to the sanitary standards of international markets and thereby enhance the admissibility of agricultural products to these markets. The needs identified in the SPS system at appraisal were prioritized vis-à-vis requirements of two production chains (beef, and uchuva-Physalis), which at the time were showing a competitive potential particularly in the US market and which were expected to have a spill-over effect to other chains. The beef chain was chosen within the livestock sector because its products were highly complex in sanitary terms, not only because they face zoo-sanitary barriers

4

but also other food safety-related barriers. Although the investment plan may have been designed on the basis of overcoming barriers for a particular fruit (uchuva), its range covered the needs being faced by a large share of the fruit and vegetables chain to successfully access international markets (c) Project Management, Monitoring and Evaluation (US$2.1 million, about 7% of total project costs) to ensure proper project management and execution and finance a Project Coordination Unit which was to be attached to the Ministry of Agriculture, and be responsible for the general management of the project.

1.6 Revised Components N/A

1.7 Other significant changes 1.7.1 Restructuring. Cumulative extensions of the Closing Date (Level Two Restructurings) totaled 21 months. The original and final closing dates were March 31, 2010 and December 30, 2011 respectively. The primary reason for both extensions was to permit the completion of individual subprojects under Component 1 and to allow for the completion of the National Biosafety Laboratory (NBL3Ag) under Component 2 of the Loan. 1.7.2 Reallocation of funds. The reallocation increased the amount of resources to Component 2 and 3 by US$600,000 and US$1.3 million respectively; and reduced Component 1 by US$150,000. This reallocation of funds was done based on: (i) the allocation of the “contingency” expense category to specific project components and activities; (ii) that the objectives and results of activities related to the R&D agendas of Component 1 were achieved (and surpassed) with fewer than expected resources spent; (iii) the extension of the closing date required additional resources towards Component 3; and (iv) that the execution of Component 2 was having higher than budgeted costs.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 2.1.1 Background analysis: The sector and country analysis portrays an exhaustively analyzed project, sourcing from Bank, international and Colombian research documents, and supporting the first pillar of the National Development Plan: foster sustainable economic growth and employment generation by implementing labor and pension reform, promoting competitiveness, innovation and technology, fostering trade integration and increasing private participation in the delivery of public services and infrastructure. However, as discussed below, key aspects of the analysis were incomplete with ramifications for achieving some of the expected results and benefits for the final target population: farmers. 2.1.2 Assessment of project design: Project design correctly prioritized key public goods and services needed to increase the competitiveness of the agriculture sector of Colombia and facilitate the transition to trade liberalization. By focusing investments in strengthening the SPS system and investing in agriculture R&D, the Government of Colombia and the Bank ensured the continuity of sector investment priorities beyond political cycles. Nevertheless, several elements/lessons were cited as essential for successful agricultural development projects, but their reflection in project design was uneven. The incorporation of small farmers through a dialogue and agreement with other actors and the networking of the Productive Partnership Project (also financed by the Bank) were expected to ensure the proper participation of these groups in project

5

investments. In spite of this shortcoming, PTA investments, in particular with respect to its R&D Competitive Fund, did provide evidence of production and productivity impacts at the farm level. Although there’s no specific indicator in project design on the participation of small farmers in the project, the project did provide significant benefits for both the national SPS and agriculture R&D systems. 2.1.3 The original flexible design of the project allowed for it to become integrated within the MADR, and in particular the DDTPS (without becoming a side-project without an institutional anchor). It also introduced an innovative way for investing in Agriculture R&D through a value chain approach and prioritization of activities by involving all stakeholders. This flexibility allowed for making important SPS investments for Colombia (as well as the Andean Region), but the complexity of the investments was not fully acknowledged or planned in the initial design. The Competitive Fund was seen as an important mechanism for making transparent public investments, leveraging private sector funding, however a few shortcomings were also identified from project design: (i) no clear mechanism to establish inter relationships among the subprojects; and (ii) no clear way for transferring technologies to farmer groups in a systematic way. Nevertheless, these shortcomings were then balanced by the results of the impact evaluation, which showed a significant farm level positive impact on production and productivity. Another shortcoming of the project design has been that both the SPS investments and the R&D Competitive Fund, required sophisticated technical, analytical and operational skills, exceeding the capacity of the MADR and many agriculture R&D institutions. This resulted in delays on SPS investments and a concentration of funds in a few large institutions that captured most of the competitive funding. Overall, both project components (agriculture R&D and SPS) introduced innovative and important mechanisms and investments in Colombia, but had a few shortcomings in terms of design, with an over-emphasis on value chain development. 2.1.4 Objectives and indicators: The PDO1 did reflect the context in which the project was prepared (trade liberalization), but was ambitious and not completely aligned with project activities. It had two distinct targets: on one hand the Agricultural R&D System and on the other, the Sanitary and Phytosanitary (SPS) System. However, the PDO-level indicator was linked closely to the SPS component rather than the agriculture R&D part of the PDO itself. By only measuring market share (indicator that is very difficult to influence through R&D investments in the short term), this subjected the project’s success to the performance of activities under Component 2, which only represented 24% of project costs, and with clear problems of attribution towards the PDO. Furthermore, by focusing only on Uchuva and Beef, this further narrowed the chances of a positive outcome with respect to the great diversity of supply chains supported by project activities in both Components 1 and 2. However, Component and Activity – level indicators in the Results Framework were more accurately representative of the planned investments. 2.1.5 Poverty targeting: The PDO had no explicit poverty focus and the project was not particularly targeted to the poor. But the project was expected to benefit many small producers participating in supply chains targeted and/or included in project activities. The project design included a Small Producers Development Framework (SDF) in order to ensure the inclusion of small producers in the country’s competitiveness and export strategy (out of the small producers

1 The Project Development Objective is to strengthen the National Agricultural Science and Technology and Sanitary and Phytosanitary Systems by supporting the joint participation of both the public and private sectors, through the mechanism of production chains, thereby contributing to the competitiveness of Colombian agriculture and improving the accessibility of export-potential products to international markets.

6

participating in agriculture supply chains expected to benefit from the project, 2% of them were estimated to be indigenous people). Further, the percentage increase in the average annual production value of the supply chains participating in the R&D investments was expected to increase by 100% over baseline. 2.1.6 Components and organization: Components were three, and some 56% of available funds financed agriculture R&D subprojects through the Competitive Fund of Component 1. An Impact Evaluation of the R&D investments done through the subprojects showed that investments were concentrated in the fruit and vegetable supply chains as planned during project appraisal, and that the Competitive Fund allowed for the co-financing of investments from project beneficiaries (up to 50% of the total cost of subprojects), providing important resources towards the project objectives and the participation of the productive sector in the subprojects. 2.1.7 Within Component 1, the methodology used to develop the prospective agendas2 for agriculture R&D in each supply chain, and the consultation processes between producers and R&D organizations was successful and exceeded targets as originally planned, although was more time consuming than expected and therefore the agendas were finalized after the initial allocation of R&D resources had been completed. Nevertheless, both the prospective agendas and the Competitive Fund yielded significant results both at the farm level, as well as lessons learnt and methodology at the institutional level, which is currently being used as input to the future investments in R&D at the national and multisector level, and serving as model for more inclusive R&D process; for matching R&D demand with supply; and for co-financing R&D between public and private sector. 2.1.8 Component 2 was also designed to create the capacity of the SPS system, and did provide relevant and useful investments for ICA and INVIMA, improving their functioning, and allowing for adopting measures that contributed to improve access of Colombian products to foreign markets (signature of 96 international SPS protocols during the life of the project) , certifying the country free of Foot and Mouth Disease without vaccination and the establishment of disease free areas of several animal and plant diseases (the most important being , Brucelosis, Tuberculosis, Bactrocera, controlled Fruit Fly, and the maintenance of the disease free status for Highly Pathogenic avian Influenza and Mad Cow Disease (OIE recognizes Colombia as a country of insignificant risk). Even though this Component has had significant economic and capacity building impact, it did not always reach complete national coverage of activities due mainly to a lack of clear criteria for prioritizing investments within the SPS system. A specific and successful approach of this component was to implement low tech approaches towards SPS, such as the Agriculture Best Practices ("Buenas Practicas Agricolas") and the approval of export protocols with many countries, and all of the targets for training and capacity building were exceeded. Even though the project design did not include the need for specific counterpart funding from the SPS institutions to ensure the operation, maintenance and continuation of project activities and investments, ICA and INVIMA have benefited directly from these activities and has allowed these institutions to use the experiences and lessons learnt, maintaining and operating country investments and complying with international protocols. In the case of the eradication of FMD the country has complied with the commitments’ of the Hemispheric Plan of Eradication of Foot-and Mouth Disease (PHEFA) which is being successfully executed in the majority of countries of South America.

2 A prospective agenda is a set of areas and prioritization done by stakeholders within a supply chain to arrive at a medium to long terms strategy for private and public investments (see link with published agendas: http://www.minagricultura.gov.co/02componentes/02cyt_09f_rtdoagen.aspx).

http://www.minagricultura.gov.co/02componentes/02cyt_09f_rtdoagen.aspx

7

2.1.9 Financing: With an estimated total cost of US$30 million, the MADR also provided resources to expand the number of subprojects financed under the Competitive Fund (a total of 117 subprojects were funded with resources from the Bank’s Loan and 474 subprojects were funded by MADR directly). Also, 50% of the funding of the subprojects was provided by supply chain participants and beneficiaries. 2.1.10 Risk mitigation: Types of risks likely to influence the project were accurate as far as they went, but suggested mitigation was superficial in some cases and certain risks were not mentioned. Such analysis obviously could not have predicted the braking of diplomatic relations with Venezuela that closed the cattle exports to that country, or the delay in the signing of the FTA with the USA. However, the belief that monitoring and follow-up mechanisms to allow for agreements and non-controversial solutions could mitigate difficulties with international acceptance of SPS protocols was inconsistent with the quite inflexible and known international standards for SPS certifications and procedures. Furthermore, it was unrealistic to expect/assume that a commitment from MADR to use the Project’s Competitive allocation of funding for R&D would be continued beyond project closing, given the fact that the S&T agenda for the entire country (other economic sectors) is not only shaped by the MADR, but by other institutions. Nevertheless, the agriculture R&D agendas (agendas prospectivas) did provide a base of consensus among supply chain stakeholders on priorities and financing on investments regardless of public sector instruments and programs. Finally, although the construction of the National Biosafety Laboratory (NBL3AG) was mentioned in the Annex of the PAD within a list of several activities and without much details, the decision to finance such a complex investment through the PTA, although it was (and is) amply justified, should have arisen as a potential major risk in the design phase of the project. 2.1.11 Participatory processes: PTA acknowledged the importance of demand driven processes for the allocation of agriculture R&D funding and small farmer participation to subproject success, effectiveness and sustainability. PTA was also developed based on the chain approach, which included a wide range of grower associations (Gremios) involved in the chain. The majority of Gremios identified at appraisal were associated with the 20 selected production chains at the national level and 76 organizations of the chains at the regional level. At appraisal, an estimated 120 organizations and Gremios represented an estimated 80,000 small producers (excluding the cafeteros - coffee producers) within these supply chains. Among the 523,000 coffee producers about 73% had a farm size of less than 5.0 hectares. In addition about 70% of the total number of producers were represented by its federation (Federacion Nacional de Cafeteros). At appraisal, a strong organizational structure existed in Colombia regarding access to productive factors and a capacity for dialogue with policymakers and agencies in charge of policy implementation, however a second set of organizations were also identified, who were highly vulnerable because of the weak or non-existent organizational representation and their rather scant ability to access markets and scientific and technological developments in particular, and to whom benefits were accrued in a more uncertain way. Evidence from the evaluation of the Competitive Fund showed that some of the investments were driven mainly by the R&D institution, than the producer organizations, which can have implications for beneficiaries’ willingness to invest in and adopt the technologies produced. However, given the impact of the R&D subprojects in farmer production and productivity, as well as the important levels of co-financing by Gremios, (with “recursos parafiscales”) are indicators that R&D subprojects are likely to be sustainable.

2.1.12 Adequacy of government commitment: The Government had embarked by 2004 on the negotiation of a Free Trade Agreement (FTA) with the United States. The Government was

8

committed to increase the competitiveness of the agriculture sector in order to prepare for a more open economy. In this context, improving agricultural competitiveness, developing the rural areas, and improving the living standards of the rural population, was a priority for Colombia to becoming a peaceful3 and productive country. The Government and the Bank thus embarked in the identification of the two key areas that would enable the sector to increase its competitive position in light of the FTA. The PTA was thus a direct result of this situation and Government strategy, receiving strong support and collaboration from the GOC.

2.2 Implementation 2.2.1 Readiness to implement: Overall, the project readiness at approval was acceptable. An Operations Manual had been prepared and a mixed of PCU and MADR staff were available to begin execution as planned (although effectiveness delays occurred due to the delay in the ratification by Congress). However, hindsight shows that the PAD underestimated the complexity for achieving international recognition of admissibility of beef products, as well as the required institutional capacity to execute the Competitive R&D funds and to supervise complex SPS investments. The project lacked planning for ensuring quality project management skills to supervise the technical feasibility studies and engineering designs for the NBL3AG to address foot and mouth disease, which ended up being one of the main causes of project delays. It also lacked the planning for the development of a communications strategy, which was eventually reflected in the lack of internal communications among subprojects and between components. Efforts to resolve this issue during project implementation were costly, involving a high level of inputs in consultant services and supervision by the Bank and Borrower. This situation was further complicated by the hiatus in the process of achieving certification of the NBL3AG, given investigations launched by the Contraloría General de la República (CGR) in the use of fiscal resources, during which the works for concluding the Laboratory were halted for 24 months.

2.2.2 Subproject approval and review process: Overall, stakeholders of the agriculture R&D Competitive Fund support the mechanisms as a transparent and objective way of allocating public funds, so the overall process of evaluation and supervision of subprojects has been deemed appropriate by most market participants. However, it is to be noted that it was a drawback not to have had the results from the agriculture R&D agendas before the initial call for proposal had been launched (at least that was the original plan as per the PAD). Furthermore, it was noted at the midterm review that there were no clear strategy/procedures for incorporating the recommendation of the expert review panel before subproject implementation. The expert review panel had to approve or deny funding for subprojects, and the recommendations for improvement were not systematically integrated in an ex-ante fashion for project implementation. This was later corrected. Finally, there were no format for subproject progress reports and the Project Management Information System (SIGP) was not available for the first call for proposals, which produced difficulties in subproject implementing agencies and delays. Again, these issues were solved by the second call for proposals. 2.2.3 The National Biosafety Laboratory Level 3 Agriculture (NBL3Ag): As a part of the strengthening of the SPS laboratory network, given the raising threat of Foot and Mouth Disease (FMD) to the livestock industry of Colombia and as a basic component of the prevention program

3 The term peaceful terminology is drawn from the PAD, as rural incomes was something that was a priority for the Government at the time to reduce violence in rural areas and provide alternative income sources for the rural population.

9

to maintain the FMD disease free status as required by OIE and the PHEFA, it was deemed necessary to have an in-country Laboratory Biosafety Level 3 Agriculture (NBL3Ag). This investment was amply justified from the point of view of the large returns to the local livestock industry (local consumptions and exports), but also for the Region (only Laboratory of this level of biosafety in the Andean Region). Although Colombia (thanks in part to project activities) have gained and maintained the certification of free of FMD (neighbor countries of Venezuela and Ecuador are not certified), the NBL3Ag has not yet been completed due to delays in the execution and contracting of the final works and adjustments to the Laboratory to obtain Certification. The Bank project team reviewed and supervised a comprehensive Action Plan (AP) in October 2010 prepared by the Borrower to conclude the necessary works and modifications to the National Biosafety Laboratory in order to achieve Certification (Biosafety Level 3 Agriculture) by project closing. Although this AP was deemed acceptable and technically and financially sound, the AP initiation was not implemented until November 2011 due to uncertainties in the Government funding of the additional works and the contract management delays between MADR, the contractors and CIAT (the fiduciary agency managing MADR’s resources). Given the delays in the implementation of the AP, three project closing date extension were requested, and two were approved by the Bank (the last project closing date extension was not approved). The completion of the AP is scheduled for September, 2012 (works are currently under way financed by Government resources) and the Bank team has confirmed that the completion is likely to meet its target and certification possibly by end of 2012. 2.2.4 Project leadership and team continuity: After the mid-term evaluation and the delays observed in the conclusion of the works for the NBL3AG, initiated an even closer, more hands-on and proactive partnership between the Bank team and the MADR, characterized by an intense focus on quality, realism, and flexibility. During implementation, the Bank and MADR partnered to raise the following issues: (i) the level of importance for addressing environmental issues in the agriculture R&D subprojects; (ii) promoting the need for Colombia to have a NBL3AG; and (iii) promoting the project model of a Competitive Fund for funding agriculture R&D in the country. The Bank project team, incorporated a Biosafety expert with experience in certification of Laboratories and protocols, as well as active participation from environmental safeguards expert in every project supervision mission. There were tensions between MADR and CGR at times with regards to the AP for the NBL3AG, were the Bank had to at times intervene to remind authorities of the importance of the investment. Of critical importance to the project’s success was the continuity of the commitment by MADR of the project’s objectives, investments and approaches, as well as the core technical teams in both institutions, and in particular the local Bank Staff. 2.2.5 Mid-term Review: The project Mid-term Review (MTR) in 2009, analyzed the performance of the Competitive Fund of Component 1 based on workshops and consultations at the regional and national level with project beneficiaries and supply chains, and field visits. The Bank team acknowledged, inter alia, the need for addressing issues pertaining to the transfer of technologies produced by subprojects to the final producers, as well as the need for R&D institutions working on the same supply chains or problems to interact with each other. The mid-term review also recognized the concentration of resources from the Competitive Fund in two agriculture R&D institutions, and the need to reach out to other players and actors, fostering more of an “applied research perspective” rather than “laboratory work”. Most MTR recommendations were implemented with positive effects, but the reduced level of transfer of technologies to producers remained an issue given the lack of resources and constraints built-into the subproject execution agreements. It was observed, however, that few projects that were implemented over longer periods at the time of PTA closure had already had some impact at the level of production (tomato production for instance) or had contributed to raise production quality standards in

10

special subsectors such as aquaculture. The longer term impact of PTA should have to be assessed over a period of time where most of investments mature enough to show results.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 Design: The planned project monitoring and evaluation scheme was as follows: the M&E system housed within the Project Coordinating Unit (PCU) would ensure follow-up of both compliance with component goals and budgetary performance. The M&E system included a PCU-based internal planning and monitoring and evaluation activities, as well as outsourced reviews and impact assessments. 2.3.2 Implementation and utilization: MADR established a well-staffed M&E Unit within the PCU in 2005, which was steadily built - with Bank support – including a Project Management Information System (SIGP) to track, monitor and evaluate agriculture R&D subprojects. The PCU disseminated major findings and R&D Agendas for 12 supply chains via annual/special seminars and workshops involving all relevant sector authorities and stakeholders at the national and regional levels. It also trained subproject implementing agencies in evaluation techniques (several subproject completion reports were prepared in areas of economic impact and environmental assessments). Although with delays (and with Bank support), the PCU undertook a comprehensive economic impact evaluation study of the Competitive Fund, used to assess the achievement of the overall objectives of the project and to prepare this ICR.

2.4 Safeguard and Fiduciary Compliance 2.4.1 Social: Project preparation included a social assessment identifying the main socio-economic and cultural characteristics of small farmers across different supply chains (no social safeguards were triggered). A Small Farmer Development Framework (SDF) was prepared to integrate small farmers in the overall development strategy of the country, and in particular of the agriculture competitiveness strategy. 2% of the small farmer population to be impacted by the project was expected to be indigenous people. Indeed, the final beneficiaries of agriculture R&D subprojects were largely small farmers with some indigenous communities also benefiting from some of the agriculture technologies developed. 2.4.2 Environment: Efficient use and better conservation and management of natural resources were project objectives of several subprojects. The project was a Category B and a Pest Management Framework was prepared, consulted and approved by the Bank. Most activities were technically Category B. Project environmental behavior, in particular for subprojects, was governed by an Environmental Action Plan (EAP). The specific objectives of the EAP over and above the PDO were to: (a) incorporate environmental aspects in all relevant activities and technologies proposed and financed by the project; (b) evaluate the potential effects on the environment and compliance with local environmental laws and bank safeguards during the mid-term and final evaluation of subproject implementation; and (c) conduct post-project impact analysis to delineate lessons from this project. This assessment was conducted by the majority of beneficiaries who developed a Risk-Benefit analysis in case that the subprojects reach a large scale implementation in the future. An environmental specialist accompanied Bank supervision missions and subprojects in particular were supervised closely throughout. The PCU had, until the last few years of project execution, environmental specialists on staff. No serious environmental problems arose during project execution. By the end of the project 99% of the subprojects had presented the socio-environmental assessments of the potential impacts of the technologies developed with the financing of the PTA. They were all deemed acceptable.

11

2.4.3 Procurement: The project’s procurement load was heavy due to the size, diversity, type and the large number of subprojects, and the continuous need for specialist consultants. The PCU procurement team was experienced and well-qualified. Bank Procurement Post-reviews (PPR) up to 2011 rated procurement performance Satisfactory and risk as Average, with the exception of Medium Satisfactory rating due to procurement processes related to the construction of the NBL3AG. No evidence was found of deviation from Bank standards, or of misprocurement. The Bank project team worked closely with the procurement specialist, CMU and the PCU to follow up on recommendations. Two issues were problematic: (i) administration of contracts related to the construction of the NBL3AG; and (ii) delayed submission of SOEs by subprojects. 2.4.4 Financial Management/audit: Financial management (FM) performance was Moderate Satisfactory throughout project implementation. The FM assessed project risk as substantial due to the complexity of the project, which involved countrywide implementation of subprojects managed by several research entities and universities. Audits and unaudited financial reports were received within the contractual dates. The external auditors issued unqualified opinion (without exception) on the project financial statements for the audit periods 2008 to 2011, and the MADR‘s action plan adequately addressed the auditors recommendations, the only exception being a pending finding of “fiscal” nature related to the completion of the NBL3AG, still under review by the CGR until today. However, because of deficiencies on the monitoring of the subprojects, an action plan was agreed, thus the MADR exercised substantial efforts during the last year of the project to follow-up on the outstanding subprojects advances pending of justification and/or reimbursement, monitoring activities continuing during the grace period on the pending advances in the amount of approximately US$355,000, where the MADR will reimburse the Bank any unsupported or unused funds.

2.5 Post-completion Operation/Next Phase 2.5.1 Transition arrangements to regular operations: Overall, the PTA was more than just a project, and became an integral part of the MADR, and the agenda of its Technology and Sanitary Protection Directorate (DDTPS). Although at project closing there is not clear indication of a stable permanent financing mechanism for the agriculture R&D system and the SPS activities, the PTA investments are being operated and maintained and lessons learnt incorporated into new policies and programs. Under Component 1, all subprojects implemented are in the public domain and agriculture R&D institutions (be it public, academic, or private) have established procedures and strategies for transferring technologies to producers, although agriculture extension and technical assistance programs were not planned at appraisal. Certain subprojects and institutions have applied to a new MADR-funded Competitive Fund for technical assistance grants launched in 2011, which looked to bridge the gap between the upstream R&D investments funded by the PTA model and technology adoption at the farmer level. Although there has been a halt in the call for proposals for agriculture R&D initiatives put in place by the new Government in 2011, the new S&T and Innovation Fund (Fondo de Ciencia Tecnologia e Innovacion del Sistema General de Regalias) is being developed and is incorporating the experiences of the PTA, in particular those related to the competitive funding process (call for proposals) and the prioritization by stakeholder consultations and R&D agendas. With respect to the investments under Component 2 (SPS systems), both ICA and INVIMA have incorporated and budgeted as of project closing the resources needed for operating and maintaining investments and activities launched under PTA. 2.5.2 Furthermore, as of project closing, the MADR had absorbed the PCU team as well as integrated the PTA methodology for the management and supervision of subprojects into the institutional structure of MADR. The transition of the SPS investments (including the NBL3AG) to public sector institutions (ICA and INVIMA), and all the subprojects’ technologies into

12

producer groups and further R&D by institutions depends on the extent to which they are clearly and deeply rooted in the institutions’ investment plans (and budgets) and the level of participation of producers in the subprojects. 2.5.3 Operation and Maintenance: Project design included the need for ensuring that O&M costs for the SPS investments in new laboratory facilities would need to be incorporated in the budget of the responsible public sector institutions (ICA / INVIMA). In particular, the NBL3AG was not yet certified as of project closing, but as of the writing of this ICR, it was expected to be certified and operational by end of 2012, for which ICA (the institution responsible for the NBL3AG) had the needed budget and human resources to assure its operation and maintenance. The project was successful in ensuring the needed O&M costs were included in the public budget, although the NBL3AG has not yet been certified (and thus fully operational). 2.5.4 Follow-on operation: While PTA launched an important process for allocating funding to agriculture R&D and the prioritization of SPS investments, the MADR and the GOC have not signaled for a continuation of the PTA methodology for allocation of agriculture R&D funding, nor a medium-long term plan for SPS system investments. As of the writing of this ICR, the MADR was considering including SPS investments within the Rural Alliances III project in preparation, but that had not been agreed upon as of yet. However, given that the Competitive Fund methodology and staff had been fully absorbed by MADR, and that there is an apparent availability of funding in the public sector budget of MADR, if a new call for proposal was to be launched, it is not clear that a follow-on operation would be required from the Bank, as all processes and procedures had been institutionalized within the Directorate of Technology Development and Sanitary Protection (DDTPS) of MADR. Finally, the current government has taken action towards undoing MADR’s role in the agriculture R&D system, transferring to CORPOICA the role of coordinator of the system, and strengthening COLCIENCIAS as the overall regulator of the S&T agenda at the national level.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Rating: High overall relevance 3.1.1 PTA development objectives, despite their vintage, remain relevant to country and global priorities. The focus of the operationalization of the country’s agriculture R&D agenda now reflects the GOC’s strategy of investing in agriculture innovation through the broader S&T framework, administering the competitive funds through COLCIENCIAS. With respect to the SPS system objectives, the MADR and autonomous institutions such as ICA and INVIMA, now reflect the same approach and vision of PTA.

3.2 Achievement of Project Development Objectives 3.2.1 PTA achieved its objective but its influence extended far beyond the sum of its achievements. The project was an innovative and successful approach in funding agriculture R&D in the country, fostering coparticipation between the public, academic and private sectors, a critical achievement in a country where agriculture represents 9% of GDP and where 79% of total poverty is rural poverty. Furthermore, the PTA has provided valuable training and infrastructure on animal and plant health and food safety, in particular for tackling foot and mouth disease (FMD), safeguarding a livestock industry that represents 20% of agriculture GDP (since 2009 Colombia is still the only Andean Country certified free of FMD, in the whole territory, where vaccination is practiced). Even though the FTA with the United States was only signed in 2011 (much later than originally expected), the PTA has provided key public goods and services that

13

will help Colombia’s Agriculture Sector increase its competitiveness and transition towards trade liberalization. 3.2.2 The PTA’s Competitive Fund’s rules and operational methodology is widely-viewed in Colombia as a model for allocating scarce public sector resources for agriculture R&D, and for establishing new relationship between the different actors along the supply chains to establish R&D priorities and investment plans (agendas propsectivas). The ability to evaluate, change and improve the rules and procedures in the Competitive Fund permitted by project design, provided space for meaningful participation and shift away from the traditional top-down approach, all major catalysts to counterpart funding from supply chain participants. 3.2.3 On the SPS system investments, the PTA was very successful in implementing low tech approaches towards SPS, such as the Good Agriculture Practices (GAP o "Buenas Practicas Agricolas") and the approval of export protocols with many countries. Furthermore, interaction between public sector organizations such as MADR, ICA and INVIMA has seen notable. Finally, although with difficulties encountered in the execution of the investments, the availability of a NBL3AG for Colombia and for the Andean Region is a resources that will yield large economic returns by allowing for early and precise surveillance, control and monitoring of exotic or emerging animal health issues. Achievement of the objective follows below. PDO: To strengthen the National Agricultural Science and Technology and Sanitary and Phytosanitary (SPS) Systems by supporting the joint participation of both the public and private sectors, through the mechanism of production chains, thereby contributing to the competitiveness of Colombian agriculture and improving the accessibility of export-potential products to international markets. 3.2.4 The PDO was ambitious. It is difficult to attribute the impact of project activities and investments to the PDO and its indicators, but there’s also a gap between the PDO indicators and the PDO itself (the indicator is related mainly to the SPS investments). Having said this, the PTA did strengthen the National Agriculture S&T and SPS systems, so we conclude that the PDO was indeed achieved. 3.2.5 The PDO indicator specified in the PAD was “Selected production chains have improved their participation in national and international markets: increase in market share indicator, increase in export related SPS indicators for beef & uchuva production”. In order to measure the extent to which the project achieved the PDO, we analyze three distinct indicators: (i) the PDO indicator specified in the PAD (only for uchuva and beef); (ii) the market share and export indicators for other supply chains to which the project contributed (overall measures of sector competitiveness); and (iii) the evidence of strengthening of the National Agriculture S&T and SPS systems. It is important to note that in all cases, significant attribution issues arise as other factors (outside project activities) also contributed to the evolution of these indicators.

• The cattle (beef) production was in a steady decline before project appraisal (1997-2003). It increased from 3,724,000 head of cattle slaughtered at project appraisal (2004-2005) to 4,100,000 at project closing (2011).

14

source: FEDEGAN

• Although cattle exports decreased between project appraisal and closing (with a sharp

increase and decline as per the graph below) due the closing of the Venezuelan border in 2009, the market share of Colombian cattle production with respect to world cattle production (heads slaughtered), increased from 1.4% in 2005 to 1.6% in 2011.

• Colombia diversified its cattle export markets: Countries importing Colombian cattle increased from 2 countries in 2005 (Venezuela, Holland) to more than 10 countries in 2011.

Destination of Cattle Exports from Colombia

3000000

3200000

3400000

3600000

3800000

4000000

4200000

4400000

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2007

2008

2008

2009

2010

2011

COLOMBIA - Slaughtered heads of cattle

Project Begins

Project Begins

15

• According to the information of the MADR, cattle productivity has increased between

2005 and 2008 as measured by several productivity indicators and at comparable levels to countries like Uruguay.

• Uchuva production increased both in the share of total agriculture production and total hectares cultivated between 2005 and 2010.

Participation of Uchuva in the share of agriculture production and cultivated area in

Colombia

• The value and volume of exports of Uchuva increased by 10.5% and 5.3% respectively

between 2005 and 2010, however the export markets showed a concentration towards Holland as the main importer of the fruit.

Total Uchuva Exports from Colombia and Destination Markets

-

20.000

40.000

60.000

80.000

100.000

120.000

140.000

160.000

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Venezuela Líbano Antillas Holandesas Perú Resto del Mundo Total Carne Bovina

% P

ART

ICIP

ACI

ÓN

TON

ELA

DA

S

Fuente: DANE-DIAN

0,00%

0,02%

0,04%

0,06%

0,08%

0,10%

0,12%

0,14%

% Participación en Producción (Ton)

% Participación en Área Cosechada (Ha)

%

Fuente: Producción - Evaluaciones Agropecuarias Municipales. MADR-CCI.

Project Begins

Project Begins

16

• An impact evaluation study of the agriculture R&D investments through subprojects

showed positive and statistically significant impacts of investments in production, income per hectare and agriculture value added.

• Overall Agriculture Exports (in Value) increased significantly over the project period, compared to the trend before appraisal, driven mainly by cattle and flowers and plants. Without project contributions to maintaining the “free of” status in animal and plant health, the growth in agriculture exports would not have been possible.

Source: WDI

Colombia – Agriculture Exports Composition

-

1.000

2.000

3.000

4.000

5.000

6.000

7.000

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

Holanda Alemania Bélgica y LuxemburgoFrancia Suecia Resto del MundoTOTAL UCHUVA

% P

ARTI

CIPA

CIÓ

N

TON

ELAD

AS

Fuente: DANE-DIAN

0

1000000

2000000

3000000

4000000

5000000

6000000

1990

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

Export Value (USD 1000) of Agricultural Products

Project Begins

17

Source: WITS

• A self evaluation of public sector institutions in the National SPS System, in particular

ICA and INVIMA, showed that investments were relevant and useful, boding well for capacity building and institutional strengthening impacts in the strengthening of the System. Furthermore, 25 participatory agriculture R&D agendas were developed (5 more than expected) and 591 agriculture R&D subprojects executed (366 more than anticipated).

• According to the impact evaluation assessment, More than 10 institutions in the agriculture sector were left with sufficient capacity to undertake agriculture R&D planning and prioritization, leaving the country with a critical mass of institutional capacity to undertake such medium to long term planning and prioritization exercises.

3.3 Efficiency 3.3.1 A least cost approach was followed for SPS investments in order to meet the animal and plant health international standards (which yield significant positive economic rates of returns). On the Agriculture R&D investments, an economic and financial analysis was performed based on data and information from 7 selected supply chains and used the DREAM4 software. However, the supply chains selected for the economic and financial analysis during appraisal were not the same ones that eventually got funding from the Competitive Fund. Therefore, for the ex-post economic and financial analysis we selected 7 supply chains that represented 70% of the agriculture R&D investments. The methodology is detailed in Annex 3. Results are as follows:

• Expected internal economic rates of return (IERR): These ranged from 23 to 140% on the seven selected supply chains at appraisal.

• Actual IERRs: Ranged from 14% to 99% for the seven selected supply chains at project closing.

4 DREAM was developed by the International Service for National Agriculture Research (ISNAR) and is available on the International Food Policy Research Institute (IFPRI) web site.

18

• Fiscal analysis: The direct impact of revenue generation is likely to be modest in the short run and thus the project’s fiscal impact is negative. In the long-run however, experience shows that fiscal revenues/impact will benefit from the improved financial situation of beneficiaries as evidenced in the increase in income per hectare in the impact evaluation study and its effect on their purchasing power and expenditure patterns, as well as organized cost recovery provisions in investments in the SPS System.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 3.4.1 The overall outcome rating of Moderately Satisfactory is justified based on the positive outcomes for project objective, substantiated by studies/evaluations. Most project targets/indicators in the results framework were met or exceeded, showing that the project had relevance to the sector of all core elements of the original PDO, and of the project methodology and design (with lessons-based innovations/refinements). There are also promising, albeit preliminary indications of economic and financial efficiency and sustainability and of positive fiscal impact, with good outlook for sustainability of investments, through observed O&M in the larger SPS System investments. Nevertheless, there is an important need to conclude the very important investment regarding the NBL3AG, but with promising perspective of being operational by end of 2012. The aggregate performance and results including institutionalization of PTA within MADR, along with full disbursement of the Loan amount, are deemed to outweigh negative aspects - mostly beyond the project’s control.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 Poverty: As noted earlier, the project was not explicitly targeted to the rural poor but it is known that the beneficiary population was heterogeneous with a large number of small-scale farmers. The income data collected for the impact evaluation showed a statistically significant positive impact in production and income per hectare of subprojects, suggesting that the beneficiaries of the newly developed technologies have good prospects for substantially improving incomes and living conditions, with the caveat that availability of credit and risk management to absorb future production and price shocks is crucial. Some of the subprojects and the technologies developed regard subsectors where small-scale producers are especially relevant, such as “guadua” production, coastal fishing, essential oil plants, etc. The agreed follow up actions contained in the subprojects would overtime have important impacts on poor farmers and workers. 3.5.2 Gender: PTA did not include (although it could have) the explicit targeting of women because of the public good nature of the investments. That said, the question of gender was not discussed in the PAD, nor by any of the PTA evaluations/studies, so its influence on subprojects in terms of degree of women’s participation and/or direct benefit cannot be substantiated. (b) Institutional Change/Strengthening 3.5.3 Institutional evolution is summarized below, complementing Section 3.2:

• The UCP’s technical, operational and analytical skills, capacity to support and supervise subprojects and cooperate with other public sector institutions, promote the program, exploit opportunities, collaborate with multiple stakeholders, and conduct sophisticated M&E, showed notable evolution.

19

• This capacity at the UCP level was, since project closing, mostly transferred and adopted by MADR (in particular the DDPTS), institutionalizing the methodologies, information systems and approaches, contributing to the policy dialogue and their technical role within the sector.

• The performance of PTA subprojects was found not to be linked with the institutions executing the project, although most of the competitive R&D funding did go to a few organizations.

• More than 10 institutions in the agriculture sector were left with sufficient capacity to undertake agriculture R&D planning and prioritization, leaving the country with a critical mass of institutional capacity to undertake such medium to long term planning and prioritization exercises.

• Even though SPS System investments were intended at the building of institutional capacity of organizations such as ICA and INVIMA, in practice the focus of the investments was stronger on physical works, in particular in the case of the NBL3AG, resulting in uneven attention to formative activities in other areas of the SPS agenda.

• Institutional change is, realistically, a long-term effort and should be planned as such from the beginning with adequate funding, tailored programs and a flexible strategy.

(c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.4 The construction of a highly sophisticated Biosafety Laboratory (NBL3AG) was not explicitly foreseen at project appraisal, but given the urgency for addressing the risk of foot and mouth disease (FMD) with such an important livestock sector, the project invested a relatively high amount of Component 2’s resources in the construction of the NBL3AG. The impact of the delays in the operation of the NBL3AG (yet to be Certified and fully operational), has produced a negative perception of the overall project performance. Aside from the questionable decision making regarding contract administration and management of the works for the construction of the NBL3AG, the expected impact for Colombia’s SPS System (as well as for the Region) will be substantial for the competitiveness and risk management of the livestock sector. Only one of this type of Laboratory exist in South America, “El Laboratorio de Fiebre Aftosa de la Direccion de laboratorios del SENASA”, in Argentina which is recognized by OIE as a reference Laboratory for FMD. The Laboratory of PANAFTOSA, located in Brazil, is the Vesicular Diseases Reference Laboratory for South America and is also an OIE and FAO FMD Reference Laboratory. In Paraguay, the construction of a NBL3AG Laboratory was finished in December 2011 and it is process of organization and certification.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 3.6.1 The Project’s Impact Evaluation Study undertook an in-depth impact evaluation, involving beneficiary surveys, focus groups, interviews, and workshops with participants in both the agriculture R&D and SPS systems. For the Agriculture R&D investments, a control group was selected (based on those subprojects that were very close to meeting the required score to be selected). The total number of subprojects analyzed and surveyed included 82 financed by the PTA and 19 in the control group. Positive and statistically significant impacts were reported in production, income and value added variables. Data showed:

• Across the board, less than 30% of farmers new the agriculture R&D instruments available to them through the public system.

• Of those who knew about the instruments, less than 50% used them. • On average, less than 10% of farmers interviewed invested in agriculture R&D. Most

investments in R&D, were done through the gremios or agribusinesses. • Only 20% of farmers interviewed said the subproject involved other actors in the supply

chain.

20

• Farmers who were beneficiaries of R&D investments through the subprojects had 10% higher value of production, 5% higher productivity per hectare, and 7% higher value added per hectare.

3.6.2 The study recommended further improvements and comprehensive interventions. The overall perception of usefulness of project interventions was higher from the agriculture R&D institutions than for the farmers, signaling the need to bridge the gap with the adoption of technologies being produced. The overall methodology for allocating R&D funds was deemed appropriate, but it was recommended not to change criteria from year to year, as well as to ensure continuity in the call for proposals for a better planning of R&D investments in the sector from the various supply chain actors. Given that technology adoption is a medium to long term endeavor, agriculture R&D investments need to be coupled with longer-term agriculture extension and technical assistance strategies and investments. Finally, regional agriculture R&D capacity needs to be supported in order to enable other organizations to compete for the Competitive Funds, without having a few organizations absorb most of the resources.

4. Assessment of Risk to Development Outcome Rating: Moderate 4.1 Sustainability: Evaluation studies indicate generally positive economic viability of investments and sustainability of capacity created. In addition, other factors taken into account include the fact that all subprojects fully implemented to specification and functioning as intended, albeit in some cases at a later stage (for those approved in the last call for proposals); there’s a clear commitment of ICA to operate and maintain the NBL3AG, although not yet operational as of the writing of this ICR; there’s proven ability of the Competitive Fund for agriculture R&D to leverage cofinancing from other supply chain actors and institutions; there are Government R&D and SPS policies that have been updated and strengthen to comply with Free Trade Agreements and international markets; and there is strong stakeholder demand for maintenance of project-supported SPS controls and systems and infrastructure, but partly dependent on final certification of the NBL3AG. 4.2 Environmental management: The environmental integrity of subprojects was supported by an Environmental Management Plan, governing subproject approval and implementation. Furthermore, there was environmental screening of all proposals by UCP’s in-house environmental specialists and expert panel; oversight of environmental compliance by Bank supervision missions; good practices and assessment of the impact of eventual technology adoption by the population at large; and reduced use of agri-chemicals in the environment, an intended collateral effect of the animal and plant health control subprojects.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 5.1.1 This rating is based on the fact that the project objective was not clearly nor directly aligned with project activities and was ambitious. Furthermore, problems with attribution of key project indicators made it difficult to guide the prioritization of project activities and investments; PDO indicators related mainly to Component 2, not all PDO elements; and there was built in flexibility and vagueness of project investments, which led to a lack of coordination among project activities and between components. Nevertheless, the design did introduce an innovative approach to reforming the system of allocation of public funds for agriculture R&D and fostering co-

21

financing between the public and private sectors; and the project was consistent with national development plan and sector strategies. The project design took a responsible stance on fiduciary analysis and fiscal capacity to maintain SPS and agriculture R&D investments going forward and it introduced a participatory approach to prioritizing and allocating public investment planning and financing in SPS and agriculture R&D. (b) Quality of Supervision Rating: Satisfactory 5.1.2 This rating is based on the fact that even with the problems with project design, the team was able to manage and overcompensate for the shortcomings and exogenous factors that threaten project success during implementation. However, the Bank team did miss a few opportunities to revise the indicators, use flexibility in project design to advantage, use participatory processes to link subprojects to the agriculture R&D agendas, and focusing on disseminating technologies to facilitate adoption by smaller farmers. The team undertook regular supervision, including 15 missions over 7 years, well-staffed and reported and with strong follow-up; and added indicators to measure progress, given the problems with the original indicators used in the PAD. Supervision was intense, pragmatic, rigorous efforts to adjust to market and Government priorities and address the importance of (and issues with) the construction of the NBL3AG, seeking the CMU’s direct engagement with project authorities to address the problems with NBL3AG and the investigation by CGR. There was a valuable continuity and transition of Bank Task Manager and technical team during the project’s execution, with a consistent, intensive supervision of Environmental Safeguards and Procurement in the last years of execution, once investments were being realized. Finally, the team supported the UCP develop sustainable, formalized, program-based M&E and safeguards capacity, to address project design shortcomings. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 5.1.3 This rating reflects an attempt to balance performance against a series of complex factors and events, some of which were entirely beyond the control of the Bank or Borrower. In key respects the project can claim to have introduced a positive reform and to have improved efficiency and effectiveness in the allocation of public resources towards agriculture R&D, which can be largely considered successful; and also in investing in key SPS activities, with a major investment in a much needed NBL3AG, which at the time of the writing of this ICR, is not yet fully operational. Given that the issue is Bank performance over the entirety (design and supervision), but weighing the outcomes from 2005-2011, on balance the rating is Moderately Satisfactory. Furthermore, the Bank will continue to accompany the Government in the decision making process about agriculture R&D policies and programs, thus increasing the likelihood of the sustainability of project investments.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 5.2.1 This rating is based on the high degree of GOC support for preparation and desire to maximize investments and outcomes by exploiting successful economic and sector reforms of the period. At the same time, this rating is balanced by the negative performance in the administration and technical supervision of investments in NBL3AG. It is important to point out the importance of the Government’s support for a new, demand-driven approach for determining agriculture R&D priorities and allocating public funds, and the fact that the GOC adopted PTA methodology as its own for the allocation of agriculture R&D funds through call for proposals of

22

sub projects. The support to PTA’s methodology and funding for agriculture R&D has waned down in the past year with the new Government, although budgetary support to NBL3AG has continued to be strong. Finally, it is important to note that there were significant effectiveness delays which resulted in the need to extend project closing dates. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 5.2.2 Performance of the UCP was consistently satisfactory, committed effort, even with great challenges in the earlier years, to launch the call for proposals for agriculture R&D investments. The PCU coordinated PTA with relative autonomy, but changes in PTA coordinator due to changes in the political leadership of the Government produced some relatively long periods of inactivity towards the end of the project. There were some capacity gaps in response to the delays and problems with the construction of the NBL3AG, the UCP caught up and learned by doing, trying to maintain efficiency, management and oversight of project execution, M&E, FM and procurement, with periods of inactivity due to transitions in leadership. The PCU helped deliver influential training to subprojects and technical teams responsible for the agriculture R&D investments, in particular with respect to fiduciary aspects, safeguards and M&E. The M&E and supervision of environmental safeguards was a major UCP accomplishment - with strong Bank support - and its progress reporting, diagnoses and research products are of excellent quality and utility. With the exception of problems with the NBL3AG, skills of the Project Coordinators and UCP’s technical team were invaluable assets and combined with the Bank team, converted the project into a successful program. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 5.2.3 On balance, overall Borrower performance must be rated Moderately Satisfactory reflecting the complex realities of the delays in the construction of the NBL3AG, which brought the project reputational issues and due to the fact that the NBL3AG is still not fully operational, although with a high likelihood of being so in 2012. Coordinator, technical and administrative teams, close collaboration with the Bank in addressing issues with NBL3AG and bringing PTA to a successful close and accomplishments in safeguards and M&E.

6. Lessons Learned 6.1 The project generated many lessons, presented selectively below (see also Annex 7): Operational

• Operations in the rural sector with a high degree of innovation require an extended horizon, intensive supervision, continuous support for capacity building, and repeat financing to maximize impact, learning and sustainability. The time required for projects such as PTA to implement and mature greatly exceeds Bank averages. The importance to such projects of experienced Bank and Borrower project teams with continuity of leadership and a focus on capacity and institution-building cannot be over-stated. PTA was unique in its strategic, innovative approach to allocation of public funds for agriculture R&D, as well as the strategic investments in the SPS system. Responding realistically to the project’s design features and ongoing needs, the Bank team was able to include in the team highly specialized experts on a regular basis, contributing directly to the project’s outcomes/success.

• Formally establishing and staffing an M&E and safeguards unit in the UCP - which is itself embedded in MADR, the principal sector institution - immediately established the

23

mainstream credentials of this important activity. Continuous support from supervision missions and an increasing body of research products and data, plus the development of regional workshops and interactions with subproject managers, dissemination of evaluation studies, and databases, provided a strong incentive to sustain the activity, utilize its products and build further capacity within the MADR and the sector.

• A pre-defined and well-prepared portfolio of investments in SPS ready for

implementation remains prudent, and can mitigate the difficult problems with construction and operation of highly sophisticated works, which absorb much time and additional costs in “learning by doing”. But such approach still need to allow a high degree of flexibility in the execution stages, early and comprehensive attention to institution-building commensurate with the type/difficulty of the SPS investment and capacity of institutions involved in operating and maintaining such infrastructure. Furthermore, the construction of sophisticated laboratories (i.e. NLB3AG) requires specialized and continuous orientation and supervisions during the planning and execution of the construction

• For SPS interventions to be sustainable, they must have pre-established prioritization criteria and require counterpart funding. The PTA has shown that it is important to establish, during project preparation/appraisal, clear prioritization criteria for the resources within the system and institutions. Furthermore, the activities supporting the SPS system must also have specific (earmarked) counterpart funding with a clear public sector support in the promotion and diversification of the agriculture and agroindustrial exports in order to be implemented effectively and maintained afterwards.

Maximizing Impact

• Leveraging maximum impact to farmers from resources allocated for agriculture R&D subprojects requires the identification of investment proposals in a transparent, strategic and participatory manner, promoting agriculture R&D investments (at the Call for Proposals stage) to include agriculture extension mechanisms which ensure high quality, sustainable technology adoption by farmers. Studies reiterate the importance for subproject counterparts to transfer technologies to end-beneficiaries (farmers), and thus subproject design should diagnose the capacity of the organization to undertake such agriculture extension once the technology has been developed and proven. Considerable creativity is needed – starting during sub project preparation - to leverage sources of expertise in rural extension, and adequate resources to fund capacity-building activities. The strategic approach for the Call for Proposals of a Competitive Fund also avoids the effect in having a large atomized number of subproject, without much interrelation among them, and thus, with a lower level of overall impact in specific supply chains and/or geographic regions.

• Furthermore, private sector actors in the agro-industrial sector and producers should not be assumed capable of adopting the technology on their own to subproject investments in agriculture R&D. Subproject investments in R&D are necessary to stimulate technology generation and development in the sector but not sufficient to guarantee the adoption of the technology by agribusinesses and/or farmers, especially if financial resources and lines of credit are unavailable or inaccessible. Smaller scale producers/entrepreneurs in particular lack the organization, commercial records and collateral to support loans from regular banks in order to adopt new technologies

24

requiring initial capital for implementing, and thus, are likely to remain dependent on public sector support for agriculture extension and incentives for adoption of agriculture technologies in the absence of alternative financing strategies.

• However, the value chain approach towards agriculture R&D, showed to be an effective way to channel public resources for improving productivity and competitiveness. The integration of value chains both in the process of the development of national R&D agendas around specific value chains as well as their active participation in the execution of R&D subprojects allowed to maximize the effect of R&D investments on value chain development. Nevertheless, this approach requires a great deal of participation of all concerned actors in the planning and prioritization process.

Competitive Funds • The competitive grant model, around organized value chains, is a promising approach

for countries with such diverse agricultural sectors as Colombia. Progress in these countries will always be a geographical and product based mosaic that can be served better by subprojects that are designed bottom-up rather than by central research institutes (the traditional alternative), even though the academic influence in PTA subprojects may appear to be big, almost all subprojects had the participation of a final user group. This model of demand-led R&D for value chain development also improved relevance and accountability of all actors in the national R&D system, as well as increased cost-sharing of R&D investments.

• Call for proposals need to be sequenced and established in a predictable manner in order to promote the efficient programming of agriculture R&D proposals and activities. Otherwise, as was evidenced after year 2 of the PTA project, research institutions would submit proposals which may not be ready for submission, given the lack of strategic focus and the uncertainty about the financing of future rounds.

• Competitive Funds may not be the best tool for the promotion of smaller scale research organizations in countries with large and diversified agriculture sectors. As evidenced in the PTA, due to the competitive nature of the Calls for Proposal, the subprojects were concentrated in only a few research institutions that were able to dominate in most requirements and levels of co-financing. Thus, alternative (but complementary) methods of allocating agriculture R&D funds must also be available in order to also promote all types of research organizations and themes.

• The need for strategic communication at project preparation and sharing of

experiences and lessons among subprojects during implementation is a repeating theme in PTA studies. The experience with the interactions between the PCU and subprojects shows that beneficiaries and executing agencies of the subprojects can benefit greatly from a prepared communications strategy, including learning from other related subprojects in the same supply chain or area of research. Events and means to share information and experiences need attention at the aggregate level to promote general project objectives and requirements, and especially at the micro level via opportunities and channels for subprojects and beneficiaries/stakeholders to learn about each other, as well as their rights, obligations, the nature of and rationale for planned activities, and what happens once the subproject activities are completed.

25

Economic Analysis

• Evaluation results for subprojects suggest that economic returns depend mainly on the adoption rate and the size of the sector to be impacted by investments. This implies that it is necessary and priority for future agriculture R&D Projects to mainstreaming economic analysis during subproject preparation and implementation and beyond in order to provide a framework for agriculture R&D subproject managers to think about the impact of their actions in terms of costs and benefits, thereby leading to a more efficient allocation of scarce resources. The availability of data on the real impact of agriculture R&D sub-projects would assure more reliable economic evaluation for future subproject selection and decrease the wide range of assumptions taken.

• Finally, for both agriculture R&D and SPS investments, it is crucial that economic analysis be mainstreamed during the evaluation and implementation of subprojects or investments not pre-defined in the PAD. The ex-post economic analysis has been based on verifiable data about production and prices from official sources (Agronet, DANTE, ENA) and, on the other hand, on assumptions and parameters on the impact of the technology (assuming is fully adopted) built for each sub-project buy the researches. During this exercise, it becomes necessary and priority for future Projects to mainstreaming economic analysis during project implementation and beyond in order to provide a framework for project managers to think about the impact of their actions in terms of costs and benefits, thereby leading to a more efficient allocation of scarce resources. The availability acute data on the real impact of SPS and agriculture R&D sub-projects and investments would assure more reliable ex-post economic evaluations and decrease the wide range of assumptions taken.

26

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies Comments from MADR to the ICR: 1. The ICR correctly states that the PAD established 3 years for the implementation of subprojects, but it also considered the ones cofinanced in the last year will exceed the closing date of the project, for which an alternative was proposed. 2. It is important to note that most of the indicators that did not reached the levels expected, have since the closing of the project, achieved the goals. 3. It is important to clarify that the subprojects presented had to be articulated within an agriculture R&D program that defined the coordination among them. 4. It is also important to note that all subprojects included an agriculture technology transfer strategy, but only 10% was assigned to such task and it is not sufficient to guarantee the results. 5. The Impact Evaluation only showed increases in productivity and not production of subprojects. 6. The delay in the implementation of SIGP had as a consequence the delay in the revision and evaluation process of subprojects for evaluators, organizers, and beneficiaries; but did not intervene with the implementation of the subproject. 7. During the PTA there were no training on issues related to economic impacts. 8. With respect to the financial sustainability of the PTA, the ICR statements do not reflect the reality of what has occurred since 2010: (i) the real budgetary resources of ICA and INVIMA have increased; and (ii) even no new call for proposals were issued since 2008, MADR has continued financing agriculture R&D programs. 9. MADR did not provide continuity to PTA, but the strategies adopted by the project have been implemented through: (i) the allocation of resources to subprojects funded by the Fund Francisco José de Caldas and Colciencias; (ii) ICA and INVIMA have established long term programs for investing in MSF. (b) Cofinanciers N/A (c) Other partners and stakeholders N/A

27

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1 – Knowledge Generation and Innovation. 18.95 18.8 99

(a) R&D Agendas 2.8 1.95 70 (b) Competitive Fund 16.15 16.85 104

2 – Strengthening the National SPS Measures System 7.3 7.9 108

3 – Project Management, M&E 2.0 3.3 165 Total Baseline Cost 28.25 29.72 105

Unallocated 1.75

0.00 -

Total Project Costs 30.00 29.72 99 Front-end fee IBRD 0.00 0.00 .00

Total Financing Required 30.00 29.72 99

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower MADR funding of subrpojects

0.00 15.00 -

International Bank for Reconstruction and Development 30.00 29.72 99

Local Sources of Borrowing Country Cofinancing of Subprojects 2.00 10.00 5000

28

Annex 2. Outputs by Component 2.1 The project had three components, as stated below. Drawing on the Borrower Completion Report (PTA, 2011), the impact evaluation study of the PTA by Fedesarrollo, and other sources, the following summarizes the main achievements under individual components: Component 1: Knowledge Generation and Innovation (69% of project costs) Overall rating: Satisfactory 2.2 The project was to finance the strengthening of agricultural production chain actors in the definition, co- financing and implementation of agriculture R&D. This would be done by financing: (i) the development of R&D agendas for twenty production chains, which would entail the carrying out of the background studies for these agendas as well as the evaluation and strengthening of the capacity of these chains to develop, co-finance and implement these agendas; (ii) the operation of a Competitive Fund that would co-finance production chains’ R&D subprojects which are consistent with the agendas defined; and (iii) the design and implementation of a unit fully adopted within the Ministry of Agriculture (MADR) responsible for the allocation of public resources to the agendas already defined and to the development and implementation of new production chains’ R&D agendas. 2.3 The Competitive Fund was open to considering all technically feasible, economically sound and environmentally sustainable subproject proposals presented by actors within the National Agriculture Science and Technology System, based on priorities set in the R&D agendas of each supply chain, implementing capacity, and consistency with project objectives criteria. Selection criteria for subprojects evolved from the earlier calls for proposals. Originally, subprojects were required to present evidence of: (i) economic sustainability; (ii) strategic linkage to the development of the supply chain; (iii) integrity and internal consistency; and (iv) socio-environmental impact declaration. However, for the 2006-2008 call for proposals, the criteria were reduced to present evidence of the: (i) fit within the general context of the development of the supply chain; and (ii) the overall objective and impact expected. In the first call for proposals, the subprojects would be evaluated by a selection committee that would prioritize the proposals based on the impacts on the supply chains and on the regions. However, in the 2007-2008, the prioritization by region was dropped.

Figure 1 – Subprojects presented to the Agriculture R&D Competitive Fund of PTA

29

2.5 The PTA financed the systematic training of 386 staff across public sector institutions in the SPS system (including ICA and INVIMA) in the areas of traceability, food safety, certification, and primary agriculture health and safety. It also supported: (i) the first design phase of the national SPS information network; (ii) the approval of ICA’s control and surveillance authorization system; (iii) achieved the conditions required for international recognition of beef product admissibility; (v) fulfilled of conditions required for international recognition of fruit and vegetable specific product chain admissibility (uchuva); and (vi) small farmers trained in GAPs and GMP techniques. 2.6 Outputs under this component were the following:

• Total 591 subprojects were implemented: 117 Bank-supported subprojects; and 474 MADR-supported subprojects (160% more than the original target of 225).

• 32 Departments and 21 supply chains benefited from the PTA-supported subprojects. • 25 agriculture supply chain R&D agendas developed (25% more than the original target). • 7 MADR staff permanently working on agriculture R&D activities (10% more than

originally planned). • Level of co-financing from participants in subproject reached the expected 25% of

subproject costs. • Installed capacity within 10 agriculture supply chain organizations to prepare and

prioritize agriculture R&D agendas. Component 2: Strengthening of the National Sanitary and Phytosanitary (SPS) Measures System (25% of project costs) Overall rating: Moderately Satisfactory 2.7 The primary purpose of this component was to respond to the sanitary standards of international markets and thereby enhance the admissibility of agricultural products to these markets. The needs identified in the SPS system at appraisal were prioritized vis-à-vis requirements of two production chains (beef, and uchuva-Physalis), which at the time were showing a competitive potential particularly in the US market and which were expected to have a spill-over effect to other chains. The beef chain was chosen within the livestock sector because its products were highly complex in sanitary terms, not only because they face zoo-sanitary barriers but also other food safety-related barriers. Although the investment plan may have been designed on the basis of overcoming barriers for a particular fruit (uchuva), its range covered the needs being faced by a large share of the fruit and vegetables chain to successfully access international markets. 2.8 This would be done by financing: (i) training professionals and technicians attached to the SPS systems; (ii) capacity strengthening (i.e. training, development of procedures, equipment, infrastructure) to mitigate the chains-specific sanitary barriers to beef products; (iii) capacity strengthening (i.e. training, development of procedures, equipment, infrastructure) to mitigate the chains-specific sanitary barriers to fruits and vegetables; (iv) designing and implementing a national SPS information network; (v) designing and implementing an authorization system allowing for Audit and/or Test and Diagnosis Laboratory activities to be delegated to a third party; and (vi) technology transfer to small farmers on best practices (GAP and GMP) to improve the sanitary and phytosanitary conditions of agricultural production. However, certain activities related to the strengthening and operation of laboratories and inspection activities for beef products, the presentation of protocols to the EU on the Mediterranean Fly affecting fruits, and

30

the implementation of the residue control plans, fell short of their original objective (although all showed some progress). 2.9 Outputs under this component were the following:

• 386 trained in animal and plant health and authorization systems (74% of the original target of 521).

• First phase of the design of the national SPS information network has been completed, but the implementation of the inter-connectivity of the information systems of each institution is still pending (60% accomplished).

• ICA’s SPS control and surveillance system was approved5 and activities were authorized, confirming the authorization system’s operation.

• Required conditions for admissibility of beef products were met, including the establishment of 8 control posts, and recognition of Colombia as free of foot and mouth disease (FMD), where vaccination is practiced. The strengthening of the laboratories of ICA and INVIMA related to beef biosafety and other aspects was mostly completed, with the exception of the certification of the NBL3AG (expected by end of 2012).

• Required conditions for admissibility of fruit and vegetables (particularly uchuva) were mostly met, with the exception of the presentation of protocols to the EU on the Mediterranean Fly which was not completed.

• 460 extension agents trained in GAPs and GMPs (47 times more than the 150 originally planned).

2.10 Table 2.1 shows aggregate outputs by type of investment and area of intervention. Table 2.1: PTA Project Outputs

Key indicators

Base line

Target Actual Accomplished

(%) Comments Mid Term Final

Component 1: Knowledge Generation and Innovation

Number of R&D agendas developed in a participatory and decentralized manner among production chain actors

0 8 20 25 125%

Percentage increase in the co- Financing of R&D agenda by the production chain participants

- 10 – 20% 25-30% 25% 100%

No. of subprojects that successfully introduced innovations. 0 80 225 571 254%

Given that this is an impact indicator, measurement was taken based on outputs (subprojects approved an in execution)

Performance rating of the personnel of MARD with improved qualifications who would be responsible for the continued implementation of a public R&D agenda for the agriculture sector and co-financed by the private sector

- 15% 25% 35% 110%

Based on 20 MADR staff that participated in trainings and Project activities, now 7 are permanent staff of DDTPS working on R&D

5 http://www.ica.gov.co/Areas/Pecuaria/Servicios/Enfermedades-Animales/Laboratorios--Autorizados.aspx

http://www.ica.gov.co/Areas/Pecuaria/Servicios/Enfermedades-Animales/Laboratorios--Autorizados.aspx

31

Component 2: Strengthening of the National Sanitary and Phytosanitary (SPS) Measures System

Officers trained in animal and plant health related issues and food safety issues, information and authorization system.

0 414 521 386 74%

First phase of the SPS national information network in operation. 0 1 1 0,6 60%

Authorization system in operation. 0 1 1 1 100%

Fulfillment of conditions required for international recognition of fruits and vegetables (uchuva) and meat chain-specific product admissibility (beef).

0 0.5 1 87% 87%

A few planned activities were not completed, such as the NBL3Ag, the implementation of the residue management plan, and the presentation of the Mediterranean Fly protocol to the EU.

Staff trained in GAPs and GMP knowledge. 0 100 150 7313 100%

Component 3: Project Management, Monitoring and Evaluation (7% of project costs) Overall rating: Moderately Satisfactory 2.11 The primary purpose of this component was to ensure proper project management and execution and finance a Project Coordination Unit which was to be attached to the Ministry of Agriculture, and be responsible for the general management of the project. Output targets for this component included: Level of implementation of Annual Operation Plan in each year of the project; administrative efficiency such as timely delivery of reports (both technical and financial); and the number of MADR staff with improved capacity to support the delivery of new institutional paradigms such as cofinancing, decentralization, and response to demand-driven changes. 2.12 All target outputs were met and evidence of productivity under the M&E portion of this component can be seen in the list of Borrower-generated analyses and evaluations in Annex 9. The PCU produced all required performance/ progress and fiduciary reports including independent audits

32

Annex 3. Economic and Financial Analysis Introduction The ex-post economic evaluation of the Agriculture Transition Project (PTA, in Spanish abbreviations) estimates the economic returns derived from the investments on Research and Development (R&D) sub-projects in agriculture, which has brought the bulk of the resources invested (56%) in relation with the resources devoted to the research agendas (6,5%), to the strengthening of the National Sanitary and Phytosanitary Measures System (26%) and to the Project Management and Monitoring and Evaluation (11%). This annex includes, in sequential order, a brief summary of the main results achieved by the PTA, the methodological framework that has guided the ex-post economic evaluation and the selection of R&D sub-projects evaluated during this exercise, the summary of the results of the evaluation, a brief description of the expenditure on agricultural R&D in Colombia and, finally, some conclusions for future R&D projects Methodological considerations and conceptual framework During the “project appraisal” the economic surplus approach was chosen to evaluate investments in agricultural research and development (R&D) following World Bank’s good practices. This approach was preferred because it underlines the conventional economic rationale for public resources allocations to generate public goods in agriculture, which results in important social profits (Alston and other 1995). At Project preparation stage, type and size of sub-project investments was not known “a priori” and the ex-ante economic evaluation of the PTA focused on seven (7) production chains – corn, cocoa, rubber, sugar cane (panela), plantain, meat and tropical exports fruits (uchuva). The analysis was performed using the results of sub-projects funded from the first call of proposals of the Competitive Fund6 of the Ministry of Agriculture and Rural Development (MADR, in Spanish abbreviation). For the supply chains of meat and uchuva, the economic analysis was based on examples of investments on sanitary and phytosanitary measures respectively. The ex-ante evaluation used the Dynamic Research Evaluation for Management (DREAM)7, a menu-driven software package for evaluating the economic impacts of agricultural R&D. During PTA implementation, the value chains eventually funded through the agriculture R&D investments were not the same ones analysed at project appraisal. The only exceptions, meat and uchuva, were evaluated ex-ante through examples of SPS investments. Therefore, it is not possible to compare the ex- and post- economic analysis in the ICR. The PTA has funded a total of 117 R&D sub-projects through the Agricultural R&D and Innovation Fund of MADR. At project completion, an ex-post economic evaluation has been undertaken on seven (7) production chains which represented more than 70% of the PTA R&D investments and 68% of the sub – projects. Table 1 shows the distribution of subprojects among

6 Agricultural R&D and Innovation Fund- Fondo de co-financiación de proyectos de Investigación, Desarrollo tecnológico e Innovación para el Sector Agropecuario para cadenas productivas del Ministerio de Agricultura y Desarrollo Rural. 7 DREAM was developed by the International Service for National Agriculture Research (ISNAR) and is available on the International Food Policy Research Institute (IFPRI) web site.

33

the different chain and highlights the selected chains analyzed as follow: (1) pisciculture; (2) aromatic and medical pants; (3) forestry; (4) biofuel; (5) fruits; (6) guadua and (7) beef. The fact that no subproject has yet completed the process of transfer and that no massive adoption of the new technologies has occurred determines that only exceptionally one can find impact in the field. That represents an important restriction to the effective economic evaluation of the results. However, for the purpose of this evaluation, 16 sub-projects out of a total of 79 within the selected chains were chosen by the PTA Coordination Unit. The sample selection was based on both data availability and major level of technological transfer to the producers and was not a formally randomized selection of all sub-projects. However, the sample size was considered as sufficient, and was not biased towards successful projects.

Table 1: PTA Sub-proyectos Production

Chain

Number of Sub-projects

Contrapartida (Pesos)

Cofinanciación (Pesos)

Valor total

2 0 0 6

2 0 0 7

Total

(*) Total

Fondos

MADR - PTA

Pisciculture 7 13 20 7,371,291,460 6,221,709,287 6,221,709,287 13,593,000,747 Aromatic 12 12 7,023,867,285 6,448,463,285 6,448,463,285 13,472,330,570 Forestry 11 11 6,736,007,131 4,934,499,157 4,934,499,157 11,670,506,288 Ethanol 7 7 4,510,821,060 3,924,327,000 3,924,327,000 8,435,148,060 Fruits 17 17 16 4,359,452,800 3,083,591,000 1,502,000,500 1,581,590,500 7,443,043,800 Guadua 5 5 3,032,197,200 2,874,306,200 2,874,306,200 5,906,503,400 Beef 7 7 7 3,009,736,700 1,895,726,000 947,863,000 947,863,000 4,905,462,700 Fisheries. 6 6 2,704,813,948 2,189,005,620 2,189,005,620 4,893,819,568 Goat 2 2 2,931,309,000 1,345,090,000 1,345,090,000 4,276,399,000 Shrimp 2 2 2,052,641,500 1,758,017,000 1,758,017,000 3,810,658,500 Biodiesel 3 3 1,812,530,000 1,736,500,000 1,736,500,000 3,549,030,000 Vegetables 6 6 6 1,595,601,272 1,234,759,000 617,379,500 617,379,500 2,830,360,272 Potatoes 6 6 6 1,379,765,600 1,186,127,510 593,063,755 593,063,755 2,565,893,110 Dairy 5 5 5 1,321,314,530 1,034,994,014 517,497,007 517,497,007 2,356,308,544 Flowers 2 2 1,118,244,000 937,006,999 937,006,999 2,055,250,999 Cassava 1 1 974,003,500 783,456,321 783,456,321 1,757,459,821 Fique 2 2 239,231,100 228,321,400 228,321,400 467,552,500 Cotton 3 3 3 667,304 591,878 295,939 295,939 1,259,182 Total 51 64 117 43 52,173,495,390 41,816,491,671 4,178,099,701 37,638,391,970 93,989,987,061

(*) With Fund Source: PTA The Model DREAM was used to estimate the economic returns or Internal Rate of Return (IRR) and the changes in producer and government’ surpluses generated through the investment of public resources in R&D in agriculture. The DREAM model focuses primarily on the evaluation of new technologies applicable at the farm level. But while the immediate impacts of R&D often arise from technology-induced changes in yield potential and production costs at the farm level, the broader economic effects also depend upon a range of biophysical, social, and market factors, for which DREAM requires to provide quantitative estimates. The process of obtaining information on technology (potential impacts on production costs and yields if R&D successful and fully adopted) and adoption scenarios from scientists, farmers, extension workers and others is critical to the success and reliability of the analysis. However, no baseline information was available for sub-projects approved and implemented by PTA. Some of the information used for the analysis was obtained from the Project Management Integrated System (SIGP in Spanish abbreviation) that kept sub-project applications files and reports. As the PTA Monitoring and Evaluation (M&E) System did not collect information on the expected economic impacts (change in yields or production costs per hectare resulting from the

34

research) and/or diffusion potential (estimated number of farmers to be reached) for each sub-project, data were essentially obtained in the field through consultation with the scientific teams of the selected R&D sub-projects. During those field meetings, participating researchers were guiding during the evaluation of its researches and on the impact derived from their investments. Therefore, the parameters and assumptions that fed the DREAM model were built in consultation with experts for each of the 12 sub-projects finally evaluated, and then modified by the analyst for the construction of various analysis scenarios. Each sub-project was evaluated assuming a percentage of farmers (or total production) nationwide reached by the technology. The base year for the analysis corresponds to the start-up year of the sub-project (i.e. 2007 or 2008 depending on when the call for proposal was launched). As was the case at appraisal, the analysis has been carried out for a 20-year period for all sub-projects, and includes the time required for the generation, dissemination, and adoption of the technology being evaluated. The discount rate is 12% as suggested during appraisal by the National Planning Department for this kind of projects in Colombia. The option of “small open economy” was selected for all supply chains evaluated. The production figures and annual growth rates come from Agronet and the National Agricultural Survey (ENA) of DANE8. The prices of the products were taken from the Agricultural Price Information (SIPSA, in Spanish abbreviations) of the Colombia Corporation International and, unit supply (production) elasticity was assumed for all products analysed. In order to incorporate the effect of the policy distortions (taxes and subsidies) in the analysis and due to the lack of studies in Colombia, the Producer Subsidy Equivalent (PSE) was supposed to be both 0 and 10%. In the case of beef was used estimated by LJ Garay (2005). With regards the technology and data related to R&D and its potential impact (assuming the technology is fully adopted), information was collected on the number of years required to perform the R&D, the probability of R&D success, and the anticipated change in the unit cost of production (relative to the base year) and yield in both scenarios “with project” and “without project”. The adoption data collected provides information on how to transform the “potential impact” of new technology into “realized impacts”. They correspond to the time expected to elapse from the release of a new technology until the maximum adoption level is attained (the adoption lag), and the expected maximum adoption level expressed as a percentage of the production within the region that will be obtained using the new technology. The DREAM model has two forms of adoption: linear and sinusoidal. The form of adoption that has been assumed for the cases tested is the lineal form. The value of the investment refers to the costs of research, development, and dissemination of the new technology.

8 Departamento Administrativo Nacional de Estadística

35

Estimates of IERR and other benefices for investments in the Agriculture Transition Project (PTA) Table 2 provides a summary of the R&D sub-projects evaluated. Due to the relatively small universe of selection, the result of this analysis is not representative and cannot be added or extrapolated to the universe of the 117 sub-projects. The ex-post economic analysis is based on actual costs and benefits executed by each sub-project and on the incremental net benefits (i.e. “with technology” minors “without technology). Cuadro 2: Cálculo de la TIR de sub-proyectos de I+D+I financiados por el PTA

Production chain

R&D Sub-project Parameters Results

Sub-

proj

ect

valu

e M

illio

n U

S$

Prod

uctio

n va

lue

mill

ionU

S$

Prod

uctio

n C

ost

incr

ease

%

Yie

ld

incr

ease

%

Su

cces

s Pr

oba.

%

Ado

ptio

n L

evel

%

T

IR %

Farm

ers

surp

lus

US$

’000

Gov

. sur

plus

U

S$’0

00

Tilapia Nutritional improvement

599,075 125 2 30 100 4 74 7,659 565

Native fish Genetic 247,500 1.5 15 10 80 50 31 784 77 Ornam. fish Sanitary 306,805 2 5 20 100 60 20 483 0 Thyme Better practices 1,419,568 0.5 5 30 100 80 17 1621 0 Achiote Better practices 673,855 0.7 5 8 70 50 99 75,479 2,2662 Melina Seed improvement 815,653 7.3 0.39 18 80 15 18 835 90 Acacia Seed improvement 815,653 74 2 31 50 15 74 8,605 928 Teak Seed improvement 815,653 0.04 (-0.17) 21 80 30 14 682 72 Uchuva Better practices 180,834 14 (-44) 77 90 50 21 283,380 21,780 Arazá Better practices 223,457 2.2 0 3 90 100 71 21,736 0 Beef bovina Genetic 163,817 155 40 20 50 20 95 12387 246

Nutritional improvement

213,870 155 10 40 80 50 39 198 3.47

Tomato (*) Better practices 275,657 3 5 80 70 14 253 0 Source: Own elaboration based on DREAM model results Results of the ex-post economic evaluation undertaken using the DREAM model are as follows: 1. The Internal Economic Rates of Returns (IERRs) range from 14 to 99% (higher than the

comparison rate of 12%) and assumptions were made for each case. 2. The financial resources required for the dissemination and extension activities of sub-projects

to larger scales of production are not included in the calculation of the IRR. In case they were known and included in the analysis, it would decrease the value of the IERRs calculated here.

3. In all cases analyzed there is a jump in productivity (yield) resulted from the immediate effect

of the sub-project, accompanied by a slightly increase in additional production costs , except in the case of improved seeds for teak.

4. The success probability of the sub-projects evaluated varies between 50 and 100%. This depends on the stage of the research , type of sub-project and the scientific and corporate entity.

5. The adoption levels of the technology are definitive in the calculation of the IERR, which is

demonstrated for almost all the sub-projects analyzed.

36

6. In sectors of high absolute production value such as tilapia, acacia and bovine genetics, the expenditure in R&D are relatively small and therefore it does enhance the final result if the IERR. This does not happen if the sector has a low production value such as thyme, melina, tomato unless the probability of success and adoption level is sufficiently high. The use of DREAM is also illustrative to point out how the IERR decreases if the analyst supposes open trade policies or without subsidies (i.e the evaluated sub-projects receiving border protection and subsidies (assumption of a PSE of 10% ) show higher IRRs than those without).

Other benefits of the PTA, which are not specifically accounted for in the above estimates of IERR relate to the significant progress made in the process of knowledge generation through the sub-projects – within the interior of the production chain, as well as capacity building and scientific documentation of accomplishments. The sub-projects have contributed to improve the capacity of agricultural innovation services providers (researchers and research organizations) and to increase the interest and capacity of rural producers to adopt innovations, although dissemination and transfer has taken place only in some sub-projects, especially when the research was participatory and involved the farmers (“gremios”). The positive social impacthas been achieved mainly through the empowerment of producers organizations. Agricultural R&D spending in Colombia Investment in Research and Development (R&D) in Colombia – including public and private spending in all economic sectors – is among the lowest in Latin America and the Caribbean (LAC) region. As of 2009, it represents only 0.15% of GDP, against a regional average of 0.54% (1990-2009) and 0.67% in 2009. Expenditure per capita in R&D is equally small in Colombia (USD 8.04 vs. regional average USD 85.14 in 2009)9, but it is above Bolivia (USD 1.86 in 2009), Guatemala (USD 1.49 in 2009), Peru (USD 3.78 in 200410) and El Salvador (USD2.19 in 2009). Over the period 1995-2009, Colombia’s R&D expenditure has decreased from 0.29% in 1995 to 0.15% of GDP in 2009 while the regional average has slightly increased over the same period (reaching 0.67 % of regional GDP in 2009). Specifically in the agricultural sector, data on R&D are collected under the Agricultural Science and Technology Indicators (ASTI) Initiative facilitated by IFPRI11. As a whole, the LAC region is characterized by an average agricultural research intensity ratio (measured by the ratio of total public agricultural R&D spending as a percentage of agricultural GDP) estimated at 1.14% in 2006. This regional average is high compared with other developing regions of the world, such as Africa (0.65%) and the Asia-Pacific (0.42%). There are, however, very wide fluctuations among LAC countries12, where the agricultural research intensity ratio ranges from as little as 0.2% to as high as 2.0% (which is close to ratios reported in the developed world)13. The PTA’s impact evaluation study analyzes the participation of the PTA resources within the MADR’s budget, where they were assigned. Overall, 70 per cent of the PTA resources, essentially the Component on knowledge generation and innovation was allocated to the

9 After correction for Purchasing Power Parity (PPP), the figures are 14.13 for Colombia and 101.23 regional average 10 There is no specific data for Peru from 2005 11 http://www.asti.cgiar.org/colombia 12 Source: Public Agricultural Research in Latin America and the Caribbean – Investment and Capacity Trends, ASTI Synthesis Report, March 2009 13 Uruguay, Brazil, Argentina, Chile and Mexico spend a relatively large share of their agricultural GDP on agricultural research (over 1.0%), whereas agricultural intensity ratios are significantly lower in poorer countries like Guatemala, El Salvador, Paraguay or Colombai (less than 0.5%)

http://www.asti.cgiar.org/colombia

37

“Science, Technology and Innovation (ST+I)” Programme and the remainder 30 per cent to the “Agricultural Health” Programme. These two programmes represented about 15.6 per cent of the total investment in the sector in 2005 and about 10 per cent in 2010. Those figures mean very little in comparison to investments in specific programmes such as “Direct Support” Programme, which represented yearly about 55% of the total investment over the period 2005 to 2010. The ASTI Country Brief No. 39 of November 2008 highlights the following key trends since 2000 for Colombia: (i) the share of CORPOICA14 in total Colombia agricultural R&D spending and capacity has gradually declined over the past decades in favor of producer associations, other government agencies, and the university sector; (ii) Agricultural R&D spending in Colombia remained stable during 1996-2002 but has contracted substantially in recent years due mainly to severe cuts in CORPOICA’s budget; (iii) More than 90 percent of research carried out by the four principal producer associations is financed through commodity taxes levied on private sector production or exports. CORPOICA, on the other hand, received more than three-quarters of its funds from the Colombia government. (iv) Colombia’s private sector is involved in only limited agricultural R&D although it plays an important (indirect) role in financing R&D. The high economic returns from R&D investments mean that it would be profitable for the Colombian government to invest in agriculture activities such as those implemented by the PTA. However, despite the importance that these activities have for competitiveness of agriculture sector and, more broadly, for economic growth, the science and agricultural technology programs are still disadvantaged in the sector budget priorities, as mentioned above. This evaluation highlights the importance of reviewing the budget priorities and government support for R&D activities where projects like the PTA may have comparative advantages and added value Conclusions 1. The PTA has been a relatively successful project as discussed in this ICR. The results of the

Impacts and results evaluation study finalized by Fedesarrollo in 2011 has showed that there has been important progresses in the knowledge generation process as well as in staff training and documentation of the achievements. The dissemination has started in some of them and the producers are having a positive perception on the results. In general, it is expected positive changes with regard to production and income. In addition, the support provided to the SPS System has been significant and has allowed for obtaining export certifications and access to foreign markets.

2. All these achievements were possible in spite of some implementation shortcomings. The prospective agendas, which should have provided the framework for planning the allocation of resources to research sub-projects and assured the participation of the supply chains actors, arrived after the Competitive Fund had been allocated to sub-projects. That can be a reason why the process of technology transfer to the production has been slow so far and can probably explain why the positive perception on the future impacts of the project is greater among the researchers that among the producer beneficiaries as reporter in the impacts and results evaluation study. Future innovation development public policies should stress the need to prioritize closer and effective linkages between the research and producer dimensions.

14 Corporación Colombiana de Investigación Agropecuaria

38

3. The fact that no subproject has yet completed the process of transfer and that no massive adoption of the new technologies has occurred yet determines that only exceptionally one can find impact in the field. That represents an important restriction to the effective economic evaluation of the results. Over the time the economic impact of PTA investments will be more evident and thus possible to evaluate.

4. The model DREAM, or Dynamic Research Evaluation for Management tool was used to illustrate the possible economic impacts of the R&D investments The model simulates possible economic performances given the current technology accomplishments and the planned subproject’s transfer activities.

5. The ex-post economic analysis has been based on verifiable data about production and prices

from official sources (Agronet, DANTE, ENA) and, on the other hand, on assumptions and parameters on the impact of the technology (assuming is fully adopted) built for each sub-project buy the researches. During this exercise, it becomes necessary and priority for future R&D Projects to mainstreaming economic analysis during project implementation and beyond in order to provide a framework for R&D managers to think about the impact of their actions in terms of costs and benefits, thereby leading to a more efficient allocation of scarce resources. The availability acute data on the real impact of R&D sub-projects would assure more reliable ex-post economic evaluations and decrease the wide range of assumptions taken.

6. Although the above-mentioned constraint, the investments of PTA resources in R&D sub-

projects generate internal rates of returns IRR for the society in the rage of 14 to 99 percent depending on the type of the sub-project and the conditions in which these resources are applied. In all cases analyzed there was a jump in productivity (yield) resulted from the immediate effect of the sub-project, accompanied by a slightly increase in additional production costs, except in the case of improved seeds for teak.

7. The probability of success and the adoption levels are decisive in calculating the IRR, which

is demonstrated for almost all the sub-projects analyses. This is important to take account for more specific evaluations to inform and maximize the use of public resources allocation in research.

8. The use of DREAM model to evaluate R&D projects within different value chains illustrates

that the IRR is linked to government support through trade policies and subsidies: In other words, projects in value chains receiving border protection and subsidies show higher IRRs than those without. This should be taken into account for the allocation of public resources between competitive and non competitive sector in terms of progressive trade liberalization.

9. The size of the sector where the research is applied is also decisive. In sectors of high

absolute production value, the expenditure in R&D is relatively small and therefore it does enhance the final result if the IRR. This does not happen if the sector has a low production value unless the probabilities of success and adoption level are sufficiently high.

10. While the future adoption and sustainability of the sub-projects financed by PTA would be

feasible, the continuous involvement of the Colombia Government is necessary to consolidate the transfer and dissemination process and upscale the successful experience gained under the PTA and extend it gradually to the whole country. This would need not only financial support but, in many cases, political and regulatory support to those sectors whose sub-projects have shown comparative advantages.

39

11. The very high and non-declining rates of returns on agricultural R&D worldwide, together

with very low spending in Colombia compared to other LAC countries and the rest of the world, suggest that there are important investment opportunities in this domain that are not being fully exploited. This means that it is worthwhile, in principle for the government to invest in activities such as those performed by PTA.

40

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Daniel J. Boyce Lead Financial Management Specialist LCSFM FM Specialist Eduardo Brito Senior Counsel LEGAF Lawyer Jose Maria Caballero Lead Agriculture Economist LCSAR Ag. Economist Ann Jeannette Glauber Senior Environmental Specialist AFTEN Safeguards Natalia Gomez Senior Rural Development Specialist LCSAR Rural Development Jose M. Martinez Senior Procurement Specialist ECSO2 PM Specialist Matthew A. McMahon Consultant AFTAR TTL Santiago V. Sandoval Language Program Assistant LCSEN Project Assistant Samuel Taffesse Senior Operations Officer AFMZW Ops. Specialist

Supervision/ICR Claudia Mylenna Cardenas Garcia Consultant LCSFM FM Specialist

Luz Zeron Consultant LCSFM FM Specialist Santiago Torres Consultant LCSPM PM Specialist Maribel M. Cherres Wedemeyer Language Program Assistant LCSSO Project Assistant Indira Janaki Ekanayake Sr Agriculturist AFTAR Ag. Specialist Jeannette Estupinan Sr Financial Management Specialist LCSFM FM Specialist Beatriz Elena Franco Program Assistant LCC1A Project Assistant Natalia Gomez Senior Rural Development Specialist LCSAR Rural Development Ricardo Hernandez Murillo Senior Environmental Specialist LCSEN Safeguards Jaime Estupinian Consultant LCSAR Biosafety Expert Jose Luis Calderon Consultant LCSAR Safeguards Jose M. Martinez Senior Procurement Specialist ECSO2 PM Specialist Matthew A. McMahon Consultant AFTAR Ag. Specialist Willem Janssen Lead Agriculturalist LCSAR TTL Diego Arias Senior Agriculture Economist LCSAR TTL Teresa M. Roncal Operations Analyst LCSAR Ops. Specialist Erika Eliana Salamanca Duenas Program Assistant LCSAR Project Assistant

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY03 72.63 FY04 185.49 FY05 240.23 FY06 0.11

Total: 498.46

41

Supervision/ICR FY06 87.22 FY07 121.26 FY08 164.70 FY09 89.75 FY10 119.466 FY11 84.512 FY12 68.22

Total: 735.128

42

Annex 5. Beneficiary and Stakeholder Survey and Impact Evaluation Results 1. The PTA undertook an impact evaluation in 2011 which included field surveys and qualitative interviews with beneficiaries and the main stakeholders who participated in the project. The evaluation followed a value chain approach. The evaluation included PTA management, outputs, and results for both project components (institutional strengthening and knowledge generation), and, in particular, the evaluation was done in the context of the Science and Technology National Policy, the strengthening of the SPS system, the performance of ICA and INVIMA with respect to the PTA, the midterm evaluation and the impact of the Competitive Fund and its subprojects. 2. The management and outputs of PTA show a positive result. Producers and research organizations consider that the call for proposals to access the Competitive Fund was adequate. However, it is important to note that, from the management up to the impacts of the Competitive Fund, the image of the PTA is more positive among the research organizations than producers. This contrasts with the perception of the project’s impacts. For the producers, increases in production and productivity are already under way, but for the research organizations the impact will take 5 more years. However, this difference between producers and research organizations sheds light over the need to strengthen this relationship in the preparation and execution of subprojects. 3. With respect to the model of the Competitive Fund, it was found that it allocated resources according to the PTA original design and strategy, providing a higher level of resources to the fruit and vegetables supply chain. The development of this model in general terms was considered satisfactory by the organizations submitting proposals. However, it is important that for the future, there will be no constant variations in the evaluation criteria of the various call for proposals, in order to strengthen the technical capacity of the agriculture R&D system. Finally, even if the model of the Competitive Fund was characterized as a useful and functional tool for the allocation of resources, in this evaluation we were unable to compare directly other mechanisms for allocating funds for agriculture R&D given that it would have required a rigorous analysis of other agriculture R&D systems to be comparable. 4. With respect to the subprojects, the economic impacts have been found to be positive and significant in the variables directly related to the production function: production per hectare, income per hectare, and agriculture value added. However, there were no significant impacts found in other variables, such as access to markets or exports. 5. These results suggest that the adoption of new technologies and productive processes take time and follow a specific sequence. In other words, the PTA has an initial impact in productivity (production per hectare), then increases total production, and then, the market participation and exports. However, if this is the sequence, the impact of PTA on market access and exports could happen in the future. In particular, it is expected that in 5 years the impact on these variables will be reflected. 6. Another reason why the impacts have not been realized is that the PTA did not involve in a more proactive manner the other actors in the supply chain. This is shown through other evaluation tools used, such as focal groups, in which it was confirmed that the participation in the subprojects of other actors in the supply chain was low. Therefore, the recommendation is clear

43

for the PTA in the need for strengthening the relationships with other actors in the supply chain in order to increase market share and access to international markets. 7. This recommendation is reinforce by the information collected in the focal groups about the agriculture R&D agendas. The methodology is important. Equally as important are the consultation process between producers and research organizations, and the ownership of the methodology for the preparation of the agendas. However, if there is not sufficient dissemination and the actors of the supply chains are linked together in the implementation, the usefulness of these agendas would be lost. It is important to clarify that even though the agendas are a result of the PTA, these become an input that should serve in the shaping of the public sector policy. 8. About knowledge generation, the PTA surpassed the planned training. It also produced PhD thesis across the various themes being researched under PTA financing, and publication in journals. These are positive impacts. However, in terms of recommendations, even if 68% of the research organizations said that they generated new research themes, for 93% of them, the areas of research already existed, and in terms of contracting, most of them are old researchers. New researchers were generated through internships and pre-graduate students. This suggests that in the future, the PTA should strengthen the incentives to research new areas and themes, and to incorporate new human capital. Finally, it is important to note that the PTA generated a strengthening of the quality and equipement of the research institutions, which is also an important part of the strengthening of the knowledge generation. 9. PTA, even though it started as a Project financed by the World Bank, it has become an important instrument of the sectoral policy. Unfortunately, the S&T programs, regardless of their importance for agriculture competitiveness and growth, have been left behind in the priorities of the sector’s budget. The impacts and results of this evaluation should be useful for the authorities of the sector to reconsider the order of budget priorities and provide unequivocal support to S&T activities, in which projects like PTA could exist. 10. The PTA should look to create effective mechanisms to ensure the financing of producer alliances with the agriculture research organizations to have medium term projects to the strengthen of capacity of actors within the supply chain as well as the adoption and use of new technologies generated, and guarantee feedback mechanisms. 11. Although one of the requirements of the Competitive Fund was the participation of the different actors of the supply chain, it was found that in most cases, the participation in the subprojects did not go beyond the producers and the research organizations. One way to guarantee a more effective link between the different stakeholders of the PTA in the subprojects and thus increase the level of ownership beyond the research organizations, is to ensure that co-financing from all actors exist in kind and in cash. 12. As was mentioned previously, the participation of other actors in the supply chain was limited; however, the supply chain approach was always maintained at the moment of selecting subprojects. Furthermore, it is important to note that the development of the R&D agendas involved all actors in the supply chain directly, so in the future it is expected that a correct implementation of the agendas and a more rigorous follow up will occur in order to ensure a better linkage among the supply chain actors. With respect to the relevance of the supply chain approach, the different actors said that the approach was adequate given that it allowed to identify in a more comprehensive manner the needs and strengths present throughout the productive process, deriving the strategies and projects that would strengthen the overall supply chain.

44

13. With the purpose of avoiding the concentration of resources in few research organizations, the PTA must support the creation and the progressive strengthening of the regional agriculture R&D capacities. The criteria of the Competitive Fund and the levels of cofinancing could be useful towards that objective. 14. The support to the strengthening of the SPS system and to ICA and INVIMA must continue. However, it is important to establish clear prioritization criteria for the resources within the system and institutions. Furthermore, the activities supporting the SPS system must also have counterpart funding with a clear public sector support in the promotion and diversification of the agriculture and agroindustrial exports. 15. The PTA did not have much impact on the overall National S&T system with respect to new participants, nor the generation of new relations among actors or major knowledge generation and uses of available public policy instruments. In the best case scenario, the relationship and knowledge between actors of the PTA subprojects, research organizations and farmers, increased or improved. Even though the capacity of the PTA to have an impact on the national S&T system is limited by design, the strengthening of capacities, the relationships and the context of the subprojects in the detailing of criteria and procedures for allocating funds from the Competitive Fund can be a way to ensure an impact on the overall S&T system of the country. 16. There’s sufficient evidence to say that through the PTA new knowledge was generated by the research organizations, activity in which farmers participated, and that these were trained and knowledge was transferred, and that they did apply, in a successful manner in order to achieve impacts in productivity, income and value added. In other words, the PTA, through its subprojects, it impacted in a positive way each of the actors in the knowledge supply chain (generation, dissemination and use), until it was finally used by farmers and until it had produced an impact in the production units. 17. The participation of the productive sector was not only limited to the execution of subprojects, but it also provided important resources to the PTA through cofinancing. About 50% of the total costs of subprojects was provided by the productive sector, which is important evidence of the effectiveness of the program to attract private sector funding to R&D activities. 18. The research organizations were benefited with the resources from the PTA subprojects. These organizations were able to strengthen their research groups by linking personnel to activities and investments in equipment and laboratories. The resources also allowed to produce new publications which were disseminated through academic journals and books, beyond the academic training of professionals linked to the subprojects. In the majority of cases, high level staff linked to the subprojects had already worked with the research organizations. In these conditions, the incorporation of new human resources was through internships and pre-graduate students. This signals to the idea that the PTA subprojects acted as temporary “financial oxygen” to the research organizations that enabled them to expand activities, but that would go back to their original levels once the subproject ended. The fact that it was difficult for the evaluators to reach the researches that had worked on subprojects is a signal that the research organizations hired them mostly for the duration of the subproject. The recommendation to finance agriculture R&D programs would help solve this problem. This does not exempt the Government from its obligation to increase the level of resources dedicated to agriculture R&D. The support of the private sector through the para-fiscal funds is another source of funding that would help strengthen the sectoral R&D capacities. However, in order to get the private sector to contribute in a significant and sustainable way to the agriculture R&D system, there must be a clear and positive linkage between the investment and the expected results in the economic development of

45

the supply chain and its stakeholders. Such direct financial dependency of the research organization in the resources of the PTA makes those same entities perceive optimism and efficiency in several aspects of the PTA, aspects in which producers are less optimistic and more critics of. 19. To conclude with this evaluation, we make reference to the impact that this same evaluation may have, as it has allowed for the identification of the strengths and weaknesses that were present throughout the execution of the PTA. The conclusions of this evaluation will allow to focus future resources in the improvement of the instruments for subproject selection, on top of placing more attention to the establishment of sustainable relationships between producers, research organizations and other stakeholders in the supply chain. The evaluation also allowed the identification and quantification of the economic and institutional impacts, presenting solid evidence for the increase in the continuation of the Project. This evaluation will serve as input to (or baseline for) future studies in case a continuation of the PTA would occur. In the same manner, the policy recommendations presented in the evaluation will serve as a guide so that the future of the PTA be better integrated in the national S&T system, making the project an important development tool. With respect to the effectiveness of the evaluation on the project costs, we can assert that the performance was highly satisfactory, given that the project costs were only a small fraction of the overall total cost for the collection of historical information, the identification of the role of each actor in the supply chain, and the identification of the impact of each of those actors. The evaluation covered all of the aspects, and in an efficient and effective manner with the established costs.

46

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The Agricultural Transition Project (PTA) arises as an alternative to the growing necessity of the Colombian Agriculture sector to adapt to the modern schemes of production and distribution of agricultural products, which are placed within the current context of competitive and globalized economies. On the 29th of November 2004, document CONPES 3316 was issued, authorizing the Government to contract the World Bank (IBRD) for a loan of US$30 millions in order to finance the PTA. The PTA was designed to take advantage of the opportunities created as a result of the commercial openness and to galvanize efforts towards both the provision of scientific and technological assets to agriculture and the amelioration of the admissibility of products in foreign markets through the optimization of sanitary conditions and safety, provided the strategic position of the agriculture sector on international commercial negotiations. In order to achieve its objective, the PTA was structured according to three components: (i) Knowledge and Innovation Creation; (ii) Strengthening of the Sanitary and Phytosanitary System and; (iii) Management, Monitoring and Evaluation. The Knowledge and Innovation Creation component was designed with 2 subcomponents: (i) Prospective of Research, Technological Development and Innovation subcomponent; (ii) Grant Funds subcomponent. The Strengthening of the Sanitary and Phytosanitary System component was designed with six subcomponents: (i) MSF training subcomponent; (ii) Meat Chain Barriers Improvement Subcomponent; (iii) Fruits and Vegetables Barriers Improvement Subcomponent; (iv) MSF Information System; (v) MSF Authorization System Subcomponent; (vi) MSF dissemination Subcomponent. The purpose of the Management, Monitoring and Evaluation component was to secure an appropriate management and execution of the PTA through the Coordination Unit (UCP), developing administrative, operational, and managerial tasks. Additionally, the Bank performed an environmental evaluation (EA) of the PTA, establishing for the category of safeguard S2 and the category of environmental protection B (need to implement actions to avoid or reduce the negative environmental effects). Similarly, it considered the need of an environmental action plan (PAA), a framework for integrated management of plagues (MMIP) and a framework for the development of small production (MDPP), all of which included environmental mitigation and supervision practices, as well as several requirements for the execution of components 1 and 2. As of the major results achieved by the PTA, the following are mentioned: - Elaboration and Publication of 25 I+D+I prospective agendas, out of the 20 initially planned.

Among these agendas, 4 are regional and 1 in transversal topics, as a result of the interest and important additional financial and human resources contributions of COLCIENCIAS, SENA and Universidad Del Valle.

47

- Co-financing of 117 subprojects I+D+I in 17 productive chains, through public calls, which

execution period varied between 12 and 48 months, having the participation of resources coming from the private sector through parafiscal funds in 43 of them.

- Training of 386 officials of the MSF National System; strengthening of the operational of

ICA and INVIMA laboratories; capacity strengthening in ICA, INVIMA, and MCIT information systems and connectivity; communication campaign of the ICA Authorization System and training of 442 veterinarians; communication campaigns of agricultural best practices (BPA) and livestock best practices (BPG) to producers and technical assistants, who would be multipliers.

- Construction of a level laboratory of biological security 3 agriculture (LNSB3A), for the safe

handling of the foot-and-mouth disease virus, which will be operated by ICA, and it is planned to start operations at the end of 2012.

- Development of the SIGMAR (Integrated Management System) information systems, and the SIGP (Integrated Project Management System); execution of the midterm evaluation of the on-going subprojects and, of the impact and results evaluation of the PTA, carried out by UT Fedesarrollo-IQartil.

- Design and implementation of an environmental management methodology for the subprojects contracted by the PTA.

The PTA had a Monitoring and Evaluation System capable of creating reliable and timely information, which was useful for: the midterm evaluation of the PTA, the final evaluation of the impacts and results, the quarterly reports addressed to BIRF, be accountable to the CGR, provide information to MADR about the achievements of the CyT and MSF policy whenever it was needed, and, above all, to support DDTPS in decision-making processes. The contribution of the efforts carried out by the PTA regarding knowledge and innovation creation and strengthening of the Sanitary and Phytosanitary measures system, can be observed in the amelioration of the agriculture sector competitiveness, which will allow to: (i) have larger production volumes for both the domestic market and exports; (ii) discriminate products through the creation of value added that allows the penetration to receivers markets; (iii) reduce pressure coming from competitors because of the commercial openness (trade liberalization agreements); (iv) reduce public health, phytosanitary, and animal problems, as consumption standards are increased through more rigorous enforcement of sanitary and phytosanitary norms; (v) allow access and greater participation in national and international markets. This was evidenced, given that the benefited producers through the subprojects increased their productivity, income per hectare, and value added. In addition, the internal rate of return (IRR) of the co-financed subprojects was greater than the discount rate (12%), so that the products and technologies developed should be included without any resistance by the producers. As of the exports’ increase in the principal production chain selected (bovine meat and uchuva), it can be stated that the support of the National Government to those sectors, including those of the PTA, influenced positively. It is emphasized the impossibility of isolating the effect of the change produced by the PTA, considering that participation and access to markets correspond to a group of complex variables that depend on simultaneous factors beyond productivity and phytosanitary barriers.

48

The incorporation of an audit team in charge of the technical and financial monitoring of the 117 subprojects contracted through the public calls of 2006 and 2007 was the most important structural change of the PTA during its execution. However, the implementation of the PTA was also affected by the following circumstances: - The delivery of the first prospective agendas was made after almost 2 years that the PTA

begun, which impeded that these were available as an input for the co-financing of the I+D+I subprojects through the public calls.

- The public calls of 2007 and 2008 were highly demanding in human resources and time, which triggered the allocation of non-planed resources to hire personnel, computers, and communication networks.

- The SIGP implementation for the public calls operations was only feasible in 2008. During 2006 and 2007 all operations were done manually, which produced a lack of efficiency and control in the procedures.

- The weakness in the design, work amounts and budget estimation for the construction of the LNSB3A, and lack of specific experience of the team responsible of the monitoring of the construction, did not allowed its conclusion in the planned schedule.

- The training of the sanitary agencies’ officials was impossible to execute according to what was accorded in the PAD, given high Government guidelines that did not authorize the ICA and INVIMA officials to go out of the country.

- The design and implementation of the management information system facilitated the financial, acquisition, and managerial control of the PTA (SIGMAR), it took 2 years to fully operate and it meant and important investment of significant credit resources.

The continuity that governments have given to internationalization processes of the economy through trade liberalization agreements since 2005 and the consolidation of the agriculture sector as one of the economic growth engines of the country, have created conditions for the strengthening and continuity of the sanitary, food safety, and innovation policies as pillars to the improvement of competitiveness of the agricultural productive systems, and therefore, of the rural population welfare. In both, MSF and CyT, the country has created a legal and institutional framework, and policy and fiscal tools that guarantee the continuity of research, technological development, and innovation of the agriculture sector, as well as of the proper functioning of the monitoring, vigilance, and control systems for the MSF. As of the performance of MADR, UCP, BIRF, and resources managers, it is considered successful, yet having as a topic to improve in future projects, the personnel stability that works in the UCP. As of lessons learned in the execution of the PTA, there are the following: - A competitive grant fund should be established in a permanent basis, so that it can be able to

capture resources from the public and private every year, and not only whenever the Government ha resources available for such endeavor.

49

- As a previous requisite to every public call, the strategic lines to be financed should be based on the needs exposed in the prospective agendas for each chain.

- For the development of a project that includes an innovation component, it should be taken into consideration the maturity process of technologies and the implementation process at the farm level. In that regard, it should be designed an ex ante strategy to create innovation that considers technology transfer mechanisms, technical assistance and incentives for the implementation of products, technologies and protocols in the production systems.

- For the construction of infrastructure of high technological complexity, it is suggested to separate from the design of the project, a planning period that includes the elaboration of a study in which likely risks are identified, and the proper contingent measures to be adopted toward the execution of the project. In that sense, if the designs, layouts, work amounts, and technical characteristics were not contracted within the loan project framework, these should be validated with international experts.

- Design projects with more integrality and complementarity among its objectives, articulating its different components during the execution.

- The information systems supporting management operations should be available since the first moment of the project implementation. In that same line, it is important during the design of the project, to foresee the time required for the development of the information systems needed, so that they can be fed in real time.

- Strengthen key information gathering processes to carry on robust ex ante and ex post economic evaluations of the projects. From the design of the project, it should be defined the methodology to be used for these evaluations.

- Develop an environmental best practices manual for I+D+I subprojects, being a guide for both the ex ante self-assessment of environmental impacts and the elaboration of an environmental action plan applicable for the laboratory work and experimental parcels to be developed within the research frame. The aforementioned manual should be available before the opening of the public calls and its content should be applicable for both the elaboration of the proposals and the execution of the contracted subprojects.

50

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders No comments received.

51

Annex 9. List of Supporting Documents

A. Bank Project Appraisal Document (PAD) – Report 31994-CO Supervision ISRs Supervision Aide Memoires Audit Reports, Financial Management and Procurement Reviews Loan Agreements: original project and Additional Financing

B. Borrower PTA Impact Evaluation Study, Fedesarrollo 2011 Mid Term Evaluation Report, MADR, 2010 Analysis of the Results of the Project Development Indicators, MADR, 2011 Bi-annual project progress reports Sub-project completion reports and environmental and economic impact reports

Cor

di l ler

a Orie

nta l

Cor

dil le

ra C

entr

al

Cord

i l lera O

ccident a

l

PicoPicoCristóbal ColónCristóbal Colón(5,775 m)(5,775 m)

C A Q U E T ÁC A Q U E T Á VA U P É SVA U P É S

G U AV I A R EG U AV I A R E

M E T AM E T A

H U L AH U L AC A U C AC A U C A

C H O C ÓC H O C Ó

A N T I O Q U I AA N T I O Q U I A

VALLE DELVALLE DELCAUCACAUCA

RISARALDÁRISARALDÁ

QUINDIOQUINDIO

C U N D I N A -C U N D I N A -M A R C AM A R C A

DISTRITODISTRITOCAPITALCAPITAL

G U A I N Í AG U A I N Í A

V I C H A D AV I C H A D A

C A S A N A R EC A S A N A R E

C E S A RC E S A R

NORTE DENORTE DESANTANDERSANTANDER

A R A U C AA R A U C A

A M A Z O N A SA M A Z O N A S

N A R I Ñ ON A R I Ñ O

P U T U MAYO

CALD A S

CÓRDOBA

BOL Í

VA

R

BO

YAC

Á

SUC

RE

S A N TA N D E R

TOL IMA

La PedreraLa Pedrera

PuertoPuertoSantanderSantander

PuertoPuertoPizarroPizarro

PuertoPuertoLeguízamoLeguízamo

PuertoPuertoHuitotoHuitoto

MacujerMacujer

CalamarCalamar

MapiripanaMapiripana

Santa RitaSanta Rita

PuertoPuertoNuevaNueva

ChavivaChaviva

San JuanSan Juande Aramade Arama

GarzonGarzonGuapíGuapí

IpialesIpiales

GaviotasGaviotasSan PedroSan Pedro

MirafloresMiraflores

San VicenteSan Vicentedel Caguándel Caguán

GiradotGiradot

PalmiraPalmira

BugaBuga

CartagoCartago

ChiquinquiráChiquinquirá

SocorroSocorro

TurboTurbo

YarumalYarumal

QuibdoQuibdo

PuertoPuertoAsisAsis

El EncantoEl Encanto

Locas deLocas deCahuinariCahuinari

LéridaLérida

YavarateYavarate

TabaquénTabaquén

BrujasBrujas

San RafaelSan Rafael

El BaneoEl Baneo

OcañaOcaña

MaganquéMaganqué

LeticiaLeticia

MituMitu

PuertoPuertoInirídaInirída

PuertoPuertoCarreñoCarreño

FlorenciaFlorencia

NeivaNeiva

PopayanPopayan

CaliCali

PastoPasto

San JoséSan Josédel Guaviaredel Guaviare

VillavincencioVillavincencio

MocoaMocoa

ArmeniaArmeniaPereiraPereira

TunjaTunjaYopalYopalManizalesManizales

BucaramangaBucaramanga

MonteriaMonteria

SincelejoSincelejo

ValleduparValledupar

CúcutaCúcuta

MedellinMedellin

IbaqueIbaque

BOGOTÁBOGOTÁ

MA

GD

ALE

NA

LA G

UAJ I RA

AraucaArauca

La Pedrera

PuertoSantander

PuertoPizarro

PuertoLeguízamo

PuertoHuitoto

Macujer

Calamar

Mapiripana

Santa Rita

PuertoNueva

Chaviva

San Juande Arama

GarzonGuapí

Tumaco

Ipiales

GaviotasSan Pedro

Miraflores

San Vicentedel Caguán

Giradot

Palmira

BugaBuenaventura

Cartago

Chiquinquirá

Socorro

Turbo

Acandí

Yarumal

Quibdo

PuertoAsis

El Encanto

Locas deCahuinari

Lérida

Yavarate

Tabaquén

Brujas

San Rafael

El Baneo

Ocaña

Maganqué

Puerto Bolívar

Leticia

Mitu

PuertoInirída

PuertoCarreño

Arauca

Florencia

Neiva

Popayan

Cali

Pasto

San Josédel Guaviare

Villavincencio

Mocoa

ArmeniaPereira

TunjaYopalManizales

Bucaramanga

Monteria

Sincelejo

Cartegena

BarranquillaSanta Marta

Valledupar

Cúcuta

Ríohacha

Medellin

Ibaque

BOGOTÁ

C A Q U E T Á VA U P É S

G U AV I A R E

M E T A

H U L AC A U C A

C H O C Ó

A N T I O Q U I A

VALLE DELCAUCA

RISARALDÁ

QUINDIO

C U N D I N A -M A R C A

DISTRITOCAPITAL

G U A I N Í A

V I C H A D A

C A S A N A R E

C E S A R

NORTE DESANTANDER

AT L Á N T I C O

A R A U C A

A M A Z O N A S

N A R I Ñ O

P U T U MAYO

CALD A S

CÓRDOBA

BOL Í

VA

R

BO

YAC

Á

SUC

RE

MA

GD

ALE

NA

LA G

UAJ I RA

S A N TA N D E R

TOL IMA

R.B. DEVENEZUELA

BRAZIL

PERU

ECUADOR

PANAM

A

Guaviare

Negro

Cauc

a

Atrato

Vichada

Patía

Ptumayo

Casanare

Atacavi

Meta

Vaupés

Caquetá

Caguán

Magdalena

Caribbean Sea

Lago deMaracaibo

PACIFICOCEAN

To Maracaibo

To Mérida

To Guasdualito

To Ibarra

Cor

di l ler

a Orie

nta l

Cor

dil le

ra C

entr

al

Cord

i l lera O

ccident a

l

PicoCristóbal Colón(5,775 m)

75°W 70°W

75°W 70°W

5°N

10°N

0°N

5°N

0°N

COLOMBIA

0 80 160 240

0 1208040 160 200 Miles

320 Kilometers

IBRD 33388R

FEBRUARY 2009

COLOMBIASELECTED CITIES AND TOWNS

DEPARTMENT CAPITALS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

DEPARTMENT BOUNDARIES

INTERNATIONAL BOUNDARIES

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank Group, any judgment on the legal status of any territory, o r any endo r s emen t o r a c c e p t a n c e o f s u c h boundaries.