annual review - summary sheet title: proyecto ganadería ...1 annual review - summary sheet title:...

1

Annual Review - Summary Sheet

Title: Proyecto Ganadería Colombiana Sostenible – Sustainable Cattle Ranching Project, Colombia

Programme Value: £15m Review Date: September 2017

Start Date: June 2016 End Date: June 2017

Summary of Programme Performance

Year 2013-14 2014-15 2015-16 2016-17

Programme Score B A B A

Risk Rating High Medium High Major1

Summary of progress and lessons learnt since last review Overall the project has been scored an A: outputs met expectations. The project is designed to gather evidence on whether introducing silvopastoral agroforestry systems (SPS) can help reduce the deforestation caused by cattle ranching and alleviate poverty amongst small scale ranchers through the increase of milk and beef production, and the provision of alternative income through payments for providing ecosystem services such as preserving biodiversity and managing land in a way which sequesters carbon. SPS systems allow ranchers to sustainably intensify production on existing land, reducing the need for farmers to expand onto new land and thereby reducing pressure on adjacent forests. Deforestation is a major cause of greenhouse gas emissions in Colombia and across much of Latin America, and the intention is that the learning from this project could pave the way for a much wider transition to sustainable land management across Colombia and comparable regions. The BEIS project manager joined the World Bank’s mission in April 2017 to verify and gather evidence on progress so far; take the views of other stakeholders, including the Colombian Government which manages the other major donor funding from the Global Environment Facility; to assess lessons learned; and to assess the potential for an extension to the project. This annual review is based on this and other missions over the course of the last year and on the results reported in the June 2017 technical report from the lead implementing agency, FEDEGAN. The project’s original closing date is January 2018: noting this, this review has assessed and scored outputs by looking at the likelihood of meeting the end of project targets, based on the actual implementation progress at June 2017 and the trajectory of progress over the last year. The project team have made good progress over the past year and this assessment has shown that the project is currently delivering good value for money. The last annual review noted that the project was ‘at a critical moment in its life’ and a number of detailed recommendations were made, both through the annual review and through the project’s mid-term review. These recommendations have been taken forward and as a consequence the rate of implementation has improved. This has been accompanied by an increase in the rate at which project funds have been disbursed, from 35% (October 2016) to 62% (June 2017). The project team should be commended for the improvements seen over the past year, but it should also be noted that the project will still not be in a position to deliver against all of the revised output targets by the current project closing date. This largely reflects delays to implementation incurred in previous years which are detailed in previous reviews. The last annual review recommended that if the project team made sufficient progress over the next year and implemented the recommendations put forward in the mid-term review then BEIS should consider extending the programme to allow the project to meet its targets in full. The World Bank submitted a formal extension request to BEIS in July 2017. This review has found that sufficient progress has been made for this extension to be considered: we are therefore conducting a parallel but separate assessment of the case for extending the programme, which will be published separately to this review. A key driver for originally investing in the SPS programme was the value the programme has as a pilot and demonstration project. It was designed to test different approaches for driving land conversion to

1 BEIS has now moved to a four point risk-rating system of minor, moderate, major and severe, as agreed with

DFID and DEFRA in the autumn of 2016 to ensure consistency across departments.

2

SPS and to gather further evidence regarding the contribution SPS can have on reducing greenhouse gas emissions and alleviating poverty amongst small scale cattle ranchers. It was also designed to build institutional capacity for implementing SPS in Colombia. This review has found that implementation has improved significantly over the last year and that as a result the programme would be in a good position to deliver this important work if extended. This would allow the project to realise its full value as a pilot, ultimately significantly increasing the likelihood that the lessons learnt from this programme are used to pave the way for a much greater transition to sustainable livestock ranching across Colombia and Latin America. Summary of recommendations for the next year 1. BEIS and the World Bank team should work with the project’s implementing agencies to ensure that

they recognise the value of the project’s work to disseminate learning, and to ensure that they are effectively recording activity undertaken in relation to this work. The project’s implementing team should also consider maximising the learning which can be drawn from this project by producing a specific report on land conversion to iSPS, describing the lessons that have been learnt and explaining how these could be applied to similar future interventions.

2. As part of the project’s end of programme assessment the project team and the World Bank should

produce a joint report setting out their experiences relating to the design and implementation of PES schemes. This should set out specific recommendations in PES design for the Colombian Government. We recommend that the team engage the Colombian Agricultural ministry with this work, and that consideration be given to producing these reports earlier if it becomes apparent that this would be beneficial to other initiatives seeking to replicate this approach (enabling the learning from this programme to more effectively drive wider transformational change).

3. BEIS should assess the World Bank’s extension request by October 2017, ensuring thorough due

diligence to ensure an extension represents good value for money.

4. The project implementation team should recruit a new project co-ordinator as soon as possible.

5. If BEIS approve an extension to the project, the World Bank and the project team should conduct an updated assessment of the resources required to fully deliver the outputs expected in the extension phase. This could form part of the risk assessment referenced in section E.

6. The World Bank, the project team and the Colombian Government should consider how the project’s

interventions can be used to deliver results in regional and national results-based payments programmes such as the BioCarbon Fund Initiative for Sustainable Forest Landscapes.

7. BEIS and the British Embassy in Bogota should consider how they can encourage greater

cooperation between the various initiatives which BEIS and the ICF fund in Colombia. 8. BEIS / British Embassy Bogota should continue to engage with the Colombian Ministry of Agriculture

as regards its intentions to pursue further sustainable cattle intensification programmes. 9. If the project is extended, an updated risk assessment should be conducted by the World Bank

identifying any further risks to implementation and to the potential of the project to deliver the intended long-term outcomes and impacts. Appropriate mitigation plans should be put in place.

3

A. Introduction and Context

Link to Business Case:

https://edrms.decc.gsi.gov.uk/isr/ieu/PNK/Documents/SPS%20Colombia/Project%20Development/Business%20Case/Colombia%20SPS%20Business%20case.docx

Link to Log frame:

https://edrms.decc.gsi.gov.uk/isr/ieu/PNK/_layouts/15/DocIdRedir.aspx?ID=DECCISRC-127-239585

Outline of the programme This project is designed to promote the adoption of silvopastoral agroforestry systems (SPS) in Colombia. The aim is to increase the environmental and economic sustainability of cattle ranching in Colombia. Extensive ranching on degraded land is a major driver of tropical deforestation and this in turn generates significant greenhouse gas emissions. SPS systems aim to shift land-use practises by promoting farming techniques that intensify ranching on existing land. This can increase the efficiency of cattle production, providing better incomes for the rural poor, and deliver environmental benefits including reduced greenhouse gas (GHG) emissions; decreased soil erosion and water pollution; and enhanced biodiversity. The project seeks to convert 35,000 hectares of often degraded extensive (i.e. open, treeless) pastures into a richer and more productive environment, where trees and shrubs are planted interspersed among fodder crops such as grasses and leguminous herbs. The project is gathering evidence on whether introducing SPS can help reduce the deforestation caused by cattle ranching and alleviate poverty through the increase of milk and beef production, and the provision of alternative income through forestry products and payments for environmental services. The programme contains innovative quasi-experimental components which allow the effectiveness of different interventions to be compared and evaluated; a key part of the programme’s value therefore lies in testing whether (and how) SPS can be effectively rolled out at scale in order to inform larger scale programmes in the future. The Colombia Mainstreaming Sustainable Cattle Ranching Project (CMSCR) was initially funded by the Global Environment Facility (GEF) with US$7 million, using FEDEGAN as lead executing agency and involving a range of NGO partner agencies (Fundación Centro para la Investigación en Sistemas Sostenibles de Producción Agropecuaria, the Nature Conservancy and Fondo Acción). UK funding has being used to expand the reach and scope of the project including by piloting a new payment for ecosystem services scheme based on carbon sequestration; by extending the project to two new ‘deforestation hotpots’; by supporting the greater provision of technical assistance to farmers; and by assisting with the production of planting material.



4

B: PERFORMANCE AND CONCLUSIONS

Annual outcome assessment The Theory of Change for the project lists the following expected outcomes and impacts: Impacts:

1. Reduced impact of cattle farming on climate change and the environment. 2. Reduced poverty among small-scale livestock farmers in seven regions of Colombia.

Outcomes:

1. Reduced GHG emissions from cattle farming as a result of SPS adoption (SPS act as net carbon sink).

2. Increased productivity of cattle farming and increased incomes for farmers. 3. Wide range of environmental benefits including to biodiversity, water and soil, and increased

climate resilience. 4. Wider adoption of SPS in Colombia and further afield as a result of the project demonstrating

benefits. 5. Better informed Government and REDD+ policy and legal support mechanisms for SPS.

The project has made good progress over the course of the last year and overall, the outputs which have been delivered are in-line with the expectations we had for progress over the year. Good progress has also been made in finalising the monitoring and reporting elements of the project: both of these factors mean that data is now being collected which will allow the link between SPS and the expected outcomes to be assessed. The last annual review reported that a contract for monitoring the International Climate Finance KPI 6 (relating to reduced greenhouse gas emissions from cattle farming as a result of SPS adoption) had yet to be put in place. That contract, between the World Bank and The Nature Conservancy (TNC) was signed in August 2016 and work is now progressing well: see section G of this review for further details. The project is also gathering good evidence to assess the impact of SPS on increasing the productivity of cattle farming; this year’s results indicate that the increase in the production of beef and/or milk per intervened hectare in participating farms is 17%. Good data is being collected regarding the impact on biodiversity and other environmental benefits and evidence is beginning to emerge which indicates that the project is positively influencing the wider adoption of SPS in Colombia2. This said, without an extension it is likely that the confidence which could be placed in the end of project assessment (linking outputs delivered to outcomes) would be diminished relative to original expectations. This is because delivering outputs is a pre-requisite to delivering (and assessing a causal link) to outcomes and delays to implementation in previous years have not been fully overcome, reducing the level of outputs which we now expect the project to deliver by the original closing date of January 2018.

2 Strong interest in scaling up SPS and other approaches to sustainable cattle ranching is developing in Colombia.

It is difficult to establish a direct causal link between this programme and these initiatives, though a link is very likely given this project’s work to build institutional capacity in Colombia and to disseminate knowledge and learning as widely as possible. See section H for further details.

5

Overall output score and description Overall the project has been scored an A: outputs met expectations. The programme’s targets were significantly revised following the last annual review (see page 7). The progress of the project-wide indicators against the revised targets is given in the table below, which also sets out the project’s projected achievements at the current project closing date (January 2018). Based purely on the total implementation progress to date our assessment is that the project’s outputs should be scored A/B. However, this annual review is intended to assess progress over the course of the last year, rather than assessing progress over the full life of the project. The trajectory over the course of the last year shows that the rate of implementation for key indicators is now accelerating rapidly3. A number of underlying administrative issues have been resolved and this, combined with continued favourable weather conditions4 has allowed good progress to be made. We believe that the projections given for January 2018 are therefore realistic and on this basis our assessment is that the outputs which have been delivered over the course of the past year should be scored A. Progress against each sub-component of the project (as detailed on following pages) has been strong over the past year and under-delivery against the project-wide indicators is largely attributable to delays incurred in previous years, as set out in previous reviews. The table below sets out progress against the project-wide indicators. Progresses against the more detailed component specific indicators are included on the following pages. These are all taken from the project’s results framework.

Indicator(s) Progress at June 2017 and % increase from June 2016

Jan 18 end of project target

Jan 18 projected results

Area under environment-friendly cattle ranching production systems implemented in project areas (ha)

62,402 +47% 84,891 73,891

Land area where sustainable land management practices have been adopted as a result of the project (ha)

12,511 +37% 35,000 24,000

Increase in the production of beef and/or milk per intervened hectare in participating farms

17% +143% 10% 17%

Improved presence of globally important biodiversity in project areas, as measured by an increase in the Environmental Services Index (ESI) resulting from the adoption of environment-friendly SPS in participating farms in project areas, over baseline

237,578 + 0.4% 575,000 394,286

Reduction in GHG emissions from avoided deforestation and forest degradation and increase in carbon sequestration at the farm-level through adoption of environment-friendly SPS in participating farms (tonnes CO2eq)

602,503 N/A: no data for June 2016

1,600,000 1,097,143

Number of cattle ranching farms benefitting from project instruments (technical assistance, PES, or support for establishment of on-farm nurseries)

2,555 -5% 2,700 2,555

3 Land conversion to iSPS has increased from 243 to 1859ha and land conversion to SPS has increased from

9,129 to 12,511ha. The latter is projected to reach 24,000ha by January 2018, this is based on signed land-use agreements with farmers and on the number of trees delivered to date (trees which have been delivered but not planted are not included within the statistics). 4 The effects of severe El Nino droughts in 2014 and 2015 significantly delayed implementation across many of the

project sites

6

Key actions: progress made towards 2015-16 Annual Review recommendations Recommendations which have been completed “BEIS should continue to engage closely with the World Bank team over the coming months to monitor the project’s progress against the agreed milestones and action plans and in particular, to check progress on streamlining the contracting and payment processes and reducing the current backlog. This will be crucial for putting the project back on track.

The action plans which were agreed during the programme’s mid-term review mission have been carried out and a number of key issues which were delaying implementation have been resolved. The contracting and payment processes have been improved and the backlog in payments has been cleared. While the previous annual review noted that a number of these issues could and should have been resolved more quickly we are satisfied that good progress has now been made. This is reflected in the marked increase in the implementation and disbursement rates over the last year.

“The World Bank must ensure that the remaining elements of the monitoring and evaluation framework are in place as soon as possible and that there are no gaps in the 2017 results collection.” The monitoring and evaluation system is now fully established and integrated. The project is reporting actual results against all of the project specific indicators allowing a quantitative assessment as well as a qualitative one to be made. We are satisfied that this recommendation has been met, for further information see section G of this review.

“An early assessment should be made by the project team of the effect of the reduced expectations around land conversion to iSPS as part of the assessment of the project’s overall contribution to reduced GHG emissions and avoided hectares of deforestation.”

The project team made this assessment during the World Bank’s December 2016 mission to the project site. The target for reduced GHG emissions was revised from 1,990,000 to 1,600,000 tonnes (CO2 equivalent). The reduced emission reductions attributable to the lowered expectations of land converted to iSPS specifically was calculated to be 265,000 tCO2e. These results indicate a 13% reduction in GHG emissions avoided as a result of the reduced expectations around land conversion to iSPS, though if extended the project expects to convert a greater amount of land to iSPS in partial mitigation. This assessment has resulted in a better understanding of the link between iSPS and emission reductions which will prove valuable for informing future initiatives and for informing the project’s full evaluation.

“Should the project meet the requirements of the agreed milestones and action plans over the next six months, BEIS should review the case for a one year extension to the project and possible additional supervisory funding (this decision will be taken within the context or other portfolio priorities and will be based on ongoing performance).”

The World Bank has agreed a restructuring package presented by FEDEGAN and has submitted a proposal to BEIS for a two year extension to the programme. This would see the programme end date moved to 31/01/2020. We are satisfied that sufficient progress has been made over the course of the last year for an extension request to be considered and we are therefore assessing the case for granting an extension separately. The outcome of that assessment will be published in due course.

“BEIS should continue to engage closely with the World Bank team over the coming months to monitor the project’s progress against the milestones and action plans which are aimed at putting the project back on target.” The BEIS project manager joined the World Bank team in their supervisory mission to the project in April 2017 and participated in regular calls to remain appraised of progress. The project has received oversight from the same project manager at BEIS over the last year although further staff changes mean that a new project manager will now be responsible for supervision at BEIS.

7

Recommendations where work is ongoing The project implementation team should ensure that lessons learned on the effectiveness of the various interventions made are disseminated as part of the project’s communications strategy.

A key part of this programme’s value lies in its ability to act as a pilot, testing different approaches for driving land conversion to SPS and evaluating the impact of SPS on improving livelihoods and delivering forest protection and climate change mitigation outcomes. For this reason we consider the effective dissemination of learning to be central to the overall success of the project. This is a prominent reason for considering the case for extending the programme. Evidence from the project has been used in several academic publications exploring this issue and the project team has developed several strategic alliances within Colombia to help disseminate learning and share best practice. This work is key to ensuring that the project deliver long-term transformational change and further emphasis should continue to be placed on this work for the remainder of the project’s life. In particular, BEIS and the World Bank team should work with the implementing agencies to ensure that they recognise the value of this component of the work and that they are effectively prioritising and tracking dissemination efforts. “BEIS and British Embassy Bogota should continue to monitor and encourage Colombian Government engagement with the project to ensure lessons from the project are informing wider policy development, particularly in any future bids for financial support through forest funds, the NAMA facility or Green Climate Fund.” BE Bogota has advised BEIS that the project is contributing to wider work to design sustainable supply chains for beef and dairy products in Colombia, as well as the formulation of majorer level climate change and sustainability policies such as the Colombian Nationally Appropriate Mitigation Actions (NAMAs) for the agricultural sector. There is good commitment to the programme at a technical level within the Colombian Government. At a political level, the Colombian Government remains committed to addressing the environmental impacts of extensive livestock rearing and their recently announced strategy to reduce deforestation5 identifies the implications of this practice as one of its core priorities. Although we are not currently aware of any bids to forest funds, the NAMA facility or the Green Climate Fund we are aware of a specific instance where learning from the SPS programme is directly informing a project which will explore how to scale up SPS and other sustainable approaches to livestock management country-wide. This project is led by the Food and Land Use Coalition and is in the initiation stages. We also have strong anecdotal evidence to suggest that the new ‘Colombia Sostenible’ flagship programme will seek to fund a sustainable cattle intensification programme as one of its first priorities, further demonstrating a strong interest in this approach within the Colombian Government. Has the logframe been updated since the last review? The results framework (logframe equivalent) was significantly revised during the Mid Term Review in June 2016. For details of these revisions see the 2015-16 Annual Review. All of the changes proposed within the 2015-16 annual review have been made, and a number of additional changes (not discussed within the last annual review) have also been made. Details regarding these additional changes are set out below. Area under environmentally-friendly cattle ranching production systems implemented in project areas The target for the first project development objective (PDO), ‘Area under environmentally-friendly cattle ranching production systems implemented in project areas’ was revised upwards during the World Bank’s December 2016 mission. The end of project target was revised upwards from 63,000 to 84,000 ha. The change was made due to better data becoming available about farm land uses at the beginning of the project – i.e. better baseline data. The new indicator reflects the increase in land area to account for those baselines6.

5 Estrategia Integral de Control a la Deforestación y Gestión de los Bosques en Colombia – EICDGB

6 This indicator captures information about the total area of land being sustainably managed within project areas,

including existing ecosystems which are being conserved within project sites. This is as opposed to the second

8

Reduction in GHG emissions from avoided deforestation and forest degradation and increase in carbon sequestration at the farm-level through adoption of environment-friendly SPS in participating farms This PDO has been revised downwards. This directly reflects the reduced expectations regarding conversion of land to intensive SPS (iSPS) as set out in the last annual review. The revised figure also accounts for the improved land use baseline data as set out above. The project team made these modifications during the World Bank’s December 2016 mission to the project site. The target for reduced GHG emissions was revised from 1,990,000 to 1,600,000 tonnes (CO2 equivalent). The reduced emission reductions attributable to the lowered expectations of land converted to iSPS specifically was calculated to be 265,000 tCO2e. This revised assessment was made in response to a recommendation put forward in the last annual review. We are content with the revised assessment given that this is directly attributable to a change in expectations regarding land conversion to iSPS which was agreed as part of the last annual review. Improved presence of globally important biodiversity in project areas, as measured by an increase in the Environmental Services Index (ESI) resulting from the adoption of environment-friendly SPS and measured in at least two pilot areas, over baseline This PDO has also been revised downwards to account for the changed expectations around land use conversion to iSPS. The change takes the Environmental Services Index target from 750,000 to 575,000. We are again content with this revised assessment for the same reasons. Reduced soil erosion (tonnes/ha) induced by the adoption of SPS and measured in at least two pilot areas, over baseline. Finally, during the December 2016 mission it was agreed that the indicator relating to soil erosion reduction would be removed from the results framework and reported on through specific research reports instead. This reflects an agreement to carry out further research on this indicator before making a decision on the final formulation of the indicator, made during the last annual review. We are satisfied that reporting progress against this indicator through separate research reports is appropriate, and the World Bank has received and are currently reviewing a draft of the first report on this subject. Intermediate Indicators The following intermediate indicators were also adjusted as result of the project restructuring that was approved by the Bank in March 2017, reflecting recommendations of the mid-term review that took place in June 2016, and the supervision mission that took place in December 2016:

Number of cattle ranching farmers sensitized on SPS and informed about availability of credit sources. This indicator was slightly reworded as follows: “Number of cattle ranching farmers sensitized and trained on SPS and cattle ranching production” to reflect the main focus of the training on awareness and capacity building. The end of project target was increased from 4,000 to 5,000 farmers (this figure is expected to increase to nearly 18,000 in the extension phase, driven by the increasing availability of demonstrative farms and farms that have already undertaken conversion). The project has continued to deliver its strategy of training ranchers on credit sources, but given the significant constraints to credit by small-scale ranchers, training alone is expected to have little impacts. The project will continue to work with FINAGRO on the design of effective climate financing instruments to support the transformation of the project at scale.

Number of professionals and technicians trained on SPS establishment and management. The end of project target for this indicator has increased from 100 to 500. The reason for this increase was the realization that to carry out activities relating to information sharing and training more intensively, and support the transformation at scale, the project needed to provide the country with a larger number of trained professionals capable of support the increasing number of programs/projects focusing on improving the sustainable intensification of livestock production in Colombia.

indicator, which only accounts for additional areas being managed sustainably (i.e. through conversion to SPS) as a direct result of the project’s interventions.

9

C: DETAILED OUTPUT SCORING

Output Title Component 1. Improving productivity in participating cattle ranching farms in project areas through SPS

Output number per LF n/a Output Score A

Risk rating: Major Impact weighting (%): 35%

Risk revised since last AR? No Impact weighting % revised since last AR? No

Key Points This component has been awarded an A – outputs have met expectations. While conversion of land to iSPS (the most important sub-component) remains significantly below the final target this figure masks the rapid acceleration in progress over the last ten months. There has been a more than five-fold increase in the number of converted hectares and the project is consequently on track to meet its revised final target for this indicator. 2/3 of the other components have been exceeded and one is not on track to be met. Progress for the iSPS component remains significantly below the final target level and below the June 2017 milestone of 2,731ha, as agreed during the December 2016 World Bank mission to the project. However, based on the rate at which implementation has been accelerated (the area under iSPS at the last annual review was 243ha) our confidence in the project’s ability to deliver under this component has increased. The World Bank has advised that land management plans have been agreed which if delivered would ultimately result in the revised end of project target being slightly exceeded. Two of the remaining three indicators have already exceeded their end of project targets. Good progress has been made towards training professionals and technicians on SPS establishment and management, with the total rising from 190 to 370 over the course of the last year, but the programme is not on expected to meet its revised target7 of 500 technicians without an extension. Overall, the project has made good progress in relation to this component since the last annual review and the results at this stage are broadly in line with our expectations. However, the risk of under-delivery against the final iSPS target remains major. Summary of responses to issues raised in previous annual reviews “An early assessment should be made by the project team of the effect of the reduced expectations around land conversion to iSPS as part of the assessment of the project’s overall contribution to reduced GHG emissions and avoided hectares of deforestation.”

The project team made this assessment during the World Bank’s December 2016 mission to the project site. The target for reduced GHG emissions was revised from 1,990,000 to 1,600,000 tonnes (CO2 equivalent). The reduced emission reductions attributable to the lowered expectations of land converted

7 Target revised upwards from 100 to 500 in March 2017: see page 7 for further details

Indicators Progress at June 2017 and % increase from June 2016



Area converted to iSPS in participating farms 1859 ha + 665% 4,000 4,000

Increase in average stocking rate (cows/ha) in project areas

15% + 36% 10% 15%

Number of cattle ranching farmers sensitized on SPS and informed about availability of credit sources

5,628 + 18% 5,500 5,628

Number of professionals and technicians trained on SPS establishment and management

370 + 95% 500 370

10

to iSPS specifically was calculated to be 265,000 tCO2e. These results indicate a 13% reduction in GHG emissions avoided as a result of the reduced expectations around land conversion to iSPS, though if extended the project expects to convert a greater amount of land to iSPS in partial mitigation. This assessment has resulted in a better understanding of the link between iSPS and emission reductions which will prove valuable for informing future initiatives and for informing the project’s full evaluation.

“Should the project meet the requirements of the agreed milestones and action plans over the next six months, BEIS should review the case for a one year extension to the project and possible additional supervisory funding (this decision will be taken within the context or other portfolio priorities and will be based on ongoing performance).”

The World Bank has agreed a restructuring package presented by FEDEGAN and has submitted a proposal to BEIS for a two year extension to the programme. This would see the programme end date moved to 31/01/2020. We are satisfied that sufficient progress has been made over the course of the last year for an extension request to be considered and we are therefore assessing the case for granting an extension separately. The outcome of that assessment will be published in due course.

The project implementation team should ensure that lessons learned on the effectiveness of the various interventions made are disseminated as part of the project’s communications strategy. A key part of this programme’s value lies in its ability to act as a pilot, testing different approaches for driving land conversion to SPS and evaluating the impact of SPS on improving livelihoods and delivering forest protection and climate change mitigation outcomes. For this reason we consider the effective dissemination of learning to be central to the overall success of the project. This is a prominent reason for considering the case for extending the programme. Evidence from the project has been used in several academic publications exploring this issue and the project team has developed several strategic alliances within Colombia to help disseminate learning and share best practice. This work is key to ensuring that the project delivers long-term transformational change and further emphasis should continue to be placed on this work for the remainder of the project’s life. In particular, BEIS and the World Bank team should work with the implementing agencies to ensure that they recognise the value of this component of the work and that they are effectively prioritising and tracking dissemination efforts.

Recommendations BEIS and the World Bank team should work with the project’s implementing agencies to ensure that they recognise the value of the project’s work to disseminate learning, and to ensure that they are effectively recording activity undertaken in relation to this work. The project’s implementing team should also consider maximising the learning which can be drawn from this project by producing a specific report on land conversion to iSPS, describing the lessons that have been learnt and explaining how these could be applied to similar future interventions.

11

Output Title Component 2: Increasing connectivity and reducing land degradation through differentiated PES schemes


Risk: Moderate Impact weighting (%): 35

Risk revised since last AR? No Impact weighting % revised since last AR?

No

Indicators Progress at June 2017 and % increase from June 2016



Area under PES schemes in project areas (PES-1 and PES-2)

41,321ha (PES-1) 1,584ha (PES-2)

+ 35% +1200%

49,411ha (PES-1) 4,000ha (PES-2)

49,411ha (PES-1) 4,000ha (PES-2)

Number of cattle ranching farms benefitting from a PES scheme (biodiversity or carbon)

1,681 (PES-1) 1,099 (PES-2)

0% +50%

1,700 (PES-1) 1,255 (PES-2)

1,700 (PES-1) 1,255 (PES-2)

Number of focal plant species used/conserved in cattle ranching farms (25 of which are globally important species)

50 + 85% 50 50

Number of market-based / consumer initiatives designed, (including large-scale payment for environmental services mechanisms), that could support the broader adoption of SPS by the end of the project

2 NA (0 at June 2016)

2 2

Key Points This component has been scored an A – outputs met expectations. A backlog in making payments to ranchers under the GEF funded PES-1 scheme, identified during the last annual review, has now been cleared and the UK funded PES-2 scheme is now proving itself to be a working incentive and is driving the increase in conversion to iSPS. The project is on track to reach the target number of cattle ranching farms by January 2018. The last annual review scored this component B largely due to dissatisfaction among farmers caused by significant delays to payments made under the PES-1 biodiversity scheme. The backlog of payments which existed at that time has now been resolved. The project undertook a series of adjustments to reduce delays in payments to farmers, which were caused by the complex methodology applied for the establishment of baselines and on-farm verification of land use changes, as well as from coordination issues within the responsible agencies. Coordination issues were resolved by improving governance and internal processes, reducing the number of approval stages required to make a payment, and the methodology for verifying land use changes was simplified. These changes have cut the overall time for these activities from 12-18 months to 6 months; as a result all payments were made by December 2016. The project’s PES-2 scheme is designed to benefit small and medium sized farmers exclusively by providing financial support (inputs + cash payments) to help them adopt land management practices which result in farmed land becoming a net carbon sink. It has been redesigned and is now proving itself to be a working incentive. The area covered by this scheme has increased from 119 ha at the time of the last annual review to 1,584ha by June 2017. This has largely occurred because the PES-2 scheme incentivizes conversion to iSPS. Therefore the acceleration in the uptake of the PES-2 scheme has driven the increase in iSPS as discussed above: 84% of the 1,886ha planted with iSPS in the first half of 2017 was driven by the PES-2 scheme. The discussion regarding land conversion to iSPS above is therefore largely relevant here.

12

It is worth noting that the lower than expected land conversion to SPS and iSPS reflects a lower than expected conversion capacity of the farms involved in the project rather than a reduced number of beneficiary farmers relative to original expectations. The ranchers which have been enrolled in the project have an average farm size which is lower than the farm size which was originally anticipated. The farms are also more widely dispersed than originally thought. This has meant that the project has had to make a partial trade-off between delivering cost-effective technical assistance to support farmers to convert large areas of land to SPS (by working with larger, more wealthy ranchers that have a greater capacity to convert large areas of land) and alleviating poverty amongst the poorest ranchers (who have smaller farms and lower conversion capacities). The project has opted to continue working with the poorest ranchers to maximize alleviation of rural poverty. The final target for the number of cattle ranching farms benefitting from the project incentive schemes has almost been reached and is on track to be met. Summary of responses to issues raised in previous annual reviews “BEIS should continue to engage closely with the World Bank team over the coming months to monitor the project’s progress against the agreed milestones and action plans and in particular, to check progress on streamlining the contracting and payment processes and reducing the current backlog. This will be crucial for putting the project back on track. The action plans which were agreed during the programme’s mid-term review mission have been carried out and a number of key issues which were delaying implementation have been resolved. The contracting and payment processes have been improved and the backlog in payments has been cleared. While the previous annual review noted that a number of these issues could and should have been resolved more quickly we are satisfied that good progress has now been made. This is reflected in the marked increase in the implementation and disbursement rates over the last year.

Recommendations As part of the project’s end of programme assessment the project team and the World Bank should produce a joint report setting out their experiences relating to the design and implementation of PES schemes. This should set out specific recommendations in PES design for the Colombian Government. We recommend that the team engage the Colombian Agricultural ministry with this work, and that consideration be given to producing these reports earlier if it becomes apparent that this would be beneficial to other initiatives seeking to replicate this approach (enabling the learning from this programme to more effectively drive wider transformational change).

13

Output Title Component 3: Strengthening Sub-sector Institutions, Dissemination and Monitoring & Evaluation


Risk: Minor Impact weighting (%): 20

Risk revised since last AR? Yes

Impact weighting % revised since last AR?

No

Key Points This component has been scored A. The project has made good progress in the further development and implementation of a communications strategy and there is evidence that outputs from the project are being used to inform wider policy and other initiatives. The Programme has also effectively piloted the use of a new methodology for reporting on ICF Key Performance Indicator 8, on avoided deforestation. The project is now reporting progress against all indicators and sub-components. Good progress has been made in further developing and implementing the project’s communications strategy over the course of the last year. Various materials have been made available through FEDEGAN’s website and the project is also engaging a wider audience through the use of social media. The project team has also made progress on the dissemination of results and the project has been majorlighted in a number of international publications. Domestically, learning from the project has informed the development of Colombian policies relating to payments for ecosystem services and the Colombian National Development Plan. Furthermore, the project is due to host an international symposium on SPS in September and we are aware of a project being set up by the Food and Land Use Coalition which is likely to directly use learning from the SPS programme to explore how to scale up sustainable cattle management practices in Colombia. A particular area of innovation has been the development of a monitoring system to track levels of avoided deforestation and forest degradation compared to both baseline and predicted levels. This work has been delivered in partnership with Ecometrica as part of a wider ICF programme on the reporting of KPI 8 (hectares of avoided deforestation). Initial findings from this method indicate positive outcomes in terms of substantial deforestation avoided at both farm and landscape level. Further refinement of this approach and wider dissemination could be undertaken in an extension period. The last annual review noted that the project’s monitoring and reporting processes would benefit from greater coordination between partner institutions, including through the development of a shared information management system. Progress has been made and the project is addressing this in two phases, relying initially on shared filing and storage systems whilst developing an integrated platform for the whole project. Several modules of the shared platform have been developed and further benefits should be realized once the full system has been deployed.

Indicators Progress at June 2017



Number of strategic alliances established with key public and private, national and regional entities for the promotion of SPS in Colombia

9 (increase from 3 at June 2016)

10 10

M&E system established and providing timely and relevant information on project’s activities and results.

Yes Yes Yes

Communication Strategy implemented for different target audiences (mainly policy makers and farmers)

Yes Yes Yes

Information system in place for reporting farms adopting SPS, including those not directly participating in the Project.

No Yes Yes

14

Summary of responses to issues raised in previous annual reviews “The World Bank must ensure that the remaining elements of the monitoring and evaluation framework are in place as soon as possible and that there are no gaps in the 2017 results collection.” The monitoring and evaluation system is now fully established and integrated. The project is reporting actual results against all of the project specific indicators allowing a quantitative assessment as well as a qualitative one to be made. We are satisfied that this recommendation has been met, for further information see section G of this review.

“BEIS and British Embassy Bogota should continue to monitor and encourage Colombian Government engagement with the project to ensure lessons from the project are informing wider policy development.” BE Bogota has advised BEIS that the project is contributing to wider work to design sustainable supply chains for beef and dairy products in Colombia, as well as the formulation of majorer level climate change and sustainability policies such as the Colombian Nationally Appropriate Mitigation Actions (NAMAs) for the agricultural sector. There is good commitment to the programme at a technical level within the Colombian Government. At a political level, the Colombian Government remains committed to addressing the environmental impacts of extensive livestock rearing and their recently announced strategy to reduce deforestation8 identifies the implications of this practice as one of its core priorities. Although we are not currently aware of any bids to forest funds, the NAMA facility or the Green Climate Fund we are aware of a specific instance where learning from the SPS programme is directly informing a project which will explore how to scale up SPS and other sustainable approaches to livestock management country-wide. This project is led by the Food and Land Use Coalition and is in the initiation stages. We also have strong anecdotal evidence to suggest that the new ‘Colombia Sostenible’ flagship programme will seek to fund a sustainable cattle intensification programme as one of its first priorities, further demonstrating a strong interest in this approach within the Colombian Government. Recommendations The World Bank, the project team and the Colombian Government should consider how the project’s interventions can be used to deliver results in regional and national results-based payments programmes such as the BioCarbon Fund Initiative for Sustainable Forest Landscapes. BEIS and the British Embassy in Bogota should consider how they can encourage greater cooperation between the various initiatives which BEIS and the ICF fund in Colombia.

8 Estrategia Integral de Control a la Deforestación y Gestión de los Bosques en Colombia – EICDGB

15

Output Title Component 4. Project management

Output number per LF n/a Output Score B

Risk: Major Impact weighting (%): 10

Risk revised since last AR? No Impact weighting % revised since last AR?

No

Key Points As discussed in previous sections the fundamental issues with the management of the project have been addressed over the last year and the rate of achievement has improved markedly as a result. The project implementation team are to be commended for their efforts. That said, some key aspects remain below target. The project implementers have not fully responded to feedback to strengthen the project implementation team to meet the scale of the challenge and the lead project manager has recently left the team. It will be challenging to find a short term replacement and retain other project staff as the end of the project approaches: this situation will naturally change if the extension request is approved. If the project is to be extended a further step change in delivery will be required with the project potentially needing to operate in new areas (requiring new regional staff to be recruited) and to sign up many new farmers. The project implementation team provides 6-monthly reports to the World Bank. The most recent technical report was published in August and covers the period January-June 2017. These reports are of a good quality and are useful for assessing progress; this most recent report was used to inform the findings of this review. The World Bank team has conducted two supervisory missions to the project since the last annual review, in December 2016 and one in April 2017. The BEIS project manager joined the team for the latter of these trips. The World Bank team produce aide memoire’s following each of these trips, these have also proven useful for monitoring and assessing progress and have been used to inform this review. Summary of responses to issues raised in previous annual reviews “BEIS should continue to engage closely with the World Bank team over the coming months to monitor the project’s progress against the milestones and action plans which are aimed at putting the project back on target.” The BEIS project manager joined the World Bank team in their supervisory mission to the project in April 2017 and participated in regular calls to remain appraised of progress. The project has received oversight from the same project manager at BEIS over the last year although further staff changes mean that a new project manager will now be responsible for supervision at BEIS. Recommendations BEIS to assess the World Bank’s extension request by October 2017, ensuring thorough due diligence to ensure an extension represents good value for money. The project implementation team should recruit a new project co-ordinator as soon as possible. If BEIS approve an extension to the project, the World Bank and the project team should conduct an updated assessment of the resources required to fully deliver the outputs expected in the extension phase. This could form part of the risk assessment referenced in section E.

16

D: VALUE FOR MONEY & FINANCIAL PERFORMANCE

Key cost drivers and performance As of 12 July 2017, the World Bank had disbursed 61.9% of total resources to the project, equating to 55.3% of BEIS funding and 81.7% of GEF funds. The majorer disbursement rate of GEF funds reflects (a) their involvement in the project from an earlier date, and (b) the lower amount of overall GEF funding allocated to the project (see program outline on page 2). As noted in the last annual review, the significant depreciation of the Colombian peso against the dollar (ca. 60%) has contributed in part to the slow implementation / execution of the disbursed funds9, by making available more financial resources than originally budgeted. In the last annual review it was also noted that accelerating the execution of funds was critical and that it was essential that by December 2016, the project reached a budget execution greater than 45% for BEIS funds and a minimum of 75% in the case of the GEF. These targets were not reached. That said, it should be noted that the disbursement rate almost doubled over the period of 9 months to June 2017 from 34.5% in October 2016 to 61.9% in June 2017. The World Bank have advised that the disbursement rate for BEIS funds is expected to reach about 65-70% by the current project closing date in January 2018. This means that unless the project is extended significant underspend can be expected. Any BEIS funding which remained unspent would be returned at this stage.

Component June 2016 amount Executed ($)

June 2017 amount Executed ($)

% of allocated budget executed June 2017

change in %pts since June 2016

% of total executed change June 16 - 17

Component 1 2,902,000 4,964,800 52 22 42

Component 2 1,031,000 2,958,271 38 25 39

Component 3 279,000 863,878 33 22 7

Component 4 167,000 492,993 72 47 12

Total 4,379,000 9,279,943 45 21 100

The execution of funds allocated to Component 4 has increased significantly over the last year. This is

due to increasing costs associated with retaining staff 10 and managing the project (e.g. increasing the

salaries of project delivery staff, in-line with market rates and BEIS recommendations). The project has

also had new capacity gaps to fill which have arisen as a result of changing responsibilities within

FEDEGAN. Finally, the projected has contracted an additional procurement officer (0.5 FTE) to help

accelerate processing times over the last year. The execution of funds for the other 3 components has

increased by a similar amount.

VfM performance compared to the original VfM proposition in the business case

Economy

Administration fees have not changed since the last annual review.

Efficiency

The World Bank has addressed inefficiencies arising from internal factors causing the low disbursement

rate. Administrative bottlenecks associated with the PES1-scheme have been addressed and a Level II

restructuring revisited PES2-scheme’s technical assumptions. The latter supports the adoption of

9 Funds are ‘disbursed’ when the World Bank transfers money to FEDEGAN. Funds are considered ‘executed’

when spent by FEDEGAN / the project implementation partners. 10

as set out earlier in this review, salaries of implementation staff have been increased in-line with BEIS recommendations),

17

intensive silvopastoral systems (iSPS). There has been a good response to the adjustments in the PES2

scheme; this is illustrated by the increasing number of hectares converted to iSPS as set out earlier in

this review.

The detailed output scoring and overall output sections above show that there has been mixed results for

project outputs. Although some outputs are expected to reach there end of project targets, the total

amount of land area where sustainable land management practices have been adopted is expected to

be below the end of project target. This is a key output for the project achieving value for money as it is

central to its overall effectiveness. There is however expected to be a large increase in this output from

June 2017 to the EOP suggesting an improvement in efficiency.

Cost Effectiveness

1. Leverage ratio of ICF resources to private / public investment.

The amount of public financed mobilised is no longer being reported against. This is because project

participants have not been able to access credit from the Government’s FINAGRO credit line as

originally envisaged.

There has been an increase in the expected private finance mobilised since the last annual review. The

end of project expected attributed amount is £4.5m. This increase from the last annual review ($0.4m)

reflects the majorer attribution rate given to UK funds for private financed leveraged. Attribution has

increased for the UK to reflect that only private investment by the farmers themselves into SPS is

included in the amount leveraged. Other private contributions are no longer included as they were

committed before BEIS joined the programme.

2. Abatement costs in terms of £ per tonne of CO2e abated - this is calculated from data on project

costs and outcomes

Using the latest figures for the expected reduction in GHG emissions, the expected end of project

attributed abatement cost is £14.1/tCO2e abated. This is greater than the amount expected in the

business case (£8.2/tCO2e).

3. Cost per hectare converted to Silvopastoral systems.

The project is not currently measuring separate GEF and BEIS funding attributions of hectares

converted. Using the latest reported figures on financial disbursement and expected land converted to

SPS, the expected attributed cost per hectare is £645, which is an improvement since the last Annual

Review (£662); however the cost per hectare is still greater than originally expected (£540).

4. Increase of income for small-scale farmers

The original business case set out four expected outcomes related to farmers’ income:

20% increase in productive assets-based patrimony of small-scale livestock farmers through the

establishment of iSPS in Project’s participating farms, at project closing date.

10% increase in the production of beef and or milk per intervened hectare in participating farms, at

project closing date, improving GHG balance.

The conversion from degraded pastures to SPS (i.e. restored pasture, pasture with trees, live fences

etc.) is expected to increase income per hectare by at least 50% since stocking rates are likely to

double3 after seven years of the establishment of the SPS.

Total farm income will rise according to the amount of land converted to each improved land use.

As in the 2016 review, results for only the second outcome have been recorded. As of April 2017, this

measure was 17%, which is 7% above expected and a 10 percentage point increase from the previous

18

annual review. This increase in productivity will have a positive impact on the incomes of farmers from

participating farms.

Effectiveness

As shown in the overall output score and description section, revised end of project targets for outcome

indicators are not expected to be achieved. As a result, the project is unlikely to have been as effective

at reducing the impact of cattle farming on climate change and environment as expected at the time of

developing the business case. Improvements have been made with respect to these indicators over the

last year and if the expected end of project targets for these indicators are realised, the effectiveness of

the project can be expected to increase between now and the project’s completion date.

The above expected performance in increasing productivity for participating farms (i.e. increasing

production of beef and milk per intervened hectare) indicates the projects is likely to have had a positive

impact on reducing poverty among small-scale livestock farmers. The overall impact on the incomes of

small scale farmers is limited as the number of farmers using more productive Silvopastoral systems will

be reduced in line with the below target end of project achieved for land converted to SPS.

Assessment of whether the programme continues to represent VfM

The low disbursement rate of UK funds has had a substantial negative impact on the projects value for money. Crucially, the land area converted to SPS is expected to fall below its target by 11,000 ha therefore limiting both the projects impact on reducing climate change and increasing the incomes of small-scale farmers. On the other hand, it is important to consider in this assessment that the World Bank expects the targets for area converted to SPS and GHG reductions to be achieved if the project were extended so that all UK funds can be disbursed. Therefore agreed targets could be achieved with the only additional cost being administration costs. The project has also been successful in achieving other outputs as shown in the detailed output scoring. Indicators of cost effectiveness have fallen below original expectations, however a number of targets for the outcomes/outputs which these indicators are dependent on have also been revised as the project has progressed. The differences between indicators in this annual review and the business case should therefore not be interpreted as a substantial divergence in value for money. In summary, failing to disburse all UK funds has limited the project’s value for money. Despite this, the fact targets for land area converted to SPS and GHG reductions could be achieved if all funds were disbursed, demonstrates the project is on course to achieve its expected impact as set out in the revised log frame. The low disbursement rate has been caused by external factors as well as internal factors, and project management should be commended for staying on track to achieve these targets relative to UK funds disbursed. Based on this and the success of achieving other output targets, the programme is considered to represent value for money but with significant scope for improvement through the full disbursement of UK funds. Quality of financial management

Date of last narrative financial report August 15, 2017

Date of last audited annual statement May 31, 2017

19

E: RISK

Overall risk rating Major, though if an extension is approved the risk rating will move to Moderate. Overview of programme risk While delivery against many of the project indicators has accelerated significantly over the past year, with the time remaining for the project fast running out, there is no prospect of meeting many of the end of project output targets without an extension. In turn this would reduce the ability of the project to deliver outcomes that translate into long-term impacts. Colombian Government interest and engagement in the programme remains major at a working level. At a political level, the Colombian Government remains committed to addressing the environmental impacts of extensive livestock rearing and its recently announced strategy to reduce deforestation identifies the implications of this practise as one of its core priorities. FEDEGAN’s lead project manager has secured a new position with TNC. FEDEGAN is reluctant to recruit a replacement ahead of the decision on the project extension being taken. BEIS are due to make a decision on this extension in the immediate future and it is likely that this decision will have been made by the time this annual review is published. Significant delays to the agreed timeline for making this decision at BEIS could create implementation risks for the delivery partners both in terms of making it difficult to retain key staff and in terms of delaying implementation of potential longer-term activities associated with the extension. Recommendations BEIS / British Embassy Bogota should continue to engage with the Colombian Ministry of Agriculture as regards its intentions to pursue further sustainable cattle intensification programmes. If the project is extended, an updated risk assessment should be conducted by the World Bank identifying any further risks to implementation and to the potential of the project to deliver the intended long-term outcomes and impacts. Appropriate mitigation plans should be put in place. Outstanding actions from previous risk assessment None.

20

F: COMMERCIAL CONSIDERATIONS

Delivery against planned timeframe As discussed in previous annual reviews the project was slow to get moving with delays in setting up the project team and the monitoring and evaluation framework, with further delays to planting caused by unforeseen climatic conditions. This has resulted in the revision of a number of the project’s indicators. A number of the project’s indicators are now on track to be delivered by the project’s closing date but the project will not be able to scale up implementation sufficiently to meet all of the project’s targets by this date. As previously discussed in this review, we are considering the case for granting an extension to the programme. As part of this assessment we will consider whether doing so would allow the project’s targets to be met in full and if so, whether this would result in a programme which ultimately delivers better value for money overall. The project team has submitted a formal request to the World Bank which would involve spending the current programme budget over the longer period. The World Bank has approved this request and submitted it to BEIS along with a proposal for additional funding to cover its extra management costs. BEIS is currently considering the case for approving this extension.

Performance of partnership(s) BEIS’s primary relationship is with the World Bank. This relationship is working well and the World Bank has been responsive and helpful throughout the year. FEDEGAN is the lead implementing organisation and the April 2017 mission attended by the BEIS project manager showed FEDEGAN’s relationship with the World Bank is open and collaborative. The last annual review reported that there were issues with the methods of working between FEDEGAN, Fondo Accion and TNC in relation to PES1, which had led to significant delays in carrying out verification of land-use changes, processing of contracts with farmers and subsequent payments. These processes have improved significantly over the last year and the backlog of payments under PES-1 identified at the last annual review has now been cleared. Asset monitoring and control The World Bank has visited the project twice in the last year. During the December mission the World Bank team visited participant farms in the Atlantic region of the country. The BEIS project manager joined for the April mission directly engaging with the central project implementation team, and senior figures from FEDEGAN and the Colombian Environment and Agriculture Ministries.

21

G: MONITORING & EVALUATION

Evidence and evaluation The monitoring and evaluation system is now fully established and integrated. The project is reporting actual results against all of the project specific indicators allowing a quantitative assessment as well as a qualitative one to be made. KPI 6 - Net Change in Greenhouse Gas Emissions (tCO2e) – tonnes of GHG emissions reduced or avoided and KPI 811 - Number of hectares where deforestation and degradation have been avoided through ICF support (ha). The last annual review reported that a contract for monitoring KPIs 6 and 8 had yet to be put in place. That contract, between the World Bank and The Nature Conservancy (TNC) was signed in August 2016. As per TNC’s contract, GHG emission reduction and carbon sequestration measurements associated with the changes in land use categories promoted by the project are divided into three phases:

(i) Phase 1 (initial): Definition of carbon measurement methodologies to be applied at the farm

and landscape level.

(ii) Phase 2 (implementation): carbon measurement and reporting of KPI-6 and KPI-8 indicators.

(iii) Phase 3 (final): compilation of collected data and recommendations.

Phase 1 is complete, TNC has produced a document “Defining Methodologies for Measuring Carbon in Soil Categories Promoted by the Sustainable Colombian Livestock Project”, which presents the methodology being applied to carry out the measurements and field sampling, analysis of the information, and estimates of the GHG emission or reduction associated to the different land use changes categories promoted by the project.

Phase 2 is being implemented. For KPI 6, the collection of samples in the field has been finalized, and collected data is currently being analysed. A preliminary report will be produced by mid-October; a second report will be produced with data resulting from the verification of land changes reported during the second semester of 2017. For KPI 8, the team has defined the protocols for quantification of deforestation and degradation using project reported activity and national and regional data, supported by map images. TNC has worked closely with Ecometrica to align the methodological approach to the reporting platform developed by Ecometrica. The project interface on Ecometrica’s platform has already been created12. The KPI8 report for the project provides good insights on the positive impacts of the project approaches on deforestation.13

In the extension phase, the project team would continue working to generate more robust data, optimize the process and update the KPI 8 report with new farms and areas of the project. The next version of the report will include Boyaca Santander region: this has a very complex topography and the team is reviewing the input data to ensure accuracy.

KPI 11 - Volume of public finance mobilised for climate change purposes as a result of ICF funding (£) At the last results collection in March 2017, BEIS decided that KPI 11 will no longer be reported on as project participants have not been able to access credit from the Government’s FINAGRO credit line as originally envisaged and the chances of significant further public funding being leveraged are remote.

11

These refer to ICF-wide KPIs rather than the KPIs which are specific to this project, which are assessed within this review and referred to as ‘project development objectives’ 12

Available at https://icf-sps.tnc-co.ourecosystem.com/interface/ 13

Available at https://tnc.app.box.com/s/35h73u2b1ucp2ag46waqfehkmmc4spth

22

KPI 12 - Volume of private finance mobilised for climate change purposes as a result of ICF funding (£) The attribution of results to KPI 12 was amended at the 2017 results collection to only reflect private investment by the farmers themselves into SPS (for example via their own equity, obtaining debt financing if possible and through in-kind investments such as labour to implement SPS).

KPI 15 – Transformational Change

A Methodology for assessing Transformational Change was developed following the last Annual Review and a full scoring for this is outlined in Section H. The programme is currently assessed as scoring a 2 (Some early evidence suggests Transformation likely). As SPS is developing several promising partnerships, the KPI 15 methodology would be reviewed following an extension to ensure these are appropriately reflected.

Impact Evaluation

A quasi-experimental impact evaluation is being delivered under the responsibility of the World Bank, including with support from their independent Development Impact Evaluation team. The evaluation will assess the average treatment effect of the project on participants by comparing this against a matched group of farms from a control group. The Bank will also prepare a final completion report (ICR), which essentially represents a performance evaluation of the project. The selection of study participants has been subject to data limitations on key socio-economic variables and their self-reported nature. However the use of this innovative case-control approach means the study should be able to distinguish the varying impact of various forms of treatment offered, including the various types of payments and the provision of technical assistance. The evaluation also aims to see whether impacts varied among participants. In particular, it will examine the extent to which poorer farmers adopted silvopastoral practices as a result of the project’s interventions. The analysis will use a variety of metrics of the degree and type of adoption: Area converted (ha); Percent of farm converted; Increase in environmental service points; Percent increase in environmental service points; and Increase in environmental service points per hectare. To collect data for the analysis, a baseline survey was administered to all participants in the treatment group (that is, all participants in the third convocatoria) and to all members of the control group selected through matching, as early as possible following their selection. The initial land uses of all members of the treatment and control groups were mapped at this time, using the same techniques in both cases. A second survey will be conducted among all treatment and control households at the project’s close. Land use changes by treatment group participants are monitored annually as such monitoring is necessary to allow the conditional annual payments to be made. Land use changes by control group participants, on the other hand, are only monitored at the project’s start and end points. With the very limited time that remains in the project, any land use changes may not be easily observable, limiting the possibilities of a robust impact evaluation. If the project is not granted an extension then the final monitoring and survey should still be delayed if possible to give more time for effects to be seen. A programme extension would not only assure this additional time but would also increase the potential effect size given the extended duration of the treatment. Monitoring progress throughout the review period The BEIS project manager joined the April 2017 World Bank monitoring mission and was accompanied to some of the meetings by the UK Embassy in Bogota. The World Bank’s Aide Memoires from this review and from their other mission during the review period (December 2016) have been used to inform this report, as has the latest technical implementation report from FEDEGAN. The UK embassy in

23

Bogota have maintained their proactive role in monitoring project progress over the past year and the BEIS project manager has been in regular contact with the World Bank team.

24

H: TRANSFORMATIONAL CHANGE

Rating The project has been given a score of 2 (Some early evidence suggests Transformation likely) for transformational change. Evidence and evaluation Transformational Change is being assessed using cross-cutting, enabler and driver criteria. Indicators against the relevant criteria are a combination of Key Performance Indicators (KPI), Project Specific Indicators (PSI) and identifiable actions in Colombia that are known already. The end of project (Jan 2018) targets for each indicator and progress as of June 2017 are indicated in bold after each indicator: Cross-cutting Political will and local ownership: The project will seek to galvanise action by the Colombian Government and the cattle ranchers to continue the implementation of SPS and iSPS beyond the life of the project and across a greater geographical reach. The ultimate ambition would be to see SPS being implemented across Latin America as the cumulative impact of a number of interventions demonstrates the value of SPS.

PSI: Number of strategic alliances established with key public and private, national and regional entities for the promotion of SPS in Colombia – Target = 10 Progress = 9

National Planning Department to include a meaningful role for SPS in the next National Development Plan being issued in 2015 (identifiable action) Incomplete.

Enablers

Sustainable: The longevity of the intervention once the project ends will not be monitored, however evidence from previous projects suggests that SPS systems are retained for at least 4 years even once payments to farmers end. This indicates that SPS can be sustained without support once implemented. The project will build on the political support for SPS to promote a larger scale intervention driven and owned by Colombia, potentially in the form of market based mechanisms or regulation.

PSI: Number of market-based/consumer initiatives designed, (including large-scale PES mechanism), that could support the broader adoption of SPS by the end of the project. Target = 2 Progress = 2

Drivers

Capacity and capability can be increased: countries and communities have the capacities and capabilities necessary to bring the change about;

PSI: Number of professional and technicians trained on SPS establishment and management

Target = 500 Progress = 370

Innovation: innovative technologies are piloted, with the potential to demonstrate new ways of doing things, which could lead to wider and sustained change;

KPI 8: Carbon Balance where deforestation and degradation have been avoided through ICF support at the landscape level and farm level in Colombian Cattle ranching landscapes. See section G of this review.

Evidence of effectiveness is shared: approaches which have proved successful in one location are made widely available and lessons on their usefulness are credible and shared widely;

PSI: Communication strategy implemented for different target audiences (mainly policy makers and farmers.) Complete

Additional Information The project has made a concerted effort to engage the wider ranching community and there is anecdotal evidence from regional project staff that other farmers are showing strong interest in being involved in the project, and the programme's nurseries are making good sales beyond project farms. We have also received positive feedback from Colombian Ministers and the Head of FEDEGAN. Progress against the methodology indicators as set out above is positive though in places incomplete and whether the implementation of SPS in some farms will lead to a widespread take up beyond the farms enrolled in the project remains to be seen. The project is registering where it understands other farms have established

25

SPS, but they are not registering when or why this occurred, so no causal link can be confirmed. This has been addressed in a revision to the relevant project indicator (now removed from the summary above) and its methodology. This said, we are aware of strong interest in exploring how to scale up efforts to drive conversion to sustainable cattle ranching in Colombia. In particular we are aware of a specific instance where learning from the SPS programme is directly informing a project which will explore how to scale up SPS and other sustainable approaches to livestock management country-wide. This project is led by the Food and Land Use Coalition and is in the initiation stages. We also have strong anecdotal evidence to suggest that the new ‘Colombia Sostenible’ flagship programme will seek to fund a sustainable cattle intensification programme as one of its first priorities, further demonstrating strong interest in this approach within the Colombian Government.