Impacts of Certification on Small Coffee Farmers Western Kenya, 2014-2017

Prepared by COSA for the ISEAL Alliance Demonstrating and Improving Poverty Impacts Project Final Report

1

Acknowledgments

Author: Carlos De Los Ríos | COSA Senior Researcher

Research: Carlos De Los Ríos Fieldwork: The data collection team for the household survey was led by Josky Gwaka and included: Cosmas Lutomia; Jared Amunga; Vivian Makungu; Stephen Kimei; Cyprian Anyim; and Annette Wamoyi. Interviews and focus groups dynamics were led by Josky Gwaka. We appreciate the invaluable support of the Ford Foundation and the leadership of ISEAL on this topic. We thank the consistent and thoughtful guidance of Kristin Komives; and the support from Vidya Rangan and Udit Khare of ISEAL; and the invaluable expertise of Howard White of the International Initiative for Impact Evaluation. Special thanks go to the participating farmers, and the management of the farmer cooperative societies. Equally thanks to Lucy Murithi of the Coffee Research Institute; Enosh Akuma of the Coffee Directorate of the Ministry of Agriculture, Fisheries and Food Authority; George Watene of the Global Coffee Platform; Isaac Rewa of Fairtrade Africa; Marten Sievers, from the Neumann Gruppe; and Ivy Sarah Nderitu from the Kenya Coffee Traders Association (KCTA).

We also want to appreciate the invaluable comments from Vidya Rangan of ISEAL; Margret Loeffen of Fairtrade International; and Henk Gilhuiz of Rainforest Alliance.

2

Contents Executive Summary ............................................................................... 3

1. Introduction ........................................................................................ 8

The study ...................................................................................................................... 8

The Kenyan coffee context ............................................................................................ 9

The project .................................................................................................................. 10

Impact pathways of coffee certification ........................................................................ 10

2. Research approach, design and methods .......................................13

The quantitative approach ........................................................................................... 13

The qualitative approach ............................................................................................. 16

The project’s evolution and research plan adaptation .................................................. 17

3. Market structure and group dynamics .............................................19

Market structure .......................................................................................................... 19

Group dynamics .......................................................................................................... 21

The certification process ............................................................................................. 22

4. The study .........................................................................................24

Fieldwork ..................................................................................................................... 24

Characterization of the sample .................................................................................... 26

5. The intervention ...............................................................................28

6. Key Findings ....................................................................................33

Farms and farmers ...................................................................................................... 33

Household ................................................................................................................... 50

Producer Organizations ............................................................................................... 56

7. Discussion ........................................................................................60

Conclusions ..........................................................................................72

3

Executive Summary

Background ISEAL and its members are working together on the Demonstrating and Improving Poverty Impacts (DIPI) program, to understand the contribution that certification systems can make to poverty alleviation and pro-poor development. ISEAL commissioned an evaluation to identify whether and how certification contributes to improving farmer livelihoods. It is in this context that ISEAL engaged the Committee on Sustainability Assessment (COSA), to rigorously assess the early impacts of the Fairtrade and UTZ certification processes on small coffee farmers in the Mount Elgon region of Kenya. Although the region is reportedly emerging as Kenya’s next “coffee belt”, it is still one of the poorest in the country and has limited experience with standards and certification1. This study also aims to contribute to the analytical framework that ISEAL and its members can utilize to understand the impacts of certification standards under similar settings.

The Kenyan Coffee context Coffee is one of the most important export crops2 in Kenya. Its market value plays a crucial role in the livelihoods of millions of rural households3. Smallholder farming dominates Kenya’s coffee sector, producing about 65% of the total volume exported. Structurally, Kenya’s regulated coffee cooperative system requires smallholders to be organized in farmer cooperative societies (FCS). Coffee farmers must sell through registered FCS and these also typically provide other vital services such as wet milling to the farmers. To sell coffee, the FCS must, in turn, use coffee marketing agents (CMA) as the bridge between their farmers and the market4.

The project Solidaridad5 partnered with Coffee Marketing Services (CMS)6, a Kenyan coffee marketing agent specialized in sustainably produced coffee, to prepare and train farmers for certification to the Fairtrade and UTZ coffee standards. They engaged two producer organizations having low productivity and quality in Bungoma district, over approximately

1 COSA (2016). Impacts of certification on organized small coffee farmers in Kenya. Baseline results. 2 Ministry of Agriculture, Livestock, Fisheries and Irrigation (2019). http://www.kilimo.go.ke/wp-content/uploads/2019/01/Download-here.pdf 3 Ruben, R and P. Hoebink (2015). Coffee certification in East Africa: impact on farms, families and cooperatives. Wageningen Academic Publishers 4 Van Rijsbergen, B; Elbers, W.; Ruben, R.; and S. Njuguna (2016). The ambivalent impact of coffee certification on farmer’s welfare: a matched panel approach for cooperatives in Central Kenya. World Development, Vol 77. Pp277-292. 5 Solidaridad is a non-governmental organization that engages supply chain actors in innovative solutions to improve production and ensure a sustainable and inclusive economy. 6 Coffee Management Services Ltd (CMS) is a private company owned by East Africa Coffee Co LLP. It has emerged to be one of the leading coffee marketers, marketing around 23% of total Kenyan coffee. Currently CMS works with approximately 250,000 smallholders in Kenya (Kenya Coffee Platform (2019). Coffee Economic Viability Study. https://www.globalcoffeeplatform.org/assets/files/03-GCP-Tools/Kenya-Coffee-Platform-Coffee-Economic-Viability-Study-Report.pdf).

4

two years. CMS expected to close these quality and productivity gaps through certification and the associated organizational and training processes so that it could market improved and certified coffee. In general, there are two distinct avenues through which certification contributes to poverty alleviation. First, there is specific training towards certification at the producer organization and farm level can contribute to improved livelihoods through capacity building in different dimensions depending on the particular certification standard. Second, once the entity is certified, there are potential benefits in terms of market access and better prices.

The evaluation problem and research design With limited numbers of FCS and appropriate counterfactual control groups in the region that could be candidates for certification, the project faced the scientific challenge of determining attribution of the observed distinctions with sufficient statistical confidence. This was compounded by a risk of self-selection7 of producer organizations entering into certification and the potential of another selection bias8 arising from the incentive for CMS to select organizations with an already higher potential to become certified (e.g. better organized farmer groups, higher aggregate production, etc.). Since the setting severely limits the ability to establish attribution using purely quantitative methods. COSA partnered with the International Initiative for Impact Evaluation (3ie) on the development of the initial research plan to ensure state-of-the-art approaches and applied world-class methodological rigor. The result, developed jointly with ISEAL researchers, was a mixed-methods approach combining structured qualitative tools and methods together with strong quantitative techniques suitable for the local context and purposefully chosen to rigorously identify a causal chain consistent with the intervention’s theory of change. Just before the impact evaluation, it emerged that the certification project had not gone as planned but that there were also potentially useful lessons to be learned. While both of the target FCS received training towards certification, only one of them became certified, and only with Fairtrade. The initial methodological framework was designed to allow for potential changes and it was still applicable with modifications taking into greater account the market structure and group dynamics that appeared to be strongly affecting outcomes. This scenario is thus different than expected and yet presents an important opportunity to refine combined methods for addressing impacts in what is an increasingly common scenario for sustainability standards in developing economies.

The pivotal role of market structure and group dynamics By Kenyan law, all coffee farmers must sell their coffee through Farmer Cooperative Societies and these also process the raw coffee thus placing them in a position of great influence. They, in turn, must use a registered marketing agent to sell their coffee (auction or direct export) and that agent may require that their preferred mill be used for final processing. So the ability to choose is limited.

7 This is a typical problem where the beneficiaries may self-select into the certification program. Characteristics of self-selected beneficiaries may be the source of difference between target and control, rather than the intervention itself. 8 This common problem, where the selected target group has a set of characteristics that are significantly different from the control group, means that an impact difference may be potentially explained by the differences between groups, rather than the intervention itself.

5

Marketing agents are selected annually and they promote their services to the FCS for whom the expectations around services and better prices are often high. Those interviewed report that disappointment is common and both farmers and FCS managers or board members harbor notable levels of mistrust, based on lack of transparency or unfulfilled expectations. Yet, they must work together. We repeatedly observed problems of expectations and transparency that may be addressed with relatively simple interventions. On the one hand, there are substantial information gaps at the farmer and FCS level especially in regard to their limited knowledge of the markets and prices for certified coffees. The FCS do not have the capacity to manage quality controls to ensure that they are fairly paid by the downstream mills (often recommended by agent) and thus harbor a constant suspicion that mills are profiting unfairly from the lack of transparency. Even further down, a perennial problem of weaker FCS is a failure to communicate between managers or board members and the farmers about their choices, FCS operations, and finances. Given the levels of mistrust and the limited choice, the basis on which FCS decide which marketing agents to work with can change constantly. It is true that the agents can serve as a useful intermediary to help bring certification to FCS. In such a situation, especially when a FCS is weak, the marketing agents can effectively become the arbiters of access to certification. These dynamics offer insight to certifiers about how they could manage their role and what may be required in terms of transparent systems and adequate information to ensure that a FCS can determine which, if any, certification is appropriate and for the FCS to have greater agency or determination in that choice by facilitating its active presence and role from the start.

The intervention Training towards certification is the key link in the project’s causal chain, as it is an important channel for delivering new capacities and new information to farmers and their organizations about what the Fairtrade and UTZ standards are and how to meet them. CMS implemented the task, and focused on training the FCS and a set of farmers in group-based settings and demonstrative parcels. Our field assessment revealed that this and other trainings also reached the other FCS and their farmers that were expected to serve as control groups (not planning to be certified) and necessitated some adjustments in the sampling. This essentially made it difficult to assess the causality of some effects including certain economic changes e.g. productivity, coffee revenue, and coffee production costs, amongst others. Trainings in the region were substantially similar with some small differences for target groups such as greater focus on environmental protection practices. The combination of FCS training, farmer-level training (part of certification preparation), and having been one-year Fairtrade certified together offered some measurable benefits.

Key findings We acknowledge that certification is a long-term commitment, that may include a valuable process of continuous improvement, and that being certified one year will not produce all of the expected outcomes. However, we consider that just the fact of achieving certification, implies that the farmer group accomplished a significant set of preparatory steps with accompanying lessons for the farmers and the group.

6

Social We have observed a substantial increase in participation in the FCS meetings, as well as a larger number of farmers voting on important decisions. This is a common trend amongst FCS, and appears to be associated with improving governance at the organization level. According to farmers, the FCS is becoming more transparent and sharing more information with them – though not yet at optimal levels. Farmers are significantly more satisfied with their organization, showing significant improvement in approval rates of their board members. Environmental From an array of environmental practices associated with soil and water conservation and integrated pest management, we found good evidence of changes in two specific practices associated with the intervention’s theory of change: an expanded and more extensive use of shade trees and conducting regular coffee farm inspections to detect pests and/or diseases. Economic There are small but significant changes in the adoption of good coffee-specific agricultural practices. We have observed an increase in the percentage of farmers pruning their coffee in both target and control groups; however, we did not observe substantial differences between groups. We observed a significant increase in the use of chemical fertilizers with strong evidence suggesting that training played an important role in determining its adoption. The percentage of farmers using synthetic fertilizers doubled between baseline and endline in both target and control groups. Furthermore, the average amount of fertilizer used has also significantly increased in both groups. We found strong evidence of the relationship between liquidity or the increase in loans from the FCS, and the probability of using fertilizers. We observed positive changes among farmers in record keeping and understanding price formation as proxies for better farm management. There is still a long way to go however, and we did not observed significant differences between target and control organizations. Yields have increased from an average of 592 kg of coffee (green bean equivalent or GBE) per hectare at baseline to just above 706 kg per hectare at endline. The farmers with the lowest yields improved the most and yet the level of increase has been, on average, equally important in all three FCS (target and control), showing no evidence of statistically significant differences between them. There is evidence of quality improvements for which average prices have increased significantly more in the target group than in the control. While training has promoted practice improvements, it has also had an impact on production costs. We observed substantial increases in fertilizer costs and paid labor. Total costs have therefore increased in both target and control groups. Revenue from coffee farming has largely increased in both target and control groups. The poorer target farmers, at lower ends of the overall revenue distribution, showed greater improvement than control groups. For the overall average net coffee income per hectare, which increased for all groups, there were no significant differences between target and controls. But we did find improvements in the net coffee income of the poorest farmers (lower deciles) in the target group.

7

Total household incomes also significantly increased in both target and control groups due to increases in various categories including other crop income, livestock management income, business income, wage labor, transfers and remittances). We observe no statistical difference of income changes between target and control groups, even when assessing each control separately or when assessing changes across farmers with different poverty levels. Perhaps counter-intuitively, farmers increased from about four days of food insecurity to almost seven days in both target and control groups without clear reasons for that shift (no crop failures or dramatic rise in food costs, etc.) Overall, the nature of the intervention did not fully evolve such that it could present the range of possible benefits associated with certification. Nevertheless, we can state that certain aspects of the certification process evident in this intervention, particularly those related to training, have led to a number of improvements. However, because we also see improvements in many of the outcomes among the control farmers as well, we cannot attribute the change exclusively to the certification processes. Although the limited changes associated with the certification process, particularly the training, are not sufficient to alter the fact that these farmers remain essentially poor, there have been some welcome improvements in productivity and quality. These led to greater coffee income that is particularly important to the poorest farmers in the region.

8

1. Introduction

The study Funded by the Ford Foundation, the ISEAL Alliance and its members are working together on the project Demonstrating and Improving Poverty Impacts (DIPI), aiming to better understand the contribution that certification systems can make to poverty alleviation and pro-poor development. Under the umbrella of this project, ISEAL commissioned three evaluations in three different contexts, aiming to identify whether certification contributes to improving farmers’ livelihoods and to test methodologies to advance the capacity of ISEAL members to develop harmonized and effective monitoring and evaluation schemes, to facilitate learning and improvement of their systems. It is in this context that ISEAL engaged the Committee on Sustainability Assessment (COSA), to develop the case study for smallholder coffee farmers in Western Kenya, with the objective to identify the livelihood and poverty impacts of Fairtrade and UTZ certifications in smallholder coffee farmers, as well as to provide an analytical framework that ISEAL and its members can undertake to understand the impacts of certification standards under similar settings. This report presents the results of the evaluation of the early impacts of preparation for, and certification of, Fairtrade and UTZ coffee standards on organized smallholder coffee farmers in Western Kenya. In particular, this study provides answers to six key research questions:

1. What are the changes that occur at the farm, household, and cooperative levels leading up to certification to the combined Fairtrade and UTZ standards and again after three years of certification?

2. Do different types of farmers, such as those with different initial assets, poverty levels, or gender, experience differing changes in outcomes over time and what is the degree of difference?

3. Can any observed changes in farm or producer organization performance be attributed to the combined Fairtrade and UTZ standard systems?

4. What is the added value that Fairtrade and UTZ standards systems bring to producer organizations, farms, and households, beyond training? This will include but not be limited to examining the extent to which farmers and producer organization managers feel satisfied with the experience of certification (in terms of challenges and cost-benefit perceptions).

5. What contextual factors significantly influence the effect of Fairtrade and UTZ standards systems on producer organizations, farm, and household changes in performance? The factors to test for influence are: the market orientation of the program, Kenyan and global coffee prices, the producer organization management and structure, livelihood and poverty context, cultural context, and project implementation experience.

6. What are the reasons that different types of farmers (for example, those with different initial assets, poverty levels, or gender) experienced different changes in outcomes, if any such differences are identified in the quantitative analysis?

9

The Kenyan coffee context Agriculture is the cornerstone of Kenya’s economy, accounting for around 33% of national gross domestic product (GDP) and 60% of total exports, and employing more than 70% of the rural population. Coffee is one of the most important crops in Kenya’s agriculture in both market value and as an export crop9, playing a crucial role in the livelihoods of millions of rural households10. Kenya’s rich soil and temperate climate produces some of the best ‘mild-Arabica’ type coffee in the world. Smallholder farming dominates Kenya’s coffee sector. There are about 570,000 small-scale farmers organized in 421 farmer cooperative societies (FCS) cultivating coffee in 87,437 hectares (equivalent to about 0.2 hectares apiece) and producing 65% of total coffee production. The balance (ca. 35%) is produced by approximately 454 estates (plantations) in 26,067 hectares11. Average smallholder productivity is 300-400 kilos of clean coffee per hectare, while estates can produce up to 1,600 kilos. Differences arise from low use of good agricultural practices, lack of capital and limited access to credit, and little climate-change adaptation capacity12. Kenya’s coffee cooperative system is regulated by the government under the Cooperatives Act, which requires smallholders to be organized in cooperative societies. Coffee farmers must sell their coffee through a registered FCS, who also provides wet milling services. All FCS must use a coffee marketing agent (CMA) to sell coffee, either by offering it at the national weekly auction in Nairobi or through arranged direct sales to interested buyers (“second window”). The certification agreement for smallholders is almost always reached at group level, and the cooperative societies are the bridge between farmers and the market13. The study takes place in Bungoma County, specifically in the Mount Elgon region, one of the poorest in the country. It is located close to the Ugandan border. Bungoma County is reportedly emerging as Kenya’s next “coffee belt”. It has 30 active cooperative societies representing 37,568 smallholders that produce coffee in 6,230 hectares. Their geographic location puts farmers far from the Nairobi Coffee Auction, where most of the county’s coffee production is shipped. Compared to the country’s more established coffee producing areas, Western Kenya has less experience with standards and certification, and faces significant quality and processing challenges14.

9 Ministry of Agriculture, Livestock, Fisheries and Irrigation (2019). http://www.kilimo.go.ke/wp-content/uploads/2019/01/Download-here.pdf 10 Ruben, R and P. Hoebink (2015). Coffee certification in East Africa: impact on farms, families and cooperatives. Wageningen Academic Publishers 11 http://www.agricultureauthority.go.ke/wp-content/uploads/2016/06/area-under-coffee-and-Production-from-2008-20151.pdf 12 Kenya Coffee Platform (2019). Coffee Economic Viability Study. https://www.globalcoffeeplatform.org/assets/files/03-GCP-Tools/Kenya-Coffee-Platform-Coffee-Economic-Viability-Study-Report.pdf 13 Van Rijsbergen, B; Elbers, W.; Ruben, R.; and S. Njuguna (2016). The ambivalent impact of coffee certification on farmer’s welfare: a matched panel approach for cooperatives in Central Kenya. World Development, Vol 77. Pp277-292. 14 COSA (2016). Impacts of certification on organized small coffee farmers in Kenya. Baseline results.

10

The project Solidaridad, a non-governmental organization that engages supply chain actors in innovative solutions to improve production and ensure a sustainable and inclusive economy, partnered with Coffee Marketing Services (CMS)15, a Kenyan coffee marketing agent specialized in sustainably produced coffee, to develop a project seeking to prepare for, and implement certification of, Fairtrade and UTZ coffee standards in two producer organizations in the Mount Elgon region in Western Kenya16. Solidaridad funded the cash outlays for the initiative such as audit fees and required protective gear for farmers who apply agrochemicals. CMS provided training and helped FCS organize their internal control systems. While UTZ Certified is especially known for its focus on productivity and sustainable farm management, Fairtrade is particularly known for its group-level strengthening, and offering a minimum price plus premium price for coffee. CMS expected that the combined certification processes would provide support for quality improvement at both farm and producer organization levels, together with productivity enhancements at the farm level. The Mount Elgon region was chosen as it is one of the most remote areas where most marketing agents would not engage. CMS argued that coffee farmers in this region have low productivity and produce low-quality coffee, which is sometimes downgraded at the wet mill, creating poor demand and attracting lower prices in the market. In addition, CMS identified that in the region there are inadequate linkages to extension services and low value added at each stage of the value chain, implying poor returns to the farmer and their organizations. In order to tackle these critical issues, CMS proposed to assist farmers in (i) improving yields, quality, and access to markets through training; (ii) adopting sustainable coffee production practices to protect the environment; and (iii) attaining Fairtrade and UTZ certification to ascertain traceability, and to improve efficiency and access to markets. CMS expected to reduce the quality and productivity gaps through the certification process (and certification itself). For its part, Fairtrade promotes fairer trading conditions (e.g. price guarantees, price premiums), sustainable markets (e.g. engaging traders, producers and consumers), standards for producer organizations (e.g. business development, governance, environmental protection), and support to smallholders and their organizations, in order to reduce poverty and improve farmer control over their livelihoods. UTZ standards promote biodiversity and natural resource conservation, as well as farmer well-being and farm productivity, with the objective to attain sustainable and resilient rural landscapes (i.e. improved biodiversity and ecosystem conservation, efficient and profitable farming, and improved livelihoods).

Impact pathways of coffee certification There are two distinct avenues through which certification contributes to poverty alleviation (See Figure 1.1). First, there is specific training towards certification at the producer 15 Coffee Management Services Ltd (CMS) is a private company owned by East Africa Coffee Co LLP. It has emerged to be one of the leading coffee marketers, marketing around 23% of total Kenyan coffee. Currently CMS works with approximately 250,000 smallholders in Kenya (Kenya Coffee Platform, 2019). 16 The project was originally conceived for three cooperative societies. However, we found that one of the selected producer organizations had already received training towards certification before the study could begin, ruling out its presence in the target group. Also, another producer organization decided after several implementation steps to pursue other directions away from certification. Therefore, CMS was forced to replace the latter, resulting in the target group being composed of only two producer organizations.

11

organization and farm level can contribute to improved livelihoods through capacity building in different dimensions depending on the particular certification standard. Second, once the entity is certified, there are potential benefits in terms of market access and better prices. Fairtrade’s contribution in capacity typically is to provide support to small producers and their organizations to strengthen the relationship between farmers and their organizations as well as to improve their organization’s bargaining position with traders and processors (Rijsbergen et al, 2016)17. It also provides training to enhance democracy, participation, and transparency, and to ensure adequate labor conditions and environmental protection. In addition, Fairtrade provides risk-reduction incentives offering farmers guaranteed minimum prices, and an additional premium for community-level investments. Expected outputs are enhanced knowledge and capacity among smallholders and stronger, well-managed, democratic organizations for small farmers. Fairtrade’s general theory of change for smallholders expects that the outcome of their intervention will contribute to improved household incomes, assets, and standards of living, as well as less vulnerable business for smallholders with increased food security, amongst others18. The UTZ certification program supports farmers to increase their productivity, the quality of their crops and their income, while also advocating for good agricultural practices that are better for both people and planet. UTZ expects farmers adopting practices and building capacity to achieve biodiversity and natural resource conservation, farmer wellbeing, and farm productivity and profitability (UTZ, 2017)19. Both interventions also set up an enabling environment to develop the market for their standards.

17 Rijsbergen B.; Elbrers, W.; Ruben, R.; and S. Njunga (2016). “The ambivalent impact of coffee certification on farmer’s welfare. A matched panel approach for cooperatives in Central Kenya”. World Development Vol. 77, pp. 277-292. 18https://www.fairtrade.org.uk/~/media/FairtradeUK/What%20is%20Fairtrade/Documents/Fairtrade_Theory_of_Change%20-%202018.pdf 19 https://utz.org/resource-library/?fwp_utz_search_resource_library=theory+of+change

12

Fig 1.1 Chain of causation for attribution analysis

The chain of causation for attribution analysis (Fig 1.1) establishes, as an initial milestone, that certification bodies provide training to an implementing partner so that it can facilitate training to both the cooperative’s board and management, as well as to farmers. As noted in Figure 1.1, training from the implementing partner is expected to stimulate changes at both cooperative and farm levels20. From the cooperative’s perspective, training towards certification may enhance its governance and business management, and will strengthen its relationship with farmers. From the farmer’s perspective, it is expected that training will lead to adoption of a set of practices oriented towards certification and that will contribute to improved performance (e.g. quality, productivity, etc).

When the certification process starts, it is expected that the cooperatives are able to sell a growing amount of their coffee as certified, benefiting from a price floor21 (minimum price) and sometimes a price premium for certified coffee. This will in turn, incentivize farmers to continue improving quality and quantity and to sell a larger portion of their coffee as certified22 thus moving toward the desired impacts.

20 Although both Fairtrade’s and UTZ’s theories of change imply impacts at higher levels (e.g. traders, consumers, systems) 21 Only for Fairtrade 22 As noted by Fairtrade, this only occurs to a certain extent, as some producer organizations may choose deliberately to diversify buyers to reduce dependency on a single buyer, thereby limiting Fairtrade sales to have a broad portfolio of buying channels as a risk-reduction strategy.

13

2. Research approach, design and methods The proposal originally facilitated certification of three farmer cooperative societies in the Mount Elgon region in the Bungoma district. CMS chose these organizations based on their capacity to improve quantity and quality of coffee produced. During our baseline field visit (2015), we found that one of the selected producer organizations had already received training towards certification before the study could begin, ruling out its presence in the target group as it may cause potential response bias. Also, another producer organization decided after several implementation steps to pursue other directions away from certification. Therefore, CMS was forced to replace the latter, resulting in the target group being composed of only two producer organizations. The nature of the intervention focuses on coffee FCS as the entities to become Fairtrade and UTZ certified. In this context, there are not many producer organizations with similar characteristics within the same agro-ecological region to become candidates for certification (note that there are only 30 FCS in the Bungoma district). When the population is too small (i.e. few producer organizations), finding an adequate control group meeting the characteristics of the target group (i.e. counterfactual) becomes challenging. This scenario with a limited number of units of assignment poses a “small n” problem where tests of statistical significance between target and control groups may not be viable. Furthermore, in a certification context, there is risk of self-selection of producer organizations, provided that certification is a voluntary decision and a process based on farmer organizations’ self-interests and expectations. There are also potential selection bias issues arising from the presence of incentives for CMS to select organizations with higher potential to become certified (e.g. better organized farmer groups, higher aggregate production, farmers in better position to obtain certification, etc.). Both potential problems could have been addressed with a larger sample size and random selection of target and control producer organizations, but in this context, it was not possible as there is a limited number of producer organizations in the intervention area. Given the conditions of the intervention, we developed an appropriate research approach that could be useful to address similar situations facing ISEAL and its members in the future. The study follows a mixed-methods approach that combines structured qualitative tools and methods together with strong quantitative techniques suitable for the context of study23 and purposefully chosen to rigorously identify a causal chain consistent with the intervention’s theory of change. The mixed-methods approach we have designed for assessing the impacts of certification takes selection bias into consideration and combines quantitative and qualitative tools that enhance the individual strength of each methodology. Together, these factors allow for a better understanding of the causal chains and impacts of an intervention.

The quantitative approach The quantitative methodology leverages farmer-level panel data for comparing outcomes of interest amongst the target group relative to a control group (counterfactual) before and after the intervention (differences in differences)24. The graph below illustrates the evaluation process. The vertical axis (Y) represents the outcome of interest, while the horizontal axis represents time (X). The yield evolution of the target group is represented

23 The International Initiative for Impact Evaluation (3ie) collaborated on the development of the research plan and its execution and concurs that mixed methods are needed for this context. 24 While this approach allows us to rule out time-invariant factors affecting outcomes, it has the strong assumption that there is a common trend between target and control farmer’s groups.

14

by line T (T0, T1), while for the control group it is represented by line C (C0, C1). The outcome of interest is measured for both target and control group at time 0 (baseline) T0 and C0. The target group then receives the intervention and the outcome of interest is measured again for both groups after the intervention at time 1 (end line), represented by the points T1 and C1. Not all of the difference between the target and control groups at time 1 (T1 - C1) can be explained as being the effect of the intervention, given that a difference already existed between the target and control group at time 0. This is an unavoidable problem of selection bias. If the target group did not receive the intervention, the path of the outcome of interest would follow the dotted line Q, which is parallel to the line C. The differences-in-differences approach overcomes selection bias by differentiating out the original bias, generating an unbiased estimator of the impacts of the intervention: (1)

𝐷𝐼𝐷 = (𝑇& − 𝑇() − (𝐶& − 𝐶() = (𝑇& − 𝐶&) − (𝑇( − 𝐶() Note that the key assumption is that line Q represents what would have happened to the target group if it did not receive the intervention, which is parallel to line C. This is called the “common trend” assumption, which means that the differences between target and control groups without intervention is constant over time.

Fig 2.1 Differences in differences

The counterfactual selection of controls Given that the unit of analysis is necessarily the producer organization, control groups needed to be constructed at the same level. In a context where there is a small set of uncertified producer organizations in the same agro-ecological region, this becomes a challenge, as these organizations need to be as similar as possible to the target group in order to serve as a valid counterfactual. The scenario of a limited number of units of assignment poses a “small n” problem where tests of statistical significance between target and control groups may not be possible. This is the key reason why we will also use the quantitative analysis as a key element to guide our contribution analysis – in addition to its

Y

T1

QT0

C1C0

Time 0 Time 1 X

15

intrinsic explanatory value. As pointed out by White (2010)25: “Although it may not always be necessary or useful to make the counterfactual explicit, in attributing changes to outcomes to a development intervention it most likely is useful to have an explicit counterfactual”. By improving our quantitative assessment, we will improve the accuracy and robustness of the contribution analysis. Control groups for this study were selected using secondary information and key informant interviews26 as well as producer organization surveys conducted at baseline ensuring they are the closest possible matches to the target group. Originally, the control group was built using two sets of producer organizations that did not have plans to become certified in the near future. The first group was formed by two producer organizations that worked with the same marketing agent as the target organization; allowing the identification of the effects of training towards certification and certification itself, and selecting out the likely effects and influence of the implementing partner. For the second group, we also chose two producer organizations with no interaction with the target group’s marketing agent. This second group allows capturing the total value of certification beyond the effects of the implementing partner. Sampling We randomly selected 120 farmers from each of the six, selected FCS. This sample strategy is expected to have around 70 percent power for detecting a 50% increase in yield. We used yield (kg/ha) as a benchmark because it is a key indicator that captures the outputs of investments in fertilizer, biocide, labor, and farming practices. In addition, it is a continuous variable that often exhibits high variance in the experience of COSA and many other researchers working with smallholder coffee production. We used a baseline yield of 700 kg/ha with a standard deviation of 650 kg/ha calculated from a data set from Mount Elgon coffee farmers. The estimated 50 percent yield increase comes from the Coffee Research Institute (CRI), which provides some farmer training for CMS. CRI says farmers following the regimen detailed in its training experience greater than 50% increases within three years on average. The calculation for sample size allows for a 10 percent attrition rate to give a buffer for farmers who cannot be included in the end line. Analytical approach Using data collected in the baseline (2014/2015) and endline (2017/2018), the difference-in-difference model can be applied. Following Angrist and Pischke (2009)27, we estimate a more complex version of the difference-in-difference model showed in (1), while controlling for producer organization, and household characteristics, as follows:

(2) 𝑦-./ = 𝛼( + 𝛼&𝐷.2 + 𝛼3𝑇/ + 𝛼45𝑇/𝐷.26 + 𝛿𝑍./ + 𝜃𝑋-./ + 𝜀-./

where: 𝑦-./ is the outcome variable for farmer 𝑖, in producer organization 𝑔 at time 𝑡

𝐷.2 = 1 represents the target producer organization, and =0 the control group𝑘

25 White, H. (2010) “A contribution to current debates in impact evaluation”. Evaluation 2010. 16:153 26 Knowledgeable informants from the Coffee Research Institute and the Ministry of Agriculture, Livestock and Fisheries, helped us narrow down the list of candidates. 27 Angrist, J. D. & Pischke, J.S. (2009) “Mostly harmless econometrics: An empiricist’s companion” Princeton: Princeton University Press.

16

𝑍./ is a vector of producer organization level time-varying covariates

𝑋-./ is a vector of household level time-varying covariates

Based on the above formulae, we can estimate the impact of the intervention on the outcome of interest, assuming the “common trend” assumption holds. Equation (2) partially corrects the potential bias generated by observable time-varying variables at the producer organization and household level affecting the outcomes of interest. Furthermore, in order to assess the heterogeneous effects at the farm level, we incorporate an interacted term Ω-./𝑇/𝐷.2, that captures the specific effect of Ω-./ (i.e. gender, age, poverty, etc.).

(3) 𝑦-./ = 𝛼( + 𝛼&𝐷.2 + 𝛼3𝑇/ + 𝛼45𝑇/𝐷.26 + 𝛼A(Ω-./𝑇/𝐷.2) + 𝛿𝑍./ + 𝜃𝑋-./ + 𝜀-./

Finally, in order to improve the likelihood of a “common trend”, we matched households for pre-intervention characteristics, producing a sample with similar trends in both target and control groups. Matching was constructed for the target group with each control separately, and with a pooled set of controls.

The qualitative approach The qualitative approach followed a disciplined contribution analysis (Mayne 2001) framework aiming to gain a deeper understanding of the reasons for the observed outcomes and the extent to which results are “attributable” to certification efforts. This methodology relies on the notion that the intervention’s theory of change can be used to infer causation by assessing whether the intervention mechanisms have in fact occurred following the logical causal chain (White, 2012). Contribution analysis seeks to demonstrate a plausible association between an intervention and the observed outcomes, developing a logical causal change between the intervention’s inputs and outcomes. Our approach to contribution analysis in the context of a mixed-methods approach follows these steps:

1. Identify the theory of change 2. Identify the activities of key actors of the intervention’s theory of change and

confirm that those activities have occurred. 3. Gather quantitative evidence of changes in outcomes by comparing

performance of target and control groups 4. Gather qualitative data from informed participants and stakeholders to assess

whether the interventions activities could have plausibly caused the observed changes identified in the quantitative piece.

5. Explore the contextual factors, to assess the degree to which they have contributed to the observed outcomes.

While standard contribution analysis uses quantitative data to provide evidence on changes, our approach benefits from solid quantitative data and sound econometric methods to minimize selection bias and power issues in the estimation such that the evidence of changes is optimal given the conditions of a typical certification project.

17

The project’s evolution and research plan adaptation At the end of the baseline stage (2015), full implementation of the intervention was still being planned. Prior to the endline a scoping trip (2018) was undertaken to understand the fidelity of the intervention, resulting in the discovery that the project has not gone as planned and the intervention did not fully accomplish the objectives of certifying target cooperatives in both Fairtrade and UTZ standards. While both target farmer organizations received training towards certification, only one managed to become certified, and only with Fairtrade. Unfortunately, after one year of being Fairtrade certified, the producer organization decided to switch marketing agents and to stop paying certification fees, thus relinquishing the certification. However, even though the intervention was not complete, the study would aim to assess the changes caused by one year of being certified since it was determined that enough had occurred to potentially provide substantial measurable benefits to farmers (training and one year of being Fairtrade certified) – even if a longer-term effect could not be assessed. Changed field conditions, together with a tight timeline, required COSA and ISEAL to agree on an adjusted design. The new design focused the quantitative assessment on both target producer organizations and only one control organization that was not affected by CMS. The qualitative component remained the same as in the original design, but incorporating additional elements to deepen our understanding of the clearly important role of the market structure and dynamics, as well as the decision-making processes of key stakeholders in the value chain around certification. Furthermore, at early stages of the endline fieldwork, our efforts to seek cooperation from one target cooperative society for endline research were unsuccessful. Once we exhausted all reasonable efforts with this group, COSA and ISEAL jointly decided to replace it with one of the control groups that was working with CMS but which was not then slated for certification. The final design for the quantitative methods thus focuses on a target producer organization that received training for Fairtrade and UTZ certification, and which was Fairtrade certified for one year. It was compared with two separate control organizations: one sharing the same marketing agent but not slated for certification, and the other one with a different marketing agent and without any certification efforts or plans in the near future. The mixed-methods approach we designed for assessing the impacts of certification in this new scenario follows the following chronological steps from the baseline until the end of the process (Table 2.1).

18

Table 2.1 Chronological steps for the mixed-methods approach for impact assessment

Study element Step Tools Purposes

Baseline

Setting the grounds

1 Secondary data sources and key informant interviews

Identify FCS candidates for control groups

2 Interviews to key actors (FCS, CMS)

Identify activities relative to the theory of change

3 Participatory Rural Appraisal

Understand perception of FCS, identify characteristics of marginalized farmers; communicate about the household survey

Baseline data (quantitative)

1 Farm-household survey Determine initial conditions of target and control groups

2 FCS survey Gather baseline indicators

Insights (qualitative)

1 Structured key informant interviews

Validate the causal chain as determined by Fairtrade and UTZ standard systems

2 Focus groups Provide insights into the reasons for differential performance

Endline

Measuring change (quantitative)

1 Farm-household survey Measure changes between baseline and endline

2 FCS survey Measure changes between baseline and endline

Establishing attribution (qualitative)

1 Structured key informant interviews

Establish contribution by Fairtrade and UTZ standard systems as well as by contextual factors that may have affected outcomes

2 Focus groups

Determine if changes in performance for certified groups could be attributed to standard systems

Provide insights on the contribution of standard system adoption to differential performance by different types of farmers

19

3. Market structure and group dynamics

Market structure The Kenyan coffee sector’s structure and performance is influenced by the value-added activities developed across a value chain that is particularly characterized by significant regulation as well as fragmentation at the grower level and large concentration in service and marketing activities (Condliffe et al, 2008)28.

Fig 3.1 Kenya’s coffee value chain

Source: COSA Research

As mentioned before, coffee production in Kenya is conducted at two different levels. On the smallholder level, there are around 570,000 smallholders (less than 2 hectares) organized into 421 cooperatives, which account for around 77% of total coffee area (113,500 hectares) and 65% of total production (42,037 tons of coffee)29. On the estate level, around 454 estates produce ca. 44% of total coffee, producing in 23% of the total coffee area. As per the Coffee Act (2001)30, smallholders are required to sell their coffee through their cooperative society. The cooperative collects fresh cherries from farmers and wet-processes the coffee in their factories, where it is pulped, fermented, skin-dried and conditioned. This gives the cooperative substantial responsibility over the quality of coffee. In many cases, cooperative societies lack updated or adequate infrastructure for wet processing, and they spend large portions of their budget in repair and maintenance of their equipment. FCS are allowed to retain up to 20 percent of total coffee revenues from selling their members’ coffee to cover their organizational expenses and to pay for the wet-mill processing. Smallholder coffee farmers face limited choices for processing and selling their coffee. Considering that cherries must be processed within a day after harvest, and that transportation is costly (no easy access), it essentially forces farmers to work with the closest cooperative and not necessarily the best cooperative of their preference. This puts little pressure on cooperative leaders to control costs and improve efficiency (Monroy et al, 2013)31. In general, smallholders’ participation in the coffee value chain is limited to

28 Condliffe, K.; Kebuchi, W.; Love, R.; and R. Ruparell (2008). Kenya Coffee: A Cluster Analysis. Harvard Business School 29 http://www.agricultureauthority.go.ke/wp-content/uploads/2016/06/area-under-coffee-and-Production-from-2008-20151.pdf 30 The Coffee Act was passed in 2001 as a mechanism to liberalize the coffee industry and restructure the role of the Coffee Board of Kenya to regulating the industry https://infotradekenya.go.ke/media/Coffee%20Act%20Chapter%20333.pdf 31 Monroy, L.; Mulinge, W.; and M. Witwer (2013). Analysis of incentives and disincentives for coffee in Kenya. Technical Note Series, MAFAP, FAO, Rome.

SmallholdersFarmer

CooperativeSociety (FCS)

MillsCoffee

MarketingAgent

Auction Dealer

Medium and Large size estates

Buyer

Direct Sales

20

delivering their coffee to the FCS, thus there are no clear feedback mechanisms to assist farmers’ access to relevant market information. Both FCS and estates choose a commercial mill, and transport the parchment (coffee after wet-milling) to the dry-mill facilities, where the wet-processed berries will be hulled, polished and graded. Millers process coffee into seven official grades mainly based on bean size and density. For the finest coffee, the origin of the beans (location and altitude) is also very important. The overall process between the mill and the cooperative can take up to three months (Monroy et al, 2013). When milling is complete, the bagged and classified coffee is delivered to a Nairobi warehouse adjacent to the auction house, through a Coffee Marketing Agent. Producers (FCS and estates) choose a coffee marketing agent (CMA) for marketing their coffee either at the national weekly auction market in Nairobi or by arranging direct sales to interested buyers (“direct sales”). Marketing agents play a central role in the Kenyan market. In addition to creating and facilitating transactions, they also provide a range of services to cooperative societies and their farmer members, either directly or through affiliates and allied businesses. These include training, cash advances, transporting coffee to mills for dry processing, sorting, cataloguing, and providing or securing warehouse services. Marketing agents also prepare and make available coffee samples for licensed buyers prior to auction, represent cooperative societies during the auction, and finally collect and distribute proceeds after the sale (Chege, 2012)32. Once the coffee is sold by the marketing agent, the proceedings are distributed to the marketer (3.6%), the miller (1.8%) and the cooperative society (94.6%). The FCS then takes 6.1% for covering its administration costs, and 12.8% to pay for the wet-milling process, leaving the farmer with about 75.7% of the value of coffee sold. FCS collection centers do not assess or grade the coffee quality of individual farmers. The center rejects defective coffee and wet-processes all the coffee together, simply registering the weight delivered by each farmer. Millers do assess the quality of the coffee received, which is sometimes cross-checked by the FCS with other mills or institutions. As noted, the quality assessment is performed for the cooperative’s aggregate production. When the coffee proceeds reach the cooperative, it distributes the total amount (discounting administrative costs and wet-mill processing) proportional to the total coffee delivered by each farmer. So, the individual farm gate price is only known well after the sale and is a function of the overall quality of the FCS’s deliveries and the market price paid, discounted by the fees for services. This process ignores individual incentives for coffee quality, thus limiting incentives to invest in quality improvement at the farm level that may not be adequately compensated with higher prices. Marketing agents also set reserve prices for each lot of coffee and ensure its proper storage, including that coffee warehouse receipts are in order, as the law requires that tendered coffee be properly stored and registered with the Coffee Directorate until auction. Such receipts essentially represent legal tender against the physical asset making the management of this process a critical role. Until 2006, all coffee sales were required to go through the Nairobi Coffee Exchange (NCE) where it was purchased by licensed coffee dealers through a competitive bidding system (Monroy et al, 2013). Since then, a direct sales mechanism, called a second window, has been allowed. Direct sales require that a marketing agent negotiate directly with a buyer outside the country and register a signed sales contract with the Coffee Directorate33 (former Coffee Board of Kenya), the statutory body that regulates the coffee industry. While these transactions can be more profitable and are highly desired by producer organizations, 32 J. Chege (2012). “Value addition in coffee industry in Kenya: Lessons from cut flower sector”. Kenya Institute for Public Policy Research and Analysis (KIPPRA). ICBE-RF Research Report No. 21/12. 33 It is under the Agriculture Fisheries and Food Authority (AFFA).

21

they are a less common. Approximately 85% of total coffee sales, on average, go through the auction. Kenya’s centralized market system, with a central auction and the noted legal restrictions on market actors (who can buy and sell), generates its own somewhat unique market dynamics. It functions fairly well even though it is also the subject of some criticism. The controls it imposes play a positive role contributing to an overall level of quality that is important to Kenya’s global reputation and price, and provides a level of disciplined controls on transactions that reduce fraud and improve transparent market price and quality discovery functions. But the system also limits the diversity of market actors and can reduce incentives at the farm level. By dampening the range of available options in the market, there is a strong reliance on a limited group of marketing agents and mills. In terms of standards and certifications, these must rely substantially on the same market actors to help introduce and, to some extent, coordinate their programs.

Group dynamics Small farmers rely on their cooperative societies. These legally registered entities are governed by board members and managed by a Chief Executive Officer (CEO) who is appointed by the board. Farmers are expected to participate in annual general assemblies and vote on important decisions for their organizations. Board members are elected for a 3-year period, with the possibility to be re-elected. The structure appears functional and somewhat controlled. However, key informants consider that many FCS in the region face serious challenges. Reports typically note poor levels of actual governance, limited management capacity, lack of overall transparency, and little or no information or training to their farmer members (unless externally funded). Farmers typically complain about lack of transparency around price levels and uncertainty about payments that are often delayed (farmers can sometimes receive payments six months after delivering their coffee to the FCS). Farmers have the power, bestowed upon them by the Cooperatives Act, to make changes in their organizations. In theory, they can remove the board as long as there are cogent reasons for that, but this power is not often exercised. With farmers kept uninformed, it is difficult to demonstrate bad faith or convincing reasons for dismissing a board of directors. Producer organizations enter into annual contracts with marketing agents and so may change marketing agents frequently. The contracting, although allowing eventual change, can keep only one market option open at a time. Selection of marketing agents is essentially based on a combination of services that are offered (e.g. cash advances, affordable inputs, and training). In practice, the marketing contracts and associated services can be quite lucrative and so agents do lobby board members to be selected. Farmers are also likely to exert pressure on the board to change marketing agents, typically fueled by promises of better prices and services, even if these are not often met. Board members of all the producer organizations surveyed, have considered certification to be a key and positive issue that could influence their decision of which marketing agent to use. Both farmers and board members have high expectations from certification: as a perceived opportunity for higher prices, access to direct export sales (second window), and access to training and projects or funds to improve their infrastructure and the capacity of the producer organization. However, some stakeholders noted that certification is sometimes used as “bait” to lure new cooperatives to an agent because there is only a rather limited market for certified Kenyan coffee. In response to an open question about the benefits perceived about certification, farmers responded that price, market access and funding for projects (e.g. wet mill improvements), were valuable potential benefits. Other potential benefits such as quality improvement, productivity improvement, improved record keeping systems, or improved governance, that

22

may have been evident if the certification had continued beyond its brief period, were not recognized by these group of farmers. Marketing agents are highly regarded by producer organizations as potential partners. Nevertheless, they are not immune to constant change and competition, as all the producer organizations we assessed have recently changed their marketing agents. Board members and farmers consider that marketing agents are often not fully transparent. In the majority of focus groups, respondents considered that marketing agents tend to take advantage of farmer groups. Marketing agents reportedly consider many producer organizations to operate unprofessionally and to be unreliable to work with and yet rely on them for most of their business. So, there is a dynamic tension between the two. Although the market structure is designed to keep upstream and downstream entities legally independent and not in collusion, many marketing agents are said to be controlled by or associated to mills with, for example, the same or related owners. Therefore, producer organizations are unlikely to have much influence on which mills are selected to work with. This of course can dampen competition and the price farmers pay for services, and one of the ways that producer organizations and farmers claim these manifest negatively, is the concern that the mills report the quality of coffee delivered as lower or downgraded from what organizations claim they test, including in cupping facilities. If true, this also undermines many long-term efforts, such as cupping labs and Q-grader trainings, to provide skills to organizations and farmers so they could take control of their own destiny, knowing their actual quality and thus theoretically at least strengthening their negotiating position.

The certification process This research study was designed to assess the certification impacts on two producer organizations and their farmer members. One producer organization received training towards certification through CMS (2016) but later decided to switch marketing agents and stop the certification process. The other producer organization received training towards Fairtrade and UTZ certification, and achieved Fairtrade certification (June 2016). Unfortunately, after one year of becoming certified, the producer organization decided to stop working with CMS, and relinquished the certification by stopping payment of Fairtrade fees. In both cases, the lack of market incentives for certification contributed to their choices. It is important to note that while the marketing agent does not offer certification itself, it is the one who sells the coffee in the market and generates expectations for both cooperative societies and farmers about the possibility of selling coffee as certified and possibly receiving a price premium. If the market for certification is limited and the marketing agent is not able to sell the coffee as certified it will generate distrust from the cooperative and its members with the marketer. This will result, as happened in our case, in the cooperative deciding to switch marketing agents and stop paying the certification fees, even though they still consider certification a good opportunity. Farmers and board members were nevertheless highly satisfied with the outcomes of the training they received. When they became certified, they had high expectations of selling their coffee as Fairtrade. The producer organization claims that the coffee was blended with others in the mill, and therefore could not be sold as Fairtrade. This prevented the organization from obtaining a Fairtrade price or premium for their coffee. Board members considered this unexpected outcome to include a lack of transparency from the marketing agent and felt it to constitute “not honoring his word” as further cause to switch marketers. They were conscious that they would lose the current training that they valued and may potentially not be able to have another agent help them achieve certification or market their coffee that way. Farmers and board members alike supported this decision.

23

From the marketing agent’s perspective, the market for Kenyan Fairtrade certified coffee is limited and has been shrinking over time. Given that there was no market for their product, they decided to sell it as conventional, looking for the best price. In summary, within such a market where demand for certification is quite limited, what we observed is substantially a problem of expectations and transparency. On the one hand, there are several information gaps between farmers and board members. So, they act based on expectations and limited understanding of market dynamics. On the other hand, there is insufficient transparency down the value chain as Board members claim the agent does not inform them and they have little knowledge of the actual market demand and prices for certified products. Improving market information, such as prices and volumes transacted for certified coffees, would be beneficial to all. Many internal transactions may not lend themselves to more transparency, but certainly the greater availability of official transaction data from auctions and exports would help – cooperatives find it difficult to access that data. Knowing how much certified coffee is transacted and at what price would temper expectations. Farmers would also benefit considerably from a simple control: to improve the quality arbitration process. This would enhance trust, help ensure fairness in the market, and even incentivize quality efforts; these can, in combination, offer potentially valuable income benefits to farmers and their organizations. The expectations and inaccurate information circulating in Kenya about various aspects of certification is detrimental to all. It may not be feasible for the standards bodies to increase their presence in the country to alleviate this information vacuum. They might however be able to nominate or support local affiliate organizations that can foster conversations with government and industry (e.g. Kenya Coffee Platform), local media, and exchanges with farmers and cooperatives and help ensure that the domestic information about certifications, their processes, market conditions, and farmer-level engagements serves everyone better. Certification processes where marketing agents act as the key channel of transmission for training farmers and cooperative representatives, faces several challenges to ensure long-term continuity. Fairtrade is aware of this and seeks always to establish some direct relationship as well (e.g. Fairtrade Africa field staff and information from FLO-CERT). First, the relationship between marketing agents and cooperatives is not often long-term, thus interrupting training processes. There is strong competition between marketing agents and contracts typically last one year. This provides incentives to marketing agents to persuade farmers and board members by offering a set of services and better prices to get the business. The unstable relationship between cooperatives and marketing agents may limit the ability of standard schemes to engage with continuity in the market. Second, institutional weakness and limited information about certification among producer organizations diminishes their ability to fully value and even realize the benefits of certification. Third, a limited market for Kenyan Fairtrade certified coffee significantly affects interest. Considering that producers typically prioritize better prices (even though there are other benefits such as training and capacity building), not having a premium or even a preferred market access thus reduces the appeal. Lastly, while most key informants acknowledged certification as a key issue that can be implemented through a marketing agent to improve quality and productivity, they concurred that it may not be the ideal vehicle as farmers and producer organizations tend to mainly consider short-term prices and services as the key drivers for decision-making.

24

4. The study

Fieldwork As per the final research plan for the endline, the study group for the quantitative assessment was composed of one of the target producer organizations and two controls. Within each producer organization, at the baseline, we randomly selected 120 farmers. This sample size allowed for a 90% power for detecting a 50 percent increase in yields34 within each organization, and allows for a 10% attrition rate to provide a buffer for farmers who cannot be included in the endline. The follow-up quantitative data collection took place between September 24th and October 17th, 2018. We targeted a total of 355 farmers in the three selected organizations; however, we only managed to gather data on a total of 321 farmers, implying a total attrition rate of 10%35. The main reasons for attrition were:

• No availability of a household member knowledgeable about coffee farming (38% due to non-availability of farmer even after several visits)

• Relocated farmers (26%)

• Sensitive health conditions (21%)

• Others (15%)

Table 4.1 Final household survey sample and attrition rates

(†) NA: Not available; R: Relocated; H: Health conditions; O: Others

The qualitative data collection was developed between December and January. We performed group interviews with board members to better understand their decision-making processes around marketing agent selection and board membership, the context and the market on certification, and perceptions around the costs of benefits of certification. Further, we conducted a specific survey around membership and organization services offered to farmers. Finally, we developed focus groups with farmers within each organization to capture their perceptions on their role in the value chain, perceptions around certification 34 Yield was used as a benchmark as it is considered a key indicator that captures the outputs of investments and farming practices. We used an expected value of 700 kg/ha with a standard deviation of 650 kg/ha, using data from the Coffee Research Institute for the Mount Elgon region. 35 Attrition was random and contemplated in the sampling strategy.

Producer Organization

Total sample

Final sample

Attrition rate

NA R H O

Target 119 103 13% 6 4 5 1

CMS Control 120 113 6% 3 2 0 2

non-CMS Control 119 105 12% 6 3 2 3

Total 358 321 10% 15 9 7 6

Attrition reasons†

25

as well as to assess potential confounding factors, selection bias and testing the statistical validity to provide additional evidence for quantitative results.

Table 4.2 Qualitative tools in producer organizations

(†) Only one focus group per Producer Organization. We had 10 participants in each focus group.

Furthermore, we developed structured interviews for a set of key informants to better understand the context in which the project evolved, with special attention to the role of markets and the value of certification36.

Table 4.3 Key informants

36 Unfortunately, we have not yet reached all the actors we wanted to interview due to their tight agendas and the time of the year we planned to execute the interviews (December – January). However, we managed to partially complete this task, and we expect no significant changes on our study as all interviewees converge in their general assessment.

Producer Organization

Board group interview Survey Focus

groups†

Target X X X

Target - B X X

CMS Control X X X

CMS Control - B X X X

non-CMS Control X X X

non-CMS Control B X X X

Institution

Coffee Directorate of the Ministry of Agriculture, Fisheries and Food Authority

Global Coffee Platform

Fairtrade Africa

Neumann Gruppe

Kenya Coffee Traders Association (KCTA)

26

Characterization of the sample Farmers in the sample are older (around 56 years old on average) with more than 25 years of experience in coffee farming. Education levels are very poor; while on average farmers have more than eight years of education, only 71% have completed primary schooling and 37% have completed secondary school. Around 71% of households have men as the main decision makers for coffee farming. Households have on average seven members, with around three members sustaining the households’ economy.

Farmers in the target group are younger (and less experienced) than farmers in Control-I; and on average, less educated than farmers in both control groups.

Table 4.4 Household characteristics

Sample farmers are smallholders, with a bit above one hectare of total land, out of which around 33% is planted with coffee. Just over half or 52% is usually used for other crops (mainly maize, beans, and bananas), 5% in pastures, and the remaining 10% is distributed between natural forest, fallow area, and other areas. Farmers are located in remote areas, a few hours away from the capital of Bungoma, and approximately two kilometers away from the nearest commercial center, which offers basic goods to purchase. Farmers are located in hilly areas located on average at 1625 m.a.s.l. Higher altitude farmers belong essentially to the target organization, where around 31% of farmers live above 1700 m.a.s.l. Around 81% of farmers deliver their coffee directly to the producer organization, which may be in some cases a few hours away from their farms. While the target group displayed some statistically significant differences with the control group in basic household and farm characteristics, such differences in absolute terms are not significant with the exception of the altitude where a large portion of target farmers are located.

Household characteristics Target

Farmer age 54.1 58.9 ** 54.8 56.9 *

Farmer years of experience 24.4 28.1 * 25.8 27.0

Farmer school grades completed 7.5 8.9 *** 8.6 ** 8.8 ***

Male principal decision maker (percentage) 65% 76% * 70% 73%

Number of household members 6.6 6.7 7.0 6.8

Dependency ratio (percentage) 40% 40% 41% 40%

CMS Control non-CMS Control

Control Group

27

Table 4.5 Farm characteristics

Farm characteristics Target

Farm size (hectares) 1.2 1.3 1.0 1.2

Coffee area (hectares) 0.3 0.3 0.3 0.3

Altitude (masl) 1,697 1,583 *** 1,604 *** 1,593 ***

Distance to nearest commercial center (km) 2.5 2.6 1.6 *** 2.1 *

CMS Control non-CMS Control

Control Group

28

5. The intervention

Training towards certification at the farm and producer organization levels is a key link in the project’s causal chain, as it is an important channel for delivering information to farmers and their organizations about what the standards are and what and how to be able to meet those standards. Trainings at the producer organization level occurred in both target and control groups. Training was provided by the Ministry of Agriculture, the Coffee Research Foundation and the Cooperative Bank. While we cannot differentiate the trainings received, we can ascertain that training topics were fairly similar, and highly valued by board members. Farm-level training has not been exclusively offered for target group farmers. Control group farmers received training from their marketing agents (i.e. CMS, Sasini), the Coffee Research Institute, and the government (i.e. Ministry of Agriculture, Ministry of Cooperatives, amongst others). Training towards Fairtrade and UTZ certification offers some relevant and distinct topics beyond those that are typically offered in local training efforts to the control groups (e.g. environmental practices and organizational or governance skills) but most of the topics are similar. From the evaluation perspective, the counterfactual has been severely affected by the presence of other training initiatives offering similar topics to the control groups (e.g. agronomic practices), limiting the validity of the comparison group. While training has been observed in target and control producer organizations, it did not reach the majority of farmers in either case. From our field assessment, we identified some key challenges for training delivery and training attendance, which lessen the ability of training to reach a wider audience. The limited number of farmers trained in the target group reduces the ability to effectively measure changes at the group level. Producer organizations actively seek training, but cost limits their access. All producer organizations in the sample group declared they paid for training their board members and officers37. However, due to limited budgets, not all members or key management personnel were trained. Training is highly valued and is regarded as critical to improve the producer organization’s performance. Key topics typically delivered in these trainings are governance, leadership, accountability, and transparency. While we cannot distinguish whether training at the producer organization level was different for target and control producer organizations, we can ascertain that topics delivered were fairly similar. Training is not exclusive for target group farmers. In the baseline report, we highlighted that both target and control group farmers received training; and that the target group had a significantly lower percentage of representatives attending training. The follow-up survey captured training received during the evaluation period (2015-2017), showing an important increase in the percentage of farmers trained in all producer organizations (target and control), while maintaining about the same statistically significant differences between target and control organizations as in the baseline. The latter implies that changes in the percentage of farmers trained is similar in both target and control groups. The average number of hours attended per representative has not significantly changed over time either, keeping differences between target and control groups similar as during the baseline. Total average number of hours attended per year is around 12, exhibiting large variability between and within producer organizations.

37 Training for producer organizations’ board members and officers was mainly offered by some public institutions (e.g. State Department for Cooperatives, Kenya Agricultural and Livestock Research Organization)

29

Table 5.1 Percentage of farmers trained

Note: BL refers to “BASELINE”, while EL refers to “ENDLINE”

Training did not reach all farmers. Approximately only 30% of farmers received training in preparation for Fairtrade and UTZ certification. Similarly, training in control groups was not widely disseminated amongst farmers. In order to better understand the intervention, we ask farmers and board members about the training received, and use this information to develop a simple model to assess the probability of being trained. We observed that extension services in the region face delivery problems and are exposed to cultural barriers that prevent it from reaching a wider audience, indirectly biasing training. For instance, training in the region is typically delivered to groups of farmers through farmer field schools and demonstration parcels. Due to budget restrictions, trainings are limited to a few locations within the area of influence of the producer organization, making it too costly for farmers far from locations or for the producer organization to transport farmers to the training location. In fact, we observe that distance and altitude play a significant and negative effect on the probability of being trained. The model below shows that farmers who complete secondary schooling are 11% more likely to be trained, implying a strong association between education (i.e. ability to speak Swahili and to understand training materials) and receiving training. In fact, some farmers mentioned that low education plays an important role in determining participation, as trainings are typically delivered in Swahili, and some farmers just speak their local languages. There is also a claim that materials are not easy to understand, generating disappointment and reluctance to attend trainings. Furthermore, there is a claim that farmers (mostly older farmers) are culturally reticent to change especially if their coffee farming practices are inherited. The latter is also supported by the fact that older farmers are less likely to participate in training. We observe that farmers whose income significantly relies on coffee are 15% more likely to participate in training sessions, implying that demand also arises from need. Farmers relying mostly on coffee as their main source of income are more interested and prone to change than others who did not exclusively depend on coffee. Training also depends positively on the number of household members, suggesting that time to attend training sessions is also a relevant aspect (i.e. more members, more probability to attend trainings). We also found that women are more likely to participate in training sessions, and that gender is a relevant determinant of training. We also note that there is not a significant effect of wealth38, and that there are no significant differences between target and control groups.

38 Using factor analysis, we developed an asset index, which comprises human capitals (years of education) and physical capitals (livestock (TLU), coffee area, number of rooms in their dwelling, access to electricity, number of frying pans available at home (taken from the PPI), whether the household has radio, television, bicycle and motorbike; as well as the number of agricultural tools). In the regression analysis we used the quintiles of this asset index.

BL EL BL EL BL EL BL EL

Training 24% 30% 33% 39% 39% 50% 36% 44%

Training hours 15.6 13.5 7.4 9.8 13.6 15.5 10.6 12.9

Control GroupTarget CMS Control non-CMS Control

30

Table 5.2 Determinants of training (probit Training = 1)

Note: Pooled model using baseline and endline data

Training towards certification offers some strong commonalities with regular training offered in the region, but also exhibits some differences. Farm-level training towards Fairtrade and UTZ certification includes some specifics in order to achieve certification, such as environmental protection practices, in which the target farmers received significantly more training than their peers in the control groups. However, training in the target group also shares some key topics with trainings in control groups (e.g. coffee farming operations, record keeping, health and safety), where we do not observe statistically significant difference amongst target and control groups. Similarities in topics such as coffee farming operations (i.e. good agricultural practices in coffee), offer common opportunities to target and control farmers to improve coffee quality and productivity, limiting the ability of the control groups to act as adequate counterfactuals to assess changes in economic conditions (e.g. productivity, coffee revenue, coffee production costs, amongst others). A closer look at coffee farming operations, allowed us to identify the specific topics developed in such training sessions. In general, 89% of trained farmers declared being trained in pruning, 79% in application of chemical fertilizers, 70% in soil fertility management (70%); and 67% in good harvesting practices, with no statistically significant differences between target and control groups. However, target farmers do receive significantly more

-0.003 **

0.113 ***

-0.079 *

0.027 ***

-0.001 **

-0.025 **

0.005

0.153 *

0.254 ***

-0.016

0.081 **

Number of observations 603

Pseudo-R2 0.1303

Marginal Effects

Distance to nearest commercial center (km)

Farmer age

Farmer complete secondary school (percentage)

Male principal decision maker (percentage)

Number of household members

Altitude (masl)

Variables

Asset index (quintiles)

Target group (dummy, if target = 1, otherwise = 0)

Income from coffee greater than 70% (percentage)

Frequency of participation in the producer organization

Year (dummy, if endline = 1, otherwise = 0)

31

training in soil conservation practices and in natural fertilizers use and application, consistent with the interventions’ plan, and less on integrated pest management.

Table 5.3 Percentage of farmers trained by topic (2015-2017) (as a percentage of trained farmers)

Training topics Target

Coffee farming operations 89% 71% * 94% 84%

Record keeping 11% 14% 10% 12%

Marketing 25% 14% 10% * 12% *

Health and safety 7% 5% 6% * 5%

Environmental protection 32% 19% 10% 14% **

Farm business and financial management 7% 17% 0% * 8%

Gender 4% 2% 0% 1%

Other 4% 10% 4% 7%

Control Group

CMS Control

non-CMS Control

32

Fig. 5.1 Percentage of farmers trained by topic (2015-2017)

Training is highly valued by farmers and their organizations. On average, 73% of farmers who received training consider that their farms significantly benefited from training received; while only 2.5% declared they did not benefit at all from trainings. Both farmers and their board members, argue that trained farmers are adopting mainly practices that do not require heavy capital outlay (e.g. pruning, application of organic manure, sucker removal, amongst others).

0%10%20%30%40%50%60%70%80%90%

100%Pruning

SoilFertility

SoilConservation

Plantation

GoodHrvstPrac

WaterContaminationChemicalFertilization

NaturalFertilization

IPM

HandlingAgrochem

Biodiversity

Target CMS Control non-CMS Control

33

6. Key Findings In this section, we present the impacts of the intervention combining our quantitative methods and qualitative insights following a contribution approach. Given that the control groups have also received some fairly similar training as the target group, and that all three Producer Organizations have worked with CMS in at least one year between baseline and endline, this section assesses the impacts of differentiated training towards certification and being one-year Fairtrade certified. We acknowledge that certification is a long-term commitment, and that having one year of certification will not produce the expected outcomes. However, we consider that just the fact of being certified, even for one year, implies that the farmer’s group is accomplishing a certain set of steps towards certification. We present the results using a pooled version of the control groups39 as the counterfactual, given strong similarities in their evolution over time in terms of training to farmers and improved governance. This allows for a stronger matching at baseline, increasing the likelihood that the “common trend” assumption holds between target and control groups. We use a simple version of the difference in differences model40 for all intermediate outcomes (e.g. agricultural practices). For final outcomes (e.g. yields, net income from coffee), we will use more complex versions of the model41.

Farms and farmers We have observed important average changes42 over time in practice adoption (environmental, agronomic and economic), as well as in the services offered to farmers, and economic returns to coffee (higher yields, prices and net income from coffee farming). We have observed no significant differences between the target and control groups (either separately or pooled). Impacts of training towards certification are not obvious as the control groups have also received training in similar topics (see Section 5). The contamination of the control group introduces a potential bias and results in no apparent impact. However, the project’s causal chain still applies, as those farmers who receive training are typically more likely to adopt sound agronomic practices. In fact, we have observed an important correlation between training at the farm level and practice adoption, which is part of the causal chain from certification bodies. Training itself does not lead to adoption, as it needs to be delivered on a regular basis by trainers whom farmers trust. There is also the need to consider the time it takes farmers to make the decision to adopt a particular practice as farmers have been growing coffee in a different way for decades. Finally, there are other market (credit) and non-market (distance to commercial centers) restrictions that can prevent a farmer to adopt. Furthermore, we have also observed an important correlation between practice adoption and trust to the cooperative. The higher the farmers’ level of trust in the cooperative, the more likely they are to adopt practices. This is another key aspect of the project’s theory of change, where training at the producer organization level can also trigger change at the farm level.

39 Any salient feature when comparing target and control groups separately will be adequately addressed. 40 See equation 1 41 See equation 3 42 Changes were observed in a subset of farmers.

34

Environmental Practices From an array of environmental practices associated with soil and water conservation and integrated pest management, we found evidence of changes in two specific practices associated with the intervention’s theory of change: an expanded and more intensive use of shade trees; and conducting regular visual examinations on their coffee farms to detect pests and/or diseases. We do not find strong evidence of changes in other practices. Increasing use of shade trees and decreasing use of soil ridges and drainage channels. Measuring the percentage of farmers using practices for soil conservation and plant water use-improvement indicates whether farmers are taking reasonable care to prevent soil erosion, maintain soil structure, and promote water percolation. Practices assessed by the survey were: drainage channels, soil ridges, live or deadwood fences, hedgerows and shade trees. Quantitative evidence shows an increasing number of farmers using shade trees in both target and control groups. It also shows a relevant decrease in the use of soil ridges and drainage channels. Farmers in the target group reported using between one and two (on average 1.5) practices, a number similar to baseline figures. However, the control group has significantly increased the number of soil and water management practices incorporated in their coffee farm as compared to the target group (from 1.4 to 1.7). We observe that the number of farmers using at least one of the practices mentioned above, is reasonably high and has significantly increased in both target (82% to 95%) and control groups (85% to 92%). This is mainly associated to the increase in the number of farmers using shade trees. The percentage nearly halved, however, for farmers using at least two conservation practices, with a significantly higher improvement in the control group.

Table 6.1 Soil & water conservation practices

Control group farmers have increased water conservation measures, while target group shows no significant improvement. We have observed no significant changes in the percentage of farmers using water catchments within the target group. However, control group farmers have doubled the existence of water catchments.

Baseline Endline Baseline Endline

Soil & Water conservation practices (number) 1.5 1.5 1.4 1.7 -0.3 *

Soil & Water conservation 1+ 82% 95% 85% 92% 6%

Soil & Water conservation 2+ 44% 38% 39% 56% -24% **

Soil & Water conservation 3+ 18% 13% 14% 18% -10%

Target Control Diff-n-Diff

35

Fig. 6.1 Water catchments usage (% of farmers)

There is a significant increase in the percentage of farmers using at least one IPM practice, yet there is large room for improvement. The percentage of farmers using at least one IPM tool has significantly increased in the target and control groups (from 35% to 84%) as farmers have started widely conducting regular visual examinations on their coffee farms to detect pests and/or diseases (from 32% to 82%). Other IPM practices are still not widely used such as keeping records on pest infestations, treatments, and results; use of traps, repellants and natural pesticides; create or preserve places for beneficial predators; or plant species that repel coffee pests. When assessing the percentage of farmers using two or more IPM practices, we observe that only 14% of sampled farmers perform another practice beyond conducting regular examinations. This evidence suggests that there is much to do in pest management in the region to minimize exposure to pests and diseases, for ensuring a less vulnerable coffee farm. This is particularly important when 62% of farmers declare pests and diseases as an important source of risk on their coffee farms, and 53% declared being affected by pest and diseases in the last season.

Fig. 6.2 Number of Integrated Pest Management (IPM) tools used (% of farmers)

32% 31%

18%

35%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Baseline Endline Baseline Endline

Target Control

35%

84%

38%

83%

4%13% 11% 15%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Baseline Endline Baseline Endline

Target Control

IPM1+ IPM2+

36

The target group exhibits an important improvement in their perception of their communities’ care for the environment. While target farmers’ perception of their own care for the environment has slightly changed, the average perception on their community’s care for the environment exhibits an important increase43. While we do not observe a significantly different change in terms of good environmental practices between target and control groups, there is evidence of improvements in perceptions of change around good care for the environment in target communities beyond control farmers.

Table 6.2 Farmers’ perception on care for the environment

Agronomic practices Training towards certification and training received by the control group shared similar topics in particular related to coffee farming operations. We found some changes over time in the application of key coffee-specific agronomic practices (e.g. pruning) and use of inputs (e.g. chemical fertilizers, use of pesticides and paid labor), closely associated to training. This effect is not captured in the differences-in-differences estimation as control group farmers were also trained, and this is a key mechanism of transmission of the effects. Furthermore, we have observed an important correlation between training and practice adoption. This insight sheds a light on the importance of standards beyond the certification itself, as practice adoption is critical to improve yields and revenues. No significant changes in weeding practices. We did not observe changes in weeding practices over time, mainly due to the already-high levels at baseline (95% of farmers weed their coffee farms). Furthermore, the intensity of weeding remained similar to rather low baseline standards (2.5 out of a standard of 4 times per year).

43 The change over time is statistically significant; however, the differences in differences estimator do not show statistical significance.

Baseline Endline Baseline Endline

Perception of household care for environment (index 0-1)

0.64 0.64 0.63 0.62 0.01

Perception of community care for environment (index 0-1)

0.61 0.63 0.59 0.59 0.02

Farmers good care for environment (percent)

57% 60% 53% 48% 7%

Communities good care for environment (percent)

45% 53% 43% 42% 9%

Target Control Diff-n-Diff

37

Table 6.3 Coffee-specific agronomic practices

There is evidence of changes in pruning practices over time but not between target and control. We have observed an increase in the percentage of farmers pruning their coffee in both target and control groups; however, we did not observe substantial differences between groups. We also assessed the different types of pruning (trim small branches only to direct growth; stumping, uprooting or top work; cut trees close to the ground; removing secondary branches; de-suckering; and pruning shade trees), and found no evidence of an improvement in the number of pruning practices performed by coffee farmers in both groups. However, we observed an important improvement in shade tree management in both target and control groups, and a systematic reduction in the percentage of farmers removing secondary branches.

Table 6.4 Pruning practices

Baseline Endline Baseline Endline

Weeding (percentage) 99% 96% 96% 99% -5% *

Weeding (times per year) 2.7 2.6 2.7 2.6 0.0

Pruning (percentage) 77% 84% 83% 90% 1%

Pruning (types of pruning) 1.5 1.6 1.6 2.0 -0.3 *

Prune coverage (percent of coffee trees) 53% 60% 56% 58% 5%

Change cycle (percentage) 1% 68% 74% 70% 70% ***

Target Control Diff-n-Diff

Baseline Endline Baseline Endline

Trimming 30% 27% 40% 25%

Stumping 4% 5% 3% 5%

Cutting close to the ground 14% 14% 15% 28%

Removing secondary 36% 1% 40% 3%

De-suckering 49% 44% 56% 59%

Shade trees 21% 65% 11% 81%

Target Control

38

Changing coffee tree cycles improved significantly in target group. We found that farmers in the target group have significantly engaged in changing the periodic cycle of their coffee plantations (revitalizing production from trees by cutting them off at ground-level and then allowing them to regrow), a practice that we did not observe at baseline, significantly increased to control group levels. We have observed a significant increase in the use of inputs over time. The percentage of farmers using synthetic fertilizers has doubled between baseline and endline in both target and control groups alike. On the other hand, the use of chemical pesticides has also increased but to a lesser extent, while the percentage of farmers using natural fertilizers – already relatively high - has increased mostly in the control group, to reach a similar level as the target group.

Table 6.5 Use of inputs

Pooling baseline and endline data we developed an econometric model to assess whether there are signs of correlation between training and practice adoption. Our model focuses on training but has incorporated a set of controls to refine the estimation, such as existence of shocks, participation in the producer organization, role of coffee in the household’s overall economy, distance to markets, access to credit, age, education, gender and household size. Our results, noted in table 6.6 below, show that there is a positive and statistically significant relationship between receiving training and the use of fertilizer or pesticide application. While the model’s intention is not to assess causation44, this correlation suggests that an increase of 10% in the number of farmers trained increases the percentage of farmers adopting fertilizers by 1.88%. Not surprisingly, the asset index shows that wealthier farmers are more prone to apply fertilizers or pesticides. These conclusions apply similarly to target and control farmers.

44 The relationship more training more practice adoption, can also be stated in the other way: the more practices you actually do, the more likely you are to demand training. However, we must say that training is generally provided at the producer organization level, and the farmers that are typically trained are younger, more educated, male, with larger household sizes, less altitude and closer to a commercial center, richer and with higher participation in the producer organization (see Table 5.2).

Baseline Endline Baseline Endline

Use of chemical fertilizers (percentage) 27% 53% 33% 58% 1%

Use of natural fertilizers (percentage) 74% 75% 65% 79% -13%

Use of chemical pesticides (percentage) 18% 21% 13% 27% -11%

Target Control Diff-n-Diff

39

Table 6.6 Determinants of use of agricultural inputs (probit Use = 1)

These results show the value of training for coffee farming, which shows a pathway along which certification can contribute directly to improve coffee incomes and reduce poverty. One key element to assess though, is that training by itself will not necessarily trigger practice adoption. First, there are a few steps between training and adoption, and these have to do with the logical sequence of learning by a farmer to be ready to adopt a specific practice. It entails acceptance, trying, realizing the value of training, and finally adopting. Given that coffee farming is an annual process, where the results of any change can be assessed after one year, the adoption process may take longer. Second, there are other, important restrictions to adoption that have to do with liquidity and access to credit. Lastly, the type of training matters. It is not just training that helps; the ability to convince the farmer that it is a good idea helps as well but it takes time, requires multiple visits, peers to do the same, and trust in the technician, amongst others. Economic practices We have observed significant changes in record keeping and understanding price formation over time, however no differences between target and control organizations. While changes are important, there is still a long way to go to reach the majority of farmers. We have observed an important correlation between trust in the cooperative and practice adoption, which validates the project’s theory of change where investments at the cooperative level to improve governance and transparency lead to better results at the farmer level. Significant improvements in record-keeping, but still room for improvement. We found that farmers in each producer organization are increasingly keeping records of their coffee farming activities. Improvements are observed in every record-keeping category. Farmers are more likely to keep records on production and sales (55%), purchase and use of inputs (25%), labor payments (15%), and coffee diseases or damage (10%). Despite the

0.188 *** 0.152 ***

0.065 *** 0.048 ***

-0.017 0.034

0.215 *** 0.092 ***

YES YES

Observations 605 605

Pseudo-r2 0.113 0.103

Gender (Male = 1, Female = 0)

Year (dummy, if endline = 1, otherwise = 0)

Pest = 1Variables

Fert = 1

Training

Asset index (quantile)

Probit

Target group (dummy, if target = 1, otherwise = 0)

40

important changes observed, there is still a relatively low adoption of the record keeping practice to make informed decisions on their coffee.

Fig. 6.3 Record keeping (% of farmers)

Farmers are more knowledgeable about coffee prices; however, there are large information gaps that undermine trust. Lack of price transparency undermines trust. A farmer’s major concern is related to the lack of understanding of how prices are determined. Some coffee marketing agents (CMA), in order to market a producer organization’s coffee, offer board members and farmers better prices for their coffee (amongst other things). Farmer focus groups showed deep concern with lack of price transparency at both the producer organization level and the CMA level. There has been an important improvement in the degree of understanding of price formation, but there is still a lot of work to be done for farmers to at least have a basic understanding of how prices are determined. Improvements have not been different between target and control groups.

Table 6.7 Price formation

As noted in the baseline and in the market structure and dynamic section, trust is a key asset in the Kenyan coffee value chain. In fact, we have observed a strong correlation

45%53% 55%

69%

19%

30% 27%33%

9%14%

11%

21%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Baseline Endline Baseline Endline

Target Control

Recordkeeping1+ Recordkeeping2+ Recordkeeping3+

Baseline Endline Baseline Endline

Full understanding of price formation (percent) 5% 8% 7% 12% -3%

At least basic understanding of price formation (percent) 26% 35% 34% 42% 2%

Target Control Diff-n-Diff

41

between trust in the producer organization45 and the use of inputs (fertilizers) and practice adoption. This implies that the more transparent transactions are, the more likely farmers trust their organizations, increasing the likelihood of adoption. We found similar results for target and control farmers, as well as for male and female farmers.

Fig. 6.4 Trust and practice adoption

Services Services offered by the cooperatives have significantly increased over time. Farmers are receiving more credit and inputs from the cooperative, as well as more of the other services related to the provision of quality seeds, technical assistance, and transportation services. However, we have also noted that there is significant room for improvement since the cooperatives only reach a minority of farmers. Farmers have reported an important increase in the provision of services. The most important change observed is related to credit services. This is consistent with the significant increase in the percentage of farmers that received credit in general. While credit from banks, NGOs, and microfinance institutions also has been available in the last few years and has significantly increased, credit from the producer organizations has made the difference, driven mainly by the relationships with the marketing agents. Farmers have also reported an important increase in the provision of inputs and high-quality seedlings. This is consistent with the higher demand for inputs observed in the endline survey. Finally, there is also an increase in the percentage of farmers receiving transportation services and technical assistance. This trend is consistent in all producer organizations.

45 Trust is defined as farmers considering that their cooperative society acts in farmer’s best interest when taking decisions. This comes from the question “Does your cooperative society acts in your farm’s best interest in making decisions?” with potential answers: always, sometimes, rarely, never. We consider the farmer trusts its cooperative society when responses are either always or sometimes.

35%

60%

27%

50%

0%

10%

20%

30%

40%

50%

60%

70%

Baseline Endline

Fertilizer use (% of farmers)

Acts on behalf of farmer Does not act on behalf of farmer

3.8

4.6

3.3

4.3

0

1

2

3

4

5

Baseline Endline

Number of practices (0 - 8 practices)

Acts on behalf of farmer Does not act on behalf of farmer

42

Table 6.8 Producer organization services to the farmers

Coffee profitability There has been an important improvement in yields over time. While we observe no statistically significant difference between target and control groups at average yields, there is evidence of changes at the lower end of the distribution. Prices represent to some extent, average coffee quality. We have observed important changes in prices as reported by the target producer organization and CMS-Control. Consequently, while there are no significant differences in the change in coffee revenues (and in coffee revenues per hectare), we observe relevant changes at the lowest deciles of the distribution. Costs have significantly increased in both groups, especially due to higher investments in fertilizers and higher paid labor days. Higher revenues outweighed cost increases, triggering higher net incomes from coffee production. There is some evidence of larger changes in the target group when assessing coffee net incomes per hectare; however, when calculating the total net incomes from coffee there is no significant difference. Finally, farmers have seen positive changes in the last few years and have positive expectations about the future. Yields have significantly improved over time; we observe slight positive differences between target and control groups at lower ends of the distribution, which vanish in higher ends of the distribution. Yields have increased from an average of 592 kg of GBE per hectare (baseline) to just above 706 kg of GBE per hectare (endline). This increase has been, on average, equally important in all three cooperatives (target and control), showing no evidence of statistically significant differences between target and control groups. The figure below shows that in the target group, changes arise mainly from a larger concentration of farmers in the upper end of the distribution (i.e. only a group of farmers with very high yields) whilst in the control group higher yields come from a shift in the distribution towards the right (i.e. a larger group of farmers with higher yields)

BL EL BL EL

Number of services offered (0 - 6) 0.4 1.2 0.8 1.5 0.0

Offer credit or financing3% 32% 1% 40% -10%

Providing or selling inputs6% 44% 19% 48% 9%

Information on input providers19% 8% 38% 11% 16% **

Providing high quality seedlings8% 12% 15% 23% -4%

Transportation services10% 21% 4% 21% -6%

Provide technical assistance6% 14% 6% 13% 0%

Target Control Diff-n-Diff

43

Fig. 6.5 Distribution of yields over time, for all producer organizations

Assessing the impact of the program not only on average yields, but also on the overall distribution, shows statistically some differences between target and control groups at the lowest quantiles of yield distribution. The table below shows that at lower levels of the distribution there is a positive difference between target and control groups – although not statistically significant – that reduces (and becomes negative) as the assessment is performed at higher levels of the distribution. This result may suggest that while training has happened in both target and control groups, training in the target group affected more the lower levels of the distribution, while training in the control group affected more in the upper levels. In other words, training in the target group has been significantly more effective for poorer performers; while training in the control group has benefited more the best performers.

Table 6.9 Yield (kg GBE per hectare)

As mentioned before, most training efforts have been oriented towards improving coffee farming operations, where we have observed important improvements in some practice adoption and input usage over time. Furthermore, all producer organizations and key informants in the region acknowledge the value of training, especially in a region with few extension services available. Farmers and board members declare that practice adoption (while not easy to achieve) could generate important changes in yields, especially in a

Baseline Endline Baseline Endline

Average 680.5 782.8 506.6 617.0 -8.2

Quantile 0.2 291.7 350.7 236.1 236.1 59.0

Quantile 0.4 428.5 511.9 362.1 415.9 29.7

Quantile 0.6 641.1 691.1 438.0 692.7 -204.7

Quantile 0.8 1,063.7 1,079.7 663.0 940.7 -261.8

Target Control Diff-n-Diff

0.0

002

.000

4.0

006

.000

8.0

01D

ensi

ty

0 1000 2000 3000Yields (kg GBE per ha)

Target (2014) Target (2017)

Target group

44

region with relatively low yield levels associated with low levels of practice adoption and input usage. Farmers declared having met people doubling their yields by simply applying some of the practices and field improvements taught on the training sessions. Practice use is highly correlated with higher yields. In order to assess whether such practices have some relationship with yields, we developed a simple fixed effects model using our panel dataset, controlling for factors such as percentage of unproductive young trees, improved varieties and the presence of large shocks affecting coffee production. Our estimations show the importance of practice adoption to improve yields. Just adopting a specific practice seems to have positive impacts on average yields. For example, doing two or more IPM practices (not including pesticide application of any sort), will increase yields in 255 kg of GBE. Our results show the value of the causal chain generated by training, which promotes good agricultural practice adoption. Once practices are adopted, farmers observe significant improvements in yields.

Table 6.10 Fixed effects estimation (Dependent: Yields (kg of GBE)

There is evidence of coffee quality improvements. Prices have been significantly higher in target group than in non-CMS control (although similar changes than in CMS control). Prices are often driven by the quality factors of grade and cup profile. Thus,

Marginal Effects

172.455 ***

133.068 **

122.896 *

255.001 ***

162.502 *

-72.890

-21.836

YES

Number of observations 578

R-sq within 0.200

R-sq between 0.140

R-sq overall 0.162

Variables

CONTROL

Synthetic fertilizer use (percent)

Synthetic pesticide use (percent)

Use at least two IPM practice (percent)

Weeding 2+ times per year (percent)

Pruning 2+ techniques (percent)

Soil conservation practices 2+ (percent)

Natural fertilizer use (percent)

45

the price a producer organization receives starts with the quality farmers deliver. At the end of the selling season, the producer organizations determine a single amount per kilogram to pay all the farmers who delivered coffee. By law, this amount must be a percentage of what the producer organization received, but no less than 80 percent. Therefore, the price farmers receive is directly related to the overall quality of the producer organization’s coffee as delivered to the Nairobi auction by the marketing agent. Farmers in our focus groups are well aware that the prices received are correlated with the quality delivered. However, farmers also recognize that given that all coffee is delivered to the same processing stream, with no provision made for price differentials, there are no incentives for a farmer’s willingness to make extra efforts to improve quality. With no quality differentials, there is no return to the investment made in adopting specific practices for quality improvement, especially when other farmers did not make such investments, thus keeping the overall average quality low. However, key informants and board members mentioned that given the low-quality levels to which coffee in the region is exposed, the application of very basic practices that may improve quality might also improve quantity. In fact, we found that all of the studied producer organizations have seen improvements in quantity, but only a few have witnessed quality improvements. Farmers and board members of our target organization recognize high improvements in the quality of coffee sold in the auction market, by reducing from 300 to 73 the number of low-quality coffee delivered. Non-CMS Control has observed important reductions in “mbuni” (low quality coffee delivered); while CMS Control did not report any quality changes in the coffee delivered to the cooperative society (from farmers perspective), nor on the coffee sold to marketing agents (from the cooperative society perspective).

Fig. 6.6 Price evolution, by producer organization (In US$ per kg GBE)

Revenue from coffee farming has significantly improved; we observe slight differences between target and control groups at lower ends of the distribution. Training has promoted the application and use of some basic agro-economic practices amongst farmers. This should have an effect over productivity and quality. We have seen important changes in productivity in both target and control groups; however, they changed at similar rates resulting in no significant average effects of the program on productivity (only at lower ends of the distribution). On the other hand, we have witnessed important changes in prices, which may reflect improved quality (only for target and CMS-control farmers).

1.91

2.26 2.152.46

1.71

2.84

0.00

0.50

1.00

1.50

2.00

2.50

3.00

Target CMS Control non-CMS Control

2014 2017

46

The combination of both effects turns into a positive but not statistically significant change in revenue per hectare. The table below reports on changes using the pooled counterfactual, which includes non-CMS control, which have shown a decline in the price received in USD. Results show larger effects on the lower quantiles of the distribution, following the yield pattern.

Table 6.11 Coffee Revenue per hectare (USD)

However, when assessing the impacts with each control separately, we observe significant changes in coffee revenue per hectare as compared with non-CMS control (Control-II) where average prices decreased; and no significant changes with CMS-Control (Control-I), which exhibit similar rates of growth on productivity and quality.

Fig. 6.7 Coffee revenue per hectare, by producer organization (In US$ per ha)

Furthermore, when assessing the overall coffee revenue per hectare distribution, we observe larger positive changes for the poorer farmers (at lower levels of the distribution), following the pattern observed in yields. Again, training in the target group seems to be more oriented to results among the poor than training in the control group.

Baseline Endline Baseline Endline

Coffee Revenue per hectare (USD) 1,335.1 1,922.0 1,107.0 1,325.9 368.0

Quantile 0.2 508.7 823.5 478.1 444.5 348.5

Quantile 0.4 796.6 1,145.2 828.1 1,053.2 123.6

Quantile 0.6 1,275.2 1,740.6 1,033.8 1,477.7 21.6

Quantile 0.8 1,709.9 2,585.5 1,426.1 2,118.1 183.6

Target Control Diff-n-Diff

1362 1303

861

1951

1157

1654

0

500

1000

1500

2000

2500

Target CMS Control non-CMS Control

2014 2017

47

Production costs have significantly increased but much more in the control group to reach target levels. While training has promoted practice improvement and therefore yields and quality, it has also had an impact on production costs. We have observed important increases in fertilizer costs per hectare, as well as in paid labor. Total monetary costs have therefore increased in both target and control groups. Coffee farming in Western Kenya is heavily dependent on family labor. While it has reduced significantly more in the target group than in the control group, relative weight of family labor on overall coffee labor is still high and changes are statistically similar between target and control groups. Total monetary costs per hectare are relatively low as compared to other countries, where average production costs are around USD 800 per hectare. Around half of total costs come from hired labor (harvest), and to a lesser extent, expenditures in fertilizers. In fact, farmers tend to consider coffee as a crop that requires no further investment and provides easy cash. As noted on the baseline report, lack of trust and low expected returns end up in inefficient underinvestment at the farm level.

Table 6.12 Monetary Costs per hectare (USD)

There is some slight evidence of impacts in net income from coffee production, driven by larger revenues as compared to non-CMS control. Assessing profitability per hectare show signs of larger improvements in the target as compared with the control group, driven by price and cost differences between target and non-CMS control farms. Larger net incomes from coffee production arise from higher revenues (higher prices) and a significantly lower increase in production costs. Differences in coffee net incomes per hectare are approximately US$ 500 per year. However, given that average coffee area is 0.3 hectares, absolute differences are about US$ 174. Such differences represent on average 24% of total household incomes.

Given differences in price changes specially between CMS Control and non-CMS Control, we presented the aggregated impact assessment, as well as the dissaggregated one to compare the target group with each of the control groups46. As expected, we observe important differences in

46 Note that differences in target group outcomes are due to matching techniques

Baseline Endline Baseline Endline

Fertilizer cost per hectare (USD) 81.2 92.1 37.3 67.7 -19.5

Pesticides cost per hectare (USD) 14.3 7.0 4.0 11.5 -14.8 **

Labor cost per hectare (USD) 116.9 134.3 53.6 170.9 -99.9

Total monetary cost per hectare (USD)

212.2 233.4 95.0 250.1 -133.9 *

Days of family labor 136.9 72.2 108.4 101.4 -57.7 ***

Ratio family labor / total labor 94.3% 82.2% 92.5% 80.3% 0.1%

Target Control Diff-n-Diff

48

coffee net income (total and per hectare) when comparing the target and CMS control groups; and no difference between target and non-CMS control. On average, when comparing the target and aggregate control groups, we found no significant evidence of impacts.

Table 6.13 Net Income from coffee production (USD)

Baseline Endline Baseline Endline

Revenue per hectare (USD) 1,335 1,922 1,107 1,326 368

Total monetary cost per hectare (USD)

212 233 95 250 -134 *

Net Income per hectare (USD) 1,123 1,689 1,012 1,082 496 *

Coffee Revenue (USD) 284 495 259 435 35

Coffee Costs (USD) 46 75 29 99 -41

Net Income from coffee (USD) 237 420 230 337 76

Target Control Diff-n-Diff

Baseline Endline Baseline Endline

Revenue per hectare (USD) 1,363 1,951 1,303 1,157 734 **

Total monetary cost per hectare (USD)

214 237 97 185 -64

Net Income per hectare (USD) 1,149 1,715 1,206 983 788 ***

Coffee Revenue (USD) 285 498 296 310 198 **

Coffee Costs (USD) 46 75 31 57 3

Net Income from coffee (USD) 239 423 265 254 194 **

Target CMS Control Diff-n-Diff

49

Average differences in coffee net income per hectare are driven by differences between target and non-CMS control farms. Differences between target and CMS control farms is not statistically significant.

Fig. 6.8 Coffee net income per hectare, by producer organization (In US$ per ha)

Farmers and producer organizations have positive perspectives around coffee farming. Farmers’ perceptions reflect what we have observed in the data. Coffee farming is not always profitable (only around a quarter of the sampled farmers consider that coffee has been profitable in the last three years). But there are important signs of positive perception in the target group. While the ratio of farmers that would be happy or satisfied if their children become coffee farmers is rather large, we have observed an interesting decrease in both target and control

Baseline Endline Baseline Endline

Revenue per hectare (USD) 1,280 1,868 861 1,654 -205

Total monetary cost per hectare (USD)

208 229 114 298 -164 *

Net Income per hectare (USD) 1,071 1,639 747 1,356 -41

Coffee Revenue (USD) 273 480 243 684 -235

Coffee Costs (USD) 47 74 40 152 -85 *

Net Income from coffee (USD) 227 406 203 532 -150

Target non-CMS Control Diff-n-Diff

1149 1206

747

1715

983

1356

0

300

600

900

1200

1500

1800

2100

Target CMS Control non-CMS Control

2014 2017

50

farmers alike. This is a trend we may want to revert as it can potentially compromise the future of coffee farming, especially when farmers are relatively old. Farmers have strong beliefs in a positive future for coffee farming. Farmers see higher yields and potential from training to improve their incomes. Perspectives remain at high levels, although target group has slightly better perspectives. Finally, the importance of coffee farming is high in both groups, but in the target group there are more farmers claiming that coffee is an important source of income (especially as a cash source), as opposed to the control group, where there was a decline.

Table 6.14 Perceptions around coffee business

Next generation index: constructed using the responses from the question “Would you be happy if any of your children choose to become coffee farmers?” The larger the index, the “happier” the farmer is.

Perspective index: constructed using the responses from the question “What would you say about the prospects for coffee farming in your region over the next five years?” The larger the index, the more positive the response is.

Coffee importance index: constructed using the responses from the question “How beneficial is coffee farming to your overall economic situation?” The larger the index, the more important it is.

Household We have seen that total household income47 has significantly increased over time, mainly due to increases in coffee income, as well as in livestock by-products and wage income. Despite significant increases in income, monetary poverty has not reduced. In fact, while poverty headcount remains unchanged, poverty gaps48 have significantly reduced. This implies that the distance to the poverty line was significantly high at the baseline, and that while changes have occurred it was not enough to move farmers out of poverty.

47 For comparability purposes, we transformed all monetary values into American dollars, using average exchange rates for the season. 48 Poverty gap is the distance between actual income and the poverty line, for all the poor.

Baseline Endline Baseline Endline

Coffee is profitable (percent) 26% 37% 21% 19% 13%

Coffee is sometimes profitable (percent)

62% 55% 68% 69% -7%

Next generation index 0.82 0.77 0.87 0.78 0.05

Perspective index 0.72 0.76 0.81 0.74 0.10 **

Coffee importance index 0.71 0.78 0.84 0.74 0.16 ***

Target Control Diff-n-Diff

51

We have observed important changes in household and agricultural assets at the household level, explained by higher incomes. Notwithstanding, food insecurity has dramatically increased. Total household income Household incomes have significantly grown over time but with no differences between target and control groups (either pooled or separately assessed). Similarly, we found no evidence of statistically significant differences in per-capita household incomes, although they have also exhibited significant growth over time. Total household income has significantly increased in both target and control groups due to both increases in coffee net incomes and increases in other sources of income (i.e. other crops income, livestock management income, business, wage labor, transfers and remittances). We observe no statistical difference of income changes between target and control groups, even when using each control separately or when assessing changes in the distribution. The role of coffee farming in overall household incomes, remains the same as in the baseline, showing no evidence of relevant change over time (although both have increased, coffee income and total income). We observe that while the target group farmers have increased their incomes from livestock management, control farmers are earning more wage income from non-agricultural activities.

Table 6.15 Household income and its components

The relatively low weight of coffee on overall household income also plays a role in the success of any program oriented to coffee improvement. On the one hand, richer households with greater possibilities to invest are less dependent on coffee income and more dependent on business activities and wage income. On the other hand, poorer households, whose income depends significantly more on coffee production, are less positioned to invest in coffee farming and have less family labor to rely upon. Total household income has significantly increased in both target and control groups. Higher incomes are partially due to higher coffee net incomes, but also due to higher income

Baseline Endline Baseline Endline

Net household income (USD) 1,894 2,587 1,792 2,200 285

Coffee (percent) 22.5% 23.9% 22.2% 22.4% 1.3%

Other crops (percent) 27.2% 18.3% 34.2% 28.1% -2.7%

Livestock (percent) 15.3% 20.3% 16.1% 14.5% 6.7%

Business (percent) 11.8% 8.3% 17.5% 5.2% 8.8%

Wage (percent) 18.4% 17.8% 4.5% 20.0% -16.1% **

Transfers (percent) 4.9% 11.3% 5.4% 9.9% 2.0%

Target Control Diff-n-Diff

52

from livestock management, and off-farm wage income. Similarly, per-capita household income has significantly increased in both target and control groups. We found no evidence of a differentiated path when contrasting separately the target group against each control group as when comparing with the overall distribution.

Fig. 6.9a Total household income (In USD)

Fig. 6.9b Per-capita household income (In USD)

Poverty We describe poverty evolution using different measures of poverty and assess the correlations of poverty with specific outcomes. We found that changes in total household income over time have contributed to reduce the poverty gap, but not the poverty headcount, due to the little household income coffee farmers in the Bungoma region get.

1,904

1,187

2,609

2,201

-

500

1,000

1,500

2,000

2,500

3,000

Target Control

2014 2017

350

180

418

319

- 50

100

150 200

250 300

350 400

450

Target Control

2014 2017

53

We use different measures to assess poverty. For income poverty, the standard considers an individual as poor if their income falls below a poverty line. Here, we looked at two different approaches for establishing poverty: 1) the Poverty Probability Index (PPI), which uses a set of ten questions tailored to individual countries to estimate the likelihood that households will fall below a poverty line; and 2) income poverty lines that use data from our farmer survey to estimate total household income based on a set of questions used in the COSA household survey, designed to capture income from coffee, other crops, livestock production, wage income, business income, and gifts and remittances received. Measuring total household income allowed us to compare it with different poverty lines. In this case, we follow World Bank’s different poverty lines which represent different standards of living. The PPI will estimate the likelihood of a household of falling below the food and the national Kenya poverty line as of 200549. The World Bank’s poverty line will estimate the percentage of households whose per-capita daily incomes fall below the US$ 1.9, US$ 3.2, and US$ 5.5 poverty lines. We also used an asset-based identification of households’ distribution by formulating a profile of capital stocks (human capital, household assets, and agricultural assets) a household possesses. We categorized farmers in the sample using questions from the COSA household survey that quantified household assets. This way of identifying farmers does not provide information about the poverty levels of farmers, but help us identify relative wealth. Using the PPI poverty likelihood, we observe that the control group underperformed as compared to the target group. In both scenarios (likelihood of being poor under the food poverty line and the national poverty line), the target group did not show signs of significant changes over time. However, in the control group while monetary poverty remains unchanged, the likelihood of being poor (i.e. PPI) has slightly increased50 51. We observe a similar pattern when assessing the target group against each control group separately. While monetary poverty hasn’t significantly moved, we have observed a significant reduction in the poverty gap across years. When assessing poverty using the World Bank poverty lines52, we observe no signs of poverty reduction in both target and control groups. As noted in the baseline, general poverty is high across the region, and it is not borderline but rather concentrated at very low levels of income. The poverty gap shows the difference between the poverty line and the actual income, and is expressed in USD per member per day. Our results show that the average gap has significantly reduced from an average of US$ 1.2 to US$ 1.0, following the income improvement over the 2014 – 2017 period. However, it implies that farmers are on average still very far from reaching the World Bank poverty line (US$ 1.9). This in fact implies that while farmers have improved in their average coffee incomes, there is a long way to go to help farmers move out of poverty. We have observed an important improvement in the farmer assets index. Farmer capital has increased over time in both target and control groups. We have observed that the most important changes are: access to electricity (from 20% in the baseline, to 60% in the endline), television (from 10% to 20%), motorbikes (from 3% to 10%), and the stock of agricultural assets, which almost doubled.

49 We use the PPI score based on the 2005 poverty line, as the updated version of the PPI was released after the endline survey. 50 The PPI has been recently updated (December 2018). We had to use the previous PPI (2015) as when we developed the survey (endline) the updated version was not released. 51 The PPI questions are essentially about household composition and access to certain assets. 52 We use updated version of the poverty lines following the World Bank, where each poverty line represent different standards of what poverty means.

54

Changes in assets over time are mostly correlated with changes in income than with changes in poverty rates, given the significantly large poverty gap.

Table 6.16 Poverty changes over time

Asset index: Factor analysis applied over a set of assets: Tropical Livestock Units (TLU), coffee area, number of rooms in the household, electricity (yes / no), frying pan (yes / no), radio (yes / no), television (yes / no), motorbike (yes / no), bicycle (yes / no), an index of agricultural assets, and total number of years of education of the household head. The index is built for all the panel data, using weights from baseline.

Food security We define food insecurity as the number of days in which any household member had to skip meals or significantly reduce their food intake. While incomes have increased over time, general poverty exhibits no signs of improvement. However, food insecurity has shown a small but worrisome increase in both target and control groups. This is partly due to reliance on maize whose price has increased and also suggests that not only is farmers’ total income not sufficient but also that it is distributed at one time of year (the month where coffee payments are expected). Food insecurity has increased over time. We have observed that on average food insecurity has increased from four days to almost seven days. This has happened in both target and control groups. In the target group the number of food secure farmers has dramatically reduced (from 73% to 57%) while in the control group it has increased. However, when assessing food insecurity, it seems that while in the target group food insecurity is mild, in the control group it has significantly increased in the extremes.

Baseline Endline Baseline Endline

PPI Food14% 12% 10% 15% -7% *

PPI Poverty Line38% 38% 34% 43% -10%

WB PL $1.9 (percent)90% 89% 94% 92% 1%

Poverty Gap WB PL $1.9 (USD) 1.2 1.0 1.3 1.1 0.0

WB PL $3.2 (percent)95% 94% 97% 97% -2%

WB PL $5.5 (percent)98% 98% 97% 100% -1%

Asset Index0.10 0.19 0.10 0.22 -0.02

Target Control Diff-n-Diff

55

Table 6.17 Food security

Fig. 6.10 Changes in food insecurity (percent)

Food insecurity has its peak between April and June. Food insecurity is not likely to happen with the same probability every month. In fact, we have observed a repeated pattern in 2014 and 2017, showing that food insecurity is more likely to happen between April and June, a period of time that is strongly correlated with coffee payments. This sheds a light on potential opportunities for improvements at the producer organization level to provide liquidity to farmers in most critical months of need.

Baseline Endline Baseline Endline

Food Secure (percentage) 73% 57% 57% 61% -20%

Food Insecure 1-9 days (percentage) 15% 22% 29% 8% 27% ***

Food Insecure 10-19 days (percentage) 8% 10% 7% 10% -1%

Food Insecure 20-29 days (percentage) 0% 6% 1% 5% 2%

Food Insecure 30+ days (percentage) 4% 5% 7% 16% -7%

Target Control Diff-n-Diff

-50%

-40%

-30%

-20%

-10%

0%

10%

20%

30%

1-9 days 10-19 days 20 - 29 days + 30 days

Target Control

56

Fig. 6.11 Critical months for food insecurity

Producer Organizations We have seen important changes at the producer organization level in terms of improved governance and an expansion of service delivery. This correlates with higher farmer participation and satisfaction with their board members and organizations. Volume sourced in target producer organization has reduced. The number of active members in target group has reduced as a group of farmers have shifted towards other organizations. Fewer members result in less coffee sourced. Moreover, we observe a reduction in the average kilograms of coffee sourced from farmers53. Our control groups have had a different experience. In CMS-control the average number of active farmers has increased at a similar rate as the total coffee sourced. In non-CMS control, the average kilograms of coffee sourced from farmers have significantly increased, regardless of the slight reduction in active membership.

53 Producer Organizations’ data may slightly differ from farm level data, as not all coffee produced by farmers is delivered to the Producer Organization. Also active membership is rather low in the target group as compared with control organizations (See Table 3.2).

0%

10%

20%

30%

40%

50%

60%

70%

80%

Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec

2014 2017

57

Table 6.18 Producer Organization members and volumes sourced

High ratios of female and youth membership in target Producer Organization. Female membership in the FCS is significantly larger than in control groups and in peer organizations in the Bungoma region. The presence of young farmers (less than 35 years old) in the target group is relatively large, suggesting an auspicious future for the organization as younger farmers step in and are likely to be more educated and open to change, and therefore more likely to adopt new technologies.

Table 6.19 Producer Organization membership structure (2017/2018)

Target farmers more actively participate in the producer organization. We have observed a significant increase in participation in the producer organization’s meetings, as well as a larger number of farmers voting in important decisions. This is a common trend amongst producer organizations, and seems to be associated with better governance at the producer organization level. In fact, all three producer organizations’ board members and CEO’s received training in topics associated to improved governance and leadership, as well as in practical aspects such as financial management, entrepreneurship and bookkeeping. These trainings are highly valued by the producer organizations’ representatives, who declared they even have to make financial efforts to pay for those services.

BL EL BL EL BL EL

Active members 1,334 921 1,540 2,630 2,234 2,176

Coffee Sales (tm fresh cherry) 524 300 671 1,096 553 849

Average kg fresh cherry per farmer 393 326 436 417 247 390

Control-I Control-IITarget

Target Control - I Control - II

Total members 1,528 3,600 3,017

Active members (percent) 60% 73% 72%

Female members (percent) 46% 30% 31%

Young members (percent) 34% 40% 1%

58

Table 6.20 Farmer participation in the producer organization

Services to farmers have significantly increased. Farmers have reported an important increase in the provision of services. The most important change observed is related to credit services. This is consistent with the significant increase in the percentage of farmers that received credit in general. While credit from banks, NGOs, and microfinance institutions has significantly increased in the last few years, it is the credit farmers received from the producer organizations that has made the most difference, and it is driven mainly by the relationships with the marketing agents who can facilitate financing. by the relationships with the marketing agents. Farmers have also reported an important increase in the provision of inputs and high-quality seedlings. This is consistent with the higher demand for inputs observed in the endline survey. Finally, there is also an increase in the percentage of farmers receiving transportation services and technical assistance. This trend is consistent in all producer organizations. Farmers are significantly more satisfied with their producer organizations. Farmers rate their board members highly in each of the three producer organizations. Furthermore, the percentage of farmers that consider that their producer organization acts on behalf of the farmer’s interest has significantly increased from 45% to 72% in the target group, higher than changes in the control group. All farmers value their producer organizations significantly more. In the target group the percentage of farmers that considers their producer organization as valuable or highly valuable has increased from 48% to 85%, an important change that is not statistically different to what has also happened at control cooperatives.

BL EL BL EL

Participation (percentage) 81% 92% 93% 97% 7%

Always participates (percentage) 69% 69% 84% 81% 2%

Vote on important meetings (percentage) 74% 87% 75% 88% 1%

Diff-n-DiffTarget Control

59

Fig. 6.12 Farmer perception of their producer organizations (% of farmers)

45%

72%

60%

75%

48%

85%

60%

90%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

Baseline Endline Baseline Endline

Target Control

The PO acts in farmers interest (percentage) PO is valuable (percentage)

60

7. Discussion In this section, we offer are deeper understanding of the main changes that occurred to the target farmers associated to the intervention by providing answers to the main research questions posed in the terms of reference. Research Question 1 What are the changes occurring at the farm, household, and cooperative levels leading up to certification with the combined Fairtrade and UTZ standards and again after three years of certification? We have observed significant changes at the farm, household and cooperative levels as shown in Section 6 for both target and control groups. For instance, at the farm level, we have observed significant changes over time in practice adoption (environmental, agronomic and economic); however, we have also acknowledged that there is large room for improvement. More intensive use of good agricultural practices has led to improved yields and coffee quality that outweighed higher production costs. At the household level, higher net incomes from coffee farming partially explain higher household incomes. Better coffee performance is not largely significant for overall household income as it accounts for roughly 24% of total household income. These higher incomes, however, are not enough to move farmers out of poverty, as the distance between incomes and the poverty line is still great. Finally, at the producer organization level, we have observed improved service delivery and quality as well as signs of improved governance and transparency. Training is at the center of the discussion as a key explanatory variable for such changes and the key instrument for certification bodies to promote change. While training has reached far more farmers than at the baseline, it has not covered the majority of farmers. We estimate that the revenue generated by the implementation of good agricultural practices is greater than the implementation costs. Given that farmers have low levels of implementation of good agricultural practices, it suggests there is great room for improvement. Practice adoption has improved. Training has played an important role in practice adoption improvement as both are strongly correlated (See Table 6.6). Training in the 2014-2017 period has significantly improved, as did the percentage of farmers using some key economic, environmental and agronomic practices. The graph below shows the number of representative practices applied by coffee farmers in their coffee plots, where the difference between trained and untrained farmers stands out, especially in the endline. Despite the observed changes in the percentage of farmers implementing good agricultural practices, there are two key things to discuss: (1) the need to strengthen and promote the implementation of good agricultural practices to a larger audience; and (2) improve the degree of adoption within each practice. For instance, while an average 43% of farmers use synthetic fertilizers, the amount used is low with average expenditure per hectare only reaching US$ 70. It means that there is a need to not only expand the percentage of farmers using improved fertilization but also the intensity of that use. Another example has to do with the implementation of integrated pest management techniques (IPM), while 61% use at least one IPM practice, only 12% use at least two practices, and 1% use three practices out of a possible total of four.

61

Fig. 7.1 Number of practices (0 - 8)

Coffee yields have exhibited significant increases. Increases in yields have been similarly important in all three cooperatives. Changes in the target group have been larger at the bottom of the distribution as opposed to changes in control groups. Such changes are strongly correlated with practice adoption (See Table 6.10).

Coffee quality has also improved. Reports from target cooperative’s board members coincide with farmer discussions during the focus groups. Data on prices show a 30% increase over time in both target and CMS-control, even though the market price went down. Price increases also coincide with higher quality coffees demonstrated by a higher percentage of those classed as AA and AB coffee beans, and fewer defects.

As a result of higher yields and higher prices (only the target and CMS-control farmers), revenue from coffee farming has also significantly increased.

Costs of coffee production have increased. Cost of production have risen mainly due to more intensive use of paid labor, although it is still highly dependent on family labor (around 80% of total labor). Nevertheless, costs of production in Mount Elgon are about a fourth of average costs per hectare amongst poor farmers in other coffee production regions. This is explained by the still relatively low level of penetration of practice adoption and high dependence on family labor.

Net income from coffee has significantly increased. Higher coffee revenues have significantly outweighed implementation costs, implying positive returns on investments in practice adoption and inputs usage.

Total household income has grown. Increases in coffee income, livestock proceeds and wage income explain increases in total household incomes. The relatively low weight of coffee in overall household income also plays a role in the success of any program oriented to coffee improvement. On the one hand, wealthier households with greater possibilities to invest are less dependent on coffee income and more dependent on business activities and wage income. On the other hand, poorer households, whose income depends significantly more on coffee production, are less well positioned or able to invest in coffee farming and may have less family labor to rely upon.

Poverty shows no sign of recovery. Monetary poverty rates are high. Despite significant growth in total household incomes, rates of monetary poverty remain stagnant. While

3.7

4.5

3.1

3.8 3.7

4.9

3.4

5.0

-

1.0

2.0

3.0

4.0

5.0

6.0

2014 2017 2014 2017

Control Target

Untrained Trained

62

poverty headcounts are unchanged, poverty gaps have significantly reduced. Nevertheless, incomes from coffee farmers in the Mount Elgon region are still a long way below the World Bank’s US$ 1.90 per person per day poverty line.

Food insecurity has increased. Despite higher incomes, food insecurity has increased. This suggests that farmers’ total income is not enough and we also know that it is unequally distributed over time, essentially associated with the period in which coffee payments are expected (March – June). This fact poses a challenging message for poverty alleviation, which not only relates to higher coffee incomes, but also on the evenness of the distribution of incomes received by the farmer. Producer organizations are being more democratic. Farmers in the target group exhibit greater participation in their producer organizations, and have an improved perception of transparency and better governance.

Producer organizations have improved their service delivery. Farmers have reported an important increase in the provision of services. The most important change observed is related to credit services. This is consistent with the significant increase in the percentage of farmers that received credit in general. While credit from banks, NGOs, and microfinance institutions has also been available in the last few years, and has significantly increased, it is the credit through the producer organizations that has made the difference, mainly driven by the relationships with the marketing agents. Farmers have also reported an important increase in the provision of inputs and high-quality seedlings by the producer organization. This is consistent with the higher demand for inputs observed in the endline survey. Finally, there is also an increase in the percentage of farmers receiving transportation services and technical assistance. This trend is consistent in all producer organizations. Producer organizations have improved trustworthiness amongst farmers. Farmer perceptions on the role of the cooperatives have significantly increased as can be shown by the percentage of farmers that consider their cooperatives valuable and believe that they act in the farmer’s best interest.

Training is a catalyzer for change. Training is strongly associated with practice adoption at the farm level and with improved governance and transparency at the cooperative level. While training has been reaching more farmers, it is still limited to a few, as only 30% of target farmers received training (and 45% of control farmers). Among the main reasons for not training most farmers we have:

• Training restricted to a few locations (for budget reasons), making it too costly for some farmers to attend

• Language barriers as trainings are mostly held in English / Swahili, and some farmers do not feel comfortable as they mostly speak their local language

• Materials are not clear enough for farmers

• Farmers are culturally resistant to change, therefore to trainings

• Farmers’ expectations around trainings are not always met

Improving the scope of training may be important, as more farmers will be more likely to apply good agricultural practices and see their productivity improve. Furthermore, cooperatives can strengthen even more their position among farmers and with marketing agents.

63

Research Question 254 55 56 Do different types of farmers such as those with different initial assets, poverty levels or gender, experience differing changes in outcomes over time and what is the degree of difference? In general, the starting point matters. With equalizing training (reaching poorer farmers), we have observed that the number of practices adopted by farmers have equalized along the asset and income distribution. However, when assessing the percentage of farmers using more than five practices (more engaged farmers), we have observed that while changes have occurred at all levels of the distribution, differences across groups remain the same. Yields have also exhibited larger improvements in the lower end of the income and asset distribution, suggesting that the implementation of very simple practices among the lower performers may cause greater changes. This is also supported by the fact that poorer households have shown larger increases in cost of production, implying that efforts have been made that were compensated with improved performance. Finally, we have observed that producer organizations have focused their attention on poorer farmers to provide services, which improved the cooperative’s credibility amongst farmers. Changes have accordingly been more marked for poorer farmers in the target group. Training equalizing among target segments. Training in the baseline was concentrated on wealthier and younger farmers. Such differences tend to significantly reduce (or disappear) at the endline, especially in the target group. Unevenly distributed training may exacerbate differences amongst farmers as it is associated with enhanced productivity and quality.

Table 7.1 Training (percentage of farmers)

54 In this section we have just considered a simple difference in difference with no matching or control for other variables as we are partitioning the sample to assess any potential differences based on the starting point. 55 We use income terciles to strengthen our results, given the limited sample size we are working with. 56 See Appendix 7.2 for a detailed assessment of all the indicators

Baseline Endline Baseline Endline

1 13% 20% 28% 43% -7%

2 32% 35% 36% 45% -5%

3 35% 43% 40% 44% 5%

1 12% 36% 45% 39% 30% **

2 24% 27% 28% 49% -18%

3 33% 31% 31% 46% -18%

Male 21% 29% 42% 42% 9%

Female 27% 32% 32% 45% -8%

Young 27% 30% 32% 42% -7%

Old 13% 33% 57% 57% 20%

Target Control Difference

% of farmers trained

Asset tercile

Income tercile

64

Equalizing training brings about equalizing practice adoption at early stages. In the baseline, we observed a strong correlation between the number of practices implemented in the coffee farm and its assets or income tercile, that significantly reduces at the endline, especially in the target group. In this sense, we can say that poorer farmers performed better than better-off farmers did. However, when looking at the percentage of farmers using more than five key practices, differences remain between asset and income terciles. This suggests that training is not enough to overcome initial differences when the threshold of analysis is rather high. Regular, simple and understandable training performed by trustworthy trainers, may contribute to improve outcomes especially at lower levels of the distribution.

We do not observe differentiated change between age and gender groups as both have exhibited similar growth. Notwithstanding, when assessing changes in the percentage of farmers applying five or more practices, we have observed that female-headed households and younger farmers are more likely to improve.

Table 7.2 Number of practices and percentage of farmers applying five or more practices

Yields have improved and production costs have also increased at lower levels of the distribution. We observe higher changes in the lower asset and income tercile than in

Baseline Endline Baseline Endline

1 2.7 4.0 3.3 4.6 0.0

2 3.3 4.0 3.8 4.7 -0.1

3 3.7 4.7 3.9 4.7 0.2

1 2.8 4.0 3.4 4.4 0.1

2 3.0 4.2 3.8 4.7 0.3

3 3.6 4.4 4.0 4.9 -0.1

Male 3.0 4.0 3.5 4.5 0.0

Female 3.2 4.3 3.8 4.7 0.2

Young 3.1 4.2 3.7 4.6 0.1

Old 3.3 4.1 3.9 4.8 0.0

1 13% 30% 16% 56% -22% *

2 23% 39% 23% 59% -20%

3 35% 57% 34% 60% -4%

1 16% 36% 14% 52% -18%

2 12% 33% 25% 59% -13%

3 33% 47% 38% 65% -12%

Male 21% 32% 15% 50% -24% *

Female 22% 43% 30% 61% -10%

Young 19% 41% 25% 58% -12%

Old 33% 33% 29% 57% -29%

Target ControlDifference

Percentage of

farmers using 5

or more key

practices

Asset

tercile

Income

tercile

Number of key

practices

applied in

coffee farming

(Min 0 - Max 8)

Asset

tercile

Income

tercile

65

the richest segments. This happens in both target and control groups. It appears that improving yields further is more difficult when the yields are higher to begin with, or that with the same effort it is easier to improve yields in the lower end of the distribution. This evidence was confirmed by our qualitative work, as most board members expressed that they have seen important yield and quality improvements, especially among the lower performers, which they attribute to the implementation of simple and basic practices. Furthermore, poorer farmers show higher increases in total production costs, mainly driven by higher use of fertilization and paid labor, suggesting that poorer households have been increasing their investments in coffee production and that such investments have paid off. Despite those efforts, wealthier farmers continue spending three times more per hectare than poorer farmers. On the other hand, we have seen that male farmers and younger farmers have experienced larger yield improvements than their older and female peers, as they have also increased their coffee investments significantly more (more fertilizer application and more paid labor). Age differences are explained by higher education and proneness to participate in trainings. Gender differences are explained by significantly different household size, as typically in female-headed households only 29% live with their husbands, whereas in male-headed households, 95% live with their wives and kids.

Table 7.3 Yields (kg GBE per ha) and production costs (USD per ha)

Baseline Endline Baseline Endline

1 566 713 451 754 -157

2 706 912 437 757 -114

3 820 744 593 767 -249

1 408 595 388 692 -116

2 654 811 462 675 -55

3 879 896 667 915 -230

Male 519 704 473 723 -65

Female 762 833 509 769 -189 *

Young 676 821 485 761 -130

Old 667 601 583 719 -202

1 64 114 93 179 -36

2 159 155 138 203 -68

3 454 436 180 367 -205

1 52 100 90 175 -37

2 72 149 141 245 -26

3 396 333 196 364 -231

Male 158 209 108 254 -96

Female 209 205 152 257 -108

Young 172 208 137 250 -77

Old 287 196 154 296 -232

Target Control Difference

Total monetary cost (USD per

ha)

Asset tercile

Income tercile

Yield (kg GBE per ha)

Asset tercile

Income tercile

66

Coffee investments paid off. Higher yields significantly outweighed higher production costs. This is true for the overall sample, but in particular for poorer households that have significantly reduced the coffee profitability gap with wealthier households. Better coffee performance in poorer households is reflected in better relative performance in poorer households. Furthermore, young farmers have better capitalized training as they have outperformed older farmers in coffee productivity. This is also reflected in total household income.

Table 7.4 Coffee net income (USD per ha) and total household income (USD per member per year)

Baseline Endline Baseline Endline

1 995 1664 882 1536 15

2 1159 2055 830 1406 320

3 1221 1427 1116 1275 47

1 617 1348 754 1412 73

2 1274 1897 886 1201 308

3 1288 1856 1266 1542 293

Male 871 1508 952 1312 277

Female 1237 1864 957 1414 170

Young 1119 1820 929 1376 254

Old 1030 1288 1125 1431 -48

1 172 258 140 206 20

2 469 331 265 349 -222

3 447 712 279 419 126

1 85 248 77 216 24

2 181 326 170 288 27

3 656 556 484 513 -128

Male 408 275 189 322 -265 *

Female 297 460 250 335 78

Young 348 419 228 329 -30

Old 278 258 255 344 -109

Control DifferenceTarget

Total household

income (USD per member)

Asset tercile

Income tercile

Coffee net income (USD

per ha)

Asset tercile

Income tercile

67

The number of food insecure farmers has increased and there is no evident correlation with the starting point. Target group farmers have increased their food insecurity levels more than the control group by a margin of a few days. The slightly more affluent of these farmers (all are below the poverty line) exhibited larger decreases in food security that the data does not explain and would merit further inquiry during a subsequent field visit. However, we do not observe a similar pattern in the control group. Table 7.5 Food security (% of farmers)

More services to poorer farmers result in improved trust and fidelity with the producer organization. Data shows that service provision in the last three years has been more oriented to poorer farmers. This correlates with more farmers considering that the cooperatives provide valuable services to their members’ coffee farming operations. As mentioned before, more trust in the cooperative is also correlated with a higher rate of practice adoption. The more trust farmer have in their organization, the more likely they are to be convinced that the set of practices promoted will contribute to their production objectives.

Baseline Endline Baseline Endline

1 60% 50% 41% 52% -21%

2 55% 61% 57% 67% -4%

3 91% 65% 73% 79% -33% **

1 52% 56% 39% 52% -9%

2 67% 58% 62% 63% -11%

3 75% 58% 75% 88% -29% **

Male 65% 38% 37% 68% -57% ***

Female 67% 68% 66% 66% 2%

Young 65% 53% 53% 64% -22% **

Old 73% 80% 86% 82% 10%

Target Control Difference

Percentage of food secure households

Asset tercile

Income tercile

68

Table 7.6 Producer organizations

Research Question 3 Can any observed changes in farm or PO performance be attributed to the combined Fairtrade and UTZ standard systems? The nature of the intervention allowed us to develop a mixed-methods approach that combines quantitative and qualitative tools to identify a causal chain consistent with the intervention’s theory of change. The intervention focuses on few units of assignment posing a small “n” problem that limits the power of the quantitative impact estimations (See section 2 for further explanation). Furthermore, we have observed important contamination in the control group as farmers received similar training from other institutions. This is where qualitative tools play an important role when combined with quantitative evidence to better identify attribution following the causal chain, consistent with the intervention’s theory of change.

Baseline Endline Baseline Endline

1 32% 81% 47% 84% 11%

2 31% 89% 66% 91% 33% **

3 89% 82% 66% 89% -30% **

1 48% 91% 51% 92% 2%

2 39% 86% 67% 85% 29% **

3 48% 77% 67% 89% 7%

Male 32% 79% 42% 84% 4%

Female 53% 87% 69% 90% 13%

Young 46% 84% 59% 88% 9%

Old 36% 83% 70% 93% 24%

1 0.1 1.0 0.5 1.5 -0.1

2 0.5 1.3 0.9 1.5 0.3

3 0.8 1.3 0.8 1.5 -0.2

1 0.4 1.2 0.7 1.6 -0.1

2 0.3 1.0 0.8 1.5 0.1

3 0.5 1.3 0.8 1.5 0.1

Male 0.3 1.2 0.6 1.3 0.2

Female 0.4 1.2 0.8 1.6 0.0

Young 0.4 1.2 0.7 1.5 0.0

Old 0.1 1.1 1.0 1.6 0.3

Target Control Difference

Number of services

delivered by the producer organization

(Min 0 - Max 5)

Asset tercile

Income tercile

Percentage of farmers that

considers their producer

organization is valuable

Asset tercile

Income tercile

69

Our quantitative assessment finds no evidence of impact of the intervention when assessing a matched differences-in-differences approach57. However, we were able to identify three key issues:

(1) There is strong evidence of changes over time in the target group in key outcome variables. Changes over time seem to be consistent. Data shows that training has led to improvements in practice adoption, which are strongly correlated with improvements in coffee productivity and quality. Such changes have impacted net income from coffee production, and to a lesser extent total household incomes. Poverty levels have not moved as the difference between actual incomes and the poverty line is still rather large. On the other hand, training at the producer organization level has improved governance and democracy, as well as service delivery. This was reflected in improved farmer valuation on the role of the cooperative.

(2) Training has been the trigger for such changes. Data shows that training is strongly correlated to practice adoption.

Following the intervention’s causal chain, there is evidence that training itself is important for improving relevant outcomes (practice adoption, yields, income from coffee, household incomes, amongst others). While training towards certification cannot be strongly differentiated from other local training initiatives, we can assert that the training causal chain has partially achieved its goals. Unfortunately, the project never managed to be completed. Information gathered during the scoping trip for the end line phase of the project showed that the intervention did not fully accomplish the objectives of certifying target cooperatives in both Fairtrade and UTZ standards. While both target farmer organizations received training towards certification, only one managed to become certified, and only with Fairtrade. Unfortunately, after one year of being Fairtrade certified, the producer organization decided to switch marketing agents, and to stop paying for certification fees, thus losing the certification. During the period in which they had Fairtrade certification, unfortunately CMS was not able to sell the coffee as certified, thus the target cooperative did not receive any price premium. This was a big part of the reason why the target group decided to stop working with CMS, as they considered they did not make any effort to do so. This fact does not allow assessment of the role of certification as a market developer for coffee farmers.

Research Question 4 What is the added value that Fairtrade and UTZ standards systems bring to POs, farms, and households, beyond training? This will include but not be limited to examining the extent to which farmers and PO managers feel satisfied with the experience of certification (in terms of challenges and cost-benefit perceptions). The target producer organization received training towards Fairtrade and UTZ certification, and achieved Fairtrade certification. Unfortunately, after one year of becoming certified, the producer organization decided to stop working with CMS, and relinquished the certification. In both cases, the lack of market incentives for certification contributed to their choice of marketing agent. Farmers and board members were nevertheless highly satisfied with the outcomes of the training they received. When they became certified, they had high expectations of selling

57 We tested several approaches for matching (PSM nearest neighbor, kernel and radius matching as well as entropy balancing techniques). All of them result in relatively similar outcomes.

70

their coffee as Fairtrade. Unfortunately, when it did not happen, it severely disappointed both farmers, board, and PO management. The market for Kenyan Fairtrade certified coffee is limited and has been shrinking over time. Given that there was no market for their product, they decided to sell it as conventional, looking for the best price. With a market with limited demand for certification, we have observed that there is a problem of expectations and transparency. There are several information gaps between the farmers and board members, so they act based on expectations and limited understanding of market dynamics. On the other hand, there is not sufficient transparency down the value chain as Board members claim the agent does not inform them and they have little knowledge of the actual market demand and prices for certified products. The expectations and inaccurate information circulating in Kenya about various aspects of certification is detrimental to all. It may not be feasible for the standards bodies to increase their presence in the country to alleviate this information vacuum. They might be able however to nominate or support local affiliate organizations that can foster conversations, local media, and exchanges with farmers and FCS and help ensure that the domestic information about certifications, their processes, market conditions, and farmer-level engagements serves everyone better.

Research Question 5 What contextual factors significantly influence the effect of Fairtrade and UTZ standards systems on PO, farm, and household changes in performance? The factors to test for influence are: the market orientation of the program, Kenyan and global coffee prices, the PO management and structure, livelihood and poverty context, cultural context, and project implementation in practice Project implementation matters of course. In this case, the intervention did not fully accomplish the objectives of certifying target cooperatives in both Fairtrade and UTZ standards. While both target farmer organizations received training towards certification, only one managed to become certified, and only with Fairtrade. Unfortunately, after one year of being Fairtrade certified, the producer organization decided to switch marketing agents, and drop the certification. Market matters. When the producer organization got certified, they had high expectations of selling their coffee as Fairtrade. However, given that there was no market for their product, they decided to sell it as conventional, looking for the best price, generating frustration amongst farmers and board members. Certification, offered through marketing agents as the key channel of transmission for training farmers and cooperative representatives, faces several challenges to ensure long-term continuity. First, there is no long-term relationship between marketing agents and cooperatives. There is strong competition between marketing agents to contract with cooperatives and such contracts typically last one year. This provides incentives to marketing agents to persuade farmers and board members offering a set of services and better prices to get the business. The unstable relationship between cooperatives and marketing agents may limit the ability of certification bodies to engage in the market. Second, institutional weakness and limited information about certification in farmers and producer organizations does not help acknowledge the value of certification, beyond the role of marketing agents. Third, limited market for Fairtrade certified coffee significantly affects appropriation from cooperatives. Considering producers typically seek better prices (even though there are other benefits such as quality and productivity improvement of training towards certification), no premium (no market) and minimum price significantly below the market pose limited “value” for board members. Lastly, while most key informants acknowledge certification as a key service offered by a marketing agent to improve quality

71

and productivity, farmers and producers consider mainly short-term prices and services as the key drivers for decision-making.

Research Question 6 What are the reasons that different types of farmers (for example, those with different initial assets, poverty levels, or gender) experienced different changes in outcomes, if any such differences are identified in the quantitative analysis?

We found that training was an equalizer. The number of farmers trained increased more in poorer households. Furthermore, equalizing training brings about equalization in practice adoption, although differences remain in the percentage of households using a large number of practices. Finally, equalizing practices implemented in coffee plots produced significant increases in yields. (See Research Question 2). Quality has exhibited signs of improvement in both target and non-CMS control, partially due to the training received – according to key stakeholders, and farmer’s focus groups. This was reflected in the higher prices received. On the other hand, CMS control did not show any signs of quality improvement.

72

Conclusions

Although the expected combined effect of the Fairtrade and UTZ standards did not materialize, there was sufficient achievement in the preparation and in achieving Fairtrade certification that warranted understanding how this affected these small farmers and their organizations and how the process works in Kenya. This study focuses on the most prominent impact pathway that emerged: the effects on farmers’ livelihoods of preparation towards Fairtrade and UTZ standards and being certified for one year. Target and control groups have received some fairly similar training in both delivery (group-based training with demonstrative parcels), and topics (focus on coffee farming operations). While quantitative tools are not qualified here to assess attribution, we rely on a combination of methods, using the quantitative data to inform our qualitative assessment and vice-versa. We found important differences in a before and after assessment of the target group. However, similar changes have also occurred within the control group for many of the factors measured. While this does not at all negate the achievements of the efforts related to certification, and there is clear evidence of their contribution, it makes it impossible to attribute the outcomes solely to those. At the producer organization level, we have observed an improvement in governance and transparency, as well as in the extent to which services reach farmers. This conclusion is backed not only from our qualitative assessment, but was also validated by farmers. These changes were not exclusive to the target group, as the control organizations have also experienced similar changes. The common leading factor triggering change at the producer organization level is the training received by board members and managers. At the farm level, we have observed important changes in some environmental effects (e.g. shade tree management and basic integrated pest management tools) and agronomic practices (application of fertilizers and pesticides). These changes are consistent with the intervention’s theory of change. However, these changes also occurred to the control groups’ farmers. The importance of training to practice adoption was also verified using econometric modelling calculating the probability of using a specific practice that a farmer was trained on. Building on this modelling perspective, we also found that training in all cooperatives has been biased towards younger, more educated and wealthier farmers living close enough to training locations. Changes in practices have also triggered change in yields. Both target and control farms have significantly increased yields. Changes in yields at the target group are consistent with the theory of change which expects that practice adoption will lead to higher yields. The correlation between training and higher yields has also been assessed using simple econometric modelling and qualitative tools. We found strong evidence supporting the fact that higher practice adoption leads to higher yields. We found an important increase in overall coffee quality, expressed also in higher prices for farmers. Changes in quality happened in the target group in full harmony with the expected causal chain. Our field assessment showed that higher prices were not common to all producer organizations. Non-CMS control did not show any sign of higher quality at the farm level. This information was confirmed by statements from board members. This implies that target farmers are taking good advantage of trainings, resulting in improved yields. In summary, training has promoted the application and use of some basic good agro-economic practices amongst farmers. This has in turn improved productivity and quality for most farmers. Higher productivity has led to increased production (given the same coffee area), and improved quality has led to better prices. Higher revenues are the corollary of this effect. Similarly, the adoption of such practices has led to significant increases in

73

production costs, which have been overcome by the increased revenues, making the investment much more profitable for farmers. As a result, net income from coffee farming has doubled in both target and control groups, implying no differentiated effects from training towards certification as compared with regular training offered in the region. Furthermore, we have found evidence of statistically significant changes in yields and net incomes from coffee farming at the lowest levels of the distribution. This suggests some greater effect on the poorest amongst the target group. Important changes have occurred in the Mount Elgon region between 2014 and 2017. Net income from coffee farming has almost doubled, and total incomes have also increased. However, there is still much room for improvement, as practice adoption is really incipient and there is high potential for increased productivity and quality. Nevertheless, we should be cautious about ascribing coffee a strong role in poverty alleviation as the poverty gap remains rather large, and the growth potential is limited by small holdings and the persistently low market prices cross many years.

~ ~ ~