4Mj- Consultative Group onlqpl International Agricultural Research

To Iy lqqq

The Impact of the International

Agricultural Research Centers:

Measurement, Quantification,

and Interpretation

M. P. Collinson and E. Tollens

ISSUES IN AGRICULTURE 6

1M1p 1y 1p 14V Msp-Msk sp

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

"Issues in Agriculture" is an evolvingseries of booklets on topics connectedwith agricultural research and devel-opment. The series is published bythe Secretariat of the ConsultativeGroup on International AgriculturalResearch (CGIAR) as a contributionto informed discussion on issues thataffect agriculture. The opinions ex-pressed in this series are those of theauthors and do not necessarily reflecta consensus of views within the CGIARsystem.

Published by the Consultative Groupon International Agricultural Research,CGIAR Secretariat, 1818 H St., N.W.,

3 Washington, D.C., 20433, UnitedStates. (1-202) 473-8951; Fax (1-202)473-8110, July 1994.

The Impact of the InternationalAgricultural Research Centers:Measurement, Q3uantification,

and Interpretation

M. P. Collinson and E. Tollens

ISSUES IN AGRICULTURE, NO. 6ConsultativeGrl... , A.g.r.i t ..... R e.......................R.es.er.

,9 *ZS000gg00|0|00 s- B B',.,.'., ............... .. .... . .............. ,.SEEg 'B ::

...... '" ss osf0 ...... =D . ..

public and ivteetordnor......... .. .s p....prts a. neto r

GroupS B Svo' ::tb:ihe in 1971 BSB

SSSSSSS:2 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ . ..... . . ......

IkJ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~. g. .. .. .. .. .. -.. .. ... .. .. ... .. ..

Th Wol Bankg . te Ioo an AgriclueOgntio ol,e Nfa..cff.Xtin - - -, a. tg -n.te ars,

iE (UNDP) gEare cspnor f h

CGAR Th ifma fte ru sa seio ofica ofthe ~ which piovids teOIARsysemigaGB wit 'a'

Serearatn.alingo 5C The: OGIA:R is ass::ise byg:g afTiei & 9- A= iugittee,5 withggggiii,i a_ Secretariatg atF g-A B B

' .=GWeW'aa ,,DsadCaaaaeth edn

.,W

4o~ cBut:: :::el by>a5f.,.,<: seRvral,,5R,5>>.f:A5 7,.gg --Euv , e

contis. Dvlig N conymmeso h GA rCh=-- , ;>ga::R-azl,sg hI4iafaBX'Bg hi4nesa.R MexicoR Nigeri. the

ContentsPage

Acronyms and Abbreviations Used in This Paper ........ iii

Introduction ............. ............................... 1

History of Impact Assessment in the CGIAR ................ 1

The Research and Development Process and IARCs ..... 3

The Development Process ...................................... 4Implications for CGIAR Impact ................ .............. 6

A Conceptual Framework for Impact Assessment ........ 7

The Research Cycle and Feedback ............ ............. 7Institutional Impact ............................................ 10Scientific Impact ............................................ 10Impact Independent of Farmer Decision Making.. 11

Impact in Farmers' Fields and Beyond:Levels and Methods of Assessment ............. 1............. 11

Evaluation on a Farming System Level ........ ........ 12Evaluation on a Global Level ............................... 14Economic Models ........................... : 16

Challenges and Strategies for IARCs .......................... 17

Planning .............................. 18Donor Needs .............................. 19

Conclusion .............................. 21

References .............................. 21

i

e.>... .. .::: 1 ...

Acronyms and AbbreviationsUsed in This Paper

CGIAR Consultative Group on InternationalAgricultural Research

CIAT Centro Internacional de Agricultura Tropical

CIIFAD Cornell International Institute for Food,Agriculture, and Development

CIMMYT Centro Internacional de Mejoramientode Maiz y Trigo

CIP Centro Internacional de la Papa

IARC International Agricultural Research Center

ICARDA International Center for Agricultural Researchin the Dry Areas

ICLARM International Center for Living AquaticResources Management

ICRISAT International Crops Research Institute forthe Semi-Arid Tropics

IFPRI International Food Policy Research Institute

IIMI International Irrigation Management Institute

IITA International Institute of Tropical Agriculture

ILRAD International Laboratory for Research onAnimal Diseases

IRRI International Rice Research Institute

ISNAR International Service for NationalAgricultural Research

NARS National Agricultural Research Systems

R & D Research and Development

TAC Technical Advisory Committee (of the CGIAR)

USAID United States Agency for InternationalDevelopment

The Impact of the InternationalAgricultural Research Centers:

Measurement, Quantification, andInterpretation*

M. P. Collinson' and E. Tollens 2

Introduction

The mission of the Consultative Group on Interna-tional Agricultural Research (CGIAR), with its seven-teen international agricultural research centers, is toimprove the welfare of poor people in developing coun-tries in ways that also improve the future productivecapacity of their natural resources while protecting thequality of our wider environment. Impact studies helpus to understand how technology influences agricul-tural production and the welfare of agricultural produc-ers and consumers. This information, in turn, can beused to improve the efficiency of resource allocation forresearch at the international agricultural research cen-ters (IARCs). Many practical obstacles must be over-come, however, before such studies can fulfill theseroles.

This is an abbreviated version of a paper of the same title bythe authors which is forthcoming in Experimental Agricul-ture (1994) Vol. 30. Experimental Agriculture is publishedby Cambridge University Press. We are grateful for permis-sion to reproduce significant parts of that article.CGIAR Secretariat, The World Bank, Washington, D.C.

2 Department of Agricultural Economics, Catholic Universityof Leuven, Leuven, Belgium.

Impact studies have another benefit. Recent re-ductions in funding for international agricultural re-search have created new pressures on the centers todemonstrate the impact of their results. Evidence ofsuccess, provided by impact studies, helps donor rep-resentatives defend CGIAR funding in their domesticbudget process. In a time of recession, when burgeon-ing environmental problems and the breakup of theSoviet Union bring new demands on aid, donors needstronger and clearer evidence of the value of theirinvestments in the CGIAR.

History of Impact Assessmentin the CGIAR

In 1979, eight years after its founding, the CGIARsponsored a review of impact assessment methods andresults (Scobie 1979). Scobie found that the high-yieldvarieties introduced in the mid- 1960s benefited mainlylow-income consumer groups. He also concluded,however, that they were not an effective means toredistribute incomes among rural groups in whichproductive assets are not equitably distributed. Hefurther concluded that international investments inagricultural research could be expanded significantlyand maintain an attractive economic rate of return. Theliterature on impact assessments that has accumu-lated since then supports his conclusion (Evenson1992).

Before 1985, impact assessments in the CGIARsystem were dominated by studies on the short-strawedrice and wheats developed by the International RiceResearch Institute (IRRI) and Centro Internacional deMejoramiento de Maiz y Trigo (CIMMYT), part of the vastbody of literature on the 'green revolution." Usingnational-level statistics, the United States Agency forInternational Development (USAID) documented the

2 development and global spread of high-yield varieties of

BE^G': .ffi..6.M.^6 .X.. .B..sS.BB .. .a.>...........M .. .... .... .. t . ... .... .... iiE .........

rice and wheat. Such publications began in 1969 withdata from the 1965-66 crop year and continued through1986 (Dalrymple 1986a, 1986b). These USAID-sup-ported studies were later extended to hybrid maize(Timothy, Harvey, and Dowswell 1988). CIMMYT hasrecently updated the data for maize and wheat (CIMMYT1992), and updated figures for rice and wheat show theextensive impact of the short-strawed materials.

Today the developing countries produce some 460million tons of paddy rice annually from more than 140million hectares, two-thirds (67 percent) of which isplanted with seeds based on IRRI's semidwarf materi-als. Asia produces 91 percent of this total, and riceprovides between 35 and 60 percent of householdcalories for its 2.7 billion people. Similarly, the devel-oping countries grow some 226 million tons of wheatfrom 100 million hectares, the seed for more than 60percent of which is based on CIMMYT semidwarf mate-rial.

The benefits of such advances in research continueto multiply long after the initial breakthrough. Duringthe two decades of the 1970s and 1980s, the rice yieldin eleven green revolution countries in Asia increasedby 63 percent, from 2.03 tons per hectare at the begin-ning of the 1970s to 3.31 tons per hectare by the end ofthe 1980s. In the 1980s, wheat yields in developingcountries increased by 37 percent, from an average of1.64 tons per hectare to 2.24 tons per hectare. Fourdimensions of the diffusion process for research andtechnology sustain the flow of benefits:

* Farmers using the new technology get higheryields year after year.

i Further adaptations continue to raise yieldceilings.

* Adaptations extend the technology to farmersgrowing the crop under different soil and waterconditions. 3

... . . : B. . . .. . ..

Adaptations extend the crop to areas previouslyunsuited to it, creating a new cropping opportu-nity for farmers working there.

Studies by IRRI have shown that poor urban andrural consumers have benefited from the reduction inthe real price of rice caused by higher production.Benefits have spread beyond the irrigated areas. As aresult, labor demand in new areas has brought immi-gration of labor, thereby helping to equalize wage ratesacross environments (David and Otsuka 1991).

During the period 1984-86, the CGIAR donorsfunded a major study of IARC impact (Anderson, Herdt,and Scobie 1988). It was supported by twenty-sixmonographs, including case studies (mostly qualita-tive) of IARC impacts on individual countries, threeregional studies, and other studies on topics of specialimportance to the donors. A parallel study on theimpact of CGIAR training in developing countries wasalso published by the CGIAR Technical Advisory Com-mittee (TAC) (TAC 1986).

These studies will not be repeated. In the future,the main responsibility for impact assessment will restwith the centers themselves. Most IARCs, however, arenot organized for systematic assessment. Self-select-ing successes often become the focus for ad hoc studies;cases of negative returns to research investments aredownplayed or obscured (Anderson and Herdt 1990).Some CGIAR centers have revised their impact assess-ment needs, responding to the pressures of five-yearexternal reviews, growing constraints on funding, andthe adoption of more formal management processes ina search for greater efficiency. For example, both theInternational Center for Agricultural Research in theDry Areas (ICARDA) and International Crops ResearchInstitute for the Semi-Arid Tropics (ICRISAT) recentlyrecruited agricultural economists to study impact as-

4 sessment.

Impact assessment continues to pose major re-source allocation dilemmas for individual centers andfor the CGIAR system as a whole. Full-blown impactassessment of all its research products, in collabora-tion with its diversity of clients, may require a doublingof the total CGIAR budget.

The Research and DevelopmentProcess and IARCs

Three levels of activity characterize agriculturalresearch:

* strategic research, which is mission focused,seeking to understand those natural and humanprocesses identified as important to the solutionof a specified problem;

* applied research, which uses existing knowl-edge to identify approaches and develop proto-type technologies to solve problems of wide-spread importance; and

- adaptive research, which aims to articulatefarmers' problems, identify appropriate ap-proaches to solution and relevant prototype tech-nologies, and fit these to the particular circum-stances of a specific group of farmers.

Another category of research adds to the body ofknowledge available to be used, when relevant, in thedesign of prototype solutions. Basic research, oftentermed "blue sky" research, has no specific problemfocus and is not found in the CGIAR.

Figure 1 shows a research paradigm that linksthese three levels and follows the "farmer back tofarmer" configuration. In this model, farmers' problemsare identified in the diagnostic stage of the adaptiveresearch cycle, ideally by using methods that involvethe farmers themselves. Many of these problems can be 5

Figure 1An idealized research paradigm(adapted from CoUinson, 1982)

Target group farmersin a region

(1) Survey diagnosis of ON-FARM (3) Experiment onfarmer priorities, apparently

resources and environment ADAPTIVE relevant materialsproblems RESEARCH and technology under

and development farmers' conditionsopportunities

(2) Identification and evaluationof materials and techniquesoffering potential for problem

or opportunities

(4) Unsolved technical STATION- (6) Body of knowledge ofproblems and possible BASED materials and techniques

new practices and TECHNICAL suitable for the climatematerials relevant RESEARCH and soils of the region

to farmers' developmentopportunities

(5) Commodity and disciplinaryresearch, solving prioritytechnical problems and

investigating possible newmaterials and practices

(9) Principles,

(7) Transitional NTERNATIONA and methods forproblems often RESEARCH manipulation of

poorly understood researched phenomena

(8) Strategic researchunderstanding natural

6 phenomena

,$Du~~~~~~~~~~ ............ gD

solved within a two- to five-year research cycle byadapting technologies that are available from previousformal research or from the experiences of other groupsof farmers.

Some problems need new options and prototypes.If they are important to enough people, they find a placein the applied research agenda at a national or regionallevel: Problems whose underlying processes are poorlyunderstood also find priority in the strategic researchagenda-again, if the problems are important to enoughpeople. At the strategic level, many such problems arerelevant to a number of countries and are researchedmost cost effectively at the regional or internationallevel. These cycles of applied and strategic research willoften occupy a ten-year period.

Perhaps the greatest challenge facing the agricul-tural research community is to build the capacity tooperate this research paradigm effectively, in partner-ship with resource-poor farmers in developing coun-tries.

In practice, these three levels of research do notfollow as neat a sequence as Figure 1 suggests. Existingknowledge often can offer some solution. Yet it isfrequently clear to researchers that better solutions arepossible. Choices among options, rather than one finalanswer, are increasingly important. The more optionsthat are available, the greater the chance that one willfit the circumstances of any given farming system, andthat farmers will readily adapt it to their needs.

Supplying nitrogen to plants is an example. It canbe done in many ways: by moving to new land; rotatingwith green manures; pumping nutrients with legumi-nous trees; adding compost or organic animal manure;adding inorganic nitrogen out of the bag; and, for sometypes of plants, fixing nitrogen from the atmosphere. Ifnitrogen fixation ever can be introduced to other planttypes, it will be a major research breakthrough. Thiseffort has already spanned decades but is still pursuedas another option, perhaps a superior one, for farmers. 7

The Development Process

Once appropriate technology is available, otherresearch and development (R & D) sectors need tomobilize its diffusion. It takes time for innovations tospread across target communities, through farmer-to-farmer contact or even an efficient extension service.Farmers themselves will usually experiment on smallparts of their fields until convinced of a novel technology'svalue to them. Making new methods accessible tofarmers may sometimes require making credit availableso they can afford the purchase. This demands effectiveenabling institutions and innovative rural banks. Adop-tion by 80 percent of farmers is often assumed as aceiling, and it may take a decade to achieve this degreeof acceptance. Thus, when new knowledge is neededfrom strategic research, a twenty-year period is notunusual from the initiation of research to developoptions to full benefits of results by farmers.

Many factors inhibit performance in the other R &D sectors, and IARCs sometimes invest in solutions tosuch inhibitions when it seems important to theirresearch interests. Where the market niches are toosmall to attract commercial seed producers, for ex-ample, Centro Internacional de Agricultura Tropical(CIAT) has had success promoting skilled local farmersas bean seed suppliers to their communities. Vegeta-tive propagation at Centro Internacional de la Papa(CIP) has helped promote local production of improvedmaterials with potatoes, and at the International Insti-tute of Tropical Agriculture (IITA), efforts to use cassavaseed for propagation also aim to reduce bottlenecks onthe spread of improved cassavas.

For roots and tubers that are usually propagatedvegetatively, low multiplication rates make diffusion of

8 *All dollar amounts are U.S. dollars.

new plant material particularly slow. Thai scientistsselected a CIAT-developed cassava clone, CM 407-7.After several years of testing, it was released as Rayong3 in northern Thailand in 1984. Innovative farmerswere supplied with 600 stakes each and gave 80 percentof harvested stakes to neighbors. The area for stakemultiplication was only 16 hectares in 1986. Eventhough it is estimated that some 70,000 hectares wereplanted with the new material in 1990, this is still only4 to 5 percent of the cassava area in Thailand. Adoptingfarmers get 10 to 15 percent more revenue based on a5 percent starch premium and in 1990 earned anestimated $3.8 to $4.6 million30 ' in extra income. In thelong run, Thai cassava will become more competitive inthe European market from this kind of innovation(Henry 1991).

Extra production from such innovations may cre-ate surpluses where markets are limited. Research cancreate new market opportunities. Integrated cassava-drying projects in Latin America have been promotedand supported by CIAT and other R & D institutions.From a beginning in 1982 with a single factory for driedcassava chips for animal feed in Colombia, this indus-try grew to 153 factories in five countries by 1990.Product differentiation - adding dried cassava foranimal feed to a market exclusively for fresh cassava forhuman consumption -created a wider cassava marketwith more stable prices, thereby stimulating the adop-tion of dried cassava processing technology.

In Colombia, introduction of improved productiontechnology has been integrated with the cassava-dryingplants in the expectation that a more stable marketwould encourage farmers to increase and intensify theirproduction. Between 1982 and 1990, the proportion ofcultivated land devoted to cassava quadrupled amongproducers in the Cordoba area, and areas of fallow and 9

yams were reduced. The value added to small farmers'incomes from dried cassava production in 1990 wasestimated to be $6.6 million, with $1.4 million to localprocessors: reduced imports of sorghum accumulatedsavings of $6.0 million of foreign currency by 1990. InColombia, processing each 1,000 metric tons of driedcassava is estimated to generate 185 person-years ofdirect labor and 37 person-years of indirect labor (Henry1991).

Although, as these examples show, centers haveintervened in the wider sectors of R & D, many bilateraland multilateral development agencies are active inthese sectors. It is clear that the CGIAR, with a budgetrepresenting 6 to 7 percent of total developing countryagricultural research budgets, has a real comparativeadvantage only in a limited sector of the sequence.Because the centers' global and regional mandates arefor strategic and some applied research, many of theresults are intermediate products, to be shaped byfurther applied and adaptive research to the circum-stances of the markets formed by diverse groups offarmers within each country. National agriculturalresearch systems (NARS) have a clear advantage in thisapplied and adaptive research; at the same time, im-proved contacts between NARS and farmers betterinform the research agenda for IARCs.

Implications for CGIAR Impact

This role in providing many countries with unfin-ished intermediate products has two key implicationsfor the assessment of IARC impact. First, NARS form ageographically widespread and diverse set of clients.CIMMYT, perhaps the most global of the centers, inter-acts with up to 100 countries every year. Second, IARCsare highly dependent on the performance of other

10 institutions for successful impact in farmers' fields: on

ii B:. } omB....'...>-gitiiiiEEEE-EB-

the extension services, on available infrastructure, onmarket access for both products and for inputs, onpolicies and, most immediately, on NARS.

The stronger NARS benefit most from CGIAR inter-mediate products (Anderson, Herdt, and Scobie 1988).Weak NARS are unable to adapt IARC products andunable to feed back information to help formulate arelevant international research agenda. The CGIARacknowledges that practice falls far short of the R & Dsequence idealized in Figure 1; its mandate includesbuilding national capacity to organize and operate aneffective research process. Some 20 percent of CGIARresources are invested in capacity building with NARS.This role involves CGIAR scientists in training, develop-ment of methodology, and collaborative research withnational scientists, including adaptive research whenthe aim is a better interaction with farmers. In Africa inparticular, IARCs are drawn into applied research andadaptive research, because NARS of some countrieshave little capacity for these activities. Yet the realcomparative advantage of an international effort re-mains in strategic research relevant to the problems ofmany countries. The fact that IARCs play a role only ina narrow sector of the total R & D sequence is oftenoverlooked by donors eager for evidence of impact infarmers' fields.

A Conceptual Framework forImpact Assessment

The CGIAR centers have two broad categories ofimpact. They have direct impact on production, con-sumption, and human welfare. They also have indirectimpact on the research capabilities of NARS and univer-sities in developing countries, and on the general un-derstanding of nature, enlarging the scientific stock ofknowledge. Although CGIAR centers have a large effecton building scientific capacity in universities, this is anoften neglected aspect of their work (Wilson 1989). 11

The framework in Figure 2 illustrates the diversityof products from IARCs and the complexity of the chainof repercussions resulting from farmers' adoption ofnew technologies. Some indicators listed merely recordimplementation; others quantify its effects.

The Research Cycle and Feedback

The conceptual framework shown in Figure 2 isrelevant to both planning the allocation of researchresources and the subsequent ex-post evaluation of theimpact of research products. The current drop in CGIARfunding has increased the need to choose among re-search initiatives and to explain those choices whenstakeholders, many NARS, and many donors seek cen-ter expertise to address their priority problems. As aresult, IARCs have growing interest in planning re-source allocation.

Setting priorities involves comparing the benefitsexpected from alternative research initiatives with theirestimated costs and the probability of success. Oneway to measure potential benefits is to estimate currentlosses caused by the various problems under consider-ation. Several IARCs, including CIP, ICRISAT, and theInternational Center for Living Aquatic Resources Man-agement (ICLARM), have made such calculations inpreparing new 1994-98 five-year budgets. CIP hasrecorded and published its process of priority setting(Collion and Gregory 1993).

Solving a research problem is often a continuingprocess. New knowledge from strategic research allowsmore options and more effective technologies. Theadoption of a succession of improvements graduallyreduces the available benefits identified in the initialassessment. Although the greater understanding de-rived from continued strategic research will usuallyyield additional benefits, at some point the resourcesinvolved-the scientists and funds-would bring greater

12 benefits if they were applied to another problem. This

Agricultural research in CGIAR Centres

Indirect impact Direct impact C

Institutional impact Scientific impactProduction, consumptionand human welfare

Impact and effect

Building capacity in NARS Advances in science and technology With farmer adoption of Without farmer adoptionand universities -fundamental research: increased new technologies -biological control of pests

-training knowledge and understanding -improved food policies-networking -strategic research; new and v-collaborative research improved research tools and I-improved research process approaches r Improve quality of manpower Additions to body of knowledge Increased production and/or higher More sustainable productionGreater effectiveness of quality production and/or more stable (improved resource management)research institutions production and/or lower cost production -resource conservation

-reduction of pesticides 0-improved environment quality 0

Measurement of indicators M

-number of people trained: -number and quality of publications -adoption rates of new technologies -reduced use of chemicals, l-number of research reports, -participation in international -improved quality of agricultural products fertilizers and pesticides

journals, etc. scientific conferences -improved farm income -loss of species U2-number of network meetings; -improved nutritional status and -pollution

visits of research workers; -human welfare Iamount of germplasm received;number of joint experiments

Co.

is the issue of the costs and benefits of more informa-tion. A key role for research managers is to identifywhen to switch resources to new problems.

Maintenance research is an exception to this ideaof a reducing benefit pile. Continuing efforts are neededto control expected mutations of disease vectors byidentifying new sources of resistance. With rice, forexample, the fight to control the brown planthopper hasbeen essential to the protection of previous yield gains,particularly in Indonesia. The hopper had always beenpresent as a rice pest, but the denser plant canopy ofthe semidwarf varieties provided a moist, shady envi-ronment that favored it. IRRI collaborated with govern-ments to introduce resistant varieties, the first of whichwere quickly overwhelmed by a second biotype of thehopper. A second round of new materials remainsresistant but will probably provide only temporaryrespite. If such maintenance research stops, the risk ofa food crisis increases.

CIMMYT has estimated that 50 percent of its wheatresearch has been devoted to keeping ahead of mutat-ing pathogens. Genetic studies at the center haverecently identified the basis of a durable form of resis-tance to leaf rust, one of the three major rust diseasesof wheat, an achievement recently confirmed after sev-eral years of testing by the U.S. Department of Agricul-ture (USDA) (Ingersoll 1992). Again, a rough assess-ment is helpful in allowing stakeholders to judgeCIMMYT's achievement. The yield losses that can beavoided if all new wheat materials incorporate this traitshould conservatively total 1 percent of the annual cropvalue, on the order of $135 million each year to produc-ers and consumers in developing countries. CIMMYTwill save on the costs of maintenance research, therebyreleasing resources to address other problems.

A yield loss assessment helps identify the potentialbenefits from strategic research programs that bringnew understanding as the basis for new options to solve

14 a problem of broad scope. Centers can use such

assessment of the numbers of farmers likely to benefitand the estimated level of benefit for each farm toinform, and even excite, their donors, as demonstratedby the following examples.

Nonchemical control of the Mexican bean weevil, apest of the stored bean crop in Latin America and Africa,has been achieved by adding natural resistance tocommercial varieties. Over time, CIAT's work to incor-porate the arcelin gene from a wild bean species intovarieties grown by farmers will save producers hun-dreds of millions of dollars and will also protect theenvironment by offering an alternative to pesticides.

The International Laboratory for Research on Ani-mal Diseases (ILRAD), a strategic research laboratorythat has worked for fifteen years to control theileriosisand trypanosomiasis in cattle, is developing bioeconomicmodels to value the losses from these diseases. Recenttests of the models have estimated annual losses fromtrypanosomiasis in Zimbabwe at $6.2 million. Furtherrefined, these models will be used to estimate conti-nent-wide losses from the disease and to assess theeconomics of alternative control options.

As outlined earlier, the full research cycle, fromspecification of the problem to full farmers' adoption oftechnical solutions, may occupy a twenty-year period-too long for the redesign or adjustment of ongoingprograms. More important to research managers isshort-term feedback to identify new options and toimprove the shaping of technological solutions to farm-ers' circumstances. NARS have always provided feed-back to IARCs on the value of the materials supplied tothem through the international trials networks. IARCs,in their collaboration with NARS, widely promote theneed for on-farm research in which farmers assess newmaterials and practices in their own fields. This re-search is increasingly perceived as the exposure, testmarketing, and adaptation of technology options. Arecent example from Malawi documents farmers' com- 15

'";L'<UR;,~~~~~~~ ............... ,.'.,@T

parisons of maize varieties for traits important to them;these included yield, processing and cooking efficiency,and storability (Smale and others 1993). Such assess-ments provide early information on the acceptability ofmaterials and practices to specific farming communi-ties and on the criteria farmers will use in their evalu-ation.

Recent IARC initiatives have moved this trendfurther by emphasizing the importance of farmer as-sessment as early as possible in the choice and designof prototype technologies, including new plant variet-ies. This approach prevents the waste of scarce humanand budget resources on the development of plant typesthat farmers can easily identify, even in early breedingcycles, as being unacceptable to them.

In a state-of-the-art pilot study in Rwanda, CIATand NARS staff compared results from farmer andbreeder selections of bush beans. They found thatfarmer participants chose breeders' materials that wouldperform well in their own home ecosystems. Thetwenty-one cultivars selected by farmers outperformedtheir local mixtures 64-89% of the time over a four yearperiod, with average production increases of up to 38%.(Sperling, Loevinsohn, and Ntambovura 1993). Eventhough farmers' choices had to meet a number of theirown selection criteria, planted in their local environ-ments, these cultivars still outperformed breeders' se-lections in terms of yield-the breeders' primary crite-rion. In this program, farmers now evaluate cultivarsfive to seven cycles before they would have been ex-posed to breeder-selected materials in conventional on-farm testing.

This pioneering study shows that breeders, select-ing for wide adaptation at on-station sites, cannotcompete with groups of farmers who know their own

16 ecosystems and can recognize key characteristics of

.................' : : "....$,B,,¢ ... '. , , - ' s..... ..... § 'E REf . a.-sF.F. .....iEERE

cultivars likely to fit them. It is a clear step towardsolving the problem of how to meet the needs of the vastdiversity of small, resource-poor farmers in drylandagriculture.

Manufacturing companies in the private sectorcarefully define market niches and create, modify, orrepackage products to fit their targets. These Rwandanfarmers, all women, carried their knowledge of theirlocal ecosystem, and therefore their own selection cri-teria, with them to the research station. They wereencouraged to use their knowledge, and each left witha mixture of bean varieties she judged best for her owncircumstances. More such mechanisms are needed toexpose farmers to technology options that may beuseful to them. Agricultural research planners maketoo little use of the strategy, common in the commercialworld, of a variety of products appropriate to a diversityof market needs.

Institutional Impact

Horton (1990) identifies the principal impact ofIARCs as institutional-creating stronger national agri-cultural research institutions. He includes intermedi-ate technologies as products that strengthen NARS asinstitutions. This definition supports a point empha-sized in the paper: the CGIAR's dependence on theeffectiveness of a range of other institutions, particu-larly NARS. The conceptual framework in Figure 2,however, illustrates the more conventional view thatinstitutional impact improves the functioning of thetarget institutions.

This conventional definition embraces human re-source development through training, a role under-taken by most IARCs in their areas of expertise. Fromthe inception of the CGIAR in 1971 to 1990, IARCs had 17

trained 45,000 developing country scientists. Some25,000 were trained from 1985 through 1989: morethan 9,000 from Africa, 6,600 from Latin America,6,800 from Asia, and 2,700 from West Asia and NorthAfrica. Most training was in short courses of one weekto six months. The five-year total includes 1,012scientists at M.Sc. level, 786 at Ph.D. level, and 348 inpostdoctoral studies. The data show how many at-tended courses; the impact of this major effort is muchmore difficult to measure. Although some IARCs domonitor their trainees, they usually do so to improvecourse content and organization rather than to evaluatetheir investment. At IRRI, however, training impact hasbeen assessed by criteria such as the professionalgrowth and job productivity of the alumni and their useof the knowledge they gained from their courses(Domingo and others 1989).

Within the CGIAR, the International Service forNational Agricultural Research (ISNAR) is responsiblefor building capacity in organization and managementwith agricultural research institutions; irrigation man-agement is the responsibility of the International Irriga-tion Management Institute (IIMI). Indicators of suc-cessful implementation are problematic in these areas,because there is no satisfactory way of evaluatinginvestments in institutional capacity building by IARCs.But CIMMYT has attributed the fall in the coefficient ofvariations of its yearly international maize trials, from25 percent in 1974 to 16.5 percent in 1982, to learningthrough collaborative interaction between center andnational scientists (Byerlee and Moya 1992). Internalreports from IARCs highlight that, although scientistsfrom developing countries appreciate training, the lackof opportunity to mobilize new skills because of mori-bund institutions reduces morale and increases staffturnover. In Africa, in particular, the financial prob-lems of NARS-low salaries and very low operating

18 budgets-erode the morale of both managers and scien-

tists and often overwhelm efforts to improve institu-tional effectiveness.

Scientific Impact

Some of the original IARC directors saw publica-tion as incidental to their mission. Recently, sincestrategic research has been cited as the most appropri-ate focus for the CGIAR, the five-year external reviewsof the centers have increased their emphasis on evalu-ating publication records. Citation analysis is anaccepted measure of scientific impact. Among IARCs,ILRAD-wholly devoted to strategic research-has astrong record. Other high performers include ICLARMand the International Food Policy Research Institute(IFPRI). Such analyses can readily be obtained forcenters, or even center staff, from commercial agenciesthat specialize in citation.

Impact Independent of Farmer Decision Making

The impact of IARC research on the lives of re-source-poor farmers is the dominant preoccupation ofCGIAR stakeholders and scientists. As Figure 2 shows,this can occur independently of farmers' decisions. Thebest-known example from CGIAR research is the con-trol of the cassava mealybug by IITA and its partners inAfrica, using aerial diffusion of a parasitic wasp(Epidinocarsis lopezO, identified as a mealybug preda-tor by CIAT in Latin America. This sophisticated appli-cation of biological control resulted in enormous sav-ings of staple food across the cassava belt of Africa. Anindependent evaluation of the program (Norgaard 1988)estimated minimum benefits of $2.2 billion by the year2003, for an expected total expenditure of $14.8 millionduring the period 1978 through 2003. The fact that thisprogram will be active for a twenty-five year period (eventhough this solution bypassed the farmer decision-making, adoption, and diffusion process) again showsthe long time horizons involved in R & D. 19

Sch opportunities are rare. Crops such as cotton,for which the farmer sells the seeds in the course ofmarketing the unginned lint, allow improved seed to bedistributed back to growers. The rate of diffusion inthese circumstances depends on two factors: theamounts of breeders' seed made available, and theefficiency of the collection of cotton and the distributionof seed. Even with farmers as a captive market, how-ever, materials cannot be imposed on them; their cir-cumstances still need to weight the direction of thebreeding program. For example, the introduction ofAkala and Deltapine cotton varieties in Turkey in the1 960s created a situation in which a high proportion ofcotton had to be picked in a short period of time,requiring large amounts of casual labor that was expen-sive and difficult to find. The traditional varietiesfruited over a longer period, spreading the demand forlabor to a level that could be provided by the family.Farmers responded by reducing their area of the crop(Kiray and Hinderink 1968).

Impact in Farmers' Fields and Beyond:Levels and Methods of Assessment

The key to economic, social, and environmentalbenefits from research investments is to design tech-nologies farmers choose to use. As Figure 2 indicates,farmers' decisions to adopt new technology bring im-pacts at the farm household level, on family income andwelfare and on the physical and human resources usedin farming. Economic repercussions extend beyond thefarm, through the markets that receive the increase inproduction and supply the new fertilizers, herbicides,and pesticides. If the scale of adoption is broad enough,these repercussions are felt in the regional, national,and even the global economies.

Impact on the farm household is the most commonand probably the most useful level for IARCs to monitor

20 to generate information for internal planning and donor

... . ... .. . . . . .st g ... .g .. .....: .. g :R... .. ''"g ,giEE..-.. ....i'.... .g 2 ii g

interest. Measures of impact start from counting "adopt-ers," farmers who use the technology. For example, aCIMMYT on-farm research project on maize in Panamafocused on the maize/bean rotation system. After fouryears, 61 percent of farmers had adopted improvedweed control, and 43 percent had adopted some form ofreduced tillage; 35 percent had adopted improved vari-eties by 1985 and 74 percent by 1989, and the use ofrow planting had increased from 30 to 80 percent (CIAT,CIMMYT, and CIP 1992). In this study, adoption wasused as a measure of success, and researchers as-sumed that farmers adopted the new varieties andtechniques because they brought benefits.

Historically, much impact assessment has beenbased on measuring the change in the productivity ofthe crop or animal enterprise using the new technology.This approach requires more than simply identifyingadopters. Extra data might include:

* the land area or number of animals to which thenew technology is applied;

* the yield increment to the technology;

* increased stability of yield over time and reduc-tion of risks;

• the cost reduction achieved when lower costsare the source of benefits;

* the net benefits obtained (the value of the incre-mental yield less the costs of obtaining it); and

* the contribution of each component of the tech-nology (the variety and each new managementpractice) to the yield increment and sometimesto the net benefits obtained.

CIAT and NARS in Colombia and Venezuela imple-mented an integrated pest management program forrice, based on economic thresholds, which illustrates 21

both cost reduction and environmental impact as sourcesof benefits. In Colombia, the total insecticide andfungicide applications were reduced from nine per cropcycle in 1980 to three in 1990. In Venezuela, monitor-ing began in 1988 when more than 60 percent offarmers made two or more applications; by 1990, morethan 90 percent were making one application at most(CIAT, CIMMYT. CIP 1992).

Evaluation on a Farming System Level

Beyond the enterprise, at the level of the wholefarm, the analysis of benefits is more complex, moredetailed data are needed, and the cost of impact assess-ment increases. Interactions among the enterprisesbring "opportunity" costs and benefits into the calcula-tion. In smallholder agriculture, with subsistence forthe family a high priority and family labor the majorinput, evaluation at the whole farm level is an impera-tive.

A comparative evaluation of the use of fertilizer oncotton and maize in the same farming system showedwidely different results between an enterprise basis anda system basis in which the interaction between enter-prises in their demands for labor and cash are ac-counted for. On an enterprise basis, fertilizer showeda cost/benefit ratio of 3.6:1 on cotton compared with3.1:1 on maize. On a system basis, fertilizer on cottonshowed a return of 3.1:1 compared with 4.8:1 for maize,reflecting a reversal of choice for investing very scarcecash in fertilizer (Collinson 1968). This case illustratesthe importance of evaluation within the whole systemfor smallholder agriculture. It also illustrates the extradata collection costs needed to capture labor profiles forthe whole farm system. A full month-by-month profileof labor use by enterprise and by operation is requiredto identify the interactions that influence how farmerschoose to invest in fertilizer. Although the enterprise isa valuable context for the generic evaluation of proto-

22 type technologies, only evaluation in the whole farm

. >:::. B B .:S.:§ ::.:S.:,': >>><i,.__..i.iB...2. ...T ............,S'.:,'.X,X,'SS,":;By W h...'"S,,E'SS' ' ,,:...',"""'=."'"''"""'""'"'"'' ' ;ESESES.ES:...........

system can identify options and adaptations that ad-dress the priorities and circumstances of the farmers inthat system.

A similarly intensive study recently reconsideredthe impact of the green revolution on the poor. Itcompared the change in incomes of different classes ofrural people in rice-growing villages in southern Indiaand concluded:

...evidence from the resurvey villages showsthat small paddy farmers (+90 percent) andlandless laborers (+125 percent) gained thelargest proportional increases in family in-come between 1973/4 and 1983/4....Thesechanges are corroborated by measuredchanges in the real value of household con-sumption expenditure, by a sharp improve-ment in calorie and protein intake, and bythe growing importance of higher qualityfoods in total household expenditure (Hazelland Ramasamy 1991).

The complexity, cost, and time required make sucha study a 'one off' event and render this level of detailimpractical as a routine follow-up to farmer innovation.Even this intensive state-of-the-art study may not cap-ture the social costs or benefits from such innovation.The sustainability issue revolves around external fac-tors-practices that degrade natural resources andgenerate costs that are avoided by the individual but fallon society as a whole. The measurement of suchexternalities, including resource and environmentaldegradation, is the subject of intensive contemporaryresearch. To date there are no accepted methods ofmeasuring the costs and benefits of this dimension.

Evaluation on a Global Level

When a new technology has been widely adopted,its aggregated effect on crop production can be used 23

- - ze ~. .. . .. ->g :::a.:: . ;. S <<. . ..... ::o: ::::g.- g*;~~~~~~~~~. ....... C. -:.2::>ff0

through national statistics to make estimates of im-pact, even on a global level. Some global and continen-tal impacts of the green revolution's short-strawed riceand wheat varieties have been noted. On the whole,because of the time required for adoption to be reflectedin national production statistics, they are of limited usefor feedback to adjust ongoing programs. Furthermore,as Table 1 shows, aggregate national data, particularlyfor crops used heavily for farm family subsistence, canbe unreliable. Finally, cause and effect are less easilyrelated at the aggregate level. Production can rise fromincreases in the area planted as well as from yieldimprovements. But where national data are reliable(particularly where land use areas and yields are welldocumented or survey data can be linked to increasesin seed or input sales), they are cheap to obtain andanalyze and offer strong reinforcement to donors ofmoney well invested in the past.

Table i "

Comparative Cassava Production Figuresfor Nigeria, 1979-82-(in thousands of tons)

Federal Cental Food aI. ... U.S.

Ofice of Bank of AgriCuture Dtent ofYear Statistics Nigeria Organization WsrNG-):cultwre

1979 1.621 1,976 105 100

1980 1,492 1,988 11,000 131

1981 872 2,159 11,000 1

1982 943 2,308 11,700 11700

Sour..e-Stifel 1992.2 4 t - - ; ~__ _ _ _ _ _ _ _ _ _ __ . .. . . ............. .

igg.gg-g .ggg g... igggggiiiig ............. -i i gg gig. g ' ........ ................ .... - g gEiiiii2-:.... .. . ..... ... ,:

In some cases, centers have collaborated with andsupported government statistical departments, supple-menting the data to be captured in routine samplesurveys. One recent example is CIAT in collaborationwith the Government of Rwanda, where new varieties ofclimbing beans were reported to be popular amongfarmers. In 1992, CIAT collaborated with both theDepartement des Statistiques Agricoles, which regu-larly documents trends in the 93 percent of Rwandanhouseholds that are dependent on agriculture, and theInstitut des Sciences Agronomiques du Rwanda toevaluate the spread of climbing beans. They havealways been popular in the Gisenye and Ruhengeriareas of the north and west, but surveys as late as 1986showed that only 5 percent of farmers in the central andsouthern areas grew climbing beans. The 1992 samplesurvey, however, reported improved varieties of climb-ing beans on 43 percent of farms, representing 450,000rural households. Estimates from the survey, carriedout in one of Rwanda's two bean seasons, gave anannual value of net benefits of between $4 million and$8 million for Rwandan farmers (Sperling 1993).

This tyfpe of documentation has increased duringthe past two years. Table 2, from a recent publicationby the Latin America-based centers, demonstrates theimpact of their collaboration in germplasm develop-ment with the national programs in the region (CIMMYT,CIAT, and CIP 1992) . Benefits from the improvedmaterials have been allocated equally between IARCsand NARS.

For beans, maize, rice, and wheat, the impact forwhich is clearly documented, the annual value of in-creased production is $1,050 million, or nearly tentimes the total 1990 budgets of the three centerscombined. Because international markets for rice and 25

.Table. 2 . . =~~~~~~~~~~~~~~~~~~~~~~~~~~~~,,,.,.... .:.'5......''. '.:.i'. .......... .......... ... ...............

Progress in Genetic Irovet in BeanisMaize, Rice and Wheat-in atow A1erica Unti 1990

Achievement ~ e#nb Msiae Rice eS

Number of teleased. varieties, lHnes, orboth orsgatingfromCIAR centers 9- 86 69 134

Area grovm with these varieties, 1990(thousands of hectares) 370 2,002 1,208 4,074Number of vaieties released withptatda m, if mcentrs, 8 111 60 272

Area grownw tese vweties, 1990(thousands of hectares) 60 1,848 800 4,560

'Criop :aea ifetd s mtes er~Is,1990(%s) : 4.9 16.7 25.1 81.2

Estim atd rprducion ro thescemateras 199 ''''u s i"'''' tons, , ,,,122 1,696 836 3,523

imad cxtra value, 1990~US $ milionl ' 60.8 203 209 567

Price saving to consumers, 1990 (%J 5 0 24 0

Costs of centers' researh progrms,l9w (VUS nUons)* 3.3 8.8 4.0 -,4

Intemal rate of return on researchprograms up to 1990 616 5 69 67

X Includes overhead.

Source: CIAT, CIMMYT, and CIP 1992.

beans are limited, as production went up, prices wentdown, making consumers the principal beneficiaries ofnew technologies for these crops. Thirty-three varietiesof potatoes developed at CIP have been released in LatinAmerica, but the impact has not yet been documented.For other commodities, such as pasture and cassava,research started from a much more limited knowledge

26 base, and impact is only now being observed.

........... o

Economic Models

Aggregate production data can be multiplied by theprice of the product to give a simple economic valuationof benefits. Such simple valuations, however, do notreflect the full economic consequences of the increasesin production, because the impacts of technologicalinnovation are confounded by the effects of changingmarket prices and changing policies on the target cropas well as by the effects of similar changes on othercrops. In addition, although more efficient productionmeans cheaper consumer prices, it may also meanchanged demands in the labor, machinery, and sup-plies markets.

Economic models capture these repercussions andoffer an aggregate measure of the economic gain createdby technological innovation. The internal rate of return,shown in the final line of Table 2, is one such measure.Based on the gain created, it specifies the rate of returnto the investment in germplasm research. A detaileddescription of types of economic models and a list ofresults from their application can be found in Evenson(1992).

Such models have their disadvantages. For non-technical audiences, they offer less transparent resultsthan the less sophisticated aggregated adoption, area,and production data, and they identify cause-and-effectrelationships less clearly. As mentioned earlier, diverseagencies contribute to the total R & D process and thusto the CGIAR's dependence on others to mobilize itsproducts. It is often unclear in model applicationswhether the benefits that stream from new technologyare appropriately allocated over the full range of invest-ments that contribute to technology adoption and diffu-sion.

As in a recent CIMMYT study of wheat research inNepal, it is often appropriate to acknowledge that inno- 27

vation would occur without a national research pro-gram-in the Nepal case, as spillover from research inIndia and Pakistan. In this study, the returns toresearch are measured by the economic gains accruingto the investments made by the national research pro-gram, net of the contribution of the spillover and of thereturns to investments by other sectors in the R & Dprocess (Morris, Dubin, and Pokhrel 1992).

Finally, it is important to note that the interna-tional centers have wider criteria for success thaneconomic surplus alone. They are interested in alleviat-ing poverty, reducing farmers' risks, and sustaining thenatural resource base. Although economic models canbe used to show the distribution of benefits betweenproducers and consumers, they cannot yet capture theother dimensions important to the mission of the CGIARcenters.

Challenges and Strategies for IARCs

The centers must help donors assure their con-stituencies that investments in research are valuable.They must also meet their own needs for priority settingand program planning. The centers are most efficient inproducing intermediate products that are useful tomany countries. Such products need further researchto tailor them to farmers' circumstances. Beyond that,institutional and policy support are also needed toenable farmers to exploit these products.

Historically, researchers in general, and particu-larly international researchers, have depended on oth-ers to identify the technology needs of small farmers andto mobilize their research results to meet those needs.This dependence is the first major challenge for IARCsbecause researchers feel pressure to justify their work

28 in terms of progress in development, without having any

influence over many of the factors that propel develop-ment (Hardie 1988).

The difficulty of linking research to the market haslong been acknowledged, in industry and in private-sector agricultural research. For each product thatsucceeds in the market, a typical manufacturing com-pany generates fifty-eight new product ideas. Afterbusiness analysis, seven of these generally reach thedevelopment stage. Of these seven, six are eliminatedduring development, testing, or commercialization.Almost 75 percent of new product expenses (and thusthe work of eight out of ten development scientists andengineers) are devoted to projects that will not bejustified in terms of commercial success (Booz, Allen &Hamilton Inc. 1968). The preceding example reflects anaverage success rate of 14 percent for the manufactur-ing companies surveyed. It demonstrates that researchcannot be planned with certainty. Serendipity remainsan important element.

In agriculture, the vagaries of the weather are anadded source of uncertainty. Production, and thereforefarmers' incomes, varies from year to year. This uncer-tainty also complicates and often prolongs the researchprocess. DeKalb Seed Company makes some 5,000crosses to identify one new commercial maize hybrid.Pioneer Seed Company has released an average oftwelve commercial conventional hybrids per year dur-ing the past decade on an annual budget of approxi-mately $19 million. Although similar data for CIMMYTcannot be used as a test of comparative performance,they are of interest. National programs in developingcountries released more than 300 maize varieties orhybrids containing CIMMYT material between 1981and 1990 (Table 3), an average of thirty per year on anannual average maize program budget (including over-head) of approximately $12 million (CIMMYT 1992).

Improved maize based on CIMMYT materials nowcovers just less than 10 million hectares-or some 12 29

- :-..~Table 3=: .

Malze'varieties and hybrids containiag CIMMYT germplasmreleased in developing countrites,by region. 1966-90.

Releases containng C.MMYT germplaszn

Code 1 Code 2 Code 3 TotalSub-Saharan Africa:1966-70 0 0 3 31971-75 4 1 6. 1l1976-80 6 4 3 13:1a981-85 - .: : . E 13 14 25 52

,1986-90 '12 17 20 4'9-Total ' 5 36 57 128

1960-70 0 0 0 019t7175 0- 2 0 21976-,,,S,,O -0,- 1 0 11981-=8=5:' 1 2 0 31986-90 0 .4 0 4Total 1 --. 9 0 10Asia:1966-70 0 0. 3 31971-7S 0 0 2 21976-80 6 1 I- '9 161981-85 9 2 22' 331986-90 10 7 33 50Total 25 10 69C 104Latin America:1966-70 8 3 9 -20

:1A971-75 4 2 8 141976-80 6 -; " = ' H .11 6 - 0 -231981-85 26 16 33 75198690 1z2 23 36 71Tot, 5:. 55 92 203An 4eveloping countries,:1966-70' 8A-:: .. ::,- S3 9 261971-75 8 3a 8 291976-80 18 16, 6 531981-85 48 33' 33 1631986-90 34 47 36 174Total 116 102 92 445

Code I = Direct use of CIMMT germplasn.Code 2 = Selection from CItMMYT trials.Code 3 = Contains some,CIMMYT germplasm.

Sopuree: Lopez, Pivira, M.&. And M.L. Morris. 1994. impacts ofIternational Ma=ze Breedi Research In the Developig World,1966-. Mexico D.F: eCTMT.

30 _ _ _ _: _ ---.-. _=

percent of the total maize area of developing countries.These numbers reflect the situation of many drylandcrops in agricultural sectors dominated by small farm-ers. The potential for further impact remains huge;achieving it will require governments and the interna-tional community to address the performance of comple-mentary functions essential to the R & D sequence,such as produce marketing, farm input supply, farmcredit, and extension services. Research will not alwaysbe the priority candidate for investments. Unfortu-nately, the adaptive research function remains weak.There is only very sporadic coverage of small-farmsectors by cadres that understand commrunities andcan enroll farmers in partnership by the use of partici-patory techniques. This direct exchange between re-searchers and resource-poor small farmers is a prereq-uisite for better market information and better balancein the demand for and supply of technology.

The second major challenge faced by IARCs is theirdiversity of products with a clientele that is scatteredacross the developing world. A typical center mayinteract with thirty national programs and have consid-erably more than 100 research projects. The cost ofassessing the impact of these projects on so manyclients is prohibitive, especially because weak agricul-tural statistics mean that primary data must be col-lected to measure impact in most partner countries.

The key to monitoring, evaluation, and impactassessment is the same as that for better articulation ofsmall farmers' technology needs: an institutionalizedcadre of professionals with skills at the grassroots levelto whom this work is routine. Continual interactionswith farmers and communities, an integral part of theadaptive research function, generates powerful feed-back on the effectiveness of the R & D sequence. Fromexperiences with a variety of farm-level models, a consen- 31

...................... . ..:. . .z . ..> . . . '' .....

sus is emerging that a participatory approach thatmobilizes both the communities and the public servicetogether will make good use of scarce professionalresources. A concerted effort to rationalize this grassrootsfunction would widen coverage and generate informa-tion for better decision making at the program andpolicy levels.

Planning

Monitoring the research cycle is important to IARCs.Preliminary assessment and intermediate feedback,within the timeframe of the cycle, are particularlyimportant to priority setting and program planning.Evidence of early adoption, or early farmer assess-ments, offer the benefit of timely feedback to modifycontinuing programs.

Special studies of final impact are of little use toprogram planning because the R & D cycle throughdiffusion of the technology to final impact assessmentis too long. IARC programs normally will have moved onbefore the results of such assessments are available tofeed back into the planning process. Where research forbetter solutions will continue, the impact of an earlieriteration can measure the residual benefits for theevaluation of further research. At the same time,special studies of final impact do add to our knowledgeof the technology adoption and diffusion process. Whenthis kind of research is undertaken, an explicit aim inits planning should be to use the information for asmany purposes as possible to offset the heavy overheadcosts of primary data collection. Subsequent studiescan be integrated with the collection of governmentagricultural statistics where this is dependable andwhere there is similar value to NARS partners in dem-onstrating the success of previous investments of pub-

32 lic funds.

B.. . . .. .. . . . . .i

Wherever primary data collection is planned andthe products of more than one IARC have helped toimprove local farming, there is scope for intercentercollaboration in sharing fieldwork expenses.

Donor Needs

Donors that sponsor development are interested inthe effectiveness of their funding. The current reces-sion coincides with a burgeoning demand for aid forEastern Europe, for United Nations activities, and forthe environment. The scarcity of funds has sharpeneddonors' interest in evidence of impact, and IARCs areresponding, some more strongly than others.

Preliminary assessments of the potential impact ofresearch are valuable to donors, particularly when aresearch calendar highlights mileposts as intermediatesteps toward solution of problems and adoption of thetechnology by farmers. Although they are not measuresof impact, such achievements allow donors to monitorprogress and help reconcile them to the longer timehorizons of strategic research programs.

The expansion of research on natural resourcemanagement in the CGIAR will involve longer-termprograms and less definable products. In many cases,the product will be an understanding of natural pro-cesses and the avoidance of losses, often measured interms of topsoil saved, intact groundwater, or otherexternal factors. Clear preliminary planning is neededbefore investments to realize such products will beaccepted. Planning should emphasize intermediateachievements and projections of the expected socialbenefit of resource management interventions. Onedimension of the ecoregional strategy recently adoptedby the CGIAR is to show early benefits to the productionsystem in which sites are located; an understanding ofthe interactions between human decisions and naturalresource processes accumulated over the long term. 33

The CIMMYT data in Table 3 illustrate the record-ing and presentation of intermediate achievements. Bymaintaining a data base of plant material sent tonational programs and the pedigrees of materials re-leased by them, CIMMYT is able to highlight the impor-tance of its germplasm. These are milepost achieve-ments that demonstrate the effectiveness of the center'spartnership with national programs on three conti-nents.

Donors use the documentation of final impact todemonstrate the success of their funding in their do-mestic budget process. Because studies of final impactare expensive and must be selective, the aim is to showexamples of high returns on IARC investment. Benefitflows can evaluate returns beyond the costs of theimmediate program, for example, against the total in-vestment in research in the commodity and even againstthe investment in the center as a whole.

Although economic models are useful for suchstudies, for the greatest impact it is important todocument farmers' adoption, improvements in house-hold situations, any differential impact on livelihoodbetween men and women, and the aggregate impact onproduction. Such parameters add value for publicrelations purposes and for donors. IARCs think thatsuch studies should be done in partnership with na-tional programs, to help convince their Ministries ofFinance to improve research funding as an investmentin the nation's future.

IARCs also bring significant returns to the domes-tic agricultural economy of some of the CGIAR's majordonors. Quantifying these returns can be valuable tofuture center support. To date, only Australia hasquantified the benefits it has received from improvedwheat materials made available from CIMMYT:

"Since 1977, Australia has contributed an averageof US$2.8 million per year to core programs of the

34 CGIAR Centers. Of this some 6% has gone to CIMMYT's

syB g..... g lB........ .. . . .... . . . .......................::: gv. lll. iz;

wheat program . . . Australia has received overall costreductions averaging some US$75 million per year re-sulting from the improved wheat varieties derived fromCIMMYT" (Brennan 1989).

The Brennan study attracted wide attention inAustralia:

"Mr John Kerin, Minister for Primary Industriesand Energy, recently announced that the use, in Aus-tralia, from 1974-1990, of wheat germplasm importedfrom CIMMYT has resulted in additional income for thewheat industry of over two billion dollars-a sum suffi-cient to fund both ACIAR [the Australian Council forInternational Agricultural Research] and Australia'scontribution to the CGIAR at their present levels for thenext 100 years!" (Tribe 1991).

There is also an important educational task inraising awareness among donors and their constituen-cies on the nature of the research process. Theirexpectations should be based on an understanding oftwo characteristics of the process: the uncertainty ofresearch as a business and the time it takes to complete.

Conclusion

Research initiatives are defined by the problemsthey seek to solve, not the product they hope to identify.Failure is common. Even with success, the final natureof the product, and therefore its congruence with mar-ket needs, is initially unknown. These uncertainties areheightened by the difficulties of identifying market needsamong small farmers to help shape the product as theresearch process progresses.

Better market information will reduce the failurerate of new technologies and enhance the efficiency of 35

. . . : . . . . .

the research process. This can be achieved by widepromotion of and support for adaptive research, and byrationalization of the organization of field-level staffworking in research, extension, and evaluation. Ratio-nalized field-level organization can easily provide peri-odic impact assessment information for clients rangingfrom research managers to policy makers and donors.Routine information flows from the field will enhancethe relevance of decisions at each of these levels to theneeds of developing country rural populations.

36

References

Anderson, Jock R., and R. W. Herdt. 1990. "Reflectionson Impact Assessment." In Ruben G. Echeverria,ed., Methods for Diagnosing Research System Con-straints and Assessing the Impact of AgriculturalResearch. Vol. 2. The Hague: ISNAR.

Anderson, Jock R., R. W. Herdt, and Grant M. Scobie.1988. Science and Food: The CGIAR and Its Part-ners. Washington, D.C.: CGIAR and The WorldBank.

Booz, Allen & Hamilton, Inc. 1968. "A Program for NewProduct Evolution." Quoted in Robert R. Rothberg,ed., Corporate Strategy and ProductInnouation. 2nded. The Free Press, 1981.

Brennan, J. P. 1989. 'Spillover Effects of InternationalAgricultural Research: CIMMYT-based Semi-DwarfWheats in Australia." Agricultural Economics 3:323-32.

Byerlee, D., and Piedad Moya. 1992. "Ex-post Assess-ment of Research Impact: CIMMYT's Experience."In David R. Lee, Steven Kearl, and Norman Uphoff,eds., Assessing the Impact of International Agricul-tural Research for Sustainable Development. Pro-ceedings from a 1991 CIIFAD Symposium at CornellUniversity, Ithaca, N.Y.

CIAT, CIMMYT, and CIP. 1992. CIAT, CIMMYT & CIP:Their Role in Agricultural Research in Latin Americaand the Caribbean. Cali, Colombia: CIAT.

CIMMYT. 1992. CIMMYTAnnual Report 1991. Improv-ing the Productivity of Maize and Wheat in Develop-ing Countries: An Assessment of Impact. MexicoD.F.: CIMMYT.. 37

;~~~~~~~~~~~~~~~. . . . . .. .. . .................................. ,.., ,T ..... j . .. T....... .. : :, ,' . , . ,

Collinson, M. P. 1968. The Evaluation of InnovationsforPeasant Farming. East African Journal of RuralDevelopment. Vol. 1, No. 2. Kampala, Uganda:Makerere University.

Collinson, M. P. 1982. 'Farming Systems Research inEast Africa: The Experience of CIMMYT and SomeNational Agricultural Research Services, 1976-81."International Development Papers No. 3. East Lan-sing: Michigan State University.

Collion, M. H., and P. Gregory. 1993. Priority Setting atCIP. Lima: CIP/ISNAR.

Dalrymple, Dana G. 1986a. Development and Spreadof High Yielding Rice Varieties in Developing Coun-tries. Washington, D.C.: USAID.

Dalrymple, Dana G. 1986b. Development and Spreadof High Yielding Wheat Varieties in Developing Coun-tries. Washington, D.C.: USAID.

David, C. C., and K. Otsuka. 1991. The Green Revolu-tion and Income Distribution Across Production Envi-ronments in Asia. Los Banos, Philippines: IRRI.

Domingo, A. A., L. D. Barnachea, S. S. Mina, and D. R.Minnick. 1989. The IRRI Short Term TrainingProgram: An Impact Assessment, Training and Tech-nology Transfer Department, IRRI. Los Banos, Phil-ippines: IRRI.

Evenson, R. E. 1992. "Notes on the Measurement of theEconomic Consequences of Agricultural Researchin Investments." In David R. Lee, Steven Kearl, andNorman Uphoff, eds., Assessing the Impact of Inter-national Agricultural Research for Sustainable De-velopment. (Proceedings from a 1991 CIIFAD Synm-posium at Cornell University,) Ithaca, N.Y.

Hardie, John D. M. 1988. "Measuring Development38 Effects of Agricultural Research in the Third World:

~~~~~~~~~~~~~~~~~~~~~~...a...........

Case Study Methodologies." Paper prepared for theRutgers University/ISNAR Agricultural TechnologyWorkshop, July 1988.

Hazell, Peter B . R., and C . Ramasamy. 1991. The GreenRevolution Reconsidered. Baltimore: John HopkinsUniversity Press.

Henry, G. 1991. "sAdoption of Cassava Technologies:Constraints, Strategies, and Impact." Working Pa-per No. 93. Trends in CIAT Commodities 1991.CIAT, Cali, Colombia.

Horton, Douglas E. 1990. "Assessing the Impact ofInternational Research: Concepts and Challenges."In Ruben G. Echeverria, ed., Met hodsforDiagnosingResearch System Constraints and Assessing theImpact ofAgricultural Research. Vol. 2. The Hague:ISNAR.

Ingersoll, Bruce. 1992. "Breakthrough Promises toEnd Blight That Afflicts Wheat Crops." Wall StreetJournal. October 23.

Kiray, M., and J. Hinderink. 1968. "InterdependenciesBetween Agroeconomic Development and SocialChange" Journal of Development and Social Change.4:4.

Morris, Michael L., H. J. Dubin, and Thaneswar Pokhrel.1992. "Returns to Wheat Research in Nepal."CIMMYT Economics Working Paper Series 92-04.CIMMYT, Mexico D.F.

Norgaard, R. B. 1988. "The Biological Control ofCassava Mealy Bug in Africa." American Journal ofAgricultural Economics 70(2): pp 366-371.

Scobie, Grant. 1979. "Investment in InternationalAgricultural Research: Some Economic Dimen- 39

sions." Staff Working Paper No. 361. World Bank,Washington, D.C.

Smale, M., Z. H. W. Kaunda, H. L. Makina, and M. M. M.K. Mkandwire. 1993. Farmers' Evaluation of NewlyReleased Maize Cultivars in Malawi: A Comparisonof Local Maize, Semi-flint and DentHybrids. Lilongwe,Malawi, and Harare, Zimbabwe: CIMMYT.

Sperling, Louise. 1993. "The Impact of ImprovedClimbing Beans in Rwanda: Preliminary Results ofCollaboration, DSA/ISAR/CIAT." Paper presentedat the seminar on bean crop research, Institut desSciences Agronomiques du Burundi, Bujumbura,Burundi. Feb. 1993. Cali, Columbia CIAT.

Louise Sperling, Loevinsohn, M.E. and B. Ntambovura.1993. "Rethinking the Farmer's Role in Plant Breed-ing: Local Bean Experts and On Station Selection inRwanda." ExperimentalAgriculture, Vol 29 No.4. pp509-519.

Stifel, Laurence D. 1992. 'Impact Assessment inAfrica: Experiences of IITA and Other IARCs." InDavid R. Lee, Steven Kearl, and Norman Uphoff,eds., Assessing the Impact of International Agricul-tural Research for Sustainable Development. Pro-ceedings from a 1991 CIIFAD Symposium at CornellUniversity, Ithaca, N.Y.

TAC. 1986. Training in the CGIAR System. Rome: TAC.

Timothy, D. H., Paul H. Harvey, and Christopher R.Dowswell. 1988. Development and Spread of Im-proved Maize Varieties and Hybrids in DevelopingCountries. Washington, D.C.: USAID.

Tribe, D. E. 1991. Doing Well by Doing Good. Agricul-tural Research: Feeding and Greening the World.

40 Leichhardt, NSW: Pluto Press Australia Ltd., in

.| -s. .::. ::s ..... . - v. ............... ............................ .s:g .: $gB0:: i ..... iiEf-iig:.....i.iEg> .;.: .:; ...-.-E:-::i::EEE::EE :: -: : :::::::::::-::.:...::

association with the Crawford Foundation for Inter-national Agricultural Research.

Wilson, Lawrence A. 1989. The Establishment ofLinkages Between Universities and InternationalAgricultural Research Centersfor the Development ofThird World Countries. Ibadan, Nigeria: IITA.

@ Printed on Recycled Paper 4 1