beneficiary assessment for agricultural...

AGRICULTURAL KNOWLEDGE & INFORMATION SYSTEMS (AKIS)GOOD PRACTICE NOTE

Beneficiary Assessmentfor Agricultural Extension

A Manual of Good Practice

The World BankRural Development FamilyAgricultural Knowledge & Information Systems (AKIS)

Work in progressfor public discussion

Lawrence F. Salmen (SDV)

October 2000

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

wb406484

Typewritten Text

72130

wb406484

Typewritten Text

Agricultural Knowledge & Information Systems is a thematic team focusing on agriculturalextension, education, and research within the Rural Development Department of theEnvironmentally & Socially Sustainable Development Network of the World Bank.

iii

Contents

Foreword iv

Abbreviations v

Agricultural Extension and the Beneficiary Assessment Approach 1

Designing the Beneficiary Assessment 1Understanding the Context 2Setting Objectives 2Selecting Institutions and Field Researchers 2

Institutions 2Field Researchers 3

Preparing Terms of Reference 4Sampling Frames 4Preparation of Interview Guides 5Methodology 6

Conversational Interviews 6Focus Group Discussions 7Participant Observation 8

Institutional Assessment 8

Implementation 9Training of Field Researchers 9Monitoring and Evaluation 10Preparation of the Final BA Report 10Internalization 11

Dissemination and Documentation of BA Impact 11

Annex: Terms of Reference for Beneficiary Assesments for Agricultural Extension 13

iv

Foreword

Agricultural extension programs areunder increasing pressure to showresults in order to maintain financing

from public budgets. Experience indicatesthat agricultural extension systems must re-spond to farmers’ real needs, if they are toshow results and be effective. Evaluating thequality and effectiveness of extension servicesis difficult, but is essential to improving pro-gram operations and maintaining funding.Beneficiary assessment is a tool that hasproven effective in providing decisionmakerswith information on how farmers view exten-sion services. This AKIS Good Practice Notedescribes good practice in carrying out ben-eficiary assessments.

“AKIS Good Practice Notes” are designedto disseminate views, experiences, and ideasthat may assist World Bank Team Leaders, na-tional counterparts from Borrower countries,and other partners to prepare and implementprojects to strengthen agricultural research, ex-tension, and education programs. The GoodPractice Notes contain valuable information

about lessons learned from innovative experi-ences in World Bank projects and elsewhere,and make this information readily available forcomment and use by project teams.

The Agricultural Knowledge and Informa-tion Systems (AKIS) Thematic Team includesWorld Bank staff working in or interested inresearch, extension, and education programs.The team objective is to enhance the effective-ness of Bank support to agricultural knowledgeand information system development. This, inturn, contributes to the Bank’s objectives of al-leviating poverty, ensuring food security, andimproving sustainable management of naturalresources. The AKIS team emphasizes policy,institutional, and management issues associatedwith agricultural research, extension, and edu-cation. Other thematic teams focus on technicalissues. The team mission is to “promote the de-velopment of sustainable and productive agri-cultural research, extension, and educationsystems in Bank client countries.” This “AKISGood Practice Note” was prepared by LawrenceF. Salmen (SDV).

Marie-Hélène CollionChair, AKIS Thematic Group

v

Abbreviations

AKIS Agricultural Knowledge and Information Systems Thematic Team of the World BankBA Beneficiary AssessmentCG Contact GroupNGO Non-Governmental OrganizationPRA Participatory Rural AppraisalTOR Term of Reference

Agricultural Extension and theBeneficiary Assessment Approach

1

The farmer is at the core of the agricul-tural enterprise. Climate, soil, seeds,roads, and insects affect agricultural pro-

duction. The key person who determines howthese factors are managed is the farmer. It is thefarmer who decides on what crops, and how,when, and where to plant and cultivate them.The farmer also decides on what livestock willplay a role in the farm enterprise. Informationpassed on to the farmer will only be effective ifit is put to use by the farmer. Agricultural ex-tension is basically a system to improve farmincome through the provision of information to

farmers. Extension programs are informationservices and, as with any service, the best judgeof the quality of the service performed is theclient, in this case the farmer. In extension work,assessment of its effectiveness includes not onlythe quality of the services, but also whether itsucceeds in reaching large numbers of targetfarmers. A sound appraisal of the effectivenessof an agricultural extension program must startwith listening attentively and systematically tothe farmer. The input derived from listening tofarmers is the essence of the beneficiary assess-ment approach described in this manual.

Because Beneficiary Assessment (BA) is prima-rily a management tool, the objective of a BA,the way it is conducted, and the use to which itis put should be determined by managers ofagricultural extension agencies. Terms of refer-ence (TOR) must be prepared to guide the se-lection of persons inside or outside the extensionagency to conduct the BA, and supervision ofthe work (see section “Preparing Terms of Ref-erence” and Annex).

The following issues play a part in the de-sign and execution of a beneficiary assess-ment:

Designing the Beneficiary Assessment

• Understanding the context• Setting objectives• Selecting institutions and field researchers• Preparing terms of reference for BA imple-

mentation• Sampling frames• Preparing interview guides• Methodology• Institutional assessment• Training• Monitoring and evaluation• Report preparation• Dissemination of findings.

2 Beneficiary Assessment for Agricultural Extension

Understanding the Context

To implement a beneficiary assessment, onemust first have a good understanding of theenvironment in which it is to take place. Thiswill be particularly important if the person do-ing the assessment is new to the project or lo-cale. It is important to become familiar with thesociocultural setting and the institutional envi-ronment where the project is taking place tofully understand the context. The following arerecommended as initial steps in assessmentdesign:• Reviewing relevant documentation such as

identification, appraisal, and supervisionreports; any previous related studies; andbroader social and economic analyses—forlocal, regional, and national levels;

• Interviewing key persons involved in de-veloping, implementing, and evaluating theproject in other agencies, such as nongov-ernmental organizations (NGOs), govern-ment, or donor institutions; and

• Exploring sites covered—and time permit-ting, those not covered—by the programbeing assessed. Limited immersion in theenvironment being affected by the interven-tion is in order; this should involve unstruc-tured conversations with randomly selectedintended beneficiaries, and other key people(diverse farmers chosen at random, commu-nity leaders, and members of local govern-ments and NGOs).

Setting Objectives

Once the context in which the BA is to be imple-mented is clear, it will be easier to determineBA objectives that are feasible and realistic. Themanager of the agricultural extension agencymust develop the objectives of the BA. This as-sessment must be of use to the manager as heor she attempts to implement and evaluate aproject that meets the needs, and helps realizethe aspirations, of its intended beneficiaries. Indetermining objectives, and depending on the

degree of decentralization of extension activi-ties, the central manager may want to consultwith regional managers. While the general ob-jective of a BA will be to increase the effective-ness and sustainability of an agriculturalextension program, particular objectives willlikely include:• Assessing the nature and degree of benefi-

ciary satisfaction with the extension service;• Determining the degree, and increasing the

rate, of participation in extension-relatedactivities;

• Identifying and helping the extension ser-vice reach the targeted segments of thepopulation, regardless of income, ethnicity,or location;

• Assessing willingness to share in the costsof extension, thereby increasing sustain-ability;

• Providing feedback on the utility and ap-propriateness of the technologies and infor-mation supplied;

• Ascertaining farmer appreciation for, andinvolvement in, local public and private in-stitutions;

• Determining the level of awareness of theexistence and nature of the extension ser-vice and ways to relate to it; and

• Discovering and examining factors under-lying motivation (or lack thereof) to imple-ment advice of extension agents.

Selecting Institutions and Field Researchers

Institutions

Ideally, BAs of agricultural extension projectsshould be carried out by the extension agenciesthemselves, in partnership with an NGO, a uni-versity research center, or a consulting firm. Thecontracted agency then identifies and recruitsqualified persons to join the BA teams and helpscoordinate the training and implementationprocess. Placing part of the responsibility forexecuting the BA with an outside agency shouldincrease the quality and credibility of the BA,

3

as it is done with the involvement of a “neutral”party not seen as overly identified with theproject itself.

The mix of in-house and external expertisein BA work combines the advantages of famil-iarity and internalization associated with in-house involvement, with technical expertise andcredibility that comes with the participation ofan external agency. A small, new extensionagency may lack credibility and qualified per-sonnel, so the BA work should be done by anexternal institution. A larger and well-estab-lished agency should have the BA work doneby an internal client-feedback or monitoring andevaluation unit.

A director should be chosen who will beresponsible to the manager of the extensionagency for selecting and supervising the BAteams. This person must have a good fieldknowledge of agricultural systems and avail-able technologies, and a good appreciation ofthe BA approach as well as the objectives of theassessment to be undertaken. In addition, theBA director will need good analytical and writ-ing skills to effectively oversee the tabulationand analysis of assessment reports, and to pre-pare the final report. When a decision is madeto involve an external agency, a lump-sum con-tract can be given by the extension service (withan appropriate TOR). The extension service canthen liaise with the person/unit in the publicentity responsible for monitoring the BA work.The BA director may be from the extension ser-vice or the external agency, as determined byextension management.

In selecting the individual and/or institu-tion to manage a beneficiary assessment for anagricultural extension agency, it is important tokeep in mind that the BA is an activity that,when done well, brings three diverse and oftendisparate entities into a functional and mutu-ally-reinforcing partnership: the grassrootscommunity of farmer-beneficiaries, the imple-menting agency (extension agency), andpolicymakers in government and donor agen-cies. The best BA practitioners should be able

to communicate well with each of these layersof international development.

Field Researchers

BA teams should include people who are famil-iar with the local culture, and have sound con-versational ability in the language of thebeneficiaries. The selection process for the BAteams is an important one, because it determinesto a large extent the quality of field interviewsand the integrity of information gathered.

The BA teams must be able to conduct con-versational interviews with people of modestmeans who do not know them. Responses mustbe elicited in such a way that they can be re-corded in an orderly and intelligible manner,and ultimately be used to improve the livingconditions of these people. Although the asses-sors ideally should be university graduates, in-dividuals without a university degree, but withgood communications and writing skills, can doan effective job.

A good BA interviewer will be:• A good listener;• Sensitive to local culture;• Respectful of all persons, regardless of status;• Knowledgeable about local agriculture;• Unobtrusive;• Open and engaging;• Proficient at recalling interviews; and• A competent writer.

Having good recall is an important charac-teristic for the assessor, who should try to mini-mize note-taking during interviews. Theinterviews should be done in an informal, con-versational manner, as opposed to a more struc-tured questionnaire style. Note-taking duringan interview may create an atmosphere of anxi-ety or mistrust, inhibiting the free flow of infor-mation. Because assessors will be writing notesbased on their interviews—as opposed to fill-ing out forms—clarity and precision of writingwill be important.

Despite the fact that most BA interviewersare social scientists, the above-mentioned char-



acteristics are more important than the aca-demic discipline of the interviewers. BA teamsshould be gender balanced because personsnormally communicate more openly with per-sons of the same gender, particularly on sensi-tive topics often covered by BAs.

Preparing Terms of Reference

The terms of reference for BA implementation(see Annex for Prototype) should elaborate onthe following areas:• Brief note on background and justification

(rationale);• Specific purpose and objectives;• Methodology—techniques to be used;• Research issues and themes to be addressed

(including the preparation of an interviewguide);

• Sample size;• Reporting (forum, frequency, to whom);• Time frame for implementation (generally

between four weeks and six months, de-pending on the sample size); and

• Budget (costs can range from US$20,000 to60,000).

Sampling Frames

In determining the sample size, the primaryconcern is that the beneficiary population in-terviewed should be large enough to serve asthe basis for management decisionmaking.Salmen (1995) stated: While statistical samplingprocedures may serve as a general guide, thesewill suggest sample sizes greater than thoseneeded for beneficiary assessment. Because ofthe in-depth, qualitative methodology em-ployed in this approach, long conversationalinterviews, often complemented by participantobservation, can provide a great deal of under-standing from a relatively small number of ben-

eficiaries.”1 BA work uses purposive samplingbased on parameters determined by the extentof extension coverage, the variability of targetcommunities, and the complexity of the pro-gram. Depending on variability of the clientgroup for extension services, the sample shouldbe stratified according to ethnicity, class, in-come, farm size, gender, and other relevant vari-ables. Within the stratified sample groups,sampling is random.

For agricultural extension BAs, interviewsshould be conducted with representativesamples of all major stakeholders. The mostimportant, and largest, group to be interviewed,naturally, is the direct beneficiaries, the poorerfarmers of the communities where extensiontakes place. Communities to be interviewedshould be selected at random with the numberof farmers to be interviewed varying accordingto the size of the community (usually 5-15percent of the community). Other key stake-holders to be interviewed in the selected com-munities would include: extension workers (seeBox 1), community leaders (formal and infor-mal), local government officials, NGOs, seniorofficials of the agriculture line ministry, andsocial and economic policymakers. For controlpurposes and to better understand the natureof demand and dissemination, it will be neces-sary to include representative samples with resi-dents of communities that are similar to thosereceiving extension but which are not coveredby extension.

By including interviews with representativesamples of these other key stakeholders, it be-comes possible to provide project managerswith a more comprehensive and accurate pic-ture of the issue at hand. In addition, it enablesBA teams to provide more relevant and realis-tic recommendations. BAs that include focusgroups and participant observation (see below)will also select representative groups and case

1 Lawrence F. Salmen, “Beneficiary Assessment: An Approach Described;” Environmental Department Papers, PaperNo. 23, World Bank, July 1995, p.4.

5

Box 1 Extension Agent-Farmer Interaction

The interaction between extension agents and con-tact farmers has generally had positive results, as re-flected in the greater increases in productionexperienced by contact farmers compared to farm-ers who did not have direct contact with extension.These latter, which is most of the farmers, may feelresentment toward the extension worker, who gen-erally has little or no time to spend with them. Somecontact farmers voice disappointment about exten-sion agents who lecture them on research themesconsidered relevant to farmer needs, without listen-ing to farmerswho express their real needs.

Extension agents, of course, have their own per-spectives, which must be understood in any compre-hensive attempt at reforming extension services.Many extension agents complain of lack of supportfrom their own central offices. They receive too littlefuel for their vehicles. They are given little or nochance for promotion. They are left on their own of-ten in remote corners of the country with inadequate

supervision. Perhaps most frustrating to extensionagents is their perceived inability to resolve many ofthe problems faced by farmers. In Ghana, fully two-thirds of the extension agents interviewed for a ben-eficiary assessment stated that they could solve fewerthan 40 percent of the farmers’ problems. Althoughmuch of the reasoning for this perceived shortcom-ing lay in the lack of resources of the farmers, roughlyone-third of the extension agents said that the majorobstacles preventing them from assisting farmers asthey should were constraints imposed on them bythe extension agency that limited them to dissemi-nating knowledge gained during their monthly train-ing, and by their own ignorance. Insights such asthese, gained during the conversational interview-ing that is central to the BA approach, can be invalu-able to the management of an extension agency as itseeks to improve the performance of its staff, andthereby enhance the quality of its services to thefarmer.

studies, respectively, and take more time, orinvolve larger teams of interviewers.

Preparation of Interview Guides

The interview guide plays an important role inthe implementation of the BA. Given that this isthe main instrument used to obtain informationfrom the assessment, the development of its con-tent should be a collaborative effort largely be-tween the project manager and the BA studydirector. It should also involve the BA teammembers to obtain their inputs and providethem a sense of ownership of the exercise.

The BA technique of conversational inter-viewing uses as guides themes based on areasof interest and operational relevance to projectmanagement, instead of administering mostlyclosed questions, as is generally done in tradi-tional surveys. This technique allows for asmoother flow of information, and often bringsto light new information that was not previouslyconsidered by project management. The follow-ing are sample interview themes:

• How has agricultural production changed(increased/decreased) over the previousthree years?

• Why has this change occurred?• What knowledge and information did farm-

ers obtain from contact with the extensionservice?

• How many farmers obtained informationdirectly from the extension service? Howmany obtained information indirectlythrough their friends, neighbors, or rela-tives?

• How useful was the information obtainedfrom the extension service? How manyfarmers adopted new technologies or inno-vations as a result of this information?

• How effective was the extension service andthe extension agent?

• How could the extension service be im-proved?Interview guides need to be tailored to the

particular group thus separate guides wouldneed to be developed for farmers, extensionagents, and government officials. Regardless of



who is being interviewed, the guide should notinclude more topics than can readily be memo-rized by the interviewer and discussed duringa one hour interview.

Although the dominant mode of inquiryused in beneficiary assessments is the qualita-tive technique of conversational interviewing,there are certain kinds of data that can best beaddressed in a quantitative way. These includetopics requiring prioritization, and certaintouchstone issues that may best be reduced to asimple yes/no response such as: Have youheard of the extension agency? On balance, haveyou benefited from this extension service? Wereyou consulted on the kind of assistance youwanted from the extension agency?

Methodology

Conversational Interviews

A conversational interview is the basic tool ofinquiry for the BA practitioner. These interviewsoften take place in the home of the interviewee,where the farmer will be most comfortable. In-terviews should be conducted in the local dia-lect, in such a way that open-ended questionsrevolve around a number of themes or topicsthat project management has selected. The ob-jective is to gain in-depth information on ben-eficiary views in relation to a planned orongoing activity, by encouraging beneficiariesto speak freely and bring to light issues of con-cern to project management.

Interviews may be conducted on a one-to-one basis or in focus groups. The advantages ofindividual interviews are that people are likelyto speak more freely, without worrying whatpeers or other community members may think.Lower-status or introverted members of com-munities may not feel comfortable speakingout in groups. Use of focus group interviewspermits a wider coverage of people, and mayprovoke insightful commentary stimulated bypeer interaction. Focus groups are, however,

more difficult to quantify, and attribution ofresponses to specific individuals more diffi-cult.

In undertaking conversational interviews,the interviewer should establish trust and goodrapport with the respondent to enhance the like-lihood of gaining unsolicited information(which could be as important as the thematicareas identified in the interview guide). The tim-ing of the interview, its duration, and the timeof day, should all depend on what is most con-venient for the intended beneficiary. Interviewsshould be completed in under one hour (bothto accommodate interviewees and to facilitaterecall). Note-taking should be kept to a mini-mum, and should be completed as soon afterthe interview as possible.

Unguided discussion is apt to be vague andtherefore of little use for decisionmaking; prob-ing for specifics is often required. For example,if the intended beneficiary of an extensionproject stated that he or she did not benefit fromit, the interviewer should probe to find reasonsfor this dissatisfaction, preferably prioritizingreasons (up to three).

Conversational interviews are simply a wayto approach a subject in a natural manner. Ask-ing direct questions on sensitive topics can putpeople on the defensive; responses given maybe ones that the respondent feels the interviewerwants to hear. The use of indirect questioningis meant to elicit a more valid expression ofopinion, or of fact. Below are examples of inter-viewing techniques.• Assessment of quality of extension service: “I’ve

noticed that a number of farmers in thiscommunity have experienced good growthwith their crops this year [including yours,if noticeable]. I know the extension servicehas been active in this area. The goal ofextension is to increase agricultural produc-tion and thereby improve the living condi-tions of farmers…”

• Responsiveness of project to needs and priori-ties of farmers: [For those farmers who are

7

familiar with the extension message] “Insome parts of this country extension workhas clearly had a positive effect on agricul-tural production. Would you say this is thecase with your particular farm? When theextension agent visits you (or a neighbor),does he spend time listening to your con-cerns or is he mainly transmitting messagesof his own?”

• Role of farmer-to-farmer information exchange:“Clearly extension agents do not have thetime to contact all farmers individually. Per-haps you or others in the community re-ceive information from other farmers whohave had direct contact with extensionworkers …”

• Suggestions to improve project: “Putting your-self in the place of the head of the agricul-tural extension agency, what would you doto best serve the needs of the farmer?” or“Some farmers complain about the exten-sion service, some praise it. In a poor coun-try like this extension has its constraints, likeall government services. Yet few woulddeny that extension could do better. Thechallenge is to give specific suggestions forimprovement…”

Conversational interviewing with both members andnonmembers of contact groups (CGs) in ten Africancountries revealed that in all but one (Mali) the CGgenerally failed in its task as diffuser of informationreceived from the extension worker. Part of the prob-lem was the minimal supervision and publicity givento contact groups. In Senegal, perhaps the countrywhere CGs were weakest, a number of farmers whowere listed as belonging to CGs did not, when inter-viewed, know that they were members. Only a thirdof the farmers in areas covered by the extensionagency knew of the existence of CGs. Eliminating theCG members, fully 90 percent of the non-CG farmersdid not know anything about the existence of CGs.While part of the problem lay in the absence of aninformation campaign relating to CGs on the part of

the extension agency, another clear obstacle was thelack of incentive for the CG member to pass informa-tion on to his non-CG member farmer. These insightsregarding the degree of knowledge about the CG andthe lack of motivation of the CG member to relay ex-tension messages to non-CG neighbors came directlyout of the conversational interviewing of farmers. Theimportance of these insights can be seen in the deci-sions to strengthen CGs in several countries ordownplay them, to the point of elimination in Senegal,and in a general trend toward more participatory ex-tension models involving entire communities.

Source: Lawrence F. Salmen,2000. “The Voice of the Farmerin Agricultural Extensions,” AKIS Discussion Paper, WorldBank.

Box 2 Interviews with Extension Contact Group Members and Nonmembers

Focus Group Discussions

In addition to enabling a wider coverage of thebeneficiary population in a given time period,focus group interviews can serve as a cross-check to individual interviews carried out in theBA. The groups should normally have 6-12people with common characteristics (for ex-ample, groups of intended beneficiaries may becomposed of married women, male heads ofhouseholds, youth from 15 - 25). There are times,however, when it may be of use to purposelymix the constituents of a focus group—say, withmembers and nonmembers of contact groups—to better appreciate the nature of conflict andcommunication between them, and provide theopportunity for indigenous solutions.

The interview guide should be used in con-ducting these interviews. The interviewer takeson a facilitative role, guiding the discussion tocover topics from the thematic guide and en-suring that everyone has an opportunity to par-ticipate. This will generally entail encouragingthe more reticent, introverted persons to speakup while providing less encouragement to thosemost apt to dominate the discussion. A re-searcher should also be present to take notes.



While the difficulty of quantifying focus groupdiscussions may be considered a liability, theirutility as a cross-check and as a fairly rapid andeasy-to-read barometer of the mood of a com-munity on many topics make focus groups auseful component of the BA approach.

Participant Observation

This technique generally involves living in atarget community for a lengthy period. Duringthis stay, it is expected that the participant ob-server will establish enough rapport and in-volvement to enable him/her to representaccurately the conditions within the communityas they relate to project objectives. The partici-pant observer normally spends from one tothree weeks in a given community. The re-searcher will focus on the areas of concern iden-tified in the interview guide. Particular issuesrelated to agricultural extension that have beenthe object of inquiry using participant observa-tion include: the relationship between contactgroups and other community associations, therole of gender in dissemination of the extensionmessage, and the nature and extent of interac-tion between an extension agent and the com-munity served.

During this stay in the community, the par-ticipant observer should prepare case studiesof 5-10 households based on repeated visits andobservation. Participant observation, beingcostly and time-consuming, should be used se-lectively on topics of particular interest, that areof a sensitive nature, and that lend themselvesto this form of intensive personal interaction.

Because participant observation is time-con-suming and costly, it may not be feasible to in-clude it in many BAs. The value of close,protracted observation of persons who becometransformed from interviewees to acquaintancesand perhaps friends should, however, be recog-nized and built in to all BA work. Part of the rea-son for the success of the BAs done on agriculturalextension in Guinea for example, was that the in-

terviewers lived in each community in the samplefor roughly 5 days. This mini-participant obser-vation provided an in-depth understanding of thecultural context surrounding the project interven-tion, which gave added weight and relevance tothe BA team’s observations.

Important factors in undertaking partici-pant observation include:• The reason for the participant observer’s stay

should be explained to everyone at the outset:Communities, while informed of the natureof the participant observer’s stay in the com-munity, should ultimately see the partici-pant observer as more than an acquaintance,more like a friend.

• There should not be over-identification with anygroup, but rather accessibility to all: A few closecontacts from diverse major segments of thepopulation should be cultivated. Theyshould represent key social groups such aslandowners and landless members and non-members of contact groups. The participantobserver should join in major organizationsand activities of the community, to be ap-preciated and identified as a participant.The observer should retain independenceyet demonstrate some level of involvementin the affairs of the community.

• The focus of inquiry must be consistent with thatof the BA: The participant observer shouldbase structured conversations on topics inthe interview guide, and discuss relevant is-sues emerging from the guide with repre-sentative members of the community.

Institutional Assessment

Successful development is promoted and sus-tained by institutions. Although people arerightfully said to be the focal point of develop-ment, it is the way people are organized andrepresented in institutions that gives them thevoice and opportunity for their own advance-ment. Despite their obvious importance, how-ever, institutions are often neglected in social

9

analysis work. Beneficiary assessments for ag-ricultural extension must include an under-standing of those institutions necessary for thesound and sustained workings of extension, asan integral part of their analysis. Key institu-tions would include: community associations(those that represent the entire community,leadership councils, women’s associations,youth and sports clubs, and credit unions);NGOs that work in the community; and localgovernment. In each institution there are vari-ous stakeholders or target informants to be in-terviewed around certain cogent topics.

Informant groups to interview for the insti-tutional assessment include:• Institutional leaders (all);• Members (sample stratified by age, gender,

and if relevant, income/class, ethnicity);and

• Nonmembers (small sample, similar strati-fication as preceding);

• Community leaders, local government of-ficials.The institutional assessment should attempt

to determine the role of the group in relation tothe extension program. Topics for the interviewwould include :

• Objectives of the institution and the de-gree to which these objectives have beenachieved;

• Criteria for membership (if any);• Track record—major accomplishments;• Role as development agent; potential as

catalyst for participation;• Relationship to other institutions, public

and private; and• Potential as vehicle for empowerment; ef-

fectiveness.Although the BA would usually include in-

terviews with senior government officials, athorough understanding of national institutionsis not within the purview of most BAs of agri-cultural extension. The focus for institutionalassessment in the BA of extension is at the mi-cro level, where the farmers live and work andwhere extension must be effective to succeed.

These four techniques of conversational in-terviewing, focus groups, participant observa-tion, and institutional assessment are central tothe BA approach; other survey techniques, rang-ing from traditional questionnaires to thoseused in participatory rural appraisal (PRA), mayusefully be introduced to complement the fourmain techniques.

Training of Field Researchers

The training of local researchers should takefrom two to five days depending on the experi-ence of team members. The training takes theform of orientation, because BA is a simple-to-use technique, and should be relatively easy toimplement. The following (collapsible) trainingschedule is recommended:

All field researchers would undergo the firsttwo days of training:

Implementation

Day 1:• Introduction to the project and its objectives;• Introduction of the BA approach and dis-

tribution of course materials;• Illustration of BA as a management tool to

be used at all phases of the project cycle (de-sign, implementation, and evaluation),through presentation and discussion of casestudies; and

• Familiarization with the reading materialsprovided.

Implementation


Day 2:• Discussion of the following BA techniques:

conversational interviews, focus groups,and participant observation;

• Review of the interview guide;• Tips on report writing;• Simulation of individual interviews and fo-

cus group discussions using the interviewguide;

• Note taking based on simulation exercises;and

• Review of note-taking.Those field researchers with little experience

in qualitative research would continue for threeadditional days:

Days 3–4:• Field exercise and pretest of interview

guide. The team will choose one or morecommunities to apply the draft interviewguide. Most team members will conductconversational interviews with at least fivepersons selected at random as representa-tives of the diverse elements (such as age,gender, status) of the community. The re-mainder will conduct either focus groupsor participant observation.

Day 5:• Morning: Team members will review notes

and relate experiences to the entire group;and

• Afternoon: Critique by the study directorand the project manager; suggestions forimprovement of applied techniques and theinterview guide.Training should involve social fund staff,

even beyond those participating in the assess-ment work. Such involvement will allow fortailoring of the methodology to the particularcontext of communities and operations, andexposure of fund staff to participatory appraisaltechniques that may be incorporated into theirown work.

Monitoring and Evaluation

Adequate monitoring is a key aspect of the BAprocess. Monitoring and evaluation of the BAis the overall responsibility of the extensionagency manager, assisted by an external tech-nical advisor. This monitoring is necessary toensure that the BA stays true to its objectives.The monitoring process must ensure that BAresearchers are effectively gathering informa-tion and not biased in their recording of inter-views, and that information gathered is relevantto project management needs.

The most crucial moment for the monitor-ing of the BA is the interim review, whichshould come roughly one-third of the waythrough the fieldwork. The BA team shouldprepare a brief progress report for this reviewpresenting initial findings, suggesting revisionsto the interview guide, and making other rec-ommendations for BA improvement. This in-terim review allows for changes in the BA basedon actual field experience, yet still leaves suffi-cient time remaining for the improvements toaffect the overall assessment.

When beneficiary assessment work coversan extensive area, logistical and budgetary con-straints may preclude persons coming togetherfor a mid-term review. In this case, the SF man-ager and the BA director should visit each sitecovered by a regional team and hold mini-in-terim reviews on progress achieved in each area.Findings from these localized reviews can thenbe compared and synthesized at national head-quarters, giving an understanding of overallprogress with the BA.

Preparation of the Final BA Report

The final beneficiary assessment report is a criti-cal part of the BA process, because it summa-rizes the findings of the field research and alsobecause the recommendations it provides serveas a guide to project management. Given the

11

goal of serving a wide range of users, two kindsof final reports for the BA may be considered:the comprehensive for discussion with donors,and the abbreviated for local use. The compre-hensive final report should attempt to quantifyfindings to the greatest extent possible. Re-sponses to a few key questions should be cat-egorized according to thematic areas of theinterview guide and presented as percentages.The findings of focus group discussions shouldbe summarized by groups and by the regionswhere they took place. The report should focusas much as possible on issues of relevance andimportance to project management.

The comprehensive final BA report shouldcontain the following:• An executive summary;• An introduction that gives background

information on the, the objectives of theBA, and a description of the methodologyused;

• Findings of the BA as they relate to the in-terview guide (this section should includetabular data);

• Any other relevant information; and• Conclusions and recommendations.

The abbreviated report should contain onlythe essentials of sample size, interview guide,conclusions (as related to key findings), andrecommendations. The length of final reportswill vary according to the subject matter, need,

and context (5-10 pages for abbreviated, and 25-40 pages for comprehensive).

Internalization

Much as an intended beneficiary has to gainownership of a development initiative for it totake root and flourish, so the agricultural ex-tension agency has to own a beneficiary assess-ment so that it too takes hold and becomes theinstrument of change it is intended to be. Expe-rience demonstrates that BA work will be farmore likely to gain its desired impact if it hasan institutional home within the implementingagency. Although many of those who conductthe BA should be external to the agency, theyneed a counterpart unit within the agency toserve as intermediary between themselves andthe extension management. The functions of thisunit would be to (a) assist agency managementwith the BA design; (b) participate in the imple-mentation of the BA; (c) monitor progress;(d) assist in the preparation and disseminationof the final report; and most important, (e) helpinternalize the findings and recommendationsof the BA in such a way that they result in con-crete changes in the practices and policies of theagency. In this way the feedback from the com-munities of intended beneficiaries has the de-sired effect of improving the work of theextension agency.

Dissemination and Documentation of BA Impact

Beneficiary assessment is an integral part of anextension agency’s developmental activity; assuch, its findings are to be used to further theobjectives of the agency. The major audience forthese findings is the management of the agency

itself. Other audiences are also important, how-ever. Dissemination of the findings of a benefi-ciary assessment should start with the origin ofthese findings, the communities of intended ben-eficiaries, for two reasons: First, these beneficia-

Dissemination and Documentation of BA Impact


ries can confirm or refute findings, thereby rein-forcing the message or calling for further inquiry.Second, as a matter of respect: this is the storytold by members of the community for their ownbetterment, so they should be the first to hear it.

A second external audience for dissemi-nation is the agricultural policy formationsector of government. Sharing BA results withthese public sector officials will help bringabout needed program coordination as wellas enriching policy formulation with exten-sion experience. A third audience is the WorldBank and other donors. Both government anddonors may best be reached through semi-nars, workshops, and roundtable discussions.When the lessons of the BA have wide appli-cation, the general public may be reached viapublications.

Finally, an important part of the BA pro-cess that is often neglected is the documenta-tion of how BA findings have affected projectactivities. This should be done at the end of theBA, after the report is given to project manage-ment. The task manager should make sure thatany follow-up actions that are taken as a resultof BA findings be kept in the project file as wellas the final BA report. A manager appointedwhile a project is going on will thus have up-to-date information on the BA, and be likely tocontinue to follow the approach in use. By docu-menting the process, findings, and impact ofbeneficiary assessments, and keeping this infor-mation as a permanent part of the file, the BAprocess becomes a learning and feedback com-ponent generating continuous project improve-ment.

The candidate who wishes to conduct a benefi-ciary assessment should address each of the fol-lowing issues. Elaboration of each is providedin this manual.

Rationale and Objectives

Why is this beneficiary assessment (BA) beingcarried out? Who are the primary and second-ary audiences for it? What are the major objec-tives of the BA?

Generally, the beneficiary assessment isdone to provide feedback to the managementof the agricultural extension agency, whichshould result in project improvements. Second-ary audiences would include donors and thehost-country government, central and local.More particular objectives might include: de-termining the level of satisfaction of intendedbeneficiaries, understanding the degree andmanner in which farmers have participated invarious phases of the implementation of theextension projects, and learning how stakehold-ers feel the project could be improved.

Methodology

Beneficiary assessment is a qualitative re-search tool used to provoke policy and pro-gram change. As such, to the degree possible,findings are to be quantified. The core tech-niques of BA are: (a) conversational interview-ing among representative groups of keystakeholders (intended beneficiaries, contrac-tors, NGOs, government officials); (b) focusgroup discussion, particularly with intendedbeneficiaries; (c) participant observation, and(d) institutional assessment. This core may

Annex: Terms of Reference for BeneficiaryAssessments for Agricultural Extension

readily be complemented by such PRA tech-niques as mapping, wealth ranking, and Venndiagrams.

Sample Framework

The sample size must be established accordingto what is considered significant by the exten-sion agency’s management. Given the use of in-depth probing and qualitative techniques,smaller samples than what are normally con-sidered to be statistically significant will suffice,although they must allow for meaningful cross-tabulation and be of sufficient size to be usefulfor decisionmaking. Samples should be repre-sentative of both numbers of people reachedand numbers of subprojects funded. Stratifica-tion should be by gender, ethnicity (where rel-evant), project type (health, education, water),and region of country.

Research Issues/ Interview Guide

The research issues for the BA will be deter-mined by the extension agency manager in con-sultation with the BA director. They will beaddressed largely by interviewing, using a ba-sic interview guide that may be modified foruse with different stakeholder groups. Thisguide would include the following topics:• Exposure to extension service—how people

learned of its existence and what they knowabout it;

• Participation—degree and nature of in-volvement in decisions regarding extensionin community;

• Dissemination—utility of contact group/family as relay for extension message;

13


• Partnerships—collaboration with other en-tities: local governments, NGOs, private sec-tor; degree, utility, advisability for each;

• Satisfaction with extension regarding objec-tives, mode of operation, and relevance tofarmers’ needs; and

• Recommendations for improvements in ex-tension service.

Dissemination

The value of a beneficiary assessment correlatesdirectly with the effect it has on influencing ac-tion. Whereas the extension agency manage-ment is the immediate and, generally, mostimportant consumer of BA findings, a numberof other groups should benefit from these aswell: the intended beneficiaries of the project,the local (and international) NGO community,and local and central government. An impor-tant component of the TOR for a BA done onextension work is a dissemination plan detail-ing how these various stakeholders are goingto be reached: papers, workshops, seminars,audiovisual media, and other means.

Schedule

A brief breakdown of the phases of BA activityshould include:

Training and field testing of interview guide—usually one week

Field work (including interim progress review)—usually 4-6 weeks

Data tabulation and analysis—usually 2 weeksFinal report preparation—usually 2 weeks

Total time for most BAs is from two to threemonths. This amount of time varies consider-ably from project to project, the sample size,experience of the BA team, and logistical diffi-culties in reaching interviewees. It is importantto keep in mind that this schedule is for oneround of interviewing. Beneficiary assessmentis best conceived and practiced as an iterativetool providing periodic feedback to a social fundover time, at say 18-month intervals.

Budget

This will vary even more than schedules, ac-cording to local costs. The key categories hereare personnel, expenses (food and lodging),travel, and office supplies. Fees for personsskilled in qualitative research techniques areconsiderably higher than those paid to enu-merators who apply questionnaires. The aver-age cost for BA work on agricultural extensionhas fallen in the range of US$20,000 to US$60,000per survey.

beneficiary assessment for agricultural...

Documents