thinking and evaluating systemically · piala pilots: • ghana 2015, roots & tubers...

Extractofconfigura.onalanalysistable:

Thinking and evaluating systemically The use of PIALA (Participatory Impact Assessment & Learning Approach)

for evaluating commodity chain development in Ghana

Adinda Van Hemelrijck (adinda@piala.org)

Glowen Kyei-Mensah (gkmensah@pdaghana.com)

ASSESStowhatextentimpactsoccurred

(ornot)

DEBATEwhere/how

impactscanbeenhanced

EXPLAINwhy

impactoccurred(ornot)

PIALAwasdevelopedwithsupportfromIFAD

(Interna.onalFundforAgriculturalDevelop-

ment)andBMGF(GatesFounda.on)toaddress

challengesofcomplexityinevalua.ngagricul-

turalmarketsystemdevelopment.

Challenges:

• howtoevaluate‘opensystems’(medium-N)

• rigorouscausalinferenceintheabsenceof

crediblecontrolgroups

• howtotriggerlearningaboutcontribuFonsto

sustainableimpact

PIALApilots:

• Ghana2015,Roots&TubersImprovement

andMarke5ngProgramme(RTIMP)

• Vietnam2013,DoingBusinesswiththeRural

PoorProgramme(DBRP)

IntroducFon

PROCESS:

• Reconstructand

visualisethe

programme’sTheory

ofChange(ToC)

•  Iden.fycausalclaims

andmechanismson

whichtofocus

• Ar.culateassump-

Fonsandformulate

quesFons

• Createsharedunder-

standingandowner-

shipoftheToCand

focusofevalua.on

DESIGNDECISIONS:

• Scope&scale

Importantfor

drawingconclusions

aboutcontribu.ons

toandsustainability

ofimpact

• Levelengagement

Importantfor

ownershipand

systemiclearning

PHASE1:Focusing&framing

PROCESS:

• Sample‘open

systems’and

popula.onswithin

thesesystems

• DesignparFcipatory

methodsandsurvey

toinquirethecausal

linksintheToC

• Collectdatausingthe

samesetofmethods

acrossthesample

•  Inquirecausesand

explana.onsbyusing

‘sensing’methods

(e.g.causalflow

mapping)andby

cross-checkingand

collaFngdataalong

theToC

DESIGNDECISIONS:

• Counterfactual

approach

Importantfor

samplingof‘systems’

andpopula.ons

PHASE2:CollecFng&linking

PROCESS:

• Organisesense-

makingworkshops

toengagestake-

holdersinvaluing

contribu.onsto

impact

• Analyseandcompare

differentsystemic

configuraFonsacross

thesampleof

‘systems’

DESIGNDECISIONS:

• Scaleandlevelof

engagement

Importantforsolid

debateenabling

‘voice’(e.g.nrof

workshopsand%

beneficiariesatlocal

andaggregated

levels)

PHASE3:AnalysingPIALA Purposes

Causalclaims&linksintheToCoftheRTIMP:

M2b: Training & starter pack for commercial

seed growers to multiply certified R&T seeds

C3a: R&T processors grow and develop into

GPCs that are profitable enterprises

O3: Enhanced R&T

processed volumes of

high quality at scale

O2: Enhanced

R&T productivity

and production

at scale

M2c: Farmer Field Forums (FFF) engage

farmers, extension agents and researchers in

developing, demonstrating and promoting

appropriate R&T production technologies

C2a: Resource-poor R&T farmers & seed

producers gain access to and adopt improved

R&T seed varieties, technologies & inputs to

improve crop husbandry, soil fertility and

pest management practices

C2b: Resource-poor R&T farmers organise

and register as FBOs that can access credit

and bargain better market prices

C1b: Resource-poor R&T processors, farmers

& seed producers commercialize and establish

effective supply chain linkages

C1a: R&T supply chain farmers & processors

are capable of developing and implementing

viable business and marketing plans

C3c: R&T supply chain farmers and

processors gain access to business financing

and market-linking services

M3b: Subsidized upgrading of advanced R&T

processors into Good Practice Centres (GPCs)

that demonstrate and promote good quality

processing & management practices

C3b: R&T supply chain processors gain

access to and adopt standardized processing

technology and good quality

management practices

O1: R&T supply chain

actors effectively solve

their supply & demand

issues and timely obtain

technical support,

resulting in sustainable

and inclusive CCs

linked to old and new

markets

I2: Improved R&T-

based livelihoods for the

rural poor in CC

catchment areas

M2a: R&D for developing bio-agents

M1c: Information, Education &

Communication (IEC) about CC support

services, inputs and technologies

M1a: Training of resource-poor farmers and

processors involved in the R&T supply chains in

business development and marketing

M3c: Co-financing of R&T supply chain farmers

and processors by matching 40% RTIMP funds

with 50% loans from PFIs and 10% self-financing

through the Micro-Enterprise Fund (MEF)

M3a: Training of artisans to produce and maintain

standardized processing equipment

for R&T supply chain processors and GPCs

I1: Rural poor people

in CC catchment areas

have increased access

to food & income to

sustain an active and

healthy life

M1: District Stakeholder

Forums (DSFs) for addressing

supply & demand issues and

technical support needs of R&T

supply chain actors members

M1b: Supply Chain Facilitation (SCF) and

market linking through the Initiative Fund (IF)

EC1

EC3c

EC1

EO2

EO1

EC3b

Systemic Theory of Change Approach!

Contribution Claim of RTIMP Component 3

ê Enhanced Processing (O3)

Contribution Claim of RTIMP Component 2

ê Enhanced Production (O2)

Contribution Claim of RTIMP Component 1

ê Enhanced Market-Linking (O3)

Contributions of RTIMP Components 1, 2 & 3

ê Improved Livelihoods (I2)

DSF FFF GPC MEF

MEF GPC Evidence Strength

FFF C2a+C2b àO2

Evidence Strength

M1c+M1b+

O3+O2

+O1àC1b

DSF Evidence Strength

O1+O2+O3 àI2

Evidence Strength (M3c)+C1a+M3b

àC3c

(M3b)+C3c àC3b

à O3

M2a+M2b+

(M2c) àC2a C1a+(M1) àO1

Tano North (Apesika) (CZ) 1 1 1 1 3 6 5 5 5 5 4 4 5 5 5

Techiman (CZ) 1 1 1 1 4 5 5 5 5 5 4 4 5 5 5

Gomoa East (SZ) 1 1 1 0 2 5 3 5 5 5 4 4 5 5 6

Assin South (SZ) 1 1 1 1 3 4 4 6 5 4 3 3 4 4 4 Birim Central (CZ) 1 1 1 1 3 3 4 5 5 4 3 4 4 4 5

Nkwanta South (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5

Upper West Akim (CZ) 1 1 1 1 2 4 4 5 5 4 3 3 5 4 5

Ashanti Mampong (CZ) 1 1 1 1 3 4 5 5 5 5 3 3 5 4 5

West Gonja (Damongo) (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5

Abura Asebu Kwamankese (SZ) 1 1 1 1 3 3 5 5 5 6 3 3 5 4 4

Nanumba North (NZ) 1 1 N/A N/A 5 5 5 3 3 5 4 5 East Gonja (NZ) 1 1 N/A N/A 4 3 5 3 3 5 4 5

Central Gonja (NZ) 1 1 N/A 2 3 5 5 4 5 2 2 5 4 5

Suhum (CZ) 1 1 1 0 3 4 5 4 4 5 2 3 5 3 5

Adansi South (CZ) 1 1 1 1 2 4 5 4 3 4 2 2 3 3 5

Ahafo Ano South (CZ) 1 1 1 0 2 2 4 5 4 5 2 2 5 3 5

Kintampo South (CZ) 1 1 N/A N/A 4 4 3 2 3 5 3 5

Wa East (NZ) 1 1 0 0 2 2 5 4 5 5 2 3 5 3 5

North Dayi/ Kpando (SZ) 1 1 1 0 2 2 6 2 3 5 2 2 5 2 5

Agona East (SZ) 0 0 1 0 2 3 6 4 4 4 2 2 6 2 6

Pru (CZ) 0 0 N/A N/A 2 2 4 2 2 5 2 5

Ho Municipal (SZ) 1 1 0 0 3 2 5 5 5 5 3 3 5 2 5

Tano North (Dua Yaw Nkwanta)

(CZ) 1 1 1 1 2 2 4 3 3 4 2 2 3 2 3

Wassa Amenfi West (SZ) 0 0 N/A 1 2 5 4 4 5 1 1 6 2 5

Kumasi Metropolitan (CZ) 0 0 N/A 1 1 4 1 1 4 1 1 4 1 4

Sampled districts"

ê"

Scoring of consistency & strength "of each causal link and evidence (0-6)"

Presence of mechanisms

(1=yes ; 0=no) "

PIALApilotinGhana

30randomsupplychainsystemsof4commodi.esin25districtsin8regions

836randomhouseholds1180quasi-randombeneficiaries(45%women)

750parFcipants(>30%beneficiaries)in23localand1na.onalsensemakingworkshops

PIALAisanapproach,notasingle

methodology.Itcanembedanymethod,so

longitisconsistentwithprinciplesofsystemic

thinkingandenablingvoice.

PIALA phases & elements

PHASE 1:

Focusing & framing

the evaluation"

PHASE 2:

Collecting & linking

the data"

PHASE 3:"

Analysing

contributions "

RIGOUR"

INCLUSIVENESS" FEASIBILITY"

Systemic ToC approach"

Participatory mixed-methods"

Multi-stage sampling of/in ‘open systems’"

Participatory sensemaking"

Configurational analysis"

6!

hbps://www.ifad.org/topic/

overview/tags/piala