thinking and evaluating systemically · piala pilots: • ghana 2015, roots & tubers...
TRANSCRIPT
Extractofconfigura.onalanalysistable:
Thinking and evaluating systemically The use of PIALA (Participatory Impact Assessment & Learning Approach)
for evaluating commodity chain development in Ghana
Adinda Van Hemelrijck ([email protected])
Glowen Kyei-Mensah ([email protected])
ASSESStowhatextentimpactsoccurred
(ornot)
DEBATEwhere/how
impactscanbeenhanced
EXPLAINwhy
impactoccurred(ornot)
PIALAwasdevelopedwithsupportfromIFAD
(Interna.onalFundforAgriculturalDevelop-
ment)andBMGF(GatesFounda.on)toaddress
challengesofcomplexityinevalua.ngagricul-
turalmarketsystemdevelopment.
Challenges:
• howtoevaluate‘opensystems’(medium-N)
• rigorouscausalinferenceintheabsenceof
crediblecontrolgroups
• howtotriggerlearningaboutcontribuFonsto
sustainableimpact
PIALApilots:
• Ghana2015,Roots&TubersImprovement
andMarke5ngProgramme(RTIMP)
• Vietnam2013,DoingBusinesswiththeRural
PoorProgramme(DBRP)
IntroducFon
PROCESS:
• Reconstructand
visualisethe
programme’sTheory
ofChange(ToC)
• Iden.fycausalclaims
andmechanismson
whichtofocus
• Ar.culateassump-
Fonsandformulate
quesFons
• Createsharedunder-
standingandowner-
shipoftheToCand
focusofevalua.on
DESIGNDECISIONS:
• Scope&scale
Importantfor
drawingconclusions
aboutcontribu.ons
toandsustainability
ofimpact
• Levelengagement
Importantfor
ownershipand
systemiclearning
PHASE1:Focusing&framing
PROCESS:
• Sample‘open
systems’and
popula.onswithin
thesesystems
• DesignparFcipatory
methodsandsurvey
toinquirethecausal
linksintheToC
• Collectdatausingthe
samesetofmethods
acrossthesample
• Inquirecausesand
explana.onsbyusing
‘sensing’methods
(e.g.causalflow
mapping)andby
cross-checkingand
collaFngdataalong
theToC
DESIGNDECISIONS:
• Counterfactual
approach
Importantfor
samplingof‘systems’
andpopula.ons
PHASE2:CollecFng&linking
PROCESS:
• Organisesense-
makingworkshops
toengagestake-
holdersinvaluing
contribu.onsto
impact
• Analyseandcompare
differentsystemic
configuraFonsacross
thesampleof
‘systems’
DESIGNDECISIONS:
• Scaleandlevelof
engagement
Importantforsolid
debateenabling
‘voice’(e.g.nrof
workshopsand%
beneficiariesatlocal
andaggregated
levels)
PHASE3:AnalysingPIALA Purposes
Causalclaims&linksintheToCoftheRTIMP:
M2b: Training & starter pack for commercial
seed growers to multiply certified R&T seeds
C3a: R&T processors grow and develop into
GPCs that are profitable enterprises
O3: Enhanced R&T
processed volumes of
high quality at scale
O2: Enhanced
R&T productivity
and production
at scale
M2c: Farmer Field Forums (FFF) engage
farmers, extension agents and researchers in
developing, demonstrating and promoting
appropriate R&T production technologies
C2a: Resource-poor R&T farmers & seed
producers gain access to and adopt improved
R&T seed varieties, technologies & inputs to
improve crop husbandry, soil fertility and
pest management practices
C2b: Resource-poor R&T farmers organise
and register as FBOs that can access credit
and bargain better market prices
C1b: Resource-poor R&T processors, farmers
& seed producers commercialize and establish
effective supply chain linkages
C1a: R&T supply chain farmers & processors
are capable of developing and implementing
viable business and marketing plans
C3c: R&T supply chain farmers and
processors gain access to business financing
and market-linking services
M3b: Subsidized upgrading of advanced R&T
processors into Good Practice Centres (GPCs)
that demonstrate and promote good quality
processing & management practices
C3b: R&T supply chain processors gain
access to and adopt standardized processing
technology and good quality
management practices
O1: R&T supply chain
actors effectively solve
their supply & demand
issues and timely obtain
technical support,
resulting in sustainable
and inclusive CCs
linked to old and new
markets
I2: Improved R&T-
based livelihoods for the
rural poor in CC
catchment areas
M2a: R&D for developing bio-agents
M1c: Information, Education &
Communication (IEC) about CC support
services, inputs and technologies
M1a: Training of resource-poor farmers and
processors involved in the R&T supply chains in
business development and marketing
M3c: Co-financing of R&T supply chain farmers
and processors by matching 40% RTIMP funds
with 50% loans from PFIs and 10% self-financing
through the Micro-Enterprise Fund (MEF)
M3a: Training of artisans to produce and maintain
standardized processing equipment
for R&T supply chain processors and GPCs
I1: Rural poor people
in CC catchment areas
have increased access
to food & income to
sustain an active and
healthy life
M1: District Stakeholder
Forums (DSFs) for addressing
supply & demand issues and
technical support needs of R&T
supply chain actors members
M1b: Supply Chain Facilitation (SCF) and
market linking through the Initiative Fund (IF)
EC1
EC3c
EC1
EO2
EO1
EC3b
Systemic Theory of Change Approach!
Contribution Claim of RTIMP Component 3
ê Enhanced Processing (O3)
Contribution Claim of RTIMP Component 2
ê Enhanced Production (O2)
Contribution Claim of RTIMP Component 1
ê Enhanced Market-Linking (O3)
Contributions of RTIMP Components 1, 2 & 3
ê Improved Livelihoods (I2)
DSF FFF GPC MEF
MEF GPC Evidence Strength
FFF C2a+C2b àO2
Evidence Strength
M1c+M1b+
O3+O2
+O1àC1b
DSF Evidence Strength
O1+O2+O3 àI2
Evidence Strength (M3c)+C1a+M3b
àC3c
(M3b)+C3c àC3b
à O3
M2a+M2b+
(M2c) àC2a C1a+(M1) àO1
Tano North (Apesika) (CZ) 1 1 1 1 3 6 5 5 5 5 4 4 5 5 5
Techiman (CZ) 1 1 1 1 4 5 5 5 5 5 4 4 5 5 5
Gomoa East (SZ) 1 1 1 0 2 5 3 5 5 5 4 4 5 5 6
Assin South (SZ) 1 1 1 1 3 4 4 6 5 4 3 3 4 4 4 Birim Central (CZ) 1 1 1 1 3 3 4 5 5 4 3 4 4 4 5
Nkwanta South (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5
Upper West Akim (CZ) 1 1 1 1 2 4 4 5 5 4 3 3 5 4 5
Ashanti Mampong (CZ) 1 1 1 1 3 4 5 5 5 5 3 3 5 4 5
West Gonja (Damongo) (NZ) 1 1 1 0 3 4 5 5 4 5 3 3 5 4 5
Abura Asebu Kwamankese (SZ) 1 1 1 1 3 3 5 5 5 6 3 3 5 4 4
Nanumba North (NZ) 1 1 N/A N/A 5 5 5 3 3 5 4 5 East Gonja (NZ) 1 1 N/A N/A 4 3 5 3 3 5 4 5
Central Gonja (NZ) 1 1 N/A 2 3 5 5 4 5 2 2 5 4 5
Suhum (CZ) 1 1 1 0 3 4 5 4 4 5 2 3 5 3 5
Adansi South (CZ) 1 1 1 1 2 4 5 4 3 4 2 2 3 3 5
Ahafo Ano South (CZ) 1 1 1 0 2 2 4 5 4 5 2 2 5 3 5
Kintampo South (CZ) 1 1 N/A N/A 4 4 3 2 3 5 3 5
Wa East (NZ) 1 1 0 0 2 2 5 4 5 5 2 3 5 3 5
North Dayi/ Kpando (SZ) 1 1 1 0 2 2 6 2 3 5 2 2 5 2 5
Agona East (SZ) 0 0 1 0 2 3 6 4 4 4 2 2 6 2 6
Pru (CZ) 0 0 N/A N/A 2 2 4 2 2 5 2 5
Ho Municipal (SZ) 1 1 0 0 3 2 5 5 5 5 3 3 5 2 5
Tano North (Dua Yaw Nkwanta)
(CZ) 1 1 1 1 2 2 4 3 3 4 2 2 3 2 3
Wassa Amenfi West (SZ) 0 0 N/A 1 2 5 4 4 5 1 1 6 2 5
Kumasi Metropolitan (CZ) 0 0 N/A 1 1 4 1 1 4 1 1 4 1 4
Sampled districts"
ê"
Scoring of consistency & strength "of each causal link and evidence (0-6)"
Presence of mechanisms
(1=yes ; 0=no) "
PIALApilotinGhana
30randomsupplychainsystemsof4commodi.esin25districtsin8regions
836randomhouseholds1180quasi-randombeneficiaries(45%women)
750parFcipants(>30%beneficiaries)in23localand1na.onalsensemakingworkshops
PIALAisanapproach,notasingle
methodology.Itcanembedanymethod,so
longitisconsistentwithprinciplesofsystemic
thinkingandenablingvoice.
PIALA phases & elements
PHASE 1:
Focusing & framing
the evaluation"
PHASE 2:
Collecting & linking
the data"
PHASE 3:"
Analysing
contributions "
RIGOUR"
INCLUSIVENESS" FEASIBILITY"
Systemic ToC approach"
Participatory mixed-methods"
Multi-stage sampling of/in ‘open systems’"
Participatory sensemaking"
Configurational analysis"
6!
hbps://www.ifad.org/topic/
overview/tags/piala