program assessment guide_pelletier_5.1.12

The Program Assessment GuideStructuring Contextual Knowledge and Experience to Structuring Contextual Knowledge and Experience to

Improve the Design, Delivery and Effectiveness of Improve the Design, Delivery and Effectiveness of Nutrition InterventionsNutrition Interventions

David PelletierDavid Pelletier

Division of Nutritional SciencesDivision of Nutritional Sciences

Cornell UniversityCornell University

CORE Spring MeetingMay 1, 2012

Wilmington, DE

AcknowledgmentsAcknowledgments

Collaborators on the MN Tool ProjectCollaborators on the MN Tool Project Allison Corsi, Lesli Hoey, Suzanne Gervais, Allison Corsi, Lesli Hoey, Suzanne Gervais,

Cornell UniversityCornell University

Funders and Facilitators of the MN Tool ProjectFunders and Facilitators of the MN Tool Project The World BankThe World Bank Micronutrient InitiativeMicronutrient Initiative Centers for Disease ControlCenters for Disease Control UNICEFUNICEF GAINGAIN A2Z (USAID and Gates Foundation)A2Z (USAID and Gates Foundation)

?

The Challenge

Why Implementation Why Implementation Science?Science?

Efficacy-to-EffectivenessEfficacy-to-Effectiveness Knowledge-to-ActionKnowledge-to-Action Evidence-Based InterventionsEvidence-Based Interventions Evidence-Based DecisionsEvidence-Based Decisions Evidence-Based Policy MakingEvidence-Based Policy Making Scaling UpScaling Up Etc.Etc.

p2

OutlineOutline

1.1. Genesis of the projectGenesis of the project

2.2. Overview of the PAGOverview of the PAG

3.3. Adaptations and next steps for the PAGAdaptations and next steps for the PAG

4.4. Q&AQ&A

Genesis of This ProjectGenesis of This Project

Micronutrient Forum (May 2009) Micronutrient Forum (May 2009) • From Evidence to Large Scale ProgramsFrom Evidence to Large Scale Programs

The Innocenti Process (Sept 2008)The Innocenti Process (Sept 2008)• Country implementers, donors, researchersCountry implementers, donors, researchers• Review of evidence for effectiveness at scaleReview of evidence for effectiveness at scale• Identification of overarching issues needing attentionIdentification of overarching issues needing attention

Country implementers requested stronger Country implementers requested stronger guidance on identifying and resolving guidance on identifying and resolving implementation bottlenecksimplementation bottlenecks

The Micronutrient Program ToolsThe Micronutrient Program Tools Program Assessment Guide (PAG)Program Assessment Guide (PAG) Program Documentation Guide (PDG)Program Documentation Guide (PDG)

Laying the Groundwork

Three-DayWorkshop

Follow-Through

The PAG Process

3-12 months 1-3 years

Program Assessment Guide (PAG)Program Assessment Guide (PAG)

1. Action Plan to Address Barriers & Enablers

2. Operations Research Agenda

3. Issues for Inclusion in M&E

4. Strategic Plan to Build Support, Capacity & Sustainability

Outputs

Participatory Procedures To Strengthen:

• The Systematic Integration Of Evidence, Contextual Knowledge & Experience

• Shared understanding, commitment, ownership, motivation & capacity to advance the micronutrient agenda

•Links with the broader nutrition and health agendas in the country

Processes

Nine modules:• Purpose• Rationale• Instructions• Templates• Examples

Laying the GroundworkModule 1: Clarifying the Problem and the Solution Module 2: Goals and Associated ValuesModule 3: Delivery SystemsModule 4: Hard to Reach PopulationsModule 5: People, Roles and Responsibilities

Program Assessment Guide

Building or Strengthening the InterventionBuilding or Strengthening the Intervention Module 6: Needs, Inputs, Activities and System ChangesModule 6: Needs, Inputs, Activities and System Changes Module 7: Action PlanningModule 7: Action Planning Building Support Systems and the Enabling EnvironmentBuilding Support Systems and the Enabling Environment Module 8: Monitoring, Evaluating and Quality Improvement Module 8: Monitoring, Evaluating and Quality Improvement Module 9: Organizing, Leading and ManagingModule 9: Organizing, Leading and Managing

Program Assessment Guide

1. 1. Delivery Systems as Social Systems (Module 3)Delivery Systems as Social Systems (Module 3)

2. The Five Needs Tool (Module 6)2. The Five Needs Tool (Module 6)

3. The Stress-Testing Tool (Module 6)3. The Stress-Testing Tool (Module 6)

4. Clear definition of roles and responsibilities (Modules 5,7) and 4. Clear definition of roles and responsibilities (Modules 5,7) and responsibility for follow-up (Module 9)responsibility for follow-up (Module 9)

Some Distinctive Features of the PAG

Module 3: Delivery systems as social systems Module 3: Delivery systems as social systems

A

National Regional Clinic Community

D HJ

Functionaries (barriers and enablers)

Household& Caregiver

Significant Others (barriers and enablers)

CB EI

GF

District

National Oblast Clinic Community

Caregiver

Rayon

• How Will Gulazik Get from the National Level to All Caregivers?• How Will Gulazik Education Get to All Caregivers?

Module 3: Delivery systems as social systems Module 3: Delivery systems as social systems

Five Universal Needs1.Awareness2.Knowledge, Information, Skill3.Motivation/ Commitment4.Resources5.Support from Others

Module 6: Strengthening the delivery system

Village-levelHealth Volunteer

Role/Responsibility:

Provide mothers with information and education to encourage proper use of Gulazik

Awareness

Information, Knowledge,& Skills

Commitment/Motivation

Resources

Support from others

Inputs, Activities& System Changes

What is required in order for these needs to be met?

What does the village health volunteer need in order to be successful?





Module 6: What does each functionary and significant other NEED in order to fulfill their role/responsibility of the Sprinkles program (on a daily basis) and what inputs, activities and system changes are required to ensure their needs are met?

Systematizing and stress-testing contextual Systematizing and stress-testing contextual knowledge and experience (Module 6)knowledge and experience (Module 6)

Proposers Challengers




Awareness



Resources

Support from others








Identifying Functionaries and Significant Othersin the Delivery and Support Systems

A


D HJ

FunctionariesCaregiver

Significant Others

CB EI

GF

Rayon

Systematizing and stress-testing contextual Systematizing and stress-testing contextual knowledge and experience (Modules 6-8)knowledge and experience (Modules 6-8)

Proposers Challengers




Awareness



Resources

Support from others








Identifying Functionaries and Significant Othersin the Delivery and Support Systems

A


D HJ

FunctionariesCaregiver

Significant Others

CB EI

GF

Rayon

Actions to address barriers and enablers in the Action Plan

Questions for Operations Research

Issues to track in monitoring and QA

Outputs

Implementation WGs OperationsResearch WG

Monitoring, Evaluationand QI WG

Coordination & Support WG

Relationships Among Working Groups

Module 9: Responsibilities for Follow-Upto the Workshop

Experiences from Kyrgyzstan,Bolivia and HaitiExperiences from Kyrgyzstan,Bolivia and Haiti

1. Sponsorship, Responsibility, 1. Sponsorship, Responsibility,

Commitment, Follow-upCommitment, Follow-up

2. Timing2. Timing

3. Preparation3. Preparation

4. Participants4. Participants

5. Contextuality (selection and sequencing)5. Contextuality (selection and sequencing)

6. Time constraints6. Time constraints

7. Facilitation7. Facilitation

Before the workshop

During the workshop

Next Steps for the PAG

Produce additional versions for:Produce additional versions for:• More complex interventions (ag, livelihood, social protection, etc)More complex interventions (ag, livelihood, social protection, etc)• Integrated (multi-intervention) programsIntegrated (multi-intervention) programs• Sub-national planning (district, project level)Sub-national planning (district, project level)• Diverse countriesDiverse countries

Partner with NGOs to gain experience applying the PAG Partner with NGOs to gain experience applying the PAG in the above contextsin the above contexts

Develop and field test the Adoption Guide, Facilitators Develop and field test the Adoption Guide, Facilitators Guide and Workshop Planning GuideGuide and Workshop Planning Guide

Develop and fiedld test training & coaching approachesDevelop and fiedld test training & coaching approaches

Q&AQ&A

An ExampleAn ExampleBreastfeeding BCC programs often improve EBF rates by a Breastfeeding BCC programs often improve EBF rates by a

modest amount, e.g., 20% at baseline to 40% at endline.modest amount, e.g., 20% at baseline to 40% at endline.

Using the five needs tools, analyze why this might be?Using the five needs tools, analyze why this might be?



Module 6: Strengthening the delivery system

Behavior-centered design of interventions and delivery and support systems (PAG)

Results Frameworks , Logic Models (USAID, PIP, etc)

High Level Frameworks for Scaling Up (WHO/RH, WHO/ExpandNet, MSI, CMH, CORE, etc)

PAG in the Context of OtherFrameworks and Models for Scaling Up

GeneralGuidance

DetailedAnalysis &Planning

program assessment guide_pelletier_5.1.12

Health & Medicine