Monitoring & Evaluation Plansand

Understanding data & information flow

Guides for Demonstrating Program Progress and Success

Learning Objectives

Describe basic elements of M&E Plans Understand basic function of Plan components Identify opportunities for data demand and use Apply the Information Use Map

Session Overview

Definition and function of M&E Plan Describe components of a Plan Basic rules to guide M&E Plan development Introduce Tool: Information Use Map Introduction to workshop group work:

Developing components of an M&E plan

M&E Plans – Definition1

Document describing all M&E activities in a program Program objectives, interventions developed to

achieve them, & procedures to be implemented to determine whether or not objectives are met

Expected results of the program and how they relate to goals and objectives

Data needed, how it will be collected & analyzed Information use, including resources needed to do so How the program will be accountable to stakeholders

1MEASURE Evaluation website, https://www.cpc.unc.edu/measure/training/mentor

M&E Plans - Function Guides implementation of program M&E

Enhances coordination, standardization States how program will measure achievements

Accountability Documents stakeholder consensus

Transparency & responsibility Helps achieve program results

Ensures good use of data Preserves institutional memory

A living document, adjusted for program modification

M&E Plan Components Introduction Program Description

Goals and objectives M&E Frameworks

Conceptual, Logic, Results Indicators

Presented in a both a Matrix & Indicator Reference Sheets

Data sources, collection & reporting systems Plans for data use & dissemination

Information Use Mapping Tool as an option

M&E Plan Components

Capacity needs for Plan implementation Funding, TA, staff, equipment (computers,

GPS) Analysis of constraints & potential solutions Plans for demonstrating program impact Mechanism for Plan updates

M&E Plan Components - Introduction

Program Context National, community-based Nature of problem within the target population

Purpose of the Plan Description of development process

Stakeholders involved Consensus process

M&E Plan Components – Program Description

Problem Statement What is the nature of the HIV-related issue being

addressed? Goal and Objectives

What is the ultimate outcome of the program (goal) What are the shorter-term aims (objectives)

Program Description Intervention(s), geographic scope, target population,

duration

M&E Plan Components – M&E Frameworks

Conceptual: shows how program fits into causal pathway to desired health outcome

Logic: shows program components that will contribute to the long and short-term outcomes

Programmatic-specific frameworks Results for USAID/PEPFAR

M&E Plan Components – Indicators Selection based on

Logic models Donor requirements

Presented in 2 ways Indicator Matrix (pp. 40-45, VN National Plan)

Table presenting indicators including information on data source, frequency, who is responsible, etc.

Indicator Reference Sheets (pp. 46-107) Detailed sheet describing each indicator, how to

measure it, underlying assumptions & interpretation considerations

M&E Plan Components – Data Sources used for inputs to indicators

Existing/routine data Planned studies Special systems/studies for program

Collection tools for program Patient records, supplies, clinic records for use of

services Reporting systems

Upward and downward flow of data

M&E Plan Components – Data Use

Specify plans for data use Identify

Databases for storage Who the users are Dissemination methods including

Reports, media, speaking events

Apply Information Use Map tool to understand data flow and enhance use

Understanding Data Flow

Helps us better understand our role in health information system and importance of collecting data

Identify opportunities for improving data collection, analysis, increase availability, and ensure its use

Higher levels: district, province, national

Analysts, evaluators

Service Delivery Point

Feedback Managers,

Government, Donors

Program

Compiled data

Clinical histories, service

statistics

Reports

Information Flow

Reasons to Assess Information Flow

Local data not used locally Higher-level information does not return back

to local level Local data not assessed in broad context Reports may not reflect what is being

collected & is needed (e.g., sex differentiation) Little incentive to produce high-quality data

16

INFORMATION USE MAP

Information Use Mapping Purpose

Describe existing flow of health information to identify opportunities for improving its use

Description Identifies gaps and opportunities for using information Identifies opportunities for additional feedback

mechanisms Identifies points where analysis & data could support

programmatic decision making

Information Use Map: Swaziland National HIV/AIDS ProgramMay 2005

Data Collection Compilation Storage Analysis Reporting UseS

NA

P/M

OH

SW

N

ER

CH

AR

egio

nal

NG

Os

Gov

ernm

ent F

acili

ties

Priv

ate

Clin

ics

Client data collected in electronic

patient record systems

Client data collected in electronic

patient record systems

Client data collected

in registers

Client data stored in electronic

patient record systems

Staff compiles into

monthly summary

sheets

Regional facilities’ monthly

summary sheets compiled

Data entered into Access at MOH Health Statistics or NERCHA

Reporting to WHO or

GFATM

Information Use Map: Swaziland National HIV/AIDS ProgramMay 2005

Data Collection Compilation Storage Analysis Reporting Use

SN

AP

/MO

HS

W

NE

RC

HA

Reg

iona

lN

GO

sG

over

nmen

t Fac

ilitie

sP

rivat

e C

linic

sClient data collected in electronic

patient record systems

Client data collected in electronic

patient record systems

Client data collected

in registers

Client data stored in electronic

patient record systems

Staff compiles into

monthly summary

sheets

Regional facilities’ monthly

summary sheets compiled

Data entered into Access at MOH Health Statistics or NERCHA

Reporting to WHO or

GFATM

Opportunities for Use

Feed

back

Loo

ps

Opportunities for Use

Opportunities for Use

Opportunities for Use

Opportunities for Use

Key Messages

Actual flow of data and information can reveal barriers to improving data quality and use

Information Use Map can highlight intervention points

21

M&E Plan Components Capacity needs for plan implementation

Identify resources needed to implement plan Funding, technical capacity, equipment etc.

Analysis of constraints Be realistic; may be connected to above

Plans for demonstrating program impact Evaluation probably not covered by monitoring indicators

Mechanism for Plan updates Depending on length of program - annual, semi-annual

An M&E Plan should be

Practical Accessible to intended users

Feasible, realistic, & diplomatic Legal & ethical Accurate

Reveal technically correct information

M&E Plan Development Advocate for the need for M&E Assess program information needs Assess existing system capabilities to address these

needs Achieve consensus and commitment among

stakeholders Indicators & reporting structure

Prepare document for final approval

M&E plan should be written during the initial stages of program development

Workshop Group Project

Developing components of an M&E Plan

Workshop Group Project:Developing Components of an M&E Plan

Goal Put workshop learning to practical use with real

life case studies All case studies for projects provided by you In total, 5-6 groups

Minimum of 3, max of 4 people Sign up for top 2 choices after this session

Workshop Group Project:Assignment

Total of 5-6 groups Look to author for missing information, or create

plausible conditions As part of workshop sessions, all groups

Logic model Some indicator selection Some tool utilization

Remaining work is your choice: Focus on some M&E Plan components & use tools presented Develop real components – detailed Nature of program description may dictate some of what

needs to be done

Information Use Map Activity for Group Work:

Complete the Information Use Map for your organization as data flows now

Review the map and discuss among your group how the flow of information could be improved: How else could data be analyzed? Are there opportunities for feedback mechanisms? Is data being used by all stakeholders?

Note potential interventions based on your discussion. Make a 2nd version of the map to illustrate an improved

flow of information

28

Small Group Activity: Report Back1. One group member remain at the table to present the map

to visitors2. Each table moves to the right to visit the table next to

them.3. The table presenter spends 5 minutes explaining how they

improved information flow in their map and how this would facilitate use of data.

4. Repeat steps 2-35. Everyone returns to original tables6. Consider improving your map based on what you learned

from your neighbors.

MEASURE Evaluation is a MEASURE project funded by theU.S. Agency for International Development and implemented bythe Carolina Population Center at the University of North Carolinaat Chapel Hill in partnership with Futures Group International,ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do notnecessarily reflect the views of USAID or the U.S. Government.MEASURE Evaluation is the USAID Global Health Bureau'sprimary vehicle for supporting improvements in monitoring andevaluation in population, health and nutrition worldwide.