Monitoring and Evaluation: Child Health Programs

Session Objectives

By the end of this session, participants will be able to:• Identify interventions and approaches to improve child

health• Describe M&E challenges of specific child health program

areas• Design an M&E framework for an identified intervention• Choose core output and outcome indicators for a specified

intervention & recognize their strengths & limitations

Current Situation

• Each year nearly 11 million children die before the age of five, 30,000 every day,largely from preventable causes.

• 33% of all child deaths occur in the first month of life.

• 41% of these deaths occur in Africa, which has only 10 % of the world’s under-five population.

• 50% of these deaths occur in only six countries; 90% of these deaths occur in 42 of 192 countries.

Global Targets

• Millennium Development Goal (2002): • To ensure a two-thirds reduction in under-five mortality

by 2015 from the base year 1990• Related targets

– Target 2: Halve between 1990 and 2015, the proportion of people who suffer from hunger

– Target 6: Reduce by three-quarters, between 1990 and 2015, the maternal mortality ratio

– Target 7: Have halted by 2015 and begun to reverse the spread of HIV/AIDS

– Target 8: Have halted by 2015 and begun to reverse the incidence of malaria and other major diseases

– Target 10: Halve, by 2015, the proportion of people without sustainable access to safe drinking water and basic sanitation

Child Health Programs

ImmunizationControl of diarrheal diseases & acute

respiratory infections (ARI)Prevention of HIV in infants and young

childrenRoll Back Malaria (RBM)Integrated Management of Childhood Illness

(IMCI)Newborn health Nutrition

Newborn Intervention Packages

• Pre-pregnancy health

• Care during pregnancy

• Care during delivery

• Postpartum care of the mother

• Newborn care– Essential newborn care– Extra newborn care– Emergency newborn care

IMCI Strategy: Components

Improving familyand community

practicesImproving health

workers skills

Improving Health System

Key Family & Community Practices I

• Exclusive breastfeeding

• Appropriate complementary feeding

• Micronutrient intake• Providing stimulating

environment

• Immunization• Handwashing and

hygiene• Insecticide-treated nets• HIV prevention and care

Growth Promotion and Development

Disease Prevention

Key Family & Community Practices II

• Increased fluids & continued feeding

• Appropriate home treatment

• Prevention/management of injuries/accidents

• Prevention of child abuse & neglect

• Male involvement in care provision

• Recognition of dangers signs

• Appropriate providers• Treatment compliance• Accepting referral• Follow-up• Adequate antenatal and

delivery care

Appropriate Home Care

Care Seeking

M&E Challenges

M&E Challenges: Immunization

• Age groups differ among data sources

Routine data

• Estimation of denominators

• Accuracy, completeness & timeliness of reporting

Survey data

• Card availability & reliability of recall

• Sampling errors & sub-national estimates

• Survey timing in relation to campaigns

• Use of data for timely program action

Class Activity: Funny Numbers for Immunization Coverage

• Example 1: For 6 months in a row, one health center recorded coverage over 100 percent.

• Example 2: Last year, 51 out of 100 districts in the country reported higher figures for DTP3 than DTP1 coverage.

• Example 3: As of May, most health facilities in the country were reporting 30% to 40% measles coverage, which is approximately what one would expect. One health center, however, reported 100% measles coverage.

M&E Challenges: Diarrhea, ARI, Fever

• Seasonality of disease

• Selection bias in health facility data

• Difficulty of standardizing & comparing across different malaria transmission settings

• Changing definitions (ARI)

• Nonstandard indicators & survey questions on water and sanitation

• Validation

Class Activity: Management of Childhood Diarrhea

Time Indicator/definitions % of children < 5 years with diarrhea in

past 2 weeks

Before mid 1980s ORS Use Ghana 1988 34.8%

Mid 1980s ORT Use: treatment with ORS or sugar/salt solution

Mid to late 1980s ORT use: treatment with ORS or recommended home fluids

Ghana 1993 37.1%

1990s Increased fluid intake Ghana 1998 56.1%

Currently Increased fluids and continued feeding during illness

Ghana 2003

Ghana 2008*

10.5%

11.0%

M&E Challenges: PMTCT (I)

• Wide range of interventions– Degree of integration

• Non-standard output index

• Replacement feeding M&E is complex

• Impact indicators difficult to obtain

• Follow-up of mother-infant pairs

M&E Challenges: PMTCT (II)

• Estimating the population at risk

• Political and ideological factors

• Rapid scale-up of interventions

• Ethical issues

M&E Challenges: Newborn Interventions (I)

• Definitions of births, deaths, and newborn period

• Data quality– Omission of still birth & early neonatal death– Heaping on age at death– Sampling errors on survey estimates– Sensitivity of perinatal (late fetal plus early neonatal)

mortality rate to changes in data quality

M&E Challenges: Newborn Interventions (II)

• Measurement of neonatal morbidity

• Lack of standardized output indicators

– Facility-level and outreach monitoring

• Outcomes needed for both mother and baby

• Attribution is difficult due to packaged services

Class Activity: Neonatal Health

In Area A, there are 5000 live births, 100 neonatal deaths, and 20 of the deaths are due to neonatal asphyxia. In Area B, there also 100 neonatal deaths, and five deaths are due to neonatal asphyxia. In Area B, there are 2000 live births. What conclusions can you draw from these data?

M&E Challenges: Child Nutrition

• Complexity of child nutrition programs

• Accurate measurement of child’s age

• Need for large sample sizes to detect change in breastfeeding

• More complex survey questions– Interviewer or respondent error

M&E Challenges: Facility IMCI

• Complexity of indicator measurement

• Quality of health facility data

– Observation bias

– Courtesy bias

– Selection bias

M&E Challenges: Disease Surveillance

• Use of non-standard case definitions• Accuracy, completeness and timeliness of

reporting• Representation• Errors in descriptive information about reported

case• Lack of laboratory support for outbreak

confirmation or patient management• Infrastructure and communication constraints

Class Activity: Disease Surveillance in Mirriah District

• What might account for the increase in the number of new cases observed during the two most recent years?

• How might you explain the discrepancy between the hospital cases and reported cases?

• What is the effect of including the children without fever status recorded on the chart in the case definition?

Adapted from CDC Case Study No. 891-903

M&E Challenges: Mortality

• Reliability of various data sources

• Measuring short-term changes

• Data quality

• Inconsistent definitions of newborn, infant and child death

• Misclassification of cause of death

• Demonstrating mortality impact

Class Activity: Mortality

During the year 1997, data from birth registers showed that 8776 live births were recorded in the municipality of Mamfi, out of which almost 7744 were children of mothers residing in the area. One hundred and thirty-nine (139) fetal deaths were recorded also from mothers living in the area. Two hundred and seven (207) deaths of babies under one year of age were registered to residents in the area of Mamfi, with the age distribution shown on your handout.

(1) Calculate the infant mortality rate (IMR)

(2) Calculate the early neonatal mortality rate (ENMR)

An investigation conducted in hospitals and maternity wards revealed that approximately 65% of births (alive and dead) took place in hospitals. Every one of such events was scrutinized, consulting the mothers’ medical charts and notes taken by the physician and nurses inside the labor room. Findings showed that thirty live births have been mistakenly declared as fetal deaths and were recorded as such. Twenty-two fetal deaths were erroneously declared as live births.

(3) Calculate the accurate IMR & ENMR

(4) How were the rates affected by errors in the definition of live births and fetal deaths?

(5) To what extent are the corrected rates representative of the municipality of Mamfi?

ILLUSTRATIVE EXAMPLE

IMCI

PROCESS

Train first-level health workers & district supervisors in IMCI

FUNCTIONAL OUTPUTS

•No. of health workers trained in IMCI

•No. of district supervisors trained in IMCI

SERVICE OUTPUTS

•Improved HW skills

•Improved HW competence

•Reduced missed opportunities for vacc. & treatment of childhood illness

•Increased client satisfaction

OUTCOMES

•Improved caretaker knowledge & practices

•Early case management

•Appropriate care seeking

•Compliance with treatment

IMPACT

•Reduced infant & child mortality

•Improved infant & child health & nutrition

Logic Model for Improved Health Worker Skills

WHO Priority Indicators of Health Worker Skills (I)

• Assessment– Child checked for three danger signs– Child checked for the presence of cough, diarrhea, and

fever– Child’s weight checked against a growth chart– Child’s vaccination status checked– Index of integrated assessment– Child under two years of age assessed for feeding

practices

WHO Priority Indicators of Health Worker Skills (II)

• Correct treatment and counseling– Child needing oral antibiotic and/or antimalarial is prescribed the

drug(s) correctly– Child not needing antibiotic leaves the facility without antibiotic– Caretaker of sick child is advised to give extra fluids and

continued feeding– Child needing vaccinations leaves facility with all needed

vaccinations– Caretaker of child who is prescribed ORS and/or oral antibiotic

and/or oral antimalarial know how to give the treatment

• Correct management of severely ill children– Child needing referral is referred

Sources of Data: M&E of IMCI Training (I)

• Health facility surveys (e.g. Service Provision Assessment– Inventory of resources and support services– Provider interview– Observation of services provided– Client exit interview

Sources of Data: M&E of IMCI Training (II)

• Supervisory checklists

• Review of national, district, and facility records

• Demographic surveillance

• Mortality survey

• Vital registration (if 90+% complete)

• Qualitative studies

Case Study

Despite substantial efforts to improve infant and child health and nutrition in Lakkha District in the past decade, about one out of five babies born in a given year die before they reach their fifth birthday, many during the first year of life. In 2002, the Government recognized that improving the quality of care for sick children at the health facility alone would have a limited impact on reducing child mortality. Lakkha District was chosen, therefore, as a pilot site for the development and implementation of a household and community-based approach to promote key household practices for child survival, growth and development . Please refer to the handout for further instructions.

References

• Arimond, Mary and Marie T. Ruel. 2003. Generating indicators of Appropriate Feeding of Children 6 through 23 Months from the KPC 2000+. Washington, D.C.: FANTA.

• Bos, E. and A. Batson. 2000. Using Immunization Coverage Rates for Monitoring Health Sector Performance: Measurement and Interpretation Issues. HNP Discussion Paper. Washington, D.C.: World Bank.

• Gage, Anastasia J., Disha Ali, and Chiho Suzuki. (Forthcoming). A Guide for Monitoring and Evaluating Child Health Programs. MEASURE Evaluation. Chapel Hill, NC: Carolina Population Center, University of North Carolina.

• United Nations. 1983. Manual X: Indirect Techniques for Demographic Estimation. New York: United Nations.

• WHO. 2001. Indicators for IMCI at First-level Facilities and Households. Geneva: WHO.

• WHO. 2005. National Guide for Monitoring and Evaluating Programmes for the Prevention of HIV in Infants and Young Children. Geneva: WHO.

• WHO and UNICEF. 2005. World Malaria Report 2005. Geneva: WHO.