document of the world bankdocuments.worldbank.org/curated/en/900461467993169295/...i document of the...

i

Document of The World Bank

Report No: ICR00003551

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-42500, IDA-45970)

ON A

CREDIT

IN THE AMOUNT OF SDR 188.9 MILLION (US$280 MILLION EQUIVALENT)

TO THE

FEDERAL REPUBLIC OF NIGERIA

FOR A

MALARIA CONTROL BOOSTER PROJECT

March 31, 2016

Health, Nutrition and Population Global Practice Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS

(Exchange Rate Effective March 31, 2016)

Currency Unit = Nigerian Naira (NGN) NGN 1.00 = US$ 0.01 US$ 1.00 = NGN 199.25 XDR 1.00 = US$ 1.41

FISCAL YEAR

July 01 – June 30

ABBREVIATIONS AND ACRONYMS

ACT Artemisinin-based Combination Therapy AF Additional Financing AIDS Acquired Immune Deficiency Syndrome AMFm Affordable Medicines Facility-malaria ANC Antenatal Care BCC Behavior Change Communication BMGF Bill and Melinda Gates Foundation CD Country Director CDD Community Directed Distributors CDI Community Directed Intervention CMU Country Management Unit COMPASS Community Participation and Action for the Social Sector CPS Country Partnership Strategy CSS Community Systems Strengthening DALY Disability-Adjusted Life Year DDT Dichlorodiphenyltrichloroethane DFID Department for International Development DHS Demographic Health Survey FA Financing Agreement FBO Faith-based Organization FCT Federal Capital Territory FGON Federal Government of Nigeria FMOF Federal Ministry of Finance FMOH Federal Ministry of Health GDP Gross Domestic Product GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GNI Gross National Income HFA Health Facility Assessment HIV Human Immunodeficiency Virus HMIS Health Management Information System HSS Health Systems Strengthening

iii

IDA International Development Association IE Impact Evaluation IEC Information, Education, and Communication INMCH Integrated Neonatal, Maternal, and Child Health INT Integrity Vice Presidency IP Implementation Progress IPT Intermittent Preventive Therapy IRI Intermediate Results Indicator IRR Internal Rate of Return IRS Indoor Residual Spraying ISR Implementation Status Report ITN Insecticide Treated Mosquito Net IVM Integrated Vector Management LGA Local Government Areas LLIN Long-Lasting Insecticidal Nets LMIS Logistic Management Information System LQAS Lot quality assurance sampling M&E Monitoring and Evaluation MCBP Malaria Control Booster Project MDG Millennium Development Goal MDG-F Millennium Development Goals Achievement Fund MICS Multiple Indicator Cluster Survey MIS Malaria Indicator Survey MNCH Maternal, Newborn and Child Health MPP Malaria Plus Package MTR Midterm Review NEEDS National Economic Empowerment and Development Strategy NGO Non-Governmental Organization NIFAA Nigerian Inter-Faith Action Association NMCP National Malaria Control Program NMSP National Malaria Strategic Plan NPV Net Present Value PAD Project Appraisal Document PDO Project Development Objective PFMU Project Financial Management Unit PIF Project Implementation Facilitator PIU Project Implementation Unit PMV Patent Medicine Vendor PP Parent Project PRS Poverty Reduction Strategy PSC Project Steering Committee PSM Procurement and Supply Management QER Quality Enhancement Review RBM Roll Back Malaria RDT Rapid Diagnostic Test RF Results Framework

iv

RVP Regional Vice Presidency SCA Subsidiary Credit Agreement SCM Supply Chain Management SFH Society for Family Health SMOH State Ministry of Health SOML Saving One Million Lives SP Sulphamethoxazole-Pyrimethamine SUFI Scale Up for Impact SuNMaP Support to National Malaria Programme TB Tuberculosis UNDP United Nations Development Programme UNICEF United Nations Children's Emergency Fund USAID United States Agency for International Development US-PMI United States President’s Malaria Initiative WHO World Health Organization

Vice President: Makhtar Diop

Country Director: Rachid Benmessaoud

Practice Manager: Trina Haque

Project Team Leader: Noel Chisaka

ICR Team Leader: Noel Chisaka

ICR Team Author: F. Brian Pascual

v

NIGERIA Malaria Control Booster Project

TABLE OF CONTENTS

Data Sheet A. Basic Information ..................................................................................................... viiB. Key Dates ................................................................................................................. viiC. Ratings Summary ..................................................................................................... viiD. Sector and Theme Codes ........................................................................................ viiiE. Bank Staff .................................................................................................................. ixF. Results Framework Analysis ..................................................................................... ixG. Ratings of Project Performance in ISRs ................................................................. xivH. Restructuring ........................................................................................................... xivI. Disbursement Profile ............................................................................................... xvi 1. Project Context, Development Objectives and Design ............................................... 1

1.1 Context at Appraisal ............................................................................................. 11.2 Original Project Development Objectives (PDOs) and Key Indicators ................ 21.3 Revised PDO (as approved by original approving authority) and Key Indicators,

and reasons/justification ........................................................................................ 21.4 Main Beneficiaries ................................................................................................ 31.5 Original Components (as approved) ..................................................................... 31.6 Revised Components ............................................................................................ 41.7 Other significant changes ...................................................................................... 4

2. Key Factors Affecting Implementation and Outcomes .............................................. 62.1 Project Preparation, Design and Quality at Entry ................................................. 72.2 Implementation ..................................................................................................... 92.3 M&E Design, Implementation and Utilization ................................................... 112.4 Safeguard and Fiduciary Compliance ................................................................. 132.5 Post-completion Operation/Next Phase .............................................................. 15

3. Assessment of Outcomes .......................................................................................... 153.1 Relevance of Objectives, Design and Implementation ....................................... 153.2 Achievement of PDOs ........................................................................................ 173.3 Efficiency ............................................................................................................ 213.4 Justification of Overall Outcome Rating ............................................................ 223.5 Overarching Themes, Other Outcomes and Impacts .......................................... 223.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ... 23

4. Assessment of Risk to Development Outcome ......................................................... 235. Assessment of Bank and Borrower Performance ..................................................... 23

5.1 Bank Performance ............................................................................................... 235.2 Borrower Performance ........................................................................................ 25

6. Lessons Learned ....................................................................................................... 267. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 27Annex 1. Project Costs and Financing .......................................................................... 28

vi

(a) Project Cost by Component (in USD Million equivalent) .................................. 28(b) Financing ............................................................................................................. 28

Annex 2. Outputs by Component ................................................................................. 29Annex 3. Economic and Financial Analysis ................................................................. 33Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 43

(a) Task Team members ............................................................................................ 43(b) Staff Time and Cost ............................................................................................. 44

Annex 5. Beneficiary Survey Results ........................................................................... 45Annex 6. Stakeholder Workshop Report and Results ................................................... 46Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 47Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 57Annex 9. List of Supporting Documents ...................................................................... 58Annex 10. Results Framework and Monitoring ............................................................ 60Annex 11. Impact Evaluation Summary ....................................................................... 66MAP .............................................................................................................................. 75

vii

A. Basic Information

Country: Nigeria Project Name: Malaria Control Booster Project

Project ID: P097921 L/C/TF Number(s): IDA-42500, IDA-45970

ICR Date: 03/31/2016 ICR Type: Core ICR

Lending Instrument: SIL Borrower: FEDERAL GOVERNMENT OF NIGERIA (FGON)

Original Total Commitment:

XDR 121.70M Disbursed Amount: XDR 178.29M

Revised Amount: XDR 188.17M

Environmental Category: B

Implementing Agencies: Federal Ministry of Health, National Malaria Elimination Program, and the State Ministries of Health and State Malaria Control Programs from Kano, Jigawa, Gombe, Bauchi, Akwa Ibom, Rivers, and Anambra.

Cofinanciers and Other External Partners: External partners included Roll Back Malaria (RBM), The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), U.S. President’s Malaria Initiative (USG/PMI), and U.K. Department for International Development (DFID) B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 01/26/2006 Effectiveness: 05/15/2007 05/15/2007

Appraisal: 07/10/2006 Restructuring(s):

06/16/2009 07/26/2011 08/04/2011 06/28/2013 06/23/2014

Approval: 12/12/2006 Mid-term Review: 06/15/2010 10/28/2010

Closing: 03/31/2012 03/15/2015 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Satisfactory

Borrower Performance: Moderately Satisfactory

viii

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

NA

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Moderately Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 38 38

Health 23 23

Other social services 2 2

Sub-national government administration 33 33

Tertiary education 4 4

Theme Code (as % of total Bank financing)

Child health 14 14

Health system performance 29 29

Malaria 29 29

Other Private Sector Development 14 14

Population and reproductive health 14 14

ix

E. Bank Staff

Positions At ICR At Approval

Vice President: Makhtar Diop Gobind T. Nankani

Country Director: Rachid Benmessaoud Hafez M. H. Ghanem

Practice Manager/Manager:

Trina S. Haque Laura Frigenti

Project Team Leader: Noel Chisaka Ramesh Govindaraj

ICR Team Leader: Noel Chisaka

ICR Primary Author: F. Brian Pascual F. Results Framework Analysis

Project Development Objectives (PDOs)The PDOs are (i) to ensure that the target population will have improved access to, and utilization of, a well-defined set of Malaria Plus Package interventions (MPP); and (ii) to strengthen Federal and States ability to manage and oversee delivery of malaria plus interventions. Revised PDOs (as approved by original approving authority) NA (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values

(from approval

documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Percentage of children < 5 years with fever treated with an effective antimalarial within 24 hours from onset of symptoms

3.7% 80.0% 15.0% 45.0%

Date achieved 08-Dec-2006 15-May-2007 04-Aug-2011 03-Mar-2015

Comments (incl. % achievement)

EXCEEDED: The target was revised from 80% to 15% to reflect a more realistic end target based on experiences from other countries with ongoing artemisinin-based combination therapies (ACTs) use, global availability of ACTs, focused treatment intervention, and the need to reduce artemisinin-resistant malaria parasites. The achievement represents a significantly higher rate than the 30% of children < 5 years in non-Project States treated with effective antimalarial within 24 hours of onset of symptoms.

Indicator 2: 3.6% 60.0% 74.4%

x

Percentage of children < 5 years who slept under an insecticide-treated nets (ITN) the night preceding the survey Date achieved 08-Dec-2006 15-May-2007 03-Mar-2015


EXCEEDED: The actual value at the end of Project surpassed the target by 24%. The achievement is significantly higher than the 37% rate of children < 5 years in non-Project sites who slept under an ITN the night preceding the survey.

Indicator 3: Percentage of pregnant women who received two or more doses of intermittent preventive treatment (IPT)

8.3% 25.0% 52.6%

Date achieved 08-Dec-2006 15-May-2007 03-Mar-2015


EXCEEDED: The Project surpassed the target by over 110% and reflected a significantly higher proportion compared to non-Project sites where only 41% of pregnant women received 2+ doses of IPT.

Indicator 4: Percentage of pregnant women who slept under bed net night before the survey

4.1% 50.0% 73.4%

Date achieved 12-Aug-2006 15-May-2007 03-Mar-2015


EXCEEDED: The actual value at the end of Project surpassed the target by 47%. The achievement is significantly higher than the 48% rate of pregnant women in non-Project sites who slept under an ITN the night preceding the survey.

Indicator 5: Number of direct Project Beneficiaries of which female

NA 16,000,000 18,000,000

Date achieved 08-Dec-2006 4-Aug-2011 03-Mar-2015 Comments (incl. % achievement)

EXCEEDED: The target was attained well before the end-of-Project household survey was conducted.

Indicator 6: Number of children under five who received Vitamin A supplementation

NA 6,000,000 26,465,138

Date achieved 08-Dec-2006 4-Aug-2011 03-Mar-2015


EXCEEDED: The target was attained well before the end-of-Project household survey was conducted. The data came from Maternal, Neonatal, and Child Health (MNCH) week post campaign surveys

Indicator 7: Percentage of States regularly using monitoring and evaluation data to manage malaria and Maternal and Child Health (MCH) programs

0.0 80.0% 100.0%

xi

Date achieved 30-Sep-2006 15-May-2007 03-Mar-2015


Dropped Indicator: This indicator was removed because the Project had consistently met its target since 2009. Thus, it added limited value and was not relevant to track. The indicator was included in this table to maintain a measure of how the Project performed in achieving PDO II.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values

(from approval

documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Percentage of households with at least one ITN/ long-lasting insecticidal nets (LLIN)

2.6% 80% 99.6%



EXCEEDED: The Project surpassed the target by 25% and helped increase percentage of households with at least one ITN/LLIN by 38 times the baseline value.

Indicator 2: Percentage of children aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or recommended home fluids

25.3% 50.0% 59.4%



EXCEEDED: The Project surpassed the target by 19% and helped increase the percentage of children aged 0-23 months with diarrhea received ORS and/or recommended fluids by 2.5 times baseline value.

Indicator 3: Percentage of women with children <1 year of age who received antenatal care (ANC) during last pregnancy

63.0% 72.0% 87.5%

Date achieved 08-Dec-2006 15-May-2007 03-Mar-2015 Comments (incl. % achievement)

EXCEEDED: The Project helped increase the percentage of women who received ANC by 39%.

Indicator 4: Number of Long-lasting insecticide-treated malaria nets purchased and/or distributed

0 12,400,000 51,392,304

Date achieved 08-Dec-2006 15-May-2007 07-Jan-2015 Comments (incl. % achievement)

EXCEEDED: The Project surpassed the target by three times.

xii

Indicator 5: Percentage of health facilities using laboratory confirmation as the basis for treatment of fever in children >5yrs and adults

5.0% 30.0% 31.4%

Date achieved 01-Nov-2009 15-May-2007 07-Jan-2015


EXCEEDED: The Project surpassed the target and increased the percentage of health facilities using laboratory confirmation by 6.3 times the baseline value.

Indicator 6: Percentage households sprayed with indoor residual spraying (IRS) in the last 12 months

3.0% 80.0% 83.0%

Date achieved 12-Aug-2006 31-Mar-2012 07-Jan-2015


EXCEEDED: This indicator reflects coverage of IRS in the entire state. However, the Project States only supported IRS only in two Local Government Area (LGAs) across the six supported States. Hence the coverage shown reflects post IRS reports /surveys, which is 83.0.

Indicator 7: Number of LLINs distributed in 22 additional states

5,200,000 20,700,000 32,592,304

Date achieved 26-Jul-2011 26-Jul-2011 03-Mar-2015


EXCEEDED: The Project surpassed the target by 58%. This reflected the additional support that the Project provided on an urgent basis in distributing 30 million LLINs to 22 non-Project States.

Indicator 8: Percentage of children <5years of age with fever in the last 2 weeks treated with an effective anti-malarial within 24 hours of onset of symptoms in communities with community-directed intervention (CDI)

0.0% 15.0% 40.0%

Date achieved 01-Nov-2006 16-June-2009 03-Mar-2015


EXCEEDED: The Project surpassed the target by 167%. Results were from the Impact Evaluation end stage that focused primarily on patent medicine vendors (PMVs) and CDI States.

Indicator 9: Percentage of care givers who knows at least two symptoms of malaria

62.3% 70.0% 85.0%

Date achieved 30-Sep-2010 16-June-2009 03-Mar-2015


EXCEEDED: The Project surpassed the target by 21%. Results from Impact Evaluation end stage that focused primarily on PMV ad CDI States

xiii

Indicator 10: Percentage of mothers of children less than five years of age in CDI communities who know at least two ways of preventing malaria

0.0% 60.0% 55.5%

Date achieved 30-Jun-2009 16-June-2009 03-Mar-2015


PARTIALLY ACHIEVED: The Project made significant progress in employing CDIs to increase knowledge and awareness, but was just short in achieving the target.

Indicator 11: Number of rapid diagnostic tests (RDTs) distributed to health facilities

0 12,000,000 14,400,000

Date achieved 30-Jun-2009 15-May-2007 03-Mar-2015 Comments (incl. % achievement)

EXCEEDED: The Project surpassed the target by 20%.

Indicator 12: Percentage of PMV knowing the correct dose of ACT for children < 5

0.0% 80.0% 60.0%

Date achieved 31-Jul-2009 16-June-2009 03-Mar-2015 Comments (incl. % achievement)

PARTIALLY ACHIEVED: Results from Impact Evaluation end stage that focused primarily on PMV ad CDI States

Indicator 13: Percentage of sampled PMV that had ACTs in stock at the time of assessment

0.0% 80.0% 73.3%



PARTIALLY ACHEVIED: PMVs provide an important avenue to access ACTs. The FGON will need to work with PMVs to improve and maintain adequate levels of ACTs. Results from Impact Evaluation end stage that focused primarilyon PMV ad CDI States

Indicator 14: Percentage of sampled community-directed distributors (CDDs) with no stock -outs in the last three months

0.0% 80.0% 39.1%



NOT ACHIEVED: With CDDs representing an important avenue for ACTs, the FGON will need to strengthen Community Health Systems to ensure that stock outs are not experienced in more than 20% of CDDs. Results from Impact Evaluation end stage that focused primarily on PMV ad CDI States

xiv

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 06/29/2007 Satisfactory Satisfactory 0.44 2 12/20/2007 Satisfactory Satisfactory 19.71 3 06/20/2008 Satisfactory Satisfactory 28.96 4 01/10/2009 Satisfactory Satisfactory 44.57 5 06/30/2009 Satisfactory Satisfactory 74.48 6 12/16/2009 Satisfactory Moderately Satisfactory 85.75 7 06/24/2010 Satisfactory Satisfactory 118.62 8 12/22/2010 Satisfactory Moderately Satisfactory 133.94 9 08/08/2011 Satisfactory Moderately Satisfactory 151.38

10 01/31/2012 Satisfactory Moderately

Unsatisfactory 156.27

11 08/07/2012 Satisfactory Moderately Satisfactory 164.16 12 02/10/2013 Satisfactory Satisfactory 183.07 13 07/30/2013 Satisfactory Moderately Satisfactory 194.34 14 03/12/2014 Satisfactory Moderately Satisfactory 211.45

15 09/29/2014 Moderately

Unsatisfactory Moderately

Unsatisfactory 260.83

16 04/07/2015 Moderately Satisfactory Moderately Satisfactory 276.13 H. Restructuring

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

06/16/2009 Y S S 68.8

Additional Financing (AF) of US$ 100 million to support expanded, supplemental components on Health System Strengthening, Community System Strengthening, communication, and procurement. The AF also supported scale up of LLIN distribution to universal access levels. The Project retained 4 of the six original PDO indicators, dropping indicators on infants who were fed breast milk only in the last 24 hours, and percentage of States using M&E data to manage malaria/Maternal and Child Health Programs.

xv


Board Approved

PDO Change


Amount Disbursed at



DO IP

07/26/2011 N S MS 145.80

Reallocate proceeds of AF to finance the cost of distributing 30 million undelivered LLINs, which were purchased by the FGON and RBM partners, to 22 non-Project States. The restructuring also added an additional intermediate indicator to measure the number of nets distributed to these non-Project States. The Bank agreed to proceed with the restructuring before the more major restructuring planned one month later given the pressing need to distribute LLINs before the rainy season and that the FGON had a well-developed plan for distributing LLINs.

08/04/2011 N S MS 149.40

Based on results of the Midterm Review (MTR) in November 2010, the FGON requested to undertake the following changes: refine and merge the PDO and intermediate indicators under the Parent Project and AF; accelerate implementation through enhanced activities; provide additional support to strengthen program management and risk mitigation measures; and extend the Project closing date by 15 months to June 30, 2013. Specific changes to the PDO indicators included: an addition of one core mandatory indicator to measure female beneficiaries; replacement of indicator on Breast Feeding with Vitamin A coverage to better align with Project activities; and revision of target from 80% to 15% for percentage of children under 5 treated with anti-malarial.

06/28/2013 N S S 185.9

Extension of the closing date by 12 months to June 30, 2014 to provide sufficient time to complete implementation activities that were delayed due to the INT investigation. The restructuring also reallocated

xvi


Board Approved

PDO Change


Amount Disbursed at



DO IP

undisbursed funds to the Federal level from State Subsidiary Agreements of both underperforming States and Kano when it decided to discontinue from the Project.

06/23/2014 N S MS 236.56

Extension of closing date by 9 months to March 31, 2015 to complete the following: LLIN distribution in Project States; Behavior Change Communication (BCC) campaigns; end-of-Project Lot Quality Assurance Sampling (LQAS) household survey; and respond to State-level demand for implementation supervision.

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1. The Federal Republic of Nigeria, Africa’s most populous country with a population of 140 million at the time of project appraisal, represented one fifth of the Sub-Saharan African population. Its land area of 351,649 square miles was divided administratively into 36 States – which were further subdivided into 774 local government areas (LGAs) – and one Federal Capital Territory (FCT) located at Abuja. The population was equally divided between Muslims and Christians, with Sunni Muslims mainly concentrated in the northern part of the country while Christians dominated the central and southern parts. 2. During the first half of the 2000s, Nigeria benefitted from oil revenues and exhibited an average Gross Domestic Product (GDP) growth rate of 9.7%. With the overall decline in the economy in 2005 outside of the energy sector, however, inflation rate rose to 15.6% while the Gross National Income (GNI) per capita hovered at US$660, making Nigeria one of the poorest countries in Africa. At the time, Nigeria ranked at the lowest scale of the UNDP Human Development Index, where it placed 158 out of 177 countries. Moreover, the country suffered significantly from the burden of childhood communicable diseases, including malaria. An analysis of data from 1999 by the Federal Ministry of Health (FMOH) estimated that at least 50% of the population (70 million) had at least one episode of malaria annually while children aged under 5 years (about 24 million) had 2 to 4 attacks of malaria annually. Consequently, nearly 30 percent of Nigerian deaths among children and 11 percent among pregnant mothers were attributed to malaria. The northern States experienced some of the highest child mortality rates in Africa. 3. Although the Federal Government of Nigeria (FGON) had recognized malaria as a major public health problem and used the primary health care system to address it, the growing concern about the increasing burden of disease and death due to malaria spawned a renewed focus with the adoption of its first National Malaria Control Policy in 1997. The launch the following year of Roll Back Malaria (RBM), a multi-partner global collaboration with a special focus on high transmission areas of Africa that aimed to reduce the burden of malaria by 2010, generated the support and momentum that helped Nigeria set up its National Malaria Control Program (NMCP) within the FMOH, to coordinate the control and prevention efforts. Due to weak malaria control efforts however, access to effective interventions remained low, particularly among the poor. In the early 2000s for example, only 34 percent of febrile children received anti-malaria treatment (chloroquine) and only 17 percent of pregnant women received Intermittent Preventive Therapy (IPT). And despite endemic malaria, the ownership and use of long-lasting insecticidal nets (LLINs), one of the main preventive interventions against malaria, remained rare. Data from the NMCP showed that only 6.8 percent of households in 2005 owned at least one Insecticide Treated Net (ITN); while the 2003 Demographic and Health Survey (DHS) in Nigeria found that only one percent of children under five and only 1.3 percent of pregnant women slept under an ITN. 4. At the time, the malaria response from the FGON centered around the 2001-2005 National Malaria Strategic Plan (NMSP) that anchored its approach based on the RBM strategy. The NSMP involved a multi-pronged intervention that included: i) prompt and effective case management, including the use of artemisinin-based combination therapies (ACTs) for treatment of uncomplicated malaria cases, with an initial focus on children under five; ii) IPT of malaria in pregnancy; and iii) integrated vector management (IVM) including the use of ITNs, indoor residual spraying (IRS), and environmental management. The Plan incorporated other cross cutting interventions such as advocacy; communication and social mobilization; effective program management; monitoring and evaluation (M&E); and partnership and collaboration. 5. Despite this comprehensive set of interventions, coverage of key preventive and curative interventions remained inadequate and well below the targeted 60% level at which major impact of the interventions on malaria

2

burden would be expected. As such, the likelihood of reducing the prevalences of malaria, child mortality, and maternal mortality to meet the health-related Millennium Development Goals (MDGs) appeared unlikely. As a result, the FGON began engaging various stakeholders (RBM, State Ministries in Nigeria, LGAs, and others) in developing an accelerated and intensified plan for its 2006-2010 Country Strategic Plan: A Road Map for Impact on Malaria in Nigeria. This revised strategy aimed to: i) address national health and development priorities, including the RBM goals and the MDGs; ii) provide an M&E framework to ensure Nigeria deploys an appropriately evaluated and documented, evidence-based and cost-effective package of interventions; iii) scale-up key preventive and curative interventions; and iv) facilitate efficient collaboration among all the partners in the public sector, the private and commercial industry, and civil society. At a total cost of US$845 million, the Strategic Plan supported all 36 States, plus the Federal Capital Territory, to achieve results at national, regional, and community levels. 6. The US$180 million IDA-financed World Bank Malaria Control Booster Project (MCBP) was intended to be implemented for five years (May 2007 to March 2012) and serve seven States1, which represented more than 25 percent of the country’s population. The proposed MCBP complemented the goals of the 2006-2010 Country Strategic Plan. In addition, the Project aligned well with Nigeria’s 2004 National Health Policy that identified malaria control as a priority health program, and the Presidential Initiative for Accelerated Achievement of the MDGs that included malaria control as a key component to achieving the health-related MDG. The MCBP also supported the FGON’s 2005 National Economic Empowerment and Development Strategy (NEEDS) – Nigeria’s Poverty Reduction Strategy (PRS) – that emphasized the development of strategic plans for malaria as a specific action under the health component of the PRS. From an economic standpoint, the FGON recognized that the cost of malaria treatment and lost economic productivity due to malaria constituted a financial burden on the economy and an obstacle to achieving its PRS. Lastly, the MCBP followed the lead of the 2005-2009 Country Partnership Strategy (CPS) that expected to achieve the following results: i) improved service delivery for human development; ii) improved environment and services for non-oil growth; and iii) improved transparency and accountability for better governance.

1.2 Original Project Development Objectives (PDOs) and Key Indicators 7. The PDOs were (i) to ensure that the target population will have improved access to, and utilization of, a well-defined set of Malaria Plus Package interventions (MPP); and (ii) to strengthen Federal and States ability to manage and oversee delivery of malaria plus interventions. 8. The PDO indicators consisted of the following: (i) percentage of children under the age of 5 years with fever treated with an effective antimalarial within 24 hours from onset of symptoms; (ii) percentage of infants aged 0 to 6 months who were fed only breast milk in the 24 hours preceding the survey; (iii) percentage of children under the age of 5 years who slept under an ITN the night preceding the survey; (iv) percentage of pregnant women who received two or more doses of IPT; (v) percentage of pregnant women who slept under an ITN the night preceding the survey; and (vi) percentage of Participating States regularly using M&E data to manage malaria and/or maternity and child programs. The Project Appraisal Document (PAD) listed 19 intermediate results indicators (IRIs) intended to monitor progress.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. The PDOs remained the same throughout the Project. Of the six original PDO indicators, one core indicator based on the Bank’s new mandate was added – the number of female beneficiaries; one was dropped – the use of M&E data for program management since the target had been consistently met and maintained over time; and one was replaced – the indicator on the coverage of Vitamin A supplementation replaced the indicator

1 Kano, Jigawa, Gombe, and Bauchi in the North; Akwa Ibom, Rivers, and Anambra in the South.

3

on Breastfeeding since the original indicator did not align well with activities that supported the Project. The new indicator on Vitamin A supplementation does not provide a timeframe, however, making it unclear if the target was to be achieved on an annual basis or overall through the life of the Project. Based on the description on the results achieved as provided in the ISRs, the team understood the target to be achieved through the life of the Project. In addition, the end target of 80% for the percentage of children < 5 years of age treated with an anti-malarial was revised with a more realistic end target of 15% as part of a restructuring in August 2011. Based on available evidence at the time, the need to change the target became necessary for the following reasons: i) comparable attainments in other countries at the time using ACTs was around 7-12% among children under five years after the drug policy transitioned to ACT use from Sulphamethoxazole-Pyrimethamines (SPs); ii) the global cost of ACTs remained very high and a barrier since the Affordable Medicines Facility-malaria (AMFm), a facility for subsidized malaria ACTs, had not yet been established, making the target too ambitious; and iii) the availability of ACT continued to be a challenge globally. In addition, the shift in focusing treatment only on confirmed malaria not only greatly improved the rational for ACT use, but also reduced the actual ACTs needed as a result of cutting unwarranted treatment of children solely based on the presence of fever. In the long-term, the focused treatment and limited use of ACT were anticipated to contribute to reducing artemisinin-resistant malaria parasites. At the time, given the 7-12% utilization rate, making the revised target of 15% actually higher than expected.

1.4 Main Beneficiaries 10. The main beneficiaries from the original Parent Project (PP) were pregnant women and children < five years of age within the seven Project States, with significant spillover effects in improvements in maternal and child health due to the comprehensive nature of the MPP. Based on a revised malaria strategy in 2008 that switched the focus to universal coverage, the main beneficiaries considered at risk shifted from children < 5 and pregnant women to the entire population.

1.5 Original Components (as approved) 11. Component 1: Strengthening leadership and coordination capacity of the federal government to control malaria (Estimated cost at appraisal: US$86.5 million; International Development Association (IDA) financing: US$86.5 million). To support the ability of FMOH staff to undertake these essential functions, the Project will support: i) strengthening government to acquire and distribute commodities2 to combat malaria, including 2 LLINs per household in 90% of population in Project States, ACT for children < 5 years, minimum of 2 IPTs for pregnant women during the 2nd and 3rd trimesters, and materials for IRS; ii) improving M&E for evidence-based management; and iii) strengthening coordination and oversight capacity of NMCP. 12. Component 2: Strengthening the capacity of SMOHs and LGAs to deliver MPP interventions at state and community levels in Project States (Estimated cost at appraisal: US$71.5 million; IDA financing: US$71.5 million). The Project will expand the capacity of the SMOHs and the LGAs in the Project States, so as to rapidly expand MPP services at both the facility and the community levels. The Project will provide appropriately tailored support to the States to improve their planning, implementation, coordination, and supervision capacities. The MPP services include the following: i) increasing access to, and utilization of, malaria preventive measures including MPPs, targeted use of IRS, IVM and distribution of LLINs; ii) increasing access to, and utilization of, effective diagnostic treatment services; iii) strengthening community mobilization and communication for behavior change; iv) strengthening partnerships for resource mobilization and implementation; v) improving M&E for state evidence-based management; and vi) enhancing capacity at State level for Project coordination and oversight.

2 Cost of commodities included among others US$14 million for LLINs, US$6.1 million for ACT, US$250,000 for SP for IPT, and US$1.2 million IRS.

4

13. In addition to the allocation set aside for Components 1 and 2, an additional unallocated amount of US$22 million was set aside for contingencies. In total, the Program Cost was US$180 million.

1.6 Revised Components 14. While no components were revised, Additional Financing (AF) in 2009 added four enhanced, supplemental components to be undertaken in concert with the existing components from the original Project (see June 16, 2009 Restructuring in next section)

1.7 Other significant changes 15. Amendment to the Financing Agreement (Approved by the Country Director (CD) on 11/19/2007). In response to the FGON’s request on 10/09/2007, the Financing Agreement (FA) was amended following a dispute raised by the States regarding the procurement of Project Implementation Facilitators (PIFs) as a condition of funds disbursement to the States. The PIFs were intended to build capacity in specific areas of need and expedite project implementation. The States contended that a) the capacity assessment carried out by a joint FMOH and World Bank team in January 2007 as the basis for establishing the need for hiring PIFs was flawed, b) significant capacities had been built in several States since the assessment through the assistance provided by other development partners, and c) many of the identified capacity gaps could be remedied through an infusion of Project funds. The States therefore resisted the hiring of PIFs, which blocked disbursement of funds needed to cover for the operational expenses, significantly hampering implementation. As a remedy, States were given 6 months to demonstrate their capacity to implement the Project without PIF assistance. In addition, a joint team carried out a second capacity assessment in early-2008. At the end, the second assessment found that all States required PIF assistance in the areas of logistics & supply management and in M&E. Changes to Part B of Section IV under Section 2 of the FA included: Exempting the “Operating Costs” category from the overall disbursement condition, thus allowing the States

to access Project funds to finance initial 6-month operating expenditures; Indicating that only States deemed to be in need of PIFs will be subject to the disbursement conditions

specified in the FA, while others can start drawing without restriction on the amounts allocated to each participating State.

16. Level II restructuring to provide AF, add supplemental components to support expanded activities, and add IRIs (Approved by the CD on June 16, 2009). In response to the FGON’s request, the CD approved US$100 million to scale up the MCBP for three years through the original closing date of the PP on March 31, 2012. The need for the AF resulted from program policy changes in 2008 that entailed scaling up LLIN distribution to universal access levels by 2010, considered a more optimal approach to malaria control. This meant distributing 4.1 million additional LLINs to Project States beyond the 8.3 million LLINs originally planned. The AF was to also support the costs associated with expanded, supplemental activities focused on diagnosis, logistics, and community mobilization, and complimented by an impact evaluation study to assess the effect of scaled up of project-supported activities. An unallocated amount of US$20.1 million set aside for contingencies. The AF retained the original PDOs, four of the six original PDO Indicators. Sixteen new IRIs were added to the RF to assess the progress of the supplemental activities (see Annex 10). Component A: Health Systems Strengthening (HSS) (Estimated cost at restructuring US$25.4 million).

To enhance activities under the original component 1 by supporting i) scale up Malaria Laboratory Diagnosis and Improving Quality Assurance for Malaria Diagnosis (US$17.0 million), ii) monitoring of drug and insecticide resistance (US$5.75 million) at sentinel sites at both Federal and State level, and iii) Supply Chain Management (SCM) and Logistic Management Information System (LMIS) that will add malaria to ongoing routine logistics activities by the FMOH for HIV and tuberculosis (TB, US$2.65 million).

Component B: Community Systems Strengthening (CSS) (Estimated cost at restructuring US$7.0 million). To enhance activities under the original component 2 by supporting pilot activities at the community

5

level: i) Health Service Delivery through Community Directed Intervention (CDI) (US$2.0 million); ii) Strengthening Health Service Delivery through the Private Sector (US$2.0 million); and iii) Increasing Demand for Health Services by intensifying Behavior Change Communication (BCC), including engaging religious leaders of all faiths from the Nigerian Inter-Faith Action Association (NIFAA) to deliver malaria messages and materials (US$3.0 million). An impact evaluation (IE) was to be conducted to provide a formal evaluation on the impact of the pilot interventions.

Component C: Communications for advocacy, program implementation, and results (Estimated cost at restructuring US$10.0 million). To enhance activities under the original component 2 by strengthening and implementing national communications strategy to ensure effective advocacy, strengthened program implementation, and an emphasis on results.

Component D: Commodity Procurement (Estimated cost at restructuring US$37.5 million). To enhance activities under the original component 1 by supporting the procurement of ACTs, SPs, and LLINs in view of the universal coverage strategy.

17. Level II restructuring to reallocate proceeds of the AF Credit and revise the RF (Approved by the CD on July 26, 2011). In response to the FGON’s request on 1/6/2011, the CD approved to reallocate unused AF funds in the amount of US$22.30 million to finance the cost of distributing 30 million undelivered LLINs to 22 non-Project states. The undelivered LLINs were secured through the FGON and RBM partners, including the GFATM under the universal coverage strategy. The justification for the reallocation entailed strengthening the federal capacity to manage universal coverage of LLINs that ensured that gains made in other States (including the Project States) were not brought down by lack of progress in adjacent States. Under the restructuring, the RF was also revised to include an additional IRI that measured the number of nets distributed in these additional States (second Annex 10). The Bank agreed to proceed with the restructuring before the more major restructuring planned for one month later given the pressing need to get the LLINs distributed before the upcoming rainy season and the well-developed work plan by the FGON to distribute LLINs. The Bank made a strategic decision to undertake the restructuring prior to the political transition that were to happen in the aftermath of the general election in Nigeria; it was also the Bank’s response as a development partner to ongoing global malaria policy changes. 18. Level II restructuring to revise the Project’s RF, introduce enhanced implementation practices, and extend the closing date (Approved by the CD on Aug 4, 2011). On 3/3/11, the FGON requested for the restructuring based on the results of the Midterm Review (MTR) completed in November 2010. According to the MTR, the Project only progressed modestly in implementing treatment measures and HSS, particularly in the areas of procurement, supply chain management, M&E, and communications. The restructuring, which took place after the newly elected government took office, aimed to increase efficiencies and involved the following changes: Refine and merge the PDO and intermediate indicators of the PP and the AF to achieve a comprehensive,

harmonized result framework (see Annex 10). Enhance how the following activities were implemented: (i) ACT and IPT utilization; (ii) ITN use; (iii) IRS

coverage; and (iv) delivery of MPP interventions through Integrated Neonatal, Maternal and Child Health (INMCH) weeks.

Provide three additional consultants to the NMCP to help support and strengthen program management, procurement, and communication activities.

Extend the Project closing date to June 30, 2013 to ensure full implementation of activities and demonstration of impact.

19. The Integrity Vice Presidency (INT) findings in 2011 resulted in changes to internal control management and procurement oversight. In February 2011, INT as the World Bank’s independent body that investigates allegations of fraud and corruption noted serious financial anomalies across the Federal and State levels, particularly in Akwa Ibom and Rivers. The findings included numerous irregularities in the training

6

programs and procurement of low-value goods (mainly national procurement) that did not require review and approval from the Bank. The INT report also highlighted system weaknesses, noncompliance with Bank procedures, and gaps in due diligence in the use of funds earmarked for program implementation. In response, the Project ring fenced components exposed to abuse through e-payment of all transactions and prior review of all procurement packages. The Bank increased oversight and supervision of Project. During this time, the Bank maintained strategic engagement with relevant ministries at the Federal and State levels. The scope of work within high-risk units was reduced until adequate staff replacements were in place and the capacity ensured. Instituting these measures delayed Project activities at the Federal level and in two States, especially for procurement of antimicrobial commodities which were critical to overall service delivery. 20. Kano State terminates its Credit with the Bank. On October 2012, Kano State elected to discontinue its participation by opting out of the Project and terminating its access to the IDA Credit. According to the Governor of Kano, the State had enough resources to fund its program, and as such not willing to continue with the World Bank support. At the time, the Kano State Government made a decision to withdraw from a portfolio wide engagement with the Bank, affecting other ongoing Projects such as the HIV/AIDS Program Development Project II and the National Urban Water Sector Reform Project. Actions undertaken to close Kano’s malaria engagement included among other things (i) refunding ineligible expenditures that accrued under the Project (USD $11,846.52) according to independent reviews by the Bank INT and Office of the Accountant General of the FGON and (ii) refunding the balance in their designated account into the Bank’s account. 21. Level II restructuring to extend the Project closing date and to adjust the Project’s legal document (Approved by the Regional Vice Presidency (RVP) on June 28, 2013). The FGON’s request to restructure the Project followed a proposal by the joint Project Steering Committee (PSC) on October 22, 2012. The restructuring sought i) a 12-month extension of the Project closing date from June 30, 2013 to June 30, 2014 and ii) a re-allocation of undisbursed funds to the Federal level from State Subsidiary Agreements of both underperforming States and Kano which at the time had just decided to discontinue from the Project and terminate its credit with the Bank. The closing date extension intended to provide the Project sufficient time to complete the activities that had stalled for 8 months in the aftermath of the 2010 INT investigations. The reallocation of resources, on the other hand, was anticipated to bolster the economies of scale for procuring the remaining Project commodities3, as well as to assist in fully utilizing the Project funds. This was meant to increase the efficiency of the Project’s implementation. Funds against the PP had been fully disbursed (US$180 million), while only 5.9% of the US$100 million AF had been disbursed. Because the 12-month extension resulted in a cumulative extension of 27 months beyond the original Project closing date, the request required RVP clearance. 22. Level II restructuring to extend the closing date (Approved by the RVP on June 23, 2014). The Bank approved the FGON’s request to extend the closing date of the Credit by nine months to March 31, 2105. The extension provided additional time to complete the following: i) LLIN distribution campaign in the six Project States; ii) BCC campaign for demand-side interventions; iii) end stage evaluation of the project; and iv) increased State level demand for implementation supervision. Ratings at the time were S for Progress towards PDO and MS for IP. No change in PDO and RF. Cumulative disbursement at the time was 99% against the original credit and 57% against the AF. Given that the restructuring resulted in extending the closing date 36 months beyond the original closing date, RVP approval was required.

2. Key Factors Affecting Implementation and Outcomes

3 The commodities included i) 6.75 million nets that helped partially cover the gap in LLINs that emerged as a result of the population growth and decay of LLINs over time since the 2009/2010 distribution campaign; ii) IRS in all States by 2014, with the aim to cover 85% of households from two LGAs per State; iii) 6.2 million additional ACTs to ensure the supply chain pipelines in the medium term beyond June 2013; and iv) a final round of Lot Quality Assurance Sampling (LQAS) survey in the first quarter of 2014.

7

2.1 Project Preparation, Design and Quality at Entry 23. Soundness of background analysis: Throughout the preparation stage, the Bank worked in close collaboration with development partners and technical agencies like the RBM Partnership, WHO, Department for International Development (DFID), and United Nations Children's Emergency Fund (UNICEF). The design of the operation took into account a program strategy that WHO recommended as effective and proven for the control of malaria and was being used successfully in other countries. In doing so, the analysis took into account lessons learned from international and regional experiences, such as the need for a strong leadership and commitment at all levels of government; a targeted technical approach using a package of effective tools to control malaria, combining effective case management with preventive measures; a strong results-based M&E system to drive decision-making and implementation; a partnership with private sector and community-based organizations to achieve large-scale, sustainable impact; and decentralized implementation and control of finances to promote program ownership by local governments, with the national malaria control programs retaining leadership in specific aspects of implementation, such as procurement of essential commodities. 24. From a country specific perspective, the Project took an approach that complemented lessons learned from Nigeria’s 2001-2005 Country Strategic Plan, such as strengthening program management at all levels; developing the national M&E capacity; and building the capacity of health staff as well as that of community-based health workers. In addition, the Bank incorporated key elements from previous and ongoing projects4 in Nigeria, including the importance of i) strengthening program ownership and commitment at all levels, ii) developing greater engagement with States, stakeholders, and partners, iii) linking annual work programs and resources with targets for results, and iv) enhancing institutional capacity particularly in the procurement of commodities. 25. The PAD highlighted two risks rated as substantial. The first related to Nigeria’s institutional and implementation capacity constraints that could potentially hinder Project implementation. As a mitigation measure, the Bank team proposed to finance capacity strengthening through training, contracting with PIFs, and resource mobilization for sustained Program development. The second substantial risk was potential delays in procurement. The Bank team anticipated that incorporating direct payment and early tendering would respond to this risk of non-availability. No Quality Enhancement Review (QER) was conducted during the preparation stage. 26. Assessment of the Project Design: Based on these lessons, the Bank developed a Project that took advantage of the existing framework and infrastructure for malaria control adopted by the NMCP, through its collaboration with the RBM Partnership; this arrangement complemented and harnessed the funding and activities of other partners, namely WHO’s involvement in case management, UNICEF’s work on developing the national strategy for ITN, M&E, and support for coordination and communication across agencies and commercial stakeholders, United States Agency for International Development’s (USAID’s) initiative on developing guidelines for malaria in pregnancy, DFID and GFATM’s contribution to the RBM Partnership, and the Bill and Melinda Gates Foundation (BMGF). Moreover, the design was visionary in employing a Malaria Plus approach that situated malaria within the overall maternal and child health landscape, when most malaria projects were designed as single disease vertical projects. This clearly pushed the Bank and the government at both levels to engage broadly and seek ways to strengthen the delivery of a comprehensive package of services, moving beyond the immediate boundaries of just malaria. The Project adopted a phased approach by focusing only on seven States based on limited IDA project financing and previous experience showing targeting all or a large number of States slowed implementation. The selection of these States came about as a result of a consultative process that

4 Lessons are from Bank-financed projects as well as from projects financed by other donors. The Bank Team worked closely with partners managing the GFATM Project, DFID, Community Participation and Action for the Social Sector (COMPASS) Project by USAID, and others.

8

considered a list of criteria.5 This was a sensible approach given that Project funds were not sufficient nor meant to support the Government’s strategy nationwide, even though malaria endemic persisted over the entire country albeit at varying degrees. This synergy helped bring to bear everyone’s contribution towards maximizing the malaria control effort in Nigeria. Other design alternatives were considered but rejected. 27. The 5-year timeframe for the original credit (May 2007 to March 2012) was reasonable given the country’s own multi-year National Strategic Plan 2006-2010, whose framework guided the design of the Project. Of the original US$180 million IDA Credit, US$22 million was designated as contingency funds, while US$158 million was allocated to the Project components. Specifically, US$86.5 million was designated for Component 1 of the Project to strengthen the capacity at the Federal level by enhancing procurement, M&E, and oversight functions. Under Component 2, $71.5 million was designated to strengthen the health system to improve delivery of the MPP in the target States, including increased access, utilization, implementation of preventive and treatment services. The allocation appeared justified given that both levels required additional boosts and funding to strengthen capacity and undertake implementation. The MCBP intended to increase coverage of malaria-specific interventions with effective service delivery through the FMOHs and State Ministries of Health (SMOHs), broader health-system development, and capacity building across multiple sectors. The Project envisioned supporting the country-led programs, including the NMCP with an emphasis on outcomes, flexibility in approaches, and partnerships with civil society organizations and partner agencies. It must be noted that the Project’s comprehensive and coordinated effort to control Malaria was the first of its kind in the country. 28. The project established PIUs at the Federal and State levels with staff seconded from various posts within the Ministry of Health. However, certain staff positions were specifically recruited to meet specific needs, including PIFs who helped develop procedures and resource mobilization for sustained program development and implementation at the State level. Using existing structures and personnel helped to quickly establish the Project infrastructure. The structure of the Federal and State were well defined and their responsibilities outlined in the main and subsidiary agreements on the basis that capacity strengthening for both the Federal and State PIUs would be created to facilitate program supervision, M&E, procurement, and policy at the Federal level and the implementation of activities at the States. The approach was innovative due to a design focused on the strengthening of State PIUs to function and implement activities with direct financing through subsidiary agreements, without solely relying on the Federal PIU as in past projects. At the Federal level, the design called for establishing a PSC responsible for overseeing the Project. The PSC was chaired by the Minister of Health and consisted of representation from FMOH, Federal Ministry of Finance (FMOF), States and LGAs, other development partners, Faith-based Organizations (FBOs), and the private sector. In addition, funds were made available to finance contracts for an M&E Agent to assist the NMCP develop an information system that measured output indicators to track Project activities, quality of services, and their outcomes. The M&E Agent was also to provide support in developing a management cycle for using data routinely for decision-making. In addition, a contracted Federal level Logistic Support Specialist would assist States with tracking their own allotments and consumption of commodities. A Social Marketing Agent and Procurement Specialist were also hired to assist with Project implementation. To mitigate the substantial risk in potential delays in procurement, the Project utilized direct payment and early tendering. 29. The project facilitated CDI training in the States, with CDI interventions being implemented in all (but Akwa Ibom) States through NIFAA, which paired an imam with a pastor as members of Project Teams in each State. Consistent messages about malaria were announced in Mosques and Churches, which helped improve interpersonal BCC at the community level. As part of the CSS of innovative approaches introduced in the AF, the

5 Selection criteria: i) mortality rate among under-fives exceeded 260/1000; ii) access to effective primary health care services was poor to non-existent (based on access within 5 kilometers; iii) access to secondary facilities was very poor or non-existent (based on required travel time exceeding half a day); iv) the States demonstrated commitment to implement large scale campaigns to cut child mortality and/or a comprehensive malaria booster program; v) documented Plasmodium Falciparum resistance to chloroquine and SP (the first line treatments in use) exceeded 85 percent; vi) an implementable State-Level RBM Strategic Plan for malaria control (based on federal guidelines and internationally recommended best practices) existed; and vii) no other significant donor aid for malaria control was currently available.

9

project conducted an Impact Evaluation in Gombe and Anambra designed to look at the effectiveness, jointly and in isolation, of CDI and Patent Medicine Vendor (PMV) interventions on key outcomes such as sleeping under bed nets, prompt access to ACTs for treatment, and incidence of malaria. Traditionally, CDIs and PMVs represented significant and popular avenues in Nigeria for managing malaria. CDIs for example played a role in dispensing malaria commodities by giving community members the authority to decide on options for distribution mechanism, timing of distribution, and selection of volunteer community directed distributors (CDDs). PMVs on the other hand usually appeared as the first choice in health care in the country and served as the primary sources of orthodox drugs for both rural and urban populations, especially the poor. In addition to selling drugs, PMVs also functioned as a major source of advice about illness and drug therapy. Results from the IE were expected to inform the FMOH and States how well these interventions worked on improving health outcomes.

2.2 Implementation 30. A QER of the MCBP’s implementation stage was held on October 15, 2009 as part of the overall effort by the Health, Nutrition, and Population in Africa Unit (now under the Health, Nutrition and Population Global Practice) to strengthen portfolio performance of ongoing projects by identifying key achievements, systemic challenges and constraints, lessons learned, and strategic options. The rationale for including the MCBP in the review revolved around Nigeria’s slow progress in improving its health indices during the preceding 10 years and the size of the Nigerian health portfolio, which was about 1/3 of the entire health portfolio of the African region. The discussion focused on three broad categories as described in Table 1. Table 1. QER Discussion Points and Action Plan

Discussion Categories Action plan moving forward 1. Institutional Arrangement: i) Progress of disbursement; ii) delay in engaging PIFs; and iii) delay in participation of FBO in implementation.

i) Disbursement is on course; ii) NMCP planned an orientation program for the PIFs within the month; iii) Bank provided $2million in project support to NIFAA to establish its secretariat and run its operation for the following two years.

2. Implementation: i) Support for capacity building to PIUs were needed at Federal and State levels; ii) project team at Federal level visited States at great frequency and State levels came to Abuja for several meetings offering opportunity for interaction; and iii) incentives to PIU members not part of Project design due to MOF’s concern for sustainability.

i) Hired KPMG tax and audit firm to assist with capacity building; ii) FGON and State level took ownership of malaria control through various documents and having malaria control as a budget line; iii) continue discussion on performance incentive.

3. M&E: i) Difference in RF b/w PP and AF given that the AF has specific indicators for the health and community systems strengthening and community components; ii) delay in reporting results of 2006 baseline due to lack of NMCP capacity to collate and analyze the data, which was eventually contracted out.

i) Project needs to develop table of planned surveys until the end of project; ii) Health Facility Assessment (HFA) 2009 would serve as a baseline upon which the HSS component of AF would be evaluated; iii) hire 3rd party firms to ensure independence of data; iv) use two non-participating states as control for the Lot Quality Assurance Sampling (LQAS) household survey and HFA; v) PIFs to assist in routine data collection and analysis; vi) for IE, the Project should be wary of attribution but emphasize contribution. However, having comparison groups would be helpful in establishing attribution.

31. Mid-Term Review (MTR). At the MTR, which was conducted on 10/18/2010-11/12/2010, the Project was rated as Satisfactory for its progress in achieving the PDOs. Despite making headway in implementing operationally vertical program areas that were highly visible, such as the rolling out bed nets, the Project was slow in areas that required integration into the ongoing program and/or dependent upon sustained efforts of the entire machinery, such as procurement, supply chain management, M&E, and communications. The six main challenges addressed in the MTR with their mitigation measure are described below.

10

Table 2. MTR findings – challenges and mitigation measures Challenges Mitigation measures 1. Delayed distribution of LLINs. a) Finalized contact with NIFFA; b) ensured the CDIs were fully

functional; c) revitalize national BCC campaign. 2. Slow progress in improving the delivery of HSS component.

a) Finalize contract with Society for Family Health (SFH) to spear head increased private access to ACTs through PMV; b) improve availability of ACTs and Rapid Diagnostic Tests (RDTs) and IEC/BCC for increased uptake of the service at health facilities.

3. The Malaria Plus Interventions. a). Use the National Neonatal Maternal Child Health weeks as part for a rapid catch process through the leadership of Nigeria’s National Primary Health Care Development Agency.

4. Slow pace of disbursement. a) Improve procurement TA; b) improve the level of technical assistance in the NMCP to ensure project designated actions are conducted in a timely manner; c) track the PIFs performance.

5. Ambiguity of certain intermediate outcome indicators that affected their measurability and lack of unified RF that both reflected PP and AF indicators.

a) Recruit a firm to carry out follow-up HFA comparison with the baseline HFA; b) use Health Management Information System (HMIS) data to measure progress of treatment activities in the public sector treatment since 2006; c) recruit firm to conduct next 2 LQAS and Maternal, Newborn and Child Health (MNCH) week assessments; and d) streamline the RF, indicators, and targets to ease collection of performance data.

6. Limited impact on PDOs given the delays in starting up the project, the low fund utilization to date, and the long lead-time required for start-up of crucial interventions such as CDI, PMV, and HSS measures.

The mission agreed to restructure the program and seek management approval on 03/03/11 for a project extension.

32. The proposed changes were formally incorporated in the Project through a restructuring that was requested by the FGON on 03/03/11 and approved by the Bank on 08/04/11. 33. Delay in contracting PIFs. At the State level, in addition to the signing of the Subsidiary Credit Agreement (SCA), a condition of disbursement was the procurement of State level PIFs. However, there had been considerable resistance initially from the Project States to the hiring of the PIFs. A second capacity assessment of the seven Project States undertaken by a joint supervision missions with the FMOF and FMOH in June 2008 found that all 7 States required the assistance of PIFs in the areas of logistics & supply management and M&E. Based on this assessment, the procurement of the PIFs were initiated in all seven States, thereby fulfilling the disbursement condition specified in the FA. However, the initial contention from the States resulted in several months of delay and transaction cost of undertaking a Project restructuring. It took three years before PIFs at all States were hired and in place to provide much needed implementation support. As evidence of the need for PIF support, capacity at the States improved drastically upon the hiring of PIFs including undertaking i) training of State level trainers in HMIS and M&E tools, ii) training for State level trainers in case management and diagnosis, and iii) recruitment of CDI focal points. 34. Program Policy Change. As part of the 2008 program policy change for which AF had been requested, and in keeping with the global move towards 100% coverage of households with LLINs as the optimal strategy for malaria control, a detailed joint plan of action was agreed to by the government and the partners to saturate the country with LLINs, including the Project States. The strategy involved utilizing a comprehensive and effective set of interventions as recommended by the World Health Organization (WHO) for malaria control and as agreed to by the FGON to ensure attainment of universal coverage by 2010. This commitment meant implementing a series of stand-alone net distribution campaigns conducted across the country in 2009-10 and distributing over 60 million nets within a two-year period, a scale up of a magnitude not witnessed anywhere else in the world. The shift in policy to universal coverage resulted in a 4.1 million gap in LLINs in Project States, which brought the total Bank procurement needs to 12.4 million LLINs. With Bank support, all seven States received their original

11

share of the 8.3 million LLINs. The additional 4.1 million LLINs required for filling the gap arrived by January 2010. As part of the FGON’s strategy to provide LLINs to all households in Nigeria, 63 million LLINs had been secured through the FGON and RBM partners, including the GFATM. However, due to a funding gap of US$22.30 million required for the operating costs to distribute the LLINs, 30 million LLINs had not been delivered to 22 non-Project States. 35. Volatility in ratings for progress towards achievement of PDOs and implementation progress during the last year of the Project. Between April 2014 and March 2015, the ISR rating for progress towards achieving the PDOs moved from S to MU to MS, while the rating for implementation progress moved from MS to MU to MS. Progress towards the achieving the PDOs was downgraded in April as a reflection of survey results from the 2013 DHS that showed a reduction in both coverage and utilization of malaria interventions in both Project States and non-Project States. While several reasons might have explained the results6, the Project team felt that the next nine months of Project had to intensify the BCC campaigns to improve availability of LLIN, ACTs and RDTs in the Project States in order to reach indicator targets by the end of the Project. AS for the implementation progress, the downgrade reflected the delay in the implementation of LLIN redistribution, which eventually affected the PDO achievement. Based on the outstanding progress during the last nine months of implementation, both in terms of outcomes and intensity, both PDO achievement and implementation progress rating were upgraded to MS. Result of the remarkable progress was evident in the results of the 2015 LQAS, in which all PDOs targets were surpassed.

2.3 M&E Design, Implementation and Utilization 36. M&E design. When the Project started, the M&E unit of the NMCP had just been established and required strengthening. Moreover, data received through the recurrent health information system were incomplete and exhibited substantial reporting delays and gaps. Medical records in health facilities were also incomplete and unreliable. The Project design focused on the need for systematic and regular M&E aggregate data from the States, through a decentralized M&E system, to assess the status of the overall process and introduce program and policy changes as needed. The Project’s M&E plan, which came about after extensive consultations with the NMCP and RBM partners, included four components: LMIS to track all project commodities, such as ITN and ACT from their point of entry into Nigeria through to

the decentralized distribution and storage points in the Sates and LGAs. Rapid population-based survey – conducted at baseline, mid-Project cycle, and end-of-Project using a unique

LQAS methodology pioneered through the Project – to track coverage and use of ACT, ITN, and IRS. Two control States would be included in the assessment to help determine exogenous influences on key program indicators.

HFA – conducted at baseline and end-of-Project – in Project States and two control States to determine the quality of care at a sample of health facilities and PMVs by assessing human, physical, and material inputs available at service delivery points.

Special Studies will be carried out as a means of enhancing rapid M&E approaches.

6 Reasons for regression on achieving the PDOs: 1) LLIN life span is at least three years. Since the initial distribution in the Project States was done in 2009/2010, the current LLINs available at the time were less than half of the original numbers covered due to both degradation and attrition, impacting coverage and utilization; 2) The timing in the implementation of the DHS survey was not optimal to malaria transmission and hence the population’s perceived risk was low. This would led to a reduction in utilization of both prevention and treatment interventions; 3) The sampling methodology may have under sampled high risk population most likely to use malaria interventions; 4) The indicator on utilization of ACT by children under five was also impacted by a change in policy that required ACT use given to those that had positive confirmation of malaria infection as shown by Rapid Diagnostic Tests and not merely the presence of fever as was being done at the time of project development. These changes in indicators as a result of global consent will impact greatly on the end stage result.

12

37. At the State level, PIFs were to be brought in to i) support data collection teams comprised of the SMOHs and the private sector, ii) help conduct HFAs that would be used to determine the quality of care of health facilities, and iii) strengthen the M&E by emphasizing the use of data for monitoring and evaluating implementation strategies and process indicators. At the federal level, a contracted M&E Agent was to assist the M&E Unit of the NMCP to strengthen its capacity to aggregate, analyze, and report to decision makers in an accessible, rapidly usable form. To ensure coordination between the Federal Government and the States, an annual Forum for Health Results would take place that brought together both levels of government in order to review M&E data, identify priorities, and develop action plans to overcome challenges. Given that the RBM approach fostered an outcome-driven program within the FMOH before the Project, the M&E design for the MCBP promoted a more pro-active and responsive engagement among partners to combat malaria. 38. The Project had six PDO indicators, of which five were focused on monitoring the progress in delivering MPP interventions in Project States under PDO 1. These five PDO indicators adequately measured progress in providing relevant services to the most vulnerable groups that include children < 5 years of age and pregnant women. On the other hand, the only PDO indicator designed to track the progress of strengthened ability at the Federal and State level to manage and oversee MPP delivery as described under PDO 2 was one that monitored the use of M&E data by Project States to prepare annual work plans. This however was dropped as part of the AF Restructuring in 2009 based on the argument that the indicator met its target consistently and thus added limited value. For the remainder of the Project, no PDO indicator was used to directly assess the achievement of PDO II, while four of the fourteen IRIs were used to monitor PDO II. Given the number of surveys that the Project was anticipated to conduct, as well as the delay in conducting the mid-cycle LQAS survey in Q2 of 2009, it would have been worthwhile to keep the original PDO II indicator to track sustained levels of commitment by the Federal and States levels to maintain M&E capacity. In addition, other indicators could have been incorporated that assessed the capacity at both levels to manage MPP interventions. With a significant amount of funding set aside for large commodity purchases and small-scale activities, the RF could have included an indicator to assess logistics or fiduciary oversight related to commodities purchased. This would have been reasonable given that the PAD identified weak procurement in Nigeria as a significant risk. Having had such an indicator might have signaled financial irregularities in the Project well in advance of the GFATM report that exposed the problem. Overall, the set of PDO indicators were heavily weighted towards monitoring PDO I. 39. M&E implementation. The RF underwent three modifications to account for changes in the Project’s focus. For example, new IRIs were added to the RF to reflect new activities introduced during the project implementation stage (for more detailed description, refer to Section 1.7 Other Significant Changes under relevant Restructuring and to Annex 10). As part of the AF in 2009, 16 new IRIs were introduced to monitor progress of additional activities. In addition, a restructuring in 2011 introduced a new IRI that tracked net distribution in 22 States outside the seven Project States. These changes resulted in having to track 36 IRIs, many of which had inherent problems of definition and effective tracking. Through another restructuring in the same year (2011), the Project dropped 26 and revised 7 IRIs that posed measurement challenges, while adding 4 others that ensured efficient and effective tracking and enhanced the Project’s capacity to monitor implementation progress. Indicator targets were adjusted to better estimate the expected contribution of the Bank credit in the Project States. The final Results Framework (RF) included six PDO indicators and 14 IRIs (Annex 10). 40. As part of ongoing discussions to coordinate overall effort within the Project and the country in general, all partners agreed on a common set of M&E instruments – including using LQAS for routine monitoring, the HFA, LMIS, HMIS, and IE. This agreement ensured that the core of implementation of the national program focused directly on results. 41. In addition to routine information systems being weak, ITN use rates and early treatment rates were household level behaviors that were best measured through household surveys. The fact that 70% of fevers were treated in the private sector also argued against relying on public sector information systems. Thus, the Project had to rely on surveys to track progress. The NMCP with Bank support undertook a baseline LQAS household

13

survey in 2006 to establish measures of key indicators set by the Project. However, changes in the NMCP leadership, including the Project Coordinator, resulted in some disruption of planned activities. Thus, completing the planned HFA that would have served as baseline for the assessing progress made on HSS interventions was delayed by several months. Conducting the mid-cycle LQAS was also delayed (planned for Q2 2009 and deferred to Q1 2010) at the request of the NMCP and partners, recognizing i) delays in the net distribution campaign that would have affected findings, ii) the onset of the rainy season, and iii) difficulties in finding an appropriate consultant to undertake the exercise. A new Project Coordinator who was appointed in December 2009 however helped move things forward and catch up for lost time. By 2010, signs of the effort to develop the M&E unit, along with a core group of M&E experts, became evident as the country gained the capacity to adequately monitor the Project’s progress and to conduct its own surveys such as the LQAS, HFA, and the Malaria Indicator Survey (MIS). Key outputs that followed included the LQAS in 2010 and HFA in 2010 to measure mid-cycle progress. The end-of-Project LQAS and HFA were completed in 2015. The baseline, mid-cycle, and end-of-Project surveys all used the Comparison States (Kaduna and Delta) to isolate impact of Project interventions on Project States. 42. The Project facilitated in developing tools that harmonized Procurement and Supply Management (PSM) and M&E management, and provided training in both service management and logistics management information systems. This produced a robust LMIS design that addressed supply chain management gaps for drugs and other commodities. This has impacted greatly on collection of malaria data and also routine collection of malaria information from the LGA, to the State and to the Federal level. 43. Utilization. The Project aided in strengthening the HMIS and in formalizing tools for quarterly reporting on community distribution and usage; services statistics from the health facilities and management issues relating to reporting. By the end of the project, the system put in place provided quality data that tracked and reported regularly on available commodities and impact on service delivery. This, along with the LQAS, ensured that any potential leakage of nets was minimized, and the impact on the beneficiaries was maximized. 44. Although none of the States were using M&E data for managing malaria programs when the Project began, all States were using M&E data by 2015 to plan for their malaria interventions and were submitting M&E data on a quarterly basis to the Federal level. Both the Federal and State level were strengthened tremendously as robust M&E units were established that could support monitoring of implementation and real time reporting of malaria data. Moreover, the core group of M&E experts led country malaria-based surveys such as the LQAS and MIS, and guided the development of modules on malaria for DHS. The NMCP has taken the lead for the supervision and monitoring functions of all program interventions. 45. Findings from the LQAS surveys were used during annual work planning sessions to restructure and improve the projects as well as to set targets for the following year. In between the LQAS, results from the Multiple Indicator Cluster Survey (MICS), DHS, and MIS Surveys were used throughout the Project to track progress of the Project. These were helpful in assessing trends of progress; despite the different methodologies that produced different results, all consistently exhibited positive upward trend in Project indicators.

2.4 Safeguard and Fiduciary Compliance

46. Safeguard. Two safeguard policies were triggered by the Project: Environmental Assessment under the Bank’s Operational Policy/Bank Procedure 4.01 and Pest Management. The Project received a Category B environmental rating at the appraisal stage and was not expected to generate adverse environmental effects and social impact given the risks associated using dichlorodiphenyltrichloroethane (DDT) and other pesticides for the IRS, and with the handling of healthcare wastes related to the diagnosis and treatment of malaria (needles and syringes, gloves, and glass slides). To address potential negative impacts however, the NMCP prepared an IVM Plan that highlighted options for vector and human activities; identified DDT use for IRS as an integral part of the

14

IVM Plan; and made recommendations regarding capacity building, training, and awareness building to ensure proper and effective implementation. With regards to healthcare wastes, an addendum to the harmonized medical waste management plan of the Abidjan-Lagos Corridor HIV/AIDS Project was prepared, given the absence of a national healthcare waste management plan for Nigeria. 47. The Project introduced waste management practices in IRS, medical commodity procurement, and LLINs campaigns. Capacity for the oversight of the medical waste management was strengthened to ensure that commodities procured through the project were disposed properly at waste facilities. Moreover, the NMEP developed i) policy guidelines on Environmental Health and Waste Management Safety Guards and ii) a training manual to guide appropriate measure in Nigeria. The States, with support from the Bank and the NMEP, conducted environmental and social audits. 48. Fiduciary Compliance. In April 2010, an assessment by the GFATM found flaws in the procurement supply management and poor oversight arrangement that had failed to properly identify and address risks. In February 2011, the INT requested the assistance of the Bank’s Africa Region Unit and other external donors to conduct a multi-phased proactive diagnostic review of the Project's fiduciary and logistics system to assess the risks which had been brought to light in the GFATM’s report. The review included forensic examination of payment vouchers, visit to state facilities and meetings with key implementation partners and allied stakeholders. Following the INT review that identified evidence of funds mismanagement resulting from Project weaknesses in internal control systems, noncompliance with Bank fiduciary procedures, and some fraudulent expenditures, the client with support from the Bank conducted an in-depth financial management assessment from the inception of the program based on the high-risk areas highlighted in the INT review. This assessment identified approximately US$6.5 million of ineligible or potentially fraudulent expenditures. In response, the FGON replaced senior management of Project Implementation Units (PIUs) and Project Financial Management Units (PFMUs) at the Federal and in Akwa Ibom and Rivers States, and took measures to increase its fiduciary oversight function. Moreover, the Government’s Economic and Financial Crimes Commission followed through with punitive actions. Also, the Program ensured that the LMIS addressed the SCM gaps underscored in the report. The office of the Auditor General, with strong support from the World Bank, also undertook a detailed review of a wider cross section of fiduciary transactions. A separate detailed review was carried out by Bank and the Controller Accountant General to examine all expenditures from the inception of the Project. The FMOF set-up a cross-sectoral committee of Technical Officers (a.k.a. Tiger Group) to further review the INT findings and draw up recommendations for further actions to be executed by Federal Civil Service Commission. During this time, the Project adhered strictly to other follow-up actions (e-payment, cancellation of foreign travels, prior review of all procurement packages, etc.). The Federal and States reimbursed all ineligible expenditures. Specific reviews of key vulnerable activities such as the bed net distribution were carried out to learn lessons and strengthen implementation. The Bank’s Country Management Unit (CMU) provided guidance and leadership in ensuring that the INT recommendations were followed through, allowing the project to continue with its implementation. The Bank Project Team continued to monitor progress by obtaining periodic status reports on all ongoing procurement packages from the NMEP. To assist with the steep learning curve required of the new staff members, the Bank supported the development of a structured capacity building plan that included: i) capacity building sessions; ii) frequent supervisory visits and ad hoc support; iii) adding an experienced consultant to help strengthen the procurement unit at the Federal level; and iv) ensuring that the PFMU oversee the accounting unit. According to INT’s Report7, the response of the CMU, Client, and Bank Project Team created a best practice for how to respond to an INT Review of a project implementation. 49. The procurement assessment conducted at Project preparation focused on procurement of big-ticket items and was not flagged as a fiduciary risk. And the INT found that large-scale procurements were implemented reasonably well, with ITNs and other commodities generally ending up in their supposed destination. The fraudulent activities occurred in the small procurement of items such as training and travel expenses. Although the

7 A Collaborative Approach between the Bank and Clients to Address Project Risks: the Case of the Malaria Booster Program (in draft).

15

small procurements did not meet the threshold to require prior Bank review, the total of these small purchases added to big amounts. This could have been flagged as a fiduciary risk in a country with weak procurement systems at both the Federal and States level. The Project might have mitigated the risk by i) limiting travel/operating expenses and small scale- procurement, ii) contracting out training components and employing performance based awards, iii) utilizing an e-procurement system that could help monitor and flag questionable activities; and iv) incorporating an indicator in the RF that assessed fiduciary compliance vis-à-vis procurement. 50. Disbursement. The total IDA Credit financing for the Project was US$280 million (US$180 million for the PP and US$100 million for the AF). As illustrated in Annex 1, the Project utilized US$276.13 million (98.62% of total financing) of the finances, including US$38.23 million of the US$42.1 million unallocated amount (US$22.0 million under the PP and US$20.1 million under the AF). The original unallocated amount of US$22 million under the PP found its use in contributing to the LLIN distribution procured by the GFATM to 22 non-Project States, while US$16.23 million from the remaining unallocated amount under the AF was reallocated to support the economics of scale through the procurement of commodities and assist in improving efficiency of Project implementation. With regards to FGON’s planned contribution of US$80 million, this came in various ways including in providing seconded Federal level MOH officials to many PIUs, as well as in States through complement procurement of drugs, RDTs, and LLINs. In Anambra for example, the State government contributed to further LLIN distribution following a gap that ensured the successful completing of the last net campaign distribution. The main and subsidiary agreements with the Federal and States posed a challenge however in that the Project could not fully document line item contributions that point to the overall inputs that were coming from the Government and the States as part of the US$80 million.

2.5 Post-completion Operation/Next Phase 51. To help sustain the project’s gains and the Government’s effort to control malaria, the Bank’s ongoing US$500 million Project, The Saving One Million Lives (SOML) Initiative Program for Results, that run from 2015 to 2019 will assist with the integration of essential priority interventions into primary health care, equitable increase access to, and utilization of quality health interventions, including for malaria. Beyond the Bank’s Project, the NMEP also designed its own malaria project with an integrated testing, treatment, and logistics management program that was packaged as a public/private partnership. The Project’s 4-year budget was estimated at NGN 72 billion, the financial analysis of which was conducted through the private sector. While the Federal Executive Council and the National Economic Council under previous Nigerian President Jonathan had already approved the draft, the proposal was still awaiting a review and final approval from President Buhari’s new government. It should be noted that moving forward would require commitments from development partners. As such, the Federal Government and development partners are in advanced discussion about opportunities to use innovative financing opportunities to fund the gaps in malaria control activities.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation 52. Relevance of PDO and design/implementation (combined implementation): Substantial 53. Relevance of PDO: Substantial. The PDOs remained consistent with Nigeria’s 2009-2013 NMSP where the FGON aimed to scale up for impact (SUFI) a package of interventions focused on prevention, treatment, and behavior change through a strengthened health system. During this period, an RBM Country Report on Nigeria (Progress and Impact Series, 2012) highlighted the importance of ongoing efforts to reduce the malaria burden, which remained unacceptably high. Towards the end of the Project Cycle, the MCBP’s objectives continued to be consistent with the 2014-2020 NMSP, particularly in ensuring that targeted population utilized appropriate preventive measures; availability of appropriate antimalarial meds and commodities required for prevention and

16

treatment of malaria; and strengthened governance and coordination of all stakeholders for effective program implementation. The Bank’s 2010-2013 CPS for Nigeria underscored the need for all development partners to help Nigeria expand malaria prevention, care, and treatment programs, similar to what the Project aimed to achieve. The 2014-2017 CPS that followed asserted the importance of increased access and utilization of quality services for infectious diseases that remained with a very large burden of disease, which included malaria. 54. Relevance of design and implementation: Substantial. The Project’s planned activities were consistent with the stated objectives, while the operational modalities responded to the needs of the country. For example, most of the allocation ($75.5 million) went to the Federal level for the procurement and logistics of commodities, which involved procuring and distributing 2 LLINs per household. This was in line with the national strategy where LLIN was used as the primary method of vector control. And given the emphasis on better access to prevention and treatment interventions mentioned in the various NMSPs, the comprehensive approach taken by the Project of integrating a broad segment of the Northern and Southern populations, remained relevant throughout the Project cycle and backed by global partners. In step with the RBM’s Country Report on Nigeria (Progress and Impact Series, 2012) that emphasized the need to work collaboratively in a broad partnership fashion, the Project’s coordinated efforts and open line communication with partners paid dividends – the Project found itself responding successfully and quickly to re-allocating funds to cover the costs of distributing the nets purchased by the GFATM to 22 additional States outside of the Project States, as requested by the FGON. In addition, the decision to move towards universal coverage in 2008 as the optimal method to control malaria amplified the emphasis placed on diagnostics, health systems development, and community systems, along with a required increase in commodities and technical & fiduciary systems, to which the Project responded. 55. The Project’s approach of staffing and creating functional PIUs at State level was considered best practice among partner agencies such as the GFATM, which has since then replicated the model in their own projects at the State level allowing them to deal with the States directly and manage projects more closely. This brought about robust teams that could meet the demands of the malaria program. Moreover, the role that NIFAA played in BCC by spreading messages about LLIN exemplified Nigeria’s goal of implementing interventions by involving civil society organizations, traditional and religious leaders and the private sector. Working with NIFAA was innovative at the time since the Bank had never engaged Faith Based Organizations for BCC. In the context of Nigeria, it was important to maintain ties to both religious groups, to come together to undertake a unified effort for the benefit of the country. Other innovative strategies introduced as part by the Project have since then been tested or expanded beyond the Project States. For example, IRS is now being implemented in small pilots generally at less than LGA level, through support from the United States President’s Malaria Initiative (US-PMI) and the Millennium Development Goals Achievement Fund (MDG-F). With regards to the IE designed by the Bank to incorporate an evidence-base assessment of interventions, development partners have agreed to its use as part of the harmonized set of tools to track progress in the battle against malaria. The MCBP became the first Bank Project in Nigeria to incorporate an IE into its design, which provided an important tool for evaluating what aspects of the Project were effective and should be scaled up. 56. Given the weak capacity in Nigeria, the Project integrated a practical approach of bringing in technical support to oversee specific tasks, including an M&E Agent and Logistic Support Agent at the Federal level. This allowed the Project to quickly gain that expertise and support needed to oversee the Project. The design also called for PIFs at the State level. The Project however experienced months of delay in hiring the PIFs due to objections from the State. This might have been avoided by communicating with and involving the States better and earlier on the process. Not only did this result in months of initial lag in activities, but also incurred transaction costs involved in undertaking a second needs assessment in the Project States and in amending the financing agreement to resolve the dispute that States could not access funds to finance operating costs unless PIFs were in place. The significant role that PIFs played in strengthening the capacity of the States became evident once they were hired, as implementation quickly progressed along with increased disbursement.

17

57. Although fiduciary weaknesses detected by INT threatened the implementation of the Project, the Bank Project team reacted quickly to resolve the problem by initiating an in-depth review by the Bank’s FM team, which led to the introduction of corrective measures and a turnaround of the project after a disbursement lag of six to eight months. In conjunction with Operations Staff, the response involved various units within the Bank, as well as other donors active in the sector: GFATM’s Office of Inspector General, USAID’s Office of Inspector General, and United Nations Development Program. To look into the details of issues raised, the Project Team established a committee (Tiger team) consisting of the Project Team, PFM, Procurement, and the Bank’s External Affairs Department while ensuring that all concerned stakeholders were involved in the process. Despite the various Bank units involved, the Bank operated as an integrated unit in terms of examining project implementation behaviors that could impede development outcomes. The Project Team’s response was considered best practice by INT and highlighted in the FY14-FY17 CPS as so. Early engagement of national counterparts was especially important as it fostered client ownership and buy-in. In light of the INT, the Project moved on by managing the risk and continued to deliver benefits.

3.2 Achievement of PDOs 58. Rating: Substantial. The overall rating is Substantial based on the following sub-ratings: i) PDO 1 – High; ii) PDO II – Substantial. A split evaluation was not needed since the only revision to the PDO was a change in one target; there was no change in how the Project was evaluated. 59. The Project exceeded all six formal PDO indicators as summarized in Table 3 below. Although the PDO indicator on the percentage of States regularly using monitoring and evaluation data to manage malaria and MCH programs had been dropped, this indicator was included below as a seventh indicator to provide a measurable gauge of how well the program performed in achieving PDO II. Table 3. Summary of PDO Achievement PDO Indicator Baseline Target Actual PDO Focus PDO Achievement Indicator 1: Percentage of children < 5 years with fever treated with an effective antimalarial within 24 hours from onset of symptoms

3.7% 15.0% 45.0% I Exceeded

Indicator 2: Percentage of children < 5 years who slept under an ITN the night preceding the survey

3.6% 60% 74.4% I Exceeded

Indicator 3: Percentage of pregnant women who received two or more doses of IPT

8.3% 25.0% 52.6% I Exceeded

Indicator 4: Percentage of pregnant women who slept under bed net night before the survey

4.1% 50.0% 73.4% I Exceeded

Indicator 5: Direct Project Female Beneficiaries NA 16 million 18 million I Exceeded Indicator 6: Number of children under five who received Vitamin A supplementation

NA 6 million 26.5 million I Exceeded

Indicator 7: Percentage of States regularly using M&E data to manage malaria and MCH programs.

0% 80% 100% II Exceeded

60. PDO I: To ensure that the target population will have improved access to, and utilization of, a well-defined set of MPP. Rating: High. The coordination between the Federal MCBP and the States played a critical role in establishing an enabling environment that allowed the NMCP to oversee Project and collaborate with partner programs, as well as for the States to implement activities at the local level, all with the goal of achieving the Project Objectives. The achievements in PDO indicators 1 through 6 (Table 3) provided evidence of improved access and utilization of MPP in the Project States. The progress demonstrated in the indicators related to ITN use (2 indicators) and treatment (2 indicators) represented significant achievements given that baseline values at the start of the Project were in the single digits. On average, these four indicators increased over the baseline value by 504%, and 64% over the target values. The Project also succeeded in surpassing the target for the number of

18

children < 5 years who received Vitamin A supplementation, an important component of the MPP given that complementary intake of micronutrients including Vitamin A can help prevent death from malaria-related complications. In addition, the Project exceeded the female beneficiary core sector indicator mandated by the World Bank. 61. An end-of-Project survey conducted during the last three months of the Project (January to March 2015) compared Project States with Comparison States.8 The results showed how much more the Project States improved compared to the Comparison States. For example, the survey found that while both groups had a high proportion of households that owned at least one LLIN (99.2%), likely due to the success of the nationwide LLIN distribution following the 2008 decision to attain universal LLIN coverage, the proportion of children under 5 years and pregnant women who slept under LLINs in the Project States (68.7% and 69.3%, respectively) compared to the Comparison States (36.6% and 47.9%, respectively) statistically differed significantly (Figure 1). Figure 1. Ownership and use of LLIN by children under 5 years of age and pregnant women

62. The survey also discovered a statistically significant difference between the two groups in the percentage of children under 5 years with fever who were treated with an effective antimalarial within 24 hours from onset of symptoms (Figure 2), with the Project States reaching a higher rate (43%) than the Comparison States (30%). Figure 2. Percent of children under 5 years with fever who received a laboratory test and treated with ACT within 24 hours of symptom onset

63. With regards to pregnant women, a significantly higher proportion in the Project States appear to have received 2 or more doses of IPT (51%) than the Comparison States (41%), despite pregnant women in the both groups equally receiving Antenatal Care (ANC) (Figure 3).

8 LQAS household survey that used Delta and Kaduna as Comparison States.

30%43%

0% 10% 20% 30% 40% 50%

% who received ACT within 24 hours of symptomonset

Project States (n=1,235 households)Comparison States (n=361 households)

Z = 4.37, P < 0.05

47.9%

36.6%

99.2%

69.3%

68.7%

99.2%

0% 20% 40% 60% 80% 100%

Pregnant women who slept inside the LLIN

Children under 5 years who slept inside LLIN

Ownership of at least 1 LLIN

Project States (n=1,235 households)

Comparison States (n=361 households)

Z = 11.05, P < 0.05

Z = 7.47, P < 0.05

19

Figure 3. Percent of pregnant women who received ANC and who had 2+ doses of IPT

64. Surpassing the PDO targets came about in part as a result of meeting the following IRIs: (i) percentage of households with at least one ITN/LLIN (baseline: 2.6; actual: 99.6; target: 80); (ii) percentage of women with children < 1 year of age who received ANC during the last pregnancy (baseline: 63; actual: 87.5; target: 72); (iii) number of RDTs distributed to health facilities (baseline: 0.0; actual: 14,400,000; target: 12,000,000); (iv) percentage of health facility using laboratory confirmation as the basis for treatment of fever in children < 5 years of age (baseline: 5; actual: 31.4; target: 30); v) percentage of aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution and/or recommended home fluids (baseline: 25.3;actual: 59.4; target: 50). Activities carried out under Component 2 of the Project (see Annex 2) bolstered the capacity of the Project States, which in turn contributed to successfully implementing MPP of interventions. A key activity involved the CDI training in the States, with CDI interventions being implemented in all (but Akwa Ibom) States through the NIFAA. The training helped improve interpersonal BCC at the community level. According to the end-of-Project household survey (2015), 44.6% of respondents in Project States heard malaria from worship places compared to 19.4% in Comparison States. Moreover, knowledge about malaria did appear to have surpassed or come close to meeting end-of-Project targets in areas with active CDI activities, as assessed by the IE, such as the percentage of caregivers who knows at least two symptoms of malaria (baseline: 63.3; actual: 85.0; target: 70) and percentage of mothers of children less than five years of age in CDI communities who know at least two ways of preventing malaria (baseline: 0.0; actual: 55.5; target: 60). The BCC helped spread the message within the Project States regarding the malaria treatment and preventive services, resulting in increased utilization of such services, as compared to Comparison States. The level of PMVs’ contribution to malaria control at the community level, as measured by the proportion of PMVs that can identify primary methods of prevention and treatment, increased from 42% (baseline IE 2013) to 60% (end stage IE 2015). In addition, the scale up of the MNCH weeks could have boosted MPP intervention consumption as evident from results from both the May and November 2014 MNCH week campaigns that showed an increased uptake of Vitamin A (greater than 80%) and deworming (40%). MNCH contributed to increasing the utilization of malaria interventions due to the high level of behavior communication messages and activities carried out during the MNCH campaign period. 65. Numerous surveys were conducted by other partners in Nigeria throughout the life of the Project, which provided an independent gauge of how well the Project progressed in reaching its targets. Despite the different magnitude in point estimates found in the various surveys, that more likely reflected differences in the methodology used in the assessments and this making direct comparison invalid, the results from all surveys showed upward trends in the indicators monitored by the Project. For example, the MIS 2011 corroborated the findings of the 2010 mid-cycle household LQAS survey that showed an upward trend among Project States in the uptake of preventive measures. The MIS showed that the Project States fared much better than the country on average. Net utilization among children under five years of age and pregnant women increased from 3.6% (LQAS 2006) to 44.8% (LQAS 2010) and 74.4% (LQAS 2015), and from 4.1% (LQAS 2006) to 40.4% (LQAS 2010) to 73.4% (LQAS 2015) respectively. The 2011 MIS found that 51% of children under five years and 53% of pregnant women within Project States slept under nets; the national averages from the MIS were 29% and 34%, respectively. Similarly, the percentage of under five year old children who used effective anti-malaria medicine within 24 hours of onset of fever in the Project States improved from 3.7% (LQAS 2006) to 5.8% (LQAS 2010)

41%

88%

51%

87%

0% 20% 40% 60% 80% 100%

% of pregnant women who received 2+doses of IPT

% of pregnat women who received ANC

Project States (n=1,235 households)

Comparison States (n=361 households)

Z = 3.35, P < 0.05

20

to 45% (LQAS 2015). The 2011 MIS found this figure to be 38% within the Project States, while the national average was 26%. The findings of the MIS show that against national averages, the Project States did much better. 66. The results above showed vastly improved capacity of Project States to implement and deliver MPP, as compared to Comparison States or the national average. The Project contributed to the enhanced capability of the Project States to use mass distribution campaigns to provide universal coverage of LLINs, selective use of ACTs to treat malaria through laboratory technicians with better diagnostic skills as well as the training skills to continue with increasing laboratory human capacity, and better understanding and integration of community BCC strategies that incorporated NIFAA to capture both Christian and Muslim messaging media. 67. PDOII: To strengthen Federal and States ability to manage and oversee delivery of Malaria Plus interventions. Rating: Substantial. While the Bank team dropped the PDO indicator target for the percentage of States regularly using M&E data to manage malaria and MNCH programs due to its target being met early and consistently (actual: 100; target: 80), the indicator substantiated the success the States had in managing and overseeing the interventions through the coordinated work of the PIFs, M&E Agent, and Logistics Agent. It would have been worthwhile to have had other PDO indicators that focused on the overall management of the Malaria Project by the FGON and the States, including critical areas as financial management and logistics. As such, rating PDO II, though compelling given the data, hinges mostly on achieving the IRIs targets. 68. The Federal level played a critical role in creating an enabling environmental for the implementation of activities, coordination of partner programs, and providing oversight function to the States. This included procuring LLINs, ACTs, and RDTs at the Federal level that not only helped maximize economies of scale, but aided the States in meeting the following IRI targets: (i) LLIN purchased and distributed (baseline: 0; actual: 51,392,304; target: 12,400,000); (ii) number of LLINs distributed in 22 additional States (actual: 32,592,304; target: 20,700,000); and (iii) percentage of households sprayed with IRS in the last 12 months (baseline: 3; actual: 83; target: 80). In addition, the Project supported capacity building for malaria laboratories in the Project States through a laboratory strengthening component. This facilitated the development and review of tools (guidelines, protocols, and implementing mechanisms) that supported the scale-up for both malaria microscopy and RDTs and enhanced quality assurance for malaria laboratories. The Project also engaged SMOHs in preparing for scaling up their capacities to conduct microscopy and RDTs as described by the NMCP malaria laboratory diagnosis policy. 69. The Project procured microscopes, along with more than 14 million RDTs. Accordingly, the number of RDTs distributed to health facilities and the percentage of health facilities using laboratory confirmation as a basis for treating fever both exceeded project targets. The States not only benefitted from the improved capacities of laboratory technicians in microscope diagnosis for malaria, but also in other tropical diseases. With regard to monitoring of drug and insecticide resistance, the Project aptly supported sixteen existing sentinel sites and assisted in upgrading three new ones by building up the institutional and human capacities, improving operational management, and establishing quality assurance mechanisms. 70. The Project facilitated the recruitment and hiring of commodity logisticians and pharmacists as needed, and provided training for all malaria commodities and logistics staff. With project support to strengthen medical warehouses across the Project States, commodity quantification, drug availability in facilities, and reorder processes improved. For example, the logistics consultant worked closely with the NMCP and partners to refine the national logistics plan, conduct training on micro-planning for Project States and assessed the readiness and suitability of the warehouses in the States. The Kano campaign was very successful and largely peaceful, with a very high beneficiary turnout. The end process assessment showed, among other things, that an average of 90-95% of the targeted households received the nets and 60.9% slept under the net on the night prior to the assessment. Such enhanced capacity contributed to accelerating the implementation of MPP interventions, including LLIN purchased and distributed.

21

71. Thus, the Project made significant progress under PDO II despite the lack of good key indicators to substantiate the achievements, much of which enhanced the capacity of Nigeria to undertake sole responsibility of the management and delivery of Malaria Plus interventions. This included, as illustrated above, the Federal government’s ability to do large procurement of commodities, the capacity of States to employ microscopy and RDT skills to confirm malaria and deliver malaria treatment through improved warehouses and clinics, and the elevated level of the country’s capacity in managing operations, logistics, and surveillance. Without these strides and coordinated efforts, the achievements documented under PDO I would not have been possible given their role in making MPP available and accessible to the population. 72. IE on CDI and PMV in Gombe and Anambra. Analysis of data from the IE study showed that the targets for net use and receipt of treatment among children < 5 years of age and pregnant women in communities with CDI were exceeded or nearly met as described earlier, providing some indication of their impact in improving behavioral change. Results were mixed however with regards to ensuring that CDIs and PMVs had stocks of proper medications: i) percentage of sampled PMV that had ACTs in stock at the time of assessment (actual: 73.26; target: 80); and ii) percentage of sampled CDDs with no stock-outs in the last three months (actual: 39.10; target: 80). The results indicated that stock-outs was a problem among CDD, making them an unreliable source of drugs; the problem existed as well with PMV but less so. However, the delay in conducting the baseline IE (April 2013) resulted in a shortened timeframe from which to measure that impact of the interventions. In addition, the supply chain challenges as well as policies related to using RDTs by PMVs might have dampened result findings (see Annex 11 for more detailed summary of IE). As such, the IE may not have captured a larger impact had it benefited from a longer assessment period. Given the Nigeria’s emphasis on strengthening community systems and involving the private sector, the findings illustrated that additional effort needs put in to fully incorporate CDDs and PMVs in the fight against malaria. As the first health sector IE linked to the Bank’s portfolio in Nigeria, the MCBP IE was particularly important as a catalyst in stimulating broader interest within the health sector in rigorous evaluation of health policies and programs for evidence-informed policymaking. Consequently, this IE was instrumental in launching further evaluation work including of Nigeria State Health Program Investment Credit (NSHIP) and of programs that did not receive World Bank financing, such as the Subsidy Reinvestment and Empowerment Programme Maternal and Child Health Project and the Quality Improvement and Clinical Governance Initiative. Table 4. Summary of Efficacy Rating PDO PDO Rating Overall Efficacy Rating PDO I: To ensure that the target population will have improved access to, and utilization of, a well-defined set of MPP

High Substantial

PDO II: To strengthen Federal and States ability to manage and oversee delivery of Malaria Plus interventions

Substantial

3.3 Efficiency 73. Rating: Substantial. As indicated on Table 3 below, the Internal Rate of Return (IRR) and the Net Present Value (NPV) were both very high indicating that the Project was economically viable. The calculated Cost Benefit ratio was also very low meaning that the project was highly cost effective or cost efficient (see Annex 3 for assumptions applied to calculations). Table 5. Economic Analysis Indicators Benefit Valuation Methods and Discount Rate Assumptions

GNI per capita, 6% discount rate GDP per capita, 6% discount rate Present Value of benefits (billion US$) 3.086 3.892 Present value of cots (million US$) 192.2 192.2 Net Present Value (NPV, billion US$) 2.89 3.7 Internal Rate of Return (IRR, %) 65% 70.5%

22

Cost Benefit Ratio 0.06 0.05 74. Results from the 2015 LQAS showed increased access and utilization of malaria interventions. Vulnerable groups (women 15-49 years and children under 5 years) experienced exponential improvement in health results in terms of the disease burden, deaths averted, and lives saved, an indication that these groups benefitted from the interventions. The 1,970,092 in total disability-adjusted life years (DALYs) averted represented more than three times of that estimated at baseline (see Annex 3 for detailed calculation). The foregoing analysis therefore shows that the MCBP was good for the people and beneficial to the country and in most part achieved its objectives. These interventions were effective in contributing to overall reduction of malaria morbidity and mortality at the Project States. The children under five gained the most positive results with the various interventions. According to the economic analysis conducted when the AF was proposed, reducing bottlenecks or attaining coverage and utilization of 75% or more reduces mortality rate among children under 5 years of age and maternal mortality rate on average by 35% and 2.0%, respectively. PDO indicators made significant strive towards reaching this mark in Project States (including Kano) that accounted for 25% of the Nigerian population, especially for ITN use given role as an effective method of vector control. Along with the Project’s impact compared to Comparison States as previously illustrated and the Project’s support in distributing LLINs to 22 non-Project States, the Project contributed to the improvements seen in the disease burden. 75. Programmatically, the Project financed both systems strengthening for capacity building in program implementation and procurement of anti-malaria commodities including the design of malaria interventions. The Project scaled-up capacity building and implementation at both Federal and at State levels, and internalized the use of annual project planning to guide on resource utilization and strengthened the project understanding of procurement processes. The Project was designed to ensure that adequate coverage of interventions was attained for maximum impact, which the Project succeeded in achieving. With regards to value for money, the Project added value towards efficient planning for implementation, utilization of resources, and the high coverage of interventions. In particular, the Project helped improve implementation of interventions that increased coverage and access to LLINs, increased availability of ACTs and RDTs in health facilities, and strengthened M&E and fiduciary management practices, all of which contributed to reducing the overall impact of malaria in children under five and pregnant women. Although most other non-Project States had begun to receive equally as much support from other partners as the Project States by 2010, the survey results exhibited that Project States were able to add value to their support as evidenced by the increase in access /coverage and utilization rates when compared to Comparison States. The FGON’s capacity to successfully request additional funding after the initial 2007 IDA exhibited its bolstered leadership and effective Malaria Program. The cost effectiveness of the project and the benefits on the target population make the case for continued intervention after the project phase and for future sustainability. 76. The Project initially suffered initial delay due to capacity issues and need for the States to have PIFs on board which took almost a year post effectiveness. In addition, findings from the INT investigation resulted in temporary expenditure suspensions and added level of scrutiny that protracted procurement processes. All these subsequently culminated to needing to extend the Project three times to ensure that the credit was fully utilized and PDO objectives attained. Despite these, the Project exceeded all PDO indicators and overall Project disbursement reached 95% by Project closure on March 15, 2015.

3.4 Justification of Overall Outcome Rating 77. Rating: Satisfactory. The overall rating is based on the following sub-ratings: (i) Relevance of the objectives and design/implementation – Substantial; (ii) Efficacy in Achieving the PDOs – Substantial; (iii) Efficiency – Substantial.

3.5 Overarching Themes, Other Outcomes and Impacts

23

78. Poverty Impacts, Gender Aspects, and Social Development. The Project contributed to delivering life-saving interventions and treatment to children < 5 years of age and pregnant women, groups who were most at-risk of suffering complications due to malaria. In addition, the comprehensive approach of the MPP meant that the population reached by the Project also benefitted from other health outreach programs, such as receiving immunization and deworming for children and general health promotion education for pregnant women. The success of the Project likely eased the burden of the government due to the economic cost of malaria treatment and lost economic activity. Given that the Project also played a role the net distribution in 22 non-Project Sates, its impact goes beyond what had been originally planned. 79. Institutional Change/Strengthening. The Project helped Nigeria build its government capacity and develop solid foundation for implementing a comprehensive health initiative, as demonstrated in how the country met and surpassed indicators mentioned in the previous sections. The decision to incorporate staff from other parts of the MOH into the PIUs strengthened the MOH’s overall human resource capacity by increasing the skills set of many of its staff who had been seconded to the PIUs. With the project now closed, key staff who return to old posts within the MOH can now provide technical expert support to other parts of the MOH. In addition, critical infrastructure, such as a functioning M&E, has been put in place. The staff experience and improved infrastructure are anticipated to serve the national public/private partnership on malaria control, which has been presented to the President for approval. They also help contribute tremendously to the Banks ongoing SOML Initiative. In areas where additional support may still be needed, the NCMP has since 2007 gained experience in establish deep connections within a broad partnership on malaria control, including the GFATM, USAID-PMI, and DFID. 80. Other Unintended Outcomes and Impacts (positive or negative)

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops NA

4. Assessment of Risk to Development Outcome 81. Rating: Moderate. Nigeria remains committed with continuing with the progress made under this Project. The current 2014-2020 NMSP emphasized ongoing universal access to prevention and treatment through broad partnerships, including community organizations and the private sector. Nigeria continues to work with global partners who are assisting with enhanced capacity of the government. Nigeria is in the process of developing a public/private partnership on malaria control, which aims to sustain the gains made under this Project. Moreover, the country continues to be engaged with the World Bank through the SOML initiative priority interventions that include malaria. Although the Federal and State governments had contributed to the Project through its own budget in various ways, the country now suffers from a fiscal crunch due to dampened oil prices. Also, any proposed initiative to undertake new projects needs still needs to be sustained through partner commitments.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 82. Rating: Moderately Satisfactory. The Bank designed a project aligned with Nigeria’s 2006-2010 Country Strategic Plan by focusing on a comprehensive, multisector approach. This was clearly expressed in the PDOs given the emphasis on MPP and coordination between the Federal and State governments. The design also

24

considered the environmental impact of the Project by incorporating an IVMP as well as addressing healthcare wastes as part of a medical waste management plan. Furthermore, the Bank took advantage of existing structures and government personnel from which to build the Project, such as using FMOH staff to fill positions within the MCBP. This, along with hiring technical experts as agents who could assist the project implementation, helped with quickly setting up the Project at the ground level. 83. The Bank coordinated with the government, other development partners, private sector, and Non-Governmental Organizations (NGOs) regarding the scope of the work. For example, joint FMOF, FMOH, and Bank missions conducted assessments to evaluate implementation capacity in the States. And while the Bank team could not have foreseen the shift in the country’s priority towards universal access to malaria interventions or the need to urgently provide support to development partners, such as distributing LLINs purchased by the GFATM to 22 non-Project States, the Project’s emphasis on a flexible design that integrated collaboration with development partners allowed the Bank team to quickly and successfully respond to the changing needs. 84. Although the reviews of the States’ capacities called for joint missions with the Federal level, the process could have benefitted from more involvement and input from the States. As such, the States disputed the findings as they related to the need for PIFs, which stalled the Project for several months due to the delay in releasing funds to States to finance operational costs. This resulted in the need to undertake a restructuring three years into the Project before PIFs were finally hired in all States and in place to provide much needed implementation support. In addition, the RF could have benefitted from a more balanced set of PDO indicators, including those that measured progress towards achieving PDO II. A formal Quality at Entry Assessment was not conducted for this Project. (b) Quality of Supervision 85. Rating: Satisfactory. The Project’s Task Team Leader changed three times over the life of the Project. However, each one of the Team Leaders had experience in managing Bank-funded development projects. Also, there was no indication that these transitions had any negative effect on the supervision of the Project. Throughout the life of the project, the Bank task team remained proactive with its supervision and implementation support, exhibited flexibility in the level and type of support it engaged in, and showed willingness to work with partners and donor agencies. A review of the Bank’s supervision budget (Annex 4) indicates that the Bank team remained engaged in the Project, allocating 405 staff weeks and US$1,530,437 between FY08 to FY15 to supervise project implementation that amounted to US$191,305 per year. 86. As evident from the Implementation Status Reports (ISRs), the team was candid in highlighting potential obstacles and offering solutions on how they could be resolved, including hiring consultants to ensure on ground follow up support. This was enhanced through quarterly, focused support from the Bank’s Headquarters, as well as tripartite meetings that involved relevant units within the Bank, NMEP, and FMOF to discuss project progress and implementation challenges. All missions involved both Federal and State reviews. And as needed, the project engaged in State visits to directly oversee implementation issues. As an example, the mission conducted a managers’ planning workshop in three States in 2012 to discuss program implementation progress and prioritization for the remainder of the project. Moreover, the Bank was responsive to the FGON’s five requests for restructuring that were approved within months from receiving the written requests from the FGON. The AF allowed the FGON to scale up interventions and accommodate a new universal access strategy. The team quickly acted as well on an urgent request to reallocate funds to finance the cost of distributing LLINs to 22 States outside of the original seven States selected for the Project. The task team also conducted separate meetings with partners including with DFID’s Support to National Malaria Programme (SuNMaP), UNICEF, USAID, WHO, and SFH to discuss issues of mutual collaboration in project activities. 87. The task team provided high-level advocacy and support. In cases where the PIU team underwent reorganization as a result of the INT findings, the mission helped facilitate a smooth transition by coordinating

25

with incumbent PIU teams, as well as providing necessary terms of references to reaffirm Project arrangements and to guide in the recruitment of new team members. The mission also provided training to new officers at the Federal and State levels who came on board in the aftermath of the INT investigations. Based on the INT report, the task team fully engaged other units in the Bank early on to develop an action plan to mitigate the weaknesses highlighted in the report. In addition, the CMU played an active role in adeptly conveying the INT findings and recommended corrective measures to the clients, and in keeping the Country and Sector Directors abreast with updates. The Project Team, Federal and State PIUs, and INT worked in concert, which led various client constituencies to own the process. Thus, despite the INT, the task team was not deterred, and worked on increasing its vigilance and control on all high value procurement packages. The Bank’s response to the INT findings not only mitigated risks but enhanced/improved/strengthened project controls and oversight management/arrangements among various internal and external stakeholders. (c) Justification of Rating for Overall Bank Performance 88. Rating: Satisfactory. The overall rating is based on the following sub-ratings: (i) Bank Performance at Ensuring Quality at Entry – Moderately Satisfactory and (ii) Quality of Supervision – Satisfactory.

5.2 Borrower Performance (a) Government Performance 89. Rating: Moderately Satisfactory: The FGON continued to show its commitment to malaria control evident in the development of the 2009-2013 NMSP, and a subsequent 2014-2020 NMSP in which the Federal and State governments committed to covering 75% of the overall financing for the malaria program by 2015. Along with these, the FGON request for a US$100 million AF to undertake enhanced activities and a reallocation of US$22 million to distribute LLINs demonstrated the government’s choice to take on additional IDA financing to scale up interventions. On the ground, the government displayed its mettle by coordinating with partners to successfully provide over 60 million nets within a short period, a massive scale up that had never been done anywhere else in the world. Its success hinged on the capacity of States to undertake proper micro-planning of net distribution, which facilitated in procuring millions of LLINs by through the Federal Government and their eventual distribution by the States through state wide net distribution campaigns and routinely at antenatal clinics. Despite this, the Project experienced several challenges that stalled the Project. For example, the Program underwent two changes in leadership in a span of 8 months, including the Program Coordinator. The changes slowed down implementation, even as new management was put in place due to the time required to settle into and learn the Program obligations. In addition, the procurement capacity and poor financial internal control and low regulatory compliance posed a challenge four years into the Project. Procurement at the Federal and State levels remained major challenges throughout the Project, creating major implications for the availability of high value commodities such ACTs and RDTs at health facilities. Moreover, there were instances where extant guidelines for critical interventions were not developed or disseminated to the States, LGAs, and service delivery points, resulting in deficit of much needed skills. Also, the Federal government could have better supported States and LGAs with consistent and adequate supervision and training. Moreover, the Borrower itself noted the high attrition rates of the few available trained staff. 90. The INT review revealed significant weaknesses in procurement and financial management at the Federal and State levels. Throughout the INT investigation, however, the FGON should be commended for working collaboratively with the Bank’s Project Team and INT Unit to resolve the fraud and corruption identified in the INT report. The FGON took disciplinary actions on indicated staff and reconstituted the PIUs and PFMs. In addition, it ensured that the Program adhered strictly to the recommendations for remedial measures. By the closing of the Project, supervisory and monitoring functions of all program interventions were being done more regularly with the NMCP taking the lead.

26

(b) Justification of Rating for Overall Borrower Performance 91. Rating: Moderately Satisfactory. The overall rating is based on a Moderately Satisfactory rating for Government Performance.

6. Lessons Learned 92. Flexibility and ability to respond to changing environment. Given the pace of research and development of new knowledge about diseases and their management, Projects that assist countries in disease control must be capable of responding to innovative and current changes to strategies. In this case, the Project quickly and promptly underwent additional financing to assist Nigeria in enhancing its activities in support of the NMCP and in a reallocation of funds in order to provide universal access to LLINs. This allowed the country to be at the forefront of managing the malaria. 93. Utilizing a comprehensive Malaria Plus approach. Although most malaria projects are single disease vertical projects, the MCBP employed a Malaria Plus approach that situated malaria within the overall maternal and child health landscape. This clearly pushed the Bank and the government at both levels to engage broadly and seek ways to strengthen the delivery of a comprehensive package of services, moving beyond the immediate boundaries of just malaria. 94. Extensive use of surveys to understand implementation performance. The LQAS survey, HFA survey, and malaria indicator surveys were strong and credible survey methods that provided implementation information for timely management decision making. Incorporating these into the Project’s monitoring system was key in driving the wider Bank health portfolio to invest in M&E measures for other follow on Projects like the SOML. 95. Enhanced coordination between the Federal Government and the States. Because of the distinct roles that the Federal and the States had in ensuring the Project’s success, it was critical to ensure a coordinated effort and unity between the two. Among other things for example, greater involvement and feedback from the States at the beginning might have resulted in more rapidly and smoothly incorporating PIFs into the Project. This would have prevented delays and boosted implementation from the outset as had been anticipated in the design. Also, the MCBP managed the lack of human resource and technical capacity at both levels of government, including across different States, by seconding Federal staff to the State PIUs and with contracts to experts who could quickly bring in skills at both levels and provide institutional training. 96. Addressing INT reviews proactively and collaboratively. Taking actions early to address Project risks provide the opportunity to engage clients and relevant partners proactively and collaboratively. Unlike other projects that would have normally suspended implementation, the Bank’s Project team consulted early with the client and various units within the Bank on developing self-corrective measures. The quick action showed the Bank’s and the client’s seriousness about dealing with problem. Establishing remediation steps reduced the negative impact of the INT findings, while continuing to implement the Project and providing benefits to the target population. 97. Mitigating fiduciary risks in countries with non-existent or weak procurement systems. While the Bank normally undertakes a procurement assessment during Project preparation, this has often focused on large-scale procurements. The experience from this Project illustrated that the Bank needs to also needs to be alert to Projects with numerous small scale procurements that all add up to a big amount. Potential mitigating actions include i) limiting travel/operating expenses and other small scale procurement, ii) contracting out training components and employing performance based awards, iii) utilizing an e-procurement system that could help

27

monitor and flag questionable activities; and iv) incorporating an indicator in the RF that assessed fiduciary compliance vis-à-vis procurement. 98. Benefits of global partnerships. The Bank’s close association with other development partners helped establish the groundwork for working together and created a more harmonized effort in controlling malaria. Because of this relationship, partners shared information with each other relevant to malaria control, which fed into distinct projects within the partnership. Such a relationship benefitted Nigeria as when Bank stepped in to assist the GFATM with LLIN distribution. More significantly, this information sharing was the basis that alerted the Bank to the risks identified by GFATM’s Office of Inspector General, which led to the INT review. 99. Involvement of an inter-religious organization in a religiously diverse setting. The involvement of NIFAA contributed to increased BCC through mosques and churches that facilitated close community level interpersonal messaging. This helped create an inter-denominational harmony. In addition, NIFAA’s efforts not only provided an avenue to strengthen behavior change strategies for malaria, but also for other communicable diseases and MPP related health interventions.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies. See Annex 7 (b) Cofinanciers (c) Other partners and stakeholders

28

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Strengthening leadership and coordination capacity of the federal government to control malaria, original Parent Project

86.50 86.50 100%

Enhanced Health Systems Strengthening through Additional Financing

25.40 25.4 100%

Enhanced commodity procurement through Additional Financing

37.50 37.5 100%

Strengthening the capacity of SMOHs and LGAs to deliver MPP interventions at state and community levels in Project States, original Parent Project

71.50 71.50 100%

Enhanced Community System Strengthening through Additional Financing

7.00 7.0 100%

Enhanced communication and outreach through Additional Financing

10.00 10.0 100%

Total Baseline Cost 237.90 237.90 100% Physical Contingencies Through original Parent Project Through Additional Financing

22.00 20.10

22.00 16.23

100%

80.75%

Price Contingencies

0.00

0.00

0.00 Total Project Costs 280.00 276.13 98.62%

Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00

Total Financing Required 280.00 276.13 98.62%

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 80.00 80.00 100% International Development Association (IDA)

280.00 276.13 98.62%

29

Annex 2. Outputs by Component Component 1: Strengthening leadership and coordination capacity of the federal government to control malaria. To support the ability of the Federal Ministry of Health (FMOH) staff to undertake these essential functions, the project supported and strengthened: Procurement and logistics of commodities for accelerated implementation of the malaria plus interventions. ACTs, RDTs, LLINs were procured as part of the original Parent Project component as well as for the Additional Financing. Through Project support, 18.4 million kits of RDTs were procured and distributed, resulting in 31.4% of facilities in the Project States having RDTs, compared to 20.4% in Comparison States. The Project embarked on training for all malaria commodities and logistics staff and supported the strengthening of state medical warehouses across the six states. The strengthening improved commodity availability, quantification and reorder processes. Capacity building was conducted with support from USAID Deliver project. Commodity logisticians and pharmacists were recruited. This improved the feedback on commodity quantification and drug availability in facilities. In addition, US$22.5 million secured under the Additional Financing was used to support LLIN distribution campaigns in 22 non Project States for LLINN procured by the Global Fund. Moreover, the project supported military hospitals with ACT as part of the federal responsibility for malaria control. National Monitoring and Evaluation for evidence-based management. Both the Federal and State level benefitted from establishing robust M&E units that supported monitoring of implementation and real time reporting of malaria data at both Federal and State level. At project development, none of the Project States were using M&E data for managing malaria programs. However, by end of project, all the Project States were using M&E data to plan for their malaria interventions and were submitting M&E data on a quarterly basis to the federal level. The national level also developed a core group of M&E experts that developed the capacity to conduct LQAS without much external assistance. As a result, the national team led the process in conducting country malaria based surveys such as Malaria Indicators surveys and guided the development of modules on malaria for demographic health surveys. By the Project closing date, the Project successfully implemented and completed an end-stage survey and a Health Facility Assessment survey. Through the Additional Financing to enhance Health System Strengthening, the project was able to support institutional and human resource capacity building and the establishment of integrated logistics management system for HIV, TB and malaria through contracting to a Logistics implementing agent. Coordinating program activities at the national and state level and across programs. The strengthened national level not only provided coordination across the Project states but across the entire country. At the State level, the PIUs were strengthened over time to perform similar functions including supportive supervision. The enhanced capacity contributed to the outcomes of interest – the PDO and intermediate indicators and the following: 1. Strengthening of advocacy and utilization of malaria interventions: Federal and State supported BCC

including the presence of NIFAA and other partners. Through the Additional Financing that focused on enhanced activities, BCC and advocacy were prioritized at both Federal and State levels, through NIFAA and supported by State officers in each of the Project States.

2. Coordination of Maternal, Neonatal and Child Health weeks: Both the Federal and States contributed to the States success in the implementation of MNCH weeks which were used as a major part of driving the Plus component of the malaria plus package. In particular, the distribution of Vitamin A to children under five was essential to achieving a PDO indicator. The last four rounds conducted across the Project States resulted in 15,065,138 children under five receiving Vitamin A, thus exceeding the target by 251%. The table below for example shows the number of children provided with Vitamin A during the MNCCH weeks and how the Project contributed to the control of intestinal helminthes as part of the broader MPP strategy:

30

Data summary for MNCH weeks rounds of November 2013 and May 2014 Intervention and Indicator November

2013 May 2014 Comment

Target population for Vit. A in the 6 Total Booster States 5,667,937 5,841,789 Total no. of children under 5 years administered Vit. A in the 6 Booster States

5,012,321 5,277,948

Coverage % for Vit. A in the 6 Booster States 88.4 90.3 Target more than attained Target population for deworming in the 6 Booster States 4,784,219 4,008,781 Total no. of children under 5 years administered deworming tablets in the 6 Booster

2,201,826 1,590,445

Coverage % for deworming tablets in the 6 Booster States

46.0 39.6 Project contributing to malaria Plus interventions.

3. Increased supportive supervision from both Federal and State levels: There was increased support from the

Federal and States for supervision and monitoring, which was now being conducted more regularly on a quarterly basis.

4. Strengthened Administrative units to manage program implementation: Both Federal and States had Project implementation Units (PIU) with staff seconded by MOH as well as specifically recruited by the programs to enhance implementation. PIUs were established and staff were seconded from Ministry of Health or specifically recruited by the project. This allowed the creation of a robust team that was able to meet the required demand of the malaria program. The creation of effective and functional PIUs at State level was seen as best practice and has been copied by Global Fund.

5. Strengthened coordination and management of partnerships for malaria control at both Federal and State level. By project Closure, the Federal and States programs were able to harness the potential of partners to contribute towards the implementation of both Federal and state malaria strategic plans.

6. The Additional Financing allocated for Health Systems strengthening supported the scale up of malaria laboratory diagnosis and improved the quality assurance for malaria diagnosis. The Project provided capacity building and supervision for malaria laboratory in the Project States, as well as engaged State MoH on planning for scaling up diagnostic capacities through increasing the number of facilities able to conduct microscopy and RDTs as described by the NMCP malaria laboratory diagnosis policy. This facilitated in the development /review of tools (guidelines, protocols, and implementing mechanisms) that supported the scaling up for both malaria microscopy and RDTs and malaria laboratory quality assurance.

7. The monitoring of drug and insecticide resistance also improved as an outcome of the Additional Financing. The project was able to support the strengthening of sixteen existing sentinel sites and upgrading of three new sites as well as provide institutional and human capacity building including operational enhancement and establishment of quality assurance mechanisms.

Component 2: Strengthening the capacity of SMOHs and LGAs to deliver MPP interventions at state and community levels in Project States. The strengthening of the PIUs supported the expansion of the State Ministry of Health (SMOH) and the Local Government Authorities (LGAs) in the target states capacities to rapidly expand the Malaria Plus services at both the facility and the community levels. The outcomes of interventions are described below: LLIN Coverage/Access. More than 24.8 Million LLINs were procured and distributed through state wide net distribution campaigns and routinely during antenatal clinics. In order to meet the request of Government, the Project used more than US$22 million of the credit to support distribution of LLINs in 22 states in collaboration with the Global Fund for AIDS, TB and Malaria (GFATM). The results from the end stage LQAS 2015 survey show that 99.6% of the houses had at least one Insecticide Treated Net (ITN). The Project exhibited an upward trend in ownership of ITN, from 2.6% in 2006, then 81.3% during midterm evaluation in 2010, and ending at

31

99.6% by the end term review in 2015. This represented an 18.3% increase between 2010 and 2015. The Project exceeded the target for LLIN coverage/access. LLIN Utilization: Results from the ends stage LQAS 2015 revealed that the percentage of children under five who slept under ITN during the previous night in Project States was 74.4%. This represented a 27.8% increase from the 2010 LQAS which showed 47.5% utilization. Comparison States, on the other hand, experienced a decrease in utilization from 37.1% in 2010 to 36.2% in 2015. Even though net coverage was similar, the fact that the Project States had a systematic program support for BCC messaging through NIFAA might have helped increase the uptake in utilization. This same impact from BCC could also explain the increased LLIN utilization in pregnant women in Project States, which showed the percentage of women sleeping under and LLIN the previous night increasing from 40.4% in 2010 to 69.3% in 2015, a 28.9% upswing. The change in LLIN utilization among pregnant women in Comparison States was less dramatic with an increase from 39.0% in 2010 to 47.9% in 2015, an increase of 8.9%. The target for LLIN coverage has been exceeded. Use of ACTs: The utilization of Artemisinin Combination Therapy (ACT) by children under five reflected a change in policy that took place during Project implementation. The new policy required that ACT be given only to children that had positive confirmation of malaria infection as indicated by Rapid Diagnostic Tests (RDTs) or microscopy; the mere presence of fever did not qualify children to receive ACT as was initially designed at the beginning of the Project. Despite the change during the implementation phase, the end stage LQAS 2015 illustrated an increase in ACT utilization, from 3.7% in 2006, 5.4% in 2010 and to 45.0% in 2015. In addition to the change in policy for who qualifies to receive ACT, the target for the percentage of children who received ACT was revised downward from 80% to 15% during a Project restructuring in 2010. Given the available evidence that came to light at the time, the need to revise the target became apparent since i) no program had been able to attain more that 20% ACT utilization among children under five years; ii) the global cost of ACTs remained very high; and iii) the availability of ACT continued to be a challenge globally. At a time when the Affordable Medicines Facility-malaria (AMFm), a facility for subsidized malaria ACTs, had not yet been established, the Project team considered the original target too ambitious based on these existing barriers. Secondly, the shift in focus to treating only confirmed malaria greatly improved rational ACT utilization and reduced the actual ACTs needed as a result of cutting unwarranted treatment of children solely based on the presence of fever. In the long term, the focused treatment and limited use of ACT were anticipated to contribute to reducing artemisinin-resistant malaria parasites. The end line LQAS survey showed that the target for utilization of ACT in children with fever within 24 hours has been exceeded. Diagnostic treatment: Through the Project, more than 15 million RDTs were procured, which facilitated the capacity of Project State to respond and comply with the new policy of treatment of only confirmed cases. In addition, Project funds helped in procuring microscopes, and in turn strengthening the skills of laboratory technicians in microscope diagnosis and other tropical diseases. By 2015, 27.9% of children under five received RDTs prior to treatment of fever, a stark jump from the zero level at project development. In addition, the 31.4% of the health facilities in Project States had RDTs compared to 20.4% in Comparison States. This highlighted how the Project States enhanced capacity for evidence based policy implementation. Behavior communication: The high level of behavior communication messages and activities carried out during the MNCH campaign period supported and contributed to scaling up of the Malaria Plus Package of interventions. Results from both May and November 2014 MNCH week campaigns exhibited increased uptake of both Vitamin A (greater than 80%) and deworming (40%). Through the Additional Financing, the Project engaged the Nigeria Interfaith Action Association (NIFAA), strengthened the IEC/ BCC mechanism through use of interfaith action as well as interpersonal communication at community level. Community System Strengthening: The project brought on board community level and facility level initiatives. The support of community interventions included strengthening the Role Model Mother approach, which later transformed into a community health worker. This intervention was delivered using the Community

32

Directed Interventions (CDI) approach derived from the past experience of treating onchocercaiasis under the Oncho Eradication Program. As part of the Additional Financing in 2009, the Project assessed demand side interventions through CDIs and the private sector by way of Patent Medicine Vendors (PMVs) in rural and peri-urban areas of two states (Anambra and Gombe). The CDIs expanded the delivery of a comprehensive package of malaria control interventions, including training, educational materials, and commodities (LLINs and ACTs.). The level of PMVs’ contribution to malaria control at the community level, as measured by the proportion of PMVs that can identify primary methods of prevention and treatment, increased from 40% (MIS 2010) to 42% (baseline IE 2013) to 60% (end stage IE 2015).

33

Annex 3. Economic and Financial Analysis Economic analysis takes for granted that the project is technically sound and the institutional arrangement is effective at project implementation. An economic analysis leaves no doubt whether or not the project contributes to the country’s welfare. It defines the objective(s) of the projects: improving the health status of Nigerian population; looking at alternative ways of reducing morbidity, and prolonging lives of target groups (women and children). The analysis asks who are the winners and losers. Who received the benefits and who bore the costs? Were the target groups adequately attended to and deaths averted for pregnant women and children under 5 years? These are critical situations analyzed using different models. A good project contributes to the country’s economic output and has potential to make everyone better off. Two methods of health sector valuation were applied for this analysis. The first method used DALYs as a primary health indicator to evaluate the efficiency and outcome of the Project. One DALY represented one lost year of “healthy” life. The sum of DALYs across a population, or the burden of the disease, measured the gap between current health status and an ideal health situation where the entire population lived to an advanced age, free of disease and disability. Calculation and Assumptions:

DALY = YLL + YLD…………………………………………………………………............. (1) DAYLaverted = DALYbaseline – DALYintervention…………………………………..……….…....... (2)

YLL = years of life lost due to premature death, K = age-weighting modulation constant (e.g. K=1), or not specified (K=0), C = adjustment constant for age weights (GBD standard value is 0.1658), r = discount rate (GBD standard value is 0.03), a = age of death (years) β = age-weighting constant (GBD standard value is 0.04), L = standard life expectancy at age of death (years), I=number of incident cases in the reference year, DW= is the disability weight. Average life expectancy in Nigeria is 54 GDP/Capita = 2970, Maternal age category (15-49), Child age category (< 5years)

The second employed a cost effectiveness analysis (cost benefit ratio) that entailed (i) comparing the baseline scenario before the Project intervention with the cost benefits of the Project after discounting and (ii) calculating the cash flows, net present values (NPV) and the internal rate of return (IRR).

YLL NCe(ra) / (B r)2 e(Br ) (b r)(L a)1 e(Br )a (B r)a1 ..............................(3)

)4(..........

)1)(/)(1(

1)(1))(()/( )())((2)(

rL

arBaLrBra

erLK

arBeaLrBerBKCeIDWYLD

34

Method One: Between 2005 and 2010, DALYs due to malaria reduced from 26,020,035 DALYs to 25,002,226 DALYs (source: Global Burden of Disease Study). This represented a little more than 1 million DALYs (1,017,814) per year, or 18,518 lives saved per year (with the greater reduction among children under 5 years of age at about 783 thousand DALYS reduction per year, 14,500 lives saved per year) through all of the interventions (child immunization, provision of LLINs/ITNs insecticide nets, antenatal cares and outpatient care in the entire population (Table 3.1). Table 3.1. Nigeria – DALYs due to Malaria (2005-2010) 2005 2010

<1m 1112520 852314

1 to 11 m 7423870 7227590

1 to 4 12929400 12603100

5 to 9 797527 874171

10 to 14 403325 380618

15 to 19 458971 405047

20 to 24 572165 511889

25 to 29 458861 432826

30 to 34 317578 306216

35 to 39 223866 209317

40 to 44 216866 189097

45 to 49 196976 173977

50 to 54 189779 174119

55 to 59 171625 161643

60 to 64 183781 162070

65 to 69 162877 147814

70 to 74 122271 116262

75 more 75772 72146

Total 26020035 25002226

The vulnerable groups (women 15-49 years and children under 5 years) were the winners especially the children under 5 years showing exponential improvement in the health results in terms of the disease burden, deaths averted or lives saved with the interventions. Total DALYs averted of 1,970,092 was more than three times estimated at baseline (Table 3.2).

35

Table 3.2. Nigeria– DALYs due to Malaria By sex (Maternal and Child) at end of Project in 2015 2005 2010 2013

Child < 5 years DALYs 22,108,007 24,917,007 20,235,931

Total 22,108,007 24,917,260 20,235,931

Discounted Child 2,809,253 <4,681,329>

Females DALYs

15 to 19 96092 90491 73,438

20 to 24 109926 106327 87,455

25 to 29 95229 95159 78,278

30 to 34 68932 70493 59,058

35 to 39 50276 49098 40,378

40 to 44 38313 35734 29,772

45 to 49 31053 29232 23,426

Total Maternal DALYs 489,821 476534 391,805

Discounted Maternal DALYs <13287> <84729>

Total discounted Maternal and Child DALYs 2,795,966 <4,766,058>

Total DALYs averted Maternal & Child by end of Project 2015 <1,970,092>

Nigeria per capita GDP = USD 2970

Monetary Value of lives saved by end of Project 2015 USD 5,851,173,240

36

Method Two: Table 3.3. Summary Results of the Economic Analysis Indicators Benefit Valuation Methods and Discount Rate Assumptions

GNI Per Capita GDP Per capita 6% discount

rate 12% discount rate 6% discount

rate 12% discount rate

Present Value of benefits (billion US$)

3.086 1.686 3.892 2.12

Present value of cots (Million US$) 192.2 142.2 192.2 142.2 Net Present Value (NPV, Billion US$) 2.89 1.54 3.7 1.982 IRR (%) 65% 65% 70.5% 70.5% C/B Ratio 0.06 0.08 0.05 0.07

Key assumptions:

1) Discount rate assumptions:

6% or the new bank wide directive 12% the discount rate used to be applied in the bank 2) Valuation of DALYs:

GNI per capita for 2014=2970 US$ GDP per capita projected to 2017 using 5.04% annual growth rate Cost distribution: For parent project the costs were distributed using the disbursement curve For additional financing the total actual cost was distributed equally over the 7 year implementation period (2009 to 2015) Scenarios: Table 3.4 (figure 1 in the excel model) – Valuation of DALYs using the projected GDP per capita and discount rate of 6% Table 3.7 (figure 4 in the excel model) – Valuation of DALYs using the projected GDP per capita and discount rate of 12% Table 3.5 (figure 2 in the excel model) – Valuation of DALYs using GNI per capita and discount rate of 6% Table 3.6 (figure 3 in the excel model) – Valuation of DALYs using GNI per capita and discount rate of 12%

37

Table 3.4. Valuation of DALYs using the projected GDP per capita and discount rate of 6%

Costs (US$) Benefits

Year Parent project Additional Financing Total costs

Total DALYs

GDP per Capita Total Benefits Net Benefits

2007 0 0 0 0 1977 0 0

2008 12743329.79 0 12743329.79 0 2083 0 -12743329.79

2009 9862116.7 12,571,428.60 22433545.3 0 2196 0 -22433545.3

2010 44297349.6 12571428.6 56868778.2 0 2315 0 -56868778.2

2011 20282654 12571428.6 32854082.6 0 2514 0 -32854082.6

2012 13335615 12571428.6 25907043.6 0 2740 0 -25907043.6

2013 21148934 12571428.6 33720362.6 0 2980 0 -33720362.6

2014 52822788 12571428.6 65394216.6 0 3203 0 -65394216.6

2015 5597178 12571428.6 18168606.6 0 3376 0 -18168606.6

2016 0 0 0 0 3559 0 0

2017 0 0 0 1970092 3751 7389815092 7389815092

NPV $192,245,557.27 $3,892,862,405.42 $3,700,616,848.15

IRR 71%

C/B $0.05

Key assumptions

2010 2011 2012 2013

GDP per capita 2315 2514.1 2739.9 2979.8

Total averted thru course of project for 10 years is 1,970,092

GDP growth Rate 5.40%

38

Table 3.5. Valuation of DALYs using the projected GDP per capita and discount rate of 12%

Costs (US$) Benefits


Total DALYs


2007 0 0 0 0 1977 0 0

2008 12743329.79 0 12743329.79 0 2083 0 -12743329.79

2009 9862116.7 12,571,428.60 22433545.3 0 2196 0 -22433545.3

2010 44297349.6 12571428.6 56868778.2 0 2315 0 -56868778.2

2011 20282654 12571428.6 32854082.6 0 2514 0 -32854082.6

2012 13335615 12571428.6 25907043.6 0 2740 0 -25907043.6

2013 21148934 12571428.6 33720362.6 0 2980 0 -33720362.6

2014 52822788 12571428.6 65394216.6 0 3203 0 -65394216.6

2015 5597178 12571428.6 18168606.6 0 3376 0 -18168606.6

2016 0 0 0 0 3559 0 0

2017 0 0 0 1970092 3751 7389815092 7389815092

NPV $142,252,187.43 $2,124,395,252.66 $1,982,143,065.23

IRR 71%

C/B $0.07

Key assumptions

2010 2011 2012 2013

GDP per capita 2315 2514.1 2739.9 2979.8



39

Table 3.6. Valuation of DALYs using GNI per capita and discount rate of 6%

Costs (US$) Benefits (US$)


Total DALYs


2007 0 0 0 0 2970 0 0

2008 12743329.79 0 12743329.79 0 2970 0 -12743329.79

2009 9862116.7 12,571,428.60 22433545.3 0 2970 0 -22433545.3

2010 44297349.6 12571428.6 56868778.2 0 2970 0 -56868778.2

2011 20282654 12571428.6 32854082.6 0 2970 0 -32854082.6

2012 13335615 12571428.6 25907043.6 0 2970 0 -25907043.6

2013 21148934 12571428.6 33720362.6 0 2970 0 -33720362.6

2014 52822788 12571428.6 65394216.6 0 2970 0 -65394216.6

2015 5597178 12571428.6 18168606.6 0 2970 0 -18168606.6

2016 0 0 0 0 2970 0 0

2017 0 0 0 1970092 2970 5851173240 5851173240

NPV $192,245,557.27 $3,082,325,071.74 $2,890,079,514.47

IRR 65%

C/B $0.06

Key assumptions

2010 2011 2012 2013

GDP per capita 2315 2514.1 2739.9 2979.8



40

Table 3.7. Valuation of DALYs using GNI per capita and discount rate of 12%

Costs (US$) Benefits (US$)


Total DALYs


2007 0 0 0 0 2970 0 0

2008 12743329.79 0 12743329.79 0 2970 0 -12743329.79

2009 9862116.7 12,571,428.60 22433545.3 0 2970 0 -22433545.3

2010 44297349.6 12571428.6 56868778.2 0 2970 0 -56868778.2

2011 20282654 12571428.6 32854082.6 0 2970 0 -32854082.6

2012 13335615 12571428.6 25907043.6 0 2970 0 -25907043.6

2013 21148934 12571428.6 33720362.6 0 2970 0 -33720362.6

2014 52822788 12571428.6 65394216.6 0 2970 0 -65394216.6

2015 5597178 12571428.6 18168606.6 0 2970 0 -18168606.6

2016 0 0 0 0 2970 0 0

2017 0 0 0 1970092 2970 5851173240 5851173240

NPV $142,252,187.43 $1,682,072,487.44 $1,539,820,300.01

IRR 65%

C/B $0.08

Key assumptions

2010 2011 2012 2013

GDP per capita 2315 2514.1 2739.9 2979.8



41

Table 3.8. Estimated Number of Beneficiaries in Malaria Control Booster States

Population Category Analysis of Beneficiaries Total Target Beneficiaries* AKWA IBOM

5 million Child<5yrs-20% 1,000,000 1,000,000Women of child bearing age-15% 750,000 Pregnant women-5% of women of child bearing age 37,500 37,500

ANAMBRA

4.8 million Child<5years of age-20% 960,000 960,000Women of childbearing age-15% 720,000 36,000Pregnant women-5% of women child bearing age 36,000 36,000

BAUCHI

5.5 million Child<5yrs-20% 1,100,000 1,100,000Women of child bearing age-15% 825,000 Pregnant women-5% 41,250 41,250

GOMBE

2.8 million Child< 5years-20% 560,000 560,000Women of child-bearing age-15% 420,000 Pregnant women-5% 21,000 21,000

JIGAWA 5 million Child< 5years -20% 1,000,000 1,000,000 Women of child bearing age-15% 750,000 Pregnant women-5% 37,500 37,500KANO

11million Child<5 years -20% 2, 200,000 2,200,000Women of child-bearing age-15% 1,650,000 Pregnant women-5% women of childbearing age 82,500 82,500

RIVERS

6 million Child<5 years-20% 1,200,000 1,200,000Women of child-bearing age-15% 900,000 Pregnant women 5% of women of childbearing age 45,000 42,500

TOTAL 40.1 million (Total)

8,318,250

29.1 million (w/o Kano) 6,035,750 *Target beneficiaries: Large estimate of target beneficiaries about 15 million benefited from project.

42

Conclusion

The analysis clearly shows the Internal Rate of Return (IRR) and the Net Present Value (NPV) are both very high indicating that the Project was economically viable. The calculated Cost Benefit ratio is also very low meaning that the project was highly cost effective or cost efficient. The vulnerable groups (women 15-49 years and children under 5 years) are the winners especially the children under 5 years showing exponential improvement in the health results in terms of the disease burden, deaths averted or lives saved with the interventions. Total DALYs averted of 1,970,092 was more than three times estimated at baseline. The foregoing analysis therefore shows the Malaria Control Booster Project was good for the people and beneficial to the country and in most part achieved its objectives. The children under five gained the most positive results with the various interventions. The cost effectiveness of the project and the benefits on the target population make the case for continued intervention after the project phase and for future sustainability.

43

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Richard Allan Consultant AFTH3 - HIS Consultant Bayo Awosemusi Lead Procurement Specialist GGODR Procurement Suprotik Basu Public Health Specialist AFTHE - HIS Public Health Marcello Castillo Consultant Consultant Maria Eugenia Bonilla-Chacin Sr. Economist GHNDR Economy Serigne Omar Fye Consultant GENDR Consultant Ramesh Govindaraj Sr. Health Specialist GHNDR TTL Meri Helleranta Consultant Consultant Eva Jarawan Lead Health Specialist GHNDR Health Hisham Abdo Kahin Counsel LEGAF Counsel Luc Lapointe Procurement Consultant GGODR Procurement Marc Nene Consultant GHNDR Consultant Chukwudi H. Okafor Sr. Social Development Specialist GSUSD Social Development Africa Eshogba Olojoba Lead Environmental Specialist GENDR Environment Adenike Sherifat Oyeyiola Sr. Financial Management Specialist GGODR Financial Management Chau-Ching Shen Sr. Financial Officer LOAG2 Loan Officer Therese Tshimanga Language Team Assistant AFTH3 - HIS Team Assistant Joseph J. Valadez Sr. Monitoring & Evaluation AFTHD - HIS Monitoring & Evaluation Gert Van der Linde Lead Financial Management Specialist GGOPS Financial Management William Vargas Consultant Consultant Liliane Vert Consultant AFTH3 - HIS Consultant

Supervision/ICR Amos Abu Sr. Environmental Specialist GENDR Environment Sunday Achile Acheneje Procurement Specialist AFTPE - HIS Procurement Adewunmi Cosmas Ameer Adekoya Sr. Financial Management Specialist GGODR Financial Management Akinrinmola Oyenuga Akinyele Sr. Financial Management Specialist GGODR Financial Management Mary Asanato-Adiwu Sr. Procurement Specialist GGODR Procurement Bayo Awosemusi Lead Procurement Specialist GGODR Procurement Maria Eugenia Bonilla-Chacin Sr. Economist GHNDR Economy William R Brieger AFTHD - HIS Noel Chisaka Sr. Public Health Spec. GHNDR TTL Boubou Cisse Sr. Human Development Economist AFTEW - HIS Human Development Alain Daudrumez Consultant AFTHD - HIS Consultant Abiodun Elufioye Program Assistant AFCW2 Program Assistant Ramesh Govindaraj Sr. Health Specialist GHNDR TTL Ogo-Oluwa Oluwatoyin Jagha Sr. Operations Officer OPSRE Operations Benjamin P. Loevinsohn Lead Public Health Specialist GHNDR Public Health Ngozi Blessing Obi Malife Program Assistant GENDR Program Assistant Dinesh Nair Sr. Health Specialist GHNDR TTL Ayodeji Oluwole Odutolu Sr. Health Specialist GHNDR Health Specialist Anne U. Okigbo Consultant GHNDR Consultant Adenike Sherifat Oyeyiola Sr. Financial Management Specialist GGODR Financial Management

44

Mehmet Onur Ozlu Sr. Urban Economist GSURR Urban Economy F. Brian Pascual Operations Officer GHNDR ICR Author Jumana N. Qamruddin Sr. Health Specialist GHNDR Health Joseph J. Valadez Sr. Monitoring & Evaluation AFTHD - HIS Monitoring & Evaluation William Vargas HQ Consultant ST GHNDR Consultant Mark Zeydler-Zborowski HQ Consultant ST GSURR Consultant

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY06 23 204.74 FY07 36 232.87

Total: 59 437.61 Supervision/ICR

FY07 0 -0.16 FY08 30 221.47 FY09 58 500.75 FY10 106 655.33 FY11 61 382.50 FY12 43 380.83 FY13 43 375.23 FY14 35 345.02 FY15 28 203.15 FY16 11 50.18

Total: 415 3,114.30

45

Annex 5. Beneficiary Survey Results (if any)

46

Annex 6. Stakeholder Workshop Report and Results (if any)

47

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

NIGERIA MALARIA CONTROL BOOSTER PROJECT 2007-2015.

10 PAGE SUMMARY FOR ICR (ABSTRACTED FROM 31 PAGE REPORT FROM NMCP)

DREFT END OF PROJECT CYCLE REPORT

PREPARED & SUBMITTED BY

Dr. John Onyeokoro. MBBS, MPH. MD/CEO, Healthwatch Resources Ltd.

www.healthwatchresources.org

48

Chapter 4: FINDINGS.

4.1. Strengthened Capacity of the health System to deliver Malaria -Plus Packages in the States.

It was intended that under this component, the project will help build the capacity of the

SMOHs and the LGAs in the target states in planning, implementation, coordination and

supervision of activities aimed at expanding the Malaria Plus Interventions Package.

This capacity strengthening was to be undertaken by the FMOH and partners, and where needed,

contracted management entities, referred to as Project Implementation Facilitators (PIFs), the

selection of which was to be subjected to international competitive bidding. Each state was

supposed to sign an individual service contract with the highest evaluated contractor and the

contract with the PIF will be financed through the project.

The major findings in designated thematic areas were as follows:

4.1.1. PROGRAMME MANAGEMENT.

4.1.1.1 Training:

This featured most prominently in the entire project States, as well as the Federal Level. While

all the states varied on the choice of trainings conducted, there was no instance of pre-training

needs assessment in any of the States.

While Anambra State spent about N138, 592,746.00 training involving about 3,655 persons,

corresponding details like the type of trainings conducted, the duration and cost per head were

still being computed as at the time of report.

Gombe State spent about N45, 406,572.00 on the training of clearly identified persons, with

specific training costs, and location of training sessions both within and outside Nigeria.

Though about 10,755 persons were reportedly trained by the Bauchi State project, there is no

breakdown of who were trained in any designated training exercise, nor the unit cost of such

exercise.

Jigawa State trained over 5,275 persons including hospital and project staff, CDDs, Journalists,

Gunduma Council Directors, Community Volunteers used in IRS, etc. Rivers State grouped the

trainings according to Departments and functional units of the project, as well as the type of

training conducted. However, total or unit training costs were not yet accessed as of now.

4.1.1.1. a. IDENTIFIED TRAINING SCHEDULE- RIVERS STATE:

Training of Secretarial staff on Management of Secretarial Personnel.

Training of admin staff on documentation & proper record keeping

Training of PFMU officers on Flexible Accounting

Advanced Management Course in ASCON

4.1.1.1. b. IDENTIFIED TRAINING SCHEDULE- GOMBE STATE:

49

Acquisition of new skills on procurement of goods, works and services Project Financial Planning & disbursement Training on contemporary issues in project management and experience sharing Financial management and disbursement training Training on Guidance and counselling Project Monitoring & Evaluation Training on Auditing Procurement Process TOT on Pharmacovigilance TOT on FANC Advanced Computer Training Training on strategic planning and management. Training on Prudence and Accountability in Public Finance Management Advanced Training of trainers Training on strategic planning and management. workshop on Procurement management Training on managing people in project Study tour to Kenya on malaria prevention and control Study tour to Zambia on IRS implementation Use of computer for financial management Defensive driving skills Training on management of registry functions. Training on Project Monitoring and Evaluation Computer application for secretarial studies Advance management training course for health officers Computer Record management course Advanced Project Management Course Management Course for Secretaries Human Resource Management Supply Chain Management Training methods and presentation skills Facility Maintenance Project Management Course Advance management Course Training on Performance Management for Project Managers

The spotlight on training details provided by Gombe and Rivers States does not in any way suggest

that the other participating states did not conduct the requisite trainings as at when due. It only

highlights the extra care taken by the states to clearly justify how, and when the trainings were

conducted, where applicable.

Contribution of the project to local resource mobilization and partnerships were also variously

captured by the states. Generally, it served as a catalyst for resource mobilization. In Jigawa State,

the project was accommodated within the premises of the State Government facility; hence no

extra cost was incurred on accommodation/housing.

50

Details of the contributions of the project to partnerships like the MNCH week, routine

immunization, polio eradication, high level advocacy, support to NHMIS, etc. were varied across

the implementing states, and will be highlighted in the final report.

4.1.2. CASE MANAGEMENT.

All participating States demonstrated that the project scaled up diagnostic, therapeutic and

preventive abilities of the recipients in their localities.

The following key observations were made by the Head of Case Management of the Program, Dr. Ntadom:

Vector control with Larvicidals was piloted in areas where was a preponderance of natural

water bodies, like Rivers State.

PPMVs were piloted in Gombe and Anambra States.

Some generalised drug therapeutic efficacy tests were carried out, with support from the Bank. Activities undertaken in this thematic area include the distribution of ACTs, RDTs, LLIN, Vector

Control through IRs, and BCC activities to enhance uptake.

The project brought about an increase in the number of health facilities providing ACTs from 398

in 2007, to 594 in 2014, an increase of 67%. There was also an increase in the scope of indoor

residual spray in Anambra, and Gombe States.

In Jigawa State, the coverage rate in 6 LGAs, involving over 300 structures was reported to be

between 82-92%. Rivers State had an elaborate Vector Control Programme, details of which

included, but not limited to the following:

Baseline data collection in preparation for IRS

Training of spray operators, supervisors team leaders in Ikwerre LGA.

Training of 320 House hold enumerators and 32 supervisors.

Social mapping of Ikwerre & Etche LGA’s by GIS expert

Training of 956 spray operators and 32 supervisors for IRS exercise

Training of 32 waste management’s and 6 coordinator

Sitting of 32 soak pits in the 32 wards of Ikwerre and Etche LGA.( To ensure that effluent

waste from IRS is not discharged into the environment)

Deployment of IRS waste management coordinators during and after IRS

There was sufficient evidence in all states to show how the distribution of LLINs from the

project significantly increased the baseline figures at the onset of the project, and how this has

had an add on effect on other sources of LLINs, notable among which is from the Global Fund for

AIDS, Tuberculosis and Malaria over the past seven years of project implementation.

4.1.3. MONITORING & EVALUATION.

The overarching goal of Monitoring and Evaluation is to contribute to the strengthening of the

health system through the project.

51

The objective is to introduce the decentralized M&E system as results based management tool to

assess the status of the overall process and introduce program changes as needed. Collection of

several types of information and tools are needed for management purpose that will be supported

by the project9:

(i) Process indicators (inputs, activities, and outputs).

(ii) Results indicators (primarily outcomes);

(iii) Logistic Management Information System (LMIS) data.

(iv) Health Management Information System (HMIS) data.

(v) Lot Quality Assurance Sampling systems (LQAS).

Primarily, this is achieved through to improved collection, quality and utilization of routine data

to monitor the implementation of malaria related interventions through the Health Management

Information System (HMIS).

Key activity Components of the State Level M&E include, but not limited to the following:

Monthly Data Collection and dissemination meetings with FPs

Quarterly Supportive Supervisory Visits.

House hold survey using LQAS.

Health facility (HFs) assessment survey.

Dissemination of LQAS hand tabulation report.

Capacity Building of MOH, FPs, LGA M & E Officers, DSNOs, Data Officer, PIU/PFMU on new

M&E data tools and supervisory checklist

Collation & Sending of monthly surveillance and Prevention/ Treatment data to NMEP.

Key house hold practices (KHHP) Survey.

Quarterly data dissemination meetings with partners.

Prior to the onset of the use of DHIS 2 as a platform for data collection, the programme was using,

and still uses the NMEP (Excel) as a platform for data collection and utilization. The introduction

of the DHIS 2 resulted in certain modifications in the malaria data subset, such that some data

components were dropped to be able to accommodate more disease entities. Consequently, the

process of migration from NMEP data base to DHIS 2 is still on-going.

The M&E project focal person mentioned the following as some of the benefits of the project in his

unit{

Improved data quality.

9 World Bank Project Appraisal Document 2006

52

Where supportive supervision and data quality assessment visits were undertaken, the staff

in visited states were put through various aspects of their work where deficiencies were noticed,

resulting in appreciable capacity building. However, the states were not doing same to staff at

LGAs and facilities/service delivery points. The SS and DQA visits resulted in improved rate of

data traffic to the Federal level from the States and SDPs.

The regular Programme Management Unit (PMU) meetings at the Federal Level, and the

Programme Implementation Unit (PIU) meetings at the state level have also served as training for a,

where the capacity of designated staff to fill forms have been tremendously built. It is expected

that this will be cascaded down to the LGAs and facility levels for a more holistic capacity building

across board.

4.1.4. PROCUREMENT & SUPPLY CHAIN MANAGEMENT.

The Procurement of commodities is the prerogative of the Federal Government, with the project

supporting and strengthening the procurement systems that ensures continuous supply of key

commodities like ACTs, IM Artemether, IPTs, LLIN, RDTS, etc.

The over-arching objective, which is to strengthen logistics and supply chain management of

antimalarial medicines/commodities, was attained in various degrees in the participating States,

and the Federal Government.

State level procurement is categorized into three broad headings:

4.1.4.1. Project Commodities:

These include, but not limited to the following;

a. ITNs/LLINs

b. ACTs

c. SP

d. RDTs

e. IRS chemicals

f. ORS Sachets

g. Others like albendazole, cotrimoxazole, Amoxicillin, Paracetamol and Vit. B complex

syrups, Fersolate and Folic Acid tins (tablets), etc.

4.1.4.2. Consultancy Services.

4.1.4.3. General Works and Services.

It was observed that the Procurement Unit in all participating states, as well as the Federal level

gained considerable experience and capacity building through the adherence to the Bank’s

guidelines.

53

It was also observed from the state reports that delays in project implementation arose from the

interference of Government in routine processes like engagement of firms, consultants and the

supply of goods.

There were various reports indicating the use to which the monies received were put to use in

meeting the above needs by the participating states.

4.1.5. ADVOCACY, & COMMUNITY & SOCIAL MOBILIZATION.

Advocacy, Community and Social mobilization activities are cross cutting, and usually impact on

all areas of programme delivery.

To a great extent, all participating states found and utilized the need for targeted advocacy to

constituted authority to elicit the buy-in to the project. This was seen as a veritable means of

engendering short term program success. In addition, it was also seen as a means of ensuring

programme ownership and sustainability. A detailed presentation of my findings will be made

later.

4.2. Strengthened Leadership capacity of the Federal Government

This component is intended to enhance coordination and oversight of all administrative and

technical aspects of the project, including standardization of implementation capacity and

intervention delivery at state and local levels, and sharing experiences across states. The Director

of the NMCP (NMEP) serves as the Project Coordinator and the Assistant NMCP Director is

Coordinator of the Federal component.

Critical to achieving the core Leadership function by the National Malaria Elimination Programme

are the following:

Adequate coordination of project activities;

Timely identification and resolution of issues affecting or potentially affecting

implementation;

Adequate administrative support to the NMEP, SMoH, and implementing partners.

Provision of project inputs in a timely and cost-effective manner; (v) appropriate

management of project resources in accordance with bank requirements for procurement and

financial management;

Effective monitoring and progress reporting.

The National Programme has had a high turn- over of leadership over the duration of the project,

with the fourth National Coordinator currently at the helm of affairs. While this may not have had

some considerable adverse effects on project delivery, its effect on institutionary memory in the

context of effective and efficient programming cannot be downplayed.

In the opinion of the National Coordinator; Dr. Mrs. Nnenna Ezeigwe, the most critical challenges

faced by the project over its duration include, but are not limited to the following:

54

Procurement- There was a perception by the private sector of exclusion in the award of

contracts.

Guidelines and SOPs-There were instances where extant guidelines for some critical

interventions, like IRS, etc. were not developed. In other cases, developed guidelines were not

disseminated to service delivery points, Local Governments and States. This resulted in relative

skills deficit at service delivery points.

Chapter 5: LESSONS LEARNT AND BEST PRACTICES.

5.1. Lessons Learnt.

There is increased awareness among the people of Nigeria about malaria and preventive

measures: People are now generally more conscious of the dynamics of malaria transmission and

the preventive measures individuals and groups can take to keep safe from malaria.

The NMEP has adapted the Global Malaria Programme recommended strategies, most of which

are already enshrined in the national malaria policy; thereby leading to the institutionalization of

Global best Practices in Malaria Control & Prevention.

Reduced Incidence and Prevalence: Before the year 2000, malaria prevalence was higher than

50%. At the time the Malaria Indicator Survey (MIS) was conducted in 2010 this has dropped to 42%.

Further, a recent mapping study jointly conducted by Glaxo-Welcome Trust at KEMRI and the NMEP

demonstrated a gradual shift from holo-endemicity to meso-endemicity of malaria. By this more

than 80% of the population now reside in areas where the prevalence is between 5-10%. This

corroborates empirical observation which indicates that malaria incidence has reduced.

Enhanced the chance of attaining MDG 4: According to the 2013 NDHS, the under- 5 mortality

has declined by 31% and now stands at 128/1000 Live Births. This is largely due to the impact of

various malaria interventions because in 2010, malaria was responsible for 30% of death in this age

group. At the moment, this proportion has reduced to 25%.

States where Booster Project was implemented did far better in the relevant malaria

indicators than the rest. A major success story of this intervention is in Misau LGA, Bauchi State

where confirmed malaria cases dropped from 14,949 to 4,819(67.8%) after application of IRS

State Programme Managers’ capacities were tremendously built by the project. This cascaded

to other categories of project staff in the various participating states. Consequently, greater

understanding of the project objectives and goal was demonstrated in all participating states

towards the end of the project cycle.

There was little or inadequate opportunities for project monitoring and supportive

supervision in states and service delivery points. Where supportive supervision was undertaken,

this was not sustained.

55

5.2. BEST PRACTICES.

Local innovations: The Booster project has provided opportunities for the development of

local innovative approaches that promote community participation in malaria case management and

control. The involvement of Community Resource Personnel (CORPS) and the Nigeria Interfaith

Action Association (NIFAA) are cases in point.

The Introduced, but short-lived Monthly Integration Committee meetings in Awka, Anambra

State worked very well initially collating reports on the various disease activities during the periodic

M&E visits. Though this was not sustained, it is worth revisiting, since it is a step forward in the

Country’s quest for integration of services at facility and other levels.

Chapter 6: CONCLUSION

Suffice it to say that encouraged by the achievements of the Booster project and other

partner supported interventions, the Federal Ministry of Health refocused the programme towards

a malaria-free Nigeria and developed the new generation National Malaria Strategic Plan 2014-

2020.

The goal of the NMSP is to reduce malaria burden to pre-elimination level of less than 5%

prevalence and bring malaria-related mortality to zero by 2020. In line with this vision, the

national programme was renamed National Malaria Elimination Programme (NMEP) at the last

National Council on Health meeting in August 2013.

The WBMCB Project provided the initial financial frontloading for the development of a robust

RBM partnership financing to scale up proven interventions in the implementing States and

strengthening capacity for malaria implementation at both State and Federal levels.

It is hoped that the observed gains of the project could be scaled up through introduction in

other designated States in the Federation.

However, it is not all good news. Some while, States could have performed better, one or two

others spent so much money on capacity building, which they could not justifiably validate. There

was also no capacity needs assessment by any of the States prior to the trainings. The Monitoring

and Evaluation capacities of the states remain relatively weak, due partly to the high attrition rate

of the few available trained M&E staff in the LGAs.

Chapter 7: RECOMMENDATIONS.

The NMEP should undertake a capacity needs assessment prior to conducting any future

training for programme staff. This should be done at Federal and State levels to identify and

prioritize the capacity needs of staff at various levels.

56

A robust capacity building plan should be instituted and implemented after the capacity

needs assessment. This plan will be shared to all stakeholders for implementation in different

government and partner supported projects.

There should be regular project monitoring and supportive supervision to help build the

capacity of project staff in states and Service Delivery Points.

There is a need for independent monitoring of Programme performance at National, State

and LGA levels.

There should be closer linkage with similar projects like the Global Fund Malaria Project.

National and State level Programme staff should be involved in future pre-project planning

meetings, with other relevant stakeholders or partners.

57

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

58

Annex 9. List of Supporting Documents Federal Government of Nigeria. 2006-2010 Country Strategic Plan. Federal Government of Nigeria. Nigeria’s 2004 National Health Policy Federal Government of Nigeria. Presidential Initiative for Accelerated Achievement of the MDGs Federal Government of Nigeria. 2005 National Economic Empowerment and Development Strategy (NEEDS) Federal Government of Nigeria. End of Project Malaria Household Survey in Nine States of Nigeria, National Malaria Elimination Programme. Federal Ministry of Health, Abuja, March 2015. Federal Government of Nigeria. Nigeria Health Facility Assessment 2009. By Malaria Booster Program, NMCP, FMPH. UNDATED. Federal Government of Nigeria. End of Project Health Facility Assessment in Nine States of Nigeria. By NMEP FMOH. March 2015. IFC Marco and National Population Commission of Federal Republic of Nigeria. Nigeria Demographic and Health Survey 2008. Abuja and Claverton. November 2009. IFC Macro, NMCP-FGON, National Population Commission - FGON, and DHS IFC International. Nigeria Malaria Indicator Survey 2010 Final Report. Abuja and Calverton. June 2014. IFC International and National Population Commission of Federal Republic of Nigeria. Nigeria Demographic and Health Survey 2013. Calverton and Abuja. June 2014. The World Bank and DFID. Country Partnership Strategy for the Federal Republic of Nigeria (2005-2009). Report No. 32412-NG. June 2, 2005. Federal Government of Nigeria. 2009-2013 National Malaria Strategic Plan. Federal Government of Nigeria. 2014-2020 National Malaria Strategic Plan. RBM Country Report on Nigeria (Progress and Impact Series, 2012). UNICEF, Federal Government of Nigeria- National Bureau of Statistics. Nigeria Multiple Indicator Cluster Survey 2007 Final Report: Monitoring the Situation of Children and Women.

59

UNICEF, UNPF, DFID, Federal Government of Nigeria- National Bureau of Statistics. Nigeria Multiple Indicator Cluster Survey 2011 Main Report: Monitoring the Situation of Children and Women. The World Bank Group and UK DFID. Country Partnership Strategy for the Federal Republic of Nigeria (2005-2009). June 2, 2005. The World Bank Group. Project Appraisal Document on Proposed Credit to the Federal Republic of Nigeria for a Malaria Control Booster Project. November 14, 2006. The World Bank, UK DFID, USAID, and the African Development Bank. International Development Association Country Partnership Strategy for the Federal Republic of Nigeria (2010-2013). July 2, 2009. The World Bank Group. International Bank for Reconstruction and Development, International Development Association, International Finance Corporation, and Multilateral Investment Guarantee Agency – Country Partnership Strategy for the Federal Republic of Nigeria for the Period FY2014-2017. March 13, 2014.

60

Annex 10. Results Framework and Monitoring

Results Framework and Monitoring NIGERIA: Malaria Booster Control Project Restructuring

Project Development Objective (PDO): The Project Development Objectives are: (i) to ensure that the target population will have improved access to, and utilization of, a well-defined set of Malaria Plus Package interventions (MPP); and (ii) to strengthen Federal and States ability to manage and oversee delivery of malaria plus interventions.

Revised Project Development Objective (PDO): No change

Fin

al I

ndic

ator

PDO Level Results Indicators

Status Indicator type

Introduction Comments

Con

tin

ue

Dro

pp

ed

New

Rev

ised

PA

D

AF

Res

tru

ctur

e FA

PDO1 Indicator One: Percentage of children less than 5 years of age with fever treated with an effective anti-malarial 10within 24 hours from the onset of symptoms11

X Outcome X The target was revised from 80% to 15% to reflect a more realistic end target based on experiences from other countries with ongoing artemisinin-based combination therapies (ACTs) use, global availability of ACTs, focused treatment intervention, and the need to reduce artemisinin-resistant malaria parasites.

Indicator Two: Percent of infants aged 0-6 months who were fed breast milk only in the last 24 hours12

X X Dropped since it did not align well with activities that support the Project. Replaced by indicator on Vitamin A supplementation

10 The specific anti-malarial is ACT. 11 Target has been revised to be more realistic 12 No longer relevant as program will be using child health weeks to deliver MPP

61

PDO2 Indicator Three: Percentage of children < 5 years who slept under an LLIN/ITN the night preceding the survey

X X

PDO3 Indicator Four: Percentage of pregnant women who received two or more doses of IPT

X X

PDO4 Indicator Five: Percentage of pregnant women who slept under an ITN the night preceding the survey

X X

Indicator Six: Percentage of States using M&E data as a basis for preparing annual work plans for their malaria and MCH Programs13

X X Dropped since the target had been consistently met and maintained over time.

PDO5 Indicator seven: Number of under five children who received vitamin A supplementation

X X Replaced PDO indicator on breastfeeding. New indicator better aligned with the activities that support the Project. However, a timeframe was not included for this indicator.

PDO6 Indicator Eight: Direct Project Beneficiaries, Of which female14 X X Core indicator

Intermediate Results Intermediate Result (Component One): Original Credit: Strengthen the Capacity of the Federal Government to Provide Malaria Control Leadership and Coordination over the Medium and Long-Term. Additional Financing: Health Systems Strengthening for Improved Program Implementation and Better Outcomes for the Malaria Plus Package Intermediate Result Indicator One: Number of ACT doses

distributed (disaggregated by under 5 and over 5) X

Intermediate Result Indicator Two: Number of distribution sites for pediatric ACT per 10,000 population and percentage of distribution sites with no pediatric ACT stock-outs in the last three months prior to the HFA

X

Intermediate Result Indicator Three: Number of LLIN and doses of SP distributed to LG distribution sites per 10,000 population by service type (public and private)

X

IRI1 Intermediate Result Indicator Four: Number of LLIN distributed per year

X

IRI2 Intermediate Result Indicator Five: Number of LLIN distributed in 22 additional states

X

13 Target achieved consistently and adding limited value 14 This is a mandatory core indicator. Beneficiaries will include pregnant women receiving IPT and children under five sleeping under bednets and those treated with effective anti-malarial.

62

Intermediate Result Indicator Six: Data validation system in place for M&E data collection and utilization.

X

Intermediate Result Indicator Seven: Population-based surveys and HFAs carried out. NMCP with the assistance of the M&E agent, triangulate the information from population-based studies and HFA for national level policy making.

X

Intermediate Result Indicator Eight: M&E data used at least annually to modify and improve annual work plans for the National Malaria Control Program.

X

Intermediate Result Indicator Nine: Annual reports of project attainments are produced and submitted to donor and collaborating agencies.

X

Intermediate Result Indicator Ten: Percentage of distribution sites with no SP, LLINs, and pediatric and adult ACT stock-outs in last 3 months prior to the HFA.

X

Intermediate Result Indicator Eleven: Percentage of service delivery points providing regular information to update the LMIS on drug use, balances and orders.

X

Intermediate Result Indicator Twelve: LMIS print-outs produced quarterly.

X

IRI3 Intermediate Result Indicator Thirteen: Percentage of health facilities using laboratory confirmation as the basis for treatment of fever in children.

X

IRI4 Intermediate Result Indicator Fourteen: Percentage of households sprayed with IRS in last 12 months

X

Intermediate Result (Component Two): Original Credit: Strengthen the Health System to Improve Delivery of the Malaria Plus Interventions in the Target States Additional Financing: Health System Delivery through Community System Strengthening, and Communications for Advocacy, Program Implementation, and Results IRI5 Intermediate Result Indicator One: Percentage of households

with at least one ITN/LLIN X

Intermediate Result Indicator Two: Percent of children aged 0-23 months with cough and fast/difficult breathing in the last 2 weeks who were taken to a health facility or received antibiotics from an alternative source.

X

63

IRI6 Intermediate Result Indicator Three: Percentage of children aged 0-23 months with diarrhea in the last two weeks who received oral rehydration solution (ORS) and/or any home fluid.

X

Intermediate Result Indicator Four: Percentage of children 12-23 months that are vaccinated with the DTP3 and measles vaccines.

X

Intermediate Result Indicator Five: Percentage of women with children12 – 23 months who received an ITN/LLIN during last pregnancy.

X

Intermediate Result Indicator Six: Percent of pregnant women who received two or more doses of IPT.

X

IRI7 Intermediate Result Indicator Seven: Percentage of women with children <1 year of age who received ANC care during last pregnancy.

X

Intermediate Result Indicator Eight: Percentage of children less than 5 years of age with fever treated with an effective anti-malarial within 24 hours from the onset of symptoms.

X

Intermediate Result Indicator Nine: Percentage of health facilities that treated children < 5 with febrile disease with an effective anti-malarial using the IMCI protocol by type of service (public and private).

X

Intermediate Result Indicator Ten: Percent of mothers who recognize signs of severe illness in children and know when and where to obtain help.

X

Intermediate Result Indicator Eleven: Percentage of children with fast or difficult breathing or chest in-drawing who were treated for pneumonia.

X

Intermediate Result Indicator Twelve: Percentage of caretakers of under-5 children able to mention two danger signs that would require them to take the child to the health facility.

X

Intermediate Result Indicator Thirteen: M&E data used at least annually to analyze and to improve annual work plans for the SMOH, including State and LGA implementing partners and PIFs.

X

64

IRI8

Intermediate Result Indicator Fourteen: Percentage of children less than 5 years of age with fever treated with an effective anti-malarial within 24 hours from the onset of symptoms in communities piloting CDI.

X

Intermediate Result Indicator Fifteen: Percentage of households with at least one ITN/LLIN the previous night in communities piloting CDI.

X

IRI9 Intermediate Result Indicator Sixteen: Percentage of sampled CDDs with no stock-outs in the last three months15

X

Intermediate Result Indicator Seventeen: Percentage of targeted PMVs in peri-urban and urban areas providing authorized and effective anti-malarials.

X

IRI10 Intermediate Result Indicator Eighteen: Percentage of sampled PMVs that had ACT in stock at the time of assessment.

X

Intermediate Result Indicator Nineteen: Percentage of mothers of children < 5 years of age in CDI communities exposed to mass media IEC messages aimed to increase use of ACT, LLINs and IPT.

X

IRI11 Intermediate Result Indicator Twenty: Percentage of PMVs knowing the correct dose of ACT for under-5 children.

X

IRI12 Intermediate Result Indicator Twenty-One: Number of RDTs distributed to health facilities.

X

Intermediate Result Twenty-Two: Percentage of children less than 5 years of age diagnosed as fever or malaria receiving a prescription of ACT, as observed at the public health facility

X

Intermediate Indicator Twenty-Three: Percentage of public health facilities which prescribed ACT to at least one child less than 5 years old diagnosed as fever or malaria, as observed during the facility assessment.

X

IRI13 Intermediate Result Indicator Twenty-Four: Percentage of caregivers who knows at least two symptoms of malaria.

X

Intermediate Result Indicator Twenty-Five: Percent change in client population’s behaviors following introduction of key BCC and IEC messages.

X

15 This will be a new module in the Household Survey. IE data will also be available in three States.

65

IRI14 Intermediate Result Indicator Twenty-Six: Percentage of mothers of children < 5 years of age in CDI communities who knows at least two ways of preventing malaria

X

66

Annex 11. Impact Evaluation Summary

IMPACT EVALUATION SUMMARY FOR THE IMPLEMENTATION COMPLETION REPORT

The impact evaluation (IE) integrated into the Malaria Control Booster Project focused on the Community Systems Strengthening component of the Additional Financing (AF) and, specifically, on the delivery of publicly-funded malaria prevention, diagnosis, and treatment services through Community Direct Distributors (volunteers) and private sector Patent Medicine Vendors (PMVs), which together accounted for US$4 million of the Project’s resources allocated during the AF in 2009. The IE was conducted in Anambra State and Gombe State to test these two pilot interventions, which were implemented between April 2013 and March 2015. The objective of the IE was to test the impact of these two novel service delivery channels on health and other malaria-related outcomes in the population. The IE results show that the MCBP in Anambra and Gombe States was successful in engaging community-level providers with sufficient knowledge of malaria prevention and treatment to provide effective malaria-related health services. There is some evidence that the interventions improved elements of household-level knowledge of malaria prevention and treatment, though this does not appear to have translated into improved malaria-related health and other outcomes. There are several potential reasons why, overall, the CDD and PMV interventions did not produce positive impacts, each of which contains important policy lessons that can be categorized as follows: (i) operational design and implementation (e.g. the scale, length and intensity of implementation; integration of operational research into program implementation; more emphasis on consumer education; mechanism to ensure subsidy pass-through to the consumer; complementarity of PMVs and CDDs and strengthened referral links, etc.); (ii) health system constraints (supply chain challenges); and (iii) policy context (e.g., policy regarding RDT use by PMVs; targeting the 5-12 age group; and using OR as well as economic evaluation to complement effectiveness analysis) (please see Annex 11.1 and Annex 11.2 for the policy conclusions of Anambra and Gombe States). Although IE implementation did not start until late 2012 with the collection of the baseline survey, discussions on its design were first held in 2007 and the IE concept note was approved in 2009. As the first health sector IE linked to the Bank’s portfolio in Nigeria, the MCBP IE was particularly important as a catalyst in stimulating broader interest within the health sector in rigorous evaluation of health policies and programs for evidence-informed policymaking. Consequently, this IE was instrumental in launching further evaluation work including of Nigeria State Health Program Investment Credit (NSHIP) and of programs that did not receive World Bank financing, such as the Subsidy Reinvestment and Empowerment Programme Maternal and Child Health Project and the Quality Improvement and Clinical Governance Initiative. The MCBP IE experience also contains important lessons for future evaluations, particularly of World Bank-assisted projects. In particular, IE should be complementary to standard project M&E and operational research activities, but for this to be the case the latter may need reinforcing. Second, to maximize the value of the IE for informing policies and operations, interventions that are the subject of an IE should begin implementation as early as possible in the project cycle. Ideally, the IE will be included in project design documents, at least in preliminary form. Third, TA for IE-type activities should be integrated into overall project supervision and capacity building for project implementation.

67

ANNEX 11.1: ANAMBRA STATE MALARIA CONTROL BOOSTER

COMMUNITY SYSTEMS STRENGTHENING INTERVENTIONS

Conclusion and Policy Recommendations

Project Context. The US$180 million World Bank-assisted Malaria Control Booster Project (MCBP) became effective in May 2007. US$100 million Additional Financing (AF) was approved in June 2009 to respond to the then recently updated National Malaria Strategy.16 The AF focused on moving towards universal coverage of the population with key malaria prevention and treatment interventions and put greater emphasis on diagnostics and health systems development. 17 The Community Systems Strengthening (CSS, US$7 million) Component of the Additional Financing included pilots for two novel service-delivery channels to complement the delivery of publicly-funded malaria-related services through public primary healthcare facilities. Under the Community-Directed Interventions (US$2 million) Community-Directed Distributors (CDDs) were trained to provide malaria control services to members of their extended families. Under the Patent Medicine Vendors (PMVs) pilot (US$2 million) private drug retailers were trained in the clinical diagnosis of malaria and the appropriate sale of subsidized artemisinin combination therapies (ACTs). The CDD and PMV interventions were implemented from April 2013 to March 2015.

Evaluation Design. The Malaria Control Booster Project Community Systems Strengthening Impact Evaluation was designed to test the effectiveness of the CDD and PMV interventions, together and in isolation, in improving malaria-related health and other outcomes. The study uses a randomized controlled trial design. Balance tests carried out following the baseline survey confirm that the randomization was successful in creating groups of households that were, on average, equal across outcomes and therefore provide a valid basis for estimating the impacts of the CDD and PMV interventions on household and individual-level outcomes.

Within wards receiving one or both treatments, the selection of PMVs and CDDs was non-random, and selection mechanisms likely favored better connected, more knowledgeable persons. This is particularly true for PMVs. In the case of CDDs, this cadre of health workers was largely created through the intervention, and so there is no control group to compare to in terms of provider-level outcomes. For this reason, the IE is not able to directly test the impact of the CDD and PMV interventions on outcomes at the CDD or PMV level.

Results – Provider Level. With this important caveat, the IE survey results show that it is possible to engage a set of community-level providers with high levels of knowledge in basic malaria prevention and treatment. This is likely due both to the selection mechanisms and to training provided through the project.

Results – Household Level. There is some evidence that the interventions improved elements of household-level knowledge of malaria and treatment-seeking behavior, though this does not appear to have translated into improved malaria-related health and other outcomes. When accounting for the fact that we expect to observe some statistically significant differences between treatment and control groups by chance when looking across a range of variables, we cannot reject the hypothesis that the CDD and PMV interventions did not produce any household-level impacts.

16 Federal Ministry of Health, Nigeria, 2008. 17 The Malaria Control Booster Project closed in March 2015.

68

Key Findings Related to the Causal Pathway. There are several potential reasons for the lack of conclusive evidence that the CDD and PMV interventions produced positive impacts, each of which contains important policy lessons, which fall under three main groups: (1) operational design and implementation; (2) broader health system constraints; and (3) policy context..

1. Operational Design & Implementation

First, the scale of implementation of both interventions may not have been sufficient to generate broader population-level outcomes. On average, 7 PMVs and 30 CDDs were trained per health facility catchment area, to serve an average population of 18,81318. PMVs were found to be the second-most popular place to seek care for malaria, after public health facilities, but this does not necessarily mean that care was sought at a PMV trained under the MCBP (34% of fever cases in the four weeks preceding the survey that sought care for their disease reported doing so from a PMV). Only a very small portion of fever cases (3%) reported seeking care for malaria from community health workers (including CDDs).

Second, it may be that the length and intensity of implementation was not sufficient for the intervention to take root and produce household-level outcomes. The intervention was rolled out over a period of two years, with limited supervision and support to CDDs beyond initial training, and limited monitoring for compliance (to ensure, for example, that PMVs pass ACT subsidies through to consumers). Similar future initiatives should be accompanied by a stronger process of ongoing monitoring and operational research, where feasible, to assess implementation problems and likelihood of success as part of the intervention. In particular, operational research, including impact evaluation, should be considered in the early stages of the project so that these experiences can inform mid-course corrections and later phases of the project, in addition to future policies. Sufficient time should also be factored in for the interventions to take root and for their full functioning prior to evaluating impact. In the case of the IE of the PMV and CDD interventions, this was not possible due to the closing of the Malaria Control Booster Project.

Third, there is a break in the causal pathway between knowledge and behavior. High levels of household-level knowledge on malaria prevention does not translate into high-levels of net use. Furthermore, only one out of every three households recognize ACT as the best treatment for malaria, and among those fever/malaria cases reported in the household survey who took some form of medicine for treatment, only 30% could confirm that they were treated with ACT. In both cases, this suggests further consumer education efforts are needed and underscores the importance of appropriate diagnostic, prescriptive, and patient education practices by healthcare providers, including community agents such as PMVs and CDDs (among those fever cases that were confirmed malaria positive using microscopy or RDTs and that took medication to treat their illness, only 46% could confirm that they were in fact treated with ACT).

Fourth, in seeking to make ACT more accessible and affordable, one important potential challenge that requires further investigation is in ensuring that the subsidy offered to PMVs (or any other healthcare provider) for the purchase of ACT is passed on to consumers. Solutions may include unique labeling, and perhaps the use of SMS technology to validate retail price (and quality). This would require further education of consumers, however.

Fifth, the results of the study suggest PMVs and CDDs should be seen as complementary. PMVs are expected to be relatively more motivated by profit, while CDDs are expected to be relatively more intrinsically motivated. This is demonstrated, for example, by the fact that about one in three CDDs reported carrying out prevention-related activities (promoting net use; net-hanging assistance) in the two weeks preceding the survey, whereas the proportion of PMVs doing these activities is close to zero. On the other

18 This number is based on the total population in the catchment area reported by each primary health facility in Anambra State.

69

hand, PMVs are a preferred source of treatments, with 34% of fever cases in the four weeks preceding the survey seeking care at a PMV versus only 3% at a community health worker (including CDDs). It may be worth considering whether CDDs could be incentivized to do more promotion (two out of three did not report any such activities in the past two weeks) with PMVs focusing on diagnosis with RDTs and treatment. This relationship could be formalized through referral links, including linkage between the CDDs and PMV, and their relationship with the public sector to better address the existing service delivery constraints based on the respective comparative advantage of each platform/agent.

2. Health System Constraints

Both CDDs and PMVs suffered from high rates of stock-outs of ACTs and, for CDDs, of RDTs. Supply chain failures are a well-documented issue within the Nigerian health system, and impact evaluation survey results show that 41% of public primary healthcare facilities in study wards experienced an ACT stock-out in the past three months. PMVs experienced relatively fewer stock-outs than did CDDs as they were not fully reliant for ACTs on the public distribution system. Still, ACTs procured from markets habitually used by PMVs would not be subject to the subsidy offered through the project and so are not necessarily a perfect substitute for ACTs procured and distributed through the MCBP. For any intervention aiming to use local actors to improve access to malaria or other health services, resolving higher-level issues in the supply chain is critical.

3. Policy Context

First, at the time the project was conceived national policy did not permit PMVs to use RDTs for malaria diagnosis (this restriction remains in place, though the latest version of the National Strategic Plan suggests that lifting this restriction should be considered). Instead, PMVs were trained in clinical diagnosis of malaria. Household survey results show that clinical diagnosis is more likely to lead to presumptive treatment: 60% of those diagnosed through microscopy or RDT were confirmed malaria positive, whereas as 92% of those diagnosed clinically took some form of medication. As an important community-level provider of health services, strategies should be developed to train and incentivize PMVs to use RDTs (the fact that current policy considers allowing PMVs to use RDTs is an important first step). For optimal care continuum, a referral system between CDDs and PMVs could be considered. Consumer education campaigns may also be needed to encourage patients/customers to demand an RDT test prior to purchasing medication. Using ACTs to treat only confirmed cases of malaria is important both in terms of efficient use of public (and private) funding, correct treatment for patients not suffering from malaria, and reducing the risk of drug resistance.

Second, the results show highest malaria prevalence among children between 5 and 12, the group not prioritized by the intervention. Traditionally, most interventions related to malaria focus on children under five and pregnant women as they are the most vulnerable groups to malaria. The relatively high percentage of malaria prevalence among children between 5 and 12 suggests, however, that efforts to address malaria in this population group, which may have developed some immunity and therefore be asymptomatic, remain important. In particular, there is evidence showing that untreated malaria infections can negatively affect children’s health and cognitive function which could impact education achievement and the development of human capital. Also, asymptomatic individuals can act as reservoirs of malaria, complicating the push towards malaria elimination.19

19 Halliday, Katherine, Georgeo Okello, Elizabeth Turner, Kiambo Njagi, Carlos Mcharo, Juddy Kengo, Elizabeth Allen, Margaret Dubeck, Matthew Jukes, and Simon Brooker. 2014. Impact of intermittent screening and treatment for malaria among school children in Kenya: a cluster randomised trial. PLoS Med.

70

Future Implications for Operations & Research. The results of this study suggest that significant improvements to the design of the CDD and PMV interventions need to be made if these are to be effective, and that further piloting and testing are needed prior to any scale-up. Furthermore, any such programs should be accompanied by regular supportive supervision and monitoring and operational research to provide ongoing guidance to the community-level providers and identify critical issues (such as lack of commodities) early.

A greater emphasis on collecting reliable cost/expenditure data is needed to understand the eventual cost-effectiveness of any such redesigned intervention.

Further research carried out as part of the MCBP IE will explore variation in treatment effects depending on the distance of the household from a trained provider and on household socioeconomic status.

71

ANNEX 11.2: GOMBE STATE MALARIA CONTROL BOOSTER

COMMUNITY SYSTEMS STRENGTHENING INTERVENTIONS

Conclusion and Policy Recommendations

Project Context. The US$180 million World Bank-assisted Malaria Control Booster Project (MCBP) became effective in May 2007. US$100 million Additional Financing (AF) was approved in June 2009 to respond to the then recently updated National Malaria Strategy.20 The AF focused on moving towards universal coverage of the population with key malaria prevention and treatment interventions and put greater emphasis on diagnostics and health systems development. 21 The Community Systems Strengthening (CSS, US$7 million) Component of the Additional Financing included pilots for two novel service-delivery channels to complement the delivery of publicly-funded malaria-related services through public primary healthcare facilities. Under the Community-Directed Interventions (US$2 million) Community-Directed Distributors (CDDs) were trained to provide malaria control services to members of their extended families. Under the Patent Medicine Vendors (PMVs) pilot (US$2 million) private drug retailers were trained in the clinical diagnosis of malaria and the appropriate sale of subsidized artemisinin combination therapies (ACTs). The CDD and PMV interventions were implemented from April 2013 to March 2015.

Evaluation Design. The Malaria Control Booster Project Community Systems Strengthening Impact Evaluation was designed to test the effectiveness of the CDD and PMV interventions, together and in isolation, in improving malaria-related health and other outcomes. The study uses a randomized controlled trial design. Balance tests carried out following the baseline survey confirm that the randomization was successful in creating groups of households that were, on average, equal across outcomes and therefore provide a valid basis for estimating the impacts of the CDD and PMV interventions on household and individual-level outcomes.

Within wards receiving one or both treatments, the selection of PMVs and CDDs was non-random, and selection mechanisms likely favored better connected, more knowledgeable persons. This is particularly true for PMVs. In the case of CDDs, this cadre of health workers was largely created through the intervention, and so there is no control group to compare to in terms of provider-level outcomes. For this reason, the IE is not able to directly test the impact of the CDD and PMV interventions on outcomes at the CDD or PMV level.

Results – Provider Level. With this important caveat, the IE survey results show that it is possible to engage a set of community-level providers with high levels of knowledge in basic malaria prevention and treatment. This is likely due both to the selection mechanisms and to training provided through the project.

Results – Household Level. There is some evidence that the interventions improved elements of household-level knowledge of malaria and treatment-seeking behavior, though this does not appear to have translated into improved malaria-related health and other outcomes.22 When accounting for the fact that we expect to observe some statistically significant differences between treatment and control groups by chance when looking across a range of variables, we cannot reject the hypothesis that the CDD and PMV interventions did not produce any household-level impacts.

20 Federal Ministry of Health, Nigeria, 2008. 21 The Malaria Control Booster Project closed in March 2015. 22 There are some exceptions, such as the observed improved result for self-reported malaria by pregnant women under the joint treatment arm.

72

Key Findings Related to the Causal Pathway. There are several potential reasons for the lack of conclusive evidence that the CDD and PMV interventions produced positive impacts, each of which contains important policy lessons, which fall under three main groups: (1) operational design and implementation; (2) broader health system constraints; and (3) policy context.

1. Operational Design & Implementation

First, the scale of implementation of both interventions may not have been sufficient to generate broader population-level outcomes. On average, 7 PMVs and 22 CDDs were trained per health facility catchment area, to serve an average population of 10,17023. PMVs were found to be the second-most popular place to seek care for malaria, after public health facilities, but this does not necessarily mean that care was sought at a PMV trained under the MCBP (22% of fever cases in the four weeks preceding the survey that sought care for their disease reported doing so from a PMV). Only a very small portion of fever cases (6%) reported seeking care for malaria from community health workers (including CDDs), and the fact that Community Health Workers (which includes CDDs) were the least accessible category of health worker suggests the intervention may have suffered from insufficient scale.

Second, it may be that the length and intensity of implementation was not sufficient for the intervention to take root and produce household-level outcomes. The intervention was rolled out over a period of two years, with limited supervision and support to CDDs beyond initial training, and limited monitoring for compliance (to ensure, for example, that PMVs pass ACT subsidies through to consumers). Similar future initiatives should be accompanied by a stronger process of ongoing monitoring and operational research, where feasible, to assess implementation problems and likelihood of success as part of the intervention. In particular, operational research, including impact evaluation, should be considered in the early stages of the project so that these experiences can inform mid-course corrections and later phases of the project, in addition to future policies. Sufficient time should also be factored in for the interventions to take root and for their full functioning prior to evaluating impact. In the case of the IE of the PMV and CDD interventions, this was not possible due to the closing of the Malaria Control Booster Project.

Third, there is a break in the causal pathway between knowledge and behavior. High levels of household-level knowledge on malaria prevention do not translate into high-levels of net use. Furthermore, less than half of households recognize ACT as the best treatment for malaria, and among those fever/malaria cases reported in the household survey who took some form of medicine for treatment, only 40% could confirm that they were treated with ACT. In both cases, this suggests further consumer education efforts are needed and underscores the importance of appropriate diagnostic, prescriptive, and patient education practices by healthcare providers, including community agents such as PMVs and CDDs (among those who took ACT to treat their fever, only 40% could confirm that they were treated with ACT).

Fourth, in seeking to make ACT more accessible and affordable, one important potential challenge that requires further investigation is in ensuring that the subsidy offered to PMVs (or any other healthcare provider) for the purchase of ACT is passed on to consumers. Solutions may include unique labeling, and perhaps the use of SMS technology to validate retail price (and quality). This would require further education of consumers, however.

Fifth, the results of the study suggest PMVs and CDDs should be seen as complementary. PMVs are expected to be relatively more motivated by profit, while CDDs are expected to be relatively more intrinsically motivated. This is demonstrated, for example, by the fact that about one in three CDDs reported

23 This number is based on the total population in the catchment area reported by each primary health facility in Gombe State.

73

carrying out prevention-related activities (promoting net use; net-hanging assistance) in the two weeks preceding the survey, whereas the proportion of PMVs doing these activities is less than one in ten. On the other hand, PMVs are a preferred source of treatments, with 22% of fever cases in the four weeks preceding the survey seeking care at a PMV versus only 6% at a community health worker (including CDDs). It may be worth considering whether CDDs could be incentivized to do more promotion (two out of three did not report any such activities in the past two weeks) with PMVs focusing on diagnosis with RDTs and treatment. This relationship could be formalized through referral links, including linkage between the CDDs and PMV, and their relationship with the public sector to better address the existing service delivery constraints based on the respective comparative advantage of each platform/agent.

2. Health System Constraints

Both CDDs and PMVs suffered from high rates of stock-outs of ACTs and, for CDDs, of RDTs. Supply chain failures are a well-documented issue within the Nigerian health system, and impact evaluation survey results show that 27% of public primary healthcare facilities in study wards experienced an ACT stock-out in the past three months. PMVs experienced relatively fewer stock-outs than did CDDs as they were not fully reliant for ACTs on the public distribution system. Still, ACTs procured from markets habitually used by PMVs would not be subject to the subsidy offered through the project and so are not necessarily a perfect substitute for ACTs procured and distributed through the MCBP. For any intervention aiming to use local actors to improve access to malaria or other health services, resolving higher-level issues in the supply chain is critical.

3. Policy Context

First, at the time the project was conceived national policy did not permit PMVs to use RDTs for malaria diagnosis (this restriction remains in place, though the latest version of the National Strategic Plan suggests that lifting this restriction should be considered). Instead, PMVs were trained in clinical diagnosis of malaria. Household survey results show that clinical diagnosis is more likely to lead to presumptive treatment: 66% of those diagnosed through microscopy or RDT were confirmed malaria positive, whereas as 87% of those diagnosed clinically took some form of medication. As an important community-level provider of health services, strategies should be developed to train and incentivize PMVs to use RDTs (the fact that current policy considers allowing PMVs to use RDTs is an important first step). For optimal care continuum, a referral system between CDDs and PMVs could be considered. Consumer education campaigns may also be needed to encourage patients/customers to demand an RDT test prior to purchasing medication. Using ACTs to treat only confirmed cases of malaria is important both in terms of efficient use of public (and private) funding, correct treatment for patients not suffering from malaria, and reducing the risk of drug resistance.

Second, the results show highest malaria prevalence among children between 5 and 12, the group not prioritized by the intervention. Traditionally, most interventions related to malaria focus on children under five and pregnant women as they are the most vulnerable groups to malaria. The relatively high percentage of malaria prevalence among children between 5 and 12 suggests, however, that efforts to address malaria in this population group, which may have developed some immunity and therefore be asymptomatic, remain important. In particular, there is evidence showing that untreated malaria infections can negatively affect children’s health and cognitive function which could impact education achievement

74

and the development of human capital. Also, asymptomatic individuals can act as reservoirs of malaria, complicating the push towards malaria elimination.24

Future Implications for Operations & Research. The results of this study suggest that significant improvements to the design of the CDD and PMV interventions need to be made if these are to be effective, and that further piloting and testing are needed prior to any scale-up. Furthermore, any such programs should be accompanied by regular supportive supervision and monitoring and operational research to provide ongoing guidance to the community-level providers and identify critical issues (such as lack of commodities) early (approximately one in five CDDs noted that stock-outs of RDT and ACT resulted from a lack of planning and/or unexpected demand).

A greater emphasis on collecting reliable cost/expenditure data is also needed to understand the eventual cost-effectiveness of any such redesigned intervention.

Further research carried out as part of the MCBP IE will explore variation in treatment effects depending on the distance of the household from a trained provider and on household socioeconomic status.

24 Halliday, Katherine, Georgeo Okello, Elizabeth Turner, Kiambo Njagi, Carlos Mcharo, Juddy Kengo, Elizabeth Allen, Margaret Dubeck, Matthew Jukes, and Simon Brooker. 2014. Impact of intermittent screening and treatment for malaria among school children in Kenya: a cluster randomised trial. PLoS Med.

75

MAP