mali - strengthening reproductive health - … · web viewin 2016, through the rbf pilot about...

Document ofThe World Bank

Report No: ICR00004095

IMPLEMENTATION COMPLETION AND RESULTS REPORT(IDA-H7530)

ON A GRANT

IN THE AMOUNT OF SDR 19 MILLION

(US$30 MILLION EQUIVALENT)

TO THE

REPUBLIC OF MALI

FOR A

STRENGTHENING REPRODUCTIVE HEALTH PROJECT (SRHP)

August 2, 2017

Health, Nutrition and Population Global PracticeAfrica Region

CURRENCY EQUIVALENTS

(Exchange Rate Effective February 28, 2017)

Currency Unit = West African CFA franc (CFAF) CFAF 618.65 = US$1

US$1.35 = SDR 1

FISCAL YEARJanuary 1 – December 31

ABBREVIATIONS AND ACRONYMS

ANJE Alimentation du Nourrisson et du Jeune Enfant, Infant and Young Child FeedingASACO Association de Santé Communautaire, Community Heath AssociationBCC Behavioral Change Communication C4D Communication for Development CADD Cellule d’Appui à la Décentralisation at à la Déconcentration, Unit for

Decentralization and De-concentration SupportCBA Cost-benefit AnalysisCFU Comité des femmes utilisatrices des services de santé, Committee of Female Health

Service UtilizersCPF Country Partnership Framework CPS Cellules de Planification et de Statistiques, Planning and Statistics UnitCSCOM Centre de Santé Communautaire, Community Health CenterCSREF Centre de Santé de Référence, Referral Health CenterCYP Couple Year Protection DFM Direction des Finances et du Matériel, Finance and Materials Directorate DHS Demographic and Health Survey DNP Direction Nationale de la Population, National Population Directorate DNS Direction Nationale de la Santé - DNS), National Health DirectionDPM Direction de la Pharmacie et des Médicaments, Directorate of Pharmacy and

MedicinesDPO Development Policy OperationDRH Direction des Ressources Humaines, Human Resource DirectorateERR Economic Rate of Return FENASCOM Fédération Nationale des Associations de Santé Communautaire, National

Federation of Community Health AssociationsFMA Financial Management Agency FP Family PlanningHIV Human Immunodeficiency VirusHMIS Système Local d'Informations Sanitaires, Local Heath Management Information System

ICR Implementation Completion Report IHP+ International Health Partnership IUD Intra-Uterine Device

MDG Millennium Development Goals M&E Monitoring and Evaluation MICS Multiple Indicator Cluster Survey MOF Ministère de l'Économie et des Finances, Ministry of Economy and FinanceMOH Ministère de la Santé et de l'Hygiène Publique, Ministry of Health and Public

HygieneMOU Memorandum of UnderstandingMTR Mid-term Review NGO Non-Governmental OrganizationOOP Out of pocket expenditurePIU Project Coordination Unit NPV Net Present ValuePDDS Plan Décennal de Développement Sanitaire et Social, Ten-Year Plan for Health and

Social Development 2014-2023PDO Project Development ObjectivePRODESS Programme de Développement Sanitaire et Social, Health and Social Development

ProgramPAD Project Appraisal Document PPF Project Preparation FacilityRBF Results-Based Financing RH Reproductive HealthSG Secretary General SRHP Strengthening Reproductive Health ProjectSWEDD Sahel Women's Empowerment and Demographic Dividend Project USAID United States Agency for International Development UNFPA United Nations Population FundUNICEF United Nations Children's Fund WHO World Health Organization

Senior Global Practice Director: Timothy Grant EvansCountry Director: Soukeyna Kane

Sector Manager: Trina HaqueProject Team Leader: Aissatou Diack

ICR Team Leader/Author: Jenny Gold

MALIStrengthening Reproductive Health Project (P124054)

TABLE OF CONTENTS

Data SheetA. Basic Information…………………………………………………………………………B. Key Dates………………………………………………………………………………….iiC. Ratings Summary………………………………………………………………………….iiD. Sector and Theme Codes………………………………………………………………….iiiE. Bank Staff………………………………………………………………………………....iiiF. Results Framework Analysis………………………………………………………………

ivG. Ratings of Project Performance in ISRs…………………………………………………..ixH. Restructuring………………………………………………………………………………x I. Disbursement Graph……………………………………………………………………….x

1. Project Context, Development Objectives and Design2. Key Factors Affecting Implementation and Outcomes3. Assessment of Outcomes4. Assessment of Risk to Development Outcome5. Assessment of Bank and Borrower Performance6. Lessons Learned7. Comments on Issues Raised by Borrower/Implementing Agencies/PartnersAnnex 1. Project Costs and FinancingAnnex 2. Outputs by ComponentAnnex 3. Economic and Financial AnalysisAnnex 4. Bank Lending and Implementation Support/Supervision ProcessesAnnex 5. Beneficiary Survey ResultsAnnex 6. Stakeholder Workshop Report and ResultsAnnex 7. Summary of Borrower's ICR and/or Comments on Draft ICRAnnex 8. Comments of Cofinanciers and Other Partners/StakeholdersAnnex 9. List of Supporting Documents

MAP

4

A. Basic Information

Country: Mali Project Name:ML-Strengthening Reproductive Health

Project ID: P124054 L/C/TF Number(s): IDA-H7530

ICR Date: 05/21/2017 ICR Type: Core ICR

Lending Instrument: SIL Borrower:GOVERNMENT OF MALI

Original Total Commitment:

XDR 19.00M Disbursed Amount: XDR 5.47M

Revised Amount: XDR 6.56M

Environmental Category: B

Implementing Agencies: Ministry of Health and Public Hygiene (MOH)

Human Resource Directorate (DRH) National Health Direction (DNS) Unit for Decentralization and De-concentration Support (CADD) Finance and Equipment Directorate (DFM) Planning and Statistics Unit (CPS) Directorate of Pharmacy and Medicines (DPM)

National Directorate of Population (DNP)

Cofinanciers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 02/28/2011 Effectiveness: 05/23/2012 03/15/2013

Appraisal: 09/08/2011 Restructuring(s):10/02/201412/29/2016

Approval: 12/20/2011 Mid-term Review: 06/30/2015 06/10/2015

Closing: 02/28/2017 02/28/2017

C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Unsatisfactory

Risk to Development Outcome: Significant

Bank Performance: Moderately Unsatisfactory

Borrower Performance: Moderately Unsatisfactory

5

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Moderately Unsatisfactory

Quality of Supervision: Moderately Unsatisfactory Implementing Agency/Agencies: Moderately Unsatisfactory

Overall Bank Performance: Moderately UnsatisfactoryOverall Borrower

Performance: Moderately Unsatisfactory

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA): None

Problem Project at any time (Yes/No):

YesQuality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Unsatisfactory

D. Sector and Theme Codes Original Actual

Major Sector/Sector

Health

Health 75 75

Public Administration – Health 25 25

Major Theme/Theme/Sub Theme

Human Development and Gender

Health Systems and Policies 100 100

Reproductive and Maternal Health 100 100

Private Sector Development

Public Private Partnerships 10 10

E. Bank Staff Positions At ICR At Approval

Vice President: Makhtar Diop Obiageli Katryn Ezekwesili

Country Director: Soukeyna Kane Ousmane Diagana

Practice Manager/Manager: Trina S. Haque Jean J. De St Antoine

Project Team Leader: Aissatou Diack Aissatou Diack

ICR Team Leader: Jenny R. Gold

ICR Primary Author: Jenny R. Gold

6

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)To improve access and use of quality reproductive health (RH) services by women of reproductive age, in selected regions of Mali.

The selected project intervention regions are Sikasso, Koulikoro, Ségou and peri-urban Bamako. Revised Project Development Objectives (as approved by original approving authority)N/A

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Modern contraceptive use among women 15 to 49 years old (percent for the project intervention regions)

Value quantitative or Qualitative)

8.0 percent national;10.0 percent project regions 15 percent

21.0 percent national;21.5 percent project regions

Date achieved 12/20/2011 02/28/2017 02/28/2017Comments (incl. percent achievement)

Target was surpassed. Per the latest Multiple Indicator Cluster Survey (MICS)1 in 2015 the use of modern contraceptives in the project regions was 19 percent. There was an increase of about 9 percent in the project regions, since the 2010 MICS baseline of 10 percent. The MICS also shows an increase nationally from 2010 to 2015, from 8 to 16 percent. The survey conducted at the end of the project (February 2017) by the MOH and UNICEF2 estimated that 22 percent of women 15 to 49 years old in the project regions use modern contraceptives. A total increase of about 11 percent, since the MICS 2010 baseline. Data from the Local Heath Management Information System (HMIS)3 show a comparable increase in the number of women 15 to 49 using modern contraceptives in the project regions: 174,147 in 2012; 248,290 in 2013; 310,113 in 2014; 284,864 in 2015; and 498,884 in 2016. Further, the 2016 HMIS data shows an increased use of modern contraceptives in each project region: 24 percent in Sikasso (baseline 7 percent); 25 percent in Koulikoro (baseline 9 percent); 22 percent in Ségou (baseline 9 percent); and 33 percent in Bamako (baseline 18 percent). This compares to 11 percent in Tombouctou, 13 percent in Mopti, 17 percent in Kayes, 3 percent in Gao, and 2 percent in Kidal (HMIS 2016). The HMIS also shows a national increased prevalence of 21 percent in 2016, which is largely attributable to the project regions. project interventions

1 United Nations Children's Fund (UNICEF), Mali Multiple Indicator Cluster Survey 2015. 2 Ministry of Health and Public Hygiene (MOH) and UNICEF, February 2017. Étude Finale du projet : Amélioration de l’accès et de l’utilisation de services de santé de la reproduction de qualité pour les femmes en âge de procréer dans les régions de Koulikoro, Sikasso, Ségou et le District de Bamako.3Système Local d'Informations Sanitaires, Local Heath Management Information System (HMIS), 2012, 2013, 2014, 2015, and preliminary data from 2016.

7

supporting these results included advocacy, training, a results-based financing (RBF) pilot and the Communication for Development (C4D) program led by the MOH with support from UNICEF. The data for this indicator are presented as an average for the project regions with a weighting by population, where relevant. Important to note is the support of other partners for interventions in Kayes and Mopti, such as France, Spain and Canada, which have also contributed to the national increased prevalence of contraceptive use. The intense C4D program in 2016 was the main social mobilization activity in the project regions, and is very likely responsible for the sharp increase in the indicator in the project regions between 2015 and 2016.

Indicator 2 : Pregnant women who have at least 4 antenatal care visits (percent for the project intervention regions)


34.9 percent national;37.8 percent project regions 43 percent


Date achieved 12/20/2011 02/28/2017 02/28/2017

Comments (incl. percent achievement)

Target was surpassed. Per the latest MICS, in 2015 the indicator was 43 percent in the project regions – an increase of almost 5 percent since the 2010 MICS baseline of 38 percent in the project regions. The MICS shows a more modest 3 percent increase nationally from 2010 to 2015, from 35 to 38 percent. The survey conducted at the end of the project (February 2017) by the MOH and UNICEF estimated 63 percent of women in the project regions had more than 4 prenatal/antenatal care visits – an increase of about 26 percent since the MICS 2010 baseline. The HMIS data protocols for this indicator are being newly developed, limiting data availability. However, in 2016, the HMIS shows about 215,629 women nationally (about 31 percent) had 4 prenatal visits. project interventions supporting this indicator included training, an RBF pilot and the C4D program. The data for this indicator are presented as an average for the project regions with a weighting by population, where relevant. Disaggregated data was not available by region. The project training and intensive C4D activities to promote 4 prenatal care visits in the project regions, very likely increased the number of women receiving multiple follow-up prenatal care visits in 2016, and results will likely continue in 2017.

Indicator 3 : Births (deliveries) attended by skilled health personnel (percent for the project intervention regions)


29.1 percent national;35.0 percent project regions 50 percent 48 percent


Date achieved 12/20/2011 02/28/2017 02/28/2017 02/28/2017Comments (incl. percent achievement)

The original and revised targets were surpassed. Per the MICS, in 2015 the indicator was 51 percent in the project regions and 44 percent nationally – an increase from the 2010 MICS baseline of almost 16 percent in the project regions, compared to a more modest national increase (about 13 percent). Complementary data from the survey conducted at the end of the project (February 2017) estimated 64 percent of women gave birth at a health center, but the survey did not ask whether the birth was attended by skilled personnel. Hence, the comparability of this data is limited. HMIS data on assisted births in health facilities, however, show a comparable increase in the indicator: 173,623 women in 2013; 205,628 women in 2014; 208,978 women in 2015; 217,377 women in 2016. The number of births attended by skilled health personnel increased by 20 percent since 2013. Further, the 2016 HMIS data shows an increase in the indicator across the project regions: 33 percent in Sikasso (baseline 12 percent); 49 percent in Koulikoro (baseline 29 percent); 43 percent in Ségou (baseline 27 percent); and 94 percent in Bamako (baseline 90 percent). This compares to 30.6 percent in Mopti, 43.8 percent in Kayes, 55.9 percent in Gao, and 59.1 percent in Kidal per the 2016 HMIS. Data for Tombouctou were not available due to the ongoing conflict. The HMIS also shows a

8

national increase in births attended by a skilled health personnel (50 percent in 2016). project interventions influencing this indicator started in 2014—key activities included training, maternal mortality audits, support to local solidarity funds in two districts to provide emergency transport to women giving birth, as well as an RBF pilot and the C4D activities. These were key interventions in the project regions during the project time frame, whereas other donor support focused in other regions. Targets were restructured in 2014 to account for the shortened time frame of the project. The data for this indicator are presented as an average for the project regions with a weighting by population, where relevant. The scale-up of the maternal mortality audits and obstetrics care in health centers in particularly increased access to quality birthing services in 2016 – when these services were fully institutionalized.

Indicator 4 : Postpartum women using modern methods of contraception (number for the project intervention regions)


0 300,000

No data was collected.

Date achieved 12/20/2011 02/28/2017 02/28/2017


No data available. The indicator depended on data collection from the NGO activities, which were not carried out. The HMIS shows the percent of women receiving postpartum care in the project regions has increased by 7 percent since 2013: 220,894 in 2013 (64.4 percent); 245,938 (65.9 percent) in 2014; 253,305 (66.7 percent) in 2015; 276,521 (71.5 percent) in 2016. However, there is no data available on the provision of contraceptives postpartum. Further, national data on postpartum care was not available for 2016 to make a comparison to the project regions. The intensive C4D activities and family planning (FP) training included guidance for FP provision postpartum, which likely influenced this indicator, although not captured.

Indicator 5 : Couple-Year Protections (CYP) reached through project interventions (number) – New indicator added in 2014


0 CYP 900,000 CYP 1,349,168 CYP

Date achieved 10/02/2014 02/28/2017 02/28/2017


Target was surpassed. Per HMIS data, the cumulative total for 2015 to 2016 was 1,349,168 CYP for the project regions, exceeding the target by 50 percent. This indicator was added in 2014 to better assess changes in contraceptive access influenced by the project. The project reports show a direct contribution of 468,000 CYP by 2017 through the purchasing of contraceptives that entered the national drug management system.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target

Values

Actual Value Achieved at

Completion or Target Years

Indicator 1: Women provided with access to a package of reproductive health (RH) services (number in Koulikoro region)

Value (quantitative or Qualitative)

0

120,000 with additional values were to be defined if RBF pilot was to be scaled-up

200,000 82,315

9

Date achieved 12/20/2011 02/28/2017 02/28/2017 02/28/2017


Not achieved. The validated reports4 from the RBF pilot in Koulikoro regions show the total number of women provided access to a package of RH services through the RBF was 41 percent below the revised target. However, there was substantial achievement towards the original target of 120,000 women in the project Appraisal Document (PAD). The target was increased in the 2014 restructuring when more financing was allocated to the RBF credit and the RBF pilot was expected to start in 2015. However, the start was delayed until 2016, with only one cycle of implementation, rather than the multiple cycles required to reinforce the approach.

Indicator 2: Health facilities that experience 2 or more weeks’ stock-outs of contraceptives during the last 2 weeks preceding the survey (percent)


No baseline was collected. less than 5 percent No data was collected.

Date achieved 02/28/2017 02/28/2017 02/28/2017


No data available. The indicator depended on the provision of support to district pharmacist by the Directorate of Pharmacy and Medicines (DPM) was delayed due the rules around the Daily Subsistence Allowance for training. At the time of 2014 restructuring, the stock management support was expected to start in 2015, including the baseline data collection.

Data from USAID5 shows the national stock of long-duration contraceptive methods was 9.3 percent for IUDs and 4 percent for implants in January 2017, compared to 21 percent and 8.6 percent in January 2015. The project likely contributed to this increased availability, given the focus on long-duration methods (in terms of procurement, but also advocacy of the Government to increase stock of long-duration methods).

Indicator 3: Health personnel trained to provide family planning (FP) services (number)


0 400 3,530


Target was surpassed. The project reports6show the number of health personnel trained to provide family planning (FP) services was achieved by 882 percent.

Indicator 4: Pregnant women receiving antenatal care during a visit to a health care provider (number in the project intervention regions)


0 500,000 400,000

165,009 in 2013;185,120 in 2014; 190,398 in 2015; 462,460 in 2016

Date achieved 12/20/2011 02/28/2017 02/28/2017 02/28/2017Comments (incl. percent achievement)

Target was surpassed. Per HMIS data, the project surpassed the revised target for the indicator in 2016, by 116 percent. Prenatal care visits more than doubled since 2014. No data is available for 2017. The 2014 restructuring revised the targets in order to account for the delayed start of the project. The project likely advanced the indicator

4 KIT-CORDAID-CGIC, Rapport final du projet pilote « PRSR/FBR » FBR-Financement Basé sur les Résultats – Approche pour accélérer les résultats en Santé de la Reproduction, April 2017.5 Data from the United States Agency for International Development (USAID) OSPSANTE portal, Monitoring Tool for Health Products, 2017.6 Data from quarterly and annual project activity reports, 2013 to 2017.

10

through training activities in 2014 and 2015, and the RBF and C4D program in 2016. The increased demand for services generated from the C4D program likely triggered the sharp increase in 2016.

Indicator 5: Deliveries by caesarean section (number in the project intervention regions)


12,756 in 2010 30,000

16,032 in 2013;17,881 in 2014; 17,826 in 2015; 18,461 in 2016

Date achieved 12/20/2011 02/28/2017 02/28/2017


Not achieved. No data is available for 2017. This project likely had a modest contribution through the obstetrical training activities in 2015, and the RH supplies and equipment distributed in 2016, as well as the RBF. Note, the baseline in the PAD is 16,151 women, which is the national number reported in the HMIS for 2010. 12,756 is the total number for the four project regions in 2010.

Indicator 6: Women provided with postpartum care within 7 days of delivering in a health facility (percent for the project intervention regions)


54.6 percent 70 percent

64.4 percent in 2013;65.9 percent in 2014;66.7 percent in 2015; 71.5 percent in 2016

Date achieved 12/20/2011 02/28/2017 02/28/2017


Target was surpassed. Per HMIS data, the target was surpassed by almost 2 percent in the project regions. No data is available for 2017. Further a health facility survey was not done to provide data to validate whether the postpartum care was received in 7 days. The indicator was likely influenced by the obstetrical training activities in 2014 and 2015, the RH supplies and equipment distributed in 2016, as well as the RBF and C4D activities led by the MOH and UNICEF in 2016. The 2010 baseline in the PAD is 55.4 percent. However, the weighted average for the intervention regions in 2010 is 54.6 percent. National data was not available to compare progress.

Indicator 7: Health personnel trained to provide obstetrics care (number) Value (quantitative or Qualitative)

0 400 806


Target was surpassed. The project reports show the target for this indicator was greatly surpassed, by more than 200 percent. The source of the data is annual and quarterly project reports.

Indicator 8: People (women and men) attending social marketing interventions that promote family planning (number for the project intervention regions)


0 600,000 742,364

Date achieved 12/20/2011 02/28/2017 02/28/2017Comments (incl. percent

Target was surpassed. Annual and quarterly project reports as well as the C4D report7 provide data on the indicator, which was achieved by 124 percent in the last year of the

7 UNICEF 2017. Rapport de Progrès (Décembre 2016- Février 2017) Accroitre la demande et l’utilisation des Services de Santé de la Reproduction (SR) et Planification Familiale (PF) aux niveaux des CSREF et CSCOM dans les 34 districts au Mali (SC160310).

11

achievement) project. Achievement was primarily through the C4D program. Indicator 9: Pregnant women provided with vouchers to access selected services and who use them Value (quantitative or Qualitative)

0 10,000 0


Not achieved. The voucher scheme was not implemented. The focus was rather on training of solidarity funds. An estimated 400 pregnant women benefited from emergency transport services from the reinforcement of solidarity funds in two districts.8

Indicator 10: Advocacy sessions held with civil society, political, religious and traditional leaders on demographic issues and family planning (number)


0 300 652


Target was surpassed. project reports show the indicator was achieved by 217 percent in the last year of the project through the C4D program implemented by the MOH and UNICEF.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP Actual Disbursements

(USD millions) 1 04/03/2012 Satisfactory Satisfactory 0.00 2 12/09/2012 Satisfactory Satisfactory 0.00 3 06/22/2013 Moderately Satisfactory Moderately Satisfactory 0.00 4 12/31/2013 Moderately Unsatisfactory Moderately Unsatisfactory 1.66 5 06/22/2014 Moderately Unsatisfactory Moderately Unsatisfactory 1.66 6 12/23/2014 Moderately Unsatisfactory Moderately Unsatisfactory 1.95 7 06/25/2015 Moderately Unsatisfactory Moderately Unsatisfactory 4.56 8 11/24/2015 Moderately Unsatisfactory Moderately Unsatisfactory 4.56 9 06/13/2016 Moderately Unsatisfactory Moderately Unsatisfactory 5.27 10 02/27/2017 Moderately Unsatisfactory Moderately Unsatisfactory 7.43

H. Restructuring (if any)

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes MadeDO IP

10/02/2014 N MU MU 1.79 Corrected for the delayed project timeframe, including cancellation of US$5 million (SDR 2.5 million), reducing the funding for the

8 Data is estimated from the membership logs of the Community Heath Associations (ASACOs) in Fana and Banamba by the Unit for Decentralization and Devolution Support (CADD) of the MOH.

12

Restructuring Date(s)

Board Approved PDO

Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IPvoucher scheme, project management, local solidarity fund support, and coalition building activities around FP issues, as well as surveys and impact evaluation.

12/29/2016 N MU MU 7.38

Cancelled and reallocated US$13.94 million (SDR 9.94 million), since project activities could not be completed in the remaining time frame. This canceled remaining funds from the activities already reduced in 2014, as well as the funds for the NGO contracting.

I. Disbursement Profile

13

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal1. The project was prepared during a period when Mali was experiencing democratic stability and reasonable economic growth (5.4 percent), with a Gross National Income of US$730 per capita, although commensurate gains in human development were not achieved. In 2011, about 50.6 percent of the 15 million people in the country lived below the threshold of US$1.25 per day. There were economic concerns related to the growing population – the total fertility rate was 6.6 births per woman. Annual population growth was 3.1 percent. 9,10

2. Health sector program. The 2007-2015 Growth and Poverty Reduction Strategy Framework aimed to increase economic growth and reduce poverty – a core pillar was access to health and social services. The project was prepared during the second Health and Social Development Program (PRODESS II), "Programme de Développement Sanitaire et Social 2007-2011". The Ministry of Health and Public Hygiene (MOH) was developing its Ten-Year Plan for Health and Social Development 2014-2023 (PDDS), "Plan Décennal de Développement Sanitaire et Social", as well as medium term planning for PRODESS III (2014-2018).

3. Governance. The PDDS was led by a newly appointed Government team in the MOH. This change reflected the Government’s efforts to renew sector leadership following alleged corruption in the management of Global Fund projects. The Government invited all partners to participate in a comprehensive assessment and investigation by the internal audit branch, i.e. the "Bureau du Vérificateur" leading to an action plan to improve governance. The new team was committed to rebuilding donor confidence, given the increasing governance concerns around country leadership and systems. In 2011, Mali ranked in the 23rd percentile for Government effectiveness and 34th percentile for corruption according to the Worldwide Governance Indicators.11 Further, decentralized management capacity of the health system in the regions and districts of Mali is weaker than at the central level, introducing additional accountability challenges.

4. Donor coordination around reproductive health (RH). During the project preparation, the MOH held donor consultations to harmonize support to PRODESS. The World Bank team engaged actively in these consultations to benefit from the International Health Partnership (IHP+) compact for donor coordination. The first compact for IHP+ was signed in 2009. The discussions to align with donors, highlighted substantial gaps, specifically in the coverage of RH and family planning (FP) interventions. The RH support would contribute to the Government’s road map to reduce maternal mortality, "Feuille de Route pour l’Accélération de la Réduction de la Mortalité Maternelle et Néonatale (2008-2015)". There was also a new donor sub-group on RH to

9 World Development Indicators, World Bank, 2011. 10 Institut National de la Statistique, 2015. Les déterminants de la pauvreté monétaire et non monétaire au Mali en 2011.11 Worldwide Governance Indicators, World Bank, 2011.

1

harmonize investments to address key financing and technical gaps, and a commitment of the Government to use PRODESS III to advance progress toward the FP priorities of Ouagadougou Partnership among Francophone countries and the Global Family Planning agenda (FP2020).

5. Reproductive health (RH) outcomes. Mali was off track to meet the Millennium Development Goals (MDGs), specifically the fifth goal to improve maternal mortality and access to RH. In 2010, maternal mortality was at 630 per 100,000 live births and neonatal mortality was 42 per 1,000 live births.12,13 The 2010 Multiple Indicator Cluster Survey (MICS) reported that only 8 percent of women (ages 15 to 49) used a modern method of contraception. Less than 1 percent of women used long-duration FP methods, specifically Intra-Uterine Devices (IUDs) and implants. About 14 percent of girls were married before the age of 15, and about 33 percent of girls 15 to 19 years of age had started their reproductive life. More than 30 percent of women reported having unmet contraceptive needs. Only about a third (29 percent) of births were attended by skilled health personnel, while about 55 percent of women gave birth in a health center. About 75 percent of women reported having prenatal care, but only about 35 percent of women had the recommended four prenatal care visits. About 3.8 percent of births were cesareans. About 57 percent of women reported initiating breastfeeding within one hour of childbirth. However, exclusive breastfeeding for the first six months was only reported by 20 percent of women. The percent of children under 12 months with complete vaccination coverage was 15 percent.14 During appraisal, several supply and demand-side challenges underlying these health outcomes were described in the context of the project (refer to box 1).

Box 1: Challenges affecting progress on RH outcomes Access to and quality of services. Although Mali had expanded the number of Referral Health

Centers, "Centre de Santé de Référence" (CSREF) and Community Health Centers, "Centres de Santé Communautaire" (CSCOMs) in the districts, access to and quality of services was a challenge, particularly in rural areas. Obstetric care was limited by large distances to travel to health facilities, as well as a lack of human resources, equipment and health commodities. Most births in CSCOMs were performed by under- qualified professionals, without skills to manage complications. Doctors and midwives lacked incentives to provide quality maternal and child care. There was also a weak decentralized drug system - in 2011, 54 percent of CSCOMs reported stock-outs of at least one essential commodity in the previous six months.

Accountability of services. The Government recognized that its health services were not producing the desired performance, in part due to motivation and accountability of service providers. To improve the efficiency of its health spending, the MOH wanted to pilot a Results-Based Financing (RBF) mechanism. During the project preparation, a small pilot was initiated in the district of Dioila.

Demand for services in communities. A main challenge related to influencing attitudes and practices that affect the decision of women to use RH services, as well as strengthening partnerships

12 World Health Organization (WHO), 2016. Trends in maternal mortality: 1990 to 2015.13 World Development Indicators, World Bank, 2010.14 Cellule de Planification et de Statistique (CPS) du secteur santé, développement social et la promotion de la famille (CPS/SSDSPF), Institut National de la Statistique (INSTAT), Enquête par Grappes à Indicateurs Multiples (MICS) 2009 - 2010, Rapport final, Bamako Mali, 2011.

2

between health services, non-Governmental organizations (NGOs) and other groups to better reach communities. There were insufficient resources for community activities to help generate demand for RH services, and weak linkages between the community and health facilities.

Financial barriers. To be effective, interventions would need to address financial barriers to accessing services. Women and their families bear the cost of transport to give birth in a health facility, as well as the cost of the delivery itself. Out-of-pocket health expenditure was 50.1 percent in 2011.15 Mali has been innovative in creating mutual health schemes through a 2010 strategy to extend insurance coverage. Through this strategy local solidarity funds, “Caisses de Solidarité Locales” have been organized to support the emergency transportation of pregnant women to the health center. However, in 2009, there were 120 local solidarity funds and community membership was 21 percent.

6. Rationale for Bank involvement. Up to 2011, the World Bank was involved in the health sector in Mali through financing a Multi-sectoral HIV/AIDS project (2004 to 2011), and a series of Poverty Reduction Strategy Credits that strengthened the health system. In late 2010, the Government asked the World Bank to develop an investment operation to address gaps in RH services coverage to implement the new PRODESS strategy (which was not yet developed) including the envisioned country action plan to advance commitments to the Ouagadougou Partnership. The resulting Strengthening Reproductive Health Project (SRHP) was designed to complement the activities of other donors (refer to Table 1 below). Specifically, the SRHP was to address challenges of access and quality in the poorest regions with the worst RH indicators (Sikasso, Koulikoro and Ségou) and in peri-urban Bamako where there are pockets of poor populations.

15 World Development Indicators, World Bank, 2011.

3

7. Links to strategies. The SRHP (the project) was aligned with the 2008 Country Assistance Strategy and the World Bank’s RH Action Plan (2010-2015). The project was expected to reinforce decentralized human resources in the health system (i.e., in districts and CSCOMs). Importantly, the project intended to support advocacy and resources to help Mali advance country commitments to increase contraceptive use as part of the Ouagadougou Partnership and Global FP2020 Agenda. At the impact level, the project would contribute to reducing unwanted pregnancy, maternal mortality and women’s empowerment to make decisions around the timing and safety of childbirth. Gender dimensions would be addressed in the design of the behavioral change interventions. The project was aligned to pillars of the World Bank’s Africa Strategy: 1) Competitiveness and Employment – as increased use of FP would help space births and reduce the dependency ratio to promote economic growth and employment of women; 2) Vulnerability and Resilience – the utilization of RH services by women would help to increase female productivity and health status; and 3) Governance and Public Sector Capacity – the RBF would strengthen accountability at the central and decentralized level in the health sector.

Table 1: Donor support to RH in Mali (allocated at time of project appraisal)16 Donor Specific Program/Area 2009 2010 2011-2016

The Netherlands Support to RH Program 2,623,828 3,279,785 15,000,000USAID Malaria, RH, HIV/AIDS 15,503,608 15,503,608 15,503,608Spain RH Kayes Region 393,574 393,574 3,200,000Canada Comprehensive life cycle approach in

Kayes, Gao. Timbuktu, Kidal 1,325,610 993,424 61,000,000

France Support to Mopti Region 1,574,297 1,967,871 9,500,000WHO Technical support in RH 1,703,951 1,744,164 1,744,164Belgium Support to Regional Programs 1,574,297 1,967,871 Not

AvailableUNICEF Support to PRODESS 5,354,751 5,068,660 5,096,784UNFPA Support for RH program and

Population Policy 1,200,000 1,200,000 1,200,000

Luxemburg Support to RH and Child Health Programs

3,000,000 3,000,000 9,000,000

World Food Program

Support to health centers in Kayes,Koulikoro, Ségou, Mopti, Gao, Timbuktu, Kidal, Bamako

4,075,585 3,648,388 Notavailable

World Bank RH in Koulikoro, Ségou, Sikasso, Bamako – previous project addressed HIV/AIDS

Previous HIV project 32,700,000

- New RH project30,000,000

Total 71,029,501 71,467,345 151,244,566

1.2 Original Project Development Objectives (PDO) and Key Indicators

16 MOH Medium-term Expenditure Framework, Special Investment Budget (2009-2011).

4

8. The PDO is “to improve access and use of quality reproductive health (RH) services by women of reproductive age, in selected regions of Mali”.17 At approval, four PDO Indicators were assigned to measure achievements of the objective:

PDO Indicator 1: Modern contraceptive use among women aged 15 to 49 years (percent for project intervention regions);

PDO Indicator 2: Pregnant women who have at least four antenatal care visits (percent for project intervention regions);

PDO Indicator 3: Births (deliveries) attended by skilled health personnel (percent for project intervention regions); and

PDO Indicator 4: Postpartum women using modern methods of contraception (number of women in project regions).

1.3 Revised PDO and Key Indicators, and reasons/justification9. The PDO did not change. There was one corrective level 2 restructuring of the project on October 2, 2014 which aimed to address the late start of the project due to the 2012 crisis. In early 2012, the political and security situation in the North of Mali heightened. Then, on March 22nd, a coup d’état toppled the President from office, before the election. The crisis affected the availability of the Government counterparts to start the project. The 2014 restructuring adjusted the expected results of the project in the context of the shortened project time frame and the delayed start of some key activities.

10. In order to measure the increased access to contraceptives, the 2014 restructuring added PDO Indicator 5: Couple Year Protection (CYP) reached through project interventions. Increasing the availability of contraceptives was a specific commitment made by Mali in 2014 and 2015 to the FP2020 Agenda.18 Increasing access to and use of longer duration contraceptive methods such as IUDs and implants was seen as an important strategy to advance results for PDO Indicator 1 in the remaining project time frame. During the restructuring, the target values of PDO Indicator 3 and Intermediate Outcome Indicator 4, were reduced to account for the delayed implementation of the regional action plans to reinforce obstetrical care. The target for Intermediate Outcome Indicator 1 was increased, since the RBF was seen as a strategy to accelerate the project in 2015. Other indicators were not changed, given the plan to intensify the project activities to achieve the expected results in the shorter time frame.

1.4 Main Beneficiaries 11. The main beneficiaries were women in the reproductive age group (15 to 49 years). This included over 2.2 million women 17 The selected regions were Sikasso, Koulikoro, Ségou and peri-urban Bamako.18 Commitments FP2020 Official Report, Mali, December 23, 2015; January 2, 2015; October 5, 2016.

5

(including about 480,000 pregnant women) who would have improved access and use of quality FP, prenatal care, assisted birth, and post-partum care services in the four project regions – Sikasso, Koulikoro, Ségou and peri-urban Bamako.

12. Secondary beneficiaries were to be targeted at both the individual and institutional levels. These included children who benefit from health services in the RH package and adolescents, men, and older women who were beneficiaries of behavioral change communication (BCC) activities to influence attitudes around RH service use—adolescents were to be a key target of BCC to influence social change around FP issues. The project also intended to strengthen the capacity of the health system including health providers in CSCOMs and CSREFs in the regions and the MOH to deliver services. The MOH was expected to have improved capacity in program management, implementation, and monitoring and evaluation (M&E). The project would mobilize change agents (adolescents, men, leaders) to influence FP issues and build capacity of the MOH, specifically at the CSCOM level.

6

1.5 Original Components (as approved)13. The original project design included three components:

14. Component 1: Strengthening Supply and Quality of Reproductive Health (RH) Service (estimated total US$7.5 million). This included the following sub-components:

i. US$1.8 million for a Results-based financing (RBF) pilot in Koulikoro to deliver a package of RH services for about 120,000 women over two years. Based on results, CSCOMs and CSREFs would receive grants to pay for services (including FP, prenatal care, assisted delivery and postnatal care), as well as money which could be reinvested in the health center for quality improvements, such as the purchase of equipment. Community-based associations would be engaged to plan and monitor the RBF, together with the Community Heath Associations (ASACOs) and Regional Health Directions. The MOH would provide overall coordination at the national level.

ii. US$2.0 million to improve the supply of contraceptives through the purchase of a buffer stock of contraceptives, training of personnel managing pharmacies at the district level in inventory management and logistical information systems, as well as training of health personnel providing FP services in CSCOMs and CSREFs. The support would reinforce the supply chain to ensure the availability of contraceptive supplies and services at CSCOM and CSREF level.

iii. US$3.7 million for capacity building in RH and obstetrics care, including equipment, such as birth tables, kits for deliveries, minor facility repairs, and training of doctors, nurses and midwives in safe delivery services. Each region would develop tailored action plans with training and other technical activities to be implemented by CSCOMs and CSREFs.

7

15. Component 2: Increasing Demand for Reproductive Health (RH) Services (estimated total US$15.6 million). This included the following sub-components:

i. US$13.5 million to strengthen outreach services and behavioral change communication (BCC) interventions by contracting of NGOs to partner with health facilities to promote the use of RH services. This involved training and paying community agents “relays” working with CSCOMs and other decentralized Government offices, such as Social Development, to promote and mobilize women to use RH services, including FP, pregnancy and vaccination services. The relay would also distribute contraceptives. NGOs would expand services to the community through mobile units, and conduct large scale BCC to influence attitudes of women, adolescents and others in communities to increase acceptance and use of RH services in CSCOMs.

ii. US$1.6 million for improving financial access. This subcomponent included technical support to assess the management and financing of local solidarity funds, which are community-based insurance schemes run by the Community Heath Association (ASACO) to finance women’s emergency transport costs to deliver in a CSCOM. These schemes are run by membership contributions from families and mobilize the community to provide timely, assisted and free transport of women to CSCOMs and CSREFs to give birth (particularly in rural areas). This subcomponent also planned a pilot voucher scheme, which would provide cash transfers to about 30,000 women living in rural districts of Sikasso, Koulikoro and Ségou to facilitate access to assisted delivery services in CSCOMs and CSREF. These activities would be complemented by the supply-side support to reinforce obstetrics care.

iii. US$0.5 million for promoting a family planning (FP)-conducive environment by strengthening advocacy activities with representatives of Government, civil society, local authorities and religious leaders. The focus was on a coalition, which would convene actors to discuss and support policy issues related to FP.

16. Component 3: Social Accountability, Project Management, and M&E (estimated total US$6.9 million). This component included: project management by the project Coordination Unit (PIU); measurement of the indicators in the results framework and the contracting of an M&E Specialist; health facility surveys; activities to disseminate the 2012 Demographic and Health Survey (DHS); human resources surveys; capacity building to improve the data quality of the routine health information system; and an impact evaluation of project interventions (specifically of the RBF).

1.6 Revised Components17. As briefly noted above, the country faced a political and security crisis in 2012, which resulted in multiple implementation challenges and revisions in activities under the three project components and associated reallocations (refer to Table 2 below). The total amount of the project at appraisal was US$30 million (SDR 19 million). The October 2014 restructuring reduced the Grant to US$25 million (SDR 16.5

8

million), cancelling US$5 million (SDR 2.5 million). The amount cancelled in 2014 was reallocated to the new US$172 million Sahel Women's Empowerment and Demographic Dividend (SWEDD) project to support Sahel regional commitments to FP2020. 19 The reallocation was to leverage additional IDA funds to support FP and women and girls’ empowerment priorities. The decision not to cancel more project funds in 2014 was based on the expectation that the project implementation would intensify in 2015 to accelerate what could be achieved in the remaining time frame. However, implementation continued to be slow in 2015, with the deepening crisis in the country and the high turnover of project staff. The 2012 crisis led Mali into a humanitarian crisis that continues to threaten the political balance and peace and security in the Sahel region and deeply impact the macroeconomic and poverty situation of the country, given a large displacement of people (about 375,000 people)20 from the Northern provinces and following communal violence. In addition to these factors, border areas of the country have become challenged by human and drugs trafficking, smuggling and small arms proliferation.21 The 2014 restructuring was highly positive to advance the project following the 2012 crisis, while the security situation in the country continued to slow implementation throughout the project period.

18. The December 2016 restructuring canceled an additional US$16.1 million (SDR 9.94 million) in uncommitted funds at the end of the project, reducing the final available Grant amount to US$8.9 million (SDR 6.56 million). As a result, the final Grant was about 30 percent of the appraisal estimate. The restructuring followed a decision of the Minister of Finance (MOF) in September 2016 during the World Bank Country Performance Portfolio Review not to request additional extensions of the project Closing Date. The project had shown poor performance for several years, and there was not yet data available on the performance in 2016. A timely decision was required to ensure that the IDA17 funds were not lost to Mali, given the many urgent needs for poverty reduction in the increasingly fragile country context. Hence, in 2016, only contracts which could be completed within the available timeframe and with a high-impact on results were maintained for financing. The amount cancelled at the end of the project was reallocated by the Government to a US$50 million Development Policy Operation (DPO) to foster inclusive growth and support pro-poor decentralized transfers and social protection.22 The reallocation was accompanied by a decision start to develop a new health project using IDA18 resources. This would allow for time to design a project that was adapted to the changed country context and responsive to lessons learned from the slow implementation of the SRHP. In 2016, the reallocation of funds to the DPO ensured the funds from IDA17 were not lost to the country and could be used for poverty reduction in the fragile country situation.

19 The Sahel Women's Empowerment and Demographic Dividend (SWEDD) project P150080 was approved in December 2014. The project includes Mali, Niger, Mauritania, Cote D’Ivoire, Chad and Burkina Faso. Additional financing for Burkina Faso was added in 2015, increasing the total amount of project financing to US$ 205 million. The project is implemented by the Ministry of Planning in Mali.20 The Internal Displacement Monitoring Centre (IDMC), 2016.21 Text adapted from Implementation Status and Results Report, June 2014.22 The First Poverty Reduction and Inclusive Growth Support Operation P157900 was approved in May 2017 for Mali. The operation will be implemented by the MOF.

9

Table 2: Formally revised project scope and activities by componentActivities Restructured changes23 Rationale

2014 2016Component 1: Original allocation US$7.5 million; increased to US$10.2 million in 2014; reduced to US$5.1 million in 2016.Establishment of RBF pilot

Increased RBF credit

Remaining amount cancelled

The RBF credit was increased in anticipation of the RBF starting in 2015. However, the contract for the RBF pilot was only signed in June, 2016, reducing implementation to eight months.

Procurement of contraceptives, RH commodities and equipment

Increased scope


Increased in 2014 to procure more supplies to impact contraceptive access. However, due to delays in the delivery of the RH supplies in the first UNFPA contract, there was no planned second procurement of RH/FP commodities because the MOH and UNFPA agreed there were adequate stocks of commodities following both the first procurement and the contributions of other partners.

Stock management training

No change

Cancelled The stock management training was not implemented due to the low Daily Subsistence Allowance guidelines for training (only resolved in 2016). Further, USAID was providing capacity building support in this area.

Training in FP and obstetrics

No change

- Training in FP remained a key part of the project.

Regional action plans No change


Action plans for RH capacity building activities in CSCOMs and CSREFs were approved in 2015, but not in additional years due to delays in the time line of the plans and capacity challenges planning the budget and activities for each health center.

Repairs to health facilities

No change

Canceled No plan was developed to repair health facilities. However, equipment and supplies were distributed to CSCOM and CSREF according to their 2015 action plans.

Component 2: Original allocation US$15.6 million; reduced to US$9.6 million in 2014; reduced to US$2 million in 2016.Contracting of NGOs (later UNICEF) for BCC and community outreach

No change


In 2014, it was expected that the NGOs would be contracted. However, this did not happen due to failed negotiations around the cost of the contracts after a 21-month procurement process. Funds were redirected to the C4D program, which was an important new strategy in 2015 to achieve the demand-side influence expected in the project’s results logic.

Health voucher scheme

Reduced Cancelled Lack of ownership to implement the scheme as planned in the PAD. Specifically, there was concern around the sustainability in the National Health Policy –these issues were not resolved. However, the Minister of Health still wanted to implement the activity in 2014.

Strengthening of local solidarity funds

Reduced Remaining amount cancelled

Activities were limited to two districts, due to a lack of agreement around the design of the technical support. The Government wanted to use a participatory diagnostic to support the solidarity funds, and not implement the study to assess the design of the funds as proposed in the PAD.

Advocacy activities on FP with a multi-stakeholder coalition


Coalition building activities became part of the new SWEDD project, which could support multi-country coalition building and advocacy activities, toward the Ouagadougou Partnership. 24

23 These are formal changes documented in the 2014 and 2016 project restructuring papers. 24 Component 3, focuses on fostering political commitment and capacity for policy making on reproductive, maternal, neonatal and child health and nutrition issues. This includes supporting the establishment of regional networks with parliamentarians, religious and traditional leaders, and civil society organizations related to demographic dividend issues.

10

Activities Restructured changes Rationale2014 2016

Component 3: Original allocation US$6.9 million; reduced to US$5.2 million in 2014; reduced to US$1.7 million in 2016.Project management and capacity building training


Reduced given the late timeframe of the project. The training activities and the purchase of vehicles needed to take place early in the project implementation to build capacity.

Health facility survey, dissemination of DHS, human resource survey and impact evaluation

Reduced Canceled The delayed timing of the survey due to the crisis. Moreover, there was a lack of willingness to implement the analytical work following the reduction of the amount of money allocated to the surveys in the 2014 restructuring. Further, the impact evaluation was expected to focus on the RBF, which was delayed.

1.7 Other significant changes

19. In addition, there were other changes in response to implementation challenges:i. Fragile country situation. Previously Mali was experiencing a period of relatively

stable democracy and growth. In 2014, Mali was identified as having a fragile country situation by the World Bank and other donors.25 While the project was not focused on the North of Mali, the country situation changed significantly in 2012, further reducing the capacity of the MOH to implement the project. According to the Worldwide Governance Indicators from 2011 to 2012, Mali fell from being in the 26th percentile to 4th percentile globally in terms of political stability and absence of violence and terrorism. In 2014, the situation in Mali had only slightly improved (moving to the 7th percentile). With these changes voice and accountability also fell, from being in the 55th percentile globally in 2011 to the 31st percentile in 2012, improving in 2014 (to the 41st percentile). There was also a reduction in Government effectiveness (falling from 23rd percentile in 2011 to 18th percentile in 2012, and 15th

percentile in 2014) and control of corruption (falling from 34th percentile in 2011 to 24th percentile in 2012, 17th percentile in 2014). 26 This change was discussed in the in the 2014 Restructuring Paper. The World Bank task team planned to support the MOH in regards to managing capacity challenges, as well as to allow for a progressive start-up of the project given that the Government staff were not working during the crisis. The fragile country context meant there was a need for closer supervision to support the MOH to implement the project.

ii. Financial management modified. The weak performance of the FMA was a problem at mid-term review (MTR), leading to a decision not to renew the FMA contract when it expired in July 2015. The major change at MTR was for the project to recruit individual specialists to fill the FMA roles.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

25 Harmonized list of fragile situations, FY13 and FY14.26 Worldwide Governance Indicators, World Bank, 2011, 2012.

11

20. The following summarizes factors related to quality at entry (in Section 3).

21. Soundness of the background analysis. The rationale for the project was strong in terms of supporting RH gaps in the new PRODESS program of the Government, which was expected to start in 2012. The preparation involved consultations with national implementers and donors, and aligning activities with the expected strategic directions and newly formed RH sub-group of donors to ensure the right geographic and intervention focus. It was critical to support the RH priorities of the new PRODESS strategy, including to increase contraceptive utilization and availability in Mali as part of the Ouagadougou Partnership. Analysis also drew on experiences from past projects, studies, best practice knowledge around obstetrics care, an RBF pilot project, the national health accounts exercise, and a 2011 Health Country Status Report. Field missions were conducted to develop descriptions of some component activities, such as the RBF and solidarity funds. Overall the technical analysis was strong, and built on clear rationale for the Bank’s support and lessons learned.

22. Lessons incorporated from previous operations. A key lesson from the previous World Bank financed Health Sector Development Program (1998 to 2006) was that the project design needed to adequately diagnose the determinants of health status and bottlenecks to be addressed by the project. The project activities needed to be designed to comprehensively address key bottlenecks, such as related to the emergency transportation of pregnant women, and the influence of community leaders on the decisions of women to use RH services. In regards to reducing maternal and neonatal mortality, lessons showed this could be done by increasing access to quality obstetrics services in health centers. Success required reducing financial barriers to pay for delivery, as well as training health workers to implement new systems and procedures to provide quality obstetrics care, and correctly identify and refer pregnancy complications. The increased proportion of skilled providers was especially important. In regards to reducing unwanted pregnancies, the key lessons were to make FP part of a broader package of maternal and child health services and ensure sufficient contraceptive availability in the supply chain. Lessons from South Asia emphasized the role of NGOs to implement BCC strategies and outreach services in local communities, influencing community attitudes and willingness to use RH services, as well as reinforcing service delivery in health centers. Lessons from RBF in other countries showed the approach could be used to help the Government to improve the quality of RH services. However, no country had successfully introduced RBF unless it was starting gradually. Hence, there was a need to start small and to generate lessons to improve the approach. The mechanisms for transparent and independent verification of RBF results was especially important. Lessons identified during preparation were directly used to shape the technical design of the project to achieve higher impact.

12

23. At preparation, however, there was a need to better review lessons and diagnosis of bottlenecks related to organizational capacity to inform the design of the implementation arrangements. The previous Health Sector Development Program had experienced implementation delays due to the weak management capacity of the MOH as well as decentralized health actors. A key challenge was the institutional arrangement among national ministries and within the agencies of the MOH. The limited capacity of these agencies to lead the project resulted in a rating of unsatisfactory at the end of the project. Further the project was extended by two years to increase disbursement, but this did not improve the achievement of outcomes.

24. Design of Components. The design of the project components had a comprehensive approach on the demand and supply side to achieve the PDO (refer to Annex 2, figure A2.1). On the supply side, the project aimed to strengthen the capacity of health professionals and facilities (e.g., skills, accountability mechanisms) to provide increased access to quality services, which were measured in the results framework in terms of the number of persons trained in obstetrics and FP, the availability of a stock of contraceptives, and number of women benefiting from services for cesareans, prenatal care and postpartum care. The RBF was key to further reinforce the delivery of quality RH services in Koulikoro, with plans to possibly expand to other regions – it was also expected to help address accountability and governance concerns around health services. On the demand-side, the project design aimed to influence community acceptance of RH and FP issues, access to services delivered by community agents, costs of delivery services, and demand to use services. The contracting of NGOs, advocacy activities, the voucher scheme, RBF score card and solidarity funds were essential to achieve these changes, which were measured in the results framework in terms of the number of women beneficiaries of the voucher scheme, and the number of social marketing and advocacy sessions. Together the improvements in quality and mobilization of new behaviors in the community would improve access and use of quality RH services by women of reproductive age in the selected project regions (Sikasso, Koulikoro, Ségou and peri-urban Bamako) as measured by the PDO Indicators. The design of the components was based directly on the Government’s strategy and country lessons, and had a strong results logic.

25. Design of implementation arrangements. The design of the institutional arrangements for the project had several challenges:

13

i. Design of the Steering Committee. A multisectoral Steering Committee chaired by the Secretary General (SG) of the MOH was made responsible for project oversight. 27

This positioned the leadership of the project below the line of sight of the Minister, despite the importance to multiple sectors. Hence, the Steering Committee was not adequately positioned to ensure Ministerial (or multisectoral) leadership of the project. Further, the steering committee included all of the project implementers. This meant that there was a lack of independent oversight of the project, implementers were overseeing their own actions. Further, the inclusive engagement of actors spread the responsibilities of the project across a large number of national agencies and departments (there were about 14 agencies on the Steering Committee) – while the focus was meant to be on results in the intervention regions, this focused the project on a series of activities by different national implementers. This design also limited the capacity of the Government to strategize results-focused actions to advance the project. Moreover, several of the ministries (e.g., education, family affairs) on the steering committee did not have a clear role in the project, making the rationale for their participation unclear. The project may also have considered including representation from civil society on the Steering Committee given their importance to the project results. Further, the mechanism to support the implementation of activities in the regions was indirect, since activities needed to first pass through several national agencies. The main challenge was the complex design of the Steering Committee to oversee the project and support decentralized implementation.

ii. Design of Project Coordination Unit (PIU). The decision to have a separate PIU was important given the multisectoral focus of the project. However, the placement outside a core unit of the Government meant the project was outside the core planning for PRODESS and could not transfer capacity to the Government at the close of the project. This was also a key lesson of the previous Multi-sectoral HIV/AIDS project (2004 to 2011). Overall, the strength of the PIU to lead implementation was limited, since it was positioned at a nexus between multiple national agencies and required approval of the SG of the MOH and agreement of other national implementers to move activities forward. Overall, the positioning of PIU limited its leadership to advance project activities.

26. Adequacy of Government’s commitment. During the preparation, there was very high commitment from the Minister of Health to the project, which was important given the poor performance of the previous health sector project. However, the same level of commitment was not uniformly there among the technical agencies (Steering Committee to the project) with regard to the design of key project activities as presented in the PAD, specifically the voucher program. The project preparation would have benefited from a more structured collaborative engagement of national and decentralized actors to outline project activities for each component.

27 The document creating the steering committee on December 2, 2013 identified members as representatives from ministries of finance, family, decentralization, education, and social development. It also included representatives from national directorates responsible for population, health, pharmacy and medicine, finance and materials, planning and statistics, and decentralization at the level of the MOH, as well as representatives responsible for budget and finance.

14

27. Assessment of risks. The overall risk associated with the project at appraisal was appropriately rated as High. The concerns identified included the limited implementation capacity of a multisectoral and complex project that is likely to trigger opposition from religious and traditional community groups and governance and fiduciary risks. Substantial training and technical support were proposed to mitigate the implementation capacity risk. The contracting of the FMA along with the use of a project implementation manual with clear procedures was proposed by the World Bank legal and financial management teams to help mitigate governance risks due to recent challenges around the Global Fund.

2.2 Implementation

28. The emergency situation in the North and slow staff recruitment delayed the start of the project. The project experienced an 11-month effectiveness delay since the MOH was not operational during the crisis in 2012. Further, the MOH preferred to build capacity of its national financial management systems, which slowed the recruitment of the FMA to meet effectiveness. The project was finally launched by the Government in December 2013. The delay was due the late creation of the project Steering Committee, and recruitment of the Project Coordinator. A qualified candidate was selected for the Project Coordinator, then late in the recruitment process, an internal candidate was assigned by the MOH, giving an unclear justification around the reason for change. The internal candidate led the PIU for about one year, resigning in January 2015.

29. Early in implementation the project provided support and advocacy to PRODESS III. The World Bank task team worked closely with the Government to help develop and disseminate the RH priorities of the PRODESS III strategy, including Mali’s National Family Planning Action Plan (2014-2018), “Plan D’Action National de Planification Familiale du Mali”, advancing commitments to the Ouagadougou Partnership. The plan included a strong design to influence the RH indicators in Mali, and leverage donor support (such as from The Netherlands, UNFPA and others).

30. During the project period, there were nine SGs and five ministers of health (with a sixth Minister at project closing). This challenged strategic leadership, decision-making and the overall authorizing environment in the Government to support the project. Further, the weak oversight of the Steering Committee (in terms of level and composition of members) meant many activities lacked clear leadership, and never reached consensus to advance actions. For example, the voucher program for pregnant women was never developed with the regions, ASACOs and other stakeholders.

31. The following are additional issues that slowed implementation:

I. Turnover of staff. The turnover and slow recruitment of staff in the PIU meant that the processing of contracts and planning of activities was often stalled.

II. Long contracting processes. Contracting processes took a long time due to difficulties convening review committees, obtaining decisions and signatures, and concerns around conflict of interest.

III. Limited field support. The low Daily Subsistence Allowance for training/supervision (1994 Decree of the Government of Mali) and increasing security constraints

15

discouraged travel of the PIU staff and other implementers outside of Bamako. This was resolved in 2016 when the MOF revised the rules of the Decree.

IV. Planning misalignment. The six-month planning delay in 2014, due to the recruitment of the project Facilitators and the lack of reliable procurement support (refer to Section 2.4), triggered repeatedly late annual planning. Further, the PIU did not conduct joint planning with the PRODESS program or consult the donor sub-group to coordinate activities. Planning required significant technical support from the World Bank to develop quality documents and budgets.

V. Weaknesses in implementation capacity. The project supported detailed action plans prepared by a number of national implementers, with weak collaboration and communication across actors –requests for project activities required many exchanges to finalize and implementers found the design of activities in the PAD as well as the World Bank’s processes for reviewing requests complex.

32. The restructuring in 2014 was a pro-active step to agree on what could be achieved in the original project timeframe. An important strategy to accelerate implementation was to increase the scope of component 1. The project was advancing well on the training of health personnel, and contracting commodities from UNFPA could further reinforce the technical platform at the CSCOM level. The voucher program for pregnant women was retained given the new Minister of Health commitment to pilot the program (refer to Table 2 for the restructuring changes). The 2014 restructuring cancelled and reallocated US$5 million to Mali’s role in the Sahel regional SWEDD project. Through the SWEDD project, Mali could engage with other countries in learning towards the Ouagadougou Partnership. However, the MOH saw it as too early to cancel funds given the recent crisis. Hence, although pro-active to achieve RH results in Mali and Sahel countries, the 2014 restructuring affected the engagement of the project implementers to conduct the daily work of the project.

33. In 2015, the MTR renewed ownership of the project, especially considering many of the staff in the MOH had changed, and there was a new opportunity to engage the MOH after the restructuring. The MTR followed a participatory process with all of the project implementers– it reviewed progress, assessed the relevance of the PDO and project design, and the bottlenecks limiting project performance. The approach resulted in a 100-day action plan to advance project implementation, with defined milestones. The main challenges at MTR were: the agreement of project actors around activities to implement in the remaining project time frame; and the performance of the FMA since in March 2015, again, the Procurement Specialist resigned, and the FMA was unable to find a replacement. An important change at MTR was the recruitment of a new Project Coordinator – the same candidate identified earlier and not hired– and the decision to hire an individual consultant Procurement Specialist.

34. The reinforcement of obstetrics care and South-South learning advanced implementation. Key activities in 2015 included: training on RBF and obstetrics; support to the Regional Health Directions to implement action plans, with awareness activities, reinforcement of FP and obstetrics services in CSCOMs and the scale-up of maternal mortality audits; the contracting of UNFPA for US$2.8 million; field missions to assess the NGOs to be contracted; and the diffusion of the DHS. In early 2016, a South-South exchange with Bangladesh engaged ministerial and technical officials from

16

the MOH, Ministry of Planning, Education, Women’s Affairs, and Religious Affairs. In particular, the South-South learning helped to build commitment to advance project activities with religious leaders, advocacy around RH issues and community-based RH activities.

35. With the PIU fully functional in 2016, implementation accelerated. The World Bank task team provided close supervision to the PIU, focusing on large results-oriented contracts, including: obstetrics related trainings for health professionals; the NGO contracts for BCC activities in each region; the contract for the RBF pilot in Koulikoro region; and a newly developed Memorandum of Understanding (MOU) with UNICEF to implement an accelerated Communication for Development (C4D) program in the four project regions. The RBF contract to accelerate Component 1 was signed in June 2016 for US$1.09 million. In regards to component 2, results were accelerated by the decision to work with UNICEF since the NGO contracts could not be advanced (refer to Section 2.4 on procurement). The MOU with UNICEF was to conduct a large-scale BCC and media campaign (US$1.7 million) to promote RH and FP in complement to its child survival messages, through the multi-actor C4D strategy.

36. The RBF and C4D program were highly successful.

Implementation of the RBF was limited to one cycle of RBF from October to December 2016, but covered 99 percent of health centers in Koulikoro. This was due to delays in implementing training and finalizing the RBF manual, data collection tools and OpenRBF portal. However, implementation reached 205 CSCOMs and 10 CSREF: 57 of the health centers had previous RBF experience and 70 were implementing RBF for the first time. Beneficiary feedback suggests the RBF influenced the motivation of health personnel and that the tripartite design, engaging the ASACO, Mayor (or Regional Health Directorate in the case of the CSREF) and a community-based organization was an effective approach to build decentralized capacity, despite the short-time frame. However, additional cycles are required to resolve issues around the verification process.

The intense nine-month C4D program ran from June 2016 to February 2017 in all of the project regions, with transformative implications. The program targeted women, youth, men, local authorities, health personnel, community health agents, and community leaders across the project regions (reaching an estimated 30 percent of the total population in the four regions) to influence the enabling environment, attitudes and acceptance of RH use, as well as actual use of services by women and adolescents. It included radio and television emissions, video from religious leaders, songs on RH services by famous musicians, peer education, community advocacy and dialogue. Outputs of the RBF and C4D program are detailed in Annex 2.

37. The final restructuring in 2016 cancelled the funds that could not be committed to high-impact activities. Due to the poor performance of the project in previous years, and results not yet being available to demonstrate the success of the C4D and RBF achievements, the MOF decided in September 2016 to reallocate the IDA17 funds to a DPO to address poverty reduction in the increasingly fragile country context. Many of the contracts for vehicles and training of national agencies which had been requested were canceled since it was too late in the project for these resources to

17

contribute to the project’s management capacity. Also, funds for activities, such as the voucher program, NGOs and surveys were canceled. However, for most other activities the funding was reduced since implementation was for a shorter duration (such as the RBF) or at a smaller scale than expected (such as the support to solidarity funds). Overall, there was a concern about losing the funds for development in Mali if the project did not fully disperse, and with so many critical development needs in the country, this was not feasible. The MOH was supportive of the cancellation in view that the funds would be reallocated to a new project in the health sector. The new health project to be financed by the Bank is now under preparation.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

This ICR rates M&E as Modest.

38. M&E design. The design of the Results Framework and M&E arrangements were adequate at project appraisal. The project included indicators to monitor all the parts of the PDO focusing on the quantity of service provision and quality aspects such as skills and access to at least the four recommended prenatal visits. The MOH was to be supported in project management by an M&E Specialist. Key data sources were the MICS and the routine health management information system (HMIS). Additional activities to address data gaps and provide analytical information absent from country systems were assigned to the Planning and Statistics Unit (CPS), such as facility-based surveys to collect data on intermediate indicator 2. The following could have strengthened the results framework:

Indicators to assess changes in attitudes and citizen engagement. Setting some priority milestones regarding attitudes on RH issues could have helped the project track changes in the demand to use services in component 2, given the importance of BCC to the results logic. Further, milestones related to citizen engagement, while not required at the time of project preparation, could have helped to track issues relating to the accountability of the project to communities.

39. M&E implementation. The M&E data were not collected as planned for the baseline surveys, since it was not possible to collect this due to the changed security situation in the country. In 2014, CPS was discouraged to implement the surveys, given the budget was reduced in the 2014 restructuring—the dissatisfaction of the implementers with the restricting stalled the implementation of the M&E data collection. Analytical activities that were consequently never implemented included the health facility surveys, human resources surveys, and the impact evaluation. Further, the M&E Specialist was only recruited in late 2015. Technical support to collect indicators was part of the MTR. The available data on the indicators was collected from the HMIS as well as surveys and studies from other partners in the country. This provided data to inform most of the project indicators. The HMIS data, however, is not as reliable as the surveys expected to support the project. For example, it is subject to reporting errors and variations in calculations of numerators and denominators.

40. After recruitment in late 2015, the M&E Specialist had mixed performance. Difficulties in monitoring project indicators were accompanied by frequent questions about definitions that had been specified at project appraisal. Data on the indicators in the

18

results framework were not available for timely project monitoring and not disaggregated to present data for the project regions or peri-urban Bamako. However, the engagement of UNICEF and the RBF pilot helped to ensure data for the end of the project. A survey was conducted by the MOH and UNICEF in February 2017 to collect data to estimate project indicators at closing, since the MICS survey data was from 2015, and more recent data was required to assess progress in 2016. The RBF also provides data on health service use and the quality of services in CSCOM and CSREF, although only from one implementation cycle. By the end of the project, most of the data gaps were filled due to the UNICEF survey at the end of the project, the RBF and the use of the HMIS.

41. M&E utilization. The Steering Committee and PIU did not use M&E data for decision-making. Overall, their results management role was very weak. Data could have been used to diagnose problems around indicators in the results framework, and focus planning on critical RH gaps, among other decisions. The project could have benefited from a package of tools to build results management capacity and encourage routine field supervision. However, the introduction of the RBF started to change the practice of data utilization in the MOH in 2016. Importantly, it provided tools to support decentralized data management. The UNICEF C4D activities and MOH support to local solidarity funds also involved participatory diagnostics to inform decision-making and program design in 2015 and 2016. At the end of the project the RBF, C4D program and solidarity fund support had started to shift the use of M&E data for decision-making.

2.4 Safeguard and Fiduciary Compliance

42. Safeguards. The project was classified as environmental category “B”, given the possible risk associated with medical waste and general health waste. Accordingly, in 2011 during project preparation, the unit responsible for hygiene and sanitation of the MOH updated the Government’s Medical Waste Management Plan with support from the Global Vaccine Alliance (GAVI), which includes funding from WHO, UNICEF, and the Global Fund and some bilateral donors). However, the PIU set-up delayed implementation of the Plan by about 10 months – the same plan was used for the SWEDD operation in 2014.

43. Financial management. At preparation, while the Government was making efforts to address governance concerns around the use of country financial management systems, the project was required to recruit a FMA (as discussed above). The FMA was to provide technical support and transfer knowledge to the MOH team to enable them to take over the project implementation by MTR. The financial management capacity of the MOH was to be assessed by mid-term to determine whether it was adequate to handle fiduciary tasks and the contract of the FMA would be revised accordingly. The recruitment of the FMA was part of the effectiveness delay – the contract was signed in March 2013 and the agency started in July 2013. Once in place, the FMA had poor performance, and did not reinforce the capacity of the MOH due to poor oversight by the Steering Committee, high turnover of staff, vacancies of key positions in the FMA, and a lack of competence in World Bank procedures.

19

44. While regular financial supervision was conducted by the World Bank, financial management was not satisfactory until the final year of the project. Further, the pace of addressing action plan items to upgrade the rating was slow. The FMA did not have stable staff to support the project, and there was very little oversight of the Steering Committee or SG, which led to the FMA outsourcing staff time to other donor projects, and not completing their required functions –the FMA paid fines to account for the lost staff time. Further, there were continuous needs for training of staff in Bank procedures, and the required management database was not installed until 2014. Despite this poor performance, the FMA contract was renewed in July 2014 for one year to avoid a break in the Grant agreement, and in hopes of having a complete PIU as soon as possible. At MTR, the audit reports of the 2013-2014 project accounts had not yet been transmitted to the Bank due to the turnover of staff in the FMA. Hence, there was a decision not to renew the FMA contract in July 2015, and the Finance and Materials Directorate (DFM) of the MOH would recruit individual specialists to support the project. During the last supervision mission of the project, there were no remaining issues related to financial management. While a key barrier throughout implementation, in June 2016 the financial management rating was upgraded to satisfactory due to the recruitment of qualified individual specialists.

45. Procurement. Procurement was also part of the responsibilities of the FMA. The Procurement Specialist was hired in December 2013, and resigned in February 2014. In March 2015, again, another procurement specialist resigned, and the FMA was unable to find a replacement. The firm could not find a long-term candidate for the established salary level. The staff turnovers delayed the approval of procurement plans, as well as the preparation of key contracts, such as for the vehicles, the health facility surveys, the NGOs, and the RBF. Furthermore, the low quality of financial documents often delayed Bank approval and related disbursements. In addition, the changes in leadership in the MOH slowed the signature of documents. For example, in 2014, the project account was inactive for eight months due to changes in MOH leadership, and the MOF needed to approve the signature change. The project’s implementation manual was revised to allow for delegation in the place of absence of the signing official, which helped handle issues when PIU staff were on leave, and when the Project Coordinator resigned – however, the signature of the MOH leadership remained a constant problem, with the high ministry turnover. At the time of the MTR, procurement was a main challenge leading to the decision to recruit an individual Procurement Specialist dedicated to the project.

46. In January 2016, the PIU finally had sufficient procurement capacity to advance large contracts. This resulted in the acceleration of the implementation of the project activities. The remaining time in the project focused on high-impact contracts, given the decision of the MOF to close the project (refer to Section 2.1). However, the NGO contracts in the project regions failed at the end of the 21-month procurement process, since the NGOs wanted additional budget to implement the accelerated 10-month program, which was not possible according to procurement rules. Hence, the negotiation with the NGOs failed at the end of the project as a result of quality and cost based selection rules – faster processing could have ensured the recruitment of the NGOs. Long procurement processes were a key bottleneck of the project, only in 2016 was the procurement rating of the project upgraded to moderately satisfactory, from moderately unsatisfactory.

20

2.5 Post-completion Operation/Next Phase47. Given the fragile country context, lessons from this operation need to be distilled prior to developing a new project. In the next year, the focus will be on analytical work and discussions with the Government to design the next operation. In terms of transition arrangements, The Netherlands has committed to support the RBF pilot in Koulikoro.

48. The priority of improving RH service access, use and quality is still highly relevant. However, there is a need to review the institutional arrangements, specifically the actors (from Government and other stakeholders) to involve in the next project at the national and decentralized levels. The design of the Steering Committee and placement of the PIU were key constraints that affected performance of this project. The next project should consider having a few carefully selected ministerial and non-Government leaders in the Steering Committee, with a PIU that can align to Government strategy and manage decentralized activities, insulated from political changes. Project facilitators could be placed in each region to build capacity within the Government. The next project should continue to collaborate with UNICEF and other agencies to build decentralized capacity in the regions and districts – this was an excellent strategy to provider closer technical support. Also, important is to re-assess the use of country financial management systems, since the use of an FMA proved to be unworkable.

49. Moreover, the new project should have a strong emphasis on demand-side activities – in addition, to decentralized capacity building in the districts. The C4D program was successful because it focused on mobilizing specific change agents, at national, regional and local levels who could influence binding constraints related to social and cultural norms, and thus accelerate results quickly. The next project should build on this approach. An institutional capacity analysis could help to better understand specific demand-side constraints as well as issues of fragility (and how they relate to supply-side issues) to improve health service utilization in local communities. The next project could then scale-up the successful activities of this project, including the community-based platform (integrating CSCOMs, community health agents, adolescents, community and religious leaders, etc.) established by the C4D program, as well as the quality, motivation and accountability tools piloted by the RBF. Leadership building strategies such as the Bangladesh Knowledge Exchange and lessons learned from the Exchange (for example, around the design of door-to-door community-based health service delivery) could also be implemented by the next operation. The opportunity is to integrate successful approaches into a package that can help to transform key health outcome indicators in Mali, as well as build on strategies to reinforce leadership and health system capacities in the fragile country context.

3. Assessment of Outcomes

50. This assessment uses a split evaluation approach given the 2014 project restructuring which changed PDO targets and added a PDO Indicator. Phase 1 covers a period from March 2012 to October 2014 and Phase 2 covers a period from October 2014 to project closing in February 2017.

3.1 Relevance of Objectives, Design and ImplementationRating: The overall relevance rating for Phase 1 is Modest and for Phase 2 Substantial.

21

51. Relevance of PDO (High for Phase 1 and Phase 2). The PDO was and remains highly relevant given the sector context, despite the changing country circumstances. The project’s PDO was consistent throughout implementation with the Bank’s CAS and with the country strategies defined by the PDDS 2014-2023 and PRODESS III (2014-2018). The PDO also remained relevant to the World Bank’s strategy, and had synergies with other West African World Bank operations (specifically the SWEDD project) and alignment to the Ouagadougou Partnership on FP of Francophone West African countries, and the Sustainable Development Goals. The project also directly supported Mali’s National Family Planning Action Plan, advancing its commitments to FP2020. In 2014, the Government of Mali committed to increase the rate of contraceptive use to at least 15 percent by 2018. In 2015, the Government committed to increasing the availability and accessibility of contraceptives throughout the country, and increase communication campaigns to promote FP use, specially focusing on adolescents. These commitments were advanced by the project.

52. Demographic challenges were and continue to be major constraints to the achievement of development outcomes. This was reinforced in the recent Country Partnership Framework (CPF), 2016-2019, which describes how “high levels of population growth present a significant economic and social challenge” given that “Mali has one of the highest fertility rates in the world with 6.9 children per women leading to an exceptionally high rate of population growth of 3.0 percent.” The total population is expected to nearly double between now and 2030. The continued high total fertility rate signals a stalled demographic transition and narrowing window of opportunity to seize the first demographic dividend through a rapid change in the population and structure whereby the ratio of working age population to dependents (older/younger) decreases.

53. Relevance of project design (Modest for Phase 1 and Substantial for Phase 2). The design concept of the project was highly relevant to address the PDO and was technically informed by a review of the best ways to improve RH and existing country experiences (refer to Section 2.1). Further, gender issues were taken into account, informing the design of BCC campaigns. However, the design of the implementation arrangements did not adequately address governance, coordination and decentralized implementation needs, particularly in Phase 1. In Phase 2, key design changes helped to optimize the existing institutional arrangements (described below).

22

Design of results logic. As described earlier, the results logic of the project had a comprehensive approach on both the demand and supply sides.

Design of institutional arrangements. As described in Section 2.1, the design of the Steering Committee and placement of the PIU limited project oversight, leadership and decision-making. Further, while the project concept was technically strong, there was a need for greater attention to constraints related to the collaboration among actors in the MOH and other sectors in the project design. At MTR, the project made two key design changes, which enabled the project to maximize results in Phase 2 (compared to Phase 1). These changes did not require formal project restructuring—an extensive restructuring (beyond the changes made in 2014) would have resulted in additional delays, considering the ongoing leadership changes in the MOH. The Government had already faced many challenges to set-up the PIU during a period of crisis in the country. The two key design changes were the following:

i. The project did not renew the FMA contract, and instead recruited individual specialists to fill financial management and procurement roles. This removed a key constraint affecting performance. Further, at MTR a highly-qualified Project Coordinator was in place to accelerate the project.

ii. The project collaborated with UNICEF to rapidly scale-up implementation of demand-side activities to address constraints related to social norms and behaviors, which were key to increase RH service utilization – as well as complement supply-side support in obstetrics and FP training and equipment. Partnering with UNICEF had the added value of building capacity within the Government to deliver health services in collaboration with stakeholders at national, regional, district and community level. Engaging UNICEF as an implementer and coordinator removed the organizational arrangement constraints previously slowing project implementation.

54. Relevance of project implementation (Modest for Phase 1 and Substantial for Phase 2). Several aspects of the implementation reduced the project’s relevance within the current structure of Mali’s health sector. For example, the project activities were spread across a large number of national implementers, with weak collaboration—and the Steering Committee provided minimal accountability, given it was managed as a collective by the implementers. The delayed start of the project, and changed security situation in the country were key factors in Phase 1. Phase 2 worked within the existing institutional arrangement to maximize the project results, particularly transformative was the C4D support in Component 2.

55. Characteristics of implementation during each phase are described below:Phase 1 Phase 2

Component 1

Implementation was reduced by delays in the launch of the RBF pilot. However, training in obstetrics and FP had started.

Implementation was maximized by the distribution of contraceptives and equipment to CSCOMs, the scale-up of obstetrics and FP training across the project regions, the realization of the RBF pilot in 99 percent of health centers of Koulikoro, and the institutionalization of maternal mortality audits.

Component Implementation was reduced by the Implementation was maximized by the

23

2 delayed contracting of the community-based activities and non-realization of the voucher program.

successful pilot of solidarity fund support in two districts, the Bangladesh exchange visit, and the widespread implementation of the C4D program28 across all of the project regions.

Component 3

Implementation was reduced by the failure to collect data as planned (such as surveys) to inform the project. However, activities to support the development of PRODESS III the National Family Planning Action Plan were significant achievements.

Implementation was increased due to the availability of M&E data from the survey conducted by UNICEF. Further the PIU was highly functional by the end of the project.

3.2 Achievement of Project Development ObjectivesRating: Project efficacy is rated as Modest for Phase 1 and Substantial for Phase 2. 56. The targets for the PDO Indicators in the project regions were largely met due to intense activities in the final years of the project. This Section assesses the achievement of the original PDO Indicators and targets, as well as those revised in the 2014 restructuring. The indicators are reviewed in terms of improving access, use and quality of RH services by women of reproductive age:

PDO Indicator 1: Modern contraceptive use among women aged 15 to 49 years

The indicator assesses the increased use FP services – an increase suggests that contraceptives were made available to prevent unwanted pregnancies, and demand was increased in the community.Key Intermediate Outcome Indicators include:#1 Women provided access to a package of RH services;#3 Health personnel trained to provide FPs services;#8 People attending social marketing interventions that promote FP;#10 Advocacy sessions held with civil society, political, religious and traditional leaders on demographic issues and FP.

PDO Indicator 2: Pregnant women who have at least four antenatal care visits

The indicator assesses access to quality RH services in CSCOMs and CSREFs as a result of the project. Receiving the recommended four visits suggests women have access to a package of services to manage their pregnancy.Key Intermediate Outcome Indicators include:#1 Women provided access to a package of RH services;#4 Pregnant women receiving antenatal care during a visit to a health care provider;#7 Health personnel trained to provide obstetric services;#8 People attending social marketing interventions that promote FP.

PDO Indicator 3: Births (deliveries) attended by skilled health personnel

The indicator assesses access to quality birthing services in CSCOMs and CSREFs to ensure safe delivery and prevent maternal and neonatal mortality.Key Intermediate Outcome Indicators include:#1 Women provided access to a package of RH services;#5 Deliveries by cesarean section;#7 Health personnel trained to provide obstetric services;#9 Pregnant women provided with vouchers to access selected services and who use them.

PDO Indicator 4: Postpartum women using modern

The indicator assesses access to and use of RH services postpartum. An increase would suggest women had access to services to manage pregnancy complications and space childbirth.Key Intermediate Outcome Indicators included:

28 The UNICEF program was designed to build institutional capacity for coordination across ministries (including Health, Communication, Social Development and Finance) and reinforce multi-stakeholder coordination in health districts (of health personnel, administrative authorities, local leaders, community agents, youth, other).

24

methods of contraception

#3 Health personnel trained to provide FPs services;#6 Women provided with postpartum care within the first 7 days of delivering in a health facility.

PDO Indicator 5: CYP reached through project interventions

The indicator assesses access to contraceptives to deliver FP services in local communities. A range of contraceptive methods need to be available in CSCOMs to enable access.#2 Health facilities that experience 2 or more weeks’ stock-out of contraceptives during the 2 weeks preceding the survey.

57. In terms of achieving the PDO, Component 1 of the project focused on "increasing access to quality RH services" (FP, prenatal care, delivery, and postpartum care) for women by: increasing access to trained personnel in CSCOMs and CSREFs to deliver services, according to national standards; and ensuring the availability of supplies, contraceptives and equipment to meet demand. The RBF specifically increased access to quality RH services in Koulikoro region. Component 2 focused on "increasing the use of RH services" by women through advocacy and social marketing sessions and reinforcement of local solidarity funds. The C4D activities in particular provided intense social mobilization and communication to address social barriers to increase the use of RH services— the scale of the program was unique and there was no comparable initiative implemented in Mali in 2016.

PDO Indicator 1: Modern contraceptive use among women 15 to 49 years old (Modest for Phase 1 and Substantial for Phase 2).58. The use of family planning (FP) services in the project regions surpassed Mali’s commitment to FP2020. The 2015 MICS data29 show that the rate of contraceptive use (weighted average for the project regions) has increased to 19 percent from a baseline of 10 percent in 2010 in the project regions. The February 2017 project survey30 suggests a further increase from 19 percent to about 22 percent in the project regions. This compares to a national increase from 8 to 16 percent from 2010 to 2015 (MICS); and an increase to 21 percent shown by the HMIS in 2016. The results surpass the 15 percent target for 2017 set by the project, as well as Mali’s FP2020 target for 2018. The numbers of women using services from the HMIS data31 show a similar increase in the use of modern contraceptives in the project regions, with an increase in 2016: 174,147 in 2012; 248,290 in 2013; 310,113 in 2014; 284,864 in 2015; and 498,884 in 2016. In each project region, the preliminary data from the 2016 HMIS show an increased use of modern contraceptives, compared to the 2010 MICS baseline: 24 percent in Sikasso (baseline 7 percent); 25 percent in Koulikoro (baseline 9 percent); 22 percent in Ségou (baseline 9 percent); and 33 percent in Bamako (baseline 18 percent). This compares to 11 percent in Tombouctou, 13 percent in Mopti, 17 percent in Kayes, 3 percent in Gao, and 2 percent in Kidal (HMIS 2016). The national increase in the

29 United Nations Children's Fund (UNICEF), Mali Multiple Indicator Cluster Survey 2015. Data on indicators in the project regions was shared by UNICEF in February 2017, since final survey report was not yet available.30 Ministry of Health and Public Hygiene (MOH) and UNICEF, February 2017. “Étude Finale du projet : Amélioration de l’accès et de l’utilisation de services de santé de la reproduction de qualité pour les femmes en âge de procréer dans les régions de Koulikoro, Sikasso, Ségou et le District de Bamako."31 Système Local d'Informations Sanitaires, Local Heath Management Information System (HMIS), 2012, 2013, 2014, 2015, and preliminary data from 2016.

25

indicator is largely due to the increase in the project regions. However, the national indicator has also increased due to support of other donors, such the French and Spanish development agencies in Kayes and Mopti. The indicator spiked in 2016 very likely due to the intense C4D activities to increase demand, since this was the main program of its type in the country. The C4D activities, while focused on the project regions, were communicated nationally in the media, and hence had a spillover effect in all regions of the country.

59. Since 2013, the project has influenced contraceptive use through advocacy and activities to strengthen the supply and quality of RH services. The project supported the development of PRODESS influencing priorities and commitment to increase contraceptive use. Between 2014 and 2016, contraceptives were purchased from UNFPA and distributed to health facilities, and 3,530 health personnel were trained to provide FP services. A 2015 UNFPA survey shows 93 percent of health personnel are trained in FP service provision nationally, and 75 percent of CSCOM and CSREF personnel were trained in the last year.32 The project likely influenced this increase in coverage. The launch of the RBF was also significant, although only one cycle was implemented – through the RBF pilot33, about 20,641 women used modern contraceptive methods in CSCOMs in 2016.

60. In the last years of the project, community-based activities increased the demand for contraceptive use. Community outreach and BCC conducted by the C4D program catalyzed women to go to the health centers to use modern contraceptives. The number of people (men, women and adolescents) who attended social marketing interventions promoting FP was 742,202. For example, 51,999 women and men were engaged by caravans traveling village to village to promote FP use, known religious leaders also promoted the use of FP in the media, and 84,077 youth were engaged in schools (Annex 2 includes the outputs of the C4D program). The data from CSCOM and CSREF in the project regions reported 582,820 women used RH services between July 1 2016 and January 21, 2017 as a result of the C4D activities – this is about a doubling of the expected number, attributable to the C4D activities.

PDO Indicator 2: Pregnant women who have at least 4 antenatal care visits (Modest for Phase 1 and Substantial for Phase 2).61. project activities likely provided more women access to quality pregnancy services. The target of 43 percent of women having at least 4 antenatal/prenatal visits was just attained for the project regions by 2015 per MICS data (examining the weighted average for the project regions). In the project regions, the MICS shows an increase from 38 percent to 43 percent from 2010 to 2015. This compares to a more modest increase nationally, from 35 to 38 percent. The February 2017 survey found 63 percent of women in the project regions had more than 4 prenatal care visits – exceeding the project target by 26 percent. The HMIS data report 215,629 women having 4 prenatal visits nationally (about 31 percent) in 2016. The indicator was not available by region or for previous

32 Programme UNFPA SUPPLIES “Rapport final de l’enquête 2015 sur la disponibilité des produits contraceptifs et produits de santé maternelle, Mali”, August 2016.33 KIT-CORDAID-CGIC, "Rapport final du projet pilote « PRSR/FBR » FBR-Financement Basé sur les Résultats – Approche pour accélérer les résultats en Santé de la Reproduction", April 2017.

26

years for comparison. This is a new indicator in the HMIS, hence the data collection protocols remain under development, limiting reliability.

62. Increases in prenatal care were supported by project activities to build capacity to deliver obstetric care, and community activities to increase demand to use services. Between 2014 to 2016, 806 health practitioners were trained to provide obstetrics care in CSCOMs. In 2015, all of the project regions implemented action plans, including advocacy and training to improve prenatal care in CSCOMs in all of the regions. In 2016, through the RBF pilot about 5,666 pregnant women in Koulikoro received at least 4 prenatal care visits – the RBF used checklists and other tools to improve the quality of pregnancy services. Further, the large-scale communication activities of the C4D program promoted and catalyzed women to use prenatal care services, engaging 306 Husband Champions to bring their wives to the CSCOMs for prenatal care (for example). The February 2017 survey shows that on average 90 percent of pregnant women in the project regions reported having at least one prenatal care visit in 2016. The HMIS shows an overall increase in women accessing prenatal care visits, with numbers doubling in 2016, which is consistent with the intensification of project activities: 185,120 in 2014; 190,398 in 2015; and 462,460 in 2016. The intense promotion of 4 visits of prenatal care by the C4D activities very likely doubled the use of prenatal care in 2016, building on training and quality improvements.

PDO Indicator 3: Births (deliveries) attended by skilled health personnel (Modest for Phase 1 and Substantial for Phase 2).63. Project activities likely influenced more women receiving quality delivery services. The 2015 MICS data indicated that about 50 percent of pregnant women (weighted average for the project regions) had childbirth assisted by skilled health personnel, against a revised target value of 48 percent in 2017 and an original target of 50 percent (pre-2014 restructuring) and a baseline of 35 percent in 2010. The MICS shows a slightly more modest national increase in the indicator from 2010 to 2015, from 29 to 44 percent. The end of project survey in February 2017 in the project regions indicated that 64 percent of women reported giving birth at a CSCOM or CSREF – an increase of almost 30 percent from the MICS 2010 baseline. However, the survey did not ask about the assistance by skilled health personnel. Nevertheless, the HMIS data on assisted births in health facilities suggest a similar increase in the indicator for the project regions: 173,623 women (50.6 percent) in 2013; 205,628 women (53.2 percent) in 2014; 208,978 (55.1 percent) in 2015; 217,377 in 2016 (60.8 percent). Further, the 2016 HMIS data shows an increase in the indicator across the project regions, compared to the 2010 MICS baseline: 33 percent in Sikasso (baseline 12 percent); 49 percent in Koulikoro (baseline 29 percent); 43 percent in Ségou (baseline 27 percent); and 94 percent in Bamako (baseline 90 percent). This compares to 30.6 percent in Mopti, 43.8 percent in Kayes, 55.9 percent in Gao, and 59.1 percent in Kidal per the 2016 HMIS. Data for Tombouctou were not available due to the ongoing conflict. The HMIS also show a national increase in births attended by a skilled health personnel (50 percent in 2016).

64. The project likely contributed to improvement in this indicator between 2014 and 2017, and to the related World Bank Core Sector Indicator, number of deliveries attended by skilled health personnel. Outputs that have contributed to this indicator include the training of personnel to provide obstetric service, and procurement

27

of equipment and supplies for health centers. The training included detection of high risk pregnancy, anesthesiology, and support to scale-up the use of maternal mortality audits to all CSCOMs and CSREFs in the project regions. Beneficiary feedback was that the mortality audits increased confidence of the community to use skill birthing services, and the assessment of deaths result in quality and accountability improvements in CSCOMs. However, the project was only modestly successful to increase the practices of cesareans. The HMIS reports only 18,461 cesareans in 2016 in the project regions, which was below the targeted 30,000. The RBF pilot provided assisted delivery services (including cesareans) to about 12,653 women—and was likely important to help improve the quality and referral of pregnancy complications from CSCOMs to CSREFs. UNICEF’s C4D program further encouraged women to have their births attended by skilled health personnel. Further, the reinforcement of the ASACO in two districts, increased membership of solidarity funds from 21 percent in 2009 to 50 percent in 2016, ensuring about 400 women benefited from emergency transport to CSCOMs for birthing services. The reinforcement of obstetrics care and scaled-up use of maternal mortality audits (institutionalized in 2015) will likely increase access to quality pregnancy care and reduce maternal mortality in the years beyond the project. Moreover, the training of personnel likely had a spill-over effect nationally, beyond the project regions, given the mobility of personnel and the complementary activities of other PRODESS donors in other regions of Mali.

PDO Indicator 4: Postpartum women using modern methods of contraception (Modest for Phase 1 and Modest for Phase 2).65. The increase in the number of postpartum women using modern methods of contraception was likely modest, although not measured. The target for 2017 was to reach at least 300,000 women. However, the indicator was not measured as the data depended on the contracting of the NGOs. The HMIS shows the percent of women receiving postpartum care in the project regions has increased since the 2010 baseline of 54.6 percent (177,805) – including an increase of about 5 points during the last year of project implementation: 203,118 (60.5 percent) in 2011; 213,924 (59.6 percent) in 2012; 220,894 in 2013 (64.4 percent); 245,938 (65.9 percent) in 2014; 253,305 (66.7 percent) in 2015; 276,521 (71.5 percent) in 2016.34 The FP training provided by the project, since 2014 included modules to strengthen FP provision during postpartum care. Further, advocacy of the project around the importance of engaging NGOs to provide a range of RH services (including FP postpartum) has helped to make this a strategy of the National Family Planning Action Plan. At the end of the project, the C4D activities likely influenced on this indicator, since FP use postpartum was promoted in the communication modules.

PDO Indicator 5: CYP reached through project interventions (Modest for Phase 1 and Substantial for Phase 2).66. The CYP increase during the project period suggesting greater access to contraceptives. According to data from the HMIS, the cumulative CYP for 2015 to 2016 was 1,349,168 for the project regions, compared to a target of 900,000 for 2017 and a baseline value of 0 at the 2014 restructuring. The numbers by region were also consistently high (322,190 in Ségou; 354,751 in Sikasso; 345,405 in Koulikoro; 326,821

34 No data was collected on whether this was within the first 7 day following delivery.

28

in Bamako). This compares with a total CYP of 395,413 in the project regions in 2014. The project directly contributed to 468,000 CYP. With the cancellation of a second UNFPA contract, not enough contraceptives were purchased to meet the target through the project alone. However, the contraceptives entered a multi-donor pool to achieve Mali’s FP2020 commitment, and the MOH and donor sub-group agreed that the supply was sufficiently met. Through the contract with UNFPA, contraceptives were distributed to health facilities in the project regions, including long-duration methods (65,000 implant kits, 14,100 IUD) and short-term methods (475,352 cycles of contraceptive pills, 19,900 gross boxes of condoms). Overall the attribution of the project to this indicator is strong.

67. Stock of long-duration contraceptive methods increased nationally. While data on the stock of RH commodities was not collected by the project, data from USAID35 shows the national stock of long-duration contraceptive methods was 9.3 percent for IUDs and 4 percent for implants in January 2017 at the close of the project, compared to 21 percent and 8.6 percent in January 2015. The project likely contributed to this increased availability, given the focus on long-duration methods in terms of procurement, but also advocacy to increase contraceptive coverage.

3.3 EfficiencyRating: Given there is no split rating for efficiency, efficiency is rated Modest for the project overall.

68. The details in this Section link to the economic analysis in Annex 3.

69. Technical efficiency was high. The interventions included in the project were technically sound. Components 1 and 2 together formed a comprehensive RH program, addressing both demand and supply-side constraints to the utilization, access and quality of RH. This type of program has been identified as one of the most cost-effective development interventions to date. The ICR cost-benefit analysis (CBA) of the project found an economic rate of return of 42 percent.

Estimated benefit of family planning (FP) interventions. In terms of the investments in FP, the conservative estimates of the impact from the CBA found that the CYP contributed by the project in 2015 and 2016 avoided an estimated 134,784 pregnancies36, of which 72,783 would have resulted in live births, 43,130 would have resulted in abortions (all unsafe abortions), 668 women would have died in pregnancy and there would have been 7,915 neonatal deaths37. The estimated cost of care avoided from the FP investment was about US$5.57 million (US$4.86 million from avoided births and US$0.7 million from avoided abortions). The productivity loss avoided was estimated to be US$1.97 million for the maternal deaths and consequent lost participation of women.

Estimated benefit of skilled delivery (pregnancy services). The CBA estimated that the project contributed to the delivery services of 143,198 women – about 30

35 Data from USAID OSPSANTE portal, Monitoring Tool for Health Products, 2017.36 Darroch JE and Singh S, Estimating Unintended Pregnancies Averted from Couple-Years of Protection (CYP), New York: Guttmacher Institute, 2011, p. 5.37 Ibid, Table 1, p.8 for Western Africa; neonatal death rate estimate from Guttmacher, 2012, p. 16

29

percent of the reported HMIS beneficiaries in 2016 (70,715) and 20 percent (72,483) in 2017. This resulted in care costs of about US$10.79 million, and is estimated to have averted about 266 maternal deaths and 3,460 neonatal deaths. The total productivity loss averted from these deaths was US$41.3 million (US$38.35 million for neonatal deaths, and US$2.94 million from maternal deaths).

70. The rationale for public financing remained high. The budget gap for the PRODESS program increased during the project due to reductions in domestic and external funding following the 2012 crisis. Hence, despite implementation delays, the project was an important source of financing for PRODESS and the National Family Planning Action Plan. PRODESS benefited through the technical reinforcement of the health sector, procurement of commodities, and the decentralized health system capacity building through the RBF, solidarity fund support and C4D program. Through these activities, the project helped to finance the advance of Mali’s commitments to FP2020 in 2014 and 2015. The project cost was about US$4.28 million in 2015. In the same period, USAID spent US$2.9 million on FP and UNFPA spent about US$0.58 million. In 2016/17, the project spent about US$4.26 million, USAID proposed a program of US$8.2 million for FP in 2016 and UNFPA a program of US$1.02 million.38 , 39, 40 Hence, project contributions were significant in comparison to other external financing post-crisis in the same regions – all donor support in the project regions slowed down during the crisis, focusing on the North.

71. Implementation strategies were highly efficient in the final year of the project. Despite long delays, the stability of the PIU in the final years of the project marked the acceleration of project activities. The project used highly efficient implementation strategies to quickly advance results, including the decision to procure commodities from UNFPA, enter an MOU with UNICEF for the community activities, and the launch of the RBF pilot. The action plan and supervision support following the MTR played an important role in boosting efficiency at the end of the project. Efficiency spiked with the recruitment of a qualified Procurement Specialist. The delayed time line however meant some activities with a high impact on disbursement could not be achieved (e.g., the NGO contracts) and the duration of implementation was limited.

72. Allocative efficiency of the project was substantial after the 2014 restructuring. There were increased resources allocated to Component 1 which increased from 25 percent to 41 percent of total budget of the project after restructuring. The health system benefited from necessary equipment and commodities to reinforce health centers. Moreover, it supported the acquisition of contraceptives that directly supported an increase in CYP, toward the PDO; and strengthened the skills of health providers. Despite the short time of the RBF pilot, the allocation of resources to RBF was a key effort to build commitment of the MOH to shift the emphasis of the health

38 UNFPA Aid Transparency Portal, Mali 2014, 2015 and 2016 program expenses See: http://www.unfpa.org/fr/transparency-portal/unfpa-mali consulted June 2017.39 USAID, U.S. Foreign Aid by Country, See: https://explorer.usaid.gov/cd/MLI consulted June 2017.40 USAID Dollars to Results, Family Planning and Reproductive Health in Mali, 2015.

30

sector from activities to results and address accountability and performance at the district level.

73. The difficulties encountered during project implementation were mitigated by the proactive approach to cancel uncommitted project funds such that they could be reallocated to the DPO and not lost to Mali. The canceled amount was 17 percent after the first restructuring and another 49 percent was cancelled just prior to closing. In, 2014, the cancellation was to help finance the SWEDD project and leverage US$165 million of additional regional IDA to enabled an FP-conducive environment in support of FP2020, including US$35 million for Mali.41 In 2016, the cancellation helped to finance a DPO of US$50 million to foster inclusive growth and support pro-poor decentralized transfers and social protection.42

74. Overall, the project maximized its use of resources to advance results, despite many implementation delays. The project was efficient in a number of ways: i) prioritizing interventions based on lessons learned around the cost-benefit, such as the focus on improving obstetrics care; ii) supporting the PRODESS program such that the project activities could contribute to health sector improvements; iii) focusing on implementation strategies to maximize results in the last years of the project; iv) increasing resources allocated to component 1 in order to maximize the strengthening of the health system in the available timeframe; and v) reallocating uncommitted resources such that they could be used by other projects. Despite these factors and the substantial efficiency in Phase 2, the substantial cancellations reduced the overall Efficiency rating to Modest.

3.4 Justification of Overall Outcome RatingRating: Moderately Unsatisfactory. 75. The intensive push on activities in the last years of the project changed the results. The project made tremendous achievements in the increasingly fragile country context. While the World Bank task team had consistently rated the project Moderately Unsatisfactory during supervision, the availability of data from the 2016 HMIS and 2017 UNICEF end of project survey provided new information on the efficacy of the project, which has increased the rating from the last Implementation Status and Results Report (ISR). Despite the large cancelation, the data suggest the increased focus on demand-side activities was transformational to produce results for the PDO Indicators. The budget estimates for the project at appraisal were based on the previous Bank project, which did not show results. Further, the last year of the project accelerated multiple years of implementation into one year.

76. The targets and indicators were highly ambitious given the fragile country context and the poor performance of the previous Bank project. The results would have been even greater had the project run longer, but the reduced scope enabled a heightened focus on the demand-side activities at scale and intensity in the final years of

41 The Sahel Women's Empowerment and Demographic Dividend (SWEDD) P150080 was approved in December 2014. The project includes Mali, Niger, Mauritania, Cote D’Ivoire, Chad and Burkina Faso. Additional financing for Burkina Faso was added in 2015, increasing the total amount of project financing to US$ 205 million.42 The First Poverty Reduction and Inclusive Growth Support Operation P157900 was approved in May 2017 for Mali.

31

the project. The project was not extended, since there was not yet data available to demonstrate the results, and the Government wanted to ensure the funds were not lost to support poverty reduction priorities, which have deepened with growing insecurity and fragility.

77. The overall outcome rating is a borderline rating of 3.54, which technically rounds to a 4, which is Moderately Satisfactory, based on the split evaluation approach. However, recognizing and weighing all the challenges of the project, the ICR assigns the project a rating of Moderately Unsatisfactory. This is also consistent with the ISR ratings for the project during the majority of its implementation period.

Phase 1 Phase 2Relevance Modest Substantial Objectives High High Design Modest Substantial Implementation Modest SubstantialEfficacy Modest SubstantialEfficiency ModestOverall rating: U MSRating value 2 4Total disbursed-US$7.78 million

US$1.79 million=23 percent US$5.99 million=77 percent

Weigh value 0.46 3.08Final Outcome Rating 3.54=Moderately Unsatisfactory

3.5 Overarching Themes, Other Outcomes and Impacts(a) Poverty Impacts, Gender Aspects, and Social Development78. The program aimed to benefit women and address gender dimensions of RH which fall particularly on the poor. The C4D program although limited in duration started to influence constraints to women’s use of RH services. The engagement of religious and community leaders, husbands, adolescents and women as ‘positive deviance’ role models was a powerful approach. Moreover, the consistent communication messages transcended the whole country through mass media, and the voice of respected leaders. Further the use of maternal mortality audits and improvements in obstetrics care will likely contribute to maternal mortality reductions in the coming years. The RBF also has the potential to improve inequalities in access to quality services– Government is looking at how to use the lessons from the RBF to reinforce CSCOMs in remote communities. Hence, the C4D program, obstetrics improvements and RBF were important to help the Government address gender and social development improvements, but require reinforcement, given their short duration.

(b) Institutional Change/Strengthening79. The following are key areas were the project built institutional capacity despite the shortened implemented time line:

32

Results-based financing. The RBF pilot was designed to support a culture shift in the public sector, moving from payment of inputs towards payment of results. There is high-level commitment of the Government to implement RBF progressively across all regions of Mali to reinforce CSCOMs and CSREFs. The Netherland Government will continue to fund the pilot started by the project.

Strengthening the technical capacity of health centers. The project likely built long-term capacity of the technical platform in health centers through the provision of medical equipment, and RH products, and quality improvement training contributing to improvements in FP, prenatal, assisted delivery care (including cesareans) and postpartum care.

Effective communication strategies to change behavior. The C4D program demonstrated how BCC activities can effectively influence demand to increase the use of services in health centers—this is a key priority of the Government’s National Family Planning Action Plan, and commitment to FP2020. The program left a package of communication tools to support implementation of communication programs with youth, community leaders, women, other after project closure.

(c) Other Unintended Outcomes and Impacts (positive or negative)N/A

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops80. The perspectives of beneficiaries regarding the project design, implementation, and results were collected through multiple methods. The findings are in Annex 5.

C4D focus groups. UNICEF conducted focus groups in February 2017 with different groups served by the program. A total of 89 individuals participated. Lessons surfaced by these sessions included that the ASACO developed capacity to coordinate community actors, that interventions were most effective when community leaders understood the positive social norms, and that religious leaders raised awareness through their sermons about RH. Findings also included that the Schools of Husbands helped to lift taboos related to pregnancy and female leaders were positive role models for increasing health center visits.

RBF assessment. Interviews and focus groups were conducted by the RBF agency in three districts of Koulikoro in March 2017. In total about 55 persons were interviewed from the Regional Health Directions, CSCOMs, CSREF, district authorities and ASACO. Lessons surfaced related to the successful influence of the RBF on key results areas: increasing service use in health centers; improving the quality of services; improving motivation of health personnel; reinforcing the coordination of decentralized health services. Lessons for Government to address related to the selection of indicators for the RBF, and the need for independent verification of results. Overall beneficiary feedback was positive, and the RBF helped CSCOMs understand quality standards, such as for skilled delivery and vaccination. However, implementation was very rapid, and more cycles of support were desired to strengthen implementation.

Regional Consultations. Structured interviews were conducted by the PIU in Ségou, Koulikoro, and Sikasso in January and February of 2017. A total of 233 individuals participated in individual interviews or group discussions, including women and girls,

33

ASACO representatives, CSCOM health professionals, youth peer educators, religious and community leaders, community health agents, and community radio. Recurring themes were identified related to the demand and use of RH services and about improvements to the supply and capacity of RH services.

4. Assessment of Risk to Development Outcome Rating: The overall risk to development outcome is Significant.81. The maintenance of the project’s development outcome will depend on the the new leadership in the MOH (the Minister changed again in April 2017) to scale-up the RBF, and continue to reinforce the quality of RH services in CSCOMs, including the platform of community health agents (reinforced by the C4D). Since all of the project’s approaches fall within the scope of PRODESS and the National Family Planning Action Plan, maintenance of the outcomes is likely. A key risk to the outcomes is further changes in the security situation in Mali, which affect the MOH and decentralized health structure.

82. The achievements of the project built decentralized capacity of the health system, specifically of the ASACO, CSCOMs and CSREFs. The capacity is likely to be reinforced by the PRODESS program. In regards to the improvements of obstetrics services, the project likely supported a new level of availability of obstetric services for women who need them, which can be further built on. The purchasing of contraceptive commodities by the project was part of the commitment of the Government to FP2020. Hence, the outcomes are likely to be reinforced by other donors (such as USAID and UNFPA). In regards, to social and behavior change, the Videos featuring rock stars, religious leaders, and other communication tools from the C4D activities can be expected to continue promoting behavior changes related to FP and RH for years to come. USAID and UNFPA have announced they will provide RH service delivery and health systems strengthening in Mali, mostly through NGOs in 2017, which would likely influence demand for RH services. The project demonstrated effective approaches for generating demand to use RH services—these approaches are likely to be scaled-up by the MOH, with the World Bank and other donors.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory83. The project had high strategic relevance (refer to Section 2.1). The US$30 million allocated to the project was an important contribution to the PRODESS and RH and FP programs of the Government. It filled a critical gap coordinated with other donors. In regards to fiduciary aspects, from preparation, the World Bank task team provided clear stipulations for financial management and procurement functions, given the recent governance concerns around the Global Fund. The decision to recruit the FMA seemed sound, despite the delays it later influenced – the condition of having an FMA was a key design factor that proved unworkable. In regards to the technical design, the logic of the project was strong, and terms of reference were developed to describe key areas of support, such as for the RBF, solidarity funds and voucher program, and

34

establish close coordination with other donors for project planning. Component 2 of the project was designed to address gender and social dimensions of RH. Further, the intervention regions of the project were selected based on a review of RH indicators and poverty data from the Growth and Poverty Reduction Strategy (2007-2015). In regards to M&E, the project was expected to have an M&E Specialist and receive support from the MOH Planning and Statistics Unit to implement surveys and analytical work. This was an appropriate arrangement.

84. During preparation, the Bank and the Government held a series of stakeholder workshops to develop the project. However, from the start, the design of the institutional arrangements of the project had weaknesses. In particular, the design of the project’s Steering Committee did not facilitate effective governance to direct the project (refer to Section 2.1). Despite this weakness, the project had strong Ministerial commitment at preparation. The key challenges were the changing security situation in the country, which delayed the effectiveness of the project.

(b) Quality of Supervision Rating: Moderately Unsatisfactory85. The World Bank task team provided dedicated support to the project. However, challenges related to the World Bank processes and the increasingly fragile country context limited satisfactory supervision of the project. The assessment of the quality of supervision considers the following factors:

Focus on development impact. The Bank’s supervision in 2013 focused on support to develop the PRODESS program and RH/FP priorities, as well as support to set-up the project. Since the 2014 restructuring and the 2015 MTR, the Bank’s supervision has focused primary on advancing high-impact activities, and supervision missions were used to review progress and agree on defined next steps and time lines. However, the cancellation of the US$5 million during the 2014 restructuring made the day to day work activities more difficult on top of an already difficult country context. Hence, while the restructuring was proactive, the cancellation of funds was too early, given the crisis context.

Supervision of fiduciary and safeguards aspect. Given the high risk documented at project appraisal, supervision activities focused extensively on financial management functions. Consequently, the Bank teams worked with the MOH to monitor and address challenges related to the FMA and procurement. At MTR, the decision to hire individual financial specialists was highly appropriate to unblock implementation delays. Safeguards were adequately addressed (refer to Section 2.4).

Adequacy of supervision inputs and processes. Having the World Bank task team leader in country enabled close dialogue and regular supervision support from 2014 onward. For example, much time was spent reviewing work plans and terms of reference to help address technical aspects. However, the need for frequent revisions of project requests meant there was a lot of back-and-forth exchanges in order to advance the project activities. This slowed progress when project requests did not receive quick non-objection. Often documents were submitted, but the next steps to incorporate the comments were delayed. Further, implementers often found the World Bank’s processes complex. Further, the lack of agreement around the technical design

35

of key activities such as the voucher program and solidarity fund support prevented implementation, and scale-up. Overall, despite the dedicated World Bank task team and long work hours, the supervision resources were inadequate to provide the required close support needed by the client in this low capacity fragile country context. For example, Bank missions were limited due to the small supervision budget and security restrictions – and the number of staff dedicated to the team was relatively small. The largest mission was for the MTR in 2015.

Candor and quality of performance reporting. Project implementation successes (and challenges) were well communicated in reporting. The World Bank task team reported honestly on the progress of the project in Implementation Status and Results Reports, Aide Memoires and management letters. The reports explain difficulties and the changes in leadership and country context.

Transition arrangements. Given the fragile country context and leadership changes within the MOH, the World Bank will assess the lessons learned before developing a new project. In terms of transition arrangements, The Netherlands has committed to support the RBF pilot in Koulikoro. Further, the C4D program will likely continue through future country support. Important to the next project is identifying the optimal institutional arrangements, specifically the actors to involve in oversight, coordination and implementation at the national and decentralized levels. During the project, the SG changed 9 times and the Minister 6 times (including in 2017 during the ICR). This provided a difficult environment for project oversight and management.

(c) Justification of Rating for Overall Bank PerformanceRating: Moderately Unsatisfactory

Ensuring Quality at Entry Quality of Supervision Overall Bank PerformanceModerately Satisfactory Moderately Unsatisfactory Moderately Unsatisfactory

5.2 Borrower Performance(a) Government PerformanceRating: Moderately Unsatisfactory86. In terms of sector planning, the Government agenda prioritizes RH outcomes. However, the resources allocated to PRODESS have been insufficient to meet its objectives. Overall the Government has allocated fewer resources to health (refer to economic analysis in Annex 2). The overall weak capacity of the MOH factored into the decision by the MOF to close the project and allocate the canceled funds to another sector. Further, problems with delayed actions and inadequate support by the Government have persisted most of the project, largely due the security and political changes in the country – commitment was revitalized at MTR, and again, through the Bangladesh Knowledge Exchange, which created an environment for innovation. In the last years of the project, the Government showed increasing leadership through the RBF and partnership with UNICEF. However, the Government could have played a more proactive role earlier in the project to plan actions and fill key functions in the PIU. Further, the institutional arrangements of the project lacked central leadership in order to engage multiple sectors, ensure lines of responsibility across agencies, make strategic decisions around how to implement interventions and resolve poor performance issues.

36

87. Overall communication was a main constraint during project implementation. The coordination of stakeholders was also weak. Most of the national implementing actors on the project Steering Committee did not contribute to the results of the project. Key national departments were those responsible for the PRODESS strategy, training, behavioral change communication (partnering with the C4D program) and RBF. Further, the Steering Committee needed to focus on subnational implementers in the regions: Regional Health Directions, District Health Teams, FENASCOM/ASACO, civil society leaders, etc. to ensure accountability to results, rather than the national agencies of the MOH. Fiduciary and monitoring aspects were another key challenge, with audit statements and procurement plans late, and many contracts not processed, although much resolved at the end of the project. Further, the MOH did not adequately support M&E of the project.

(b) Implementing Agency or Agencies’ PerformanceRating: Moderately Unsatisfactory88. Despite delays in the set-up of the PIU, at the end of the project, in the last years of the project, the PIU worked long hours to ensure the outcomes of the project were maximized. Overall the team had a high commitment to results once all staff were in place. In regards to consultations, regular meetings and planning sessions were held with project implementers, but there were many delays in processes and outputs were often weak – moreover, there was high turnover in terms of who would come to the meetings. Coordinating the national implementing actors required strong collective engagement, considering complex relationships among agencies. The national complexities often removed focus from the sub-national coordination, which was the intention of project. Further, there was weak coordination with other donor and NGO programs in defining the annual plan – the plan was only shared after the activities were identified, which meant key activities were often missing. A key challenge was the fact that PIU staff (and other implementers) could not travel outside of Bamako due to security reasons. The lack of personnel for key positions such as procurement and M&E was also a challenge for much of the project. The PIU was closed at the end of the project.

(c) Justification of Rating for Overall Borrower PerformanceRating: Unsatisfactory

Government Performance Implementing Agency or Agencies’ Performance

Overall Borrower Performance

Moderately Unsatisfactory Moderately Unsatisfactory Moderately Unsatisfactory

37

6. Lessons Learned 89. Lessons learned from the project related to i) design, ii) implementation and iii) technical interventions are described below. Several important lessons learned included: ensuring project implementation arrangements that have a clear line of accountability to achieve decentralized project outcomes, including a PIU that has sufficient strength to lead decision-making and is sufficiently insulated from political turnover; the need for close supervision to support the fragile country context; the transformational influence of intense demand-side activities on project results, with less cost than planned; and the need to adapt and simplify the design of technical interventions with local implementers.

i. Lessons related to project design. Lessons from the project design relate to the institutional arrangements and increasingly fragile country context.

Take stock of implementation arrangements at project design. The assignment of roles and responsibilities related to oversight, coordination and implementation should provide the required authority, autonomy, knowledge and influence (at the right levels) to support the results logic of the project. A stakeholder analysis could have been used to identify relevant actors and issues supporting/hindering their oversight and implementation roles.

Accountable design of the Steering Committee. The Steering Committee should provide strategic support to project decisions, rather than be engaged in the day-to-day implementation –oversight and implementations functions should be separate for accountability purposes. The Steering Committee could have engaged key decision makers, such as Ministers (Health, Planning, Communication), possibly a representative from non-Government, FENASCOM and the national association of communes, given the importance of engaging NGOs, and reinforcing the decentralized health system.

Strong PIU to coordinate the project, insulated from political change. The experience of this project underscores the need to give attention to the roles, quality, autonomy and selection of personnel tasked with project coordination responsibilities in the PIU. During the project, there were 9 SGs and 6 Ministers of Health (including the new Minister in 2017). Clear lines of delegation were important to empower the leadership of the PIU and facilitate stability of project processes. The PIU needed the authority to act on day-to-day project decisions, and to be insulated from political changes in the Ministry. The signature of the SG for project activities delayed implementation, especially during periods of leadership turnover—political assignment also delayed the recruitment of a qualified Project Coordinator until the MTR.

Position PIU to coordinate regional level implementation. In regards to coordination, a PIU anchored in the national Government was important, but main coordination of implementation should be at the regional level. In the current project, the PIU became responsible for coordinating many national implementers, which limited the direct support of activities in the regions. Meetings with project implementers and beneficiaries at project completion highlighted the need to engage local Governments to ensure results. The C4D

38

activities demonstrated that local Government is often better positioned to engage in multisectoral partnerships with diverse stakeholders engaged to deliver project activities and reach the rural communities with the poorest people. Key actors for subnational coordination (as seen from the C4D, RBF and training activities in the project) included the Mayors and commune administration, Regional Health Directions, District Health Teams, offices of Social Development, Planning, Education (schools), ASACOs (regional offices of FENASCOM), CSCOMs, CSREFs, pharmacies, community and religious associations.

Consider an extended project time line, given the low capacity context in a crisis environment. The project would have benefited from a longer implementation period given the constraints. The pace required to implement activities in the last years of the project put high pressure on the PIU and World Bank task team, who were working night and day and still did not have enough time to advance many activities. Developing the project with a longer time line or having the possibility to extend the close date when the country entered a fragile situation could have ensured a more feasible implementation time line.

Review use of country financial systems. The use of an FMA proved unworkable and delayed the project. Recruitment of individual financial specialists dedicated to work with the MOH to advance the project proved to be the best solution.

ii. Lessons related to project implementation. The project faced a range of implementation challenges, which can offer lessons for future projects.

Set-up of project coordination before the start of the project. The need for a functional PIU is of paramount importance. The long delays in hiring M&E and Procurement Specialists, the resignation of the initial project Coordinator, and the poor performance of the FMA all adversely affected project activities, disbursements, and outcomes.

Plan early capacity building of project management functions. Capacity building of the project management functions should be strategized and planned from the start of the project, anchored around the expected outcomes of the project (e.g., reinforcement of technical aspects, planning, supervision, M&E, stakeholder engagement, communication).

Plan for close supervision in the low capacity fragile context. The reduced Government capacity following the crisis, heightened the need for support and resources, specifically to coordinate actors, develop project activities, manage results and support implementation of Bank processes. Having the project leader in country from 2014 onward was important. However, despite long hours and a dedicated task team, the supervision resources were inadequate to provide the close support needed in the fragile country context. There was a need for regular missions in addition to extensive in-country support to help the Government to take forward specific project activities which proved difficult with the available supervision budget as well as with the insecurity and travel restrictions.

39

Implement activities, as part of a facilitated strategy to avoid dispersed planning of small activities and ensure quality support. project activities that were most likely to lead to sustainable outcomes were those that were designed as part of a strategy to develop institutional capacity. For example, the C4D activities, which reinforced the health districts in the project regions and the RBF scheme to change incentives produced measurable changes that are likely to continue after project completion. The project could have benefited from the elaboration of strategies which grouped smaller activities by different actors to advance defined results – this could have also simplified the approval of many small requests and provided funding for local support to facilitate implementation and quality improvements, while engaging a range of local implementers, as was exemplified by the C4D and RBF. Much effort was spent planning activities, such as training, by different implementers, but how these activities came together as an aggregate to advance results in all five regions was often less clear.

Use field assessments to take corrective actions to improve the project. The Steering Committee did not review data or collect data to make its decisions and there were few field visits. However, at the end of the project, the MOH saw that field visits could provide data to assess how to improve the project (to improve training, supply of supplies, etc.), and make corrective actions. Further, in the C4D and RBF simple dashboard, communication, diagnostic and meeting tools were reported to help support the Government to start to make decisions around the program using information from field reports. Likewise, the solidarity fund support although only in two districts used a diagnostic to assess how to resolve problems.

Timely use of restructuring and the MTR to address problems. A clear success of this project given the project coordination difficulties noted above was the well-timed corrective restructuring (2014) and stock-taking exercise during the 2015 MTR. The participatory MTR engaged all key stakeholder groups to build commitment around an action plan to accelerate project implementation for the remaining term. Following these pro-active engagements, the project could reach full implementation and produced notable outcomes in the approximately 18 months remaining.

iii. Lessons related to technical interventions. The project had successes and challenges related to technical activities, which can offer lessons.

Multi-stakeholder communication strategies can transform the demand to use health services. The C4D was scaled-up to all regions in less than one-year. The program, although short, demonstrated effective tactics for shifting behavior to increase the use of RH services, leveraging the collective influence of multiple key change agents, with targeted consistent messaging. This multi-stakeholder design addressed barriers to service use from different angles, focusing on women and change agents, with the strong influence on women’s RH decisions. Notable lessons for promoting demand-side changes were identified through a series of focus groups conducted by UNICEF. Key was the use of a 'positive deviance' approach by bringing role model mothers

40

and husbands (CFU, ANJE, Husband champions) to influence other women/men to use or support RH and FP services; also, engaging community and religious leaders was essential to the success. These leaders communicated messages to encourage the husbands and women in the community to use RH/FP services.

Adapt and simplify technical interventions with local implementers. While the design of the project built on lessons learned and evidence, the design of project activities, such as the voucher scheme and solidarity fund support as described in the PAD was viewed as complex, and there was a lack of common vision among technical agencies around how to approach activities. There was a need to work with the Government, experts and decentralized implementers to reach consensus on how to adapt these interventions within the country context.

Plan activities to build leadership to implement the project. To ensure ownership of the project, at MTR, stakeholders were mobilized and leadership commitment was renewed – important was mobilizing the Minister, but also the collaborative assessment of technical actors. Then, in 2016, the Bangladesh Knowledge Exchange was used to build commitment of a Government team from MOH, MOF and other ministries to implement new strategies to address RH in the community and with religious actors. The implementation teams of the C4D and RBF used similar strategies to build ownership of stakeholders to quickly adapt new tools and methods. Sequenced leadership building activities, anchored in results, were useful to restore momentum, onboard new leadership, improve collaboration and build confidence and know-how to implement project activities. Overall, the project could have benefited from a planned approach to build leadership around the project, given the difficult implementing environment.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agenciesN/A

(b) CofinanciersN/A

(c) Other partners and stakeholdersN/A

41

Annex 1. project Costs and Financing

(a) Project Cost by Component (in US$)Components Appraisal

Estimate (USD millions)

First Restructured Estimate (USD millions)

Second Restructured Estimate (USD millions)

Actual/Latest Estimate (USD millions)

Percentage of Appraisal

Percentage of Restructuring

October 2014 December 2016

1. Strengthening supply and quality of RH Service

7.50 10.22 5.10 4.00 53 percent 39 percent 78 percent

2. Increasing demand for RH Services


3. Social accountability, project management, and M&E


Total Baseline Cost 30.00 25.00 8.88 7.78 26 percent 31 percent 88 percentPhysical Contingencies 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Price Contingencies 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Total Project Costs 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Front-end fee PPF 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Total Financing Required 30.00 25.00 8.88 7.78 24 percent 31 percent 88 percent

At appraisal project cost was estimated at US$30 million. The above Table a shows the actual cost estimated at the end of the project was US$7.78 million.

(b) Financing (in US$)Sources of Funds Appraisal

Estimate (USD First Restructured

Estimate (USD Second

Restructured Actual/Latest

Estimate (USD Percentage of

AppraisalPercentage

of first Percentage of

second

43

millions) millions) Estimate (USD millions)

millions) Restructured Estimate

Restructured Estimate

Borrower 0.00 0.00 0.00 0.00 0.00 0.00 0.00

IDA Grant 30.00 25.00 8.88 7.78 24 percent 29 percent 82 percent

Total 30.00 25.00 8.88 7.78 24 percent 31 percent 88 percent

The project did not benefit from Government counterpart funding. The above Table (b) shows that the project activities are financed only by Ida.

44

Annex 2. Outputs by Component

The results logic of the project is summarized in figure A2.1 and the outputs by component are described below. Table A2.1 at the end of the Annex summarizes progress on the intermediate outcomes by component.

Figure A2.1 The results logic of the project

Component 1: Strengthening Supply and Quality of RH Services

This component was implemented by the PIU, the National Health Direction (Direction Nationale de la Santé - DNS), the Regional Health Directions of the intervention regions, FENASCOM, Community Health Association, (Association de Santé Communautaire -ASACO), the DFM, and the DPM.

i. The following are outputs related to the RBF pilot in health facilities in Koulikoro:43

Before launching the pilot, 37 personnel were trained in RBF in Benin. In June 2016, the MOH signed the RBF contract with the experienced NGO

consortium KIT-CORDAID-CGIC for nine-months of accelerated implementation. In August 2016, the RBF agency conducted a cascade training of 1,116 representatives

from across Koulikoro region, including 237 district representatives (about 19 per district), 410 members from ASACOs, 492 health personnel from CSCOMs and CSREF, and 27 regional officials. The training focused on the specific roles related to coordination, delivery of a package of high-quality RH services, and verification:

o The ASACO led the planning and coordination for the health centers;

43 RBF agency reports, 2016 to 2017, including presentation on RBF achievements at the stakeholders’ workshop

Impact (changes in behavior of women to use RH services; changes in quality and acess to services)More women use modern contraception (15-49 years old, including post partum) (PDO1, PDO4)

More pregnant women receive at least 4 antenatal care visits (PDO2)More births attended by skilled health personnel (PDO3)Increased coverage of family planning methods (PDO5)

Outcome (better service delivery and supply management)Women gain access to package of reproductive health services (IO1)

Fewer stock-outs of contraceptives at health facilities (IO2)More pregnant women receive prenatal care during health care visit (IO4)

More deliveries by caesarean section (IO5)More women receive post-partum care within 7 days of delivery (IO6)

Outputs (changes in knowledge, attitudes, cost of services)More health personnel trained to provide family planning and obstetric services (IO3, IO7)

Social marketing reaches more women and men (IO8)More pregnant women receive and use vouchers to access obstetric services (IO9)

Advocacy conducted for civil society, political, religious and traditional leaders (IO10)

45

o The health providers in health centers executed the plans to deliver the package of standardized RH services;

o The DRS and district health teams supervised the RBF in terms of procedures and standards;

o The Mayor (in the case of a CSCOM) and Circle Council (in the case of a CSREF) defined the priorities for the RBF in their districts; and

o Verification involved i) the district health teams and DRS at the first level of verification, and ii) the second level was an independent community-based organization engaged to verify the RBF results using a community survey, focusing on the satisfaction with the quality of care.

The establishment of the RBF platform (procedures, tools, etc.) delayed the start of the RBF, including the RBF implementation manual and local data collection instruments. The RBF was launched in October through a regional ceremony. The OpenRBF portal for data management was established in December 2016.

In total, 215 ASACO established results agreements and RBF plans (205 CSCOM and 10 CSREF). The tripartite contracts were signed between the CSCOM, community-based organization and Mayor (Circle Council in the case of CSREF). Due to the delayed timeframe, only one cycle of RBF was implemented in the regional health centers: 82,315 women and children were provided RH service (refer to box A2.1 below). The number includes the total number of women and children beneficiaries of RH package provided by the RBF agency, given the integration of childhood services.

Throughout implementation of the pilot, the RBF agency provided coaching technical support to reinforce the local implementation of the RBF in the districts. This was in addition to monthly field visits by coordination team, and meetings to share progress and problem-solve across districts. Meetings were also held at the national level to build leadership commitment to the RBF.

BOX A2.1 Services provided in CSCOM and CSREF through the RBF pilot (October to December 2016)44

The average quality score for the service package provided by a CSCOM was 66 percent and for a CSREF 69 percent. However, this score is only available for the one cycle of RBF, providing a baseline for future reinforcement.

The package of RH services to women and children included the following: 2,620 complicated cases of malaria treated in the CSREF for children 0-5 years; 1,275 correct case referral taken charge in the CSREF, from CSCOMs; 1,290 pregnancy complications taken charge in the CSREF (including cesareans); 206 cases tuberculosis DOT taken charge in CSCOMs; 43,031 cases of malaria in children under 5 years taken charge in the CSCOMs; 16,189 consultations for integrated management of illness of children 0-5 years; 15,260 children under 12 months completely vaccinated; 400 pregnant women tested for HIV and placed on antiretroviral drugs; 6,292 pregnant women treated correctly for malaria; 20,641 women used modern FP; 6,393 women receiving prenatal visits; 11,363 births assisted by qualified personnel; 5,777 women received four prenatal visits.

44KIT-CORDAID-CGIC, "Rapport final du projet pilote « PRSR/FBR » FBR-Financement Basé sur les Résultats – Approche pour accélérer les résultats en Santé de la Reproduction", April 2017.

46

ii. The following are outputs related to improving the supply of contraceptives:45

In 2015, contraceptives amounting to 468,000 CYP were purchased from UNFPA, and distributed to health facilities in 2016.

3,530 health personnel were trained by DNS, Regional Health Directions, UNICEF and the RBF agency to provide FP service.

iii. The following are outputs related to the capacity building of health facilities to deliver RH and obstetrical services:

806 health personnel were trained to provide obstetric services, such as prenatal/antenatal care, neonatal care, emergency obstetrical care, anesthesiology, and maternal mortality audits. Regional action plans in 2015 were an important contribution to this, although only funded for one year.

In 2015, the use of maternal mortality audits was scaled-up in all health districts as a result of the training and technical supported in the regional action plans.

Equipment requested and received by health centers in 2016 included: 100 gynecological tables, 280 maternity beds, 16 operating tables and lights, 65 vacuums extractors, 418 aspirators, 4 echography, 2 centrifuge, 200 vaginal spectrums, 186 midwifery kits, 510 RH kits, 8 refrigerators, 71 mother and child scales, 4 examination lights, and 8 baby warmers and baths.

The training, equipment and demand-side activities likely contributed to improvements in the quality of services in 2016 and catalyzed more women to use RH services. During the seven months of the C4D campaign health centers reported: 582,82046 women and adolescents used RH services, and there was a direct focus on promoting prenatal care, assisted birth and postpartum care (refer to box A2.2 below).

BOX A2.2 The project catalyzed more women to use RH services in 2016

According to the HMIS47, the number of women receiving antenatal care spiked in 2016: 165,009 in 2013; 185,120 in 2014; 190,398 in 2015; and 462,460 in 2016. The increase was consistent across all regions.

The HMIS also shows that the number of women with deliveries by cesarean increased during the project: 16,032 in 2013; 17,881 in 2014; 17,826 in 2015; 18,461 in 2016.

According to the HMIS, the percent of women receiving postpartum care has increased since the 2010 baseline of 54.6 percent (177,805) – including an increase of about 5 points during the last year of project implementation: 203,118 (60.5 percent) in 2011; 213,924 (59.6 percent) in 2012; 220,894 in 2013 (64.4 percent); 245,938 (65.9 percent) in 2014; 253,305 (66.7 percent) in 2015; 276,521 (71.5 percent) in 2016. No data was collected on whether this was within the first 7 day following delivery.

Component 2: Increasing Demand for RH Services

This component was implemented by the PIU, DNP, and the Unit for Decentralization and De-concentration Support (Cellule d’Appui à la Décentralisation at à la Déconcentration - CADD) and FENASCOM (reaching the ASACO). The main activity of this component was 45 Data is from quarterly and annual project reports from the PIU. 46 UNICEF 2017. Rapport de Progrès (Décembre 2016- Février 2017) Accroitre la demande et l’utilisation des Services de Santé de la Reproduction (SR) et Planification Familiale (PF) aux niveaux des CSREF et CSCOM dans les 34 districts au Mali (SC160310).47 Système Local d'Informations Sanitaires, Local Heath Management Information System (HMIS), 2012, 2013, 2014, 2015, and preliminary data from 2016.

47

the UNICEF C4D program in 2016, despite other planned activities. The C4D program carried out many of the activities planned for the NGOs. Several high-level advocacy sessions organized by DNP following the Bangladesh Knowledge Exchange also contributed to results.

i. The following are outputs to strengthen outreach services and BCC interventions:

UNICEF implemented the accelerated C4D program. The C4D activities48 built the capacity of the ASACO to implement multisector communication, social mobilization and advocacy. During the short campaign, 742,202 men and women attended social marketing and advocacy interventions promoting FP and the use of RH services. This surpassed the targeted 600,000 persons in a rapid timeframe. Box A2.3 describes (below) the C4D outputs. UNICEF estimated about 30 percent of the targeted population in the project regions was reached through direct communication activities or mass media dissemination.

BOX A2.3 UNICEF C4D communication outputs over seven months

5,946 religious, female, men and youth leaders were trained to influence community acceptance around decisions to use RH services: 1,567 community health workers; 771 health center directors; 1,079 community leaders; 671 peer educators (including 282 girls); 903 mothers promoting the nutrition of young children; 306 men from 27 Schools of Husbands schools,"Écoles des maris"; 230 youth reporters; and 121 journalists. This was achieved through a cascade training led by 240 Malian C4D trainers.

185,620 persons were informed about RH issues through film, theater and other outreach. The engagement of the Schools of Husbands schools, mobilized 1,110 men Husband champions who

openly encouraged their wives to use prenatal and postnatal consultations. 652 CSCOMs received communication and video materials, and organized advocacy interventions in

January 2017 around reinforcing community acceptance to use RH services. 83,077 youth were engaged in secondary schools, colleges and training centers in Bamako to

promote RH service use. The advocacy sessions also distributed stickers, t-shirts, etc. Over 60,000 youth used the social media tools set-up by C4D, including the Facebook page

Mali4Family and Twitter. 51,999 persons were engaged by women promoting their practices of nutrition and feeding of young

children (ANJE), "Alimentation du Nourrisson et du Jeune Enfant" and contraceptive use, “Comité des femmes utilisatrices des services de santé” (CFU) who travel villages in a caravan to promote RH service use.

Media was also used to influence behaviors, including 25,200 radio emissions; a spot on the main television channel broadcasting to about 14.4 million Malians, 12 episodes of a television series called, “La Vie de Bijou”; six songs produced by internationally recognized musicians – Djeneba Seck and Amy Sacko and the Mali Instrumental Ensemble for adults and rappers Iba one, Mylmo and Master Soumy for adolescents; and widely shared videos with known celebrities and respected Malian religious leaders speaking about RH.

ii. The following are outputs related to improving access to finance for RH services:

Capacity building of local solidarity funds. CADD conducted workshops in two districts including a participatory diagnostic identifying problem solving actions to strengthen the solidarity funds– consequently membership of the funds increased from 21 percent in 2009 to about 50 percent in 2016 – about 400 women49 have benefited from emergency

48 UNICEF 2017. Rapport de Progrès (Décembre 2016- Février 2017) Accroitre la demande et l’utilisation des Services de Santé de la Reproduction (SR) et Planification Familiale (PF) aux niveaux des CSREF et CSCOM dans les 34 districts au Mali (SC160310.)49 Data is estimated from the membership logs of the Community Heath Associations (ASACOs) in Fana and Banamba by the Unit for Decentralization and Devolution Support (CADD) of the MOH.

48

transport to give birth, since the 2015 support. ASACO led the community mobilization and some of the funds established membership cards. However, the planned study to assess the insurance schemes was not implemented, and the training and technical support was not scaled-up to other districts.

iii. The following are outputs to promote a FP conducive environment:

Bangladesh exchange. In May, 2016, the project organized a South-South Exchange with Bangladesh on Population Policy and RH (refer to box A2.3 below).

Advocacy sessions. In total, DNP held about six advocacy sessions: the exchange visit to Bangladesh; three cabinet level sessions after Bangladesh, as well as follow-up exchanges with Tunisia and Morocco around the national Population Policy. The UNICEF activities enabled the project to achieve 652 advocacy sessions, with civil society, political, religious and traditional leaders.

BOX A2.4 Catalytic learning of Bangladesh exchange

The objective of the exchange was to learn from Bangladesh’s experience in: i) multi-sectoral policy dialogue, ii) engaging religious actors around RH issues, iii) empowering women to make FP decisions, and iv) community-based service delivery. While late in the project implementation, the exchange motivated commitment in new areas: to develop RH training for religious leaders in Mali; to build ownership of the population policy by the key players in the country (Government, opposition, religious leaders, NGOs, communities, and donors); to strengthen door-to-door RH services in communities; to develop an integrated community-based service package (education, nutrition, psycho-social services and RH/FP) ; and to develop actions to strengthen public-private partnership in service delivery. In the evaluation of the exchange, all of the participants reported improved knowledge and confidence to engage religious leaders in RH activities; and new knowledge to design actions to coordinate the population policy. However, follow-up could not be realized in the timeframe of the current project. Actions to engage religious leaders are being advanced in the regional SWEDD project.

Component 3: Social Accountability, Project Management, and M&E

This component was implemented by the PIU, CPS, DNP, and the Human Resource Directorate (Direction des Resources Humaines - DRH). The main activities implemented in this component were project management costs and training. Field visits/supervision were rare and data collections were not realized. Annual planning. Planning workshops were held in Bamako annually. However, the

approved annual plan was delayed repeatedly. Data collection. There were some activities by CPS to disseminate the 2013 DHS findings

and the PDDS in the project regions. The MOH/UNICEF survey conducted at the end of the project provided the final project data, as well as data from the 2015 MICS and HMIS.

Training and conferences. The PIU joined international conferences on nutrition in Benin; on FP in Indonesia; and on private sector partnership in Senegal. The DRH also received training in leadership management, implementing agencies received training on Bank procedures, and the DFM received reinforcement in TOMPRO.

Supervision. Four vehicles were purchased for the MOH. The PIU also carried out meetings with project implementers, the World Bank and the Steering Committee to manage the project.

49

Table A2.1 Achievement of intermediate results by component

Intermediate Outcome Indicators Base-line

Source Original Target2017

Revised Target 2017

Actual achieved

2017

Extent of achieve-

ment

Component 1: Strengthening Supply and Quality of RH Services1# Women provided with access to a package of RH services (number).50

0 RBF 120,000 200,000 82,315 Modest

#2 Health facilities that experience 2 or more weeks’ stock-outs of contraceptives during the last 2 weeks preceding the survey (percent)

N/A Not collected

<5 - N/A Negligible

#3 Health personnel trained to provide FP services (number)

0 Reports 400 - 3,530 High

#4 Pregnant women receiving antenatal care during a visit to a health care provider (number)

0 HMIS 500,000 - 462,46051 Substantial

#5 Deliveries by Caesarean section (number) 12,756 HMIS 30,000 - 18,461 Modest#6 Women provided with postpartum care within the first 7days of delivering in a health facility (percent)

54.6 HMIS 70 - 71.5 Substantial

#7 Health personnel trained to provide obstetric service (number)

0 Reports 400 - 806 High

Component 2: Increasing Demand for RH Services#8 People attending social marketing interventions that promote FP

0 UNICEF

600,000 - 742,364 High

#9 Pregnant women provided with vouchers to access selected services and who use them (number)

0 Not collected

10,000 - 0 Negligible

#10 Advocacy sessions held with civil society, political, religious and traditional leaders on demographic issues and FP (number)

0 PIU, UNICEF

300 - 654 Substantial

50 RBF agency report. The package included prenatal care, vaccination, malaria care, HIV prevention, tuberculosis, integrated child health management, delivery with skilled health worker, postnatal visit, family planning, pregnancy complications/referrals. 51 Provisional data from the HMIS for 2016.

50

Annex 3. Economic and Financial Analysis

The project economic analysis is updated based on the actual implementation experience, including a review of changes in the health financing situation in Mali and the cost-benefit analysis (CBA) of the project activities.

The project context changed with the 2012 crisis. This project was expected to complement donor support from USAID and the Netherlands at the national level, Canada in Kayes, and the United Nations system in the North. The project-targeting was based on the poverty map of the Poverty Reduction Strategy Paper, focusing interventions on selected regions (Sikasso, Koulikoro and Ségou) and peri-urban Bamako. The 2012 crisis devastated communities and health infrastructure in the North of Mali, resulting in changes in the MOH and financing of the health sector.

PRODESS II was not renewed until 2014 due to the crisis and changed health financing situation.52 In order to understand the context of project achievements, key health financing changes in Mali are described.

Domestic financial resources. Figure A3.1 shows the domestic financing allocated to the PRODESS program during the project covered about 38 percent of the expected costs. The health sector budget was reduced from about FCFA 70 million in 2011 to less than FCFA 26 million in 2012. In 2017, the budget was not restored to the pre-crisis level.

Figure A3.1 The budget allocated to PRODESS, from 2004 to 2017 (FCFA millions)53

2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 20170

20000400006000080000

100000120000140000160000180000200000

MINISTRY OF HEALTH & PUBLIC HYGIENE Health & Social sector budgetLow budget estimate for PRODESS

External funding to the health sector. Figure A3.2 shows that external funding of the health sector also dropped between 2011 and 2014. There has been some improvement since 2015, but external funding remains below the pre-crisis level.

Health expenditure. The expected increased Government expenditure on health did not happen (figure A3.3). However, there was a decrease in out of pocket expenditure (OOP). Since 2010, the Government has implemented initiatives to reduce the cost of health services to households, which likely contributed to the reduced OOP. These include a

52 Priorities in PRODESS III (2014-2018) include health, social development, and the promotion of women, including universal access to health care and improvements in quality of services and solidarity in communities. 53 Data from the World Bank, 2017. Open Budget portal, Boost tool.

51

strategy to extend insurance coverage through private mutual health schemes, with a contribution of more than FCFA 400 million in 2015, partnerships with private sector providers and payment exemptions for basic health services.

Figure A3.2 External funding to the in health sector in US$(2009-2015)54

2009 2010 2011 2013 2014 2015$0

$5,000,000

$10,000,000

$15,000,000

$20,000,000

$25,000,000

$30,000,000

$35,000,000

$40,000,000

Figure A3.3 Private health expenditure (PvtHE), out of pocket (OOP) and Government health expenditure (GHE) during the project 55

2010 2011 2012 2013 20140%

10%20%30%40%50%60%70%80%90%

100%

GHE as % of total health expenditure PvtHE as % of total health expenditureOOP as % of total health expenditure OOP as % of PvtHE

The CBA considers the extent the project investments generated benefits for Mali. The findings present the Net Present Value (NPV) and the Economic Rate of Return (ERR) to compare to estimates at appraisal. The analysis assesses the contribution of the project to two results areas.

i. Improvements in maternal and neonatal mortality. The project likely contributed by increasing the use of facility based pregnancy services, including prenatal and delivery care. The project increased the coverage of personnel with skills to handle pregnancy complications, and helped to improve the technical platform in health centers by providing supplies and equipment. Moreover, the C4D and maternal mortality audits addressed demand-side barriers related to confidence of the community to use services, mobilizing additional women to use pregnancy services in health centers. In a few districts, the project also contributed to barriers of transport to health centers, while Government’s support to mutual health schemes was more widespread.

ii. Reducing unwanted pregnancies and abortions. The project likely contributed by increasing the CYP from available FP products, increasing the skills of health personnel

54 Data from the PAD (2009-2011) and DFM (2013-2015)55 Data from the WHO, 2014. National Health Accounts Database.

52

and community workers to deliver and promote FP, and empowering women to make decisions to use FP. The C4D activities at the end of the project were intensive and beneficiary feedback and HMIS data suggest that they created an enabling environment, breaking barriers (social norms, etc.), which catalyzed women to use FP.

The beneficiary population targeted by the project was women of reproductive age. The HMIS was used to estimate the beneficiary population in the project regions (Table A3.1).

Table A3.1 Population data for the project intervention areas.56

Indicators 2012 2013 2014 2015 201657 2017Population of regions 10,348,999 10,659,001 109,769,98 11,300,997 11,634,559 11,925,423Population growth rate - 3.00% 2 .98% 2.95% 2.50% 2.50%Woman of reproductive age(21% are pregnant)

2,432,015 2,504,865 2,579,595 2,655,735 2,792,294 2,862,102

The costs of the project are reviewed below by component and year (Table A3.2). i. Component 1, Strengthening Supply and Quality of RH Services. Execution of costs began

in 2014 with training activities and the contracting of UNFPA. In 2016, 50 percent of the budget was canceled.

ii. Component 2, Strengthening Demand for RH Services. The failed recruitment of the NGOs impacted the cost of this component. The C4D activities were executed in 2016, which explains the increased cost of this component in 2016. In the 2016 restructuring, 79 percent of the budget was canceled.

iii. Component 3, Social Accountability, project Management, and M&E. There were some costs in the first year related to project management and conferences. The cumulative management costs were around US$1.8 million, considering the delayed start of the project and cancelation of the data collection activities (in 2016, 67 percent of the budget was canceled).

Table A3.2 Project Costs including Price Contingencies in US$

Year 2013 2014 2015 2016 2017 TotalComponent 1 - 7,337 3,145,422 262,022 1,192,334 4,607,115Component 2 - - 462,464 1,612,701 350,096 2,425,261Component 3 38,475 281,696 673,790 739,889 109,326 1,843,176

Total cost 38,475 289,033 4,281,676 2,614,612 1,651,756 8,875,552Disbursement rate 0.1% 1.1% 18.4% 81.0% 100%

Total Grant 30 million 25 million(restructuring 1)

25 million 8.88 million(restructuring 2)

8.88 million

The CBA estimates how many women received benefits from the project. Table A3.2 shows the total number of beneficiaries by PDO Indicator. The disbursement rate in 2013 (0.1 percent) was low which suggests that the impact of the project was mainly in 2014, 2015, 2016 and 2017. While it is not possible to attribute the full improvement in the PDO Indicators to the project, the beneficiaries’ survey suggests that the project has contributed to the PDO results – there is no data on PDO Indicator 4. PDO Indicator 1 and 5 contribute to similar FP results. PDO Indicator 2 and 3 contribute to related results for safe birthing. The CBA analysis focuses on PDO Indicator 3 and PDO Indicator 5 to review the benefits of the

56 HMIS data from the MOH, 2012, 2013, 2014, 2015 and preliminary data from 201657 Estimated to be 24% of the population as from 2012 to 2015, since data is not available for 2016 and 2017.

53

project to the two main results areas.

Table A3.3 Annual beneficiaries by PDO Indicator according to HMIS data58

Year 2012 2013 2014 2015 2016 2017 estimate

1#Modern contraceptive use among women aged 15 to 49 years (number)

174,147 248,290 310,103 284,864 498,884 516,478

2#Pregnant women who have at least 4 antenatal care visits59 (number)

215,164

223,992 239,470 174,181 162,247 249,119

3#Birth (deliveries) attended by skilled health workers (number)

349,389 173,607 205,628 208,978 235,136 362,414

4#Post-partum women using modern methods of contraception (number)

0 0 0 0 0 0

5#CYP reached through project interventions (number)

- - Added in 2015

435,710 913,458 538,934

i. Investments in family planning (FP). While the total number of CYP recorded in the HMIS data was 1.3 million, only an estimated 468,000 of this was explicitly purchased through the project. To provide conservative estimates of the impact, this figure was used, even though the other project interventions likely also contributed to the increased use of FP commodities not purchased through the project. Thus, the CBA found that the CYP contributed by the project in 2015 and 2016 avoided an estimated 134,784 pregnancies60, of which 72,783 would have resulted in live births, 43,130 would have resulted in abortions (all unsafe abortions), 668 women would have died in pregnancy and there would have been 7,915 neonatal deaths61. The cost of care avoided from the FP investment was US$5,568,871 (US$4,864,414 from avoided births and US$704,457 from avoided abortions). The productivity loss avoided was US$1,966,965 for the maternal deaths and consequent lost participation of women.

ii. Investments in skilled delivery (pregnancy services). The project contributed to the delivery services of an estimated 143,198 women – about 30 percent of the reported HMIS beneficiaries in 2016 (70,715) and 20 percent (72,483) in 2017. This resulted in care costs of about US$10,787,912, and is estimated to have averted about 266 maternal deaths and 3,460 neonatal deaths. The total productivity loss averted from these deaths was US$41,300,116 (US$38,355,110 for neonatal deaths, and US$2,944,295 from maternal deaths).

The results of this ex-post analysis show a higher return on the reduced project costs compared to the ex-ante analysis of 2011. With the benefits of this gain spread across 10 years, from 2013 to 2023 due to the expected lag in benefits after the project. The total NPV benefit was US$31.0 million with an economic rate of return (ERR) of 42 percent. The initial assessment performed for the PAD estimated the total NPV benefit as US$14.9 million over 10 years and the ERR as 37 percent. Hence, the reduced project cost of US$8.88 million had a higher return than was expected from the US$30 million estimated cost at appraisal. The project had substantial efficiency in its shortened time frame, since the costs were proactively

58 HMIS data from the MOH, 2012, 2013, 2014, 2015 and preliminary data from 2016.59Between 2012 and 2014 the data is for 3 antenatal visits, since data on 4 visits was not available in the HMIS. The data for 2015 and 2016 are for 4 antenatal visits. 60 Darroch JE and Singh S, Estimating Unintended Pregnancies Averted from Couple-Years of Protection (CYP), New York: Guttmacher Institute, 2011, p. 5.61 Ibid, Table 1, p.8 for Western Africa; neonatal death rate estimate from Guttmacher, 2012, p. 16.

54

reduced and funds were directed to high-impact areas. Further, considering the reduced external funding during the years of the project (in part due to more resources being diverted to security), the amount was an important contribution to the health sector.

55

Table A3.4 Present value of net benefits of the PAD versus ICR (US$)2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023

Project Cash Flow (38,475) (289,033) (4,281,677) (2,614,613) (1,651,755) (8,875,553) Discounted Cash Flow (38,475) (262,757) (3,538,576) (1,964,397) (1,128,171) (6,932,377) Additional care cost - Facility-based delivery (FBD) (4,726,200) (4,844,364) (9,570,564)

Care cost avoided - unwanted pregnancy 1,039,888 945,353 859,411 781,283 710,257 645,689 586,990 5,568,871 Productivity loss avoided - unwanted pregnancy 367,296 333,905 303,550 275,955 250,868 228,062 207,329 1,966,965 Productivity loss avoided - FBD 7,457,968 6,779,971 6,163,610 5,603,282 5,093,892 4,630,811 4,209,828 39,939,363 Total benefits 8,865,152 8,059,229 7,326,572 6,660,520 6,055,018 5,504,562 5,004,147 47,475,199

Net benefit (cost) (38,475) (262,757) (3,538,576) (6,690,598) 2,892,617 8,059,229 7,326,572 6,660,520 6,055,018 5,504,562 5,004,147 NPV (38,475) (301,233) (3,839,809) (10,530,406) (7,637,789) 421,439 7,748,011 14,408,531 20,463,549 25,968,110 30,972,257

ERR 42%

Assumptions of ICR CBA analysis62, 63

The project contributed to 30 percent of women receiving skilled delivery and pregnancy services in 2016; and 20 percent in 2017 (143,198 women); The project contributed a CYP of 468,000 from 2015 to 2016; Women who benefited from the increased CYP also benefited from the use of modern

contraceptive services; Women who benefited from multiple prenatal visits likely also benefited from skilled delivery; 66 percent of births are in health facilities; If all women had facility-based deliveries maternal mortality would drop by 56 percent; newborn deaths would drop by 71 percent; NPV of productivity losses avoided by having fewer maternal deaths are estimated at $2,945 per women, based on current Mali GDP per capita; NPV of productivity losses avoided by having fewer neonatal deaths are estimated at $5,474 per infant, based on current Mali GDP per capita; The unit cost of births and abortions are calculated from the RBF data, $66.83 for a birth and $16.33 for post-abortion care; There are an estimated 0.288 unintended pregnancies averted per CYP contributed by the project; For every 100 pregnancies averted among women using contraceptives there are 54 percent fewer births, 32 percent fewer abortions (all would be

unsafe), 0.496 percent fewer maternal deaths, and 2.04 percent fewer neonatal deaths. A discount factor of 10 percent was used.

62 Guttmacher Institute and UNFPA, 2009, 2012 and 2014. Adding it Up: The Cost of Investing in Sexual and Reproductive Health.63 J. E. Darroch and S. Singh. 2011. Estimating Unintended Pregnancies Averted from Couple-Years of Protection (CYP). Guttmacher Institute.

56

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Specialty

Aissatou Diack Senior Health Specialist GHN07 project Management

Amy Ba Language Program Assistant AFTHE - HIS Administrative support

Aissatou Chipkaou Operations Analyst GHN13 Operations

Jean J. De St Antoine Lead Operations Officer AFTHW - HIS Operations

Francois P. Diop Sr Economist (Health) AFTHE - HIS Economics

J. C. Duarte Pacheco Blasques HQ Consultant ST GHN07 Team MemberMbaye Mbengue Faye HQ Consultant ST GSU07 Team MemberDaria Goldstein Lead Counsel LEGLE Legal

Davidson R Gwatkin Lead Health Specialist HDNHE - HIS Health

Nicole Hamon Temporary GHNDR Administrative SupportDaniele A-G. P. Jaekel Operations Officer GHN07 OperationsMaud Juquois Economist (Health) GHN13 Health Economics

Amadou Konare Senior Environmental Specialist AFTN1 - HIS Environment

John F. May Consultant GEDDR Reproductive HealthRianna L. Mohammed-Roberts Senior Health Specialist GHN03 Health DeliveryCelestin Adjalou Niamien Sr Financial Management Specialist GGO26 Financial Management

Joao C. Oliveira Senior Economist ECSPE - HIS Economics

Mahamadou Bambo Sissoko Senior Procurement Specialist GGO07 ProcurementM. Driss Zine Eddine El Idrissi Sr Economist (Health) GHN13 Health EconomicsSupervision/ICRAissatou Diack Senior Health Specialist GHN07 Project ManagementDominic Haazen Lead Health Policy Specialist GHN13 Sr Advisor/EconomicsDawn Roberts HQ Consultant ST GHN07 EvaluationM. Vadel T. Hassen HQ Consultant ST GHN07 Economics

Jenny Gold Senior Health Specialist GHN13 Team Member/ICR Author

(b) Staff Time and Cost

Stage of project CycleStaff Time and Cost (Bank Budget Only)

No. of staff weeks US$ Thousands (including travel and consultant costs)

FY11 31.06 192,967.28FY12 39.04 241,861.35

Total appraisal: 70.1 434,828.63FY13 22.31 133,632.35FY14 19.51 164,507.19FY15 16.36 123,033.99FY16 28.10 147,682.74FY17 24.75 150,122.23

Total Supervision/ICR: 111.03 718,978.50

57

Annex 5. Beneficiary Survey Results

Findings from beneficiaries were collected through multiple methods to gain the perspectives of different groups served by the project. These included a series of focus groups facilitated by UNICEF to review the C4D Program, interviews with RBF beneficiaries and interviews conducted in each project region by the PIU to collect feedback on the project’s support to health centers. The findings are described below.

UNICEF focus groups. UNICEF conducted three days of focus groups with project beneficiaries, from February 23-25, 2017. In total, 11 sessions, with about 12 participants per session64 to understand the benefits of C4D activities involving health centers, religious and community leaders, Schools of Husband, female leaders, films and media products and peer education. In total, 89 persons participated, including from ASACO and CSCOM (7), religious and community leaders (9), Schools of Husbands (4), community health agents (7), youth peer educators (48), female leaders (7), and community radio (4). The themes and lessons from these sessions after 9 months of project implementation included the following65, 66

ASACOs built capacity to coordinate community actors. The targeted involvement of different community actors (religious leaders, community agents, ANJE, CFU, etc.) from the outset of the C4D activities built a strong sense of ownership. The regional launch of the activities and training of each community group, built communication skills to influence RH and FP. Local workshops allowed community actors to form coalitions for planning and implementation of the C4D activities.

Interventions were most effective when community leaders understood the positive social norms. Interventions need to encourage community leaders to support new social norms for youth leaders to participate without inhibition. Otherwise, youth leaders are often inhibited by elders and unable to voice issues and make decisions about RH / FP, despite their own knowledge and desires.

Religious leaders served as levers of awareness through sermons they conducted on the topic of RH. They provided influential leadership to engage those with different viewpoints. This required a customized toolkit on RH to support religious leaders. Many religious leaders talked about RH, focusing on the 4 prenatal consultations, the 2 postnatal consultations and the accompaniment of the husband. Others used verses from the Qur'an to recall the traditional positive practices of FP and birth spacing.

The Schools of Husbands helped to lift taboos around the husband’s attitude towards his pregnant or breastfeeding wife. Husbands acted on the 'descriptive social norm' aspect of seeing other men accompany their wives to the health center for pre- and post-natal consultations, communication helped to promote the same attitude and behavior in other men. This experience fueled other ideas such as "The

64 The beneficiaries included representatives from each project region. 65 UNICEF 2017. Rapport de Progrès (Décembre 2016- Février 2017) Accroitre la demande et l’utilisation des Services de Santé de la Reproduction (SR) et Planification Familiale (PF) aux niveaux des CSREF et CSCOM dans les 34 districts au Mali (SC160310).66 C4D participants were interviewed during the ICR mission, from February 19 to March 2, 2017, including six persons from ASACO, peer educators and community health workers.

58

University of Grandmothers" to target more barriers to behavior change to promote mothers to adopt new behaviors.

The activities reinforced the skills of community health agents to follow women in their villages. In the interview, the community health agents reported following about 40 women during the program activities, compared to about 10 previously.

Female leaders from the ANJE and CFU were positive roles models to increase health center frequentation. Their outreach communication going from village to village addressed trust issues around going to the health center to use RH and nutritional services.

The discussions about the television series «La vie de Bijou" revealed that most young people are experiencing similar situations in their lives. This helped to promote dialogue on common RH and FP related issues that are often not discussed.

Youth peer educator (U-reporters) felt empowered, and improved their understanding of RH rights and communication skills. Interviewees reported having bigger dreams now – to be a minister, a doctor, and a child’s right advocate.

RBF assessment.67 Structured interviews and focus group discussions were conducted by the RBF agency with health workers involved in RBF in Koulikoro. The participations represented the Region Health Direction, six CSREFs, 12 CSCOMs, six district authorities, including representatives from the ASACO. In total, about 55 participants were engaged. The focus group design stratified districts and health centers by performance to collect a range of lessons and perspective. This included: i) one district with strong performance in the RBF; ii) one district with weak performance in the RBF; iii) one district that had previous experience implementing RBF before the project. The successes and challenges emerging from one cycle of RBF implementation are summarized below.

Influence on service use. The interviewees reported an observed increase in the use of services in CSCOMs, particularly vaccination, prenatal care and FP. The ASACO promoted the RBF services in the community, encouraging woman to come to the health center. Local radio and women’s groups were also engaged to bring women to the health center. Some interviewees noted seasonal and cultural factors limiting the success of the RBF. For example, the high river water limited access to the health center for pregnancy care in the winter, and health personnel not following-up with women postpartum due to traditions limiting visitors during the first week of the baby’s life. Other CSCOMs reported innovative solutions, such as asking women to rest in the CSCOM for the first 24 hours to provide postpartum care. Some CSCOMs gave soap and mosquito nets to encourage women to come for consultations, as well as reminder calls by mobile telephone.

Quality of services. The interviewees from CSCOMs and CSREF reported that the RBF increased the motivation of personnel to follow standards to deliver quality services. For example, personnel provided counseling around the importance of having multiple prenatal care visits, such that women would return for at least four visits. The RBF also helped to reinforce quality assurance processes providing a

67 Rapport de Capitalisation FBR à la Malienne, L’équipe de l’étude du Consortium KIT-CGIC-CORDAID, March, 2017.

59

quality checklist for the ASACO. All health centers noted that the overall sanitation of CSCOMs, such as birthing rooms, had improved. Some CSCOMs reported using the RBF funds to recruit additional qualified personnel to deliver services. Other CSCOMs reported changing their patient practices, such that all women are seen by qualified personnel during their visit, which was not the case before the RBF.

Motivation of personnel. Interviewees reported that one cycle was not long enough for sustained behavioral change. However, personnel showed a motivation to be recognized for providing quality services, beyond the RBF payment. In addition, the interviewees reported that the RBF motivated health personnel to be punctual, and spend more hours in the health center. Before the RBF, when women came to the CSCOM at night to give birth, qualified personnel were often not available. The opportunity to reinvest the RBF money in the health center was an important motivation. A key challenge expressed by interviewees related to the late transfer of the RBF payments, which was promised monthly and then changed to quarterly.

Coordination of health services. Some interviewees noted the RBF created a team dynamic between the CSCOM, CSREF, ASACO, administration, etc. to better coordinate services. However, others noted that more involvement of the Mayor was needed. Interviewees also noted that the RBF reinforced the relationship between the CSCOM and ASACO, which is politically difficult. Some CSCOMs noted that the ASACO was reluctant to recruit new qualified personnel or purchase needed equipment, and the RBF built the understanding to realize these needs. However, in some cases there was a mistrust that the RBF funds would be kept by the ASACO.

RBF indicator selection. In regards to the indicators, the feedback was that there should be more flexibility in the selection of indicators, such that individual districts could make decisions to address their priorities. Further, the indicators focused on RH, rather than broader health service improvements, which excluded the participation of certain personnel. Further including payment for one prenatal visit, discouraged health personnel to work hard to promote women to have four visits. Moreover, there were few indicators at the CSREF level. The interviewees also noted the need to consider different indicators (and payment levels) for health centers in rural and remote areas, where socio-cultural challenges to attract patients may be more difficult.

RBF training. Most interviewees reported that the RBF training needed to be conducted in the local language to build strong understanding. Further, follow-up supervision and coaching were important to reinforce the training and solve problems, while the availability of vehicles and cost factors limited supervision support.

Verification of the RBF services. Verification by district health teams was reported to be difficult, given the number of CSCOMs to cover with limited resources. There was also a concern that district health teams were not sufficiently independent to verify results of CSCOMs. The verification of the RBF by the community-based organization was not in place long enough for the participants to provide feedback. The main challenges of the verification were the intense process required to collect the data (including working all night) and the lack of understanding around the expected quality of the indicators: for example, one CSCOM lost its credit for giving vaccinations since they did not have a refrigerator.

60

Regional Consultations. Structured interviews were conducted by the PIU in Ségou, Koulikoro, and Sikasso in January and February, 2017. The data collection focused on two topics: the first part was on the demand for RH services and the second part was on the provision of RH services in the targeted regions of the project. The purpose was to understand how training and BCC activities implemented by the MOH and UNICEF benefited these regions. Individual and group interviews were conducted in two districts of each region with the following groups: women and girls using RH services in CSCOMs (53); ASACO president and CSCOM health professionals (18); youth peer educators and their peers (73); religious and community leaders (24); community health agents (57); community radio (8). Beneficiaries’ reactions were similar across regions, and reoccurring themes, successes and challenges are summarized below68.

The following are themes and lessons related to increasing demand for RH services: Campaigns built ownership of RH issues among religious and community

leaders. Campaigns to raise community awareness of RH issues were overall well-received. In all regions, community leaders increased their understanding of issues of birth spacing, age at first marriage, childbirth assisted by skilled staff, antenatal and postnatal care, breastfeeding, and childhood immunization. Many leaders took on RH issues as their own, recognizing the importance to their community. However, some leaders remained reluctant, and strategies to engage these leaders are important.

Communication resources built capacity of community health agents. Stakeholders reflected on what factors helped to build capacity of community agents to serve women. Noted factors included C4D communication training, community events, motorcycles for transportation (Koulikoro), and kits to support communication.

Radio was a social mobilization tool. Radio stations were reinforced by the C4D training, which enabled them to raise awareness through the production of programs, sketches, synchronized football matches and caravans.

Campaigns influenced the acceptance of RH services. In all three regions, women who participated in BCC activities were well-informed about RH issues. Health personnel reported that women come more frequently to the health center because of the C4D activities. Many mentioned that religious leaders gave sermons on RH use, influencing women’s decisions. The activities started to change the acceptance of men to allow women to use the health center – however, many husbands are still reluctant. Also, there are many unaddressed issues, such as the high financial burdens on women to pay for RH services, and stigma to access contraceptives in a public location.

Reinforcing local solidarity funds benefited women and children. The two districts reported the high value of the project support to reinforce local solidarity funds to help address the cost of accessing the health center – membership dues are now paid and women and children are receiving evacuation services. Children were added to the benefits after the training.

Peer educators increased dialogue on taboo issues in the community. Peer educators reported having skills to discuss RH and FP issues.

68 Taken from interview summaries prepared by the PIU.

61

The following are success and challenges related to improving the supply and capacity of RH services: Strengthening capacity of health centers. Health professionals expressed

satisfaction with the contributions of the project to enhance their ability to offer services. In all three regions, the provision of training (anesthesiology, maternal mortality audits, and FP), C4D communication materials and equipment (such as delivery kits) were cited as valuable, particularly in efforts to reduce neonatal and maternal mortality to promote women to use RH services. However, there were many gaps noted in the planning and implementation of the project support. It was noted that some CSREF still do not meet standards for providing delivery services, including having a rest room for after birth. In Ségou, stakeholders reported that ambulances and motorcycles that were supposed to be provided were never received and that there was a significant gap between the planning and implementation of project activities. CSCOMs interviewed in Koulikoro also expressed concern that they had not received the promised equipment. In Sikasso, stakeholders noted that some training activities planned by the health structures were not carried out.

Strengthening RH services through RBF. In Koulikoro, CSCOMs reported that the RBF increased the motivation of providers and follow-up of women throughout their pregnancy, including innovations such as calling them by mobile. However, there was a concern about the short duration of the activity, since it was the second pilot and the whole region was now trained.

Contribution of other development partners. Interviewees reflected that other development partners also contributed to the changes in RH service use. In some cases, providers were unaware of direct project contributions of the World Bank, since donor resources were pooled.

62

Annex 6. Stakeholder Workshop Report and Results

A stakeholder workshop held on February 27-28 explored the achievements, challenges and lessons from the project. Participants included the PIU, the project Steering Committee agencies, the RBF agency representatives, the C4D program coordinators, UNICEF, regional and district representatives from health and social development offices, the delegation that joined Bangladesh, representatives from all of the directorates of the MOH, DNP, representatives from the Ministry of Communication, representatives from the RH donor sub-group, and representatives from the World Bank country office. Findings from the stakeholder workshop, presented in the Table below, were reinforced through stakeholder interviews conducted by the ICR team, from February 19-28.

Comp-onent

Achievements Challenges Lessons

1 Reinforcement of the technical set-up of health structures in the regions of the project (equipment, commodities)

Trained RBF leaders in Benin

All districts of Koulikoro were trained to implement RBF, increasing motivation of ASACO

Capacity building of health providers in obstetrical care and FP

Material purchased without capacity building to manage stock

Need for close coaching on RBF in early cycles

Some RBF indicators misaligned to needs of CSCOMs/CSREF

Not enough RBF cycles for corrective actions

Lack of confidence in RBF verification mechanism

Not enough time for community scorecard or to integrate RBF and community activities

Contract to purchase commodities should include technical support element

RBF should pay for strategic indicators to improve quality of key services in the national/regional health program, and allow flexibility based on variations in district needs

RBF scale-up should be progressive to resolve implementation problems

RBF can build capacity of ASACO to link the health center to the community

2 Bangladesh exchange provided new know-how and recommendations to support population policy and community programs

Diagnosis and training of local solidarity funds in Banamba and Fana

Increased demand to use RH services in health districts through C4D social marketing sessions

Community and religious leaders, adolescents, husbands, health professionals and community agents trained

Post Bangladesh actions need to be supported by SWEDD and future projects – including involving religious leaders in RH, strengthening private sector and NGO partnership, and addressing gender violence

Non-recruitment of NGOs for implementation of activities

Non-implementation of voucher scheme

Knowledge exchange was a strategic tool to build commitment and knowledge for innovative approaches

Partnership with UNICEF enabled decentralized capacity building to achieve results

Participatory involvement of communities in the analysis and identification of actions built ownership

Targeting authorizers

63

in RH communication Mass media reached

larger population, influencing knowledge on RH

Short-timeframe of C4D

Limited scope of solidarity fund support

who influence decisions to use services in the C4D activities influenced RH acceptance

C4D communication targeting specific actors, with coherent messaging rapidly mobilized a social change process

Multisector collaboration was key to C4D success

3 Acquisition of Vehicles for MOH

Training of DFM in TOMPRO

Conferences on nutrition in Benin; on FP in Indonesia; and on private sector partnership in Senegal

Experts from DRH trained in leadership

Coordination meetings

The delay in the start-up of the project

The stability of technical and financial staff coordinating the project

Following complex procedures

The cancellation of vehicles for supervision due to late timeframe

Coordination of many national implementers

Early start-up of activities enables results

Staff loyalty allows for realization of activities

Large activities minimized procedures, rather than many smaller activities

Cancellation of activities demotivated implementers

Build capacity for field supervision early on

Strengthen regional/district coordination for results

64

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

The Borrower’s ICR (Rapport d’activités de fin de Projet de Renforcement de la Santé de la Reproduction (PRSR) du 15 mars 2013 au 28 février 2017) was submitted to the Bank in early August 2017. The brief report was prepared drawing on data collected from the final stakeholder workshop and project reports.

The first section of the report presents the project context, development objectives and design: how the project reinforced the National Reproductive Health Action Plan and PRODESS program; the PDO and PDO indicators; the project beneficiaries in the four project regions of Koulikoro, Sikasso, Ségou and peri-urban Bamako; the three project components and associated activities; the government units responsible for project implementation; and changes made to the project during the 2014 restructuring, the 2015 MTR, and the 2016 restructuring to reallocate resources at the end of the project. The second section of the report describes project outputs by component and subcomponent. These outputs are summarized in the below table.

Component 1 – Strengthening Supply and Quality of Reproductive Health (RH) Servicesi) Results-based financing (RBF) pilot: The project trained a core group of 37 ministry staff in RBF, including 12 officers from the Koulikoro region. This allowed an accelerated RBF approach to be implemented in 205 CSCOMs and 10 CSREFs in Koulikoro region in 2016. Each health center established an RBF contract with local authorities, developed a work plan for improving RH indicators, quality of services and capacity building of providers. In total, 1,300 people were trained using a cascade training approach to inform 3,530 persons in Koulikoro on RBF procedures. This support resulted in 200,977 women accessing a range of RH services in 2016 (unverified numbers from first RBF report).

ii) Improving the supply of contraceptives: The project trained staff in health centers in family planning, specifically long-term methods, and financed a stock of contraceptive products. This support increased the availability of contraceptives especially, long-term methods (IUDs and implants), such that less than 5 percent of structures have experienced an inventory shortage of implants and less than 10 percent had a shortage of IUDs.69

iii) Capacity building in RH and obstetrics: The project supported the in-service training of 806 physicians, midwives, nurses and matrons in the areas of RH, family planning, emergency obstetric care, among others. This improved the quality of services in CSCOMs and CSREFs, especially in the areas of anesthesia and resuscitation. The project also funded safe delivery and caesarean delivery kits for child birth, as well as equipment for the delivery of RH services. The activities filled a critical need to improve the technical base and consequently the quality of services for mothers and children.

Component 2 – Increasing Demand for RH Servicesi) Strengthening outreach services and behavioral change:The project engaged UNICEF to accelerate demand and use of RH / FP services through the C4D program in 2016. This C4D program enabled a mass social mobilization in the regions of the project, involving all possible target groups at the central and community level. The C4D program included a multi-media campaign and activities to promote the use of contraceptives and RH services, encouraging child vaccination, the use of neonatal and postpartum health services and FP. The C4D program resulted in 208,468 women and adolescents using FP services. Further, through the funding of the action

69 Data from the USAID OPSSANTE portal, Monitoring Tool for Health products, 2017.

65

plan activities in CSCOMs in 2015, the project supported community outreach services, such as maternal mortality audits.

ii) Improving financial access:The project, through the Unit for Decentralization and De-concentration Support (CADD), carried out the diagnosis and training to reinforce local solidarity funds in Banamba and Fana, and 400 women benefited from emergency transport services for reasons related to childbirth and one of the centers received an ambulance. The voucher program was not implemented due reasons of alignment with the national health policy and concerns of sustainability at the end of the project.

iii) Promoting a family planning (FP)-conducive environment:The National Population Directorate (DNP) organized an advocacy seminar for journalists, religious and traditional communicators to promote FP. Further, the project organized a study trip to Bangladesh, which enabled the members of the delegation to gain knowledge for the finalization of the National Population Policy document.

Component 3 – Social Accountability, Project Management and M&E:The PIU coordinated and monitored all project activities (technical activities, monitoring of indicators, financial management activities, and contracts for staff and implementing partners, such as UNICEF and the RBF pilot). Further, the PIU organized the project restructuring sessions, the MTR mission, the Steering Committee meetings, the World Bank supervision missions, and meetings of implementers for project coordination.

The Borrower’s ICR concludes with a summary of data on the project’s PDO indicators. The report uses the MICs 2015 report as the end line source of data for 2017. The final section of the report also includes a summary of the financial disbursement for the project by year and by component.

66

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

N/A

67

Annex 9. List of Supporting Documents

Key project documents Mali Strengthening Reproductive Health (SRHP) Project Appraisal Document (PAD), World Bank,

November 23, 2011 Integrated Safeguards Data Sheet, World Bank, March 8, 2011 Financing Agreement, February 24, 2012 Letter to extend effectiveness date, November 22, 2012 Restructuring Papers (October 2 2014, December 29, 2016) Comments from internal review of project documents, 2011

Materials from project implementation Aide-memoires of project missions, 2011 to 2017 (including from MTR) Annual technical program and budget documents, “Programme Technique et Budget Annuel”, 2012 to

2016 Back-to-Office-Reports and Statements of Mission Objectives, 2011 to 2014 Bangladesh Knowledge Exchange report, May 2016 Documents on project steering committee and minutes of meetings, 2013-2017 Financial management and audit reports, 2013 to 2017 Implementation Status and Results Reports, sequence 01-10 (April 3, 2012 to February 27, 2017) Official correspondence and management letters, 2011 to 2017 Project Implementation Manual, "Manuel de Procedures Administratives Financieres et Comptables" Quarterly and annual project activity reports, 2013 to 2017

Data for ICR Data on solidarity funds from Fana and Banamba, from the membership logs of the ASACOs Presentations from the stakeholders’ workshop for the SRHP, February 27-28, 2017 Summary of beneficiary interviews conducted by the PIU, from January to February 2017 Summary of stakeholder interviews conducted by the ICR team during the ICR mission Système Local d'Informations Sanitaires, Local Heath Management Information System (HMIS),

2010, 2011, 2012,2013,2014,2015, and preliminary data from 2016 UNFPA, 2016. "Programme UNFPA SUPPLIES Rapport final de l’enquête 2015 sur la disponibilité

des produits contraceptifs et produits de santé maternelle, Mali." UNICEF, Mali Multiple Indicator Cluster Survey 2015. Data on indicators in the project regions was

shared by UNICEF in February 2017, since final survey report was not yet available. Ministry of Health and Public Hygiene (MOH) and UNICEF, February 2017. “Étude Finale du

projet : Amélioration de l’accès et de l’utilisation de services de santé de la reproduction de qualité pour les femmes en âge de procréer dans les régions de Koulikoro, Sikasso, Ségou et le District de Bamako."

RBF agency reports and materials, 2016 to 2017 UNICEF Reports, documents and multi-media materials from C4D program, 2016 to 2017 USAID OSPSANTE portal, 2017, Monitoring Tool for Health Products, https://ospsante.org/

Key country documents PRODESS II and III strategy documents, Government of Mali Plan d’action National de Planification Familiale du Mali 2014-2018 World Bank, 2015, Systematic Country Diagnostic, Mali World Bank, 2015, Mali – Country Partnership Framework for FY16-FY19 World Bank, 2011, Country Assistance Strategy Progress Report, April 28, 2011 World Bank, Mali – Interim Strategy Note for FY14-15

68

https://ospsante.org/

MAP

I N S E R T

M A P

H E R E

AFTER APPROVAL BY SENIOR GLOBAL PRACTICE DIRECTOR

AN ORIGINAL MAP OBTAINED FROM GSD MAP DESIGN UNIT

SHOULD BE INSERTED

MANUALLY IN HARD COPY

BEFORE SENDING A FINAL ICR TO THE PRINT SHOP.

NOTE: To obtain a map, please contact

the GSD Map Design Unit (Ext. 31482)

A minimum of a one week turnaround is required

69