Document ofThe World Bank

FOR OFFICIAL USE ONLY

Report No: 20602

IMPLEMENTATION COMPLETION REPORT(24640)

ON A

CREDIT

IN THE AMOUNT OF SDR 35.0 MILLION (US$48.0 MILLION EQUIVALENT)

TO

THE ISLAMIC REPUBLIC OF PAKISTAN

FOR THE FAMILY HEALTH II PROJECT

JUNE 26, 2000

HEALTH, NUTRITION AND POPULATION SECTORSOUTH ASIA REGION

This document has a restricted distribution and may be used by recipients only in the performance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective May 10, 2000)

Currency Unit = Pak Rupees (Rs.)Rs. 1.00 = USS 0.0193

US$ 1.00 = Rs. 51.8875

FISCAL YEARJuly I - June 30

ABBREVIATIONS AND ACRONYMS

BHU Basic Health UnitCDC Communicable Disease ControlCPR Contraceptive Prevalence RateDFID Department for International Development (United Kingdom)DHDC District/Divisional Health Development CenterDoH Department of HealthDOTS Directly Observed Treatment Short-courseEOC Emergency Obstetric CareEPI Expanded Program of ImmunizationFIC Fully Immunized Childrenfp Family PlanningGoP Government of PakistanGoBalochistanGovernment of BalochistanGoPunjab Government of PunjabHDC Health Development CenterHMIS Health Management Information SystemICR Implementation Completion ReportICT Islamabad Capital TerritoryIDA International Development AssociationIMR Infant Mortality RateIUD Intra-uterine DeviceKfW Kreditanstalt fur Wiederaufbau (Germany)LHV Lady Health VisitorMCH Maternal and Child HealthMoH Ministry of HealthMTR Mid-term ReviewNBF Not for Bank FundingNCB National Competitive BiddingNGO Nongovernmental OrganizationPC- I Planning Commission Form IPDHS Pakistan Demographic and Health SurveyPFFPS Pakistan Fertility and Family Planning SurveyPHNS Public Health Nursing SchoolPIHS Pakistan Integrated Household SurveyPHC Primary Health CarePHDC Provincial Health Development CenterRHC Rural Health CenterRH Reproductive HealthQAG Quality Assurance GroupSAP Social Action ProgramSAPP Social Action Program ProjectSDR Special Drawing RightsTB TuberculosisTBA Traditional Birth AttendantTT Tetanus ToxoidWHO World Health Organization

Vice President: Mieko NishimnizuCountry Manager/Director: John W. Wall

Sector Manager/Director: Richard Lee SkolnikTask Team Leader/Task Manager: Hugo Diaz-Etchevehere/Muhammad Bashirul Hag

FOR OFFICIAL USE ONLY

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 136. Sustainability 147. Bank and Borrower Performance 158. Lessons Learned 189. Partner Comments 1910. Additional Informnation 19Annex 1. Key Performance Indicators/Log Frame Matrix 20Annex 2. Project Costs and Financing 23Annex 3. Economic Costs and Benefits 26Annex 4. Bank Inputs 27Annex 5. Ratings for Achievement of Objectives/Outputs of Components 30Annex 6. Ratings of Bank and Borrower Performance 31Annex 7. List of Supporting Documents 32

Annex 8. Borrower/Implementing Agencies's Contribution to the ICR 34Annex 9. Cofinanciers' Contribution to the ICR 43

This document has a restricted distribution and may be used by recipients only in theperformance of their official duties. Its contents may not be otherwise disclosed withoutWorld Bank authorization.

Project ID: P0 10414 Project Name: FAMILY HEALTH II

Team Leader: Muhammad Bashirul Haq TL Unit: SASHP

ICR Type: Core ICR Report Date: June 25, 2000

1. Project Data

Name: FAMILY HEALTH 11 L/C/TF Number: 24640

Country/Department: PAKISTAN Region: South Asia RegionalOffice

Sector/subsector: HT - Targeted Health

KEY DATESOriginal Revised/Actual

PCD: 12/17/90 Effective. 07/15/93 07/12/93

Appraisal: 05/25/92 MTR: 06/15/96 06/11/97

Approval: 02/23/93 Closing: 06/30/99 12/30/99

Borrower/lImplementing Agency: GOVERNMENT OF PAKISTAN/BALOCHISTAN & PUNJAB DOH

Other Partners: Department for International Development (DFID), United Kingdom;Kreditanstalt fur Wiederaufbau (KfW), Germany

STAFF Current At AppraisalVice President: Mieko Nishimizu Joseph D. Wood

Country Manager: John W. Wall Paul Isenman

Sector Manager: Richard Lee Skolnik Roberto Cuca

Team Leader at ICR: Hugo Diaz-Etchevehere Christopher D. Walker

ICR Primary Author: Muhammed Bashirul Haq

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=HighlyUnlikely, HlU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability. L

Institutional Development Impact: SU

Bank Performance: S

Borrower Performance: S

QAG (if available) ICRQuality at Entry:

Project at Risk at Any Time: Yes

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:

The project objective was to improve the health status of the population in general, but withspecial emphasis on high-risk and under-privileged sections of the community in the Provinces ofPunjab and Balochistan and the Islamabad Capital Territory (ICT). This objective was to beaccomplished through (a) increasing the availability and quality of maternal health services,including family planning; (b) improving the quality and integration of primary health careservices, particularly for communicable disease control (CDC); and (c) building the institutionalcapacity to realize the above objective.

The objectives were in line with the National Health Policy of the Government of Pakistan (GoP)announced in May 1990, and the Bank's Country Assistance Strategy (CAS) of June 12, 1992.The project design took into account the lessons learnt from an earlier population project, theInternational Development Association (IDA) experience in education projects, and other donors'experience in the health sector including the Asian Development Bank (ADB) and the UnitedStates Agency for International Development (USAID).

The project was co-financed with the Department for International Development (DFID) andKreditanstalt fur Wiederaufbau (KFW). The project was carried out in the context of the Bankand other donors' work with government through the Social Action Program Projects I and II(SAPP I and SAPP II). The Social Action Program provided a framework within whichgovernment addressed key policy matters in the health sector and financed certain inputs at thecommunity level. As such, the Social Action Program helped to create an enabling environmentfor the Family Health II Project.

The borrower's commitment to changing the existing situation was reflected in its PolicyFramework Paper at that time, which committed it to increases in development and recurrentspending for Primary Health Care (PHC), especially Maternal and Child Health (MCH) care; aPublic Sector Adjustment Loan; and undertakings in the Social Action Program in the form oftime-bound plans for policy and institutional reforms and improving program implementation.

The project, in retrospect, had a few shortcomings. It was complex, as it involved nineimplementing agencies and six nongovernmental organizations (NGOs) with a large geographicalspread covering 61 districts and about 54 million rural population. The complexity increased as itwas funded by three donors under parallel financing, who had different rules and procedures foroperations. For instance, the project became effective at different times (IDA - July 12, 1993;DFID - 14 months after IDA; and KfW - 16 months later) and had different closing dates (IDA -June 30, 1999; DFID - December 1999, and KfW - December 2000). At the design stage,agreement was not reached on a framework for joint monitoring by the donors, or onperformance indicators which would form the basis of project monitoring. It took nearly twoyears to reach an agreement on the indicators.

- 2 -

3.2 Revised Objective:

There was no change in the original objective.

3.3 Original Components:

The project components were ambitious in terms of scope and targets and were not in line withthe technical and management capacity of the implementing agencies. The project had four majorcomponents:

* Strengthening Health Services (US$64.5 million). This component included: (a) theintroduction of an enhanced package of maternal health services, including family planningand (b) integrating and expanding communicable disease control activities, mainly in ruralareas but with some experiments in urban areas. The project also included support foremergency referral services including obstetric emergencies and expanding the health servicesof six well-established NGOs.

i Staff Development (US$31.2 million). This component included: (a) the introduction of acomprehensive, continuing in-service training program for existing PHC staff and (b)expanded female paramedical training and improved quality.

* Management and Organization Development (US$13.2 million). This component includedactivities to improve the institutional capacity for planning and management.

* Federal Component (US$5.2 million). This component included: (a) developing primaryhealth services in the ICT and (b) strengthening health systems research (HSR) capacity.

3.4 Revised Components:

Project restructuring in May 1997 sought to bring the project in closer alignment withimplementation capacity. It involved changes in the scope of the project components in terms ofreduction of targets and deletion/addition of some activities. The deleted activities had lowpriority or had experienced prolonged implementation delays, or resources were made availablefor them from outside the project.

New activities introduced were to pilot interventions such as decentralization of health services tothe district level, tuberculosis (TB) treatment using the directly observed treatment short-course(DOTS), reproductive health, and community-based education for medical students (for PHCorientation). However, the broad original project components remained unchanged.

The total estimated project cost declined as a result of restructuring by about 25%. The IDAcredit was reduced from its original amount of US$48 million equivalent to US$28.73 millionequivalent at the time of restructuring. Later, in April 1999, the Bank further canceled an amountof US$3.7 million equivalent from the IDA credit because of slow implementation and exchange

-3 -

rate-related savings.

3.5 Quality at Entry:

In this Implementation Completion Report (ICR), the quality at entry is rated as unsatisfactory.The project was consistent with the CAS and sector strategy. The programs of the two provinceswere fully in accordance with the Social Action Program proposed by the Government to ensurebetter implementation of ongoing activities in the social sectors. However, the project turned outto exceed the implementing agencies' capacity, a fact that was acknowledged at restructuring.

There was no Quality Assurance Group (QAG) review at the time of this project preparation.However, a QAG supervision assessment in August 1997 noted that the project design was soundbut complex and extremely ambitious, and that the project was not ready for implementation atapproval.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

The Staff Appraisal Report (SAR) did not make very clear the objectives of the project. Nor did itmake very clear exactly what was to be done to reach the objectives as stated, or offer a smallnumber of indicators that would be used to measure the achievement of outcomes. Rather, theSAR noted in the annexes a large number of indicators, some of which related to the SocialAction Program and some to this project. However, it was not clear what was the exact relationof the two sets of indicators.

As the project unrolled, and as confirmed at the Mid-term Review (MTR), the Bank andGovernment agreed on a small set of mostly process indicators that could be used to track howthe project was doing. These focused largely on the amounts of services used, and clients formaternal health services, the number of places where appropriate services would be offered, andimmunization rates, as well as some indicators relating to the building of capacity.

The comments below on the achievement of objectives follow the refinement of objectives as theywere elaborated over the first few years of the project and at the MTR.

Overall, the project inputs, put in place in an enabling environment assisted by the Social ActionProgram, contributed to an important improvement in maternal health services, immunization, thestart of TB control, and helped to build important capacity in the health sector in two provinces.Thus, the project is deemed to have been satisfactory in meeting its objectives.

4.1.1 Increasing the availability and quality of maternal health services, includingfaimily planning

Based on Department of Health (DoH) service statistics, a rising trend in utilization of MCHservices was observed in the rural health facilities during the project life. DoH Balochistanprovided antenatal care to 11.5% of pregnant women in 1996 which increased to 19.8% in1998 while the number of supervised deliveries by trained birth attendants increased from3.8% to 6.3% in the same period. In rural Punjab, antenatal care services increased from 11%

-4 -

in 1996 to 23.6% in 1998 while the proportion of deliveries supervised increased from 5.7% to11% during this period. The envisaged target for the provision of antenatal care to 20% ofpregnant women has, therefore, been largely met. Higher utilization of MCH services couldprobably have been achieved if planned social mobilization activities had been implemented.

Most PHC facilities are currently providing family planning services as compared to a smallpercentage in 1993. This was believed to be under 5%, although a benchmark was notestablished in 1993 as provision of family planning services was not part of the PHC package.At project close, 83% of PHC facilities having female staff in Balochistan and 82% in Punjabwere providing family planning services and advice. The total clients for family planningservices served during calendar year 1999 were 566,000 in Punjab and 201,000 in Balochistan,which were almost three times more than the ones served two years before.

A Health Facility Survey of 128 PHC facilities, conducted in April 1999 to compare thesituation in 1993 with 1998, showed an increase over the base year in antenatal clients (52% inPunjab and 120% in Balochistan) and family planning clients (67% in Punjab and 126% inBalochistan); an improvement in the availability of contraceptives and essential drugs (anincrease by more than 100% for 15 essential drugs in Punjab and 18 in Balochistan out of 30essential drugs); and an improved female health staff situation (with females working in 87%of sample PHC facilities in Punjab and 82% in Balochistan).

A study conducted in 1999, covering 12,000 married women from 2,000 households in 80villages from eight districts in Punjab, and supplemented by a review of 22 health facilities,revealed that 18.2% of all deliveries were attended by a trained person (12.1% by doctors and6. 1% by lady health visitors (LHVs); 11.8% of deliveries took place in a hospital; 37.2%received antenatal care from an LHV or a doctor and the mean number of antenatal visits was3.9; 19 out of 22 sample facilities were providing MCH care; 21 sample facilities had an LHVin position; and 13 out of 15 Tehsil hospitals and rural health centers (RHCs) had a femaledoctor in position.

The service delivery outlets of provincial DoHs and of the provincial Population WelfareDepartments were the main source of servicing the rural clients for family planning services.The increase in the contraceptive prevalence rate (CPR) among the rural population andimproved knowledge of contraception can therefore be partly credited to the project.According to the Pakistan Demographic and Health Survey of 1990/91, the CPR was 13% inPunjab and 2% in Balochistan, while the percentage of currently married women who knew atleast one method of contraception was 80% in Punjab and 36% in Balochistan. The PakistanFertility and Family Planning Survey of 1996/97 revealed that the CPR had increased to 26.8%in Punjab and 7.1 % in Balochistan, while the percentage of currently married women whoknew at least one method of contraception had risen to 94% in Punjab and 78% inBalochistan. The comparison of CPR in rural populations of Punjab and Balochistan between1994/95 and 1996/97 also showed an upwards trend; it had moved from 14.1% to 22.2% inPunjab and from 1.9% to 4% in Balochistan.

The provincial DoHs substantially improved gender balance through increased recruitment offemale staff and provision of incentives. The stock of female paramedics at PHC and first

- 5 -

referral facilities increased more than two fold in Balochistan from 243 in 1993 to 641 in 1998and by 27% in Punjab from 2,730 in 1993 to 3,462 in 1999. More specialists (63) andequipment were made available at first level referral hospitals during 1997/99.

Support to six major NGOs for upgrading their maternity services is a successful example ofpublic-private partnership. Overall, there is a need to focus on quality of care throughfollow-up interventions for improving the outcome of obstetric emergencies and maternal andchild care.

4.1.2 Improve the qualitv and integration of primary health care services, particularly for communicabledisease control

Improvements were registered in malaria control and immunization of the target population.The pilot for TB control through DOTS strategy in Balochistan province was highlysuccessful, and World Health Organization (WHO) recommended expansion of the program.The integration of CDC programs was one of the legal covenants of the project but progresswas only registered in development of a conceptual framework for integration.

Immunization. The Pakistan Demographic and Health Survey 1990/91 revealed that theproportion of fully immunized children (FIC) in the age group 12-23 months was only 38.6%in Punjab and 17.8% in Balochistan, while pregnant women who received two or more dosesof tetanus toxoid (TT) vaccine were 23% in Punjab and 6% in Balochistan. According to thePakistan Integrated Household Survey (PIHS) of 1996/97, the immunization coverage ratesshow substantial improvement: FIC of 12-23 months age were 51% in Punjab (urban 57% andrural 49%) and 59% in Balochistan (urban 70% and rural 57%). The PIHS did not recordcoverage of pregnant women with two or more doses of TT. However, a TT coverage studyof 1999 conducted in Punjab province showed that 39% of married women had received 2-5doses of TT vaccination.

Malaria control activities. Surveillance activities have markedly improved in the last threeyears. The number of patients screened through blood tests almost doubled while the numberof confirmed malaria cases declined by about two and a half times in Balochistan and aroundfive times in Punjab. The proportion of falciparum malaria also came down from 37% to 23%in Balochistan and from 48% to 18.6% in Punjab.

TB control using DOTS strategy. The program was introduced on a pilot basis in Balochistan,and 167 sputum positive cases were detected during January 1998-October 1999. The sputumconversion rate was 89% while the cure rate was 82%. This led to a change in TB treatmentpolicy from a conventional clinic based approach to the DOTS strategy.

4.1.3 Building the institutional capacity

Continuing education. The establishment of 38 Health Development Centers (HDCs) at theprovincial and divisional/district levels led to the evolution of a system of ongoing in-servicetraining for service providers and managers. These HDCs will continue functioning asprovincial governments transfer their operational costs to their regular budgets. The targets set

- 6 -

for in-service training were also met. However, an independent review rated the quality ofin-service training as only marginally satisfactory.

Improving gender balance. Expansion of pre-service training of female paramedics wasachieved through seven Public Health Nursing Schools (PHNS) which were established orupgraded under the project. Communication programs initiated in Balochistan focused onrecruiting local young women for admission to the PHNSs instead of young women fromother provinces. For instance, all 85 trainees selected in 1999 were local young women, ascompared to just one in 1993. However, the quality of training was found marginallysatisfactory in two newly established PHNSs in Punjab province.

Improved planning and management capacity. Long- and short-term training courses abroad forDoH staff have substantially contributed to establishing a team of public health practitioners withskills in management and planning. The Health Planning Cells in the provinces are now staffed bya group of trained professionals.

Decentralization of health services to district level. The provincial DoHs made significantprogress in the decentralization of powers. Achievements in this regard include:conceptualizing a model after two years of field testing; application of the tested modelseparately at district and hospital level; and initial development of administrative and financialprotocols. The Government of Punjab also enacted an Autonomy Bill in November 1998 toprovide legal cover to the health sector decentralization initiatives.

Health Management Information System (HMIS). The system has been established forfirst-level health care facilities at outpatient level in both provinces. Nevertheless, the use ofHMIS data in planning and management is only starting to take roots.

4.2 Outputs by components:

The achievement of the physical targets, as revised during the MTR, is satisfactory. Thecomponent wise project inputs described below are only the broad details and do not representthe entire picture.

Component 1: Strengthening Health Services

MCH services were improved by the upgradation of 30 dispensaries with the addition of anMCH section; construction of 102 residences for LHVs; supply of equipment including 2,300intra-uterine device (IUD) insertion tables and kits, 2,000 MCH kits, 758 baby resuscitation kits,and weighing scales; in-service training of LHVs in areas of MCH, family planning and health andnutrition education; supply of 24 vehicles for supervision; and improved availability of femalehealth staff, medicines and logistics. The skills of 4,800 practicing traditional birth attendants(TBAs) were also enhanced through training along with the supply of delivery kits.

Health and Nutrition Education. Modest contributions were made through in-service training inareas of interpersonal communication (IPC) and nutrition education, and preparation of two videofilms for IPC training and TB DOTS promotion. A communication campaign for Student Health

- 7 -

Education and Parent Education was highly successful for motivating local young women inBalochistan to join paramedic and nursing schools.

Preventive Programs. A joint WHO, World Bank, and DFID mission reviewed implementationof TB control using the DOTS strategy in a pilot district of Balochistan in March 1999. Theresults were promising, and the joint review mission recommended program expansion. After thereview mission, DoH Balochistan expanded TB control to another eight districts. Currently 90%of TB cases are supervised by the community within the DOTS approach. A building forestablishing a Public Health Laboratory was completed in Balochistan province, but DoH couldnot fully equip it before project closing for lack of timely procurement of all needed equipment.

The DoH Punjab strengthened its malaria control program by providing 2,595 bicycles foroutreach workers, 242 motorcycles for supervisory staff, repair of 27 vehicles, and antimalarialdrugs. The DoH Balochistan provided 50 motorcycles, 100 spray pumps, 11 mini-foggers, andantimalarial drugs.

DoH Punjab strengthened its immunization program, using the project's KfW grant, byproviding 86 million disposable syringes, 10 refrigerated vans, eight cold rooms, 300 ice linerrefrigerators, 100 deep freezers, 702 vaccine storage refrigerators, 80 generators, 800 cold boxes,2,500 vaccine carriers, 9,000 flasks, 2,500 dial/VP thermometers, and eight cold chain repair kits(for some items the delivery is planned up to December 2000). The Expanded Program ofImmunization (EPI) program was also provided with 1,258 bicycles for outreach work, 252motorcycles, and 106 vehicles for monitoring and supervision. The cold chain and supervisoryvehicles, if maintained properly, should remain functional for the next ten years. The inputs inBalochistan were restricted to major repair of 23 vehicles, supply of 10 new vehicles, and twomnillion disposable syringes.

Upgrading referral services. The KfW grant contributed to refurbishing 82 district and tehsilhospitals in the areas of blood bank, diagnostic and monitoring equipment (laboratory,anaesthesia, sonography, X-ray), MCH equipment, and provision of medicines and other supplies.TIhe equipment to refurbish Intensive Care Units of Balochistan district hospitals reached in March2000, while in Punjab some equipment is planned to reach end users up to December 2000. TheIDA Credit supported procurement of 24 ambulances for district hospitals. The two provincialDoHs recruited 63 specialists during 1994/99 (44 in Punjab and 19 in Balochistan) to improveemergency handling capacity including obstetric and paediatric emergencies. However, it is tooearly to assess the impact of additional resources in enhancing the quality of and access toemergency services.

Pilot-testing Reproductive Health package. The DoH Balochistan in 1998 developed aReproductive Health package for testing on a pilot basis. The staff training, baseline survey, andimprovements in buildings were completed. An ambulance was also assigned for obstetricemergencies. Implementation is planned during 2000/01 using the Govemment's owndevelopment budget.

-8 -

Improvement in staff mix and skill mix. The DoH Balochistan recruited 18 female supervisors,398 female paramedics, and 80 female doctors during 1993-1999 at district level and below,thereby improving gender balance. Since 1997, the DoH Punjab recruited 1,862 technical staffcomprising LHVs, male medical officers, women medical officers, and specialists at tehsil anddistrict hospitals. However, PHC facilities in hard to reach areas continue to face shortages offemale paramedics. Two hundred and fifty two posts of LHVs for example, were vacant in Punjabprovince by the end of the project.

Support to NGOs for maternal health through KfW grant. In Balochistan, the FarnilyPlanning Association of Pakistan (FPAP) was assisted in establishing a maternity hospital atQuetta and in upgrading a maternity facility in the interior of the province. The latter facility isoperational while the first one would be operational by December 2000. The Lady DufferinMaternity Hospital, Quetta, was also assisted in expanding its nursing school and renovating thehospital building. In Punjab, five NGOs were assisted in establishing or expanding MCH andmaternity services in three cities. All facilities would become functional by December 2000.

Expansion of Local Government MCH services. Four MCH centers were constructed for theMunicipal Corporation, Quetta, besides supply of two dump trucks for sanitation and sprayequipment. However, the MCH centers have been awaiting commissioning for one year.

The DoH Punjab could not implement the planned Urban MCH activities in two MunicipalCorporations and inputs for sanitation because these municipalities failed to provide land forconstruction and funds for incremental operating costs. With the DFID grant, 30 MCH centers ofthe Lahore Metropolitan Corporation (LMC) were strengthened along with a health promotioncenter and training in reproductive health. The continuation of these interventions at present levelis unlikely because of budgetary shortfalls.

Component 2: Staff Development

Provincial Health Development Centers (PHDCs) and Divisional/District HealthDevelopment Centers (DHDCs). Balochistan established seven centers (one PHDC and sixDHDCs) housed in newly constructed buildings with an attached hostel, while Punjab establishedone PHDC and 30 DHDCs (two DHDCs in Punjab were not functional at project closing). Theseinstitutions provided support to the DoHs in the areas of management development, humanresource development, health and nutrition education, and monitoring and evaluation. The lastfunction was, however, not fully developed.

The DoH Balochistan prepared 33 in-service training curricula and met the in-service trainingtargets. For example, 2,710 persons were trained and retrained in technical areas and 2,744 inmanagement areas including training in HMIS. In Punjab, the PHDC prepared 20 in-servicetraining curricula and met 99% of the in-service training targets. Transfer of trained staff fromthese centers and their replacement with untrained staff was a major concern during projectimplementation.

-9-

An independent review of PHDCs and DHDCs revealed that the impact of training in enhancingthe skills of the staff was less than desirable. The recommendations of the review were discussedduring the ICR mission, and there was agreement to institute appropriate measures to retrain staff,vacant posts at PHDCs and DHDCs, and that PHDC would continue to review the curricula forcreating a better balance between theory and hands-on training.

Expansion of training for female paramedics, nurses, and postgraduate public healthtraining for doctors. In Balochistan the project substantially contributed to improving thephysical facilities (construction of hostels, MCH center, labour rooms, staff offices, equipmentand furniture) and in meeting additional staffing needs of already functioning schools, includingthree PHINSs and one Nursing School. The project also initiated educational programs to attractlocal young women for pre-service training as paramedics and nurses. The campaign was highlysuccessful. At present, all LHV trainees are local young women while in 1993 there was only one.The project, in conjunction with an ADB-funded project, contributed to increasing enrollmentcapacity for LHVs in PHNSs from 25 in 1993 to 85 in 1999. DoH also established an Institute ofPublic Health with a qualified faculty of 18 members. The first batch of trainees for a diploma inpublic health was inducted in April 2000.

In Punjab the two PHNSs at Lahore and Multan were substantially improved. In 1998, DoHcommissioned two new PHNSs in temporary premises. The permanent buildings of these twoschools and attached hostels are now complete and occupied. The newly commissioned PHNSshave only 9 tutors against the requirement of 30, based on the standards set by the PakistanNursing Council. The DoH plans to provide additional furniture and equipment in these schoolsand in all the PHDC/ DHDCs up to December 2000 using the KfW grant. The enrollment ofLHVs has increased from 370 in 1994 to 520 in 1999.

Technical Assistance (TA) and Fellowships. In Balochistan technical skills of DoH staff wereenhanced in relevant fields through 13 short-term and seven long-term training courses. Technicaland management capacity of the DoH was also expanded through 51 short-tern and ninelong-term consultants.

In Punj'ab technical assistance was provided through 18 long-term and 38 short-term consultants.The project also funded 15 long-term and seven short-term fellowships in various disciplines ofpublic health. All fellows returned back and were working in the DoH by the end of the project.

Eleven studies were done, covering a health facility survey; quality review of in-service training;quality review of civil works; unit costs of first level care facilities; institutional review of PHDCand DHDCs; situation analysis on administrative positions for females in DoH Punjab; tetanustoxoid coverage study; assessment of health services development; equipment effectiveness audit;community participation in the Lady Health Workers program; and assessment of human resourcedevelopment.

Focus group discussions in December 1999 revealed that long-term consultants were helpful ininstitutional strengthening. However, there were mixed feelings on the usefulness of short-termTA. All the studies were rated as useful except the last two.

- 10-

Twining arrangements. The PHDCs in Punjab and Balochistan established linkages with theInstitutes of Public Health at Quetta and Lahore in the area of in-service training, with theProvincial Population Welfare Departments for family planning services, with BalochistanUniversity, and with PHDCs in other provinces.

Component 3: Management and Organization Development

Decentralization. In Balochistan major preparatory work for decentralizing health systems to thedistrict level in six districts was completed a year before the project closing including short-termin-service training and re-training of health staff in supervision and management. The DoHsubmitted a proposal to the government in 1998, outlining the administrative and financialchanges required at the district level. However, no decision was taken by the Government by theproject closing date.

The Government of Punjab enacted an Autonomy Bill in November 1998 to decentralize healthmanagement to the district level, as well as the management of tertiary care hospitals. Pilot testingfor decentralizing health management to the district level in four districts showed improvements inservice delivery and governance aspects. The DoH submitted a proposal in October 1999 forapproval of the Government, outlining the administrative and financial changes required at thedistrict level. The decision to proceed with decentralization to the district level has been delayedbecause of the change of Government in October 1999.

Health Management Information System (HMIS). The district offices of the DoHs wereprovided computers and supplies for the HMIS and staff training. In Balochistan, 56% of all PHCfacilities were reporting on HMIS instruments in 1998 (against a target that 90% PHC facilitieswould start reporting by June 1999). The reporting rate in Punjab rose from 11% in 1994 to 95%in 1999 against a target of 95%. There were deficiencies in data analysis, feedback and use of datafor management and planning.

Component 4: Federal Component

The Pakistan Medical Research Council (PMRC) organized 38 orientation and eight in-depthworkshops for capacity building in Health Systems Research. More than 800 participants weretrained. In addition, two data analysis and report writing workshops were also arranged, andskills of 42 participants were enhanced. Thirty-four research studies in priority areas(excluding the National Health Survey of Pakistan) were completed and disseminated.Equipment was also provided to six research institutions for strengthening their data analysiscapacity.

The overall implementation by the ICT health administration had been slower than planned. Therewere improvements in several areas like delivery of MCH services, improved staff mix, skilldevelopment of staff through in-service training, implementation of HMIS, and initiation oftreatment of TB cases on DOTS strategy. However, immunization rates of target population werelow especially of women against tetanus and there was lack of utilization of HMIS data for

- 11 -

feedback to the facilities and re-planning.

4.3 Net Present Value/Economic rate of return:

Not applicable.

4.4 Financial rate of return:

Not applicable.

4.5 Institutional development impact:

The project successfully established an extensive network of two PHDCs and 36 DHDCs atprovince, division, and district level which are now the nucleus for work on managementdevelopment, human resource development, health and nutrition education, and monitoringand evaluation. These institutions have begun training large numbers of health staff inmanagement and in technical areas. However, currently the quality of training is onlymarginally satisfactory. The quality of training is highly likely to improve with follow-upthrough SAPP II.

Forty-three health professionals trained abroad in management, planning, human resourcedevelopment, health education, and other topics, have returned and were generally postedagainst relevant positions. These trained professionals have started contributing to improvedplanning and management such as the development of decentralized health systems, theimprovement of gender balance, expansion of TB control using DOTS strategy, anddeveloping a reproductive health pilot project. The research activities carried out under theproject have enhanced operational research capacity in the country through availability ofhuman resources trained in systems research.

The two provincial DoHs in the project benefited from the work of more than 89 short-termconsultants, both expatriate and local consultants. During focus group meetings, almost 75%of these consultants were rated as useful by the implementing agencies. In the last two years ofthe project, a local counterpart was attached to every visiting consultant which helped toenhance knowledge and skills of these individuals. Twenty seven long-term consultants alsoassisted in imparting skills to the DoH staff in areas of planning, management, human resourcedevelopment, and district health systems.

The provincial DoHs undertook extensive groundwork for decentralization of healthmanagement to district level and conducted testing in 10 districts. This initial experiencewould be an asset for the DoHs to carry forward the more ambitious decentralization agendaof the new government. The HMIS for first level care facilities has been firmly established, anddata generated should assist in improved planning.

Staff training and on-the-job experience in procurement procedures and financial management,especially in the later years of the project, have equipped some managers with skills in these areas.This would be beneficial for future procurement and for improving internal financial controls.

- 12 -

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:

Two changes of government in the early stages of the project temporarily slowed down the paceof implementation. Separate legal agreements with each of the three donors, with differenteffectiveness and closing dates, and lack of a framework for joint monitoring created a complexarrangement for implementation. Furthermore, the DFID suspended its grant 14 weeks before itsoriginal closing date of December 31, 2000 due to takeover by the military, and some of theongoing activities were left incomplete.

5.2 Factors generally subject to government control:

The two provincial governnents signed their legal agreement with DFID 14 months aftereffectiveness of the IDA Credit, and with KfW 16 months later. The delayed signing ofagreements substantially affected implementation of planned activities in the first two years ofproject life. The Finance and Planning & Development Departmnents in Punjab province startedreopening the parameters of the approved project and repeatedly raised questions about thevalidity of the approved plans. The project suffered due to frequent transfers of senior managersand project staff, and from political interference, especially in Balochistan. However, sectorpolicies favoured a shift of budgetary resources towards disease prevention and health promotionunder the framework of the Social Action Program.

During the course of implementation, the adequacy and timeliness of provision of counterpartfunds were never a problem in the provinces of Balochistan and Punjab, but adequate funding andreleases of allocated funds remained a major concern for the small federal component managed bythe Ministry of Health.

5.3 Factors generally subject to implementing agency control:

The factors subject to implementing agency control were mainly related to managementeffectiveness.

- Weak financial reporting and monitoring resulted in financial mis-management throughoutproject implementation. Although audit reports were submitted, many of them containedobservations that were required to be resolved by Government. An effective financialmanagement system for the project should have been established.

* The Director General Health (DGH) and his line managers in Punjab did not accept the"ownership" of the project in early periods. This was because a clear line of command forcarrying out project implementation was not established between the Project CoordinationUnit (PCU), the DGH and the Health Secretariat, which created implementation bottlenecksprior to MTR.

* The DoH Punjab did not agree to apply the Bank procedures for procurement of goods fornearly two years, thereby decelerating project implementation, and this impasse was finallyresolved with instructions from the Chief Minister of the province to use the procurement

- 13-

procedures of the respective donors.

* The project staff had to follow four different sets of procedures (World Bank, DFID, KfW,and GoP) to meet the requirements for procurement of goods and services, which resulted inconfusion and obstacles at different levels and even led to re-tendering of procurement ofvehicles in Punjab province.

* Land for construction was not acquired in a timely way; and building designs, consultants'terms of references and bid documents were not finalized in a timely manner.

* Most project staff, in early periods of project life, did not have sufficient experience formanaging a large size project, especially the software components.

5.4 Costs andfinancing:

The total project cost was estimated at US$ 114.0 million equivalent at the time of appraisal andwas to be financed by a GoP contribution of US$3 1.6 million equivalent, an IDA credit ofUS$48.0 million equivalent (SDR 35.0 million), a KfW grant of US$22.2 million equivalent and aDFID grant of US$12.2 million equivalent.

As of the time of closing of the IDA credit (December 31, 1999), total project expenditureamounted to US$58.6 million, or about 51% of the total project cost estimated at appraisal inUS$ terms. The total project expenditure was financed as follows: government contribution,US$8.5 million; IDA credit, US$22.3 million (SDR 16.1 million); DFID grant, US$10.8 million;and KfW grant, US$17.0 million.

Several factors explain the large difference between total project cost estimated at appraisal andtotal project expenditure, including: (a) the restructuring of the project in 1997; (b) slower thananticipated implementation subsequent to restructuring; (c) the DFID grant was not fullydisbursed because of suspension of DFID assistance in October 1999; (d) the KfW grant is stillunder disbursement until December 2000; (e) some of the project inputs were made available fromoutside the project (like contraceptives); and (f) exchange rate-related savings.

6. Sustainability

6.1 Rationale for sustainability rating.

The project's sustainability has been rated "Likely." As compared with the situation before theproject started, there is a stronger government commitment to provision of the types of healthservices which were the focus of the project--maternal and child health including family planningand communicable disease control. The prospects are good that these services will continue toexpand and become more effective. Specific examples that point in this direction include:

* Many more couples are choosing to use contraceptives, as reflected in the rising trend in thecontraceptive prevalence rate in recent years.

* The proportion of pregnant women availing themselves of antenatal care services and the

- 14 -

proportion of deliveries attended by trained personnel are expected to expand becausesubstantial numbers of LHVs have taken up jobs at rural PHC facilities and the capacity forpre-service training of LHVs has been increased by almost 50%.

* The expansion of MCH services including family planning at the community level through theLady Health Workers program is expected to continue until nationwide coverage is obtained.

* The Ministry of Health in collaboration with SAPP II donors and WHO are presentlyundertaking a strategic review of the imrnunization program for taking corrective measures.The Global Alliance for Vaccine Initiative (GAVI) has also shown an interest in supportingthe immunization program in Pakistan for the introduction of new antigens.

* A new culture of continuing education is firmly in place with the establishment of a networkof PHDCs and DHDCs.

* The TB control program is expanding in all provinces using the DOTS strategy.* The Autonomy Bill of November 1998 enacted by the Government of Punjab became the

basis for appraisal of the "Hospital Autonomy Project."* The expanded MCH and maternity services of five well established NGOs will help in

expanding services for low income groups.

6.2 Transition arrangement to regular operations:

The Government has transferred the operational costs of the project to the regular budget fromJuly 1999. This included PAK Rs 123.2 million for continuation of the jobs of paramedics anddoctors recruited under the category of "incremental staff' for delivery of health services andPAK Rs 53.1 million for continuing operations of newly established or expanded institutions likeHDCs, planning cells, and PHNSs. The PHDCs and DHDCs have been placed under theadministrative control of the Director General of Health in each province. These centers havemade twinning arrangements with other institutions for technical support. Staff training hasbecome an ongoing process. The HMIS has become functional for the routine monitoring of PHCservices in first-level care facilities.

Under SAPP II, the Bank will continue to monitor the continuation and expansion of activitiesinitiated under the project for the next three years. These will include measures for increasing thenumber of female staff in health services delivery and management; continuation of in-servicetraining and improvements in quality of training; decentralization of management to district level;and overall progress in further improving quality and access of MCH and preventive services.

7. Bank and Borrower Performance

Bank7.1 Lending:

The project was in line with the Governnent's development strategy and the Bank's CAS.Assistance to the borrower was provided during project preparation through a reasonably goodskill mix of Bank staff and consultants. Risks were appropriately identified and possible remedieswere also suggested. Lessons learnt from previous Bank operations were adequately incorporatedin the project design. However, the scope and targets were overly optimistic. The implementingagencies' financial management system and training of borrower's staff in application of Bank'sguidelines were not addressed as part of project design as a result of which serious instances of

-15-

financial mismanagement arose. In addition, as noted earlier (Section 4.1) project objectives atappraisal were not stated clearly or in terms easy to measure. There was no agreement on anappropriate set of indicators to be used for measuring the achievement of outcome, and there wasa lack of clarity about the links in terms of outcome achievement and measurement between thisproject and SAPP.

7.2 Supervision:

Supervision was done through regular supervision missions, pre-mission monitoring throughconsultants, problem-solving visits by appropriate Bank staff or consultants, and informal regularcontact with the implementing agencies. During May 1997, the project was restructured andscaled down. In association with implementing agencies, the challenges facing implementationwere documented in every mission along with options to resolve them as "agreed critical actions,"and then followed up regularly. A review of "agreed critical actions" since restructuring in May1997 showed that a total of 94 critical actions were identified jointly by the Bank missions and theimplementing agencies. Out of these, 55 (59%) were fully resolved; 24 (26%) were partiallyresolved; 12 (13%) remained unresolved; and three were addressed under SAPP II. In the latterhalf of project life, regular project monitoring was supported by two independent reviews toassess quality of the civil works and of the in-service training program, and a facility survey wasconducted to assess improvements over time. With the introduction of OP/BP 10.02 the Bankbegan to pay more attention to the fiduciary aspects of projects, and the Bank's FinancialManagement Specialists spent additional supervision time with the provinces on improving theirfinancial management systems during supervision. As a result of this increased attention, twoend-use audits and a special review were undertaken by consultants on behalf of the Bank. Thelatter review revealed serious deficiencies in the financial management systems of DoHBalochistan and Punjab.

The project was at risk for about one and a half years prior to MTR. Therefore, staff mix and skillmix of post MTR supervision missions were made more elaborate. This change in supervisionstrategy helped in improving project performance.

Weakness in supervision. A QAG rapid supervision assessment in August 1997 rated the Bank'ssupervision performance as marginally satisfactory. The supervision by the donors lackedcoordination, and KfW, and DFID were supervising their own inputs independently. The project'slegal covenants were regularly tracked.

7.3 Overall Bankperformance:

The overall Bank performance, based on the restructured project, is rated satisfactory.

Borrower7.4 Preparation:

The planning section of the provincial DoHs took a lead role in project preparation, and mosttechnical areas were adequately developed to meet pre-appraisal mission requirements. DraftPC-Is (Planning Commission Formn Number 1, the main project planning document for internal

- 16 -

government approval) were ready at the time of Negotiations. The project coordination units

were made functional before the project became effective.

7.5 Government implementation performance:

The governments' implementation efforts resulted in improved gender balance, risingcontraceptive prevalence, increased enrollment for ante-natal care, successful pilot testing of TB

control and of decentralization of health management to the district level, modest increase in the

proportion of fully immunized children against six vaccine preventable diseases of childhood,

institutionalizing HMIS and creating a new culture of continuing education. Developingpartnership with NGOs was a unique experience in the health sector. These accomplishmentswere achieved despite weaknesses in fiduciary areas, weak coordination and lack of clear roles

within the DoHs especially in Punjab, limited expertise in handling complex projects, andgovernance issues (frequent staff transfers and political interference).

Procurement. In Balochistan no significant problems were encountered. However, it took more

than two years to reach an agreement with the DoH in Punjab to enforce implementation of IDA

procedures for procurement through International Competitive Bidding (ICB).

Financial management and Auditing. The financial management aspects of the project were

not adequately addressed at appraisal which resulted in serious problems in financial managementas the project progressed. Although the DoH Punjab developed a software package and recruited

a qualified firm for managing the project funds, the financial management system remained weakin Balochistan. Submission of Audits was often delayed in meeting IDA audit requirements. The

Auditor General of Pakistan had raised some audit observations on the expenditure incurred under

the federal component during the years 1996/97 and 1997/98. These observations are yet to be

resolved by Government. In 1997, a special review performed by an independent consulting firmfrom Pakistan on behalf of the Bank, noted there were serious financial irregularities including

apparent fraud. Government were requested to take corrective measures and establish an effective

financial management system for the project. Subsequently, the ineligible expenditures were

refunded with some delay. The FY 1998/99 audit reports were due on March 31, 2000 and are stillpending. The Punjab and Federal Special Accounts still have an outstanding balance that needs to

be refunded to the Bank.

Legal covenants. The project was in default for two legal covenants: (i) delegation of powers to

District Health Officers to hire, transfer and promote staff up to Basic Pay Scale 12 in theirdistrict; and (ii) a review of the administrative and staffing implications of the reorganization ofthe communicable disease control programs and implementation of agreed measures. The first hadrepercussions across sectors and hence it was not possible to effect this change for the healthsector alone. In the latter case, there was opposition from paramedic unions.

7.6 Implementing Agency:

The perfornance of the Special Projects Unit of the DoH Punjab and the TB control section ofthe DoH Balochistan, is rated as highly satisfactory for their work in pilot-testing decentralizationof health management to the district level and TB control using DOTS strategy respectively. The

- 17-

work done by the PHDC, Balochistan, in mobilizing parents and school teachers to attract localgirls for pre-service LHV training is another example of highly satisfactory outcome. Theperformance of PHDCs and DHDCs in terns of quality of training is rated marginally satisfactory.The overall performance of the Punjab PCU in procurement of goods and of the Balochistan PCUin dealing with financial management is also rated marginally satisfactory.

7.7 Overall Borrower performance:

The overall borrower performance, based on the restructured project, is rated satisfactory.

8. Lessons Learned

Capacity building* An assessment needs to be made of the capacity of the implementing agencies at the time of

appraisal in financial management and procurement. Capacity building in these areas shouldbecome part of project design. Further, safeguards need to be built-in in the form of end-useaudit reviews.

* The project's performance indicators need to be agreed at appraisal, and costs must be builtinto the project for establishing benchmark values and then tracking these indicators throughindependent surveys.

Technical aspects.i A well-targeted conununications campaign and incentives package proved successful in

securing the enrollment of local young women in the paramedical schools in Balochistan andcan serve as a model for what might be done elsewhere.

3 Capacity building for social mobilization needs to proceed early on in the project for demandcreation and increased utilization.

Operational aspects.i Umbrella projects spanning more than one province need to be replaced with province specific

projects to reduce complexity in terms of geographical spread, the number of implementingagencies, and to embed projects better in the macro framework of provinces.

* The project had a 10 year cycle and more than six years implementation period. It is desirableto keep 3-4 years of implementation period which is more likely to equate with the maximumduration of original champions being in the same job. Further, it is difficult to prescribe aprecise 6-7 year road map and associated project inputs.

Administrative aspects.* For multi-donor funded projects with parallel financing arrangements, a well-defined

coordination mechanism among donors, including joint monitoring, needs to be defined atappraisal stage.

* An umbrella PC- I for the main components of the project should be approved at ExecutiveCommittee of National Economic Council (ECNEC) level. Smaller PC-i s then can bedeveloped for different components/sub-components and approved and subsequently revisedat departmental level.

* Guarantees need to be obtained through the legal documents for continuity of key managerial

- 18 -

and technical staff directly involved in project management.* A clause needs to be added in the Project Agreement that could minimize political interference

in future projects. One option to minimize transfers of essential project staff could be that theborrower/implementing agencies, and the Bank should agree on criteria for selection ofessential project staff or that they should have a minimum time in grade.

9. Partner Comments

(a) Borrower/implementing agency:

The Borrower/implementing agencies evaluation is given in Annex 8.

(b) Cofinanciers:

The project was cofinanced by the DFID and KfW. DFID has carried out its own evaluation atthe end of the project which is generally based on the assessment of decentralization reformintroduced at MTR, and the progress made in the HMIS in the second half of the project. Thereport is given in Annex 9.

(c) Other partners (NGOs/private sector):

Not applicable.

10. Additional Information

- 19-

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:indicator/Matrix Projected in last PSR ActuaULatest Estimate

A. Improvements in the health status of the Increase MCH service use by 20% in A survey of 128 PHC facilities, conducted inpopulation within the provinces of Punjab and Balochistan and by 15% in the Punjab. April 1999, showed improvements betweenBalochistan and Islamabad Capital Territory 1993 and 1998 in several areas: - increase(ICT) with special emphasis on increasing in number of antenatal visits by 52% inthe availability and quality of matemal health Punjab and 120% in Balochistan;services, including family planning. - increase in number of fp clients by 67% in

Punjab and 126% in Balochistan; and- significant increase in supply of drugs andcontraceptives.The HMIS data showed that DoHBalochistan provided antenatal care to 11.6%of rural pregnant women in 1996, whichincreased to 19.8% in 1998. The number ofsupervised deliveries also increased from3.8% to 6.73% in the same period. DoHPunjab provided antenatal care to 11 % ofrural pregnant women in 1996, whichincreased to 23.6% in 1999. The proportionof deliveries supervised by trained birthattendant increased from 5.7% to 11%during this period in Punjab.

Increase immunization coverage for <1 yr. to The proportion of fully immunized children65% in Balochistan and 80% in Punjab by (FIC) in the age group of 12-23 monthsthe end of project. increased from 38.6% in 1990/91 to 51 % in

1996/97 in Punjab, and from 17.8% to 59%in Balochistan during the same period.(PIHS).

Increase Tetanus toxoid (TT) coverage to Coverage in Balochistan increased from 6%50% in Balochistan and 60% in Punjab by in 1990/91 to 27% in 1997/98, and in Punjabthe end of project. from 23% to 66% during the same period.

(source: DoH service statistics)

Reduce IMR by 30% by year five. IMR fell from 105 per thousand live births in1996/97 to 89 in 1998/99 in Punjab, andfrom 108 per thousand live births in 1996/97to 86 in 1998/99 in Balochistan. Thereduction in IMR is in the range of 15-20%(source: PIHS 1998/99).

Reduce female paramedical vacancies by The achievements exceeded the target.20% by year five. Balochistan increased the stock of

paramedics from 243 in 1993 to 641 in 1999,and Punjab from 2730 to 3462 durng thesame period.

Make available full family planning services in 83% of PHC facilities with female health staffall health facilities by year four. in Balochistan, and 82% in Punjab were

providing fp services in 1999.The CPR in rural populabon increased from14% in 1994/95 to 22% in 1996/97 inPunjab, and 1.9% to 4% in Balochistanduring the same period (source: PakContraceptive Prevalence Survey 1994/95and PFFPS 1996/97).

- 20 -

Increase knowledge of contraception by 50% Currently married women who knew at leastby year five. one method of contraceptio increased from

80% in 1994195 to 94% in 1996/97 in Punjaband from 36% to 78% in Balochistan in thesame period (source: PDHS 1990191 andPFFPS 1996/97).

Deliver 75% of the total training program by The achievements exceeded the target.year five. However, improvement in quality of training is

desirable.

B. Improvements on quality of care and Reduce preventable diseases by 20 per Scientific evidence is not available to supportintegration of prmary health care services, 100,000 population especially among this. However, improvements were noted.

particularly for communicable disease control children over five years. * There are major gains towards polioeradication.* Surveillance activities for malaria controlhave markedly improved in last three years.The number of patients screened throughblood test almost doubled, whereas thenumber of confirmed malaria cases declinedby two and a half times in Balochistan and byabout five Umes in Punjab. The proportion offalciparum malaria also came down from37% to 23% in Balochitan and from 48% to18.6% in Punjab.* TB control on DOTS strategy wasintroduced on a pilot basis in Balochistan.The sputum conversion rate was 89%whereas the cure rate was 82%. This led to achange in TB management policy in Pakistanfrom a conventional clinic based approach toDOTS strategy.

C. Institutional capacity building Improve planning management capacity by The target was met except at two distridcts inestablishing Health Development Center at Punjab province. 38 health developmentdifferent levels and evolution of a in-service centers were set-up and fully operational.training system for service provider and However, there was shortage of staff in thesemanager. units.

Expand planning cells in the DoH. The planning cells were equipped and aremanaged by trained professionals.

Expand monitoning/evaluation system with The HMIS reporting rate by the first levelrelated information system support. Care health facilities increased from 11% in

1994 to 95% in Punjab against a target of95%, and 56% in Balochistan against atarget of 90% in 1999. The use of data forplanning purposes is taking root.

Establish a functioning, on-going The training targets for management havemanagement training system. been satisfactorily met. A functioning system

is in place in both Punjab and Balochistanprovince.

Decentralization of staff and financial Major preparations for decentralizing theauthorities. health system to the district level have been

satisfactorily completed. Thedecentralization to district level is highly likelyto continue, as it is a priority for the newgovernment.

Strengthen supervisory capacities. The supervisory capabilities werestrengthened by staff training in managementand supervision and by provision of 552motors cycles and 237 operational vehicles.

- 21 -

OutpUt Indicators:

Indicator/Matrix Projected In last PSR Actual/Latest EstimateNo. of Health Development Centers/training Construct 42 centers/training schools by the Target has been fully met.schools constructed end of project.

No. of Health Development Centers/training Make 42 centers/training schools functional 40 (95%) centers/training schools have beenschools made functional at the end of project. made functional.

In-service training in technical areas Train 3,210 persons in Balochistan; 12,844 Balochistan met 84% of the target ( 2.710persons in Punjab persons trained)

Punjab met 99% of the target (11,249persons were trained).

In-service training in management areas Train 1280 persons in Balochistan; 1568 Balochistan exceeded the target. More thanpersons in Punjab twice the of number persons (2,744) were

trained and re-trained.Punjab exceeded the target by 14% (1,790persons were trained).

% of Basic Health Units (BHUs) and Rural In Balochistan - Increase from 11% to 90% In Balochistan, 56% of PHC facilities were inHealth Centers (RHCs) i.e. Primary Health of PHC facilities in compliance with use of compliance with use of HMIS.Care facilities in compliance for reporting HMIS.through Health Management Information In Punjab - Increase from 11% to 95% of In Punjab, the agreed target was fullySystem (HMIS) PHC facilities in compliance with use of achieved.

HMIS.

% of MCH centers, BHUs and RHCs with Increase from 67% to 90% of MCH centres, In Balochistan, 86% of the centers havefemale health staff BHUs and RHCs with female health staff. female staff.

In Punjab, 92% of the centers have femalestaff.

% of PHC facilities having female staff and Increase the number of female staff and 83% of PHC facilities have female healthproviding family planning services provide FP services in 70% of PHCs in staff and providing fp services in

Balochistan and 90% of PHCs in Punjab. Balochistan, and 82% in Punjab.

No. of health system research studies Conduct 60 health system studies by the end 34 studies were completed and resultsawarded by the MoH of project. disseminated.

No. of NGO projects completed by end of Complete five NGO projects in Balochistan, Four NGO projects in Balochistan and sevenprogram and nine in Punjab by the end of program. in Punjab were either completed or will be

completed by December 2000.

End of project

- 22 -

Annex 2. Project Costs and Financing

Project Cost b Component (in US$ million equivalent)Appraisal Actual/Latest Percentage ofEstimate Estimate Appraisal

Project Cost By Component US$ million US$ million1. Provincial Health Services

Punjab 42.15 19.95 47Balochistan 10.55 9.94 94

2. Staff DevelopmentPunjab 18.00 11.24 62Balochistan 8.50 7.76 91

3. Management and Organizational DevelopmentPunjab 7.75 4.57 59Balochistan 3.45 3.27 95

4. Federal Component 4.40 1.87 543

Total Baseline Cost 94.80 58.60Physical Contingencies 7.30Price Contingencies 11.90

Total Project Costs 114.00 58.60Total Financing Required 114.00 58.60

Project Costs by Procurement Arrangements (Ap raisal Estimate) (US$ million equivalent)

Procurement MethodExpenditure Category ICB NCB Mthod N.B.F. Total CostNCB Othee

1. Works 0.00 13.60 0.00 3.30 16.90(0.00) (12.20) (0.00) (0.00) (12.20)

2. Goods 11.30 1.70 1.40 24.00 38.40(7.40) (1.50) (1.00) (0.00) (9.90)

3. Services 0.00 0.00 8.40 10.60 19.00(0.00) (0.00) (8.20) (0.00) (8.20)

4. Contraceptives and 0.00 0.00 11.50 8.70 20.20Medicines supplies (0.00) (0.00) (11.10) (0.00) (11.10)

5. Salaries and Operating 0.00 0.00 12.30 6,00 18.30Costs (0.00) (0.00) (5.40) (0.00) (5.40)6. Project Coordination 0.00 0.00 1.20 0.00 1.20

(0.00) (0.00) (1.20) (0.00) (1.20)Total 11.30 15.30 34.80 52.60 114.00

(7.40) (13.70) (26.90) (0.00) (48.00)

-23 -

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equival nt)Procurement Method

Expenditure Category ICB NCB :hr N.B.F. Total CostNCB Other'

1. Works 0.00 9.45 0.00 2.22 11.67(0.00) (7.77) (0.00) (0.00) (7.77)

2. Goods 7.16 2.29 1.10 13.78 24.33(4.38) (1.60) (0.74) (0.00) (6.72)

3. Services 0.00 0.00 3.06 9.21 12.27(0.00) (0.00) (2.51) (0.00) (2.51)

4. Contraceptives and 0.42 0.00 0.00 0.68 1.10Medicines supplies (0.25) (0.00) (0.00) (0.00) (0.25)

5. Salaries and Operating 0.00 0.00 5.17 0.94 6.11Costs (0.00) (0.00) (3.19) (0.00) (3.19)6. Project Coordination 0.00 0.00 1.84 1.28 3.12

(0.00) (0.00) (1.84) (0.00) (1.84)Total 7.58 11.74 11.17 28.11 58.60

(4.63) (9.37) (8.28) (0.00) (22.28)

Figures in parenthesis are the amounts to be financed by the IDA Credit. All costs include contingencies.

t Includes civil works and goods to be procured through national shopping, consulting services, services of contractedstaff of the project management office, training, technical assistance services, and incremental operating costs related to(i) managing the project, and (ii) re-lending project funds to local government units.

- 24 -

Project financing by Category of Expenditure (in US$ million equivalent)

Category of a al sallpate E4rst3-i ,f pr.JF xpegditure . A GovtiKIW ,U_ IDA 1 Ei

Civil Works 12.2 2.0 2.2 0.5 7.8 1.7 2.2 0.1 64% 85% 100% 20%Equipment, Furniture,and Vehicles,Contraceptives andSupplies 21.0 16.7 180. 2.9 7.0 4.4 13.2 0.8 33% 26% 73% 28%Training, Educ.Materials,Fellowships 8.2 0.3 1.7 8.8 2.5 0.5 0.6 7.9 30% 167% 35% 90%Salaries, andOperating Costs 5.4 12.6 0.3 - 3.2 1.9 1.0 0.7 59% 15% 333% -

Project Coordination 1.2 - - - 1.8 - - 1.3 150% - - -

Total Financing 48.0 31.6 22.2 12.2 22.3 8.5 17.0 10.8 46% 27% 77% 89%

Note: Actual/Latest Estimates were based on the data from ICS dated 05/07/00.

- 25 -

Annex 3: Economic Costs and Benefits

No economic costs and benefits analysis was carried out at the time of project appraisal or for the ICR.

- 26 -

Annex 4. Bank Inputs

(q) Missions:Stage of Project Cycle No. of Persons and Spccialty Performance Rating

(e.g. 2 Economists, I FMS, etc.) Implementation DevelopmentMonth/Ycar Count Specialty Progrcss Objective

Identification/PreparationPreparation 6 Mission Leader, Health Advisor,05/13/1991 - 06/05/1991 Implementation Specialist,

Health Planner,Management Specialist,Economist

Preparation 7 Mission Leader, Health Advisor,10/12/1991 - 10/31/1991 Implementation Specialist,

Health Planner,Management Specialist,Economist, MCH Specialist

Appraisal/NegotiationPre-appraisal 8 Mission Leader, Health01/29/1992 - 02/24/1992 Advisor, Implementation

Specialist, Health Planner,Management Specialist,Economist, MCH Specialist

Appraisal 12 Mission Leader, Health Advisor,05/21/1992 - 06/10/1992 2 Implementation Specialist,

Health Planner,2 Management Specialist,2 Economist, Public HealthSpecialist, Training Specialist,MCH Specialist

Supervision05/4/1993 - 05/24/1993 6 Mission Leader, Project S S(Was rated in Nos. "1 " Adviser, Health Planner,

Management Specialist,Implementation Specialist,Financial Analyst

01/14/1994 - 02/13/1994 5 2 Public Health Specialist, S SArchitect, Disbursement Officer,Health Planner/Economist

10/08/1994 - 10/30/1994 5 Mission Leader, Public Health S SSpecialist, Architect,Disbursement Officer, HealthPlanner/Economist

08/19/1995 - 09/13/1995 6 Mission Leader, Architect, S SHealth Planner/Economist, 2Public Health Specialist,Disbursement Officer

03/29/1996 - 04/21/1996 3 Mission Leader, Architect, HRD U USpecialist

- 27 -

11/17/1996 - 12/05/96 3 Mission Leader, Health U SPlanner/Economist, PublicHealth Specialist

05/02/1997 - 06/11/1997 3 Health Planner, Mission Leader, S SHealth Specialist

11/17/1997 - 12/17/1997 9 Health Specialist, Population S SSpecialist, Social SectorEconomist, Reproductive Health,Health and Population Advisor,Procurement specialist,Procurement analyst,Disbursement Officer, Architect

04/14/1998 - 05/19/1998 9 Mission Leader, Architect, S SManagement Specialist, HealthPlanner, Human ResourceSpecialist, Population Specialist,Public Health, Financial Auditor,Health Economist

11/23/1998 - 12/17/1998 5 Mission Leader, Public Health S SSpecialist, Human ResourceSpecialist, MIS Specialist,Architect

05/03/1999 - 05/28/1999 5 Mission Leader, Public Health, S SMIS Specialist, Reproductive/Population Specialist,Preventive Program Officer

11/23/1998 - 12/17/1998 9 Mission Leader, Architect, S SManagement Specialist, HealthPlanner, Human ResourceSpecialist, Population Specialist,Public Health, Financial Auditor,Health Economist

05/03/1999 - 05/28/1999 9 Mission Leader, Architect, S SManagement Specialist, HealthPlanner, Human ResourceSpecialist, Population Specialist,Public Health, Financial Auditor,Health Economist

ICR11/18/1999 - 12/13/1999 5 Mission Leader, Public S S

Health, MIS Specialist,Population/ReproductiveSpecialist, PreventiveProgram Officer

- 28 -

(b) Staff

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ (,0()

Identification/Preparation 45.7 160.1

Appraisal/Negotiation 70.3 246.2Supervision 193.8 339.1

ICR 26.4 60.8

Total 336.2 806.2

Source: Project Details at a Glance as of 05/09/00

- 29 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components

(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingOMacro policies O H OSUOM O N * NA?Sector Policies O H *SUOM O N O NA

Z Physical O H *SUOM O N O NAO Financial O H OSUOM O N * NAZ Institutional Development 0 H * SU O M 0 N 0 NAM Environmental O H OSUOM O N * NA

SocialO Poverty Reduction O H OSUOM O N O NAZ Gender O H *SUOM O N O NAO Other (Please specify)

O Private sector development 0 H O SU O M 0 N 0 NAZ Public sector management 0 H 0 SU O M 0 N 0 NAO Other (Please specify)

- 30 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU-Highly Unsatisfactory)

6.1 Bank performance Rating

? Lending OHS OS *U OHUI Supervision OHS OS OU OHU

F Overall OHS OS O U O HU

6.2 Borrowerperformance Rating

? Preparation OHS OS * U O HUZ Government implementation performance O HS O S 0 U 0 HUN Implementation agency performance OHS OS OU O HUOI Overall OHS OS O U O HU

- 31 -

Annex 7. List of Supporting Documents

1. Department of Health, Punjab. Presentation and Brieffor ICR Mission, November 1999.

2. Faisel, Arjumand and Kamal, Imtiaz. Institutional assessment ofprovincial and divisionalhealth development centers, Balochistan. July 1998.

3. Faisel, Arjumand and Rehman Abdur. Assessment of Health Services Development over theproject life in Balochistan. April 1999.

4. Faisel, Arumand; Qadir, Ghulam; Mahmood, Sana; and Ahmad, Saeed. Assessment of HealthServices Development over the project life in Punjab. April 1999.

5. Faisel, Arjumand and Rehman Abdur. Assessment of accomplishments under Second FamilyHealth Project in Balochistan. December 1999.

6. Faisel, Arjumand and Qadir, Ghulam. Assessment of accomplishments under Second FamilyHealth Project in Punjab. December 1999.

7. IDA. Staff Appraisal Report, Second Family Health Project, Pakistan. 1993.

8. IDA. Restructuring/Supervision Mission Aide-Memoire, May 2 - 24, 1997.

9. IDA. ICR Mission Report, November 18 - December 13, 1999.

10. Kamal, Imtiaz and Faisel, Arjumand. Institutional assessment ofprovincial and divisionalhealth development centers, Punjab. July 1998.

1 1. Vendal, Pervaiz. Quality Review of Civil Works, 1998.

12. Anjam Asim Shahid and CO. Final Ex-post Review and End-use Checks Report for FamilyHealth II Project, 1998.

13. Anjam Asim Shahid and CO. Final Ex-post Review and End-use Checks Reportfor FamilyHealth H Project, November 1999.

14. HMIS Reports for 1996/99. Provincial Departments of Health, Punjab and Balochistan.

15. Federal Bureau of Statistics, Government of Pakistan, Islamabad. Pakistan IntegratedHousehold Survey, Round 2: 1996-97.

16. National Institute of Population Studies, Islamabad and Centre for Population Studies,London School of Hygiene & Tropical Medicine, December 1998. Pakistan Fertility and Familyplanning Survey, 1996/97.

- 32 -

17. National Institute of Population Studies, Islamabad, July 1992. Pakistan Demographic andHealth Survey, 1990/91.

18. Population Council, Islamabad, September 1995. Pakistan contraceptive Prevalence Survey1994/95.

19. Population Census Organization, Statistics Division, Government of Pakistan, Islamabad.Population and housing Census, 1998: Census Bulletin 8 & 10.

20. Pakistan Medical Research Council, Islamabad. Utilization of Basic Rural Health Services inPakistan, 1993.

21. IDA. Development Credit Agreement, Credit Number 2464-PAK, April 1993.

22. IDA. Project Agreement, Credit Number 2464-PAK, April 1993.

23. DFID. Final Review of The DFID Funded Components of The Second Family HealthProject, Punjab. Western Asia Department of DFID, November 1999.

24. DFID. Final Review of The DFID Funded Components of The Second Family HealthProject, Balochistan: Western Asia Department of DFID, November 1999.

25. EPOS. Second Family health Project, quarterly Progress Report, January - March 2000.

26. Malmud, Asif. Ex-post Review of Civil Works, Second Family Health Project, August 1997.

27. Ilyas, Muhammad. Ex-post Review of Goods, Second Family Health Project, August 1997.

28. Sidat Hyder Morshad Associates (Pvt. Limited. Family Health Project II, Quetta,Balochistan: Special Review of Accounts (July 1999 to December 1997), May 1997.

29. Sidat Hyder Morshad Associates (Pvt. Limited. Family Health Project II, Lahore, Punjab:Special Review of Accounts (July 1999 to December 1997), May 1997.

30. Federal Bureau of Statistics, Government of Pakistan, Islamabad. Pakistan IntegratedHousehold Survey, : 1998/99.

31. Khan, Amanullah. Study on Persistent Low Coverage of Tetanus Toxoid Among PregnantWomen in Punjab. May 1999.

- 33 -

Annex 8. Borrower/Implementing Agencies' Contribution to the ICR

FAMILY HEALTH II PROJECTEVALUATION REPORT BY DoH BALOCHISTAN

Introduction

The Project was started during the year 1993-94, then fine turned through mid-term review inMay 1997 (with a revised PC- I at a cost of Rs.786.219 million) and closed on June 30, 1999. Theproject components included: Health Service Development; Staff Development; and ManagementDevelopment.

Health Services Development Component

Under this component civil works were completed in 22 Districts comprising construction ofresidences for female paramedics at PHC facilities, and up-gradation of dispensaries for provisionof MCH services.

Special focus was given to MCH and family planning services. The provision of family planningservices were very low at the start of the project in 1994, however, 134,000 clients benefited in1997 and 238,000 in 1999. MCH services were further strengthened through posting of 40 LadyHealth Visitors at PHC facilities and recruitment of 18 Assistant Inspectress Health Services assupervisors. After closing of the project, these posts have been taken on to the regular budget ofthe Department of Health effective July 1, 1999. Village based MCH services were also improvedby providing delivery Kits to village based TBAs and linking the TBAs with village based LadyHealth Workers and PHC facilities.

Referral system for emergency obstetric care (EOC) was strengthened through KfW funding byproviding laboratory and other diagnostic equipment at Divisional, District and Tehsil Hospitaland major RHC's. 22 Ambulances were also provided to the District Hospital and Rural HealthCentres.

Reproductive Health services were started in Quetta District as a Pilot with collaboration ofGynaecology and Obstetrics Department of Sandman Hospital, Quetta. After the closing of theproject, implementation of this pilot is planned for fimding though the development budget of theDepartment of health (DoH).

TB Control through DOTS strategy was also piloted in Mustung District. WHO mission reviewedthe Program in early 1999. The results were promising and WHO recommended Programexpansion. After gaining experience, the TB control though DOTS strategy has now beenextended to 13 districts of the province using regular DoH health budget.

A Public Health Laboratory was constructed at Quetta. It could not be made functional during the

- 34 -

project life because of late completion of the building. The Department of Health plans to equipand furnish it through its own budget to make it fully functional. Some equipment and a vehiclewere provided to Institute of Public Health Quetta which has started postgraduate training inPublic Health.

The KfW grant assisted two NGOs (Planning Association of Pakistan - FPAP, and Lady DufferinHospital) in improving and expanding family planning and matermity services including EOC. Theoutput included: renovation of Lady Dufferin Hospital building, upgradation of Nursing Schoolsof Lady Dufferin Hospital, establishment of a small maternity hospital in a remote district of theprovince, and establishment of a medium size maternity hospital at Quetta which is nearingcompletion.

Staff Development Component

One Provincial Health Development Centre (PIIDC) and a network of six Divisional healthDevelopment Centres DHDCs along with hostels were constructed, equipped and furnished andmade fully functional to work in four areas: management development, staff development, healthand nutrition education, and systems research and monitoring & evaluation. The two schools fortraining of female paramedics were also strengthened by providing residences for the tutors,hostel facilities for trainees, addition of labour suites and additional teaching staff. The staffrecruited during the project life has been transferred to the regular budget of the Department ofHealth effective July 1, 1999. The performance of PHDC and DHDCs was unsatisfactory in latterfunction.

The project during its life conducted 311 in-service training courses for improving technical andmanagement skills of the staff of the Department of Health including skill development in areas ofdistrict health management and health management information system. In total 5,451 healthpersons were trained and re-trained, 95 job descriptions were prepared, and 43 curricula fordifferent types of training were developed and implemented.

The project substantially contributed in increasing enrollment of local female in paramedicstraining schools. The total number of pre-service training schools have increased, increasing thetraining capacity by 100%. By mid 1999, almost 100% of the entrants to the Public HealthSchools were local female in contrast to one local girl in 1993.

Management Development Component

The health management information system (HMIS) was strengthened through staff training,provision of hardware and software and HMIS reporting instruments. The reporting rate fromoutpatient services improved from 45% in 1996 to 63% in 1998 against a target of 90% by theend of the project. However, the system lacks capacity for data analysis and there is little evidencethat data is being used for informed decision making.

The Health system-strengthening component finded by DFID strengthened the institutionalcapacity of health delivery system, in terms of organizational change and operational skill

- 35 -

development for decentralizing health services to the district level. The project worked towardsthis through the following outputs.

1. Strengthened district management capacity based on community participation.2. Effective and integrated training of health human resources.3. Improved district planning and information systems.4. Increased gender sensitivity to imbalances in health needs and staffing.5. Development of counterpart institutions to support in-service training, management

development and health systems research into the needs of the health department.6. Development of an appropriate structure for a decentralized health services.

Constraints/weakness

1. Base line data was not available to compare the impact of inputs.2. Project Implementation Unit Staff was not trained in financial and procurement

procedures of donor agencies as such great delays were observed during the projectlife.

3. Frequent transfers of trained staff of the project had advise effects on projectmanagement and in achieving planned targets.

Lessons Learnt

1. Pl- 1 was not consulted properly by the concerned project staff; therefore some ofpriority areas remained unattended.

2. Community involvement was not given due attention.

Failures

1. Decentralization process could not be implemented for want of approval by theGovenmment, during the project life. As such the District Health Management Teamconcept through implemented but remained un-successful.

2. Research work remained untouched due to lack of coordination between the ProjectCoordination Unit and counter part institutions.

Suggestions.

I. Initiation of a base line survey before launching future projects.2. PC-1 be prepared by those technical persons working in the area where project is to be

implemented.3. Research studies may be the part of the project and PC-1 should have flexibility to

re-fix priorities in the light of studies.4. Training of all Project staff to learn financial management, and procurement and

inventory procedures of the donor agencies to overcome financial irregularities.5. Reimbursement claims procedures may be made simple and short.

- 36 -

FAMILY HEALTH II PROJECTEVALUATION REPORT BY THE PUNJAB HEALTH DEPARTMENT

Summary of achievements

Despite many weaknesses, the DoH would like to rate overall implementation as satisfactory.Main achievements during the project life were as follows:

* After Mid-term Review (MTR) in May 1997, the major factors which contributed toaccelerated implementation were effective delegation of administrative and financial authority tothe project team and capacity building through technical assistance

A new culture of continuing education has been created

* At the beginning of the project, provision of family planning (fp) services was not apriority for the Department of Health (DoH). Now, DoH is the lead agency in providing fpservices

* GoPunjab enacted an Autonomy Bill in November 1998 and made significant progress indeveloping decentralized district health systems in four districts. The preparations are under wayto extend the decentralized systems to another six districts

LDoH tested implementation of TB Control through DOTS Strategy in one tehsil. Thereported cure rate of 59% during 10 months implementation was not very encouraging because ofgovernance issues. However, GoPunjab is committed to pursue DOTS strategy and allocated Rs52.0 million in FY00 outside the project

The project contributed in improving staff mix at PHC and first referral facilities byrecruiting 1,862 female paramedics and general duty doctors (male and female)

* &An independent quality review in 1999 showed increased utilization of PHC services asreflected by substantial increase in the number of clients for antenatal and family planning services,and improved supply of drugs and contraceptives (although still insufficient).

* The efforts in institutionalizing HMIS were satisfactory

Project performance-monitoring indicators. There was reasonable progress. The salientfeatures are: 29 out of 31 Health Development Centers are functional; 98% completion of goodquality civil works; and 92% of PHC facilities, other than dispensaries, have a female health staffagainst target of 90% by the end of the project life. Clients for antenatal care and family planninghave increased substantially over the life of the project. The revised targets for continuingeducation were almost 99% met.

- 37 -

Lessons learnt

The project had important goals with ambitious interventions in several areas, whichrequired substantial time for bringing in necessary changes. Hence impact as per design could notbe fully achieved.

Three donors were funding the project and signed separate legal agreements withGoPunjab and had different implementation arrangements. This led to complex arrangement forproject implementation. Nearly two years were spent in negotiations with other two donors afterinitiating the project with the World Bank, resulting in loss of valuable implementation time.There was a need for one single agreement with all partners.

* Donor cooperation and cohesion was lacking and there was a need to establish a formalcoordinating mechanism.

* The project was more complex than the capacity available in the DoH in 1993.

* In first three years, there was lack of clarity about the powers of the Project Director andwas the main cause of delay in implementing the project. PC1 or some other document shouldhave specified the powers to avoid confusion.

* The DoH could not benefit, as desired, from the short-term TA due to want of an effectivemechanism to review and respond to the recommendations.

The PC 1 had a rigid design and did not permit flexibility. Umbrella PCI identifying maincomponents should have been made for approval by the ECNEC, followed by smaller PC I s forvarious components for approval at the departnental level.

* There was a need to give training in fiduciary areas to all concerned staff, as early aspossible, preferably before the start of the project.

Coordination between various departments of the GoPunjab was not very efficient andwas a another major hurdle in causing delays.

* The project has been instrumental in establishing a team of public health practitioners withexperience in project management, which could manage projects more effectively in future.

The opinion about hiring consultants for deficient areas has started receiving wideracceptance among senior staff of GoPunjab.

The difficulties faced by the project in relation to the procurement activities have led to thestreamlining of government's procedures for all donor-funded projects.

Several weaknesses of the health system were identified over the project life, whichinitiated thinking for new interventions such as decentralization of health services to the district

- 38 -

level.

Detailed assessment

Project costs and expenditure. The project cost, at MTR, was reduced from US$ 82 million toUS$ 53.4 million because of implementation constraints in earlier period of project. The activitieswhich were dropped or added (without changing the project objectives and its components) wereof four types: these were i) activities for which resources outside the project were available (likecontraceptives for USD 5.2 million); ii) activities like training and deployment of village healthworkers as Government had already started the program on a large scale; iii) activities whichcould not be implemented because of slow implementation prior to MTR like incremental staffsalaries as new staff for delivery of health services was not recruited in the first three yearsbecause of recruitment bans; and iv) introduction of new activities like decentralization and TBcontrol on DOTS strategy. In early 1999, US$ 3.0 million of IDA Credit was further canceled,because of exchange rate savings and lack of commnitment by two Municipal Corporations to bearoperational costs for the expand MCH services.

The expenditure by April 30, 2000 was Pak Rupees 1.275 million (IDA Pak Rupees 487.5 million,DFID Pak Rupees 271.4 million, KFW Pak Rupees 270.5 million and GOPunjab Pak Rupees245.5 million).

Implementation status of Health Services Development Component

Inputs for PHC services. The MCH services were improved through supply of equipment like2,300 IUD insertion tables and kits, 2,000 MCH kits, 758 baby resuscitation kits and weighingscales, and in-service training of lady health visitors (LHVs) in areas of MCH, family planning andhealth & nutrition education. Skills of 2,600 practicing TBAs were also enhanced along withsupply of 10,500 disposable delivery kits and 1,000 kits for training of TBAs. DoH made limitedefforts in linking trained TBAs with Lady Health Workers and PHC facilities.

Performance of PHC services. The Bank reviewed randomly selected 100 PHC facilities inMarch 1999. The review showed improvements in several areas: increased utilization of servicesas reflected by substantial increase in the number of clients for antenatal care and family planningservices; and improved supply of drugs and contraceptives. HMIS also supported findings of thesample survey: the number of visits by the clients for family planning services were 207,000 in1996 and 566,000 in 1998 (increase by about 173% in three years); and the number of antenatalclients rose from 319,000 in 1996 to 393,000 in 1998. In spite of all these improvements, there isstill un-utilized capacity at PHC facilities.

Upgrading referral services. KiW made a major contribution to refurbish the district and tehsilhospitals by providing Blood Bank equipment for 60 hospitals, 100 ultrasounds and 100incubators. Some equipment has already been distributed and the remaining is planned to reachend users up to December 2000. Data is not available on the quality of services provided at tehsiland district hospitals. However, DoH has recruited 44 specialists for diseases of women andchildren, and anesthesia at district and tehsil hospitals. Since 1998, DoH has been providing 24

- 39 -

hour emergency services at all hospitals which are free of cost for first 24 hours.

Improvement in staff mix and skill mix. The overall availability of female health staff hasimproved. Since 1994, DoH recruited 1,862 staff comprising LHVs, male medical officers,women medical officers, and specialists for tehsil and district hospitals. However, PHC facilities inhard to reach areas continue to face shortage of female workers (252 posts of LHVs are vacant).

Partnership with NGOs. The funding for NGOs through KfW grant is more than DM 6.0million. The project supported five NGOs for establishing or expanding MCH and maternityservices in cities of Lahore, Gujranwala and Gujar Khan. All planned facilities are expected to befully functional by December 2000.

Support to local government The DoH could not implement IDA financed Urban MCHactivities in Municipal Corporations of Sargodha and Bahawalpur including inputs to improvesanitation because of lack of commitment by these municipalities to provide land for constructionand incremental resources for operation. DFID contributed in expanding MCH services of theLahore metropolitan Corporation by strengthening 30 MCH centres and by establishing a healthpromotion centre along with support for incremental operating costs. The continuation of theseinterventions is unlikely because of lack of commitment by the Corporation to support incrementaloperating costs.

Health and nutrition education. The DoH made limited contributions against the planned inputsbecause of capacity issue. The main inputs were in areas of in-service training of PHC staff, andmass media campaign to support EPI and polio eradication.

TB control on DOTS strategy. The project implemented TB control on pilot basis in one tehsilof Sheikhupura district. The pilot was not well managed because of governance issues. A total of264 cases were detected in 10 months (126 sputum positive) of which 171 were under treatment.The cure rate after eight month treatment was 59%.

Malaria control program. The project provided transport for out-reach workers and supervisorystaff (242 motor cycles and 2,595 bicycles), and limited antimalarial drugs. Improvements havebeen observed in detection and treatment of malaria cases - the conclusion is based on the resultsof an independent review of 100 PHC facilities undertaken in March 1999. The program data alsosupported this inference as surveillance system has considerably improved during 1997-98 with adrop in slide positivity rate from 5.8% in 1994 to 0.84% in 1998. The falciparum ratio has alsodropped by 50% in last five years. However, these averages should be interpreted with cautionbecause as many as 280 sanctioned posts of technical staff are vacant and there is shortage ofappropriate anti-malarial drugs. The resistance of falciparum malaria to chloroquine is alsospreading.

Immunization. The project in 1999/00, using KfW grant, arranged supply of 86 milliondisposable syringes, 10 refrigerated vans, eight cold rooms, 300 ice liner refrigerators, 100 deepfreezers, 702 vaccine storage refrigerators, 80 generators, 800 cold boxes, 2,500 vaccine carriers,9,000 flasks, 2,500 dial/VP thermometers and eight repair kits (some items are in the procurement

- 40 -

pipeline costing approximately DM 12.641 million). The EPI program was also provided 1,258bicycles for outreach work, 252 motor cycles and 106 vehicles for supervision. The cold chainand supervisory vehicles should remain reliable for next ten years.

The coverage of fully immunized children (FIC) did not appreciably increase in the last five years.The coverage of FIC <lyr as per monitoring reports of DoH, was 66%,47%, 76%, 73% and72% during 1994 to 1998 while protection of pregnant women with two or more doses of TTwas 37%, 32%, 54%, 64% and 52% in the corresponding period. The PIHS survey 1996/97 gavea coverage of FIC of <2yrs at 51%, which is much lower than the DoH monitoring system. Lowlevels of coverage by PIHS is more likely due to exclusion of FIC which could not produceimmunization card. The DOH implemented a mopping operation during July - October, 1999 andexpected to have achieved much higher coverage of FIC of 1 year The main reason for lowcoverage in the past was erratic supply of vaccines and syringes.

Staff Development Component

Provincial Health Development Centre (PHDC) and District Health Development Centres.The DoH established one PHDC and 30 DHDCs. These institutions supported DoH in areas ofmanagement development, human resource development, health and nutrition education, andmonitoring and evaluation. The last function, however, was not fully developed. These centersprepared 20 in-service training curricula and met 99% of in-service training target as agreed atMTR. There are 28 functional DHDC housed in their own buildings with an attached hostel. Anumber of posts are vacant in these centers. Two DHDCs are non functional for of lack staff(Rajanpur and Attock). Further, transfer of trained staff from the centers and their replacementwith untrained staff remained a major concern.

An independent review of this component revealed that although project targets were nearly metbut the impact of training in enhancing the skills of the staff was less than desirable. The specificareas of improvement included filling of vacant posts of technical staff and that PHDC wouldcontinue to review the curricula for creating a balance between theory and hands-on training.

Expansion of pre-service training for female paramedics. The physical facilities of alreadyfunctioning two Public Health Nursing Schools (PHNSs) at Lahore and Multan were substantiallyimproved. In 1998, DoH commissioned two new PHNSs in temporary premises. The permanentschool buildings and attached hostels are now complete and occupied. The newly commissionedPHNSs have only 9 tutors against the requirement of 30, based on the standards set by thePakistan Nursing Council . The DoH plans to fully furnish and equip these schools and the 31PHDC/DHDC by December 2000 using KfW grant. The enrollment of LHVs has increased from370 in 1994 to 520 in 1999.

Technical assistance (TA) and fellowship. The project provided TA, both long term and shortterm, to support institutional capacity building. Focused group discussion revealed that long termconsultants were very helpful for institutional strengthening. However, there were mixed feelingson the usefulness of short term TA. The project also supported 15 long term and --- short termfellowships in various disciplines of public health. All fellows returned back and are working in the

- 41 -

DoH.

Management Development Component

Decentralization of health services. The DoH made initial efforts for gradual transfer of powersto the lower formations within the existing system (to meet a legal covenant), but it did not workvery well. GoPunjab, therefore, enacted an Autonomy Bill in November 1998 to decentralizeHealth Systems to the district level and the referral hospitals. Major preparatory work fordecentralizing health systems to the district level in four districts is nearly complete. DoH hassubmitted a proposal for approval of the government, outlining the administrative and financialchanges required at the district level and appointment of chief Executive Officers in ten districts.The project trained 1,790 health staff in supervisory and management techniques (many wereexposed more than once).

Development of Health Management Information System (HMIS). The inputs provided werehardware and software, reporting instruments, and training of 3,028 persons. The reporting rateby the health facilities on HMIS instruments, covering outpatient services only, changed from11% in 1994 to 91% in 1999 against a target of 95%. There is substantial improvement in use ofHMIS data. The issues relate to modification of software to window base.

- 42 -

Annex 9. Cofinanciers' Contributions to the ICR

REVIEW OF THE DFID FUNDED COMPONENTFAMILY HEALTH II PROJECT: NOVEMBER, 1999

BALOCHISTAN PROVINCE

1. The project had been successful in developing the systems needed for operationalisingdecentralized district health system including rational budgeting and planning mechanisms,supervisory system, and in achieving the notification of District Health Management Teams(DHMTs) in six trial districts. But the Purpose of the project will only be partially achieved dueto lack of political will to implement decentralization and the persistent problem of governancemainly political interference which has affected progress towards sustainability of successfulinterventions.

2. Some evidence suggests that the DHMT continue to operate and have introduced the conceptof a team approach to problem solving resulting in addressing some minor problems, neverthelessthe overall quality of services had by and large been unaffected by the project.

3. Despite the establishment of a district planning system, the production of guidelines andmanuals and training, virtually no change in health service delivery can be attributed to this work.This is because throughout the Project life, the health department was unable to set budgets, evenfor the trial districts, which were consistent with the decentralized systems being developed. Thebudgets allocated to the districts remained inequitable and insufficient thus they detracted fromany improvements which may have been made in the areas of systems development andmanagerial capacity building.

4. Externally generated technocratic solutions, especially in the form of the establishment of newinstitutions such as the PHDC/DHDC network, are not appropriate due to a lack of Governmentownership. This problem has been further exacerbated by the persistent transfer of staff in thePHDC/DHDC which has diluted the considerable technical assistance provided to the networkand brings into question the sustainability of the work particularly in the PHDC/DHDC. It wasencouraging that agreement has now been reached on the role of the PHDC with clear leadershipcoming from the Directorate of Health. It is hoped that this will be accompanied by the allocationof a training budget for the PHDC and DHDCs so that they are able to full fill this role andimprove the quality of the training.

5. The project has supported the development and implementation of HMIS in the province. Thereporting rate for outpatient service has improved from 45% in 1996 to 63% in 1998, however,the system continues to lack capacity for data analysis and there is little evidence that data is beingused for decision making.

6. The project shows how committed staff can take forward sensitive and difficult issues, such asgender and participation even in a difficult socio political context. The project has been innovative

- 43 -

in its approach to community participation especially the aspects of health education in schools. Itis hoped that lessons can be learnt from this and other related initiatives that can then be used bythe DoH in other districts. However, in the absence of an enabling institutional environmentcommitted to managerial, financial, and administrative decentralization, it remains difficult tointegrate lessons learnt from successful pilot initiatives into the overall delivery system.

7. The number of local female entrants to paramedic training shows a marked improvement as aresult of the project. The total number of pre-service training schools have increased, increasingthe training capacity by 100%. By mid 1999, almost 100% of the entrants to the Public HealthSchool in Quetta and Turbat were local females.

8. It is acknowledged that the project under the technical leadership of the Nuffield Institute hasperhaps continued with donor led initiatives without giving due consideration to the contextespecially the political environment prevailing in Balochistan.

9. Nevertheless, the project has been fortunate to have had effective and consistent managementwhich is clearly demonstrated by the good working relationships with the DoH and other keystakeholders and had been successful in achieving project outputs.

10. The team came to the conclusion that although the project had done well in achievingits outputs this was due to good local management and a flexible relationship with the DoH butthat in the current environment many aspects were unlikely to be sustainable. Therefore, the teamconcluded that under the current political environment in the Province, it will not be possible torecommend further investment by DFID to build upon the work carried out by the project.

11. The Balochistan Health Systems Strengthening (HSS) component provides an interestingcontrast to the Punjab HSS experience. Whilst the project outputs have generally been deliveredeffectively where possible there has been little impact on the purpose. This again illustrates theimportance of the environment external to the project.

12. The major impediment in project being able to achieve its objective has been persistentpolitical interference over the project life. The political interference affected all aspects of theproject from appointment at PHDC and DHDC's selection of staff for Technical CooperationTraining (TCT) awards and appointment in key management positions at the provincial anddistrict level. The staff transfers meant that project interventions lacked continuity for successfulimplementation of reforms to affect health services delivery in any significant way. The projectand SAP policy dialogues have continually highlighted the issue and its implications for systemsdevelopment in the province, however, with little success. The continued political interferenceresults from a weak political government that is unlikely to take reforms of any significant natureforward.

-44 -

RECOMMENDATIONS

1. The team concluded that under the current political environment in the Province, it will not bepossible to recommend further investment by DFID to build upon the work carried out by theproject.

2. The PHDC and DHDCs are allocated a training budget so that they are able to full fill their roleof providing in-service training and improve the quality of the training.

3. The budgets allocated to the districts remain inequitable, insufficient and so rigid that theyundermined any possible improvements which may have been made in the areas of systemsdevelopment and managerial capacity building. Although it is understood that the decision todecentralize powers was out of the control of the DoH, it is felt that more could be done inproviding rational budgets to the districts. Therefore it is recommended that the DoH builds onthis work and allocates rational budgets to all districts.

4. While subject matter in the school education program should continue to be demand-driven,additional information on contextually-needed health issues should also be included. Effortsshould be made to identify ways of systematizing and continuing this initiative after the projectends, including identification of a social development counterpart in government to take itforward.

5. The outcomes of community participation activities in the pilot districts should be gatheredtogether in a single document to ensure that this knowledge is incorporated into future practice.In particular, the issue of how to take this to scale should be addressed. Along with any othersalient features, the positive response by communities in pilot district to presentations on genderissues in cornmunity participation and mobilization approaches should be analyzed.

6. Before the project ends, a suitably qualified gender trainer with experience in health deliverysystem be identified and hired to provide training for DHMT and to train nominated trainers fromthe five pilot districts on gender and development. The latter training in particular should alsoinclude facilitation skills.

PUNJAB PROVINCE

Health Systems Strengthening

1. The term was pleased to see the conducive atmosphere to change in the Punjab and wasencouraged by the vision and plans for reform of the health system. The project inputs have addedsignificant value to this process, in particular through the inputs in the last year that have adaptedto the changed policy environment with DoH. The detailed work undertaken by the project teamin the four pilot districts for decentralization has had a clear influence on the development of thecurrent policy and the staff training undertaken has provided a good basis for carrying forward thereform. It was felt, therefore, that the Purpose of the Punjab Health System Strengthening (PHSS)component would be largely achieved but only partially as a direct result of the project.

- 45 -

2. The process of policy change towards an appropriate model for decentralizing health services inPunjab provides interesting lessons for the future. The process was clearly iterative and its finalimplementation was driven by a high level bureaucratic and political will rather than throughtechnocratic project solutions. However, the work of the project was important in the process.Particularly important in the latter years was the flexibility and responsiveness of project inputs tothe changing policy environment.

3. The reform process is still in an early stage of implementation and although the institutionalcapacity of DoH to implement organizational change has markedly increased over the project lifecapacity remains weak. However, the senior management team of DoH have a clear vision ofHealth Sector Reform and are moulding the organization to deliver it. As part of the reformprocess, the DoH outlined a new decentralized management vision. It envisaged decentralizationto large provider institutions and also to lower levels within the department at the District level.The first phase of reforms have been initiated in autonomous hospitals and at the district level inthe form of District Health Government (DHG).

4. The District Health Government concept is based on purchaser provider split of functions andresponsibilities with delegated powers for hire and fire and financial autonomy. The DoH hasenacted a law to make provision for delegated authority and some of the Chief Executives havebeen selected. However, steps have been taken towards enhancing capacity of the DoH inimplementing reforms through contracting technical assistance from the private sector and alsofrom the Project to develop management and financial systems. The Health Sector Reform Unit(HSRU) which is co-ordinating the reform process has acquired services of consultants. However,some of the contracted consultants lack appropriate skills to undertake management reforms. Theteam reconmnends that HSRU defines the technical assistance required to reform the healthsystem in order to improve health services and then identified competent consultants to supportthe DoH in designing and managing the reforms.

5. Because of the rapid change in the policy reform process the role envisaged for PHDCs andDHDCs at the start of the project is no longer appropriate. However, the role they are expectedto play in the new health system has not been clarified. As a result the staff in the institutes whohave been trained by the project are not being fully utilized. It is important that the role of theinstitutions is clarified and they are then staffed with appropriate skills and the existing staff areplaced in the institution that is best suited to their skills.

6. Gender issues have begun to be addressed particularly at the service delivery level but willrequire significant, regular and sustained follow-up efforts to be fully mainstreamed into healthdelivery systems at all levels. There has been limited progress in the development of a genderstrategy for the DoH which would seek to encourage an increased representation of women instrategic management positions.

7. Poverty considerations in service delivery are also beginning to be addressed and will requirethe development of contextually-appropriate responses, which in turn will require a sustainedeffort in order to be achieved.

- 46 -

8. The District Co-ordinators, despite not being specialists in the field of social developmentappear to have both internalized the need for holistic social development (gender, poverty andcommunity participation) in health care delivery, and have demonstrated an awareness of itsbenefits through their evident commitment to preparing the ground for operationalization. Thechallenge now is to ensure that this holistic, integrated approach is taken up by other professionalsinvolved in systems reform, as well.

Metropolitan Corporation of Lahore (MCL)

9. The project has been very successful in achieving the agreed outputs nevertheless, it was feltthat there would only be limited achievement of the Purpose due to the lack of commitment bythe Metropolitan Corporation of Lahore (MCL) to reforming the health system. It is disappointingthat the systems planning work carried out by the MCL staff facilitated by the project is notperceived as appropriate and may be wasted. Therefore whilst the capacity of the DoH to delivermore effective services has improved, their capability has not due to the lack of commitment tothe agenda within MCL.

10. One aspect of the project has been the renovation of 30 Reproductive Health (RH) centres,however, these centres are not providing contraceptives. The provision of a wide range ofcontraceptives is a fundamental and essential part of any RH service hence these centres are notfunctioning as envisaged by MCL when it was agreed that DFID would upgrade the facilities.

11. The MCL component provides an interesting contrast to the PHSS experience. Whilst theproject outputs have generally been delivered effectively there has been little impact on thepurpose. This again illustrates the importance of the environment external to the partnerinstitution to a project's output and purpose. This crucial factor is normally placed in theassumptions column of a logical framework enabling project management to leave it out of theirdecision making on implementation. This, as PHSS has shown, seriously reduces the effectivenessof project input.

Institute of Public Health (IPH)

12. An Institutional review of the IPH took place in November 1997 recommending autonomyand reorganization before progress would be made in delivering appropriate public health training.Therefore DFID suspended support until these recommendations were implemented.

13. The IPH was granted autonomy in July 1999 and the Chief Executive Officer (CEO)appointed. However, the current plans presented by the CEO do not seem sufficient to actuallymake any difference. Therefore, at the request of the DoH DFID will consider the provision ofshort term support to the CEO in developing a strategic plan and managing the change process.

14. Although this component has been unsuccessful until now, there is undoubtedly DoHcommitment to radically changing the IPH and making it responsive to the needs of thedepartment. And as we see from our experience in the project this support is the vital ingredient

- 47 -

for success.

RECOMMENDATIONS

1. The role of the institutions such as PHDC and DHDCs is clarified and they are then staffedwith appropriate skills and the existing staff are placed in the institution that is best suited to theirskills.

2. HSRU defines the technical assistance required to reform the health system in order to improvehealth services and then identified competent consultants to support the DoH in designing andmanaging the reforms.

3. HRSU identify a competent, committed task manager to co-ordinate the social developmentand gender components of health services delivery.

4. The DoH should give serious consideration to the development and implementation of a genderstrategy for management and administrative positions in particular. If desired, technical assistancecould be offered during the remaining period of the project to develop such a strategy and identifyways to implement it. It is recommended that any such initiative should take as a starting pointexisting national policy on gender and employment developed by the Ministry for Women'sAffairs.

5. At the district level, gender, poverty and community participation should form part of anoverall, integrated social development approach to health systems delivery, to be implementedthrough partnerships between health care providers and communities.

6. The team expressed support for the new management arrangements in IPH which seek to meetthe overall needs of the health sector particularly IPH's role in the overall framework for healthsector reforn. Therefore, the team will prepare a separate submission to DFID managementwhich will recommend that a consultant is hired to work with IPH in preparing internalmanagement structure and a strategic plan of action.

7. The team will prepare a submission to DFID Management that will recommend that asubstantial new package of support for Health Sector reform is prepared to begin in July 2000.However, to ensure that reform momentum is maintained in the interim period it will alsorecommend that parts of the project components which are supporting Punjab health sectorreform should be extended to June 2000. Taking into account the lessons learnt by GoPunjab andDFID, the team recommends that the extension period is managed by British High CommissionDevelopment Section in Islamabad with a local co-ordinator based in DoH in Lahore.

Conclusions common to both provinces

1. Project management arrangements were unduly complex with involvement from DFID inLondon and BHC Development Section Islamabad, the British Council in Islamabad Quetta andLahore, and Manchester and Nuffield Institute of Health in Leeds. These management

-48 -

arrangements limited the responsiveness of the project to ground realities.

2. The lessons of the project allow a number of conclusions to be drawn. The nature of the policychange process means that it is impossible to prescribe a 5 year road map and associated projectoutputs in support. Policy reform is an iterative process, and one that requires clear politicalsupport if it is to be implemented in a meaningful fashion. Project support need to recognize thisreality and be responsive to the current needs of the partner institution: it cannot work to 3-5 yearplans for specific project inputs, or lever an agenda that is not already present. This requires aflexible approach from donors and a clear link to the Government's own planning andimplementation cycle. Clearly, this flexibility needs to be in the context of a mutually agreedframework of policy and implementation priorities. Externally generated technocratic solutions,especially in the form of the establishment of new institutions, are not appropriate to supporting areform agenda.

- 49 -