Document of The World Bank

Report No: 26049

IMPLEMENTATION COMPLETION REPORT(TF-20052; IDA-26790)

ON A

LOAN/CREDIT/GRANT

IN THE AMOUNT OF US$ 45 MILLION

TO THE

UGANDA

FOR A

DISTRICT HEALTH PROJECT

June 23, 2003

Human Development 1Country Department 4Africa Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective )

Currency Unit = Uganda Shilling USh 1 Ush = US$ 0.106US$ 1 = Ush 940.0

US$1.45924 = SDR 1FISCAL YEARJuly 1 June 30

ABBREVIATIONS AND ACRONYMSAIC Average Incremental CostAIDS Acquired Immune Deficiency SyndromeBOD-CE Burden of Disease-Cost EffectivenessCBOs Community Base OrganizationsCDC Curriculum Development CentreCHAP Community Health and AIDS ProjectCHW Community Health WorkersCP Cooperating PartnersDANIDA Danish International Development AgencyDfiD Department for International Development, United KingdomDHSP District Health Services Pilot and Demonstration ProjectDHT District Health TeamEHP Essential Health PackageEPI Expanded Program for ImmunizationEU European UnionFBOs Faith Based OrganizationsGoU Government of UgandaGPA Global Program on AIDSHMIS Health Management Information SystemHIV Human Immuno-Deficiency VirusHIV/AIDS Human Immunodeficiency Virus/Acquired Immunodeficiency SyndromeHSSP Health Sector Support ProjectIDA International Development AssociationIEC Information Education and CommunicationKABP Knowledge, Attitudes, Beliefs and PracticesKfW Kreditanstadt fur WiederaufbauM & E Monitoring and EvaluationMOH Ministry of HealthNGO Non Government OrganizationNHP National Health PolicyNMS National Medical StoresNRA National Resistance ArmyODA Overseas Development Administration (UK)OPL Operational LevelPAF Project Action FundsPCO Project Coordination OfficePHC Primary Health Care

PHCCG Primary Health Care Conditional GrantsPLWHA People Living With HIV/AIDSPYAR Person Years At RiskQAG Quality Assurance GroupSACII Structure Admustment Credit IISAR Staff Appraisal ReportSTIP Sexually Transmitted Infections ProjectSWAp Sector wide approachUDHS Uganda Demographic and Health SurveyUNFPA United Nations Population FundUNICEF United Nations Children FundUPDF Uganda People Defense ForcesUSAID United States Agency for International DevelopmentVCT Voluntary Counseling and TestingWHO World Health Organization

Vice President: Callisto E. MadavoCountry Manager/Director: Judy O'Connor

Sector Manager/Director: Dzingai Mutumbuka Task Team Leader/Task Manager: Peter Okwero

UGANDADistrict Health Project

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 85. Major Factors Affecting Implementation and Outcome 136. Sustainability 157. Bank and Borrower Performance 168. Lessons Learned 199. Partner Comments 2110. Additional Information 22Annex 1. Key Performance Indicators/Log Frame Matrix 23Annex 2. Project Costs and Financing 32Annex 3. Economic Costs and Benefits 43Annex 4. Bank Inputs 44Annex 5. Ratings for Achievement of Objectives/Outputs of Components 47Annex 6. Ratings of Bank and Borrower Performance 48Annex 7. List of Supporting Documents 49Annex 8. Comments from MoH and Borrower's Implementation Completion Report 50

Project ID: P002971 Project Name: District Health ProjectTeam Leader: Peter Okwero TL Unit: AFTH1ICR Type: Core ICR Report Date: June 23, 2003

1. Project Data

Name: District Health Project L/C/TF Number: TF-20052; IDA-26790Country/Department: UGANDA Region: Africa Regional Office

Sector/subsector: Health (91%); Sub-national government administration (5%); Health insurance (2%); Central government administration (2%)

Theme: Health system performance (P); Decentralization (S); Population and reproductive health (S); Administrative and civil service reform (S)

KEY DATESOriginal Revised/Actual

PCD: 07/31/1993 Effective: 07/17/1995 07/17/1995Appraisal: 05/28/1994 MTR: 10/12/1998 10/12/1998Approval: 02/07/1995 Closing: 12/31/2002 12/31/2002

Borrower/Implementing Agency: GOVERNMENT/MOHOther Partners:

STAFF Current At AppraisalVice President: Callisto E. Madavo Edward V. K. JaycoxCountry Director: Judy M. O'Connor Francis ColacoSector Manager: Dzingai B. Mutumbuka Jacob Van Lutsenburg MassTeam Leader at ICR: Peter Okwero V. JagdishICR Primary Author(s): Ramesh Govindaraj and Peter

Okwero

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: U

Sustainability: L

Institutional Development Impact: H

Bank Performance: U

Borrower Performance: S

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: NoBoth the outcome and borrower performance have been rated in the text as "Partially Satisfactory", but given the choices above for performance ratings, Outcome must be rated as "Unsatisfactory" and Borrower Performance as "Satisfactory".

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:The District Health Services, Pilot and Demonstration Project (DHSP) was conceived at a time when Uganda was just recovering from several years of conflict. The health services were in a state of near-collapse, and were being only marginally maintained through donor-supported programs, charitable/humanitarian organizations and religious missions. The Bank had initially proposed a Community Health and AIDS Project (CHAP) as a follow up to the Bank-supported Uganda First Health Project to address health policy issues and support efforts to contain the growing AIDS epidemic. Given the seriousness of the AIDS crisis in Uganda, however, the original project was split into two projects: the Sexually Transmitted Infections Project (STIP), which was to focus specifically on STD and HIV interventions, and DHSP, which was to support a broad set of health sector reform initiatives, including the rehabilitation of the health infrastructure in Uganda. The ICR Mission, however, could not ascertain the precise reasons why the CHAP was conceived way back in 1989, when the First Health Project was still in the early stages of implementation, and why DHSP took almost 4 years to prepare - although the need to ensure adequate collaboration with the many partners represented in Uganda as well as the complicated design of CHAP may have contributed.

The Government of Uganda’s (GOU) White Paper on Health (1992) and Three Year Plan Frame (1993–1996) laid the foundations for strengthening health services delivery in the country within the broader government reform agenda. However, despite the existence of the White Paper and the Three Year Plan, the Ministry of Health (MoH) did not have a specific health reform policy or strategy at the time DHSP was prepared. The White Paper provided the basis for the new Health Policy that was prepared in the late 1990s, while the Three Year Plan Frame provided an outline for the implementation of major policy milestones - neither, however, was a full-fledged Health Sector Strategy. The main goals of the government policy in the health sector were to: (a) mobilize additional resources to finance the health sector; (b) reallocate resources towards preventive and promotive services; (c) promote public-private collaboration; (d) strengthen planning, management, and coordination of services at various levels; (e) renovate and consolidate existing facilities and services; and (f) promote community participation in the development and management of health services. The design of DHSP also derived its concept of an essential package of health services from the influential 1993 World Development Report: Investing in Health. At the same time as these sectoral policies were being contemplated in Uganda, a number of government-wide reforms were also underway, and the health sector reforms, including those supported by DHSP, had to recognize and work within the confines of these broader reforms. Notable among these reforms were the devolution of management of delivery of services, including health services, to district administrations under the Decentralization Program; the Civil Service Reform program; and the overall economic liberalization – all of which had major implications for DHSP, as discussed below.

The goal of DHSP was to test, on a pilot basis, and demonstrate the feasibility of delivering an essential health services package to district populations within a prudent financial policy framework, and through an integrated program of policy, institutional, and financial improvements, in order to improve efficiency and equity in the provision of health services. The specific objectives were to: (a) Pilot and test new sector policies, which will facilitate the implementation of essential health services; (b) Strengthen planning and management capacity at district levels so that they are prepared to deliver essential health services; and (c) Restructure the Ministry of Health and build its capacity to provide health policy leadership and to support the governments policy of decentralization.

The objectives of the project were clear and consistent with the aspirations of government for the development of the health sector, as outlined in the Government White Paper on Health (1992) and

- 2 -

Three-Year Plan Frame, 1993-1996. The project also took cognizance of the broad public sector reform agenda of government; and focused on the major challenges facing the health sector. In addition, by design, the project was to be implemented in phases, coinciding with the roll-out of the decentralization program, such that only decentralized districts were incorporated into the project.

On the other hand, at the time of project design, Uganda's institutions – including the MoH - were still weak, as the country was just recovering from years of civil strife. The project was widely perceived to have been complex and demanding for the borrower, given the existing implementation capacities in Uganda. In particular, the project seems to have overestimated the capacity of the MoH to coordinate the project, even allowing for the capacity building activities proposed in the project. In the same vein, it significantly underestimated the difficulties involved in implementing such a complex project through the decentralized district administrations, as discussed later in the report.

3.2 Revised Objective:The objectives remained unchanged during the entire life of the project. Given the issues with regard to the original design and objectives, and the rapid evolution over the life of the project in the environment in which the project was implemented, discussed below, the ICR mission views the failure to change the objectives or formally restructure the project, even at the time of the Mid-Term Review, as problematic.

3.3 Original Components:The project had four original components namely:

(a) Pilot Activities(b) Demonstration Activities(c) Capacity Building for District Health Administrations(d) Capacity Building and Restructuring for the Ministry of Health

Pilot Activities (US$ 8.2 million): In the Pilot Phase, identification and piloting of an Essential Health Services Package was to be undertaken in three districts, during the first year. Besides, districts were to be supported in undertaking the district level support services necessary for the successful delivery of the essential health services package. The activities identified in the project’s SAR included: (a) Establishing new funding mechanisms for health services; (b) Contracting out repairs and maintenance services for vehicles, equipment and buildings; (c) Collaborating with private providers (NGOs, CBOs) including contracting specified health services from NGOs and private providers; (d) Identifying options for paying health workers at all levels in terms of their productivity in delivering essential health services; (e) Testing greater autonomy of Government health units through self governing trusts and similar mechanisms; (f) Identifying initiatives to provide essential health package to workers of private firms; and (g) Implementing community based mechanisms for promoting the sale of impregnated bed nets to encourage the use of pit latrines through water and sanitation initiatives.

Demonstration Activities (US$ 19.1 million): Commencement of the demonstration phase was to be contingent on the successful implementation of the pilot phase based on the following criteria: (a) Achievement of at least 60% disbursement of the approved budget; (b) Timely submission of satisfactory quarterly progress and financial reports; and (c) Satisfactory completion of scheduled management training programs. Seven districts were to be added to the project during the Demonstration Phase, where findings from the pilot phase were to be implemented widely and intensively. Implementation of demonstration activities was to be reviewed periodically to inform the pace of expansion and replication of essential health services and the necessary support services in each district.

- 3 -

Capacity Building for District Health Administrations (US$ 36.7 million): Project activities under this component were to be implemented in a phased manner to achieve nationwide coverage, with the objective of supporting institutional development of the District Health Teams to assume their newly acquired responsibilities under the reforms. The project was to provide computers and office equipment for district medical offices, and training to district health teams in a range of areas including planning, management, accounting health information etc. Other proposed project activities included: supporting activities to facilitate NGOs and other private health providers in providing preventive services especially in underserved areas; testing greater autonomy for government health units through Self Governing Trusts; supporting measures to improve the effectiveness of government training institutions; establishing new health financing mechanisms; reviewing and updating the relevant statutes and regulations governing the health sector; and strengthening the Public Health Directorate of the National Resistance Army (NRA).

Restructuring and Capacity Building for the Ministry of Health (US$ 8.0 million): Under this component, the project was to support the MoH at the center in realigning its role to be compatible with the decentralization policy and to strengthen the Ministry in order to provide the necessary leadership/stewardship of the sector in the new policy environment. Besides, the project was to support the MoH in formulating and implementing action plans for health services decentralization, and for restructuring the Ministry.

The components were reasonably related to the project objectives, outlined the full scope of activities to achieve the objectives; and took cognizance of the proposed institutional reforms both at the center and districts – most notably, the Report of the Civil Service Reform Program on Restructuring the MoH and the Action Plan for implementing the Health Policy Reform Program, from which a number of recommendations were adopted by the project.

However, a review of the various initiatives proposed under the four components illustrates the enormity of the task taken on by the project. It would seem unlikely, even under the best of circumstances, that one could implement a pilot within one year (particularly the first year of a project) in a country recovering from civil and political strife, evaluate it, and assimilate its lessons sufficiently to be able to extend it to the demonstration districts in the second year. The pilot involved significant investments in infrastructure and civil works, as well as the purchase of major equipment through international competitive bidding; whose procurement requires a significant lead-time. It is therefore not surprising that the pilot phase and the demonstration phase were ultimately merged without any clear outcomes emerging from the pilot, as the criteria agreed to at the onset of the project for moving from pilot to demonstration were found to be untenable.

Indeed, concerns regarding the capacity of the MoH to undertake a number of major reforms simultaneously; the quality and motivation of the health workers at the district level; and the likely transitional difficulties arising from the decentralization program were voiced by a number of stakeholders, including the GOU, during the design phase. However, it was assumed by the project team that these issues would be addressed through other parallel capacity building initiatives, such as the Structural Adjustment Credit II (SACII), which included a component on the Civil Service Reform Program, and the initiatives of the Decentralization Secretariat under the Ministry of Local Government. As it turned out, the sheer number of reforms proposed in DHSP was to constrain the implementation of DHSP. These issues are discussed in more detail below.

3.4 Revised Components:The project components were not formally revised during the duration of the project. However, as noted, the distinction between pilot and demonstration activities was abandoned during the supervision mission of

- 4 -

November 1996 (see Annex 1 for a listing of the various project activities).

3.5 Quality at Entry:Project concept, objectives and approach: The design of the project reflected and was widely seen to be responsive to the existing situation in Uganda. The country was recovering from civil strife, and the health services were run down and fragmented, relying on service delivery through vertical programs and/or projects supported by various donors. The project’s objectives, as already highlighted in Section 3.2 were consistent with government’s health priorities and reform agenda. The main challenge for the project was to harmonize, rationalize and coordinate health services delivery within a sound and clear policy framework.

In this, the project benefited from a baseline study that was required to provide information on (a) health status and disease burden indicators; (b) expenditure on different health packages; and (c) managerial capacity of districts. A health needs assessment was also carried out in all 13 pilot districts from October 1994 to September 1995. This assessment provided information about health care management capacity. Furthermore, the preparation of the project involved, at all stages, a high level of participation by representatives from government ministries and major donors in Uganda (DFID, USAID, DANIDA, SIDA, KFW), some of whom (namely, SIDA and KFW) made commitments to co-finance the project, while others (DFID and DANIDA supported parallel projects). These commendable efforts to ensure collaboration and ownership are reflected in the design of DHSP, which focused on the need to bring all key partners together to fund a common work program. Indeed, the project’s emphasis on ensuring the involvement of relevant partners might partly explain the long delay in project preparation.

Although the overall project concept was clear, the relevance of the pilot and demonstration components, and their relationship to each other and to the ultimate project goals, was less so. It was also unclear how the results - and the success or failure of the pilot and demonstration phases - would be measured, so that a progression could be made to the next stage. It was also not clear how the pilot and demonstration phases were different from each other. Furthermore, the modalities for implementation were far from explicit. In response to the government’s ambitious decentralization program, the project had a district-focus that entailed a decentralization of fiscal and management responsibilities to the districts. However, this was being undertaken at a time when the appropriate capacity and authority were still being established at district level. The district-focus, while laudable, placed a management and budgetary strain beyond the capacity of the districts to cope.

Unfortunately, the ICR mission was unable to ascertain exactly why DHSP was designed in the manner that it was. That there were relatively few Bank projects with a similar design at the time, from which lessons could be drawn, certainly made the task more challenging (comparable Health Sector Reform Projects - e.g. in Kenya, Mozambique and Mexico - were only developed in the mid-1990s). There are many questions one may ask, such as: (a) Did an expectation that other partners would jointly fund a common program under CHAP encourage the Bank to design a "quasi program/project"?; (b) Was it because of the major macroeconomic reforms (e.g. decentralization, civil service reform etc.) initiated by that government that it was decided to initiate a health reform program?; (c) Was it that DHSP was seen as a vehicle for initiating the health sector reform program or was it that DHSP was to support the MoH in implementing the government-wide reforms?; and (d) Did the pressure on the Bank team to be responsive to the obvious needs of the health sector in Uganda overtake the need for a more careful design of the project? Unfortunately, however, the mission could not get convincing answers to these questions, and, therefore, had to rely on educated guesses. On the available evidence, it would appear DHSP was designed to initiate a health sector reform program and at the same time support the MoH implement other government-wide reforms.

- 5 -

The absence of a health sector strategy until the late 1990s resulted in DHSP becoming the de-facto reform agenda. While the project recognized that reform would entail activities beyond the health sector, no measures were put to coordinate these initiatives. As such, their implementation was significantly constrained. There is also the question of whether it was reasonable to assume that reforms outside the health sector (even if they might further health reforms) could be undertaken successfully within a health project, and whether the MoH had either the mandate or the capacity to coordinate these initiatives. Indeed, the project, by design, allowed virtually any development activity to be supported and gave the districts the flexibility to finance the activities within their plans that other partners did not particularly find acceptable. However, the problem with DHSP was that several of these activities had neither been fully anticipated nor adequately planned for at the design stage. Even the project staff were not always sure of how the project’s components and activities were supposed to come together and how they related to the overall objectives of the project, although DHSP did support technical assistance (which certainly helped) to facilitate this process. In the absence of a fully effective mechanism to coordinate funding, the project made funding available for a diverse range of activities, even if the linkages between the activities were tenuous, and only indirectly linked to the primary project objectives. In essence, the broad reach and scope of the project meant that the project had great difficulty in establishing a clear focus.

There was also no monitoring and evaluation (M&E) framework agreed upon at the design stage of DHSP; nor were the key performance targets identified. The absence of specific indictors to measure the impact of the project components meant that it was virtually impossible to estimate their success and the extent to which they contributed to the project’s objectives, despite the wide range of activities supported. Despite several attempts, there was no consensus around an M&E framework during the entire life of the project – complicating its monitoring, making it difficult to keep it on track, and also making this implementation completion review particularly challenging.

In fact, the evidence suggests that the project actually achieved a great deal; for example, many stakeholders that the mission met stressed the key role played by DHSP in establishing a favorable environment for a health sector SWAp in Uganda. However, these achievements were not necessarily related to the project’s goals, or brought about by design. Critics have even argued that the complexity of the project and the confusion that it created may actually have triggered the move by the government and the partners towards a more structured approach to competing priorities, which culminated in the SWAp. While this might be too harsh a criticism, the fact remains that the laissez faire approach of the project was at least as much of a weakness as a strength.

Project location: In addition to the Ministry of Health, various other entities – such as the Ministry of Local Government, the Ministry of Finance, etc. - were considered as Project Executing Agencies, in view of the decentralized implementation and major government-wide reform elements included in the project. Both the Ministries of Finance and Local Government, however, were dropped, as it was decided that the project was to be focused largely on the health sector and be the basis for health sector reform. Strong leadership and participation from the Ministry of Health was, therefore, considered critical for the success of the project. Besides, the Ministry of Health had prior experience with an IDA project, the First Health Project, and this was considered to provide it a significant advantage for the coordination of DHSP. Furthermore, some of the staff of the First Health Project was to form the nucleus of the Project Coordination Office for the DHSP. While this reasoning on the part of the design team would seem reasonable, the fact that the project included several activities that extended beyond the health sector and/or were reliant on initiatives in other sectors would suggest that perhaps a joint, or at least a more coordinated, implementation by the relevant Ministries might have improved the implementation of the project. Unfortunately, however, the Project Steering Committee made up of relevant permanent secretaries failed

- 6 -

to successfully coordinate implementation across sectors.

Readiness for implementation: Within the context of the virtual breakdown of the health care system in Uganda at the time and the resultant time constraints, the project may have been ready for implementation at the time of entry. For example, by the time of project effectiveness, most key staff had also been put in place. The First Health Project was in the final stages of completion and the preparation of DHSP had also taken a long time. Pragmatism might have dictated that some anticipated inadequacies in the project design be left to be evaluated and addressed during the supervision missions and mid-term review. However, more could have been done at the time of project development to ease implementation and the disbursement of funds. For example, the project’s operational manuals had not been completed by the time of project effectiveness, which significantly slowed the implementation of the project. An insistence on the completion of these operational manuals by the Bank would, in all likelihood, have significantly speeded up project implementation and disbursement. Similarly, adequate assessments of the procurement and financial management systems (in addition to the creation of an M&E system), and the preparation of appropriate documents to support these critical areas might have gone a long way in alleviating the problems in implementation encountered later in the project.

Financial management and institutional capacity analysis: The project, by design, was to be implemented under a decentralized arrangement. Although concerns were repeatedly expressed about district capacity to implement the project, it was assumed these limitations would be addressed by other initiatives supporting the government’s overall decentralization program. It soon became apparent after project effectiveness that capacity, especially at district level, was grossly inadequate in a number of areas – planning, financial management and procurement. Sufficient consideration had not been made to mitigate the effect of secondary project accounts at district level to the project special account. These were to undermine project implementation in the early years. Because of the particular weaknesses around district financial management, the project had to undertake intensive training and support of district level accounting staff, through TA recruited under SIDA. Some of these constraints could have been mitigated if sufficient analysis had been undertaken at project design in the areas of financial management, civil works and procurement, with decentralization in mind.

Risk assessment and sustainability: It should be acknowledged that some of the risks associated with implementing reforms in the health sector were noted at the design stage. Thus, the risk of conservative elements in the Ministry of Health blocking or slowing the reforms was specifically considered. It was also realized that effective project implementation was contingent on successful implementation of the Decentralization Program and Restructuring of the Ministry of Health, as outlined by the Civil Service Reform Program. Certain measures were also initiated to mitigate the identified risks. For example, the Road Map in the Action Plan for Implementing the Health Policy Reform Program included some of the measures necessary, like the formation of a top level Implementation Committee to coordinate the reforms. By project effectiveness, the Ministry of Health had appointed the top-level committee. This, together with the presence of clear reform strategies and action plans under the decentralization and the civil service reform programs were particularly useful at the beginning of the project in coordinating the various reforms being supported.

But overall, the project grossly underestimated the political, institutional, and economic challenges involved in the health sector reform process. The borrower’s suggestion that greater caution be exercised in pushing such major reforms through in the relatively short life-span of a project (evidently, the borrower had envisioned a 24 year horizon for implementing these reforms) were given inadequate consideration. The project also did not have effective safeguards to ensure that the potential risks would not hamper the implementation of the project. While assurances were sought and received from the government to mitigate

- 7 -

possible risks and facilitate implementation, the project had no way of enforcing these initiatives.

Under the First Health Project, support was provided to pre-pilot implementation of key policy measures, including the essential health services package, on the understanding that, during the pilot phase of DHSP, these were to be better elaborated before the final articulation of the policy and reform strategy. However, the preparation of the policy and strategy documents were significantly delayed, and the new Health Policy and the Health Sector Strategy Plan were finalized only in 1999 and 2000, respectively. And, in the interim, there was no clear strategy that could guide the reform process, and this significantly hampered the sectoral reforms. In the absence of a policy, it was not clear to which of the many reforms being pursued by donors within DHSP the government was committed.

Similarly, even at the design stage, serious concerns were expressed about the capacity of the MoH and the district level institutions to: (a) cope with the reforms; (b) implement a broad range of health sector reforms encompassing health financing, liberalization and decentralization; (c) motivate and improve the skills of under-trained and underpaid civil servants; and (d) undertake project activities through local administrations that were unfamiliar with donor procedures. It was assumed, however, that some of the concerns would be addressed through concurrently implemented reform programs. Thus, for example, SACII - which was supported by IDA - included Civil Service Reform, and provided for a compensation reform program, to be followed by salary enhancement. However, the compensation reform has not been completed to date, and this has had a negative impact on reform efforts in the health sector.

Overall, given the many limitations in the design and implementation of the project, the quality at entry must be rated as unsatisfactory.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:The project made significant contributions to Uganda’s health sector (see Annex 2 for details). For example, it introduced the concept of an essential health package, which later formed the basis of the current National Health Policy (NHP) and Health Sector Strategic Plan (HSSP). It also facilitated institutional policy changes in the health sector by supporting decentralization and restructuring of the Ministry of Health, and supporting capacity building in planning and management, including financial management and health management information systems, in the districts. DHSP was instrumental in supporting operational studies and the review of a number of policies and programs including the development of guidelines and strategy documents. It introduced and attempted to institutionalize the concept of quality assurance in health service delivery, including the setting aside of specific funds for this purpose. It introduced gender concerns into the policy debate in the health sector and attempted to mainstream gender in health policy. In addition, the project initiated efforts to ensure sustainable health care financing, and, at least partially, laid the groundwork for the health sector SWAp adopted in 2000. It is important to note, however, that DHSP was intended to test alternative reform approaches as the basis for reform, but, as discussed in this report, the absence of a clear government reform agenda (the While Paper and Three Year Plan recognized the need for reform but did not specify a precise strategy) led to the DHSP becoming the framework for reform in the health sector.

DHSP also supported activities in other sectors, many of which transcended the Ministry of Health's stated mandate. Thus, for example, DHSP provided a platform around which difficult issues beyond the health sector - such as salary arrears, manpower issues, financial management issues outside the health sector, etc. – were taken on. While this may be viewed as a reflection of the project’s flexibility, and its capacity to adjust to the changing policy environment, the main problem was that there was no framework to

- 8 -

systematically tackle these issues. The forays into areas beyond the health sector also raise questions about the appropriateness of using a traditional health sector project to further broad government-wide reform.

Furthermore, the projected improvements in service delivery and health outcomes were largely not realized (see Annex 1 for Project Development Objective Indicators). With the exception of HIV prevalence and contraceptive prevalence rate (CPR) the other major program indicators either stagnated or deteriorated (see Annex 1, which presents a matrix based on the development objectives and indicators used to track the project in the Project Status Reports (PSR), and assesses the project's achievements in detail). There is also no evidence to demonstrate that the Pilot and Demonstration districts had better health gains compared to the other districts. There may be a number of explanations for this. The failure to realize gains in health outcomes may be partly explained by the fact that the project foresaw a major part of its funding being used for direct service delivery, but most of it was instead used for capacity building. Such a switch in project focus is not - in itself - a problem. Indeed, it was perhaps understandable, given the clear failure at project design and appraisal to adequately estimate the capacity constraints, and may even be interpreted as demonstrating flexibility and responsiveness on the part of the project supervision teams. However, the project was neither formally restructured, nor an explicit case made for changing the project's goals and priorities, in a way that might suggest that the project team was responding deliberately to perceived problems with the original design or to evolution in the policy environment. It is also likely that the macroeconomic and sectoral reforms that were underway resulted in a serious dislocation of the health care services, especially at the district level. Furthermore, the project's complexity and broad reform agenda meant that it did not have the focus necessary to realize specific program improvements. The absence of explicitly agreed outputs at the onset and failure to agree on them during project implementation almost certainly contributed to this lack of focus. This was further complicated by the fact that there was no effective internal mechanism to review program plans at the center to ensure prioritization, complementarity, and synergy. As a result of all these factors:

The pace of change in response to the reform initiatives was slow, and central-level programs tended to lretain their verticality, especially if they had separate project funding.Interventions such as nutrition, that were critically needed to improve health outcomes, were largely not ldelivered under DHSP, while civil works more than doubled in size.Factors outside the project’s control - such as the freeze on the hiring of health staff, and nonpayment lof salaries and salary arrears – seriously undermined health worker performance. The implementation of project activities was fragmented and uncoordinated, e.g., facilities were lconstructed, but were not staffed or equipped adequately.While the project was able to facilitate a certain level of collaboration between the government and the lNGO community, it had far less success in engaging the private for-profit sector, including private practitioners, in the health sector reform process, and particularly in delivery of an essential health package. Given the fact that over the project’s life there was an observed increase in the use of private health services across all income levels (see Table 1 in Annex 1), this was a significant failing.

Finally, one of the anticipated gains from DHSP was that the identification and implementation of a national essential health package, and the development of planning guidelines and technical support to districts, would lead a more efficient sectoral planning and resource allocation, a rationalization of health expenditures, and improved health outcomes. Indeed, between 1995 and 1999, district expenditure on EHP in the pilot districts did uniformly increase, on average from 33% to 62% (see Table 4 in Annex 1). The evidence also suggests that the districts were allocating an increasing share of their budgets for health. Although, per capita expenditure on health remained low across all the districts, the failure of the Pilot and Demonstration districts to perform better than other districts in terms of health indicators raises questions about the efficiency of health sector allocations in the districts, even if the effectiveness of essential health

- 9 -

care services package in furthering efficiency is taken as a given.

Overall, given that it was not successful in achieving its major objectives of testing and demonstrating the feasibility of delivering an essential health services package, but did make a contribution to policy-making and planning, as well as capacity building at central and district level, and, arguably, paved the way for the coming together of partners in a SWAp, the project’s achievement is rated as partially satisfactory.

4.2 Outputs by components:Components 1 and 2. Pilot and Demonstration Activities:

The definition and implementation of an essential health services package was seen to be central for Uganda to improve the effectiveness of the health services, and the concept was to be the guiding principle behind all the major reforms in the sector. This was to have been done during the pilot phase, but as earlier explained pilot and demonstration activities were merged and implemented without distinction. We, therefore, discuss the two components together. As part of the two components, the project supported two main activities: (a) Identification and piloting of Essential Health Services Package, and (b) District Level Supporting Services (see Annexes 1 and 2 for details).

Identification and piloting of Essential Health Services Package: Notwithstanding the technical weaknesses in the Burden of Disease-Cost Effectiveness (BOD-CE) approach, which have been discussed in several articles in academic journals, the pilot districts were able to identify the major causes of the BOD through the project. In addition, progress was made towards identifying cost-effective interventions and - much later - introducing a rational approach to planning and budgeting linked to the BOD within the pilot districts. The experience gained was incorporated in the planning guidelines, introduced to other districts, and underpins the current NHP and HSSP. The project provided the financial resources for delivering the EHP to the population in three, and later 16, pilot districts. It was out of this initiative that programs that originally had little appeal or support, like malaria and nutrition, started featuring in the district plans. However, many of these initiatives could only be implemented several years later than anticipated by DHSP.

District Level Supporting Services for the delivery of the essential health package:

Establishment of new health financing mechanisms:

DHSP was instrumental in assessing and promoting establishment of new health financing schemes. A majority of these schemes were not new, as they were already being implemented outside the project. Using the Local Government Act, the project supported the districts to promote user fees in publicly provided health facilities, develop guidelines for collection and management of user fees, and conduct a series of training initiatives. Financing options other than user fees were found to be either not feasible within the project life, or required major technical support, financing, and new legislation. Drug revolving funds were abandoned because of fragmented and uncoordinated procurement and logistics management. The Social Health Insurance report is still on the drawing table with limited options for implementation. Although a few community prepayment schemes still operate, they too are likely to become nonviable, especially with abolition of user fees. Finally, the establishment of health facility trusts is still awaiting enactment of relevant legislation.

Even though the actual collections were small – there was less than 5% cost recovery - the decision to concentrate on user fess was probably appropriate, as it provided a viable financing alternative for Uganda. Secondly, the schemes provided resources that could be channeled to areas where government funding

- 10 -

could not be applied. Due to political and equity concerns, however, user fees were abolished in March 2001. The initiatives to provide an essential health package to workers of medium and large-scale private firms was also abandoned because of the lack of a social health insurance program, despite the fact that a majority of private firms were providing health services, either directly in the workplace, or through arrangements with private facilities.

Other supporting services for the delivery of the essential health package:

Implementation of support services for the delivery of the essential health package were implemented only on a limited scale. A significant and commendable exception was the rehabilitation and equipping of health facilities and public hospitals, so as to increase their capacity to deliver the facility-based aspects of the EHP. The health facilities were also encouraged to contract out the repair and maintenance of building, equipment and vehicles, which became common practice in public hospitals. The limited support for certain support services was, to some extent, understandable, as the initiatives were complex, went beyond the mandate of the Ministry of Health, and/or required an existing legal and regulatory framework, such as assessing options for paying workers, improving effectiveness of government health training schools, etc. However, much more could have been done to, for example, promote impregnated bed nets (in addition to its important role in negotiating a tax waiver on netting materials) and increase the public sector collaboration with NGOs, CBOs, and the for-profit sector.

In sum, it is worth noting that DHSP was least successful in testing the policy initiatives that required the involvement of other sectors, such as (a) testing greater autonomy of government units; (b) identifying initiatives to provide EHP in private firms; (c) implementing community based mechanisms to promote the sale of impregnated bednets and encourage the use of pit latrines through water and sanitation initiatives; (d) initiating measures to improve training institutions; (e) reviewing and updating relevant statutes and regulations governing the health sector; and (f) identifying options for health worker remuneration based on productivity. On the contrary, the areas where DHSP was successful were the ones that were clearly outlined in the White Paper and Three Plan Frame - again emphasizing the critical need for government ownership and commitment of a sectoral reform strategy in order to ensure the success of the reform.

Overall, the implementation of these two components is rated as unsatisfactory.

Capacity Building for District Health Administrations

The project played a pivotal role in building district capacity and supporting the decentralization process in the health sector. The support virtually covered the entire range of possible activities from district level health planning and budgeting to formal (including in-service) training, rehabilitation and equipping of health facilities, and the strengthening of support services, such as financial management and procurement. Besides, newly created districts benefited from start-up packages (there were 39 districts at project design and 56 at completion). Thus, with project support, health departments benefited from new vehicles and office equipment, improved facilities following repair and renovation, direct funding for day-to-day activities, and trained health workers – including, notably, accounts officers - a majority of whom still remain in the districts or in the country. The support strengthened the capacity of the health departments in the districts to better plan and manage the resources within the health sector, and implement the programs under a decentralized arrangement. It was also noted the integration of project budget within the overall district health budget agreed during a district conference ensured that resources were better aligned and prioritized unlike the center where such a forum was nonfunctional.

However, the level of over-expenditure on capacity building activities, noted earlier, was unwarranted and

- 11 -

almost certainly undermined the delivery of services. Support to NGOs was also grossly reduced. The expenditures towards capacity building should have been better managed and regulated in order to ensure balanced expenditures at district level, value for money, and, ultimately, improvements in health outcomes.

Overall, the rating for this component is rated as only partially satisfactory.

Restructuring and Capacity Building for the MoH

Besides supporting the restructuring and reorganization of the Ministry of Health around its constitutional mandate of policy formulation, national planning, development of standards and guidelines, and providing technical support, the project was instrumental in supporting the Ministry of Health execute its new mandate. Thus, the project supported (a) the formulation and development of the new Health Policy, the Health Sector Strategic Plan, and the Nutrition and Food policy; (b) the revision of laws and regulations governing the health sector; (c) the development of various technical, policy, and strategy documents, including service standards and guidelines; and (d) the provision of technical support. Other major contributions of the project included: support to the SWAp process to ensure better coordination of investments in the health sector; reconstruction and relocation of the Ministry of Health headquarters from Entebbe to a more central location in Kampala, in order to strengthen its internal coordination; and the provision of procurement and logistic support. Virtually all the major departments in the Ministry received, at one time or the other, funding from DHSP to improve their operations, especially during support supervision. As noted, Quality Assurance was initiated with project support, and has since been adopted as a Department within the Ministry.

Through the project, extensive support – admittedly with limited success - was also given to the Health Planning and Quality Assurance Departments to improve intra-sectoral collaboration in planning, budgeting, use of resources, and support supervision. The lack of clear priorities, and the absence of appropriate fora to bring together the development partners and programs, were primarily responsible for limiting the effectiveness of the project. However, the adoption of SWAp and the increasing resources in the government budget have since brought about a major change in the intra-sectoral collaboration, with the Health Planning Department taking on a more central role in coordination, and the major technical program units becoming more amenable to collaboration. The project also prepared the leadership in the Ministry of Health to look beyond the projects and programs, think more holistically about the sector, and provide overall leadership. As a result, the Ministry is now in better position to lead the dialogue with development partners, to negotiate with other arms of government, and to support districts and other implementing entities.

Overall, the rating for this component is rated as mostly satisfactory.

4.3 Net Present Value/Economic rate of return:A NPV/ERR was not calculated for the project.

4.4 Financial rate of return:A FRR was not calculated for the project.

4.5 Institutional development impact:Despite its many limitations, DHSP has enhanced the capacity of Uganda to use available human and material resources for improved health care delivery. As a result of DHSP support, the health sector now has a better capacity to absorb funds (see Annex 1). There has also been an improvement in the absorptive capacity in the districts supported by DHSP. Thus, for example, the process indicator now in use in the

- 12 -

SWAp “Percentage of disbursed PHC conditional grants that are expended” was 50% in 1998, 55% in 2000/01, and 99% in 2001/02; the process indicator “Proportion of districts submitting completed HMIS monthly returns on time” increased from 15.6% in 1998, to 52% in 2000/01. This is as a result of the emphasis placed in DHSP on capacity building activities, such as the training of MoH and district staff and private health providers through short- and long-term training. However, this training was costly (for example, foreign training consumed nearly $1.7 million USD). Several district level staff was also lost to attrition (e.g., to NGOs or the private sector), or moved on to other roles in the district, which undermined the institutional development impact.

Overall, to a significant extent, the systems developed with project support provide a basis for building a strong health sector, even if the development objectives of DHSP itself were not always realized. The Project’s institutional development impact is, therefore, rated as high.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:Impact of HIV on Health Outcomes: HIV/AIDS had, and continues to have, a significant negative limpact on mortality outcomes. Thus, life expectancy at birth was 54 in the late 1970s and 43 in 1995 (UNAIDS 1998). Even the current malnutrition in Uganda has been attributed to HIV infections in women, who represent 70% of the agricultural workforce in Uganda, and also play a critical role in caring for PLWHA.

Shortfalls in financing and politicized priority setting: Despite reservations expressed by the lgovernment and by the MoH, who envisioned the health sector reform process over a much longer time horizon, DHSP undertook an ambitious reform program in the limited 5-6 year time frame of the project. Furthermore, the design of the project was based on an expected financing of US$ 75.1 million. Only US$ 66 million, however, was ultimately realized, because DANIDA and DFID decided to channel funds to the sector through parallel projects (whose objectives did not always mirror those of DHSP), rather than co-finance DHSP. As a result, some of the projected DHSP activities had to be scaled down. For example, a significant activity that was scaled down due to the shortfall in financing was the support that the project was to provide to NGOs. Another effect of the shortfall was that, during the last two years of the project, very little funding was made available to the districts as well as the center for the implementation of their work-plans.

5.2 Factors generally subject to government control:Closure of Bank: In 1999, the International Credit Bank was closed down by the Central Bank of lUganda. All the SIDA Grant central project accounts and the accounts of some districts were held in the International Credit Bank. The government took a long time to repay the funds held by the bank and this affected the flow of funds from the SIDA Grant. Thus, affected districts could not access DHSP funds until the government had repaid the locked-up funds. Some procurement committed under SIDA Grant also experienced considerable delays.

Poor Staffing: Districts, especially the new and remote ones, had inadequate staffing, and hence a low lcapacity to plan, implement, and account for funds. As noted, at project preparation, there were 39 districts, while by project closure there were 56 districts; thus, almost 50% as many new districts were created during the life span of DHSP.

Civil Service Reform: This proved to be a challenge, particularly at the district level. As a result of lthe reform spearheaded by Ministry of Public Service (MOPS), there was a ban on recruitment of staff.

- 13 -

This left many health facilities in the hands of untrained workers who were unequal to the task of delivering the EHP.

Decentralization: As a result of decentralization, health workers in districts were to be recruited and lpaid by the local governments. Some districts were unable to pay salaries for their health workers for as long as three years because of financial constraints. This affected project implementation. By 1998, many districts also showed a reluctance to allocate funds to PHC (Hutchinson 1998), regardless of revenue source. For example, between 1995 and 1996, Mukono District decreased the percentage of funds from user fees dedicated to improved services from 21% to 10% (although the presence of other donors meant that other funds were usually available). Kabale District, on the other hand, used 42% of user fee income for services in 1996 (HMIS, Mukono and Kabale).

Counterpart Funding and Tax Issues: As a result of the government’s financial constraints, its lcounterpart contribution was reduced from 10% to 5%. This ultimately affected the scope of the project and its financing. Even with a reduced requirement for counterpart funds, the government is far behind in the release of these funds. Many activities have, as a result, not been implemented fully. It should be noted that the MoH was, to some extent, constrained by having to pay taxes on imported goods based on the Ministry of finance (MoF) requirements. Due to delays in paying these taxes, the MoH also had to pay substantial interests and demurrage charges. Unpaid IDA funds borrowed to meet the counterpart obligations are a major issue at project closure.

Failure to Retain Trained Staff: The PCO estimates the vast majority of district staff trained l(especially Accounting Officers) left the health sector or the district, undermining continuity and representing a systemic loss. While many were redeployed in general district functions (e.g., as accountants), it still constrained the ability of the health sector to reach the targets set by DHSP.

5.3 Factors generally subject to implementing agency control:Initial size of the special account: The initial size of the special account at US$ 750,000 was linadequate for project operations, considering the decentralized implementation of the project. The long accountability cycle associated with a decentralized project was not anticipated at the project design stage. However, with assistance from the Bank, which mounted a mission to specifically address this issue, this issue was satisfactorily resolved.

Procurement delays: Delays were occasioned at various stages and by various agencies namely: by lCTB that took very long to award tenders, by Treasury that delayed in paying taxes for goods after arrival in the country, by suppliers who in many cases never met their contractual obligations regarding delivery schedules. Other delays were due to slow identification of needs and their specifications, unfamiliar IDA procedures at the beginning of the project and lack of a comprehensive procurement plan. As a result of these delays, some items critical in the delivery of the EHP arrived in the country towards the end of the project.

Cash flow Problems: After delays in executing some procurements and civil works, most major lcontracts matured after mid-term. This put pressure on the special account whose size had not taken into account this crowding of activities. As a result, work plans (central and districts) could not be promptly funded because all available funds were committed to contractual obligations for civil works and other procurements. Associated with this problem was the high threshold set for special commitments, which led to the payment of too many contracts from the special account, as they did not qualify for special commitment. Poor management of expenditure/disbursement records further exacerbated this and such the Special Account could never be replenished in time.

- 14 -

Unplanned expenditures: A number of unplanned expenditures seriously undermined project limplementation. The decision made during the Supervision Mission of 1998 to support all districts to deliver the EHP could not be implemented fully as funds had run out. Civil works almost doubled in size and technical assistance exceeded the budget by USD 2.5 million.

5.4 Costs and financing:The total project costs, as estimated and planned at project inception, amounted to US$ 75.1 million. Most of the project funds, however, had been disbursed prior to closing (see Annex for details). The planned financing was as follows:

Source(US $)

Original Cost (SAR)

Revised Cost (1999)

IDA 45.00 45.00SIDA 7.00 8.50KfW 9.00 9.00ODA 2.20 0.00DANIDA 5.00 0.00GOU 6.90 3.50Total 75.10 66.00

Although DANIDA (Denmark) and ODA (United Kingdom) initially indicated a willingness to co-fund the project, no firm assurances were received from them in this regard. In the end, both donors decided to channel their funding through separate parallel projects. Government counterpart funding was revised to 5% due to cash flow problems. Failure to secure all the funds anticipated at project design stage seriously constrained project implementation in the last two years. These, however, could have been better managed, since both DANIDA and ODA funding ended up financing the health sector program. In addition, there was a serious shortfall in counterpart funding, which was further compounded by delayed releases.

6. Sustainability

6.1 Rationale for sustainability rating:Despite the problems with DHSP, and the resultant mixed outcomes, the sustainability of the project activities – from the programmatic as well as the financial perspective - is rated as likely. This assessment is based on the fact that the institutions in Uganda, helped by the DHSP, are now stronger and the donors are working collaboratively with the GoU and each other, within an overall development framework.

Specifically, the factors favoring project sustainability are:

Some of the key reforms and initiatives supported by the project have government ownership and have lalready been integrated in operations of the MoH (they are part of the HSSP). They include Quality Assurance, HMIS, infrastructure expansion, and private-public collaboration.The close collaboration between MoH and the development partners through the SWAp process around lthe new National Health Policy and Health Sector Strategic Plan provides a solid basis for programmatic and financial continuity. Further, the creation of the Poverty Action Fund, through which the PHC Conditional Grant operates, las well as the movement towards budget support by several development partners have provided sustained avenues for funding to the health sector.Considerable capacity building in terms of human resources development, systems development and l

- 15 -

infrastructure has been undertaken, both at the central and district levels.

6.2 Transition arrangement to regular operations:There are no major transitional arrangements to regular operations required because most project activities have already been integrated into the regular operations of MoH - largely through the SWAp and the budget support arrangements.

7. Bank and Borrower Performance

Bank7.1 Lending:Overall, the Bank’s performance was unsatisfactory at the identification and preparation phase.

On the positive side, the Bank built linkages with several development partners in supporting heath sector development; some of whom funded the pre-investment studies, and later agreed to co-finance the project. However, at the preparation phase, although the multidisciplinary preparation team identified the critical heath sector issues, which were consistent with Government of Uganda's plans and priorities, the capacity for bringing about change in the sector was grossly underestimated. Although the project provided a detailed review and analyses of health and other related issues in the country, and drew on the experience of the First Health project (that, incidentally, had specifically signaled the problems of embarking upon an overly complex agenda), DHSP was an extremely complex project being implemented in a complex political and socioeconomic environment.

While district implementation was phased, as discussed above, the main criterion of assessing the success of the pilot - that of achieving at least a 60 % disbursement rate - was unrealistic and unlikely to be achievable. No lead-time had been allowed for civil works to rehabilitate health facilities, which required an assessment of the quantity and type of work to be undertaken. This assessment, which was itself delayed, was only partly completed one-and-a half years into project time. In addition, it would not have been possible to procure equipment and medical furniture from abroad in one year. There was also a delay in getting the project started. A World Bank procurement specialist estimated that it would take between 23 and 29 months (2-3 years) to get equipment from a supplier abroad to destinations in Uganda. Although health planning, progress reports, and financial reports were satisfactory, the key pilot activities for testing different ways of delivering the essential health package had not begun. Furthermore, it had been assumed in the project design that pilot districts had adequate capacity to carry out pilot activities. It, however, soon became obvious that these districts, like the rest, lacked capacity, and required to be trained and prepared for the pilot activities.

Orientation in procurement - particularly with regard to IDA regulations - is essential for project implementation success. Procurement capacity assessment is essential and was not undertaken at the time of project design. Similarly, in order to cut down the time lag between project approval and implementation, the procurement plans and project implementation manuals should have been finalized by the time of project approval; this did not occur with DHSP.

At appraisal, the Bank's performance was again unsatisfactory.

The Bank's appraisal mission included a team with a comprehensive skill mix. At appraisal, the institutional arrangements at the center and in the districts, the phasing of project implementation, the role of the NGOs, and the linkages with existing heath sector programs supported by other development partners were further reviewed and agreed to with the government. These, particularly NGO participation, were to prove useful later during project implementation. The relevant safeguard policies were discussed

- 16 -

and agreed upon with the government, including the actions the government needed to take to ensure successful project implementation. However, the institutional capacity assessment - including assessment of the procurement capacity - was never appropriately assessed at appraisal, and the complex design, which lacked focus, was not fully appreciated as problematic.

There was also an understanding that as “lender of last resort,” IDA disbursement categories would be adjusted cover gaps in grant support. However, the project documents and agreements did not adequately spell out the “rules of the game” for this process -- including a policy framework and agreement about priority setting so that the funding would be used for high rather than low priority needs not funded by other CPs. The concept of the Bank being the “lender of last resort” can only be sensibly interpreted and applied when there is a prioritized and joint expenditure program, based on jointly agreed upon strategies and action plans. The use of this concept in other contexts can lead to confusion and misunderstanding, and can potentially severely constrain the effective implementation of Bank-supported projects.

7.2 Supervision:While the Bank met its formal supervision requirement of twice a year during project implementation, and the IDA Team was able to develop good rapport and working relations with the government team, there were persistent implementation constraints in the areas of procurement, logistics, disbursement, district capacity, human resources, and monitoring and evaluation, especially in the early stages of implementation. A viable and effective monitoring and evaluation framework that is mutually agreed upon at the time of project preparation is essential for the effective supervision, implementation and evaluation of a project. This was perhaps the most overlooked area at both project design and was not effectively addressed during supervision. The monitoring of the project was negatively affected by the failure, due partly to the lack of consensus at a conceptual level, in resolving this vital issue, despite several attempts. This was raised as a concern by the Acting Country Director early in 2002 who noted that “data on access to services is at present poor, and what data exists shows that increased utilization of services has been mixed. A hard-nosed assessment is needed on whether the data supports the conclusion that the objectives have indeed been achieved.” This was reiterated in the next PSR by both the sector manager and country program manager who noted that “the issues with regard to M+E and sustainability do not appear to have been fully resolved…[and] a high quality evaluation [will be needed to] support the team’s assessment of “Satisfactory” for project outcome. Greater proactivity and more persistent follow up on these issues on the part of the Bank might have significantly mitigated these constraints. The evidence also suggests that the persistence of constraints to project implementation, even when they were appropriately identified and reported on (procurement, disbursement etc.) didn’t trigger support or action by senior management, be it in the Bank or within government.

In view of major flaws in project design that were increasingly apparent (and were noted several times by supervision missions), one would have expected the Bank and the government to seriously consider restructuring the project in mid-stream, especially since two project components were merged, and the project in its latter half, was increasingly focused on the implementation of the Health Sub-Districts concept and on supporting the Ministry of Health in its move towards a SWAp approach in the sector. The ICR team could not find a persuasive answer as why such a formal restructuring was not undertaken. While, evidently, a review team had decided - based on the evidence available at the time - that a restructuring was not warranted, the ICR did not find any documentation either documenting or justifying this decision. In the absence of evidence that there was a compelling case for not restructuring despite the identified design flaws and changes in the project environment, the ICR team is forced to conclude that the failure to restructure was a miscalculation on the part of the supervision teams.

Over time, however, the supervision improved considerably. The supervision missions involved

- 17 -

comprehensive and competent teams, and the participation of several partners, thereby ensuring adequate support to the PCO and other implementers. Indeed, the supervision teams need to be commended for their hard work and the support that they provided to the GoU in the latter half of the project in difficult circumstances, which were not helped by the complicated and overly ambitious project design. In the last two years of the project, the recruitment of a health specialist in the country office also ensured continuous support to the PCO and improved the quality of supervision. Furthermore, several of the programs supported by the project also became part of the joint GoU-Donor Joint reviews of the health sector towards the end of the project, which helped in their supervision. Accordingly, in 1998, project management was shifted from an unsatisfactory to a satisfactory rating (The Country Portfolio Performance Review rated this project unsatisfactory until 1998, based on persistent problems with disbursements and counterpart funds, and the delays due to the non-streamlined World Bank procedures and the size of the special accounts). It may well be that, had the project been restructured in an appropriate and timely manner, that the intense supervision might have yielded greater dividends in terms of outcomes.

Overall, the Bank’s supervision was only unsatisfactory initially and satisfactory in the later stages.

7.3 Overall Bank performance:Given the unsatisfactory performance of the Bank at project identification, preparation, appraisal and for part of the supervision, and despite the observed improvement in project supervision in the latter half of the project, the Bank's overall performance must be rated as unsatisfactory.

Borrower7.4 Preparation:The Borrower's performance in preparation is rated as satisfactory. During preparation, the Borrower provided the necessary leadership and support to the Bank that facilitated project development. The availability of the health policy and plan, 1993-1996 and the White Paper on Health, developed with active participation of stakeholders, were especially beneficial. The Borrower facilitated wide participation of stakeholders in preparation including the development partners. Because of this, the government was able to raise US$ 17.4 million from other donors to co-finance the project.

7.5 Government implementation performance:The government's overall performance is rated as marginally satisfactory. Over the life of the project, the government maintained its leadership and commitment to heath care delivery and overall heath sector reform. As mentioned above, this has not yet been translated into improved health outcomes, but a base has certainly been built upon which further consolidation can occur. The issue of counterpart funding not being provided, as agreed, has become a systemic feature of projects in Uganda, and needs to be addressed in a systematic fashion. The issue of the taxes imposed by government on project inputs created a serious constraint to operations. Perhaps most importantly, several fiduciary problems associated with implementation of project, including in procurement and financial management, persisted for much of the project, despite considerable support from the Bank supervision teams.

7.6 Implementing Agency:The performance of the implementing agency, i.e. the Ministry of Health, is rated as partially satisfactory. The competent leadership from the MoH and the PCO, who were saddled with a complex project, boosted the implementation of project activities, including the compilation and preparation of timely and comprehensive reports. While there were substantial delays with procurement and disbursement in the early stages of the project implementation, once the PCO had familiarized themselves with the IDA procedures, the situation improved (although some problems persisted), resulting in enhanced disbursement

- 18 -

and smoother project implementation. This, however, was only achieved through the considerable and very labor-intensive support provided to the PCO by the Bank supervision teams. One lesson learned is that the core project management team needs to be put in place before project effectiveness for faster project implementation. Also, due in part to the project’s complexity and the lack of a monitoring framework, the budget required for the PCO was triple the SAR’s estimate. Despite these problems and delays, which were compounded by the closure of the bank where the project held accounts, the project closed on schedule on December 31, 2002. There are also adequate arrangements in place for sustainability and a smooth transition to SWAp/budget support arrangements, through which the Health Sector Strategic Plan is being implemented.

7.7 Overall Borrower performance:Overall Borrower performance is assessed as partially satisfactory.

7.8 Partner PerformanceThe project was co-funded by the Government of Sweden through SIDA and the Government of the Federal Republic of Germany through KfW.

SIDA: The performance of SIDA is rated as satisfactory. SIDA provided three technical advisers who were very instrumental in the implementation of the project and overall reform of the health sector. SIDA promptly responded to all issues raised during implementation. The TA on Financial Management was highly regarded and considered to be invaluable to the implementation of DHSP by the government as well as by the other development partners.

KfW: The performance of KfW is also rated as satisfactory. KfW provided technical support and was prompt in releasing funds and responding to issues raised.

8. Lessons Learned

Impact of Decentralization: DHSP was implemented at a time when Uganda was just recovering lfrom conflict and internal strife, and furthermore had embarked on an ambitious decentralization agenda that sought to devolve the responsibility for the planning and implementation of essential public services, including health, to district administrations. Such devolution of power presents several unique challenges for the design and implementation of Bank supported sectoral projects, that need to be recognized and effectively managed, but which were underestimated by DHSP. These challenges include, inter alia: (a) Building and sustaining capacity in the decentralized entities, including ensuring adequate support from the center; (b) Ensuring an equitable distribution of resources – financial, technical and human – across decentralized units, and minimizing potential conflicts with vertical programs implemented by the center; (c) Ensuring that the decentralized decision making on the allocation of resources across various sectors by district administrations does not compromise the channeling of funds intended for health to the sector (as was the case prior to the initiation of conditional grants during project implementation by the central government in Uganda.); (d) Ensuring that the size of the special account should be carefully determined to take into account the longer replenishment cycle associated with a decentralized project; and (e) Determining how much procurement should be decentralized to ensure economies of scale.

Project vs. Program Support and a Multisectoral Context: DHSP was conceived and designed as a ltraditional project, but ended up providing broad programmatic support and gradually developing a larger group of development partners. The absence of a clear government sectoral policy compounded the situation. This raises substantial questions about the use of a traditional project as the instrument

- 19 -

for major sectoral and institutional reform. Furthermore, as the lender of last resort, IDA/DHSP ended up supporting a large number of initiatives that extended well beyond the objectives of the project, and even the mandate of the MoH. As noted, this raises serious questions about the appropriateness and effectiveness of using a health sector reform instrument to bring about reform in other sectors.

Politicized Environment: A public sector project that has a major focus on reforms is very sensitive lto the political environment. An assessment of the political climate is vital in the design and implementation of such a project like DHSP. However, these were underestimated. To be fully effective, reform efforts within the health sector need to explicitly recognize the politics and political economy of the sector (such as the viability of user fees, hospital autonomy, the geographical distribution of health facilities, the impact of the emphasis on the sub-district); the potential constraints imposed by other ongoing reforms (such as civil service reforms and the freeze on the hiring of health workers); the need to coordinate health sector reform efforts with initiatives in other sectors (such as initiatives in agriculture and education); the need for a proper sequencing of various reform initiatives and investments within the sector and across sectors; the need for early and extensive consultations with relevant stakeholders in order to ensure wide ownership; and the time required for complex reform initiatives to produce tangible outcomes. DHSP failed to anticipate and plan for such variables, and thereby failed to realize its full potential.

Accountability of Civil Society Partners: DHSP worked with decentralized and community-based lorganizations, which was a strong achievement. However, NGOs were constrained by having to work through Local Tender Boards. In addition, measures of accountability that responded better to local capacity, and that stimulated an appropriate focus on outcomes instead of inputs, needed to be developed. Such measures of accountability should be identified and agreed between the Bank and Borrower during the design of operations.

Special Accounts and Banking Issues: Projects such as DHSP that work with multiple implementing lpartners, such as districts, sub-district entities, and CBOs, FBOs, and NGOs need to make multiple advances to those implementing partners and thus require correspondingly larger Special Accounts. The criteria for the selection of commercial banks for the location of Special Accounts should also be scrutinized, revised if necessary, and strictly adhered to. Even with limited capacity in terms of logistics and human resources, absorption of funds improves when donor/ project regulations are relaxed. This was the case when the imprest system for releasing funds to districts was adopted.

Effective Monitoring and Evaluation and Procurement/Financial Management Capacity: A viable land effective monitoring and evaluation framework that is mutually agreed upon at the time of project preparation is essential for effective supervision, implementation, and evaluation of a project. Without such a system, it is virtually impossible to track the performance and impact of a project and keep the project on track. By the same token, detailed assessments of procurement and financial management, supplemented by ongoing support, are critical ingredients in effective project implementation.

Improved Donor Co-ordination and Need for Further Investments and Consolidation: Effective ldonor coordination is vital for the improved performance of the health sector. During the life of DHSP, the multiple planning and reporting arrangements of the different donors (who were implementing vertical programs) significantly burdened the district administrations. However, DHSP contributed to increasing donor coordination, and, to some extent, helped to lay the foundation for the SWAp in the health sector. While a base has been built in Uganda, the intended improvements in health service delivery and outcomes need more focus, and continuing support and investment by the donors. The government also needs to take increasing responsibility for financing the health sector. The competing

- 20 -

priorities of the government and the donors have, in the past, undermined the potential for positive change in the health sector. Further consolidation is, therefore, needed to realize the desired health outcomes in Uganda.

9. Partner Comments

(a) Borrower/implementing agency:Refer to Annex 8 for Detailed Borrower's Comments on draft IDA ICR and for the Borrower's Evaluation Report.

1. Comments of the Borrower on the ICR

1.1 The ICR prepared by IDA displays a good understanding of the design and scope of the project and the environment in which it was implemented.

1.2 We would like however, to make the following observations on some of the assessments made in the ICR. (a) Quality at Entry. In our assessment, the quality at entry was marginally satisfactory.

(b) Achievement of objectives and out puts. Our assessment is that given the complex design of the project and the challenging environment in which it was implemented, its overall achievement was satisfactory.

2. BANK AND PARTNER PERFORMANCE

Bank

2 1 Lending

The Bank performance was satisfactory at identification and preparation phases. At appraisal, the Bank's performance was satisfactory.

2.2 Supervision

The Bank's performance was satisfactory as the Bank met its formal supervision requirement of twice a year during project implementation and was able to develop good rapport and working relations with the government team.

2.3 Overall Bank performance

The Bank's overall performance was satisfactory.

Partner Performance

The project was co-funded by the Government of Sweden through SIDA and the Government of the Federal Republic of Germany through KfW.

(a) SIDA: The performance of SIDA is rated as very satisfactory.(b) KfW: The performance of KfW is also rated as very satisfactory.

(b) Cofinanciers:

- 21 -

Co-financiers comments were not completed by the time of publication.

(c) Other partners (NGOs/private sector):

10. Additional Information

- 22 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

Increased % of population accessing a basic package of services in pilot and demonstration districts.

Output Indicators:

Indicator/Matrix

Projected in last PSR1

Actual/Latest Estimate

Expand health facilities providing basic health services through physical rehabilitation.

Increased absorption capacity by districts of budgeted funds in annual work plans.

Expansion of pilot of the essential package of services from 6 to sixteen.

Relocate and restructure the Ministry of Health.

Conduct Studies to support reform.

Identify and pilot new financing mechanisms.

Training for district health team members.

Provision of medical and office equipment to districts. All districts provided with basic office equipment and vehicle.

Availability of guidelines for technical programs.

1 End of project

Annex 1 - Project Development Objective IndicatorsIndicator Baseline 1995 End of Project

2000-2002Comments Rating

Reduced mortality and morbidity from main causes in BOD

MMR 527/100,000 (UDHS 95)No change in adult mortality rate

Nutrition38% children <4 stunted5% children<4 wasted

MMR 504/100,000 (UDHS 2000/01)

Adult deaths 9 female deaths and 10 male deaths/1000 (UDHS 2000/01)

39% children<4 stunted4% children<4 wasted28% children 6-59 months Vitamin A deficient65% children 6-59 months anemic30% women 15-49 anemic (UDHS 2000)

MMR drop not statistically significant, and reflects an earlier period.

Malaria still remains the number cause of morbidity and mortality (no data).

AIDS is still the leading cause of death among 15-49 year olds – 50% of annual adult mortality and 12% annual deaths overall (CHDC and UNICEF

Not achieved (except drops in HIV prevalence (see STIP ICR).

- 23 -

1999).

Malnutrition and micronutrient deficiency also key: 50% of pregnant women are anemic and 30% of maternal deaths are attributable to anemia (Pyle et al, 2000 quoting WHO 1998 prevalence data.

Reduced IMR and <5MR

IMR 81/1000 live births <5MR 147/1000(UDHS 95)

IMR 88/1000<5MR 152/1000

Reflects a period 10 years previous40% of <5MR is related to malnutrition (UNICEF, and Pyle at al 2000.)

10% of teens and non-pregnant and lactating mothers have low body mass index (DHS, Pyle et al 2000)

Vitamin A deficiency may contribute up to 25% of child mortality (Beaton et al, 1993) – this is now being addressed within the EPI.

Not achieved

Lower TFR 6.9 (UDHS 95 using 92/94 data)

6.9 UDHS 2000/01 Was 7.3 1988 but now plateaued

Not achieved

- 24 -

Increased % of population accessing a basic package of services in pilot and demonstration districts

Only 20% of population accessing the basic package (MOH baseline estimate)MCH/FP/EPICPR all methods 15% 1995 UDHSANC 90%

Facility birth 35.4%Caesarean sections in preceding 5 years 2.6% of births 95 UDHSChildren <4 with diarrheal disease in 2 weeks before survey taken to a health facility 55.1%Children <4 with ARI in previous 2 weeks before survey taken to a health facility 61.4% All EPI 12-23 months 47.4% (increased from 31% 1988)MalariaNo obvious baseline

Water and sanitation – no strong baselineNutrition – see aboveTB/STDOther diseases

By 2000 42 units had been newly constructed and 98 rehabilitated.

CPR all methods 23% UDHS 2000/01

ANC 90% but only just over 50% received iron tablets.Facility birth 37%

Caesarean sections in preceding 5 years 2.5% of births 2000 UDHS

Children <5 with diarrheal disease in 2 weeks before survey taken to a health facility 44.9%

Children <4 with ARI in previous 2 weeks before survey taken to a health facility 64.7%

All EPI 12-23 months 36.7%(Received by 12 months as required by WHO 28.5%)

Only 13% households have mosquito nets (UDHS 2000), 8% children <5 and 7% of women of reproductive age sleep under one (3.2% of those nets treated in children, .3% of WRA slept under a treated net). 1/3 of pregnant women received malaria prophylaxis in pregnancy (UDHS 2000)

No clear data.

No monitoring framework for district comparisons, and pilot and demonstration phases merged.

Childhood disease indicators between DHS 1995 and 2000 shifted age from <4 to <5 so not completely comparable.

EPI program lost some focused management attention in shift from vertical program, as well as performance incentives such as payment to vaccinators.

Malaria policy developed. But whereas was to be $1 million allocated for ITMs, this was scaled sown due to differences of opinion about how to launch this endeavor. Locally made and treated nets are cheaper and more acceptable. TA helped draft malaria guidelines.

Mixed (see also table 1 below)

- 25 -

No improvement as above.Improved – see STIPLittle data

Still major gender issues to address when 13% households have nets but only half these accessed by vulnerable children and WRA.

Higher p.c. expenditure on health

$7-12 spent on health of which government spent $4 98/99 (Background to the Budget Ministry of Finance 98/99)

Achieved (but with limited payoffs in terms of health outputs and outcomes)

Higher per capita public spending on health (higher share of government spending on health)

IN FY 93/94, government recurrent expenditures were U.Shs 19.54 billion, while donors spent US$ 57.21 million (Picazo 1998). 2.5% of government spending was allocated to health in 87/88

FY 98/99 U. Shs. 59.76 billion, donors $73.53 million. 8% of government spending allocated to health 95/96 and 9% 97/88 (Costing the Minimum Health Care Package Uganda 1998). Other authors suggest by 1998 7% of total public expenditure was allocated to health (Hay 1998).

Difficult to separate donor and government spending in view of increasing trend to budget support.

Partially achieved (but with heavy reliance on external support)

Proportionately higher expenditure on lower level clinical services and public health

Government allocation to PHC 25.1% 92/3, to hospitals 69.7%

.

Government allocation to PHC 32.2% 95/6, to hospitals 58.4% (Picazo 1998)

At district level allocations to PHC only 25% districts spent 10% or more on basic health services FY 96/7 – so government re-earmarked conditional grants and then district non salary spending increased from U. Shs. 1.27 billion 96/97 to U. Shs. 2.57 billion 97/8.

By 2000 there had been an expansion of the pilot from 6 to 16 districts implementing the EHP, and the share of the budget going to the EHP increased from 33% in 1995 to 63% in 1999. The pilot was first expanded to 13 then to all 45 districts.

As of 1998, the Poverty Alleviation Fund (PAF) has provided funds directly to districts, now continued in the SWAp process making distinctions of the DHSP contribution difficult.

District annual work plans showed 5-10% of the total district budget was allocated to HIV, STI and TB-which were around 9% and 5% of the total BOD (BOD1995, Hutchinson 1998).

By 2000 the MOH was restructured and central staff reduced from 400 to 2000, and the MOH HQ relocated to Kampala

Achieved

- 26 -

Re-structured health services with more responsibilities to the districts for PHC

DHSP funds absorption 16% 1995-6, overall district funds absorption 60% 1995.

Only 32% of MOH departments had technical guidelines in 1995.

Of the actual FY 98/99 allocation of U. Shs. 54.3 billion, 43.1% is tied to central MOH and Mulago/Butabika hospitals, and 56.9% devoted to district management of which 38.4% is allocated centrally as delegated funds.District absorption capacity increased from 60% 1995/6 to 72.5% in 1998/9 while the absorption of DHSP grants increased from 16% in 1995/6 to 43% 1998/9 and to 70% by 2001. However 2001/02 project funds were fully committed so DHSP resources lost to the districts and only partially compensated for by PAF.Training for DHMT members in supervision (87%), planning (77%), HMIS (81%), and financial management (34%). MPHs provided for District Directors of Health Services (94%).

All districts provided with basic office equipment and vehicle, medical equipment to all public health facilities.

Impact e.g. on National Health Policy 1999 (focus on minimum package, strengthened roles for district health care, sustainable health care financing, enhanced roles for community and NGOs etc.)

Decentralization constrained by civil service reform and decentralization – local budgets unable to cope with staff, also hiring freeze. By 2001 strategy for consolidation of a PHC Conditional Grant payroll and verified salary arrears had been paid.

In the restructuring of the MOH, more role for the HPD including support supervision for district plans.

Guidelines available for 59% of technical programs in 1999 compared with 35% in 1995.

Achieved (However, in spite of increases in budgets to selected districts and increased spending on EHP, no improvement on service delivery or outcomes seen - See table 2 -4 below).

Different options for sustainable health care financing and management tested

Formal user fees in place at NGO facilities and informal fees in place in government.

Cost sharing: training undertaken, by 1998 5-10% of recurrent costs recovered from fees (up to 65% in NGOs): Government hospitals charging fees increased from 25 (1995) to 45 (1999). Fees increased from 1.3% to 9% of hospital expenditure. Government has now removed user fees below hospital level.Health insurance – study undertaken, not implemented

Numerous studies were supported on reform: support supervision, autonomy of hospitals, contracting out of clinical and other services.

Some models outside the project were tried and successful (Kisiizi Hospital scheme for community financing). Health insurance was assessed with consultancy support and deemed too costly by government, user fees were supported with

Partially achieved (but with no clear decisions on appropriate financing strategies)

- 27 -

training and guidelines but later cancelled due to political pressure and some concern about whether equity was being affected. Resource mobilization has been achieved with increased donor support and government allocations, which however rely heavily on budget support.

Greater roles for NGOs, private sector and communities in the provision of services

No clear baseline Efforts made to increase roles but no clear monitoring framework. Many initiatives unsuccessful.

Was to be $500,000 to support private sector units, this was changed. Private providers only involved since 1998. No evidence of study of NGO performance as outlined in the MTR. Funds constrained and political pressures had impact: e.g. Kamuli – ended up with a government provider for a district hospital instead of an NGO as planned. Indirect effect noted – the civil service freeze pushed people into the private sector, as new health graduates couldn’t be hired in the public sector. DCA said NGOs could supervise government facilities – not politically feasible. By 2000 over 30% of government health spending occurred through contracted services.

Partially Achieved

Effective-ness of Government Training institutions improved

No clear baseline No clear monitoring framework

By 1998 a study had been done on the lack of an HRD policy and recommended one be developed based on the EHP. Study proposed 3 models of training but then restructuring transferred schools to Education.By 1998 schools had separate bank accounts2000 – Studies conducted on management of training schools.2001 – Inter-ministerial committee established to review this.DHSP also supported the

Partially achieved

- 28 -

Makerere Public Health diploma in Health Services Management 1998-9 (30 participants per year); provided scholastic materials to schools, improved training for lab technologists, nurses and medical assistants, and supported tutor training in Mulago and Butabika.

Public health program for the army (identified target group) improved.

No clear baseline

No clear monitoring framework

Support to the army expanded over time to include civil works (Mubende, Tororo and Masindi), supervision, drugs and supplies, equipment and training (public health training to 4 staff and training for 100 vaccinators and 50 health educators). And to army casualties who live in a camp. + a community health program which extends to outlying community (malaria control, RH, school health and water/sanitation)

Achieved

Source The Republic of Uganda Public Expenditure Review Sept. 23, 2002. The World Bank

- 29 -

Table 1 -Utilization of Health Care by Those Reporting Sick by Poverty Level 1992-2000No/self treatment Government Private

1992 lowest 20% 54.14 17.62 28.23 Highest 20% 36.27 16.52 47.22 Total 44.87 18.06 37.071997 lowest 20%

56.92 18.90 24.18 Highest 20% 35.79 17.94 46.28 Total 44.80 20.88 34.322000 lowest 20%

43.92 18.93 37.15 Highest 20% 20.97 19.13 59.89 Total 30.58 19.79 49.63

Source: Household Surveys quoted in PER Sept. 23, 2002

Table 2 – Summary of Progress in DHSP Outputs by Components for 16 Pilot DistrictsIndicators Status

1995Status 1999

Status 2002

Target Comments

District absorption capacity

60% 72.5% 75% 89% Absorption of DHSP funds much less from 16% to 43%

Application of EHP

33% 63% 80% Proportion of district funds spent on actual delivery of EHP

Services contracted out

5.7% 12% 30% To CBOs, NGOs and private firms.

User fee collection User fees abolished in 2001 in government units below hospitals.

OPD Utilization 64% 68% increased (no user fees)

80% All districts

Service delivery DPT 3

72% 51% 46% 80% UDHS data. Outreach workers lost allowances. UNEPI figures higher.

DHTs trained in management

35% 77% 100%

Equipping of health offices

32% 100% 100% 100%

Rehabilitation of health units

0 units 42 130 200

HMIS monthly submission

66% 81% 100%

Financial decentralization

0% 66% 80% Proportion of un-earmarked funds

Downsizing the MOH

400 staff 220 200

Staffing of MOH 40% 69% 80% Lower figures in Annual ReviewsMOH departments with technical guidelines

35% 59% 80% Technical guidance now key role of MOH

Source: Konde-Lule J. et al. Final Evaluation of DHSP 2002.

Table 3 – Type and Level of DHSP Support to DistrictsCategory of District Key Activities Period of

SupportRounded Average Annual Support per District in Shillings

Pre-pilot (Gulu, Kabale, • Needs assessment 1994-2000: 100 million

- 30 -

Tororo) • Training• Service delivery

6 years

Pilot (Gulu, Kabale, Tororo, Apac, Kiboga, Kamuli, Iganga, Lira, Masindi, Mukono, Mubende, Rukungiri, Soroti)

• BOD study• Training• Supervision• Pilot service delivery• Rehabilitation of health units• Service delivery

1995-2000: 5 years

200 million

Capacity building 1 (Arua, Hoima, Kasese, Kisoro, Mbale, Mpigi)

• Training• Supervision• Rehabilitation of health units• Service delivery

1997-2000: 3 years

100 million

Capacity building 2 (rest of the districts)

As above mainly through NGOs

1998-2000 50 million

Source : Konde-Lule J. et al. Final Evaluation of the DHSP. 2002.

Table 4 – Proportion of District Health Expenditures on EHPDistrict 1995/96 1998/9Apac 6% 47%Gulu NA 65%Iganga 19% 48%Bugiri NA 19%Kabale 26% 68%Kamuli 29% 67%Kiboga 19% 70%Lira 38% 69%Masindi 31% 66%Mubende 10% 56%Mukono 53% 57%Rukungiri 7% 57%Soroti 18% 81%Katakwi NA 80%Tororo 2% 67%Busia NA 70%Average 33% 62%

Source : Konde-Lule J. et al. Final Evaluation of the DHSP. 2002.

- 31 -

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)AppraisalEstimate

Actual/Latest Estimate

Percentage of Appraisal

Component US$ million US$ millionPilot Activities & Demonstraton 24.27 13.02 54Capacity Building for District Health Administration 32.01 34.88 109Restructuring and Capacity Building for the Ministry of Health

18.82 17.36 174

Total Baseline Cost 75.10 65.26Total Project Costs 75.10 65.26

Total Financing Required 75.10 65.26

Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 2.00 0.00 11.20 0.00 13.20(2.00) (0.00) (10.20) (0.00) (12.20)

2. Goods 13.80 1.00 0.70 9.70 25.20(13.80) (1.00) (0.70) (0.00) (15.50)

3. Services 0.00 0.00 12.20 13.90 26.10(0.00) (0.00) (12.20) (0.00) (12.20)

4. Miscellaneous 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

5. Miscellaneous 0.00(0.00)

0.00(0.00)

5.10(5.10)

5.50(0.00)

10.60(5.10)

6. Miscellaneous 0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

Total 15.80 1.00 29.20 29.10 75.10(15.80) (1.00) (28.20) (0.00) (45.00)

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Expenditure Category ICBProcurement

NCB Method

1

Other2 N.B.F. Total Cost

1. Works 19.21 0.85 5.98 26.04() (0.00) (0.00) (0.00) (0.00)

2. Goods 9.86 0.03 0.06 1.03 10.98(0.00) (0.00) (0.00) (0.00) (0.00)

3. Services 0.53 12.29 5.26 2.47 20.55(0.00) (0.00) (0.00) (0.00) (0.00)

4. Miscellaneous 0.00 4.35 2.14 0.26 6.75(0.00) (0.00) (0.00) (0.00) (0.00)

5. Miscellaneous 0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

- 32 -

6. Miscellaneous 0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

0.00(0.00)

Total 10.39 35.88 8.31 9.74 64.32(0.00) (0.00) (0.00) (0.00) (0.00)

1/ Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.2/ Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units.

Annex 2 – Details of Component-Specific Activities and AchievementsComponent 1 Pilot and Demonstration Activities1. Identification of cost effective services

Burden of disease/cost-effectiveness (BOD/CE) study. Through the efforts of DHSP, a Burden of disease/cost-effectiveness (BOD/CE) analysis for 13 pilot districts was carried out between November 1995 and February 1996. District health teams (DHTs) were assisted to collect the necessary health data. They had been trained to carry out data analysis using the computer. Information obtained from the analysis would be used for health planning and budgeting. The following were the key outputs of the BOD/CE study:

· The burden for each of the top-ten diseases was quantified, · Health funding was estimated by source,· The cost of the essential health package (EHP) as defined by each DHT was determined and funding gaps were estimated,· The cost-effectiveness of common health care interventions was determined, · The mismatch between budgetary allocation and disease burden was determined.

2. Implementation of Cost –Effective Services to selected districts (Delivery of EHP to District Populations) In the pilot districts, DHSP has been the major source of funding for the delivery of the EHP to the district populations. DHSP on average provided about 40% of the funds spent by pilot districts in the delivery of the EHP (Excluding wages, hospital costs and most costs of drugs).3. Provision of District Level Supporting Services including:

· Strengthening of district health team supervision. DHTs were trained in support supervision, given the necessary guideline and provided with operational funds and logistics to carry out supervision

· IEC materials and services on various components of the EHP. · Social mobilization of the community by DHT on various health problems.· Provision of continuing education to health care providers. This was e central feature of

the DHSP support to the district capacity building component· Training community based heath workers such as Traditional Birth Attendants (TBAs)

and Community Health Workers (CHWs). This was a major target under the capacity building component of DHSP. TBAs were in addition equipped with the delivery kit.

· Equip and rehabilitate buildings and health facilities. In order to enhance the capacity of

- 33 -

health facilities to provide the health facility based aspects of the EHP, their rehabilitation and equipment became imperative. DHSP deployed a significant portion of its resources for this task.

· Develop sustainable repair and maintenance services for health equipment, facilities and transport. This was to include development of procedures for contracting out these services. DHSP supported this initiative through provision of resources to pay contractors, provision of information to districts on the relative cost of services and technical support to districts in the contracting process.

· Conduct necessary operational research. This effort was supported by DHSP through two ways. Students on MPH Program of Makerere University attached to districts undertook research relevant to districts with project funding. Districts separately identified operational research needs for which they budgeted and implemented though district work plans.

· Design and implement monitoring and evaluation system. During the development of district work plans, indicators for monitoring performance were identified. Monitoring and evaluation was one of the areas in which DHTs were trained. The HMIS supported by DHSP, is one of the monitoring tools.

4. Test resource mobilization and efficient use options

· Improved financial management and accounting system implementation

· Assessing feasibility of facility pricing proposals/ Establishing user-fees for publicly provided health care services. Since colonial times, public sector health services in Uganda have been provided free of charge. Various post independence governments have maintained this policy. From early 1980s, following a period of severe economic decline in the country, health sector funding could no longer sustain the large health care infrastructure that had been established, or the growing demand for health care. As a result, informal and illegal charging for health services in the public sector became widespread. An attempt was made between 1987 and 1993 to formalize user-fee as a national health policy. A bill that would enshrine the policy was however rejected by the parliament. But the Local Government Act of 1997 allows local authorities to charge fees, at their discretion, for services they provide. This formed the basis for national user-fee policy development. DHSP promoted and supported districts in establishing user-fees with two main objectives: first, to introduce the population to being cost-conscious about healthcare. Second, once this notion is well established, the risk of health care costs to households would form the basis for introducing a national health insurance. Other objectives, such as revenue generation, were considered to be of secondary importance. Most DHSP support has been for developing and disseminating user-fee guidelines and for training health workers and committees to use the guidelines. The guidelines also provide guidance on exemption and waiver mechanisms, which are implemented by both health workers and community leaders. The support included the provision of logistics such as safes, cash boxes and receipt books. While user-fees were applied widely, they were not applied in all districts or health units. A study carried out revealed that most health units had user-fee management guidelines. User-fee collections were found to have improved over the previous 2 or 3 years. The collections represented about 0.5% to 10% of the total health unit/hospital expenditures. It was interesting to find that hospitals with previously low user-fee collection dramatically increased their collection simply by reducing the number of collection centers. The efforts of DHSP culminated into a draft policy on user-fees. Government has however, revisited the issue of user fees upon realization that many people were not accessing heath services because they cannot pay user-fees. In referral and district hospitals, private wings have been established where those who can afford will be attended to. The second wing will offer

- 34 -

free services to those who cannot afford costs of the private wing.

· Feasibility of drug revolving funds. DHSP supported efforts to establish drug revolving funds. Various studies under DHSP and the DANIDA funded Uganda Essential Drugs Support Program (UEDSP) however, have shown that it is not feasible, under the existing circumstances, to establish a revolving fund at the district level. Districts receive drugs, (free of cost to themselves), from many sources. The immediate problem facing districts is therefore that of a multitude and uncoordinated methods of drug supply, funding and delivery. Against this background, the need to create a revolving fund proved difficult. The lack of supplies/logistics and a management system at the district level have compounded this situation, which has resulted in the poor management and distribution of drugs and supplies within districts.

Therefore, MOH working with different donor agencies financing drugs in the country commissioned reviews to assess the drug situation. The agencies included DANIDA, DHSP, STIP, DFID, the National Drug Authority and National Medical Stores. The reviews identified poor prescription culture and inefficient management and use of drugs throughout the country as some of the key problems. DHSP and DANIDA, in association with the National Drug Authority, supported the implementation of recommendations, which included training of health workers in rational drug use and in assessing district drug needs.

· Test community income generating activities. Community Based Income Generation for Financing Health Services. This initiative was meant to raise the income levels of communities so that they are able to pay for health services. Two schemes for income generation were supported through DHSP as pilots. One was a potato-growing project in Kabale district. A seed fund of U. Shs. I million was given to women's group to start the project. Two years later, it had generated over U. Shs. 10 million worth of assets and income. The women and their families have benefited through improved nutrition, as they were able to buy nutritious food from the proceeds of the sale of potatoes. Improved income is also expected to have made it easier for them to access health care. The other scheme, less successful, was a bakery in Gulu, started by health workers. Due to insecurity and difficulties with transportation, it was not possible to find a market for the bread. The project did not therefore last long or provide the envisaged benefits. However, questions about the extent to which the health sector should get involved in income generation were raised. It was concluded that directly supporting income generation was not the role of health sector, which lacked the necessary institutional framework to provide back up services.

· Test local health insurance proposals. Two prepayment pilot schemes were supported in Tororo and Kisiizi Hospitals. In Tororo, peasant farmers were advised to pay in advance for their hospital expenses at the time of harvesting maize and rice. The scheme was successful for a couple of years, mainly as a result of a vigilant coordinator and good harvests. As a result of poor harvests in the subsequent seasons, the scheme became unsustainable. The second, more successful scheme in Kisiizi Hospital was partly financed by DFiD. It was launched in September 1996 and evaluated in April 1997. The enrolled membership was 3 times what had been estimated. The success of this scheme has been attributed to: (a) the existence of sickness societies (or Engozi) where risk sharing is not an alien culture (b) availability of a good quality hospital C slow, non-pressuring community mobilization. The Top Management of the Ministry of Health however, observed that the success of the Kisiizi scheme was too short to be conclusive. It was recommended that the pilot be extended to 8 hospitals in different parts of the country before a decision could be taken on whether and how to adopt prepayment as a national policy.

- 35 -

· Assessing the Feasibility of Social Health Insurance in Uganda. A study financed through DHSP to assess feasibility for health insurance in Uganda, proposed three types of insurance: (a) mutual funds for large firms and civil servants (b) voluntary health care plans for the rural population and people without formal employment, and (c) a Medi-Save program in which members can save on the money they have prepaid for healthcare. In addition, the study proposed further development of private insurance, rationalization of the third-party car insurance and of mutual accident funds. A study to assess the feasibility of providing a package of health services for workers was carried out by the Health Planning Department through DHSP support. While it was found to be economically feasible for such services to be provided through insurance, insurance requires an enabling legal framework. At the moment it is deemed politically and financially not viable

5. Improve collaboration with private sector and NGO providers. The initiatives strengthened or initiated by DHSP regarding Private- Public Collaboration have been concretized in the National Health Policy. A portion of funds allocated to the Ministry of Health from the national treasury, is now used to fund NGO hospitals and health units· Review existing barriers. It was envisaged that better collaboration with the private sector would increase the capacity of the districts to deliver the EHP to district populations. In order to enhance the delivery of the EHP by the private sector, it was found necessary to review the laws and regulations governing private medical practice and to identify possible entry barriers to the health care market. Several barriers were identified and recommendations to eliminate them made. Many of these recommendations are being incorporated for adoption in the new health policy and Health Services bill. Other recommendations, such as those related to contracting, grants for NGOs, and exploring the feasibility for a wider role of health insurance are being implemented.· Involve private sector in health planning and implementation. Districts have been encouraged to allow private providers to participate in the development and implementation of district work plans with project funding. There was little success however, as districts, which had traditionally been under funded considered private providers to be competitors for scarce resources. It can be argued however, that DHSP was not assertive in promoting this collaboration because under the STI project, the same arrangement worked. · Direct support to private organizations. The project provided financial support to over 100 private providers to implement approved proposals. The proposals covered core components of the EHP namely: malaria control, reproductive health, and water and sanitation and school health. The DCA provided for USD 500,000 for the establishment of private units in undeserved areas. Each private unit was to be provided with up to USD 25,000 for equipment. This has not been done because private for profit health providers are demand and profit driven. It was realized that health facilities that are not demand-driven, and hence unsustainable, would be established through the grant. · Training. DHSP financed the training of district health staff without discrimination between public and private. The main types of training have been operational and mid level management courses· NGO Panel. A national NGO Panel was formed at the MOH to promote and put in place modalities for private-public collaboration. When districts failed to collaborate with the private sector in work plan implementation, the Panel selected private sector proposals for DHSP funding. Alongside district authorities, the panel also supervises the implementation of private sector activities funded by the project. District NGO Panels were not found necessary as an equivalent body, the District Health Management Team, already existed in each district, and comprises of DDHS's core team (or, the District Health Team, DHT) and representatives of NGOs, CBOs and hospitals

- 36 -

· Contracting Non-core Services to the Private Sector. DHSP supported and strengthened the contracting by hospitals to the private sector of non-core services such as repair and maintenance of vehicles, equipment, buildings, gardening, cleaning and catering. Support included funding for contracted out services and technical support supervision. As a result of these efforts, the proportion of health expenditure on contracted services grew from 5.7% to 12% between 1995/1996 and 1998/1999 in the 13 pilot districts.· Contracting selected Health Centers to or From NGOs and Private Providers. The DCA specifically required that studies and experimenting of contracting to NGOs be carried out nationwide, and in particular in Kamuli district and health centers in Kampala. The study to determine optimal ways of contracting was carried out in 4 districts. Kamuli district leadership however, preferred a government district hospital to be built rather than contract services from an NGO hospital. This is being done through a loan provided by the Spanish Government. Contracting out of health centers in Kampala was not carried out. This was expected to benefit from the strategic health plan for Kampala and the greater autonomy for Mulago Hospital. Both studies were supported by DHSP.· Provision of Medical Equipment to NGO Health Facilities. NGO Health facilities including Kamuli Hospital and Vira Maria Hospitals benefited from medical and, laboratory equipment procured through DHSP. All the NGO health facilities heading health sub-districts benefited from this procurement.· Supervision. The DCA provides for NGOs with good performance record to supervise Government health units and community-based activities. This kind of collaboration already existed in districts and has been enhanced further through the establishment of health sub-districts. NGO health units with good performance are in charge of a health sub-district with several health units including Government ones, to supervise. DHSP provided funds for supervision carried out by NGO facilities and personnel.

· Evolving Alternative Options for Paying Health Workers. It was envisaged that health workers paid adequately and promptly were vital for the delivery of the EHP. In light of the poor remuneration of government, which was also not paid on time, DHSP was to evolve alternative options for paying health workers. A study financed by DHSP was carried out in Kabale district to identify and recommend different ways of motivating staff. The study recommended (a) prompt payment of salaries and allowances (b) performance based rewarding (c) allowing co-ownership of motorcycles or bicycles (d) provision of housing (e) prompt appointment, confirmation and promotion of staff. The study noted that the salary of most staff was below the basic monthly expenditure. The staff therefore spent a good part of their work time making a living by other ways, such as trading and farming. The most important incentive to health workers would be to meet their basic living cost. A DHT bonus scheme was briefly tried in Kabale but was stopped, as members were not being rewarded by performance but by seniority. Motivating health workers though appropriate pay is the mandate of the Ministry of Public Service and is a core component in the on going Civil Service Reform Program. The Ministry of Health through DHSP could not carry its efforts forward though fully aware that motivated health workers are critical for the delivery of the EHP to the population

- 37 -

5. Support PHC Activities

· Evolving a Community Based Mechanism for Sale and Maintenance of Impregnated Bed-nets. Through DHSP, funds have been provided to the districts to procure Impregnated Mosquito Nets, which were to be sold to users and funds used to procure more nets. This effort was hampered by two problems namely: the prohibitive cost of the nets to users as they were until recently, taxed and the lack of a promotion strategy by the Ministry of Health which created the impression that communities do not appreciate the use of impregnated nets as an effective strategy for malarial control. DHSP lobbied against the taxes, and the nets are now tax-free.· Increase use of Pit latrines and availability of Clean Drinking Water. DHSP has provided materials for latrine construction to community members, and has supported the construction of latrines in schools and other public places, The public is being sensitized about the importance of using latrines through various ways including mass media campaigns. Through the project, water sources also have been protected and some limited sinking of bore holes done. It should be noted however, the mandate for provision of water lies with the Ministry of Water, Lands and Environment.· Procure Drugs, Vitamins, Equipment and Supplies. DHSP has, to a limited extent, funded the procurement of essential drugs by districts from the National Medical Stores. The main source of for the health sector has been the Uganda Essential Drugs Program funded by DANIDA (Denmark).

Component 2 – Capacity Building for District Administrations

1. Institutional Development of the District Health Teams

·Logistical Support. The project provided districts with: Transport (Motor vehicles; Motor cycles; Bicycles; Ambulances; Tricycles); Office Equipment (Computers; Photocopiers; Manual Typewriters; Duplicating machines; Generators; Fax machines; Safes; Filing cabinets and shelves; Furniture ·Human Resources Development. Government and NGO health workers at the district level were trained to improve service delivery, planning and budgeting, management, computing, monitoring and evaluation, support supervision. Decentralization. DHSP funded the restructuring of the district department of health, which was a basis for further decentralization of health services to the health sub district level. This has enabled the DDHS to concentrate on planning, monitoring and technical support with health sub districts. Planning. Every DHT and staff of lower level health units have been trained in planning, district planning guidelines have been supported and DHSP has also funded the annual planning exercise during which central facilitators guide the development of district work plans. DHSP introduced an element of certainty in the planning process (planning ceilings, which were always adhered to were known before hand), which has contributed to the emergence of a culture of rational and realistic planning. Districts now know the top ten diseases and the cost-effective interventions. ·HMIS. DHSP has contributed to the development and strengthening of a system for collecting, collating, analyzing, disseminating and utilization of health related data. Specific actions supported with regard to HMIS include: Training of DHTs, Hospital and Health unit management teams in HMIS; printing of HMIS forms; support supervision of health units by DHTs. As a result of these efforts, the % of health facilities submitting monthly HMIS returns has increased from 66% in 1995 to above 80% in 1999. Filing System. Because of the increasing number of records handled by the heath department at the district level and to ensure easy retrieval and utilization of information, DHSP supported the development of a filing system. Besides

- 38 -

provision of filing cabinets and meeting recurrent costs, DHSP hired a consultant who trained trainers who in turn trained DHTs and staff of health units in filing, which has improved records management at the district level.·Stores Management. Besides constructing stores, DHSP supported the revitalization of the system for managing stores: stores management training, and stock cards procurement As a result, it is now possible to know in the district stores, what has been received, issued to who and the balance with regard to drugs and other supplies·Accounting System. DHSP funds at the district level were managed within the existing financial management structure. Guidelines and formats have now been provided for utilization and accounting for project funds. This is a result of training accounts staff attached to the department of health, provision of guidelines and regular support supervision by the PCO. ·Support Supervision. DHSP supported the strengthening of the support supervision system at the district level (DHT) and central programs·Strengthening the Public Health Directorate of NRA ( UPDF)2. Testing Greater Autonomy of Government Health Units. It was envisaged that if government health units were more autonomous, they would be more effective and efficient. A study commissioned by DHSP recommended ways of making Mulago Hospital more autonomous. The recommendations are under consideration by the MOH. A consultant funded by DHSP has also reviewed the structures of referral hospitals. The structures are consistent with the greater autonomy aspirations that referral hospitals are expected to have. Hospitals, for example, have management committees that are responsible for day-to-day issues. They also now have responsibility for procuring drugs and other supplies for which they directly receive funds from the treasury. District hospitals have been decentralized to District Local Authorities. The next step is for district hospitals to gain sufficient levels of autonomy to operate efficiently without much interference from either the MOH or district authorities.3. Measures to Improve the Effectiveness of Government Training Institutions. A study to propose ways of improving health training in Government health institutions was carried out with DHSP funding. Several proposals were made and as result of the study, the HRD policy and plan for the health sector has been drafted. However, the restructuring of the Government has overtaken the implementation of the proposals. In the new structure, all health-training institutions have been removed from line ministries and placed under the Ministry of Education. The MOH is now only responsible for providing technical guidance on the training policy and for national human resource planning for the health sector.

Component 3 – Restructuring and Capacity Building for the MOH1. To restructure the Ministry around its new responsibilities. Through DHSP, consulting services were procured for the restructuring of the Ministry of Health to align it with its new role. The districts also clearly understand that direct service delivery of health care to the population is now their role. The Ministry now has a better functional definition and grouping and is now organized around its constitutional mandate, which includes:

· Policy formulation,· Setting Standards· Quality Assurance· National Epidemic Control· Capacity building of districts and other providers· Technical Support· Coordination· National Planning

- 39 -

· Coordination of Research· Monitoring and Evaluation· Resource Mobilization

Regional referral hospitals were also restructured through the efforts of DHSP. The exercise prepared the hospitals for autonomy. This has not yet been granted but the hospitals are now self-accounting and have management committees for decision making on day-to-day operational issues. Such decisions as determining need for drugs and other supplies and undertaking procurement are now taken by referral hospitals.2. Training. Many officers of the Ministry have been trained at home and abroad in various disciplines to enhance their capacity to undertake roles consistent with the mandate of the Ministry of Health. Capacity for policy formulation, strategic planning, monitoring and evaluation, provision of technical support, development of guidelines has improved. The National health policy and Health Sector Strategic Plan were for example, developed with little external technical assistance. The Ministry staff is moving forward the SWAp process again with little external technical assistance. The Ministry has officers trained in such new areas such as environmental health and health insurance and telemedicine with DHSP support. 3. Logistical Support. The project provided Ministry of Health with logistical support namely: Transport (Motor vehicles, Ambulance ( Mulago Hospital); Office Equipment (Computers, Photocopiers, Generator, Fax machines, Filing cabinets and shelves, Furniture)4. Strengthen Heath Planning and Inspection Departments

5. Introduce a Quality Assurance Program in Ministry of Health. In accordance with its mandate, it was imperative for the Ministry of Health to develop a system for ensuring that health care standards are adhered to by private and public heath care providers. DHSP has supported the Ministry of Health to develop this Quality Assurance System. As a result of DHSP support, the % of MOH departments with guidelines has increased from 32% in 1995 to 59% in 2000.Specific actions funded include:

· Paying salaries of staff of the quality assurance unit that was originally not part of the Ministry structure and staffing.

· Based on the successful pilot, the Quality Assurance Department was included as part of the regular structure of the Ministry of Health through a restructuring process funded by DHSP. Quality Assurance Departments have also been established in other ministries such as Water and Mineral Development based on the successful pilot of DHSP.

· DHSP has funded the development of guidelines and standards covering many program areas such as malaria, school health, environment health, nutrition, and reproductive health.

· DHSP has funded the training of government and non-government providers on quality assurance.

· Quality Assurance visits have been regularly funded by DHSP.6. Movement of the Ministry of Health headquarters to Kampala. Through DHSP, the Ministry of Health Headquarters was reconstructed and relocated to Kampala. This has improved the operational capacity of the Ministry as most departments are now accommodated in one building making coordination easier. The relocation has also made it easy for the clients of the Ministry to access its services.7. Formulate and implement action plans for health services decentralization and restructuring. Through DHSP, the process and system for health care planning at the national and

- 40 -

district levels has been strengthened. DHSP has supported the development of the National Health Policy and health Sector Strategic Plan. These two documents provide overall guidance to the planning process in the entire health sector – public and private. Public sector funding for the health sector is based on an agreed work plan that must be consistent with the national health policy and health sector strategic plan. The development of planning guidelines and facilitation of the planning exercise at national and district levels has been supported by DHSP. Owing to its central role in the reform of the health sector, the Health Planning Department was targeted for support by DHSP. Support included: training, logistical support and operational funds to execute the functions of the department.8. HMIS. DHSP has contributed to the development and strengthening of the System for collecting, collating, analyzing, disseminating and utilization of health related data. This system has now been constituted into the Resource Center of the Ministry. Specific actions supported with regard to HMIS and later the Resource Center include: Redesigning the HMIS system- DHSP co -funded with DANIDA and UNICEF the consultation process; Prevision of computers to central HMIS Unit; Training of DHTs, in HMIS; Support supervision of DHTs by the central team9. Financial Management. Though DHSP funding and technical assistance obtained through DHSP, districts and hospitals have been assisted to strengthen the system for management of funds. Regular field visits funded by DHSP helped in institutionalizing the system. Training of accounts staff at the central levels has contributed to the building of a strong financial management system. Computers have also been provided by DHSP making it possible to computerize the system.10. Procurement. The expertise developed through DHSP in the area of procurement is assisting the Ministry of Health in executing its regular procurement functions. The PCO system will form the foundation of the new procurement unit, which is being, formed in the Ministry of Health in consultation with the Ministry of Public Service.11. Moving Forward the SWAp Process. By design, DHSP had characteristics of a SWAp. Many donors were expected to fund a common program of work through the project that was largely sector wide. Some institutional aspects of SWAp such as using regular systems of government had some similarities with the design of DHSP. (DHSP operates within government systems particularly at district level). The design of the Uganda SWAp has therefore had a lot to learn from DHSP. Project staff has also been instrumental in the design and implementation of the SWAp because of their experience. Further support to the SWAp process includes:

· Various consultations and joint reviews have been funded by DHSP. · The process of developing the National Health Policy and Health Sector Strategic Plan,

which served as the foundation of the SWAp, also benefited from DHSP funding.

Study Tours to countries that adopted the SWAp concept earlier than Uganda were funded by DHSP. This enriched the development of the Uganda SWAp.

12. Main streaming Gender in the Health Sector. In the design of DHSP, mainstreaming of gender in the health sector was not a specific requirement. In the course of implementation however, it became clear that gender issues could not be ignored. The following actions were undertaken with project support as a contribution to the on going efforts to mainstream gender in the health sector:

(a) The review of laws and regulations governing the health sector funded by DHSP included a gender perspective. As a result, midwives are now allowed to operate maternity homes

- 41 -

(b) DHSP funded the review of the draft health policy to include a gender perspective. The final edition of the policy has specific actions for bridging gender gaps in the health sector.

(c) As a result of the DHSP initiative and the ongoing discussion on the need to address gender issues, several programs of the Ministry were reviewed to identify and bridge gender-gaps.

(d) Following restructuring, which was funded by DHSP, an analysis of the allocation of key positions at the Ministry headquarters, districts and hospitals, was done by DHSP. The analysis revealed serious under representation of women in key leadership/decision making positions. This information was made available to top management with a view that it would inform the process of recruitment and promotion and deployment.

(e) Selection of candidates for training under the project had gender as part of the criteria. Women were given special consideration, given their under representation in key leadership/decision making positions. As a result, more women than men were trained overall.

(f) Whenever a woman who was selected from in country training was breast-feeding, DHSP paid for the baby-seater and extra milk for the baby if required.

(g) Under the user fees scheme in government health facilities developed with DHSP support, there was a waiver mechanism for those unable to pay. Most beneficiaries were women.

(h) The Ministry of health headquarters although not a very tall building has lifts, making it friendly to people with disabilities.

- 42 -

Annex 3. Economic Costs and Benefits

not applicable

- 43 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle Performance Rating No. of Persons and Specialty

(e.g. 2 Economists, 1 FMS, etc.)Month/Year Count Specialty

ImplementationProgress

DevelopmentObjective

Identification/Preparation11/1993 8 1 Sr. Public Health Specialists

1 Operation Officer1 Economist1 Field Manager - ODA2 USAID2 Representative from SIDA

Appraisal/Negotiation5/1994 14 1 Sr. Public Health

Specialists2 Operation Officer3 Economist1 Staff Assistant5 Representatives ( ODAUSAID, SIDA, DANIDA2 Peer Reviewers

Supervision

10/26/1995 8 2 Operation Officer1 Sr. Economist1 Representative - ODA1 Task Manager2 Representative from SIDA1 Implementation Specialist

S S

04/26/1996 6 2 Operation Officer1 Sr. Economist1 Representative - ODA1 Task Manager1 Implementation Specialist

S S

11/08/1996 7 2 Operation Officer1 Sr. Health Economist1 Representative - ODA1 Task Team Leader1 Consultant1 Disbursement Analyst

U S

05/29/1997 8 3 Public Health Specialists2 Economist1 Team Leader1 Communications1 Implementation Specialist)

U U

11/14/1997 12 3 Public Health Specialist1 Operation Officer4 Economist

S S

- 44 -

1 Team Leader1 Health Program Officer1 STD Advisor1 Implementation Specialist

05/28/1998 9 1 Team Leader1 Public Health Specialist1 Health Specialist1 Implementation Specialist1 Procurement Specialist1 Program Officer1 Sr. Health Advisor1 Public Health Specialist1 Sociologist

S S

04/30/1999 8 1 Public Health Specialist1 Mission Leader1 Sr. Health Specialist1 Health Specialist1 Health Economist1 Consultant1 Sr. Health Advisor1 Procurement Specialist

S S

11/05/1999 15 1 Team Leader3 Health Specialists1 Procurement Specialist1 Consultant1 Malaria Specialist1 Health Economist1 SIDA, Regional Adviser2 SIDA Consultant1 USAID, Tech. Adviser1 Macroeconomist 1 Macroeconomist-Uganda1 Financial Mgt. Specialist

S S

04/18/2000 7 1 Mission Leader1 Pr. Health Specialist1 Health Specialist1 Sr. Procurement Specialist1 Financial Mgt.2 Pharmaceuticals Specialist

S S

10/27/2000 4 1 Team Leader1 Lead Health Specialist1 Health Specialist1 Sr. Procurement Specialist

S S

03/21/2001 2 1 Team Leader1 Health Specialist

S S

11/21/2001

6/2002

4

3

1 Team Leader1 Team Member1 Procurement Specialist1 Member Financial Mgt.

1 Sr. Health Specialist1 Team Leader

S S

- 45 -

1 Procurement Analyst

ICR1/2003 3 1 Team Leader, 1 Health

Specialist, 1 Public Health Specialist

S

(b) Staff:

Stage of Project Cycle Actual/Latest EstimateNo. Staff weeks US$ ('000)

Identification/Preparation 163.60 105Appraisal/Negotiation 71.00 62.80Supervision 790.60 998ICR 21 75Total 1046.62 1240.80

- 46 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingMacro policies H SU M N NASector Policies H SU M N NAPhysical H SU M N NAFinancial H SU M N NAInstitutional Development H SU M N NAEnvironmental H SU M N NA

SocialPoverty Reduction H SU M N NAGender H SU M N NAOther (Please specify) H SU M N NA

Private sector development H SU M N NAPublic sector management H SU M N NAOther (Please specify) H SU M N NA

- 47 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bank performance Rating

Lending HS S U HUSupervision HS S U HUOverall HS S U HU

6.2 Borrower performance Rating

Preparation HS S U HUGovernment implementation performance HS S U HUImplementation agency performance HS S U HUOverall HS S U HU

- 48 -

Annex 7. List of Supporting Documents

Beaton G. et al. Effectiveness of Vitamin A Supplementation in the Control of Young Child Morbidity and Mortality in Developing Countries. University of Toronto Report to the Canadian International Development Agency, 1993.

Berman P. et al. A Feasibility Analysis of Social Health Insurance in Uganda. Dec. 2001.

Child Health and Development Center (Makerere) and UNICEF Kampala. A Situation Analysis of Women, Adolescents and Children in Uganda: An Analysis for the Health Program, Sept. 1999

Hutchinson P. Ongoing Progress on Decentralization in Uganda’s Health Sector. Mimeograph Uganda Resident Mission. 1998

Hutchinson P. Household Demand for Health Services in Uganda. Assessing Outcomes for a Comprehensive Development Framework. October 1999.

Konde-Lule J. et al. Final Evaluation of the District Health Services Pilot and Demonstration Project (DHSP). September 2002.

KPMG. Recommendations for Restructuring of the MOH in Uganda. 10 Mar. 1998.

Ministry of Health. White Paper on Health Policy Update and Review.1993.

Ministry of Health. Mid-Term Review Report District Health Services Pilot and Demonstration Report. Uganda – IDA Credit No. 2679-Ug. Sept. 1998.

Picazo O. Uganda District Health Services Pilot Project Mid-Term Review of Health Care Financing Aspects. Oct. 12-23 1998.

Pyle D. et al. Nutrition Policy and Strategy; Opportunities for Programming and Impact. John Snow International, May 2000.

Uganda Bureau of Statistics, Macro USA. Demographic and Health Survey. 1995.

Uganda Bureau of Statistics, Macro USA. Demographic and Health Survey. 2000/01.

World Bank. DHSP Aide-Memoires Supervision Missions, MTR, and PSR reports.

- 49 -

Additional Annex 8. Comments from MoH and Borrower's Implementation Completion Report

Borrower's Comments on the draft Implementation Completion Report Prepared by IDA

1. Comments of the Borrower on the ICR

1.1 The ICR prepared by IDA displays a good understanding of the design and scope of the project and the environment in which it was implemented.

1.2 We would like however, to make the following observations on some of the assessments made in the ICR. a) Quality at Entry. In our assessment, the quality at entry was marginally satisfactory.The assessment of the ICR does not take full account of the following positive factors:

The project was consistent with government priorities for the health sector as reflected in the 1993-96 lhealth policy and plan.The project benefited from a baseline study.lThe reform element of the project was in line with other on going government reforms such as ldecentralization and civil service reform. There was a positive environment for reform.DHSP inherited some officers, personnel, and logistics from the First Health Project that ensured a lquick project take off.

We agree with the identified factors, which negatively affected quality at entry.

b) Achievement of objectives and out puts. Our assessment is that given the complex design of the project and the challenging environment in which it was implemented, its overall achievement was satisfactory. The assessment of the ICR does not take full account of the following:

The project made a significant contribution to the revitalisation of the health sector. It introduced the lconcept of the Essential Health Package (EHP) which is now the basis for planning and resource allocation. There is now, as a result, more rational use of resources. The project also provided the financial resources for delivering the EHP to district populations in the 13 and later 16 pilot districts. Support services such as community mobilisation and IEC necessary for delivering the EHP were also supported.

In terms of capacity building, the project supported human resource and institutional development, linfrstustrctural and systems development and provision of logistics at the district and central levels. The public health sector is now one of the most vibrant government institutions. DHSP also made a significant contribution to the institutionalisation of the collaboration between public and private sectors in health care delivery through support to NGOs.

The Project’s institutional development impact is high. DHSP has enhanced the capacity of the health lsector to use human and material resources for improved health care delivery. DHSP laid the foundation for the current SWAp process, which has provided a basis for further development of the health sector in a coordinated manner.

The sustainability of the project is rated as likely because many of the reforms and initiatives started or lsupported under the project are now part of the HSSP and have already been integrated in the regular

- 50 -

operations of the MOH through budget support and SWAp arrangements.

2. BANK AND PARTNER PERFORMANCE

Bank

2 1 Lending

The Bank performance was satisfactory at identification and preparation phases. At the preparation phase, the preparation team clearly identified the critical heath sector issues, which were consistent with Government of Uganda's priorities in addressing heath care as reflected in the 1993-1996 Health Policy and Plan and the Health Sector White Paper. The Bank also built partnership with other development partners in supporting heath sector development some of whom funded the pre-investment studies and later on agreed to co-finance the project.

At appraisal, the Bank's performance was satisfactory. The Bank's appraisal mission included a team with a comprehensive skills mix. At appraisal project institutional arrangements both at the center and districts, phasing of project implementation, the role of the NGOs, linkages with existing heath sector programs supported by other development partners were further reviewed and agreed with government.

2.2 Supervision

The Bank's performance was satisfactory as the Bank met its formal supervision requirement of twice a year during project implementation and was able to develop good rapport and working relations with the government team. This ensured timely and adequate follow up of agreed actions and early resolution of emerging issues. DHSP was a complex project whose successful implementation would not have been possible without a very supportive IDA task team.

2.3 Overall Bank performance

The Bank's overall performance was satisfactory.

Partner Performance

The project was co-funded by the Government of Sweden through SIDA and the Government of the Federal Republic of Germany through KfW.

i) SIDA: The performance of SIDA is rated as very satisfactory. SIDA provided three technical advisers who were very instrumental in the implementation of the project and overall reform of the health sector. SIDA promptly responded to issues raised.

ii) KfW: The performance of KfW is also rated as very satisfactory. KfW provided technical support and was prompt in releasing funds and responding to issues raised. No KfW funds were cancelled because the KfW agreement expired only after all the KfW funds had been utilized.

____________________________________________________________________________________Summary of Borrower's Implementation Completion Report Prepared by MOH

District Health Services Pilot and Demonstration Project(IDA-26790: Project ID: P002791)

- 51 -

1. Assessment of Development Objective and Design and Quality at Entry

1.1 Background

The District Health Services Pilot and Demonstration Project was a successor to the First Health Project. Following the completion of the First Health Project (FHP) funded through a World Bank credit, it became imperative for the Government of Uganda (GOU) to plan a systematic development of the health sector. FHP had been conceived as an emergency program to restore the functional capacity of a number of health facilities and services. The GOU and IDA had originally proposed CHAP, Community Health and AIDS Project as a successor to FHP to address health policy issues and contain the AIDS epidemic. Given the seriousness and urgency of the AIDS epidemic however, the original project was split into two projects: the DHSP and the Sexually Transmitted Infections Project (STIP) with the latter being launched earlier than DHSP. 2.2 Objectives

Specific Objectives of DHSP which never changed through the life of the project were:

i) To pilot and test new sector policies which will facilitate theimplementation of essential health services.

ii) To strengthen planning and management capacity at district levels so that they are prepared to deliver essential health services.

iii) To restructure Ministry of Health and build its capacity to provide health policy leadership and to support the government’s policy of decentralisation.

The objectives of the project were clear and consistent with the aspirations of government for the development of the health sector.

1.3 Original Project Components

The project had four original components namely: i) Pilot Activitiesii) Demonstration Activitiesiii) Capacity Building for District Health Administrationsiv) Capacity Building and Restructuring for the Ministry of Health

1.4 Revised Project Components.

Project components remained the same, but the strategy for implementing the components was changed where by pilot and demonstration components were implemented without distinction because the conditions for moving to demonstration from pilot could not be met in one year as originally envisaged.

The project components were consistent with the project objectives set out above. DHSP was however, a very complex project considering the number of reforms it was expected to undertake and the decentralized framework in which it was implemented.

- 52 -

1.5 Original and revised Project Cost

The total project costs as estimated and planned at project inception amounted to US. $75.1 Million. Only USD 66 Million was however realized because DANIDA (Denmark) and ODA (United Kingdom) who were originally to co –fund the project, changed plans and channeled their funding to the health sector through other arrangements. Government counterpart funding was later revised to 5% due to cash flow problems. Because of this shortfall several initiatives such as support to NGOs were scaled down.

1.6 Quality at Entry

The quality at entry is considered marginally satisfactory for the following reasons:

The project was consistent with government priorities for the health sector as reflected in the 1993-96 lhealth policy and plan.The project benefited from a baseline study.lThe reform element of the project was in line with other on going government reforms such as ldecentralization and civil service reform.DHSP inherited some officers, personnel, and logistics from the First Health Project that ensured a lquick project take off. Project implementation manuals were however, not put in place by project effectiveness and this lnegatively affected the take off of the project.Lessons learned from previous projects particularly the First Health Project, were taken into account in lthe design of DHSP but not to a satisfactory level. The project was for example, complex contrary to lessons and recommendations from FHP.

2. ACHIEVEMENT OF OBJECTIVES AND OUTPUTS

The project made a significant contribution to the revitalisation of the health sector. It introduced the concept of the Essential Health Package (EHP) which is now the basis for planning and resource allocation. There is now, as a result, more rational use of resources. The project also provided the financial resources for delivering the EHP to district populations in the 13 and later 16 pilot districts. Support services such as community mobilisation and IEC necessary for delivering the EHP were also supported.

In terms of capacity building, the project supported human resource and institutional development, infrstustrctural and systems development and provision of logistics at the district and central levels. The public health sector is now one of the most vibrant government institutions. DHSP also made a significant contribution to the institutionalisation of the collaboration between public and private sectors in health care delivery through support to NGOs.

The Project’s institutional development impact is high. DHSP has enhanced the capacity of the health sector to use human and material resources for improved health care delivery.

The sustainability of the project is rated as likely because many of the reforms and initiatives started or supported under the project are now part of the HSSP and have already been integrated in the regular operations of the MOH through budget support and SWAp arrangements.

Overall the project achievement is rated as satisfactory

- 53 -

3. BANK, BORROWER AND PARTNER PERFORMANCE

Bank

3.1 Lending

The Bank performance was satisfactory at identification and preparation phases. At the preparation phase, the preparation team clearly identified the critical heath sector issues, which were consistent with Government of Uganda's priorities in addressing heath care as reflected in the 1993-1996 Health Policy and Plan and the Health Sector White Paper. The Bank also built partnership with other development partners in supporting heath sector development some of whom funded the pre-investment studies and later on agreed to co-finance the project.

During preparation, the Bank fielded a team with varied expertise. Technically, the project was well prepared, and provided sufficiently detailed analysis and review of heath and other related issues in the country. The preparation benefited from several reports and experiences from government and other donor projects including previous IDA funded projects such as the First Health Project, the changing structure of government arising from decentralization, the role of NGOs and the linkages with existing donor programs support were reviewed and incorporated during preparation.

At appraisal, the Bank's performance was satisfactory. The Bank's appraisal mission included a team with a comprehensive skills mix. At appraisal project institutional arrangements both at the center and districts, phasing of project implementation, the role of the NGOs, linkages with existing heath sector programs supported by other development partners were further reviewed and agreed with government. These, particularly NGO participation were later to prove useful during project implementation. Key relevant safeguard policies were discussed and agreed with government including necessary actions government needed to undertake to ensure successful project implementation.

3.2 Supervision

The Bank's performance was satisfactory as the Bank met its formal supervision requirement of twice a year during project implementation and was able to develop good rapport and working relations with the government team. This ensured timely and adequate follow up of agreed actions and early resolution of emerging issues. DHSP was a complex project whose successful implementation would not been possible without a very supportive IDA task team. In the initial years, supervision missions involving comprehensive teams took on average two weeks. This ensured that adequate support was given to the PCO and other implementers and emerging problems resolved promptly. The duration and composition of supervision missions was to reduce markedly in last two years due to the recruitment of a health specialist in the country office who ensured continuous support to the PCO. Several of the programmes supported by the project became part of the joint GOU-Donor Joint reviews of the health sector towards the end of the project.

3.3 Overall Bank performanceThe Bank's overall performance was satisfactory. During identification, preparation, and appraisal the Bank fielded comprehensive teams with appropriate skill mix and incorporated other development partners. Supervision teams included the required expertise and local bank staff who continuously monitored project activities. These ensured good working relations between the government and Bank staff.

- 54 -

Borrower

3.4 Preparation

The Borrower's performance in lending is rated as satisfactory. During preparation the Borrower provided the necessary leadership and support to the Bank that facilitated project development. The availability of the health policy and plan, 1993-1996 and the White Paper on Health developed with active participation of stakeholders were especially beneficial. The Borrower facilitated wide participation of stakeholders in preparation including the development partners. Because of these government was able to raise USD 17.4M from some of the donors to co- finance the project.

3.5 Government Implementation performance

The government's overall performance is rated as satisfactory. This is based on the fact that government fulfilled most of the key conditions of the credit.

3.6 Implementing Agency:

The performance of the Implementing Agency- Ministry of Health is rated as satisfactory. The competent leadership from the MoH boosted implementation of project activities including compilation and preparation of timely and comprehensive reports. While there were substantial delays with procurement and disbursement in the early stages of the project implementation, once the PCO had mastered IDA procedures, these were largely resolved resulting in improved disbursement and project implementation. Despite the above delays, which were compounded by the closure of some banks where the project held accounts, the project closed as scheduled on 31st December 2002. There are adequate arrangements in place for sustainability and smooth transition to SWAP/budget support arrangements through which the Health Sector Strategic Plan is being implemented.

3.7 Overall Borrower performance:

Overall Borrower performance is assessed as satisfactory. Over the life of the project the government maintained its leadership and commitment to heath care delivery and overall health sector reform which greatly contributed to success of the project.

3.8 Partner Performance

The project was co-funded by the Government of Sweden through SIDA and the Government of the Federal Republic of Germany through KfW.

i) SIDA: The performance of SIDA is rated as very satisfactory. SIDA provided three technical advisers who were very instrumental in the implementation of the project and overall reform of the health sector. SIDA promptly responded to issues raised.

ii) KfW: The performance of KfW is also rated as very satisfactory. KfW provided technical support and was prompt in releasing funds and responding to issues raised. No KfW funds were cancelled because the KfW agreement expired only after all the KfW funds had been utilized.

- 55 -

4. Key Lessons Learned

i) In the design of a project, it is vital to assess the capacity of other support systems even when they are under other sectors. For DHSP, it was envisaged that the Ministries of Local Government and Finance had or would build the financial management capacity of districts. This was not the case and DHSP had to embark on the training of local government accounts staff whose role was vital in project implementation.

ii) A public sector project that has a major focus on reforms is very sensitive to the political environment. An assessment of the political climate is vital in the design and implementation of such a project like DHSP.

iii) A decentralized project requires more flexibility in design because of the dynamics of the local government environment. The size of the special account for example, should be more carefully determined to take into account the longer replenishment cycle associated with a decentralized project.

iv) Effective donor coordination is vital for the improved performance of the health sector. During the life of DHSP, districts were burdened by multiple planning and reporting arrangements of different un coordinated donors.

vi) Considerable time is required for lessons to be learnt from a pilot on the basis of which a demonstration is made. This was not considered in the design of DHSP where pilot and demonstration components were distinct.

vii) Many of the factors that determine health outcomes are outside the control of the health sector. While DHSP provided resources for delivery of EHP, it had no control over the recruitment and payment of health workers or ability of the population to pay for heath services. These factors are critical to health care delivery.

viii) For capacity building in health to lead to improved heath care, it must be holistic and coordinated. DHSP had little to do regarding an important aspect of capacity building for improved health care namely, recruitment and motivation of health workers. Laboratory equipment has been provided to some health facilities without laboratory personnel.

ix) The monitoring and evaluation strategy including monitoring indicators needs to be agreed well in advance. The monitoring of the project was negatively affected by the failure, due partly to lack conceptual concurrence, in resolving this vital issue.

x) The Civil Society is a viable partner with government in the delivery of health services. Effective and resourceful Civil Society Organizations are however few. To reap the benefits of working with Civil Society Organizations, there is need to relax regulations for example, in procurement. Under DHSP, NGOs found it difficult to procure through the Local Government Tender Board.

xi) Vertical programs need to be minimized or eliminated completely if a decentralized health care delivery system is to function effectively. Vertical programs affected implementation of the project, which in accordance with the policy of decentralization, supported districts to develop, approve and implement their integrated workplans.

- 56 -

- 57 -