world bank document - documents & reports · for official use only po = operational guidelines...

Document of

The World Bank

FOR OFFICIAL USiE ONLY

Rept No. 13745

PROJECT COMPLETION REPORT

INDONESIA

SECOND HEALTH (MANPOWER DEVELOPMENT) PROJECT(LOAN 2542-IND)

DECEMBER 2, 1994

Population and Human Resurces Operations DivisionCountry Department IIIEast Asia and Pacific Regional Office

This document has a restricted distribution and may be used by recipients only in the performance oftheir official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

Currency Unit = Indonesian Rupiah (Rp)(as of November, 1993)

US$1.00 = Rp 2,020Rp 1 million = US$495

(at appraisal)US$1.00 = Rp 1,100RP 1 million = US$909

FISCAL YEAR

April 1 - March 31

ABBREVIATIONS AND ACRONYMS

ADB = Asian Development BankAKPER - Nursing AcademyBAPPENAS = National Economic Development Planning AgencyBAPPEDA Provincial Economic Development Planning AgencyBLKM = Balai Latihan Kesehatan MasyarakatDEPKES = Ministry of Health (Departemen Kesehatan)DIK - Annual Routine BudgetDIP = Annual Development Budget (Daftar Isian Proyek)DUP = Annual Development Projects (preliminary)GOI = Government of IndonesiaIBI = Indonesian Midwives AssociationICB - International Competitive BiddingIDI = Indonesian Medical AssociationIEC = Information Education and CommunicationINPRES - Instruksi PresidenLCB = Local Competitive BiddingINSTALUED = Public Works UnitKLKM - Kursus Latihan Kesehatan MasyarakatLAN - Institute of Public AdministrationMCH - Maternal and Child HealthMOH = Ministry of HealthNGO - Non-Governmental OrganizationOED = Operations Evaluation DepartmentPCR - Project Completion ReportPEKARYA KESEHATAN = Nursing AssistantPIMPRO - Provincial Project OfficerPKMD = Village Community Health Developmetn Program

FOR OFFICIAL USE ONLY

PO = Operational GuidelinesPOSYANDU - Health and Family Planning Village GatheringPPAR = Project Performance Audit ReportPUSDIKLAT - Center for Education of Health ManpowerPUSDIKNAKES - Center for Education and Training of Health PersonnelPUSKESMAS = Health CenterREPELITA IV - Fourth National Five-year Development PlanSKN - National Health SystemSPK - Nursing SchoolTBA = Traditional Birth AttendantUNDP - United Nations Development ProgramUSAID - U.S. Agency for International DevelopmentWHO - World Health Organization

Tlis document has a restricted distribution and may be used by recipients only in the performance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization.

FOR OMCAL USE ONLYTHE WORLD BANK

Washinghon, D.C. 20433U.S.A.

Office of Director-GenwalOperations Evaluation

December 2, 1994

MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT

SUBJECT: Project Completion Report on IndonesiaSecond Health (Manpower Development) Project MLoan 2542-INID

Attached is the Project Completion Report on Indonesia-Second Health(Manpower Development) Project (Loan 2542-IND) prepared by the East Asia and PacificRegional Office. Part II was prepared by the Borrower.

This project was designed to assist the Ministry of Health increase the numberand quality of health care providers by expanding and improving both pre-service and in-service training. For this purpose the project provided funding for buildings, equipment,studies and technical assistance.

Shortages of counterpart funds, among other things, resulted in substantialdelays in implementation and the project closed two years later than originally planned. Theproject was restructured to reduce its size and to focus less on civil works and more ontraining and institutional development. In the end two-thirds of the civil works in the originalproject was satisfactorily completed and the impact on quantity and quality of training wasminimaL The project outcome is rated as marginally unsatisfactory and institutionaldevelopment as negligible. However, the modest benefits of this project are likely to besustained over time because of government commitment to fund health operations andmaintenance. Subsequent projects have abandoned this essentially supply-side approach totraining.

The Project Completion Report is of good quality. No audit is planned.

(1SAttachment

This document has a reeoed diibuton and may be usd by recipent only In th pedrrmanoe of thRobcialdutles. t oontnwt may not oewlwe be dieoceed wWto World Bank aufhorizaton.

FOR OFFICIAL USE ONLY

INDONESIA



TABLE OF CONTENTSPage No.

Preface ........................................ i

Evaluation Summary ................................ ii

PART I: PROJECT REVIEW FROM IBRD PERSPECTIVE ... .... 1Project Identity ................................ 1Background .................................. 1Project Objectives and Organization ................... 3Project Design and Organization ..................... 6Project Implementation ........................... 9Project Results ................................ 18Sustainability ................................. 22Bank Performance .............................. 22Borrower's Performance .......................... 23Lessons Learned ............................... 24

PART 11: PROJECT REVIEW FROM BORROWER'S PERSPECTIVE 26

PART III: PROJECT PROFILE AT COMPLETION .... ........ 29

Related Bank Loans ............................. 29Project Timetable ............................... 31Cumulative Estimated and Actual Disbursement .... ........ 31Project Implementation ........................... 32Project Costs and Financing ........................ 35Studies ..................................... 37Status of Covenants ............................. 38Use of Bank Resources ........................... 39

This document has a restricted distribution and may be used by recipients only in the performance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization. l

- j -

INDONESIA



PREFACE

This is the Project Completion Report (PCR) for the Second (ManpowerDevelopment) Health Project for which a loan of US$39 million to the Governmentof the Republic of Indonesia (GOI) was approved on April 24, 1985. The loan becameeffective in January 1986 with a closing date of September 1989. The loan wasrestructured to redirect the emphasis from civil works to improving manpower qualityin May 1988 and again in October 1990, at which time US$700,000 was cancelled.The project was closed in March, 1993, after four extensions. About 97 percent ofthe amended loan amount was disbursed.

The Preface, Evaluation Summary, Part I and Part III of this PCR wereprepared by Susan Stout (EA3PH), Dr. E. Iswandi (RSI) and Ms. Karima Saleh(Consultant). Part II was prepared by GOI. The PCR was prepared in accordancewith the Guidelines for Preparing PCRs, issued in June, 1989. The report is based ondata provided by GOI, a review of World Bank records, evaluation studies, and fieldvisits in Indonesia in 1992 and 1993.

- ii -

INDONESIA



EVALUATION SUMMARY

Project Objectives

i. This project was appraised in late 1984. The project was prepared andappraised as one of three investment loans designed to improve Indonesia's health andpopulation status. At the time, Indonesia's health status, while showing signs ofimprovement, was less than satisfactory and inferior to that of comparable lowerincome countries. Although life expectancy was improving, infant mortality and childdeath rates were unacceptably high at about 102 and 13 per thousand live birthsrespectively. Fertility was showing significant declines, but there were clear needs tofurther strengthen the provision of family planning and basic health services.

ii. Government's response to these conditions centered on the establishmentof an extensive public health service delivery system focused on a network of healthcenters, health subcenters and district level hospitals. A vigorous family planningprogram, managed by the National Family Planning Coordination Board,complemented this system, and made additional demands for the availability ofappropriately skilled health manpower at all levels of the health services network. TheFourth Population Project (SAR No. 5454-IND, PCR No. 9309-IND) addressed theinstitutional development needs of the national family planning program. The SecondNutrition and Community Health Project (SAR No. 5840-IND, PCR No. 11997-IND)was designed to develop and strengthen functional aspects of the health service deliverysystem through the promotion of outreach systems and improvements in nutritionpolicy and programming. This project was designed to complement these effortsthrough assisting the Ministry of Health to meet perceived needs for more and bettertrained health providers, particularly paramedical staff and nurses, to deliver improvedbasic health and family planning services.

iii. The project had two main operational objectives: First the projectintended to expand the output and improve the quality of pre-sernce training forparamedical staff. This was to be achieved through (i) strengthening the Center forEducation of Health Manpower (PUSDIKNAKES) within the Ministry of Health,(ii) the provision of new and/or improved physical facilities and equipment for 32schools and academies in 18 provinces, and (iii) three policy studies required to furtherdevelop and improve health manpower policies. Second, the project would expandthe capacity for and improve the quality of in-service training through(iv) strengthening the Center for Education Training of Health Personnel

- iii -

(PUSDIKLAT), and (v) the provision of new and/or improved physical facilities foreight in-service training centers to include four existing national centers, two existingregional centers, and two new provincial level centers.

Implementation Experience

iv. The project was approved by the Board in June 1985 and becarne effectivein January 1986 with a closing date of September 1989. Some implementationproblems emerged early in the project period. For example, the project encounteredbudget constraints, in the form of limited counterpart funds for the civil worksprogram, 40 percent of which was to be financed by the Government. Despite effortsto restructure the project in late 1987 implementation remained difficult and faceddelays throughout the life of the project, largely due to problems with the design ofthe management structure for the project, significant shortfalls in the availability ofcounterpart funds at critical junctures in the project plan and delays in the recruitmentand placement of technical assistance for the project. Achievements during the firsttwo years of implementation were below expectations, and even with a sharplycurtailed civil works program, cumulative disbursements totalled only US$470,000 inMarch, 1987 and US$3.49 million in March, 1988.

V. By late 1987, both Government and the Bank agreed that the project wasin need of significant adjustment. Following a major review, the loan was restructuredin May 1988 to redirect the emphasis from civil works to improving the quality of preand in-service training. Major adjustments included reductions in the civil worksprogram, the allocation of resources to the provision of books, teaching equipment andteaching materials and definition of an integrated package of technical assistancedirected at strengthening the two key project institutions. At about the same time,adequate counterpart funding was made available and cumulative disbursements roseto a total of about US$9.5 million at the end of March 1989.

vi. The thrust of the project during its first two years was primarily on civilworks. As a result, the studies on health manpower policies were delayed and couldnot significantly influence policy on health manpower. Implementation of technicalassistance and the fellowship program were also delayed. Despite the restructuringeffort and a significant reduction in the civil works program, the project, in retrospect,never attained its potential to influence human resource policies in the health sector.

vii. Responsibility for the implementation of the project was based with theMinistry of Health units responsible for pre-service (PUSDIKNAKES) and in-servicetraining (PUSDIKLAT) respectively. These units were established in conjunction withthe preparation of the project, and formalized in a decree issued in March, 1984.Formally directed by the Secretary General of the MOH, the project was initiallymanaged by two Project Offices based at Pusdiklat and Pusdiknakes. Coordinationamong the two agencies was less than satisfactory, and in conjunction with shortagesof counterpart funds, resulted in slow progress until a Project Secretariat, reporting toboth agencies, was established to facilitate implementation in late-1988.

- Iv -

viii. Disbursements and results were disappointing during the first years of theproject. By March 1988, US$33.5 million (86 percent) of the original US$39 millionwas undisbursed and 72 percent of the loan remained undisbursed in March 1989.However, following adjustments in implementation arrangements in 1988 whichreduced the project's physical objectives, there were significant improvements indisbursements. By October 1992 nearly $34 million had been disbursed. Fourextensions of the project were used to provide time for the completion of the reducedconstruction plans for the project, and to complete the provision of equipment for theschools that were established. In addition, during 1988-1992, 116 short-term and 22long-term overseas fellowships to train selected staff of Pusdiknakes, Pusdiklat andregional training institutions were completed.

ix. Utilization of technical assistance and consultant services concentrated onthe use of national consultants to assist in the design and supervision of civil works inthe first three years of the project. The restructuring of the loan in late 1987 includedspecification of an integrated package of technical assistance to assist both Pusdiknakesand Pusdiklat in aspects of curricular reform and in strengthening the management ofhealth training and fellowships. Following a lengthy recruitment and selection process,the integrated package was started in March 1990. Although this technical assistanceprovided useful outputs, it is not clear that it had any lasting influence on healthmanpower policy.

Project Results

X. Despite initial problems in implementation, the project eventually achievedtwo-thirds of its physical targets. Twenty-two of 32 schools or academies were builtand 10 in-service training centers were constructed or rehabilitated. Five out of thenine originally targeted nursing schools were rehabilitated and one new nursing schoolwas constructed.

xi. The Project's impact on institutional and policy issues was limited.Although two of the three policy studies were eventually completed, they appear tohave had little impact on health manpower training policy. Recognizing this in 1990,GOI and the Bank agreed to postpone consideration of the several institutional issuesconcerning responsibility for manpower training and linkage with broader aspects ofhealth sector development to yet to be designed subsequent projects.

Project Sustainability

xii. The project produced limited sustainable achievements. Improvements inthe macroeconomic environment and in the allocation of resources to health suggestthat the Government will not have difficulty in sustaining the recurrent costs of theschools, academies and in-service training centers established through the project. Butthe project's failure to significantly alter the institutional arrangements which governhealth manpower policy making, or move beyond identification of some of theconstraints to improved management of training institutions limits its effectiveness.It is perhaps appropriate that the Bank and the Borrower decided, during the last stages

v

of project implementation, to defer efforts to address broader issues of how best todevelop the health workforce to later discussions.

Lessons/Conclusions

xiii. Design Requires Substantial Sectoral Knowledge. In retrospect, itseems that the fundamental problems of the project grew out of the limits of the Bank'sunderstanding of the health sector in general and health manpower issues in particularat the time of appraisal. The design of the project, while consistent with healthplanning and project design principles current at the time, showed little recognition ofthe organizational and institutional setting in which the project would be implemented.The project's dominant concern with improving production capacity also reflectedlimited understanding of the performance of the health facilities that were to be staffedby the products of this investment.

xiv. Need for Analysis of Affordability. In a closely related vein, the projectwas appraised with very little attention to the capacity of the Government to providethe counterpart funds necessary to assure timely implementation. Although externaleconomic shocks, related to the decline in oil prices, could not have been anticipatedby the appraisal team, a clearer understanding of levels and trends in healthexpenditures in the public sector may have helped to avoid the devastating impact thatthe drop in counterpart funds had on project implementation.

xv. Assigning responsibility for project management to the targets ofinstitutional development risks delay in the use of project resources, and canoverwhelm the target agency. In this case, it seems clear that the both Pusdiklat andPusdiknakes, neither of which existed prior to the project, needed to concentrate theirinstitutional and managerial resources on developing their basic organizationalcapacities. Moreover, as new institutions, it would appear likely that both would havebenefited from more experience in the management of external finance beforebecoming responsible for a project of this dimension.

xvi. Early Identification of TORS and Selection Procedures for TechnicalAssistance. This project shows that unless the objectives of technical assistance arecarefully specified, lengthy delays can ensue in securing these resources. In this case,the appraisal report was largely silent on the purposes of the technical assistance to becontracted through the project, referring generally to expectations that it would assistin organizational development and improved management and planning. Once theconstruction program was back on track, however, the work of agreeing on morespecific terms of reference and securing qualified assistance to fulfill them wasundertaken with relative success. It appears likely that greater attention to planningthese inputs prior to appraisal would have improved the overall perfornance of theproject.

xvii. Need for an Evaluative Framework to Assess Project Progress. Last,the absence of an evaluative framework, integral to the project appraisal, appears tohave constrained project performance. The most glaring example of this problem in

- vi -

the present case was the absence of a discussion in the appraisal of what specificimprovements were expected in the project's general objectives ("expanding the outputand improve the quality of manpower"). In the absence of this specification, littleattention was put on a review of whether the project objectives were appropriate tohealth sector needs which reinforced the supply-side bias of the orientation of theunderlying approach to health manpower development. The appraisal also lacked aformat, in either the SAR or its working papers, which would assist Bank andBorrower management to assess progress toward the project's institutional goals.

INDONESIA



PART I: PROJECr REVIEW AND BANK'S PERSPECTIVE

1. Project Identity

Project Name : Second Health (Manpower Development)Loan No. Loan 2542-INDLoan Amount : US$ 39 million equivalentProject Period :1985 - 1989, extended to March 1993RVP Unit : East Asia and Pacific RegionCountry : IndonesiaSector : Population and Human ResourcesSubsector : Population, Health and Nutrition

2. Background

2.1 In the early and mid-1980s, Indonesia's health status was, althoughimproving, less than satisfactory and lower than that found in neighboring countries.Epidemiological conditions were largely unchanged from the mid-1970s, which buttressedGOI's growing interest in improving the performance of its public health service deliverysystem. The most prevalent causes of mortality and morbidity continued to includeinfectious, parasitic and gastrointestinal diseases that are preventable or treatable throughthe provision of basic health services.

2.2 In 1982, the Ministry of Health finalized a policy document, the NationalHealth System (SKN), which described the broad foundations for the long termdevelopment of the Indonesia's health system. This document provided a planningframework for the Fourth National Five-Year Development Plan, 1984-89 (Repelita IV)which further articulated GOI's policy goals in the health sector. Repelita IV set out anambitious agenda, aiming at achieving an increase in life expectancy to 60 years; areduction in infant mortality to 45 per thousand live births; a 50 percent reduction in theincidence of communicable diseases; and a 67 percent reduction in protein-caloriemalnutrition in children under five. Repelita IV identified five key principles as the basisfor determining health sector policies, programs and activities: (a) strengthening healthservice delivery, including measures to support and expand primary health care activitiesat the community/village level; (b) strengthening health manpower development;

- 2 -

(c) expanding activities to improve nutrition, potable water supply, and environmentalhealth; (d) establishing programs to strengthen the overall management of the system; and(e) improving the supply, distribution and quality of drugs, medicine and medicalequipment. The high priority placed on strengthening health manpower in this documentreflected a strong conviction that there was a risk that demand for health personnel,especially nurses, midwives and paramedics to staff the rapidly growing public healthinfrastructure, would exceed supply.

2.3 The public health delivery system, a network of health centers (puskesmas)and subcenters (puskesmas pembantu) evolved rapidly during the second and third fiveyear plans. The speed of growth created a concern that demand for manpower to staff thenetwork of facilities was outstripping supply. Moreover, there was widespread concernthat the quality of available paramedic manpower was low, due to the absence of asystematic in-service education and training system, poor management and technicalsupervision, low salaries and limited prospects for career development, particularly fornurses and other paramedics. During Repelita III, the Government took the first stepstoward addressing these problems through conducting a large scale inventory of availablehealth manpower. This was followed, in late 1981, by a joint USAID, WHO and Bankmission that assessed short and long term health manpower needs.

2.4 The principal short term recommendations of this mission were:

* the output of community health and hospital nurses should be increasedthrough the establishment of additional nursing schools, or Sekolah PerawatKesehatan (SPK) and for higher level nurses, Academies (Akper), inprovinces with the greatest need.

* the output of technical paramedics (dental nurses, laboratory assistants,pharmacists, radiologists, physiotherapists, anesthesiologists, medicalanalysts and health inspectors/sanitarians) should also be increased throughthe establishment of additional schools (public and private),

* the quality of all categories of health manpower (graduate and paramedic)should be improved through the establishment of additional in-servicetraining centers; and

* the management of health facilities, specifically hospitals, and thesupervision of health manpower in hospitals and health centers, should bestrengthened.

2.5 In addition, the mission recommended a number of longer-term policymeasures, including the establishment of a single, national health manpower data base andinformation system, improved policies for career development, compensation andcontinuing education for all categories of health manpower, the improvement ofcoordination between public and private sector training institutions to promote efficiency,particularly in underserved provinces, and the redefinition of central and provincial rolesto promote greater responsibility for manpower planning at the provincial level.

- 3 -

2.6 This project thus evolved out of an overriding concern with the supply ofhealth services, and a shared conviction between the Bank and the Borrower, that staffingthe evolving network of public health facilities should be the primary focus ofgovernmental intervention in planning the health workforce.

3. Project Objectives and Organization

3.1 Objectives. The project's overall goal was to support the MOH's long-termhealth manpower development plan, through (1) helping expand the output and improvethe quality of MOH paramedical manpower 1/ through (a) strengthening the Center forEducation of Health Manpower (PUSDIKNAKES) by local and overseas training and localand foreign and technical assistance; (b) providing new and/or improved physical facilitiesfor 32 schools and academies in 18 provinces; and (c) carrying out three studies requiredto further develop manpower development policies for Repelita V and beyond; and (2) helpexpand the capacity for and improve the quality of MOH's in-service education andtraining through (i) strengthening the Center for Education Training of Health Personnel(PUSDIKLAT) through local and overseas training and local and overseas technicalassistance; and (ii) providing new and/or improved physical facilities for eight in-servicetraining centers. These objectives would be accomplished through the following projectcomponents:

3.2 Project Description. The following activities were to be undertaken from1985- 1989.

Pre-Service Paramedical Health Manpower Development(PUSDIKNAKES) (US$ 40.4 million of which $14.6 million foreign)

3.3 Institutional Strengthening (US$4 million, of which US$3 million isforeign). This component of the project aimed to strengthen the Center for Education ofHealth Manpower which was formed just prior to the start up of the project. Three majorforms of institutional strengthening were planned:

(a) Systems Support. Decisions on the senior staffing of PUSDIKNAKESwere completed just prior to project startup and plans were established todevelop a work-plan specifying priority operational activities to be carriedout during Repelita IV. To assist in this process, the project planned toprovide 192 manmonths (168 foreign and 24 local) of technical assistanceservices for organizational development, management and planning,curriculum development, the development or improvement of systems for

1/ The term paramedical was used throughout the Staff Appraisal Report to encompass basic nurses,nursing assistants (pekarya kesehatan), midwives and a wide range of allied staff such as laboratoryand x-ray technicians. More recent sector work suggests that there are major distinctions in theformal and informal roles among these different categories, as well as important differences in thefactors influencing their demand and supply. This report uses the language of the original SARfor consistency's sake. However, one of the lessons learned from the project was that this usageobscures important health policy questions.

- 4 -

technical supervision, monitoring, evaluation and support of educationalprograms and for manpower data collection and processing. A total of 35fellowships-6 long-term and 29 short-term-were to be provided toPusdiknakes staff. Funds were also provided for group educationalactivities (workshops and seminars), to promote dialogue between provincialofficials and Pusdiknakes staff.

(b) Strengthening Provincial Bodies. To support MOH's long-term strategicfocus on transferring technical and institutional support for health manpowerdevelopment to the provincial level, the project planned to create educatonand training divisions in five provinces; East, West and Central Java,North Sumatra and South Sulawesi. The project provided for 10 short-termfellowships (2 per province) as well as workshops and seminars to supportthe establishment of skills in manpower planning. This activity became acovenant of the loan, and was to be achieved by March 31, 1986.

(c) Centers of Excellence. A related initiative called for the selection of 10schools and/or academies, representing each of the technical sub-disciplinesor categories of paramedical manpower as centers of excellence. The long-term goal of this initiative was to strengthen the capability of these centersso that they could provide technical and institutional support to otherschools working in the same discipline. The technical knowledge and skillsof the faculty of these 10 schools were to be strengthened through theprovision of 36 long-term fellowships, workshops and seminars. Selectionof the 10 centers of excellence became a covenant of the loan, and was tobe achieved by December 31, 1985.

3.4 Paramedical Schools and Academies (US$ 36.2 million of which $11.4million foreign)

(a) Increased capacity: The MOH planned to establish 48 new schools oracademies and to rehabilitate 42 existing schools or academies duringRepelita IV. The project provided support for the construction orrehabilitation of a total of 33 institutions within this overall plan, selectedon the basis of: (i) manpower categories in greatest need; (ii) overall healthstatus of the province; (iii) geographic location of the school/academy; and(iv) site acquisition and land availability. The project planned to providefunds for physical development (civil works, equipment, furniture, vehicles(2 per school) and books and teaching materials for 21 schools (6 new and15 existing) and 11 academies (7 new and 4 existing). Of the 11academies, 7 were to be developed as multistream academies, where groupsof two or three different subcategories of personnel would be placed on thesame site and would share common dormitory, dining, library andauditorium space. Ten of the total 32 facilities were to be nursing schoolsor academies, and the remaining 22 facilities were designated for trainingassistant phartnacists, medical analysts, laboratory assistants,sanitation/health inspectors and dental assistants.

- 5 -

(b) Support for Multistreaming: Since the concept of multistreaming forparamedical training was new to Indonesia, the project included threecovenants intended to facilitate the introduction of this concept, agreed atnegotiations: (i) establishment of a common core curricula for the differentparamedical categories to be 'multistreamed';2/ (ii) introduction ofmultistream classes and prograns on a pilot basis in the seven multistreamacademies; and (iii) establishment of a policy for future multistreamparamedical education by April 1, 1988.

3.5 Studies (US$ 0.2 million of which all is foreign). The project also providedfor short-term international technical assistance to assist in the conduct of three studieswhich were to contribute to the development of health manpower policy:

(a) Financial feasibility of the long-term health manpower development plan'stargets for the year 2000. This was to be completed by June 30, 1986.

(b) Examination of the new role, functions, and status of nurse assistantsor "Pekarya kesehatan". The future role, functions and careerdevelopment prospects for this category of paramedic was to be reviewed,and plans for their longer-term utilization, career development and trainingagreed. The review was to be completed by December 31, 1986, and aplan for utilization and career development completed by April 1, 1988.

(c) The new strategy of short-term training for health manpower through"parallel" classes (double shifts) would be evaluated to assess its impact onefficiency and training quality by December 31, 1986. Policy on thecontinuation or otherwise of parallel classes was to be completed by April1, 1988.

In-Service Health Personnel Development (PUSDIKLAT)(US$ 12.9 million of which $4.2 million foreign)

3.6 Institutional Strengthening. This part of the project aimed to strengthenthe staff, function and effectiveness of the Center for Education and Training of HealthPersonnel (Pusdiklat). Established in 1984, Pusdiklat was assigned responsibility for thein-service training for the entire public health system, not only paramedicals, through thedevelopment and coordination of programs for a set of national level in-service trainingcenters (Balai Latihan Kesehatan Masyarakat, BLKM) and provincial in-service trainingcenters (Kursus Latihan Kesehatan Masyarakat, KLKM). In addition to its responsibilityto coordinate the work of these centers, Pusdkilat also conducted short in-service trainingcourses, since Pusdkilat took over responsibility for obligatory public administration andmanagement training for MOH echelon III and IV personnel from the National Instituteof Public Administration (LAN) in 1984. It was also responsible for training trainers for

2/ 'Multistreaming' intended to accomplish efficiencies in training expenditures by coordinatingparamedic training programs on a single campus and sharing equipment, materials and facilities.

- 6 -

the BLKM and KLKM and supported pre-employment training and in-service training onbehalf of the four functional Directorates General of MOH.

3.7 To strengthen Pusdiklat's capacity to perform these roles, the projectprovided two forms of training:

(a) Pusdiklat staff. Ten long-term fellowships (8 masters level in managementand planning and 2 doctoral level in research and development) and 11 shortterm fellowships;

(b) Training Center Staff. For staff of the in-service training centers, theproject provided for (i) experiential, field-based training in communityhealth program development for 84 selected national, regional andprovincial trainers; (ii) 20 masters-level fellowships in public healtheducation; and (iii) 18 manmonths of technical assistance to support thedevelopment and implementation of these programs.

3.8 In-Service Training Centers (US$10.8 million of which US$2.5 million isforeign). At the time of appraisal, Pusdiklat had seven existing training centers. Thesehad become severely deteriorated from their initial construction. They also had insufficientcapacity to manage the in-service training loads implicit in MOH's plans for the placementof new staff during Repelita IV. The project therefore provided funds for the physicaldevelopment (civil works, equipment, furniture, vehicles, books and teaching materials)of 10 in-service training centers, including the rehabilitation of 8 in-service trainingcenters, 4 at the national level, in Jakarta, Ciloto, Lemah Abang and Salaman, and for theconstruction of 2 new provincial level facilities in Bali and Aceh, the construction of a newregional center in South Sumatra (Palembang) and rehabilitation of a regional center inMurnajati.

3.9 The project was to be organized and managed by the two Chiefs of thesecenters, who, as Project Officers would report to the Secretary General of the MOH, theformal Project Director. To ensure intra and inter-ministerial coordination, a ProjectAdvisory Board, constituting the Project Officers, the MOH Adviser, Health Technologyand Manpower, the Chief, Bureau of Planning, the Chief, MOH Inspector General and theChief, Health and Nutrition of Bappenas was established. This Board was to meet at leastquarterly to review overall project progress, approve plans and budgets for all componentsand resolve any issues affecting project goals and implementation.

4. Project Design and Organization

4.1 Design. The conceptual foundation for the project was the Ministry ofHealth's long term health manpower plan, developed during the last years of Repelita III,and specific implementation estimates for this plan developed for Repelita IV. Thestrategic focus of this plan can best be characterized as a classically supply-side approachto health workforce planning. The primary focus of the long term plan, and hence theproject design, was on the production of medical and paramedical personnel necessary to

adequately staff the Ministry's rapidly growing network of hospitals, health centers andhealth subcenters. Conceptualization of staffing needs were based on prescriptive staffingnorms. These specified required numbers of staff by type and facility, based on normativeformulation of expected roles and functions. At the time, Indonesia's approach to thedefinition of these norms was broadly consistent with conventional tenets of primary healthcare delivery. The norms sought to maximize the provision of simple, curative care andpreventive activities through health centers and subcenters, which were assumed to beattractive to the consuming public. This approach assumes there is demand for basicpreventive and curative services, and that establishment of a facility would result ineffective utilization of services. In Indonesia, as elsewhere, health planning tended toignore the existence of traditional or privately provided health services, and did little toassess actual levels of demand or need for care.J/

4.2 Consistent with this approach, the health manpower plan, and the project,exhibited a predominant concern with improving the accessibility of services, and hencewith rapidly increasing the quantity of workers, with relatively less attention to improvingor understanding qualitative dimensions of their expected and actual performance. At thestart of Repelita IV, health planners were understandably most concerned to establish theproduction capacity necessary to assure a growing supply of paramedical workers for theanticipated growth in the public health delivery system.

4.3 In retrospect, it is easy to question the appropriateness of this approach.Experience since the design of the basic health service delivery structure in Indonesia hasraised a number of concerns with this overarching concern with supply of newly trainedstaff, relative to finding ways to tap already available resources in the form of private andtraditional sources of care. The 'supply-side' approach and its application to public healthplanning increasingly has been criticized from a number of perspectives. A commnontheme is that this approach fails to incorporate workable mechanisms for making supplysystems responsive to unpredictable dimensions of demand. These systems also share atendency to develop bureaucratic, top-down managerial structures which inhibit innovationand adaptation of resources to local needs.

4.4 In summary, the design of this project rested on what would now beconsidered an incomplete and possibly inappropriate conceptual base. However, it wouldbe unfair to assess the design of the project exclusively on this criterion, since theframework was typical of the times. Moreover, the project was designed in the contextof two additional Bank investments, both of which did focus on problems of improving theperformance of the programs (for family planning, in Population IV, and for communityhealth and nutrition prograrns, in Nutrition and Community Health II) which wereprepared, appraised and implemented conterminously with this project. Creating a

3/ This approach was followed in many settings, and was actively promoted, particularly following theHealth for All by the Year 2000 Conference in Alma Ata, by most international donors active inthe health sector. Although there were critiques of the approach, including its application inIndonesia (see Hull and Hull, 1978), its appeal was nearly universal, perhaps because of the relativeease of scaling up' to a comprehensive program design once agreement on the basic nornativepicture of how services should be delivered was developed.

- 8 -

"division of labor" between the structural, infrastructure focus of the Second HealthManpower project, and the functional, performance improvement goals of the latterprojects may have been a rational approach to assisting the MOH to improve its capacityand performance through a manageable set of discrete investments.

4.5 The design of the project is thus most appropriately assessed in a contextwhich accepts the basic supply-side approach to health manpower planning. When viewedin this way, the project design appears to be internally consistent, well-linked withGovernment plans and perspectives on the needs of the sector and largely appropriate.However, several design problems in the project are apparent, even within this context.

4.6 First, the project design seems, in retrospect to have inadequately linkedinputs with its expected outputs and objectives. Compounding this, the project lacked anevaluative framework which would assist the Borrower, or Bank supervision teams (whichchanged with some frequency) to assess progress and performance. The objectives of theproject were to "help expand the output and quality of MOH paramedical personnel"through rapidly increasing the numbers and physical capacity of both pre-service and in-service training systems. The design, as described in the SAR, focused particularly on theexplication of the numbers of schools to be built, and did not specify its expectations asconcerns numbers, expected skill levels or qualitative characteristics of the manpower tobe produced through the schools. To its credit, the design did provide for a majorprogram of fellowships and large technical assistance inputs as an instrument fordeveloping improved curricula and training quality, but these areas receive little explicitdefinition in the SAR beyond specification of the numbers of manmonths of technicalassistance and fellowships to be provided. The SAR included no reference to specificoutput goals, nor did it clearly state which of the several categories of paramedical staffcovered in the project scope were of greatest interest or concern to overall sectoralperformance. Specification of the institutional development goals of the project was alsoweak.

4.7 Second, the provisions for the "software" programs to enhance qualitativedimensions of paramedical training, primarily for fellowships and technical assistance, didnot receive adequate attention at the design stage. For example, although there is areference in the SAR to collaboration with WHO for the management of fellowships, thisarrangement was dropped during project negotiations, which led to delay anddisappointments with the quality and timeliness of the fellowship programs. The SAR alsoprovided only cursory guidance on the proposed content and purposes of the 234manmonths of technical assistance which were to be provided, leading to extended delaysin agreeing the terms of reference for particular technical assistance. Ultimately terms ofreference were only agreed during the first (of four) extension periods that were requiredto complete the implementation of the project.

4.8 Third, the project called for an aggressive and large scale constructionprogram to establish new and rehabilitate existing schools and academies. Theconstruction plan was justified in the context of the MOH's long term health manpowerplans, and calibrated to reflect the role of other donors, particularly the AsianDevelopment Bank who were also supporting the manpower plan. Nevertheless, there is

-9-

the SAR provides only a cursory assessment of the feasibility of implementing the civilworks program on time, and did not analyze the availability of GOI counterpart funds toprovide for the 40 percent of the total civil works budget estimated in the total projectcosts. Implementation arrangements for the management of the civil works componentrelied primarily on existing MOH arrangements, with some provision for localconsultancies to assist in design and supervision. In retrospect, the project design seemsto have underestimated the level of effort and overestimated the availability of counterpartfunds required to effectively deliver the civil works program on time.

4.9 Organization. The project was designed to be organized and managed bythe two agencies responsible for pre-service (Pusdiknakes) and in-service (Pusdiklat)training, which were established, through a break-up of the former, in March 1984.Staff from the participating Centers, including a finance officer, and equipment specialist,a construction adviser and an architect or engineer from (MOH's INSTALMED) wereassigned to be responsible for the project monitoring, reporting and coordination. Nofurther organizational mechanisms to create links between the pre-service and in-servicetraining centers, and the functional Directorate Generals of the Ministry, beyond thoseinherent in their broader organization mandate, were specified for the program. Thisreflects the project's predominant concern with the production capacity side of manpowerplanning, and relative lack of attention to assessment or reform of the technical and clinicalroles of particular categories of paramedical personnel.

4.10 A notable feature of the organizational structure, in retrospect, is the factthat the concerned Centers were at once the target of the proposed institutionalstrengthening efforts, and the responsible implementing unit. As new units with no priorexperience managing a Bank-financed investment, it is perhaps not surprising that projectmanagement was less than smooth in the early years of implementation. Despite repeatedrecommendations from Bank supervision missions, this design flaw was not addressed untilAugust 1989. Only at this point was a project management unit directly responsible to theSecretary General and authorized to coordinate project-related activities of both unitsestablished. While this organizational design assured that project supported activitieswould be intimately linked with the evolution of the MOH's institutional capacity as awhole, it also offered little assurance that the project would receive the full time attentionof the concerned Project Officers. Responsibility for implementation of the project at theprovincial level was assigned to the Kanwil, or provincial health officer, who wouldmanage the project in the context of his regular duties. In addition, a full time projectimplementation officer was to be appointed in participating provinces to support localprocurement, the preparation of work programs and budgets and to assist with projectmonitoring and supervision. During the project, it became clear that these arrangementswere overly optimistic about the feasibility of effectively adding management of a Bankproject on to full-time management positions.

- 10 -

5. Project Implementation

5.1 The project became effective on January 21, 1986 about five months afterthe planned date of effectiveness. The implementation of the project was not smooth andreflected chronic problems relating to constraints on the counterpart funds available for theproject. Following a Bank review of the project in November 1987, the loan agreementwas amended in May 1988 to reduce the overall civil works program for the project andrefocus resources on provision of equipment and materials for the completed construction,develop an integrated package of technical assistance aimed to assist both Pusdiknakes andPusdiklat and establish a Project Secretariat. One outcome of the review was a decisionto extend the loan closing date to September 1990. In October 1990, the legal agreementwas substantially revised to focus project resources on improving the quality of training,and to reduce the civil works program. A second extension of the loan closing date, of18 months, was approved at the same time, primarily to provide for the completion of theprocurement and installation of equipment for schools and academies. Two subsequentextensions of six months each were ultimately approved, in April and December 1992 andthe project was closed in March 1993.

5.2 The implementation history of the project can be seen as occurring in threephases. During the first, dating from the project launch, held in July 1985 to mid-1988,project authorities and Bank supervision teams concentrated on initiating the first phase ofthe construction program, which focused on the construction of 17 of the originallytargeted 32 buildings. Two key constraints to the smooth implementation of what had beenconsidered a reasonable construction plan emerged early in this phase. First, governmentalbudgeting procedures dictated the use of single year contracts for design and supervisionconsultancies and required that counterpart funds not spent in a particular fiscal year revertto the treasury. This made it difficult for project authorities to efficiently allocate availableresources and seriously complicated the design and ultimately the construction program.Second, external changes in the macroeconomic environment, most particularly a rapiddecline in oil revenues, resulted in severe cuts to the MOH's development and recurrentbudget allocations. These in turn severely constrained the availability of counterpart fundsfor the implementation of the civil works program. The delay in the construction programunderstandably occupied the lion's share of managerial attention, from both Bank andBorrower throughout this phase of the project. Although initial steps toward theimplementation of the technical assistance component of the project were undertaken, thepolicy studies included in the original design were slipped to allow for the completion ofthe construction program. The implementation delays during this period were reflectedin very slow disbursement of the proceeds of the loan: By July, 1987, eighteen monthsafter project effectiveness, less than US$2 million had been disbursed.

5.3 Progress on implementation continued to be slow during 1987. In additionto the problems outlined above, there were significant changes in project management atPusdiknakes, as well as a major change in the organization and composition of the Bankteam responsible for the supervision of the project as a consequence of the re-organization.Responding to the apparent implementation difficulties, the Bank proposed andsubsequently conducted a major review of the project in November, 1987, which initiatedthe second phase of the project. The primary outcome of this review was an agreement

- 11 -

to significantly restructure the project. In particular, it was agreed that the financialconstraints on the MOH's budget made it unlikely that the originally construction prograrncould be completed, and an agreement was reached to postpone the second phase ofconstruction (the remaining 16 facilities) to a possible future project. The mission alsorecommended major revisions in the legal agreements for the project, which resulted inan increase in the percentage of Bank financing for construction from 60 percent to76 percent, a detailed reconsideration of the detailed terms of reference and recruitmentprocedures for the international technical assistance for the project, and an agreement toincrease the project's allocation for the equipment and materials to sharpen the project'sfocus on improving the quality of training.

5.4 The amendments to the legal agreements which flowed out of the November,1987 review mission eventually resulted in an agreed revised loan agreement in May,1988, which marks the beginning of the third phase of the project. During this period,project management focused on the completion of the more limited construction program,the procurement, distribution and installation of the equipment for the constructed schoolsand academies, and, implementation of the re-designed technical assistance component ofthe project. A further loan amendment, in October 1990 recorded several minoradjustments among categories of expenditure, including an increase in the allocation forproject management (exclusive of salaries and allowances) to overcome continued problemswith the availability of government counterpart funds to support these works and cancelledUS$700,000 from the loan. In addition, recognizing that its purposes were now moot, thestudy of the use of parallel classes was dropped from the loan agreement.

5.5 Overall, three major areas created problems for the implementation of theproject: delays and difficulties in organizing and implementing civil works, difficulties,during all three phases of the project, in project management, managing the procurementof goods, and equipment for the new schools and academies, and uneven attention to andslow progress on the implementation of the limited 'software' goals incorporated in theoriginal project design.

5.6 The outcome of the lengthy implementation delays, two amendments andmultiple extensions was a completed project at considerable variance from the appraisedproject. Major variances were as follows:

Pusdiknakes

5.7 Civil Works. The original project planned for the construction of 32 newnursing schools and academies, and the construction of 8 national or regional in-servicetraining centers. In 1987, in response to the slow progress in construction, the plan wasreduced to a much smaller list, resulting in a total of 21 new nursing schools or academiesat the end of the project. The project did complete construction of nine in-service trainingcenters (see Table 5.1). The mix of new construction and rehabilitation efforts within theoverall program was also adjusted, focusing greater resources on rehabilitation of existingfacilities than anticipated at appraisal. However, equipment was provided to a total of 33schools or academies, largely in line with appraised estimates.

- 12 -

5.8 Equipment and Vehicles. The original loan agreement called for theprovision of equipment and supplies for both new and rehabilitated schools and academies,as well as supplies of books for the regional and national in-service training centers. Noaction was taken on the procurement of equipment or books and materials until well intothe fourth year of the project. Moreover, the 1987 supervision/restructuring mission foundthat the original project design had significantly underestimated equipment costs, and asfor civil works, had optimistically planned that GOI would be able to finance 60 percentof the equipment, which proved unrealistic due to the fall in oil prices and a fall in thedollar exchange rate against the Yen and ECU (likely sources of supply) since the date ofappraisal. Consequently, costs for equipment, furniture and, ultimately, vehicles were re-estimated and allocations from the loans for these items increased from US$2.5 million atappraisal to US$5.2 million in the first loan amendment and US$12.6 in the second.Although the procurement process was extremely protracted, equipment was ultimatelyprovided to 33 schools and academies. In addition, project resources were used to procure45 vehicles (1 for Pusdiknakes, 1 for Pusdiklat, 1 each for 33 schools/academies and 10for BLKM and KLKM) following the second amendment. No problems in theprocurement of these vehicles were reported.

5.9 Institutional Strengthening: Fellowships. Although eventuallyimplemented fellowships, and significant inputs of technical assistance, were heavilyconcentrated in the last two years of the project. Difficulties in recruiting students withsufficient English language skills early in the project resulted in a decision to reallocatefellowship resources toward a greater reliance on in-country training than anticipated atappraisal. In addition, where the appraisal anticipated the completion of 29 short-termoverseas training courses for Pusdiknakes staff, none were completed, 3 of 6 planned longterm overseas fellowships were completed. A major substantive focus (13 of 29 short termfellowships) of the fellowship plan at appraisal was for the provision of 13 short termfellowships in technical supervision of training programs and another in planning andmanagement (6 of 29). In the end, the project accomplished little of its objectives in thisarea, providing none of the short term training programs originally anticipated, and onlythree of the six long term fellowships (see Table 5.2).

5.10 Technical Assistance. The SAR did not provide detailed explication of theexpected roles and contributions of the technical assistance that was to be provided toPusdiknakes or Pusdilcat. Early in the project, a consultant was recruited and worked for18 months to assist in the development of approaches to strengthening field-based,experiential training methods for paramedic training. Although this led to increasedawareness of the need to strengthen the use of this type of training among schools and atPusdiklat, the project did not, judging from project records, pursue this theme followingthe completion of the consultancy. Following the 1987 review of the project, theimplementation of technical assistance focused on the development of an 'integratedpackage' of technical assistance which was designed to add an integrated program offoreign and local consultancy services. Detailed terms of reference for the integratedpackage were developed with significant assistance from Bank supervision missions. Aseries of problems, attributable largely to the fact that the package was being designed tofit the needs of two institutions (Pusdiknakes and Pusdiklat) as a whole and the lack ofexperience in competitive recruitment and selection of technical assistance (e.g., lengthy

- 13 -

time to develop a short list acceptable to the Bank, delays while contract terms and originalproposals were revised at Bank insistence) held back the establishment of the re-designedTA package. Ultimately,. however, the package was completed and awarded to theAssociation of Canadian Community Colleges and P.T. Inersia Ampak Engineers and workinitiated in March 1990. Once in place, the technical assistance teamn provided valuableadvice, focusing on (i) assistance in updating and revising 13 curricula for particularparamedic specialties, (ii) advice on planning and management issues to the centralagencies as well as the three selected centers, (iii) an evaluation of the multistreamconcept, and (iv) the design and implementation of a purpose-designed program to provideoverseas 48 short-term fellowships in a range of paramedic specialties.

5.11 Strengthening Provincial Bodies. The project's effort to strengthen healthmanpower training units at the provincial level in six provinces were implemented throughthe provision of short term fellowships, workshops and seminars. Two long termfellowships in public health management were completed, but there is little evidence thatrelationships between the central manpower agencies and provincial bodies weresignificantly altered.

5.12 Selected Centers. Ten schools and/or academies were identified to becomecenters of excellence in line with the appraised schedule in 1986. The centers wereidentified on the basis of their experience, quality of training, relative proximity to Jakartaand to represent the major types of paramedical manpower included in the project. Visitsto the 10 selected centers during the 1987 review of the project found that the quality andcapabilities of the selected centers were extremely mixed, and not sufficiently reflectiveof each center's potential to contribute technically or substantively to its sister institutions.Consequently, it was agreed that this effort would concentrate on three centers from theoriginal 10. These three institutions, AKPER/Bandung, AKPER/Jakarta and AKZI/Jakartawere encouraged to develop work programs that would more clearly than heretoforearticulate their proposed relationships with sister institutions, focusing on (i) staffdevelopment, (ii) introduction of new curricula and the preparation of new teachingmethods, and (iii) preparation of basic handbooks and written materials in BahasaIndonesia to assist in the transfer of new curricula and teaching methods. Progress on theimplementation of the resulting workprograms was evaluated, in 1989, to be encouraging,although communication among the centers was limited and there were concerns that theadditional workburden of being a center of excellence would displace normal teachingworkloads. The selected centers eventually provided a total over 3,000 short-termfellowships for various paramedic trainers from across the project.

5.13 Support for Multistreaming. Progress on the development and evaluationof the multistream concept was constrained by the major delay in the implementation ofthe civil works program, since the sites where the concept was to be tested were notfinished. Once completed, the multistream academy concept does not appear to have beeneffectively implemented, largely due to considerable confusion on the objectives of theconcept, particularly among school and academy directors and lack of detailed operationalguidance from Pusdiknakes on the intent and specific expectations of the concept. Thiswas reinforced by budgetary arrangements which allocate pre-service training resourcesby functional program, and a consequent reluctance to risk not being accountable for

- 14 -

program funds because of a co-mingling of training resources at the institutional level.Compounding this problem, the delay in addressing 'software' elements of the loan untilpost-1987 delayed work on the design or refinement of elements of the curricula forparticular paramedic specialists to define 'common ground' that could become the focusof a multistream concept. Since continuation of the multiple administrative and budgetarystructures was consistent with SAR, Government continued to be reluctant to move to anintegrated administrative structure and budget at the school level. A supervision missionin March 1988 concluded that effective solutions to these problems would need to betackled under a future project. A subsequent evaluation of the views of school andacademy Directors conducted in January 1991, concluded that the concept, while sound,needed significant additional work to become meaningful.

Pusdiklat

5.14 Civil Works. Implementation of the civil works program for national(BLKM) and regional (KLKM) in-service training centers was largely consistent with theproject design. By the completion of the project, five new regional in-service trainingcenters had been constructed, four new national centers had been constructed and thenational training center in Jakarta (associated with the Pusdiklat campus) was rehabilitated(Table 5.1 .b). The completion of these works suffered from the sarne sources of delay asexperienced in the construction program for Pusdiknakes.

5.15 Equipment, Furniture and Materials. Consistent with the original projectdesign, packages of basic equipment, furniture and books and materials were procured foreach of 10 centers. Major implementation problems included, as for Pusdiknakes,underestimation of the costs of these materials at appraisal, and limited provision of booksand training materials in Bahasa Indonesia. The integrated technical assistance packageincluded review and revision of the books and materials lists for these procurements andupdated estimates were integrated into the second loan amendment.

5.16 Institutional Strengthening: Fellowships. Although significantly delayedthe planned program of overseas and in-country fellowship training was largely completed,with significant inputs through the integrated technical assistance program. As detailed inTable 5.3b, the primary variance from appraisal estimates was on the completion of longterm overseas training programs. More significantly than the numbers however, thefellowship programs, did not occur until during the first extension of the project, whichconstrained their impact on the efficiency or effectiveness of project related activities.There has not been a formal evaluation of the impact of the fellowships program on theoperations or effectiveness of either BLKM or KLKM. However, issues concerning staffqualifications appear to be less significant factors in their performance than fragmentationof responsibility for determining content for in-service training among functional andprogram units of the Ministry, Pusdiklat and provincial authorities.

5.17 Institutional Strengthening: Technical Assistance. The integratedpackage of technical assistance which was eventually put in place in March, 1990 wasdesigned to focus on the same set of issues as for Pusdiknakes, and included theimplementation of the overseas short-term overseas fellowship training programs. Five

- 15 -

Table 5.1(a): CONSTRUCTION OF PRE-SERVICE TRAINNG PROGRAMS (SCHOOL AND

ACADEMY), ACCORDING TO APPRAISAL AND IMPLEMENTATION, BY PROVINCES

No. Province Institution Appraisal CompletedNew Rehab.

1. DI Aceh School 1 0 02. North Sumatra School 1 0 03. West Sumatra School 2 1 0

Academy 1 1 04. South Sumatra School 2 0 05. Jambi School 2 1 06. Riau School 1 1 07. DKI Jakarta School 2 1 1

Academy 1 2 18. West Java School 2 2 0

Academy 1 2 09. Central Java School 1 1 0

Academy 2 2 010. DI Yogyakarta Academy 1 1 011. East Java School 2 2 0

Academy 2 2 012. Bali Academy 1 0 013. North Sulawesi School 2 0 014. South Sulawesi Academy 1 1 015. NTB School 1 0 016. East Kalimantan School 1 0 017. Maluklu School 2 1 018. Irian Jaya School 1 0 0

Total 32 21 2

Table 5.1(b): CONSTRUCTION OF IN-SERVICE TRAINING PROGRAMS (BLKM AND

KLKM), ACCORDING TO APPRAISAL AND IMPLEMENTATION, BY PROVINCES

No. Province Institution Appraisal CompletedNew Rehab.

1. DI Aceh KLKM 1 0 02. North Sumatra KLKM 0 1 03. South Sumatra KLKM 1 1 04. DKI Jakarta BLKM 1 1 15. West Java BLKM 2 2 0

KLKM 0 1 06. Central Java BLKM 1 1 07. East Java KLKM 1 1 08. Bali KLKM 1 1 0

Total 8 9 1

- 16 -

person months of assistance in organization and management were provided to Pusdiklat.There is little evidence to suggest that this technical assistance resulted in overcominglargest institutional issue facing the organization which is the lack of clarity concerningauthority over the design and conduct of in-service training between Pusdiklat and theprogrammatic units of the central Ministry of Health.

Table 5.2: LOCATION OF COMPLETED CONSTRUCTION, BY PROVINCE AND TYPE

Construction Rehabilitation

Pusdiknakes (Pre-service)

DKI JakartaAKPER, Jakarta 1ATEM/APRO, Jakarta (multistream) 1SPRG, Jakarta 1SMAK, JakartaAPKTS/AKZI, Jakarta

West JavaxPK, Sumedang 1SPRG, tasikmalaya IAKPER, Bandung 1AAK, Bandun 1

Central JavaSPRG, Semarang IAKPER/APRO, Semarang (multistream) 1AKFIS, Solo 1AKZI/AKPER/APK, Yogyakarta

(multistream) 1

East JavaSPK, Madiun ISPK, Kediri 1AKPER/AKNES, Surabaya (multistreamn) I

Sumatra SelatanKLKM, Palembang 1AKPERtAKZI, Malang (multistrearn) 1

WestSPK, Solok 1AKZI/APK, Padang (multistream) 1

SouthSPRG, Jambi 1SPK, Tanjung Pinang 1AKZI/AKFIS, Ujung Pandang (multistream) 1SPK, Ternate 1

- 17 -

Construction Rehabilitation

PUSDIKLAT (In-service)

DKI JakartaBLKM, Cilandak 1Central Pusdiklat 1

West JavaBLKM, Ciloto 1BLKM, Lemah Abang 1KLKM, Bandung 1 1

Central JavaBLKM, Salaman 1 1

East JavaKLKM, Burnajati 1

BaliKLKM, Denpasar 1

Sumatra UtaraKLKM, Medan 1

Studies

5.18 The project included three studies: (i) a review of the financial implicationsof manpower requirements of MOH's staffing plans through the year 2000, (ii) review andanalysis of the expected role to be played by 'pekarya kesehatan', or nurse auxiliaries, anew category of health assistants for which short term training was begun at about thesame time as the project was appraised, and (iii) an assessment of the impact of parallelclasses on the quality, efficiency and effectiveness of training programs at the school andacademy level. The first study, a review of the financial implications of the MOH'smanpower requirements, received considerable attention during the first year of projectimplementation, and an interim report, which emphasized problems with the availabilityand quality of data on current placements of health workforce by type and location wasreviewed by a Bank supervision mission in 1987. This mission provided detailedsuggestions for improving the study, though notably was silent on the question of thedemand for and utilization of the health workforce. Project records indicate that a studywas completed in 1988. The findings were incorporated into Bank sector work (Issues inHealth Planning and Budgeting) in 1989.

5.19 Terms of reference and detailed arrangements for the study of PekaryaKesehatan are not mentioned in project records until 1987, and the GOI did complete a

- 18 -

survey of secondary data on the placement of these workers. Although project records donot justify attribution to the project financed study, training of 'pekarya kesehatan' wasdiscontinued in 1989.

5.20 Terms of reference for a study of the impact of parallel classes on trainingefficiency, quality and effectiveness were not agreed by the time of the 1987 review of theproject. Parallel classes were initially implemented to accelerate the training of nursesduring Repelita IV. They were instituted for much of the 1984 and 1985 batch of training,but were discontinued in 1986 as concerns with an excess supply of paramedic manpowerbegan to displace concern with excess demand. The study was ultimately dropped fromconsideration in the project.

6. Project Results

6.1 This section assesses the results of the project from two perspectives. First,how well was the project managed? Were the inputs provided appropriately employed toaddress the overall objectives of the project? Second, what was the project's overalldevelopment impact, or contribution to the effectiveness of Indonesia's health workforce?The general finding that emerges in this section is that on both dimensions, the project wasless than fully effective. In part, the project's limitations can be attributed to it being arelatively large endeavor. The project stretched the capacity of the core implementingagencies, which had not previously managed a large Bank-financed effort, as evidencedby the problems and delays in achieving the project's physical targets. In addition, theproject appears to have had highly constrained results in terms of its influence on healthworkforce policy and institutional capacity. This in turn appears to be a consequence ofthe project's initial design, which, in retrospect, does not appear to have sufficientlyconsidered broader sectoral and institutional issues. Efforts were made, by both theBorrower and the Bank, to redirect project resources during the implementation period toaddress issues of training quality and the management of human resources in the healthsector. However, these efforts were limited and occurred too late in the project cycle toexert a strong influence on overall effectiveness.

6.2 Use of Resources. Despite delays, the project was able to achieve themajority of its physical goals and 97 percent of the investment was eventually disbsursed.The review of the project in late 1987 correctly identified the major sources of delay incompletion of the project's construction program, and appropriately reduced the overallsize of the program. In addition, Bank supervision missions made effective use ofinstruments to facilitate improved performance, through such steps as establishing a specialaccount for the project, which facilitated more rapid disbursement of loan proceeds, andthrough intensive efforts to assist the Borrower in identifying ways to improveimplementation through amendments to the legal documents

6.3 It is important to note that the project's civil works and equipment targetswere a "slice" of the Health Manpower Development plan that was used as the base forthe project design. This plan called for the construction of 48 new schools and academiesto assure the availability of training places to meet the growing demand for health

- 19 -

personnel. The availability of Bank financing for a significant proportion of this programmay have, despite the delays helped to assure that at least some of this construction wascompleted despite the oil price shocks and severe reductions in MOH's budget in the mid-1980s.

6.4 Utilization of technical assistance and fellowship resources, the project'sprimary instruments for achieving improvements in training quality was less successful.Beyond the implementation of an 18 month individual consultancy on improving fieldpractice and experiential training methods, which appears not to have had significantfollow-up in the context of the project, technical assistance resources were not put in placeuntil 1990, six months after the project's originally planned closing date. Not surprisingly,although the terms of this technical assistance contract were completed in a timely fashion,the findings and insights developed through the activity had little impact on overall projectresults.

6.5 The conclusion that emerges is that the project did in fact, despite manyconstraints and delays, make use of the inputs that were made available. The timing ofthis utilization, however, appears to have limited the overall results of the project.Moreover, in retrospect, it is easy to speculate that the mix of these resources, with morethan two thirds of total project resources allocated to civil works was inappropriate.

6.6 Development Effectiveness. This section reviews the developmenteffectiveness of project results concerning two of its objectives: (i) evidence ofinstitutional strengthening and improved organization in Pusdiknakes and Pusdiklat and(ii) the project's contribution to health manpower policy. The conclusion that emerges isthat the project may not have been highly effective in respect of either of these broadobjectives. This outcome appears to be primarily a function of project design, and itsemphasis on production capacity with insufficient attention to whether the projections ofthe demand for health workers were appropriate in the context of the health sector inIndonesia. This fundamental design problem could not be overcome within the boundariesof the project.

6.7 Institutional Strengthening at Pusdiknakes/Pusdiklat. The primarycomponents of the project aimed to strengthen the organization and management of thePusdiknakes and Pusdiklat respectively. Although there has been no specific evaluationof the achievement of these objectives, the implementation record of the project suggeststhat improvements in institutional capacity in either agency were limited. The fact that themanagement arrangements were significantly adjusted as late as the second amendment ofthe loan, when it was agreed that a Project Secretariat, to be financed at 100% (exclusiveof salaries) would become responsible for project management, suggests that little'strengthening' occurred in either agency between 1985 and 1990. An evaluationconducted by the technical assistance team in 1991, provides indicators of the institutional

- 20 -

and organizational problems that continued to constrain the effectiveness of these agenciesat project completion.4/

6.8 The evaluation team visited each of 33 training schools or academies toexamine the incoming students input to pre-service training, the educational process withinschools and academies, the output of the schools and academies and, experience to datewith the implementation of the multistream concept in those institutions where facilitiesdesigned for multistreaming purposes were constructed. Principal findings are useful asan indication of the institutional development agenda that still needs to be addressed at theend of the project.

Students. A new national student selection process was introduced byPusdiknakes in 1987/1988 (without project assistance). The study teamfound that the new process, which emphasized the use of exams forselecting students, still had timing problems and problems: (i) timing ofstudent selection overlapped with operating budgets at the school/academylevel were inadequate to support stepped up recruitment efforts. Last,despite the new selection process, there were reports of continued ambiguityconcerning responsibility for final selection among the Kanwil, Pusdiknakesand school/academy directors who felt a need for guidance on their role inthe selection process.

* =Educational Process. This evaluation also assessed the adequacy of schooland academy physical facilities and the availability of teaching materials andequipment, among facilities in the project. The evaluation tearn foundproblems, subsequently addressed by the project, with adequate watersupply, western style showers in the dormitories which were unacceptableto students and faculty. The majority of the Directors interviewed reportedthat they had not been involved in the design of the new facilities. Majorgaps wree also found in the quantity and quality of books and audiovisualmaterials available for teaching, materials that were only available inEnglish. Organizational arrangements for the acquisition of materials werealso inadequate. Directors and faculty thus have no control over incomingmaterial, and lack awareness of available materials. Those interviewed atthe school academy level expressed general satisfaction with the process ofcurriculum preparation and implementation, though there was (andcontinues to be) universal concern with the large gap between the theoreticalportions of curricula and limited exposure to practical and field basedexperience in applying these concepts. Resources to facilitate field work,and clear operational principles for planning field experiences wereinsufficient.

4/ Improvement of Management of Training Institutions: A Study of 32 Pusdiknakes HealthManpower Institutions: Association of Canadian Community Colleges in Association with P.T.Inersia Ampak Engineers. January 1991.

- 21 -

6.9 Management and Center/School Responsibility. The evaluation reportexamined questions concerning the organization and management of training programs inall 32 schools, and reached two broad conclusions. First, the team recommended thatPusdiknakes should revisit its role vis. each of the schools, and work to delegateresponsibility for institutional level management to the Directors and their staff. Thereport recommended that Pusdiknakes adjust its role to focus on broad monitoring of theeducational process and improve its capacity to act as a liaison between school andacademies and the Ministry of Health. It recommended further that Directors and keyinstitutional personnel at the school level receive training in long range and strategicinstitutional planning.

6.10 Impact on Health Manpower Policy. The key instruments through whichthe project was designed to influence health policy were: I) the establishment of a smallnumber of multistream academies as a way to encourage improvements in the efficiencyand effectiveness of paramedic training, and 2) design and implementation of the threepolicy studies. As previously noted, actual performance on each of these topics waslimited, and the impact of these efforts on health policy apparently very limited.

6.11 The multistream concept remained poorly understood and largely notendorsed by the Directors of schools and academies interviewed in 1990/1991. Mostschool or academy Directors reported lack of clear guidance and expectations regardingthe concept. Many were unclear on where the concept originated, or its purpose andobjectives. The evaluation found that few of the resources that were to be shared acrossprograms in multistream settings were in fact shared. For instance, in one school thecommon library had been divided into three separate areas, with individual collections andcirculation controls organized for each of the individual programs of training (organizedby category of paramedic personnel). Litle administrative coordination across sprogramsoccurs. One explanation for this result, which to date has not changed is that planning andbudgeting arrangements are fragmented along paramedic category lines. Overall, it wouldappear that the project achieved little with regard to either developing or implementationof the multistream concept.

6.12 The results of policy research on the effectiveness of the use of parallelclasses and on the role and function are similarly disappointing, though less welldocumented. Although work on both efforts was started, and a study, using secondarydata on pekarya kesehatan was completed in 1989, it is difficult to attribute the decisionto stop the program on the basis of the project-financed research. A major finding ofresearch on parallel classes was that they are ineffective if unaccompanied by the provisionof additional resources to assure reasonable student/faculty ratios and equipment. By1988, Government was increasingly concerned that the numbers of paramedics beingproduced would exceed the number of civil service posts available. A decision wasaccordingly taken to stop parallel classes in 1987. However, this decision wassubsequently reversed in 1990, when the Ministry of Health initiated a program toaccelerate the training of the bidan di desa, or village midwife, and returned to the use ofparallel classes to rapidly increase their rate of production. If the study of parallel classesdid influence policymaking on the production of health manpower, it was clearly not

- 22 -

sufficiently influential to withstand the pressures, external to the project and the projectimplementation units, to rapidly develop this category of personnel.

6.13 Three factors appear to underlie these disappointing results. First, theoverall size, and early problems with the civil works program appears to have quicklycaptured the focus of both Bank and Government supervision of the project. As a result,work to initiate the policy studies was delayed to a point where decisions that were to beaddressed through the research were already made, thus obviating any significantcommitment to the implementation of study findings. Second, both Bank and governmentmanagement teams responsible for the project went through significant and frequentchanges between project startup and completion. This lack of continuity in projectmanagement was perhaps exacerbated by the lack of detail in the staff appraisal report onthe objectives and expected outcomes of the multistream concept, the role of technicalassistance and, particularly, the expected contributions of the research program. Third,it is interesting to note that project records indicate little effort, on the part of either thegovernment or the Bank, to identify or create linkages between this project and otherprojects and activities in the health and population lending and ESW program. Thisappears to be a result of the need to concentrate supervision resources until well into thefourth year of the project on the completion of the civil works program and later, therestructuring of the loan. Once the project was restructured, delays in the process ofrecruiting and placing the integrated package of technical assistance, and a lengthy processof preparation of bid documents and bid evaluation for the procurement of the equipmentfor the constructed facilities dominated the focus of supervision missions.

6.14 Reduced Expectations. In the context of this assessment, it is importantto note that both Bank and Borrower expectations for the project were steadily andappropriately scaled down as the difficulties of completing the civil works programcontinued and awareness of the limited influence that either Pusdiknakes or Pusdiklat couldhave on decision making within the program units at MOH grew. By 1990, a supervisionmission concluded that important issues of employment and training in the health sectorand management and institutional issues would remain unresolved and could not beaddressed within the framework of the project.

6.15 From 1990 on, the the Bank and the borrower concentrated on taking thenecessary steps to complete the construction program, assure that the facilities wereequipped, and the use of technical assistance put in place to assist the Project Secretariatin the implementation of the fellowship program. In the end, the project results werebroadly consistent with the limited, input-oriented nature of its original design, whichfocused exclusively on the provision of inputs for the development of training capacity,with essentially no attention to perhaps more significant elements of health workforcepolicy.

7. Sustainability

7.1 It is perhaps not surprising, given these results, that the project producedlimited sustainable achievements. Improvements in the macroeconomic environment andin the allocation of resources to health suggest that the Government will not have difficulty

- 23 -

in sustaining the recurrent costs of the schools, academies and in-service training centersthat were established through the project. But the project's failure to significantly alter theinstitutional arrangements which govern health manpower policy making, or move beyondidentification of some of the constraints to improved management of training institutionslimit its effectiveness. It is perhaps appropriate that the Bank and the Borrower decided,during the last stages of project implementation, to defer efforts to address broader issuesof how best to develop the health workforce to later discussions.

7.2 At the same time, the project, despite its very serious limitations in bothdesign and implementation, did contribute to the Ministry of Health and Bank'sunderstanding of the importance of health manpower issues to overall sectoralperformance. The project did, however, stimulate a commitment on the part of the Bankand MOH to work toward developing a more sound conceptual and operational approachto the development of the health workforce.

8. Bank Performance

8.1 The overall design of the project was the major weakness in the Bank'sperformance. In making this assessment, it is important to note that this project occurredvery early in the development of the Bank's operational experience in the health sectorgenerally and in Indonesia in particular. The input orientation of the project design wastypical of the time, and reflected limited experience within the Bank in the design andimplementation of successful institutional development projects. The absence of a clearevaluative framework for assessing progress appears in retrospect to have been the mostsignificant flaw in the project design. It is possible that more intensive efforts to definethe desired project outputs and outcomes during the preparation process may have led toa more positive result.

8.2 Formal and informal Bank supervisions were carried out twice a year aswould be considered normal for a loan of this type. Supervision missions exercisedconsiderable flexibility, as evidenced in the efforts to improve project implementationthrough two amendments in the loan agreement, and to be responsive to Governmentrequests to extend the loan to allow completion of project activities. In retrospect, it mightbe reasonable to question whether this flexibility was excessive. It is striking, given theoverall record of project performance, that no supervision mission recommended orsuggested that the loan be canceled. Project ratings were consistently low throughoutproject implementation, and a suggestion to consider cancellation of the loan may wellhave been appropriate.

8.3 Supervision missions also appear generally to have been suitably staffed witha mix of expertise. At the same time, as detailed in Part III, it is noteworthy that theproject made relatively greater use of health specialist skills early in the implementationperiod, and of more seasoned specialists in operations and implementation in later years.A reversal of this skill mix, with greater use of senior operational staff early on in theimplementation period may have improved performance. The relative absence ofdiscussions of broader aspects of sectoral performance suggests that linkages between the

- 24 -

supervision of this project and other elements of the Bank's health and population portfoliocould have been more effective.

9. Borrower Performance

9.1 GOI's performance in managing the project is best evaluated in two phases.During the first, from appraisal through the first amendment of the loan, the project wasmanaged on a part time basis by the Directors of two newly established units of theMinistry of Health. Performance during this period reflected the difficulties of managinga project on a part time basis, as well as the inexperience of the Borrower in themanagement of a project of this size. The Borrower did, however, move to adjust thesearrangements following the 1987 review and the amendment of the project, and put thenecessary arrangements to establish a Project Secretariat into place. Following theestablishment of this Secretariat, performance on the administration of the project,maintenance of project accounts and disbursements improved significantly and can bejudged to be successful. The limitations of the project results appear to be attributableprimarily to gaps in the general project design, responsibility for which rests with both theBank and the Borrower.

10. Lessons Learned

10.1 Several lessons can be drawn from the implementation of this project.These are include the following:

Design Requires Substantial Sectoral Knowledge. In retrospect, it seemsthat the problems of the project grew out of the limits of the Bank'sunderstanding of the health sector in general and health manpower issues inparticular at the time of appraisal. The design of the project, whileconsistent with health planning and project design principles current at thetime, showed little recognition of the organizational and institutional settingin which the project would be implemented. The project's dominantconcern with improving production capacity also reflected limitedunderstanding of the performance of the health facilities that were to bestaffed by the products of this investment.

* Need for Analysis of Affordability. In a closely related vein, the projectwas appraised with very little attention to the capacity of the Governmentto provide the counterpart funds necessary to assure timely implementation.Although the shocks to the external economic environment could not havebeen anticipated by the appraisal team, sensitivity analysis of the levels andtrends in health expenditures in the public sector may have been helped toavoid the devastating impact that the drop in counterpart funds had onproject implementation.

* Assigning responsibility for project management to the targets ofinstitutional development risks delay in the use of project resources, andcan overwhelm the target agency. In this case, it seems clear that the both

- 25 -

Pusdiklat and Pusdiknakes, neither of which existed prior to the project,needed to concentrate their institutional and managerial resources ondeveloping their basic organizational capacities. Moreover, as newinstitutions, it would appear likely that both would have benefited frommore experience in the management of external finance before becomingresponsible for a project of this dimension.

Early Identification of TORS and Selection Procedures for TechnicalAssistance. This project demonstrates that inattention to the specificobjectives of technical assistance can lead to lengthy delay in securing theseresources, and consequently in disappointing outcomes. In this case, theappraisal report was largely silent on the purposes of the technical assistanceto be contracted through the project, referring generally to expectations thatit would assist in organizational development and improved management andplanning. Once the construction program was back on track, however, thework of agreeing more specific terms of reference and securing qualifiedassistance to fulfill them was undertaken with relative success. Greaterattention to planning these inputs prior to appraisal would have improvedthe overall performance of the project.

* Need for an Evaluative Framework to Assess Project Progress. Last,the absence of an evaluative framework, integral to the clearest projectappraisal, appears to have constrained project performance. The exampleof this problem in the present case was the absence of a discussion in theappraisal of what specific improvements were expected in the project'sgeneral objectives ("expanding the output and improve the quality ofmanpower"). In the absence of this specification, little attention was put ona review of whether the project objectives were appropriate to health sectorneeds and reinforced the supply-side bias of the orientation of theunderlying approach to health manpower policy. The appraisal also lackeda format, in either the SAR or its working papers, which would assist Bankand Borrower management to assess progress toward the project'sinstitutional goals.

- 26 -

PART II. PROJECT REVIEW AND BORROWER'S PERSPECTIVE

2.1 Data related to the project activities presented in Part III of the PCR areadequate and appropriate as mostly derived from project documentation collected byparticipation of the consultation with officials from all projects components andinstitutions. Thorough discussions with those officials have been also held during fieldvisits.

Project Preparation and Design

2.2 In general the Staff Appraisal Report has provided a comprehensive pictureof the nature of the project and guided the project management for right projectimplementation. The project was prepared through discussion and interview involving allrelevant components of Ministry of Health, such as Bureau of Planning, Pre-Service andIn-Service Education and Training Centers, Bureau of Personnel, Directorates of Nutrition,Health Center Development, Family Health of the Directorate General of CommunityHealth, Directorate General of Medical Care, Health Settlement Environment of theDirectorate General of Communicable Disease Control. Bappenas directed and advisedthe Depkes and the Missions in the preparation of the project in terms of planning andbudgeting policies and Ministry of Finance was involved in the discussion on the financialaspect. Other units were also contacted to obtain more information and accurate data forthis preparation.

2.3 The actual needs of Depkes were considered especially the programs relatedto the paramedical manpower development, which was the major issue in anticipatingproblems faced in the Pelita IV (four to five-year development plan). Better healthdelivery services would be achieved through increasing the number and improving thequality of the health manpower. Consequently strengthening education and training ofhealth personnel were quite right to become the focus of this project.

2.4 Considering the Financing Plan table in the Staff Appraisal Report weobserve that US$28.8 million or 68.71 percent of the loan has been allocated for civilworks which is regarded as too much compared to the allocation of the US$5.5 million or14.10 percent for furniture equipment and materials, and US$1.00 million or 2.56 percentfor group educational activities.

2.5 Although buildings for education and training facilities were urgentlyneeded, proper allocation for equipment and group educational activities would help fasterimprovement of the quality of the new graduates and existing paramedical manpower.Based on past experience a project period of 4 years might be too ambitious if not assuredby further assignment of experienced staff in similar World Bank projects management,for handling the project. The need for in-country fellowships was overlooked butfortunately corrected during project implementation.

- 27 -

Project Implementation

2.6 Although there was a substantial delay (September 89 - April 93) in projectcompletion the major objective and targets of the project has been achieved and about99 percent of the loan has been utilized. Compared to the financing plan stated in the SARwhere 60 percent of the total project costs would be from Bank loan, the projectimplementation indicated that the sharing was 61 percent loan and 39 percent GOI funds.

Civil Works

2.7 Construction of new buildings or rehabilitation of old ones for education andtraining institutions have been completed. However, the number of institutions to be builtor rehabilitated has been reduced. The reduction of the amount of civil works was causedby the need for more equipment and training has been realized. This was also due to thefact that GOI revenue was decreasing due to the drop of oil prices starting in 1985. TheGOI was then unable to provide adequate funds for the 40 percent counterpart budget forcivil works.

2.8 Therefore the increase from 60 percent to 76 percent loan funds forfinancing civil works category was very helpful and very much appreciated. Also thereallocation to more loan for educational equipment and group educational activities wasvery useful in improving the quality of education and increasing the dynamic of theteachers, trainers and students.

Educational Equipment

2.9 Notwithstanding the lengthy process of procurement, the project schoolswere very pleased that finally educational equipment could be provided. The absence ofan expert in educational equipment selection and specification and the insufficientunderstanding of the procurement committee members and related personnel in the WorldBank Guidelines for ICB were the main reason for the delay in procurement.

Technical Assistance and Fellowships

2.10 With regard to the provision of technical assistance and fellowships it washighly appreciated but it would be better if the project could start at the early stage of theproject period. This was indeed difficult as project management was handled by peoplenot having much experiences in managing similar World Bank Project activities.

2.11 There was not much time left during the project period for making follow-upto the findings and recommendations of the consultants. The concept of multistreamacademies was not clear while the consultants did not have time to develop and implementa model for that purpose.

2.12 The designated selected centers has started its activities to lead similarschools but the criteria for designating selected centers need to be revised. A selected

- 28 -

center model has been formulated by the consultant but again no time was available toimplement the model fully.

2.13 The provision of overseas fellowships was very much appreciated. Howeverthe selection of fellowships candidates was rather in a hurry and many teachers shouldleave the school at the same time as they did not want to loose the opportunity to join theoverseas training. The benefit of the custom designed overseas training could not beevaluated in terms of improvement in academic knowledge but new ideas obtained throughclose contact with foreign education systems might enhance better quality of teachingprocess in the future. The in-country fellowships training which was not included in theoriginal plan has given opportunity to more teachers without proficiency in English toimprove their knowledge.

Project Management

2.14 For reviewing the project management we divided the project period intotwo phases, the pre-secretariat period from 1985-1989 and the period of 1989-1993 wherea full-time project secretariat was established. During the first period there was no full-time staff and most of the staff were inexperienced in handling the World Bank projects.The perceived benefit in having full-time and experienced secretariat staff is:

* more time and concentration in carrying out the project administration andmonitoring the project activities;

* less top down planning;* more efficient in using financial resources; and* more able to find solution in administrative and communication problems

encountered

Bank Performance

2.15 In general, the Bank missions were recognized as qualified. We havelearned a lot from the missions and task managers. However, in some discussions thechange in task managers and mission members could cause delay in responding lettersfrom the GOI which caused delay in project implementation.

Project Sustainability

2.16 The classes and rooms, the educational equipment, the more qualifiedteachers or trainers provided by the project gave a substantial input in speeding up theinstitutionalization of the schools and training centers. This, in turn, has made routinebudget available for operation and maintaining the project education and training facilities.

- 29 -

PART III: STATISTICAL INFORMATION

Table 1: RELATED BANK LOANS

LoanlCredit Year oftrk Purpose Approval Status

Health and NutritionCr. 1373-IND To enhance GOI capacity to formu- March 1977 Compkted

First Nutrition late and execute nutrition programs, March 1982

Dcvelopment Project operations, research, manpowertraining, monitoring and evaluationby strengthening and expanding theexisting nucleus of personnel institu-

tion.

Ln. 2635-IND To improve effectiveness of health February 1983 Completed

Provincial Health care services; reduce incidence of September 1989malaria in three provinces; improvequality and utilization of healthservices through improvements inhealth centers, referral and outreachsystems and manpower development.

Ln. 22636-IND Increae the effectiveness and November 1985 Completed

Nutrition and efficiency of GOI's nutrition and June 1992

Community Project community health programs through(a) strengthening coordination andmanagement of community healthprograms; (b) further development ofnutritional surveillance; and(c) strengthening nutrition manpowerdevelopment.

Ln. 3042-IND To assist GOI improvc the delivery June 1989 Ongoing

Third Health Project of health services and raise healthstatus in two provinces through betterand more decentralized health sectorplanning, budgeting and manage-ment.

Ln. 3550-IND To (a) enhance programs for child December 1992 Ongoing

Third Community survival, safec motherhood and nutri-Health and Nutrition tion in five provinces; (b) buildProject provincial capacity to plan, monitor

and evaluate health and nutritionservices; and (c) strengthcn centralDepkes to providc technical supportto the provincc.

PopulationCr. 300-IND To (a) assist GOI in the establish- March 1972 Completed

ment of clinic-based family planning September 1981program in six provinces of Java andBali; (b) strengthen BKKBN's coor-dination capacity; and (c) improvefamily planning information andmotivation services.

- 30 -

LoanlCredit Year oftritk Purpose Approval Status

Ln. 1472-IND To (a) extend family planning pro- June 1977 CompletedPopulation 11 gram to ten additional provinces in March 1984

Outer Island I; (b) establish contra-ceptive distribution centers in Javaand Bali; and (c) expand andstrengthen population education andmotivation activities.

Ln. 1869-IND To (a) expand family planning pro- lunc 1980 CompletedPopulation III gram services to the remaining September 1984

provinces; (b) assist GOT to decen-tralize management of familyplanning program; and (c) strengthenmatemal and child health services.

Ln. 2529-IND To (a) further strengthen BKKBN June 1985 CompletedPopulation IV through completing decentralized December 1992

infrastructure in Outer Islands II,improve field communications,expand staff, strengthen IEC; and(b) usist the Ministry of Populationand Environment to perform itspolicy and coordinating role.

Ln. 3298-IND To (a) expand family planning ser- April 1991 OngoingPopulation V vice delivery to hard-to-reach popula-

tion groups; (b) enhance acceptanceof long-acting contraceptives;(c) support BKKBN's IEC, staffdevelopment, research and MISactivities; (d) promote safec mother-hood through training and deploy-ment of villagc midwives safemotherhood IEC campaigns.

- 31 -

Table 2: PROJECr TIMErABLE

Item Date Planned Date Revised Date Actual

Identification(Executive ProjectSummary

Preparation

Appraisal Mission October 1984

Loan Negotiations April 1, 1985

Board Approval May 14, 1985

Loan Signature June 18, 1985

Loan Effectiveness September 16, 1985 November 19, 1985 January 21, 1986

Loan Closing March 31, 1989 September 30, 1990 March 31, 1993March 31, 1992September 30, 1992

Loan Completion March 31, 1989 March 31, 1993

Last Disbursement June 29, 1993

Table 3: CUMULATiVE EsTIMATED AND ACTUAL DISBURSEMENT(US$ million)

1986 1987 1988 1989 1990 1991 1992 1993

Apprisal 7.0 20.3 30.8 36.7 39.0 - - -Eatimate

Actual 1.5 1.9 7.0 14.8 19.1 25.2 30.9 37.2

Actual as % of 21 9 23 40 49 65 79 95eatirmae

Actual as % of 4 5 18 38 49 65 79 95total loan

Date of Final June 29, 1993Disburaement

Comments: 1. In 1990 *greement was anmended, as of December 1990 with cancellation of US$700.02. Original Loan Amount was US$39,000.0; cancelled US$700.0.

- 32 -

Table 4: PRoJECF IMLEMENTATION

Description Appraisal Actual(PCR Estimate)

1. Pe-service ParamedicalHealth Manpower Develop-ment

1.1 Insttutional Streqthening

Organizational capability. Project provides 168 Association of Canadian Com-foreign and 24 local man- munity College served for 24months of TA services months, Four long-term

fellowships overseas weregranted and 18 short-term forstaff.

la-Country fellowship 7 long-term fellowships.

Curriculum Development. Curriculum development hasdone, but could not identifythe evidence.

1.2 Strengthenin of ProvincialEducation and TrainingDivisions

Overseas taining in manage- 10 ST fellowships two of Two long-term MPH inment, planning, and super- each of five selected Management and Planningvision provinces (West, Central,

East Java, North Sumatraand South Sulawesi)

Se-mnar and workshop.

Local trining. Local training have beencarried out for staff andincluding ACT A III (teachertraining)

1.3 Development of Selected 10 3Centes and other schoolsand acadenies

Overseas taining in pam- 36 long-term fellowships Overseas: 3 long- and 88medical technical subject. short-term for selected centers

and other schools.

In-Country fellowships 185 long-term fellows forSelected Centers andschools/academies.

Seminar and workshop In-country seminar and Completedworkshops

- 33 -


1.4 Paramedical Schools andAcadenies

Construction, equipping and 32 buildings of schools 23 buildings completed includ-furnishing institutions. and academies, located in ing 2 rehabilitation in 10

18 provinces. provinces.

Acquisition of teaching Books and teaching equipmentmaterials and books. and materials purchased for

schools and academies.

Acquisition of vehicles for 34 operational vehicles forschools. schools and academies.

2. Strengthening of Pusdiklat

2.1 Technical assistance tostrengthen the Pusdiklat'sorganization; Nationallevel.

2.2 Overseas training

Planning and management 8 master degrees for 5 long-term fellowships.and curriculum development. pusdiklat staff;

Research, development and 2 doctorate degree forevaluation Staff

Training in Administration, Short-term: 4 in 10 short-term fellows.educational technology admin/mngt 7 in educa-

tional technology

Technical Assistance 48 mammonths of technicalservices

35 short- and 3long-term fellowships havebeen granted.

2.3 In-Country fellowships

Seminar and workshops In-country seminar and Workshops and seminar haveworkshops been done.

2.4 Strengthening of Regionaland Provincial In-ServiceTraining Centers

- 34 -


Experimental, field-based In community health Completed, no follow-uptrining. development programs to

84 trainers from national,regional provincial levels;in community healthmanagement programs to16 trainers.

Overes taining in public 20 ansster in public health 9 long- and 15 short-termhealth education programs. fellowships were granted.

Technical Assistace 18 manmonths of technical Completedservices for EFBT andfellowship management.

2.5 In-Swvice Training Centrs

Construction, equipping and 4 national training centers: 10 constructions including onefumnishing of buildings. (Cilandak, Ciloto, Lemah rehabilitation have been com-

Abang and Salaman), two pletedregional and two provin-cial training centers.

Acquisition of teaching Books, teaching materials andmaterials and books. other audio-visual system and

furniture have been purchased.

Acquisition of vehicles for 8 vehicles for training 11 vehicles have been pro-training centers. centers of national, cured for BLKM/KLKM.

regional and provincial.

- 35 -

Table 5: PROJECr COSTS AND FINANCING

A. Project Costs (USS million)

Appraisal Estimates Actual atCompletion

Local Foreign Total Totalg

Land 1.85 - 1.85 0.79Civil works 28.18 7.64 35.82 19.80Furniture, equipmentand materials 3.58 6.35 9.93 17.65Technical assistanceand fellowships 0.03 4.61 4.64 5.71Studies - 0.15 0.15 JbGroup educationalactivities (training) 0.87 - 0.87 4.82Project management - - - 5.70/

Total baseline costs 34.51 18.76 53.26 54.47

Physical contingencies 3.31 1.40 4.71Price contingencies 5.21 2.67 7.88

Total Project Costs 43.03 22.82 65.85 54.47

/a No breakdown of local and foreign exchange component available./b Included in 'technical assistance and fellowships' category.Lc Includes recurrent costs financed by the government.

- 36 -

B. Project Financing

Planned Actual atSourme (LnA2542-IND Revised Revised Completion

Agre nat In 1988) (iL 1990) (03/1993)

(USS'OOO) % (USS'OOO) % (USS'000) % (USS'000) %

IBRDExpediture Categori

(1) Civil works 25,900 15,600 15,900 15,050

(2) Materials, furniture and 5,200 12,500 12,600 11.772equiprnent

(3) Consultantr ervices 2,4007,900 6,000 5,714

(4) Overeas trining 3,000

(5) Seminars, workshops 1,000 2,000 2,600 3,707and local training

(5) Projcct mranagemnct na 500 900 927(excluding salaries andallowances

(6) Unallocated 1,500 500 900

Total IBRD 39,000 59 39,000 59 38.300 59 37,170 68

Government 26,850 41 26,350 41 26,852 41 17,300 32

Total A La 68S 100 6C51S2 100 54 470 100

a/ In 1988 the Loan Agrececnt was amended, and one rnw category was added, to read: (1) Civil works (76 %); (2) Materials,furniture and equipment (100/90/65%); (3) Conahtans' ervices, in-country training and overseas fellowships (100%);(4) Serninars, workshops and local training (100%); (5) Project Managenent (excluding salaries and allowances (50%); and(6) Urallocated.

Purpose as definedStudies at Appraisal Status Impact of Study

1. Health manpower development To assess the financial resourcs required and available Financial projections Unclear, ESWStudy to achieve the targets of the Bormwer's long-term health prepared 1986 addressed issues in

manpower development plan through the year 2000. 1989

2. Review of the functions, status To establish a plan for the long-term utilization, career Completedand long-term role of auxiliary development and training of auxiliary health workers July 1989health workers (pekaryakesahatan)

3. Impact study of parallel classes To evaluate the impact of parallel classes on the Not completed Noneefficiency and the quality of training at the schools andacademics

oW3

- 38 -

Table 7: STATUS OF COVENANTS

Loan Agreement Deadline forCovenants Description of Covenants Compliance Status

Schedule 5, para. 1 Establish and maintain education 03/31/86 In complianceand training division in West,Central and East Java, NorthSumatra and South Sulawesi

Schedule 5, para. 2 Designate 10 paramedical schools 12/31/85 In complianceand academies as selected centers

Schedule 5, para. 3(a) Develop multi-stream curricula for - In compliancethe seven academies and introducesuch curricula for each of theseacademies as they begin operation

Schedule 5, para. 3(b) Establish a policy for multi-stream 04/01/88 Deferred toparamedical manpower education future projectand training

Schedule 5, para. 4 Establish systematic, cost-effective 06/30/86 In compliancespace allocation and design stan-dards for schools and academiesand in-service training centers

Schedule 5, para. 5 Carry out Part E of the project in - In complianceaccordance with terms of referenceacceptable to Bank

Schedule 5, para. 6 Furnish to the Bank the results of 06/30/86 In compliancethe study to assess the financialresources required and availablefor the Borrower's long-termhealth manpower development planthrough the year 2000

Schedule 5, parm. 7 Furnish to the Bank the results of 12/31/86 In compliancethe study to review the functions,status and long-term role ofauxiliary health workers; and

Establish a plan for the long-term 04/01/88 In complianceutilization, career development andtraining of such auxiliary healthworkers

Schedule 5, para. 8 Furnish to the Bank the results of 12/31/86 } Deferred inthe study to evaluate the impact of } agreement withparallel classes on the efficiency } Bankand quality of training; and }

Establish a policy for parallel 04/01/88 }classes

- 39 -

Loan Agreement Deadline forCovenants Description of Covenants Compliance Status

Schedule 5, para. 9(a) Acquire all land and rights for 04/01/86 In complianceeight of 15 schools, academies andtraining centers

Schedule 5, para. 9(b) Acquire all land and rights for the 04/01/87 In complianceremaining seven schools, acade-mies and training centers

Section 4.01 Audit reports due within nine 12/31/93 In compliancemonths after the end of fiscal year

Table 8: USE OF BANK RESOURCES AND MISSIONS

A. Staff Inputs (Staff Weeks)

Stage ofProject Cycle FY84 FY85 FY86 FY87 FY88 FY89 FY90 FY91 FY92 FY93 FY94 Total

Through apprital 24.2 18.3 42.S

Appraisal 45.9 45.9

Negotiations 4.0 4.0

Supervision 2.2 25.3 16.7 20.9 7.8 10.5 11.4 12.5 4.2 138M4

PCR 0.1 11.7 12.0

Total 24.7 773 2S.3 16.7 20.9 7.8 103 11.4 12.5 4.3 11.7 2233

Month/Year Duration of Number ofMission Persons Name Specializations

ThroughAppraisal

Appraisal Oct/Nov 84 24 4 Pearce, Lundeberg, Nelson, Operations/HealthSinur

Appraisal throughEffectiveness

Supervision Dec 85 7 7 Williams, Bumgarner, Health/Operations/Prescott, Sirrur, Mills, EconomicsBrooks, Krister

Apr 86 2 3 Williams, Bumgarner, Krister Health

Sep 86 27 3 Williams, Sinur, Sinclair Health/RA/Architect

Mar 87 8 2 Mahar, Sinur Economic/RA/U,Operations

Oct/Nov 87 23 4 Shanley, Hunting, Buckley, Health/OperationsBrooks

Jan/Feb 88 24 4 Shanley, Hunting, Buckley, Health/OperationsBrooks

May 88 4 1 Shanley OperationsJun 88 2 Sung, Brooks Operations

Mar 89 11 2 Shanley, Buckley OperationsSep 89 23 2 Shanley, Iswandi Operations

Jan/Feb 90 41 2 Hunting, Iswandi Operations

May/June 90 41 3 Hunting, Martins, Iswandi Operations

Sept/Oct 90 28 2 Hunting, Iswandi Operations

Completion July 93 - I Iswandi Operations/Health

world bank document - documents & reports · for official use only po = operational guidelines...

Documents