Document ofThe World Bank

FOR OFFICIAL USE ONLY

Report No: 22455-ALG

IMPLEMENTATION COMPLETION REPORT(CPL-35730; SCL-3573A; SCPD-3573S)

ON A LOAN

IN THE AMOUNT OF 40US$ MILLION

TO THE DEMOCRATIC AND POPULAR REPUBLIC OF ALGERIA

FOR

A BASIC & SECONDARY EDUCATION SUPPORT PROJECT

09/10/2001

Human Development SectorMiddle East and North Africa Region

This document has a restricted distribution and may be used by recipients only in the performance of theirofficial duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective )

Currency Unit = Algerian Dinar1.00 DZD = US$ 0.013US$ 1.00 = 77.06 DZD

FISCAL YEARJanuary I st December 31

ABBREVIATIONS AND ACRONYMSCAMEMD Centre d'Approvisionnement et de Maintenance des Equipements et des Moyens

Dicdactiques (Center for Procurement and Maintenance of Equipment andPedagogic Materials)

CNF Centre National de Fonnation des Cadres (National Staff Training Center)COSP Centre d'Orientation Scolaire et de Selection Professionnelle (School Orientation

and Selection Center)DOE Directions Chargees de l'Orientation et de l'Evaluation (Department

for Orientation and Evaluation)IPN Institut Pedagogique National (National Pedagogic Institute)INRE Institut National de la Recherche en Education (National Institute of Education

Research)MNE Ministry of National EducationPMU Project Management Unit

Vice President: Jean-Louis SarbibCountry Manager/Director: Christian Delvoie

Sector Manager/Director: Jacques BaudouyTask Team Leader/Task Manager: David Steel

FOR OFFICIAL USE ONLY

ALGERIABASIC & SECONDARY EDUCATION SUPPORT PROJECT

CONTENTS

Page No.1. Project Data I2. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 14. Achievement of Objective and Outputs 45. Major Factors Affecting Implementation and Outcome 76. Sustainability 87. Bank and Borrower Performance 98. Lessons Leamed 119. Partner Comments 1210. Additional Information 13Annex I. Key Performance Indicators/Log Frame Matrix 15Annex 2. Project Costs and Financing 18Annex 3. Economic Costs and Benefits 20Annex 4. Bank Inputs 21Annex 5. Ratings for Achievement of Objectives/Outputs of Components 22Annex 6. Ratings of Bank and Borrower Performance 23Annex 7. List of Supporting Documents 24

This document has a restricted distribution and may be used by recipients only in theperforrnance of their official duties. Its contents may not otherwise be disclosed withoutWorld Bank authorization.

Project ID: P004954 Project Name: BASIC & SECONDARYEDUCATION SUPPORT PROJECT

Team Leader: David J. Steel TL Unit: MNSHD

ICR Type: Core ICR Report Date: September 17, 2001

1. Project Data

Name: BASIC & SECONDARY EDUCATION SUPPORT L/C/TFNumber: CPL-35730;PROJECT SCL-3573A;

SCPD-3573SCountry/Department: ALGERIA Region: Middle East and North

Africa RegionSector/subsector: EP - Primary Education

KEY DATESOriginal Revised/Actual

PCD: 06/20/1991 Ef4&ctive: 03/23/1994Appraisal: 11/19/1992 MTR: 04/02/1999Approval: 03/16/1993 Closing: 03/31/2001

Borrower/Implementing Agency: GOVT. OF ALGERIA/MIN. OF EDUCATIONOther Partners:

STAFF Current At AppraisalVice President: Jean-Louis Sarbib Caio Koch-WeserCountly Manager: Christian Delvoie Pieter P. BottelierSector Manager: Jacques Baudouy Roslyn HeesTeam Leader at ICR: David SteelICR Primary Author: Dung-Kim Pham

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=HighlyUnlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: S

Sustainability: L

Instituetional Development Impact: M

Bank Performance: S

Borrower PerJbrmance: S

QAG (if available) ICRQuality at Entry: S

Project at Risk at Any Time:

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:

As indicated in the appraisal report, the project will be the first step in a long-termn process of

improving the quality of learning acquired by students and increasing the efficiency of resource use in basicand secondary education.

This objective is clearly stated and correctly reflects the government's strategy with respect toeducational quality and efficiency as well as the government's decision to limit the project scope to theimmediate needs of those sub-sectors. In the absence of a Country Assistance Strategy, this objectiveemerged from a sector study: "Democratic and Popular Republic of Algeria - Improving Quality andEfficiency in Basic and Secondary Education," World Bank Report No. 8912-AL, August 1990, preparedjointly with the government. It identified and analyzed a number of organizational/structural andpedagogical problems and offered potential solutions. The project's objective was thus both realistic andrelevant. Since the quantitative goals of increasing school enrollment were largely achieved by theGovernment, the project focused its priority to improving quality and efficiency of the sub-sectors in theareas of assessment of learning achievements, pedagogic research, and pedagogic materials. Furthermore,the project simply aimed to initiate the process of quality and efficiency reform, which will be a long anddemanding undertaking.

3.2 Revised Objective:

The project objective was not revised.

3.3 Original Components:

The Project comprises three components. A description of the Government entities involved in theimplementation of the project appears in Section 10, "Additional Information."

Component I: Design and introduction of a pilot system for assessing learning achievements, quality, andefficiency. Cost: US$ 3.7 million. This component, managed by the Centres d'Orientation Scolaire and deSelection Professionnelle - COSP (School Orientation and Selection Centers) in six wilayas aimed to (i)increase the capabilities of the Ministry of National Education (MNE) in the area of knowledge and qualityassessment; and (ii) disseminate infornation on new assessment techniques, methods, and practices withinthe education community and among decision-makers.

Component II: Pedagogic research. Cost: $US 2.3 million. This component supported training andimplementation activities linked to the pedagogic research programs of Institut Pedagogique National -IPN (National Pedagogic Institute). In 1996, the name of IPN was changed to Institut National deRecherche en Education - INRE (National Institute of Education Research). In the context of the project,this change did not affect Component II because INRE's mission was still to perform education-relatedresearch, evaluate the education system, and develop and test teaching methods.

Component III: Improved provision and more efficient use of pedagogic materials. Cost: US$ 58 million.The purpose of this component was to improve the supply and use of pedagogic materials in schoolsthrough: (i) provision of pedagogic materials for about 10,500 existing and 2,500 grade 1-6 schools underconstruction, and 750 grade 7-9 and 250 secondary schools under construction; (ii) improved provision anduse of school science laboratory products in about 3,500 grade 7-12 schools, rehabilitation of the Centred'Approvisionnement et de Maintenance des Equipements et des Moyens Didactiques - CAMEMD(Center for Procurement and Maintenance of Equipment and Pedagogic Materials) storage andredistribution facilities for such materials and the provision of related equipment and staff training; and (iii)

- 2 -

provision of spare parts, equipment, technical assistance, staff training and improved CAMEMDmaintenance facilities for improved maintenance services to schools.

The design of the three components was well in line with the project objectives. Although aware ofthe decline in educational quality and output, the MNE lacked the proper tools to measure the system'sperformance and to assess the impact of the reform measures. The activities of Components I and II filledthis gap by providing the MNE with an essential lever. Component I also included a powerful tool fordisseminating the results of the assessments to decision-makers and obtaining feedback from them.

Other elements essential to improving the quality and efficiency of the education system are anadequate supply of pedagogic materials and their efficient utilization. Most schools were without anypedagogic materials, and the supplies and equipment necessary to implementation of the reforms at thebasic education level had not been provided. Component III helped make up this deficit by supplying theschools with the appropriate pedagogic materials. It also included an important element of ensuring a moreefficient utilization of those materials by training high-level technicians and a large number of scienceteachers in their use and application.

The design of the components took into account experience from prior loans by choosing a simpleimplementation structure. Thus, implementation of each component was entrusted to one of the entitiesoperating under the authority of the MNE, the project activities being arranged to fit into each one's workprogram. However, this design did not take into account the restrictive administrative arrangement forComponent I, in which technical implementation and budgetary execution were entrusted to two differentbodies (one institution under the ministry's supervisory authority, INRE; and a central MNE directorate,the Direction de l'Evaluation, de l'Orientation et de la Communication - DEOC (Directorate ofAssessment, Orientation, and Communications). This arrangement was an obstacle to the efficientimplementation of Component I (see details below).

3.4 Revised Components:

Whereas the objectives were not revised, the "quantitative" side of Component III was slightlymodified. In physical terms, the laboratory products packaging and storage unit was dropped and theoverall spare parts requirement scaled down. The unutilized funds were reallocated to supply a number ofadditional schools whose facilities had been destroyed by acts of terrorism, a total of 193 primary schoolsand 76 general secondary schools. In financial terms, of an unutilized balance of US$11.8 million, US$5million was canceled in July, 2000 at the request of the Government, while over US$6 million financed theadditional schools.

3.5 Quality at Entry:

The quality of project preparation is considered satisfactory based on several factors, of which thefollowing are the most notable: (i) the project design was the result of an in-depth sector study, directlymeeting the priorities of the Algerian govenmment, which, at the time, was looking at the qualitative aspectsof the system after achieving substantial quantitative improvements in school enrollment; (ii) the modestscope of the objective which addressed the early stages of a longer process of improving the quality oflearning acquired by the students and increasing the sector's efficiency; (iii) given the risks associated withAlgeria's political and economic situation and with the limited project implementation capacity ofgovernment agencies, a simple project design was essential; (iv) preparatory steps taken in advance of theproject, such as the issue of invitations to bid on educational materials and of requests for proposals from

- 3 -

international technical consultants prior to loan effectiveness.

However, the quality of project preparation could have been enhanced by the presence ofperformance indicators, with clear distinction between output and outcome indicators, with base values andtarget values. These indicators would have contributed to a better assessment of achievement of thedevelopment objectives and project sustainability. Also, viewed retrospectively, while most on-site missionswere not feasible because of security concerns, including in particular the mid-term review mission, theproject could have provided for an impact evaluation study towards the end to evaluate outcomes as well asidentify areas in need of additional action. Furthermore, the study evaluating the system of textbookproduction and distribution, and the study on the maintenance and security of pedagogic materials inschools should also have benefitted from advance preparation, as indicated in (iv) above.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:

Despite the difficult economic and political climate in Algeria during the project implementationperiod, the Government has supported the project objective and continued to assign priority to improvingthe quality and efficiency of the system. Overall, the project has achieved its objective. It has provided theMNE with a basic capacity for assessment of learning achievements and education research which willserve as a foundation on which the MNE can continue to build in order to better gauge what students arelearning, what is being taught, and how it is being taught (Section 6 "Sustainability").

Pilot system for assessing learning achievements, quality, and efficiency: The objective has beenpartially achieved. The pilot system has not yet been set up. However, substantial experience in pedagogictechniques of assessment and in survey preparation and monitoring has been acquired as a result of thework carried out during the project in connection with pedagogic performance surveys of the followingdisciplines and levels: Arabic language (3rd, 6th and 9th grade of basic education), mathematics (3rd, 6thand 9th grade of basic education and I st grade of secondary education), French language (9th grade ofbasic education). DEOC also succeeded in establishing operational indicators, based on those of theOECD, to facilitate assessment of educational quality and efficiency. This task will be continuedsystematically to serve as an assessment tool.

Pedagogic research: The objective was achieved, and the outcomes are encouraging, as shown bythe mastering of the application of pedagogic research methodologies and techniques by INRE. As acorollary to the analysis of school programs undertaken by INRE, curricula and textbook content wererevised, using in-house capacities. The outcomes were widely distributed (through the publication of studyand training workshop reports), including to the decision-makers in the education system, particularly theNational Commission on Education Reform set up by the President of the Republic. Furthemmore, theeducation community now has more information available in the area of pedagogic research and newteaching methods thanks to the INRE library, which has modemized and strengthened its documentationresources.

Improvedprovision and more efficient use ofpedagogic materials: This objective was achievedbut with one significant qualification. While the "physical" inputs of pedagogic materials were fullyachieved and beneficial, CAMEMD's services remained centralized and their quality did not improvemuch. An assessment based on a sampling of teachers and students in establishments served by the Center,in Guelma and Tlemcen, revealed that the availability of equipment had led to better application ofcurricula and progressions and greater student interest in classroom and practical work. However, it

- 4 -

appeared that the services delivered by CAMEMD in the areas of procurement, repair, and maintenancewere inefficient. The quality of the materials received was variable, their arrival was often notsynchronized with the start of the school year, packages of supplies were sometimes incomplete, andschools were often turning to the private sector for repairs. The maintenance study, which was notcompleted, would have enabled CAMEMD to redefine its responsibilities within the context ofdecentralization and of a closer association with the private sector.

4.2 Outputs by components:

The project components had varying outputs.

Component I - Design and introduction of a pilot system for assessing learning achievements, qualityand efficiency (implementing agencies: DEOC and INRE)

The MNE did not have sufficient capacity to assess the students' learning achievements. Althoughthe educators were aware of the decline in the quality and efficiency of education, the reasons were not fullyunderstood, let alone acknowledged. This component aimed to improve the MNE's capacities for assessinglearning achievements and quality and to disseminate the new assessment techniques and practices to theeducation community and the decision-makers.

This component was achieved only partially. The activities linked to the design of a pilot systemfor assessing learning achievements were not fully accomplished. The discipline surveys targeting thedifferent education levels did take place, but this activity was suspended at the critical phase of analyzingthe results and preparing the reports. Consequently, neither the activities linked to tests of the system in thesix COSPs nor those scheduled for the second phase in seven additional wilaya took place. The multipliereffect of the training received abroad in assessment methodology was very modest, and the training ofon-site inspectors did not come about.

Within the framework of this component, it was also planned to develop quality and efficiencyindicators for the education system, based on the OECD model. This task was performed satisfactorily.Indicators for 1999 and 2000 were established. The 2000 report, containing 29 indicators, has been sent tothe printers and will be distributed to the education community.

Although the extent to which this component was accomplished fell short of expectations, it didcontribute to the strengthening of the DEOC's institutional capacities in the areas of survey preparation andexecution and data quality verification. It also led to the development of capacities for devisingperformance indicators for the education system.

Component II - Pedagogic research (Implementing agency: INRE)

Given the inadequacy of INRE's pedagogic research capacities it was unable to make a systematicevaluation of the impact of the reform measures, and, consequently, of the quality of the instructionprovided, or to develop curricula based on pedagogic programs. The purpose of this component was tostrengthen INRE's capacities for analyzing pedagogic programs so as to improve the consistency betweendisciplines and basic and secondary education cycles. Within the project framework, only two disciplines(Arabic and science) were covered.

Implementation of this component was satisfactory. Both the quantitative and the qualitative goals

- 5 -

were achieved, and sometimes even exceeded. As planned by the project, 36 experts in pedagogy weretrained in curriculum and textbook design. This training was then passed on to wilaya personnel.Pedagogic research programs, supported by international technical assistance, were also conducted, and thefindings distributed to large numbers of inspectors and teachers by means of seminars and publications.An assessment of the programs covered by the component (Arabic and science) was carried out and evenexpanded to other programs and other textbooks in subjects such as philosophy, history, geography,mathematics, and French. The results of the programs assessment were published in a memorandum, ofwhich 6,000 copies were printed for distribution within the education community. A large number ofseminars were organized and brochures on their conclusions also produced.

Two outcomes are worth mentioning:

- A number of textbooks in Arabic for the different education cycles, for the disciplines of science,letters, and mathematics were revised to take account of the results of the programs assessment carried outby INRE. In future, INRE will include among its functions the work of preparing textbook specifications.

- INRE was able to make the results of its pedagogic research available to the National Commissionon Education Reform and thereby contribute to the Commission's work.

Also within the framework of the component, INRE was able to add around 5,000 technicaldocumentation to its library. Although the library still lacks infrastructure and furniture, its filing systemis up and running and, with a satisfactory volume of documentation, it has been able to open its doors toeducators, researchers and students in the area of education science. The attendance rate at the library,which has become a benchmark, is a good indicator of the availability and quality of its documents and ofthe services offered to researchers and teachers.

Component II - Improved supply and more effective use of pedagogic materials (Implementingagency: CAMEMD)

In the absence of materials (particularly for grades I - 9, since implementation of the new basiceducation system required new teaching inputs), and given that pedagogic materials represent one of theessential inputs contributing to an improved educational environment, this component was designed toprovide appropriate pedagogic materials to around 14,000 primary schools (representing 85 percent ofexisting schools) and 250 secondary schools (35 percent of existing schools) and to establish a centrallaboratory products storage and packaging unit. Its purpose was also to make the use of pedagogicmaterials more efficient by strengthening CAMEMD's maintenance facilities.

The quantitative goals of this component, i.e. number of schools receiving pedagogic materials,were achieved: the 100 percent coverage rate set for the year 2000 was met, and stocks are available forthe new primary schools. A further equipment tranche was obtained to supply 269 additionalestablishments. This financing was provided by funds coming from the dropped laboratory productsstorage and packaging unit, and from the reduced needs in equipment spare parts. Activities connectedwith the unit were abandoned for reasons of security, but arrangements were made to provide thelaboratory products to the schools. A total of 27 professionals received training abroad (nine in IT andmanagement, ten in machinery/tools, four in reeling/coiling, and four in electronics).

However, the maintenance and security study included in the project, which was to have beencompleted by December 31, 1994, was launched in only early 1999. Because of the country's securitysituation at the time, the consultants were unable to conduct the study and the contract was canceled by

- 6 -

mutual agreement between CAMEMD and the Bank. Since then, with the evolving economic environmentand the expansion in the scope of CAMEMD's responsibilities, it was decided to redefine the purpose ofthe study, dividing it into two different studies, one concerning a strategy for the procurement andmaintenance of school equipment, and the other concerning the development of a maintenance master plan.At the end of the project, no action had been taken on these two studies. CAMEMD did state, however,that it planned to undertake the studies on its own after the project closing date as it recognized theirimportance.

4.3 Net Present Value/Economic rate of return:

Not applicable.

4.4 Financial rate /f return:

Not applicable.

4.5 Institutional development impact:

The project's contribution to the building of institutional capacities varies among the differentimplementing agencies within the MNE. Those capacities were acquired thanks to the commitment of theMNE's professionals, their determination to improve the system, and the productive use of the internationaltechnical assistance furnished within the project framework. Where INRE is concerned, the impact hasbeen substantial. Its staff has acquired an understanding of the techniques of analysis and design of schoolprograms and assessment of subjects taught, such that INRE has been able to expand these activities tocover subjects not included in the project. INRE professionals are now performing tasks that will enablethem to upgrade teaching methods and develop pedagogic objectives for textbook content. In the case ofDEOC and CAMEMD, the impact is less marked because of the non-completion of "qualitative" activities.Thanks to the surveys conducted, DEOC and the education community (inspectors and teachers) in thewilaya have acquired a familiarity with techniques for assessing learning achievements, supervision andsurvey methods, and validation of results. Using its own capacities, DEOC set up test surveys during aperiod when no technical assistance was available. In CAMEMD's case, this division has strengthened itsconpetence in terms of project management, enabling it to understand the need to plan, monitor, andsynchronize the different stages of procurement and delivery of the materials to schools.

5. Major Factors Affecting Implementation and Outcome

5.] Factors outside the control of government or implementing agency:

The economic crisis of 1998 and 1999 resulting from the drop in the price of oil in 1998 was ahindrance to project implementation because it led to a substantial reduction in the project budget. Thiswas a particular obstacle to Component I activities, since it affected funding for the surveys. In addition, asuccession of changes in the Bank's project teams affected "qualitative" activities to a certain degree. Thesection entitled "Bank Performance" (see below) discusses this factor in detail. In general, inadequateattention was paid to the monitoring and analysis of the assessment and pedagogic research activities.

5.2 Factors generally subject to government control:

Thanks to the Ministry of National Education 's commitment to its policy of improving the qualityand efficiency of basic and secondary education, the project was able to continue working toward its

-7 -

objective. This was in spite of the country's security situation which had an adverse influence on projectimplementation. Site visits by international technical assistance consultants and the Bank were not able totake place. In addition, loan disbursements were held up because the suppliers were having difficultiesestablishing letters of credit.

Budgetary constraints during two budget years (1998 and 1999) particularly affected theimplementation of Component I, which required funding to pay the DEOC staff participating in thesurveys. The Government's employment policy encouraging early retirement lessened the multiplier effectsof training, since certain staff took the technical training offered and then left government service. Inaddition, the complex administrative procedures involved in payment and customs clearance in generaldelayed loan disbursements, but also delayed, and sometimes even prevented, the procurement of certaintechnical periodicals and journals for INRE's library.

5.3 Factors generally subject to implementing agency control:

Because of the lack of coordination, combined with a string of changes in DEOC and INREprofessional staffs, Component I was only partially satisfactory. As noted above (see end of paragraph3.3), technical implementation and budgetary control under Component 1, entrusted to different entities,disrupted the progress of that component's activities. The slow-down in the management and overallproject coordination on the part of the Project Management Unit also affected project implementation, asthe difficulties were not acknowledged in time for corrective decisions to be made.

5.4 Costs andfinancing:

The total project cost was estimated at US$64 million, of which US$40 million was financed bythe Bank loan and the remainder by the Algerian Government. During implementation, the unused fundsresulting from the decision to drop the laboratory products storage and packaging unit and reduce spareparts requirements (resulting from an initial overestimation) were reallocated to an additional tranche ofpedagogic materials for 193 primary schools and 76 secondary schools whose facilities had been damagedby acts of terrorism.

At the Goverrnent's request, US$5 million was canceled from the loan in August 2000 following areview of disbursement forecasts. At the closing of the project books (31 July 2001), after the 4month-grace period, a total of US$30.7 million had been disbursed. The undisbursed balance will becanceled. The loan withdrawal rate was primarily affected by the following factors: (i) complex paymentprocedures (see 5.2 above) and (ii) the cancellation of two contracts for pedagogic materials under theadditional round of procurement, since the suppliers were unable to respect the delivery date imposed bythe loan closing date.

6. Sustainability

6.1 Rationale for sustainability rating:

The sustainability of this project is probable because the quality and efficiency of education arecentral elements of the reform process introduced by the National Commission on Education Reform andunder way since May 2000. Moreover, the Government is aware that the actions supported by the projectare essential to continued improvement of the system. They contribute to an understanding of the causes ofdeclining quality and efficiency so that the appropriate steps may be taken to bring about improvements.

- 8 -

The Govermment has also asked the Bank to undertake a review of public expenditure in the social sectors,to help it identify institutional reforms likely to improve efficiency and enable it to rationalize resourceutilization. This report was completed in August 2001.

More specifically, the project's sustainability is already assured by INRE, which has expanded itswork program and textbook analysis and revision to other subjects. DEOC will continue to work ondeveloping a system for assessing learning achievements, and will continue to produce quality andefficiency indicators, since these tasks fall within the scope of its mission. The project's sustainabilitywould be enhanced by the development of a strategy in pedagogic equipment procurement and maintenancewhich will redefine CAMEMD's role within the context of decentralization and of the expected role of theprivate sector. In addition, although the project did not contemplate an impact evaluation study towards theend of the project (see paragraph 3.5), the MNE has included this study in its own program. Likewise, ithas plans to train additional specialists in the development of educational indicators.

6.2 Transition arrangement to regular operations:

The project activities are already incorporated in the MNE's normal program, so that no specialtransition arrangements are required. They will continue as at present, and will be financed out of theMNE budget. The education sector, being part of human resource development, falls within theGovernment's budgetary priorities for the 2001-2004 economic recovery program. The Bank's missionwas given the following commitments by the implementing agencies.

Continuation of the learning assessment work With respect to the project's two principalactivities, the following measures will be undertaken: (i) first, the unfinished task of analyzing the surveyresults and preparing the reports will be completed. The task of establishing a system of leamingassessment will definitely be the next step, since this is part of DEOC's mission; (ii) the report on educationsystem indicators will be updated every two years and distributed to the education community and thedecision-makers.

Continuation of pedagogic research. As discussed above, the fact that assessment and revision ofthe programs have been extended to other disciplines is evidence that the project activities have alreadybeen incorporated into INRE's normal program.

Improvement of the system of pedagogic equipment procurement and maintenance.Implementation of this component has already helped CAMEMD begin to develop capacities in planningand management of procurement work. In addition, a new management structure is in place, demonstratinga commitment to redefining CAMEMD's role, with the purpose of making the procurement andmaintenance system more efficient. The two studies establishing the strategy for the system will beundertaken during 2001. One of the studies will outline a strategy for school equipment procurement andmaintenance, and the other will develop a maintenance master plan.

7. Bank and Borrower Performance

Bank7.1 Lending:

During project preparation, the Bank's performance was satisfactory. The project team had arange of pertinent qualifications, with expertise in the areas of learning assessment, pedagogic materials,and project management. Project design was based on an in-depth study of the basic and secondaryeducation sector, and reflected government priorities. Therefore, it focused on key elements for

-9-

improvement of the quality and efficiency of the education system. The project team clearly identified theprincipal risks in terms of the country's political and economic situation and the implementation capacitiesof the government agencies, and thus balanced the design of the project accordingly. In addition, since thetextbook publication and distribution system represents a determining factor in the quality and efficiency ofthe sub-sectors, and wishing to maintain the dialogue with the Government on this matter, the project teamincluded in a supplemental letter to the loan agreement the legal requirement that a study be conductedconceming the assessment of that system. In retrospect, the terms of reference of that study should havebeen a condition for loan effectiveness. Also, it would have been useful to include an impact evaluationstudy towards the end of the project in order to assess the project's achievements and identify thechallenges still remaining (particularly since the country's difficult security circumstances preventedsupervision missions and the mid-term review mission from taking place within Algeria).

The Bank prepared an appraisal report clearly describing the problems of the sub-sectors and theactivities supported by the project. The report also contains a detailed implementation timetable for projectactivities and performance indicator matrices. These indicators would have been more useful had they beenmore clearly defined, with a distinction between indicators measuring outputs and those measuring theproject's outcomes and impact.

7.2 Supervision:

Bank performance was only moderately satisfactory. This outcome was partly affected byAlgeria's difficult security circumstances, which hindered project supervision and led to high staff turnover.Faced with the circumstances, the Bank adopted innovative solutions, organizing missions outside Algeriaand calling on the services of local consultants. Despite those efforts, the continuity of the work was notsecured and decisions could not be made rapidly when difficulties arose, particularly during implementationof Component 1. Turn over in task management of the project happened several times following loaneffectiveness, and information sharing among the different teams was poor. Inadequate attention wasgiven to the monitoring and analysis of activities related to the project's "qualitative" aspects. In addition,the Bank failed to enforce the deadlines for two studies stipulated in the loan agreement, namnely a study onthe maintenance and safety of pedagogic materials in the schools (December 31, 1994) and a studyconceming assessment of the textbook publication and distribution system (December 31, 1995).

7.3 Overall Bank performance:

Overall, the Bank's performance was satisfactory, despite its only moderately satisfactoryperformance for project supervision. The Bank continued to support the Government during the difficultcircumstances, adopting innovative solutions to get around the difficulties. With government support, itorganized missions outside Algeria, called upon the services of local consultants, and also had staff go toAlgeria when security conditions there were not favorable.

Borrower7.4 Preparation:

The Government's performance was satisfactory. Its participation in the Bank's sector workresulted in production of the study entitled "Improving Quality and Efficiency in Basic and SecondaryEducation in Algeria - Report No. 8912-AL," which identified the priorities endorsed by the project. The

- 10-

Government also participated actively at the project design stage, which resulted in a tightening of theproject's scope to limit it to the immediate needs of the sub-sectors.

7.5 Government implemnentation performnance:

The Government's performance was only moderately satisfactory. Although the Governmentremained committed to the project's overall objectives, and collaborated fully with the Bank during difficultperiods, it did not meet its contractual obligations. The deadlines for two required studies were not met (astudy on the maintenance and safety of pedagogic materials, and a study concerning assessment of thetextbook publication and distribution system). In addition, no decision was taken to solve the conflictbetween DEOC and INRE, which persisted throughout the project and hampered the achievement of resultsunder Component I. Also, the lack of government counterpart funds restricted project activities during1999.

7.6 Implementinig Agency:

The implementing agencies' performance was highly satisfactory. Despite such unfavorablefactors as a succession of changes of management, a lack of budget resources, and a scaling down ofinternational technical assistance, the implementing agencies remained committed to their task andsucceeded in accomplishing most of the project activities, putting in practice the experience acquiredthrough the limited international technical assistance that it was possible to provide. In the same spirit,DEOC and INRE should have been able to override the administrative constraints and to assure a betteroutcome for Component I.

The overall coordination of project activities by the Project Management Unit (PMU) was lesssatisfactory. Meetings were rarely held with the MNE, and progress reports on the project were preparedless frequently during the final years of the project.

7.7 Overall Borrower performance:

Overall performance was satisfactory. Despite difficult security and economic situations, theGovernment remained committed to its policy of improving the education system, and provided a goodframework for project implementation under difficult circumstances. Also, thanks to its help, theorganization of project supervision missions was able to take place outside Algeria, and those Bankmissions that were carried out within the country benefited from heightened attention on the part of theGovernment.

8. Lessons Learned

A. Positive lessons leamed from the project

(i) A project with realistic objectives. designed to fit in with the Government's priorities. has a greaterchance of success. Thanks to those characteristics, this project was able to keep going during difficultperiods, requiring little decision-making on the part of the Government or the Bank.

(ii) A commitment by the government officials directly involved in project activities is essential. Thistype of commitment contributes to the continued pursuit of project objectives and lessens the risks ofchanges of priorities, even at times of changes in the political leadership.

- 1 1 -

(iii) The necessary preparatory work prior to project start-up enables a project to maintain itsmomentun and to move steadily forward as soon as the loan is effective. Within the framework of thisparticular project, invitations to bid were sent to the suppliers of pedagogic materials and requests forproposals for technical assistance and training were issued prior to loan effectiveness. This preparatorywork has significantly contributed to the degree of achievement of the project.

B. Negative lessons learned from the project

(i) It is important to establish distinct output and outcome indicators as well as outcome/impactindicators with a base value and a target value that will help assess the degree of achievement of thedevelopment objectives and the project's sustainability. Those indicators are particularly necessary forprojects with objectives of an intangible nature, such as those relating to the quality and efficiency of theeducation system.

(ii) It is worthwhile to undertake an impact evaluation of the project to provide an objective analysis ofproject achievements and identify the challenges still to be met. These results can serve as a basis for newoperations. Such a study would have benefited this project, particularly since supervision missions were sorestricted.

(iii) Greater attention should have been paid to the enforcement of contractual obligations, with anaction plan for handling cases of noncompliance with the provisions of the loan agreement. Activitiesdesignated contractual obligations are regarded as essential to a project's development. In the event oftardy compliance, or outright noncompliance, action should be taken not only to satisfy the "legal" aspectsof the case but also to ensure that the project's objectives were fully met.

(iv) It is necessarv to strengthen the system for the passing on of information when there are changes inBank proiect supervision teams. This ensures continuation of the dialogue with the borrowers, and reducesthe risk of unintentional changes in the project's priorities. The following approach would be helpful: (i)presence of the person in charge of project preparation during the first year of project implementation; (ii)updating of project progress reports and more complete project status reports, with greater emphasis put onthe achievement of the project development objectives.

C. General lessons. To have a significant impact, projects in the education sector need to place anemphasis on reforming the system. Without far-reaching reforms addressing such issues as quality,efficiency, and equity, the education system cannot continue to advance. Investing in such reforms will be alonger process and will demand greater effort. The following factors should be considered: (i) maintenanceof an ongoing dialogue with borrowers; (ii) involvement of all parties making decisions concerning thesystem or directly benefiting from it (teachers, community, parents, etc.); (iii) in-depth assessment of theorganization and capacity of the institutions; (iv) presence of an efficiently performing system ofassessment of learning outcomes, making it possible to evaluate curricula, textbooks, teaching methods andprograms, examination systems, etc.

9. Partner Comments

(a) Borrower/implenmenting agency:

During its review of the report prepared by the Bank, the Government commented that overall thereport gives an accurate picture of the project outcomes, the stages through which the project passed and

- 12-

clearly identifies the positive and negative factors that affected its implementation and somewhat impactedits outcomes. The MNE also made a few small comments regarding form, which were included in thereport. Two comments are reproduced in detail here, since they express MNE's position and are thereforeworthy of note.

(i) Paragraph 4-2. Outcomes by component. Component I: Contrary to what is stated in the report,the Ministry of Education has always, since the seventies, had sufficient human capacities to carry outleaming outcome assessments (as far back as 1976, trials were carried out by school and job orientationcounselors on tests of learning in several school sub-sectors).

(ii) Paragraph 7-5: Government implementation performance. The Government did not fail to honorits contractual obligations, evidence of this being the study on maintenance entrusted in 1995 to IBF, aBelgian consulting firm. All the equipment surveys were carried out using CAMEMD's own resources in1997 and 1998, as was the data input. It was not until 1999, during the supervision mission, that thedecision to cancel the contract was taken, since IBF had failed to meet its contractual obligations.Furthermore, the Bank indicated that terms of reference would be sent to CAMEMD the next month, i.e. inJune 1999, so that the study could be completed by a domestic consulting firm (see aide-memoire of May1999). As a result of rapid turn over in task management, the terms of reference were never sent.

The copy of the Government's cornments remains in the project files. A version of the Government'sreport is annexed hereto.

(b) Cofinanciers:

Not applicable.

(c) Other partners (NGO.s/private sector):

Not applicable.

10. Additional Information

A brief description of the principal education sector bodies involved in implementation of the project isgiven below.

A. The Directorate of Assessment, Orientation and Communication (DEOC) is the body within theMinistry of National Education that monitors student learning achievements. It has authority over theSchool Orientation and Selection Centers (COSPs), and provides them with technical supervision(definition of the broad lines of the activities program - general guidelines for orientation programs andmethods).

The COSPs are external centers responsible for activities linked to the orientation of studentsenrolled in the different education cycles, particularly at the time of transition from one cycle toanother, the provision of information to students and parents, and the assessment of learningachievements and of teaching and training programs and methods, in liaison with the other bodiesand agencies concerned. At the local level, the COSPs report to the Education Director of therespective wilaya.

The technical personnel of the COSPs comprises school and job orientation counselors who havereceived specialized university training in school orientation and selection and applied psychology.

- 13-

The COSPs have an activities program that covers the school year.

A COSP is headed by a Director, who has direct oversight over the technical staff (school and joborientation counselors) and administrative and service staff. It has an operating budget allocatedby the Government. COSPs are managed under the direct administrative supervision of theEducation Director of the respective wilaya.

B. The National Institute of Education Research (INRE) was established in 1996 to take the placeof National Pedagogic Institute (IPN). INRE is responsible for pedagogic research, the continuousassessment of the education system, and the preparation of instructional materials. INRE alsoassess the impact of education policy. INRE is administered by an orientation council andmanaged by a director general, and has a scientific council and a committee responsible forapproving approve pedagogic materials and supporting resources.

C. The National Center for Procurement and Maintenance of Equipment and PedagogicMaterials (CAMEMD) was set up in 1986. Its supervisory authority is the MNE and it isfinancially autonomous. It is responsible for the procurement and maintenance of pedagogicequipment. It works in close collaboration with the Planning Directorate, which it responsible forregistration and monitoring of equipment procurement operations and standardization. CAMEMDalso trains technicians (pre- and in-service training courses) and monitors pedagogic equipmentuse. CAMEMD maintains links with the wilaya through its eight regional antennas.

- 14 -

Annex 1. Key Performance Indicators/Log Frame Matrix

Outcome / Impact Indicators:

Indicator/Matrix Projected in last PSR Actual/Latest Estimate

Component 1: Data non available Component 1:1.1 Improved measurement techniques of 1.1, 1.2, 1.3 Results were not totallypupils' leaming achievement in the DOE at achieved, as the expected activities remainedcentral level and in six participating COSP. uncompleted.

1.2 Introduction of improved local training 1.4 Education indicators were produced buttechniques for inspectors and teachers in have not been used as assessment tools.testing, assessment and orientation andselection of pupils in the six pilot wilaya.

1.3 Expansion of the assessment system toseven additonal wilaya which are to be addedand financed by the MEN in a second phase(expected 1997-98).

1.4 Improved quality and relevance ofcurricula in the four covered subjects ascompared to international standards (OECDeducation indicators model).Component 2: Component 2:2.1 Improved measurement techniques of 2.1 Assessment techniques of INRE werepupils' learning achievement with appropriate improved through intemational technicalpedagogic research at classroom and school cooperation and training seminars, therebylevel and at central level and in the six enabling INRE to expand its pedagogicparticipating COSP. research work to other subjects (see below)

and to contribute to the work of the National2.2 Improved feed-back to and from Commission for the Education Systeminspectors and teachers on pedagogic Reform.research results in the topics covered.

2.2 This was achieved through numerous2.3 Improved linkages between curricula and information seminars and the publishing ofrevised texts for the prescribed schoolbooks study reports broadly disseminated to theat 1, 2, and 3 cycles of basic and secondary educabon community. (documents availablelevels in Arabic and Physical Sciences. in the Bank's archives).

2.4 Development of improved teacher 2.3 This was achieved and went beyonddocumentation for Arabic and Physical project expectations as the INRE workSciences at all levels of basic and secondary reached more topics than prescribed by theeducation. project.

2.5 Improved availability of pedagogic 2.4 The following results assessmentbackground informaton and improved reports were produced : (i) Arabic (grade 1,functioning of the INRE library. 3, 6, and 9 BE and 3 SE); (ii) Mathematics

(grade 9 BE, 3 SE); (iii) Physics andChemistry (grade 1 and 3 SE); (iv) French(1 st foreign language at all cycles, 2ndforeign language at grade 8 BE and 1 SE);(v) English (grade 4 and 9 BE and 3 SE).

2.5 The INRE library modernization andstrengthening in terms of documentationcontributed to increased availability ofpedagogic information. The Bank's missionnoticed the presence of education sciencestudents and researchers in the library.

Component 3: Component 3:3.1 Improved provision of pedagogic 3.1 All the materials expected under thematerials for 14,000 basic and secondary project were purchased and distributed toschools, as follows: school facilities. Field visits could not beDec. 1994 - 10% covered carried out during the ICR mission but theDec. 1995 - 25% covered mission was informed that the 100%Dec. 1996 - 40% covered coverage rate has been reached.

- 15-

Dec. 1997 - 60% coveredDec. 1998 - 80% covered 3.2, 3.5 Field visits could not be carried outDec. 1999 - 90% covered during the ICR mission. The assessmentDec. 2000 - 100% covered tools have not yet been used to assess the

quality of the system. However, during project3.2 Better use of pedagogic materials by implementation, an assessment surveyteachers in classrooms among teachers and students showed that

the availability of materals enabled an3.3 Better training courses in maintenance improved application of the program andand securty for technicians progress, and a greater interest of students

3.4 Improved provision, repair and in courses and application sessions.maintenance services by CAMEMD and its 3.3 Overseas training activities took place.regional antennas for all basic and secondary The trained technicians organized seminarsschools. and workshops for the CAMEMD staff as

3.5 Better teachers education and improved well as for the teachers in topics using thoseequipments.

pupils' leaming achievements.3.4 CAMEMD services lack effectiveness.School facilities sometimes rely on the prvatesector for repair services.

Output Indicators:

Indicator/Matrix Projected in last PSR Actual/Latest Estimate

Information source: Section "Current Statusof the Project" in last PSR (5 March 2001).

Component 1:Component 1: 1.1 Six surveys have been canied out: Component 1:1.1 Nine surveys activities on a - Arabic Language grade 3 BE 1.1 Eight surveys have been carried out:representative sample: - Arabic Language grade 9 BE - Arabic Language grade 3 BE-4 basic subjects (Arabic, Sciences, French) - French Language grade 9 BE - Arabic Language grade 6 BE-4 school levels (grade 1, 2, and 3 in BE - Natural Sciences grade 1 SE - Arabic Language grade 9 BEand SE). - Mathematics grade 9 BE - Mathematics grade 3 BE

- Mathematics grade 1 SE - Mathematics grade 6 BE1.2 (i) Communicaton of the results to the - Mathematics grade 9 BEteachers and the inspectorate. - Mathematics grade 1 SE(ii) Diffusion of the final report to the - French Language grade 9 BEeducation community.(iii) Short publications on assessment tools to The survey on Sciences in grade 1 SE didteachers. not take place.

1.3 (i) Creation of data banks and A national seminar was organized in Maystandardized test items. 2000 for the pre experimentation phase.(ii) Pedagogic support to teachers onpedagogic assessment procedures. 1.2, 1.3 The system results and tests

1.4 Seven individuals have benefited from a diffusion did not take place.1.4 (i) Overseas training of staff (14 training in Canada for 4 to 6 weeks.individuals) in designing, implementing, and 1.4 Fourteen individuals benefited from a 2evaluating internal efficiency studies. phases training, in 1995 and 1996.(ii) Local training of staff (about 1560inspectors). 1.5 Inidicators for 1999 and 2000 have been

established.1.5 (i) Setting up of a quality indicators 1.5 Education system quality and efficiencysystem aimed at providing useful, reliable, indicators, based on the OECD model wereand viable information, on the schools set up for 1999 and 2000.system.(ii) Local training of staff (inspectors,teachers, COSP counselors, universitypsycho-pedagogy specialists).

Component 2: Component 2: Component 2:2.1 To establish a protocol for pedagogicresearch.

- 16 -

2.2 Improve coherence between programs 2.2 In addition to Arabic Language andobjectives and subjects content in Arabic and Physical Sciences, the programs concerningPhysics for grade 1, 2, and 3 SE. the following subjects were evaluated:

mathematics, English, French, biology,

2.3 Overseas training in programs and history, geography, and philosophy (inschoolbooks evaluation. 2.4 The results of the programs and process).

2.4 Dissemination of brochures on the schoolbooks evaluation have been produced 2.3 Two teams of eighteen individuals wereconclusions of pedagogic research seminars in a memorandum. 6,000 copies were trained in Canada in 1995.aimed at inspectors and school managers in disseminated to the educaton community.the wiaae 2.4 A memorandum on programs evaluation

the wilayate. (6,000 copies) was diffused. An important2.5 Acquisition of basic documentation number of seminars was organized and(about 5.000 copies). brochures on the conclusions of these

seminars were also published.2.6 Subscription to foreign specialized (documentation available within the anchivemagazines. system of the Bank.)

2.5, 2.6 The INRE library benefited from anasset of 5,000 books. Subscriptions tospecialized magazines were partial due tocumbersome administrative procedures.

Component 3: Component 3: Component 3:3.1 Purchase of 13,000 compendiums for 3.1 The materials were purchased and 3.1 The expected materials were totally10,500 built primary schools, plus 2,500 to distributed. An additional tranche of materials purchased and distributed. An additionalbe built; provision of scientific equipments for is in the process of being acquired. tranche of matenals for facilities having750 basic schools under construction; suffered from acts of terrorism is in process.provision of scientific equipments for 250 This tranche will be partially executed due tohigh schools under construction. the closing of the loan.

3.2 (Indicators are not mentioned here as thesub-component related to the establishmentof a laboratory products conditioning unit has 3.3 Following the cancellation of the contract

bee cneld).. olwgtecnelao ftecnrcbeen cancelled). with the consulting firm for the initial study, 3.3 No progress on the study.

3.3 To conduct the study on maintenance the study was split into two studies on: (i) aneeds. school materals purchase and maintenance

strategy, and 3.4 Activity achieved, releasing a balance3.4 Purchase spare parts. (ii) the development of a master plan for which was used to purchase the additional

maintenance. No action was undertaken tranche of materials mentioned above.3.5 Overseas training of 16 high technicians concerning the execution of (i) and (ii).and 10 CAMEMD executives. 3.5, 3.7 Overseas training took place for 27

executives (9 in information technologies and3.6 Increase in personnel (about 30 management; 10 in machines/tools; 4 inindividuals) in regional annexes. reeling/coiling; and 4 in electronics). As a

result, seminars and workshops were3.7 Skill upgrading of about 100 high organized for other CAMEMD staff andtechnicians and CAMEMD engineers as well teachers in subjects benefiting from theas about 1,000 laboratory managers and equipments.29,000 teachers.

3.6 Infommation non available.3.8 Acquisition of technical documentation.

End of project6/13/2000

- 17-

Annex 2. Project Costs and Financing

Project Cost by Component (in US$ million equivalent)Appraisal Actual/Latest Percentage ofEstimate Estimate Appraisal

Project Cost By Component US$ million US$ millionPilot national assessment system 3.30 3.60Pedagogic research 2.00 2.90Improvement of supply and use of pedagogic material 51.00 50.60

Total Baseline Cost 56.30 57.10Physical Contingencies 2.00 89.2Price Contingencies 5.70 89.2

Total Project Costs 64.00 57.10Total Financing Required 64.00 57.10

Including contingencies

Project Costs by Procureme nt Arrangements (Appraisal Estimate) (US$ million equivalent)

Procurement MethodExpenditure Category ICB NCB Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 1.60 1.60(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 32.70 0.00 4.00 0.00 36.70(32.50) (0.00) (4.00) (0.00) (x36.50)

3. Services 0.00 0.00 6.60 0.00 6.60Technical coop. & Trng. (0.00) (0.00) (3.40) (0.00) (3.40)4. Miscellaneous 0.00 0.00 19.10 0.00 19.10Operating Costs (0.00) (0.00) (0.10) (0.00) (0.10)5. Miscellaneous 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) (0.00) (0.00) (0.00)6. Miscellaneous 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) (0.00) (0.00) (0.00)Total 32.70 0.00 29.70 1.60 64.00

(32.50) (0.00) (7.50) (0.00) (40.00)

Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent)

Procurement MethodExpenditure Category ICB NCB Other2 N.B.F. Total Cost

1. Works 0.00 0.00 0.00 0.00 0.00(0.00) (0.00) (0.00) (0.00) (0.00)

2. Goods 33.40 0.00 2.00 0.00 35.40(31.30) (0.00) (1.20) (0.00) (32.50)

-18 -

3. Services 0.00 0.00 2.60 0.00 2.60Technical coop. & Trng. (0.00) (0.00) (2.60) (0.00) (2.60)4. Miscellaneous 0.00 0.00 19.10 0.00 19.10Operating Costs (0.00) (0.00) (0.00) (0.00) (0.00)5. Miscellaneous 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) (0.00) (0.00) (0.00)6. Miscellaneous 0.00 0.00 0.00 0.00 0.00

(0.00) (0.00) (0.00) (0.00) (0.00)Total 33.40 0.00 23.70 0.00 57.10

1 (31.30) (0.00) (3.80) (0.00) (35.10)

"Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies.21Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff

of the project management office, training, technical assistance services, and incremental operating costs related to (i)managing the project, and (ii) re-lending project funds to local government units.

Project Financing by Component (in US$ million equivalent)Percentage of Appraisal

Component Appraisal Estimate Actual/Latest Estimate

Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF.Pilot national assessment 1.40 2.30 1.70 1.90 121.4 82.6system

Pedagogic research 2.00 0.30 2.30 0.60 115.0 200.0

Improvement of supply and 36.60 21.40 31.10 19.50 85.0 91.1use of pedagogic materials

- 19-

Annex 3. Economic Costs and Benefits

Not Applicable

- 20 -

Annex 4. Bank Inputs

(a) Missions:Stage of Project Cycle No. of Persons and Specialty Perfornance Rating

(e.g. 2 Economists, I FMS, etc.) Implementation DevelopmentMonth/Year Count Specialty Progress Objective

Identification/Preparation12/1991 4 1 ASS, I ED, I 00, I HR

Appraisal/Negotiation11/1992 3 1 ASS, 1 ED, I 00,

Supervision9/1994 2 1 ASS, I ED S S

4/1995 2 2 ED S S10/1996 2 1 AC, S S12/1998 1 1 HR S S5/1999 1 I ED S S11/1999 3 1 HR, 1 IMP, I MON U S6/2000 1 1 MON S S5/1993 2 1 ASS, I ED

ICR4/2001 2 1 00, 1 OA

AC: Architect; ASS: Assessment Specialist, ED: Education Specialist; HR: Human Resources Spec.; MON:Monitoring Spec.; 00: Operations OfficerNote: 5/1993 Rating for: Impl. Prog and DO = 1

(b) Staff:

Stage of Project Cycle | Actual/Latest Estimate

No. Staff weeks US$ ('000)Identification/Preparation 20 158.5

Appraisal/Negotiation 10 60.0Supervision 18 163.7ICR 5 30.0Total 53 412.2

- 21 -

Annex 5. Ratings for Achievement of Objectives/Outputs of Components(H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable)

RatingL Macro policies O H OSUOM O N * NAO Sector Policies OH OSUOM ON * NAE Physical O H *SUOM O N O NAE Financial O H OSUOM O N * NAEl Institutional Development O H O SUO M O N 0 NAE Environmental O H OSUOM O N * NA

Social] Poverty Reduction O H O SU O M O N * NA

O Gender O H OSUOM O N * NAEl Other (Please specify) O H OSUOM O N * NA

O Private sector development 0 H O SU O M 0 N * NAEl Public sector management 0 H O SU O M 0 N * NAO Other (Please specify) 0 H O SU O M 0 N * NA

- 22 -

Annex 6. Ratings of Bank and Borrower Performance

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory)

6.1 Bankperformance Rating

E Lending OHS Os Ou OHUE Supervision OHS OS O U O HUO Overall OHS Os O U O HU

6.2 Borrower performance Rating

OJ Preparation O HS * S O U O HUO Government implementation performance 0 HS 0 S 0 U 0 HUC0 Implementation agency performance O HS O S O u O HUO Overall OHS OS Ou O HU

- 23 -

Annex 7. List of Supporting Documents

Staff Appraisal Report (SAR), Basic and Secondary EducationImproving Quality and Efficiency in Basic and Secondary Education (Report No. 8912-AL)Project Status ReportsSynthese des Rapports d'Evaluation des Programmes Scolaires des Enseignements Fondamental etSecondaire (French and Arabic - Fevrier 1999)Rapport Final dEvaluation - Evaluation de Programme (Decembre 1998)Indicateurs du Systeme Educatif (Mars 2001)Evaluation de Rendement Scolaire (Mai 2000)Evaluation de Rendement Scolaire (Juin 1998)Evaluation de Programmes de l'Enseignement Fondamental et Secondaire (French and Arabic - Mars 1999)Manuel Sciences Physiques (1999)Rapport Final d'Evaluation - Evaluation de Programme - Physique Enseignement Secondaire (Decembre1998)Recueil des Questionnaires d&Evaluation des Programmes et des Manuels Scolaires (1998- 1999)Evaluation des Manuels Scolaires du Secondaire *Physique lere AS* - Rapport Final (Janvier 2000)Rapport d'Evaluation du Manuel de Francais 8eme AF - 2eme langue etrangere (Decembre 1999)Evaluation des Manuels de Chimie de l'Enseignement Secondaire (Fevrier 2000)Evaluation des Manuels de Chimie de l'Enseignement Secondaire (Fevrier 1999)R6sultats de l'Evaluation Pedagogique des Examens de Fin de Cycle (2000)

- 24 -

BORROWER'S REPORT

FROM Unite de Glestin PHONE NO. : +02 232898 Jul. 2B 200 10:42FM1 P2

Republicue Al§ rienne Democratic ue Populaire

_ . _. A

Minist6re de l'Education Nationale

RAPPORT D'ACHEVEMENT

PROJET D'APPUI A L'ENSEIGNEMENT FONDAMENTAL ET

SECONDAIRE

Pr6t BIRD n° 3573-AL

11 Jullost 2001

FDM Unite de G3estion* PHONE NO. : +02 232836 Jul. 28 2001 10:42RM P3

SOMMAIRE

I - Caractfristiques et rappel succint des objectifs du projet.

2 - RWlisation des objectifs et r6sultats par composante.

3 - Impact de d6veloppement.

4 - Principales causes ayant affect6 l'execution

4 - I - Causes externes

4- 2 - Causes internes lWEes A la gestion et l'execution

5- Performances.

5 - I - De Ia Banque

5 - 2 - De l'emprunteur

6 -Principale leVon tir&e de la realisation de ce Jor projet.

7 - Cout et financement

8- Annexes: 1- Financement du projet par composante

11- Etat des consommations

TU -Situation d6taill6e des consommations par ann6e

FROM Uniite de GestPiiHOIE NO.: +02 232898 Jul. 2 2 1043I P4

I - Carect6ristigues et bref ranDel des obiectifs.

Le projet ( Appui I 'enseignement fondamental et secondaire >>constitue la preiniere 6tape du long processus, clairement enonce et d6jientam6 par le Gouvermement en matiWre d'amnlioration de la qualite desconnaissances acquises et de lefficacit6 du systeme educatif.

Le projet s'articule autour de 3 composantes:

1-1-Composante I : Elaboration d'un dispositif pilote d'6valuation.Cette composante d'un montant de 3,7 Millions de dollars E.U, alWascompris (soit 6% du coOlt estiinatif de base du pret), appuie la conceptionet l'introduction d'un systeme pilote d'cvaluation des connaissances des6l1ves, de la qualit et de l'fficacitt .Elle devait permettre de:

1-1-1 - faire connaitre de nouvelles techniques, rnodalit6s etpratiques d'6valuation parmi les 6ducateurs et les d6cideurs.

1-1-2 - renforcer les capacit6s du controle des connaissances etd'6valuation de la qualite a titre pilote dans les centresd'olientation scolaire et professionnelle de 6 wilayas.

1-2 -Composante IT: o Renforcement du ptogramme de recherchep6dagogique de l'lNRE (ex IPN).

La deuxieme coniiposante d'un montant de 2,3 Millions de dollarsE.U ( 4 % du cout estimatif de base) devait appuyer:

1-2-1 - les activites de formation touchant les programmes derecherche p6dagogique

1-2-2 - les renforcements institutionnels de l'ex.IPN.1-2-3 - I'execution des programmes de recherche sur la conception

et l'evaiuation des programmes d'enscignement et des manuels scolaires.

1-3-Composante III: o Am6lioration de la disponibilite etl'utilisation des mat$riels p6dagogiques. Cout 58 Millions de dollars E.U(90 % du coft estunatif global). Cette enveloppe devait financerl'acquisition des equipements p6dagogiques pour 13.000 colesprimaires, 750 ecoles fondamentales et 250 1yc6es (la trancheCompl6mentaire concerne 193 autres ecoles fondamentales et 76lycees).

- L'acquisition de pieces de rechange pour la maintenance- L'informatisation de la gestion de la maintenance, formation,

etudes et assistance technique.

FROM Lk-iite de Gestion PHONE NO.: +02 2328983 Jul. 28 2001 10:431 P5

2- Rfalisation des obiectifs et r6sultats Par composantes.

L'execution du projet a colncid& avec des periodes tr6s difficilesque le pays a connues (crise econoniique et contraintes financieres, debutd'application des r6formes, problemes s6curitaires, instabilite des cadres,etc...).

Malgr6 cela, le projet a et6 maintenu avec respect des d6lais etex6cution des actions concourrant A lamelioration de la quaIit6 et del'efficacit6 du systrme.

2-1-Comgosante I -

o Elaboration d'un dispositif pilote d' valuation >oCette composante a connu beaucoup de difficult6s qui ont persiste tout aulong de l'ex6cution du projet, et ce malgr6 l'intervention, quasi-permaiente au debut de 1'UnitM de gestion du projel. Ces difficult6s,itaient dCes au fait l'ex6cution technique 6tait dorniciliee A la Directionde I'Evaluation de l'Orientation et de la Communication et la gestionfinanciere assuree par l'Institut National de Recherche en Education et ceen application du decret pr6sidentiel n°: 93-327 du 26-12-1993 portantapprobation de I'accord de pret BIRD. n° 3573-AL.

Malgr6 ces problemes qui ont entrav6 1'execution normale duprojet, l'objectif assigne a la composantc I a 0t6 atteint en partie;l'objectif n'a pas e realise en tolalite du fait du retard cumuie pour lesraisons deja signakes plus haut.

Concernant 1'l1aboration du dispositif pilote d'6valuation, lapassation des epreuves et l'adminiistration des diff6rents questionnairessocio-d6mographiques se sont d6roules selon les chronogramme arrktd,La correction des epreuves et la saisie des donnees ont ete realisees.

Les operations d'analyse, de traitement d'exploitation des risultatssont en cours (apres cl6ture du pret).

De nm8me que la construction des indicateurs avait 6t6 finalisee et ledocument en cours d'impressioni.

Aussi, A travers Ies travaux deja r6alis6es, l'objectif <Acquisitionde 1'experience dans les techniques p6dagogiques, d'dvaluation etl'6Iaboration, des enquetes )> a dte largement atteint.

Le systene pilote sera etabli et toutes Ics dispositions seront prisespour que le travail soit poursuivi et finalise.

PHOE NO.: +02 232898 Jul. 28 2001 10:44M1 P6FROM: Unite de Gestion

2-2 - Comgosante Il : Recherche pedagogique

La realisation de cette composante a 6t6 plus que satisfaisante etl'objectif attendu a 06 d6passd puisque les iquipes de l'lNRE ont rialiseplus que ce qui etait inscrit et retenu dans le projet.

Les resuitats aussi bien quantitatifs que qualitatifs ont 6td plus quesatisfaisants.

L'dvaluation des programmes a 6t6 etendue A d'autres disciplines(seules l'Arabe et les Sciences 6taient concern6es), elle a touch6 laPhilosophie, I'istoire, la Geographie, les Math matiques et la langueFran9aise.

De meme, l'evaluation des manuels scolaires a touche d'autrestitres non retenus pour le projet.

Aussi, nous pouvons estimer que cette composante a contribu6largement au renforcement des capacites institutionnelles de l'INRE qui aet6 en mesure de contribuer et mettre i la disposition de la commissionnationale de reforme du systeme 6ducatif tous ses travaux de recherchep6dagogique.

L'exp&rience acquise a et6 reinvestie: I'extension a d'autresprogrammes et d'autres manuels a ete entamee et de nouvelles equipesont 06 mises en place.

2-3- omvosante 111: Am6Iioration de I'approvisionnement :cnmat6riels didactiques o

La quasi totalit6 des acquisitions en materiels pedagogiques pr6vuspar cette composante a 6W r6alis6e et le taux de couverture des ecoles en6quipements pedagogiques a depasse les 100 %. Les equipements de latrancbe compl6nientaire de 193 Ecoles Fondamentales et 76 lyc6es ontetd acquis et instaills, coinie pr6vu avant la date de cl6ture. Toutefois,le decalage entre l'acquisition et la rentree scolaire est dii aux imp6ratifscontractuels et A 1'6ch6ance de cl8ture du pret.

L'6tude sur la maintenance et la securite n'avait pas 6te realisiecornme prevu au 31 Decembre 1994.Elle a connu beaucoup de retard,pour enfin Stre comniandee en 1995 A un bureau d'6tudes Beige (IBF).Les travaux d'enquete et de saisie avaient dt6 r6alis6s par les propresmoyens du CAMEMD en 1996 et 1997, la situation s6curitaire del'6poque ne favorisant pas la poursuite du travail par le bureau d'&tudes.Le contrat avait &6 r6silie en 1999. L'objectifde 1'etude avait 6t6 red6finiet 1'etude scindce en deux. Toutefois, la Banque s'6tait engag6e atransmettre au CAMEMD, avant la fin Juin 1999, un mod6le de terme dereferences pour aider le CAMEMD (Cf. Aide-Memoire Mai 1999).

PROM Unite de Gesticn PHONE NO. : +02 232898 Jul. 283 2001 10:45ft1 P7

Sur insistance de l'UGP et suite au changement de responsableopdr6 au niveau du CAMEMD, le volet maintenance a ete inscrit commetr6s prioritaire dans le programme d'action de CAMEMD et les deuxetudes de la maintenance ont &t6 retenues pour etre achev6es dans lesmeilleurs d6lais.

3- Impct de dielopDement:

11 demeure certain que le projet a contribue dans une tr6s large partau d6veloppement des capacites nationales.

Les exp6riences accumiulees et les capacit6s acquises par les cadresde secteur sont palpables. Ceci a 6te possible grAce a 1'engagement desdecideurs et leur volont6 d'am6liorer la qualit6 et l'efficacite du systeme6ducatif.

Sous la conduite de la DEOC et A travers tous les educateurs dessix wilayas pilotes, les m6thodes de contrBle et d'enquote et la validationdes r6sultats ont 6t6 relativement acquises.

Pour l'INRE, les resultats sont plus probants. Les techniques deconception et d'6valuation des programmes et des manuels scolaires sontmaltris6es .

La demultiplication des experiences au niveau de terrain a port6d6ji ses fruits.

Quant au CAMEMD, les cadres ont acquis une riche expdriencedans la gestion du projet. uls planifient, suivent les diff6rentes etapes del'acquisition, la livraison des materiels et la foination a leur utilisation.

4- PrinciRaux 616ments alant affect: 1'excution:

II y a lieu de signaler que 1'ex6cution des deux premierescomposantes du projet A ddniarr avec une ann&e de retard. Ce retardest dCi principalement aux diff6renits changements de cadres qu' & connule Ministere (Administration Centrale et Centres Nationaux) en 1994 et1995.

Par ailleurs, 1'execution du projet a connu tout au long de lap6riode des retards des A des facteurs aussi bien externes qu'internes etles r6sultats finaux en ont &6 affect6s.

- Facteurs externes:

Le ddroulement du projet a W queique peu entrav6 par:

FROM Unite de Gestion PHONE NO.: +02 23289E Jul. 28 2001 10:45fM P8

- Les contraintes financi6res qu'a coniu le pays suite aux effetsde la crise economique et la chute du prix du p6trole.

- Les p6riodes difficiles daes aux conditions s6curitaires qu'atraverse' le pays:

- Les changenients successifs qu'ont connu les 6quipes desuivi du projet ai niveau de la Banque.

- Les lourdeurs administratives des services des douanes etdes services de la Banque Algerienne de Developpement.

r La meconnaissance des m6canismes et proc6dures degestion de Ia Banque.

- Et A un degrd moindre, les difficult6s de communicationavec la Banque et les bureaux d'6tudes etrangers (dont le CIDE),difficultes qui resident en des encombreme4its tres frequents dest6l6communications, journ6es ouvrables et nombre d'heures permettantles contacts en tenant compte des d6calages horaires.

- Facteirs internes:

Les changements successifs des cadres et personnels opr&6s auniveau du Ministere et des centres nationaux (INRE et CAMEMD, entreauti-es), ainsi que le depart premature A la retiaite de cadres impliqu6sdans le projet ont affect6 serieusement le bon d6roulement de I'ex6cutiondu projet.

- Les difficultes qu'a connu I'Unite de gestion du projet dans lacoordination ont elles aussi retard6 1'exicution et affecte les resultats deia composante J notamment.

5- OualitW de 'intervcntion:

Tout au long de 1'execution du projet, la qualite de l'intervention3aussi bien de la Banque que ce]le du Gouvernement a W sans aucundoute tr6s positive.

C'est ce qui a fait << durer >> le projet jusqu'& son tenne,comparativement aux autres proJets d'autres secteurs et notammentdurant les p6riodes tres difficiles allant de 1993 a 1998.

Toutefois, si les performances de la Banque ont 6te tressatisfaisantes, il y a lieu de signaler l'instabilit6 des responsables quipilotaient le projet au niveau de la Banque (pas moins de 5 experts ontpris, successivement, en charge le projet appui i I'enscignementfondamental et secondaire), ce qui a cause des retards dans 1'ex6cution du

lt S:

FROM Unite de Gestion PHONE NO.: +02 232898 Jul. 28 2001 10:461M P9

En ce qui concerne l'emprunteur, il y a lieu de noter que malgro lescontraintes financieres et securitaires, le Gouvernement n'a menag6aucuni effort pour solutionner tous les problemes et surtout donner lapriorit6 au projet dans le cadre du budget d'F.quipement.

Par ailleurs, malgre les contraintes, le Gouvememnent a maintenuglobalement ses engagements avec la Banque.

6 - Princinale lecon tir6e de la rkalisation du proict

Globalement les experiences du projet ont & trks positives aussibien dans la mobilisation des moyens pour le maintenir dans les delaisimpartis A toutes les actions ayant contribu6 i 1'ex6cution << a peu presnormale ») du projet, que dans les techniques et connaissances acquises.De par lVengagement de tous les cadres directement concern6s par lesactivites du projet qui, malgr6 les contraintes et les changements, ontmaintenu le cap.

7 - Financement - Co&ts et D&caissements

Le coOt total du projet a &t6 estim6 initialement a 64 millions dedollars U.S dont 40 millions de dollars U.S a financer par la Banque et 24millions de dollars U.S par le Gouvernement.

Durant l'execution, un reliquat de 11 millions de dollars U.S avait&6 d6gag& Ce reliquat resultait de l'abandon de la realisation de l'unit6de conditionnement de produits chimniques ( eu 6gard aux r isques qu ellepresentait et les besoins minimnes en produits dans les 6tablissements ) et ala r6duction des besoins en pieces de rechange pour la maintenance(besoins surestim6s initialemeent).Le reliquat a ete renegocie avec la Banque et affecte en partie ( 6 millionsde dollars) pour l'acquisition d'une tranche compl6mentaire de materielspedagogiques au profit de 193 ecoles fondamentales et 76 lycees dont lesequipeinents ont Wt6 d6terior6s suite a des actes de terrorisme. Uneenveloppe de 5 millions de dollars U.S a 6t6 annulee du pret en Juillet2000.

A la date de la cl6ture du pr6t (Mars 2001), pras de 33,040millions de dollars U.S avaient ite d6caisses, soit 82.58 % du pr6t initial(chiffe incluant les engagements du CAMEMD pour la tranchecompl6mentaire ).

Les trois tableaux retra9ant la situation des couts et dedecaissemients par composante et annde d'cxercice sont joints en ainexe.

PHONE NO.: +02 232898 Jul. 28 2001 10:46AM1 P10FRC1M Unite de Gestion

CONCLUSION

Malgre les contraintes et les difficult6s rencontr6es tout au long del'execution du projet, les objectifs fix6s ont &6 relativement atteiiits dansla mesure ou' :

la mise en place et la constitution d'6quipes de sp6cialistes ayantacquis les coniaissances et les techniiques de 1'6valuation ont WtFrdalisees.

Ia recherche pedagogique s'est renforcee par la formation despecialistes qui ont d6ja itendu leurs activites a des disciplines et desmanuels autres que ceux retenus dans le projet.

I'approvisionnement et l'utilisation des equipements pedagogiquesau sein des etablissements scolaires se sont amdlior6es. Le rendement etles resultats aux examens scolaires &e sont nettemnent am6lior6s.

Par ailleurs, it y a lieu de noter qu'aucune personne n'a ete recrut6especialement pour le projet. Ce sont les cadres du secteur engages dans leprojet qui ont assure les activites du projet en plus de leur programme detravail.

Les experiences acquises ont 6te d'une grande richesse aussi biensur le plan de la gestion que sur le plan technique.

Aussi, I'amelioration de la qualite et de l'efficacite du systEmeeducatif continue a avoir la priorit' du secteur et constitue la pierreangulaire de la reforme du systOmc educatif.

Ainsi les manques signales et les travaux d'achevement et definalisationi du projet seront pris en charge dans le programme normal dusecteur.