document of the world bankdocuments.worldbank.org/curated/en/386861468017349143/pdf/icr9500... ·...

Document of The World Bank

Report No: ICR0000950

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-45910)

ON A

LOAN

IN THE AMOUNT OF US$90 MILLION EQUIVALENT

TO THE

STATE OF CEARÁ

WITH THE GUARANTEE OF THE

FEDERATIVE REPUBLIC OF BRAZIL

FOR A

CEARÁ BASIC EDUCATION QUALITY IMPROVEMENT PROJECT

December 17, 2008

Human Development Sector Management Unit Brazil Country Management Unit Latin America and the Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective October 23, 2008)

Currency Unit = Real (R$)

R$2.20 = US$1.00

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APRECE - Ceará Mayors Association (Associação dos Prefeitos do Ceará) ART - Accelerated Results Together BOS - Basic Operational Standards CAGE - SEDUC’s Education Coordination and Management Unit CAS - Country Assistance Strategy CPS - Country Partnership Strategy CDTP - SEDUC’s Technical-Pedagogical Development Unit CENPEC - Education and Culture Research Center (Centro de Pesquisa em Educação e Cultura) CLS - SEDUC’s Procurement Team (Comissão de Licitação da SEDUC) CPPE - SEDUC’s Education Planning and Policy Unit CRA - Learning Resource Center (Centro de Recursos de Aprendizagem) CREDE - Regional Education Development Center (Centro Regional de Desenvolvimento da Educação) ECD - Early Childhood Development (Educação Infantil, 0-6 anos) FUNDEF - Education Development and Teacher’s Incentive Fund (Fundo de Manutenção e Desenvolvimento do Ensino

Fundamental e de Valorização do Magistério) FUNDESCOLA - School Improvement Program (Programa do Fundo de Fortalecimento da Escola) GEP - Project Executive Group (Grupo Executivo do Projeto) IDB - Inter-American Development Bank IDM - Municipal Development Index (Indice de Desenvolvimento Municipal) IPEA - Applied Economic Research Institute (Instituto de Pesquisa Econômica Aplicada) IPLANCE - Ceará Planning Institute (Fundação Instituto de Planejamento do Ceará) LDB - National Education Law (Lei de Diretrizes e Bases da Educação Nacional) MEC - Ministry of Education (Ministério de Educação) MIS - Management Information System NEBE - Northeast Basic Education Project PBF - Basic Operational Standards (Padrões Básicos de Funcionamento) PDE/GIDE - School Development Plan (Plano de Desenvolvimento da Escola) PNLD - National Textbook Program (Programa Nacional do Livro Didático) PROARES - Social Reform Support Program (Programa de Apoio às Reformas Sociais) PRODEM - Municipal Education Development Program (Programa de Desenvolvimento da Educação Municipal) PROINFO - National Education Informatics Program (Programa Nacional de Informática na Educação, MEC) PROMED - IDB Expansion and Improvement of Secondary Education Project (Projeto de Expansão e Melhoria do Ensino Médio) PROURB - Urban Program of the State of Ceará SAEB - Basic Education Evaluation System (Sistema de Avaliação da Educação Básica) SAP/SADRE - Pedagogical Monitoring and Support System (Sistema de Acompanhamento Pedagógico) SAS - State Labor and Social Action Secretariat (Secretaria Estadual de Trabalho e Ação Social) SECITECE - State Science and Technology Secretariat (Secretaria Estadual de Ciência y Tecnologia) SEDUC - State Basic Education Secretariat (Secretaria Estadual de Educação Básica) SEPLAN - State Planning and Coordination Secretariat (Secretaria Estadual de Planajamento e Coordenação) SESA - State Health Secretariat (Secretaria Estadual de Saúde) SIP - School-Initiated Project (Projeto de Inovação Escolar) SPAECE - State Education Evaluatin System (Sistema Permanente de Avaliação Educacional do Estado do Ceará) UECE - Ceará State University (Universidade Estadual do Ceará)

UFC - Ceará Federal University (Universidade Federal do Ceará) UNDP - United Nations Development Programme UNESCO - United Nations Educational, Scientific and Cultural Organization UNICEF - United Nations International Children’s Fund UNDIME - National Union of Municipal Education Managers (União Nacional de Dirigentes Municipais de Educação)

Vice President: Pamela Cox

Country Director: John Briscoe Sector Manager: Chingboon Lee

Project Team Leader: Ricardo Silveira ICR Team Leader: Suzana N. de Campos Abbott

BRAZIL Ceará Basic Education Quality Improvement Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design............................................... 72. Key Factors Affecting Implementation and Outcomes ............................................ 143. Assessment of Outcomes .......................................................................................... 214. Assessment of Risk to Development Outcome......................................................... 335. Assessment of Bank and Borrower Performance ..................................................... 336. Lessons Learned ....................................................................................................... 357. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 36Annex 1. Project Costs and Financing.......................................................................... 37Annex 2. Outputs by Component ................................................................................. 38Annex 3. Economic and Financial Analysis................................................................. 42Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 43Annex 5. Beneficiary Survey Results ........................................................................... 45Annex 6. Stakeholder Workshop Report and Results................................................... 46Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR..................... 47Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ......................... 4Annex 9. List of Supporting Documents ........................................................................ 5

MAP

i

A. Basic Information Country: Brazil Project Name:

BR- CEARA BASIC EDUCATION

Project ID: P059566 L/C/TF Number(s): IBRD-45910 ICR Date: 12/19/2008 ICR Type: Core ICR Lending Instrument: SIL Borrower: Original Total Commitment:

USD 90.0M Disbursed Amount: USD 90.0M

Environmental Category: C Implementing Agencies: SEDUC Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 01/24/2002 Effectiveness: 04/08/2003 04/08/2003

Appraisal: 03/25/2002 Restructuring(s): 12/16/2003 02/09/2007

Approval: 05/30/2002 Mid-term Review: 03/13/2006 03/24/2006 Closing: 06/30/2006 06/30/2008 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately

Unsatisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Moderately Satisfactory

Overall Bank Performance: Moderately Satisfactory Overall Borrower

Performance: Moderately Satisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators QAG Assessments (if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Unsatisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 20 20 General education sector 80 80

Theme Code (Primary/Secondary) Access to urban services and housing Primary Primary Education for all Primary Primary Municipal governance and institution building Primary Primary Participation and civic engagement Secondary Secondary Rural services and infrastructure Primary Primary E. Bank Staff

Positions At ICR At Approval Vice President: Pamela Cox David de Ferranti Country Director: John Briscoe Gobind T. Nankani Sector Manager: Chingboon Lee Xavier E. Coll Project Team Leader: Ricardo Rocha Silveira Maria Madalena R. dos Santos ICR Team Leader: Suzana Nagele de Campos Abbott ICR Primary Author: Suzana Nagele de Campos Abbott F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The main objective of the Ceara Basic Education Quality Improvement Project is to promote greater quality, efficiency and equity in the provision of education services by: (a) improving education quality in both academic achievement (output) and learning environment (education inputs); (b) expanding access to drop-outs and excluded youth through the use of alternative methodologies; (c) fostering equity in the provision of

iii

education services in the poorest municipalities; and (d) strengthening the managerial and administrative capacity of the central, regional and municipal levels to deliver public education services efficiently. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 :

Fifteen percent increase over SAEB/1997 results in the number of students obtaining basic levels of proficiency in Portuguese, Mathematics and Science

Value quantitative or Qualitative)

4th grade - Portuguese: 42,5%, Mathematics: 10%, Science: 52.4% 8th grade - Portuguese: 53,4%, Mathematics: 12,3%, Science: 50%

4th grade - Portuguese: 49%, Mathematics: 11,5% 8th grade - Portuguese: 61%, Mathematics: 14%

4th grade - Portuguese: 28.4%, Mathematics: 31.2% 8th grade ; Portuguese: 65,5%, Mathematics: 27.9%

Date achieved 07/01/1997 12/31/2006 06/30/2008 Comments (incl. % achievement)

Indicator 2 : Increased educational opportunities for youngs and adults in the 15-39 age bracket to reduce illiteracy


1,200,000 (illiterate population in Ceara - IBGE data)

400,000 youth and adults trained and certified as no longer illiterate (new target)

423,500 young adults certified adn 42,084 new students enrolled


Indicator 3 : Reduction in dropout rates


state schools - 12.4% municipal schools - 11.4% data revised on 10/01/07

state schools - 7.0% municipal schools - 6.4%

state schools - 12.1% municipal schools - 7.5%

Date achieved 12/31/1999 12/31/2006 06/30/2008

iv

Comments (incl. % achievement)

Indicator 4 : Raised fundamental (1-8 grades) education completion rates


73% 80%

82% all schools 81.2% state schools82.1% municipal schools


Indicator 5 : Reduced age-grade distortion in fundamental education


55,7%

state schools - 40% municipal schools - 32% overall - 38%

32.6% all schools 35.7% state schools32.9% municipal schools


Indicator 6 : Reduced repetition rates in the fundamental education system


state: 7,6% municipal: 11%

state: 5,4% municipal: 7,6%

20.0% all schools 20.9% state schools21.2% municipal schools


(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : University training for teachers Value (quantitative or Qualitative)

4,600 teachers graduated 3,657 teachers

graduates

Date achieved 06/30/2005 06/30/2008 Comments (incl. % achievement)

Indicator 2 : Early/Basic Education training for teachers Value Early childhood: Early childhood:

v

(quantitative or Qualitative)

3,400 teachers qualified Basic Education teachers: 3,000 teachers qualified

692 teachers qualified Basic Education: 3.974 teachers qualified


Indicator 3 : Initial education training program for Health Agents and teachers in 40 municipalities

Value (quantitative or Qualitative)

1,300 Health Agents trained 5,000 Teachers trained

1,325 Health Agents trained 5,397 Teachers trained


Indicator 4 : Assuring Basic Operational Standards (BOS) for state schools


100% of state schools (686) schools.

100% of state schools with BOS implemented


Indicator 5 : Construction of new schools and acquisition of equipment


35 schools (of which 9 indigenous)

30 new schools built and 2 nearly concluded; 27 schools equiped


Indicator 6 : Accelerated Classes in grades 1 to 4 Value (quantitative or Qualitative)

140,000 students enrolled 121,551 students

enrolled


Indicator 7 : Accelerated Classes in grades 1 to 4 Value (quantitative 140,000 students

enrolled

vi

or Qualitative) Date achieved 06/30/2005 Comments (incl. % achievement)

Indicator 8 : Strengthening SADRE


100% of CREDES

100% of CREDEs and 100% of state schools with SADRE implemented


Indicator 9 : Reformulation of Televised Education Program- Telensino Value (quantitative or Qualitative)

210,000 new students covered Program dropped


Indicator 10 : PRODEM implementation


60 municipalities

55 municipalities with PRODEM implemented; 5 additional municipalities with PRODEM implementation at 50%


Indicator 11 : School Development Plans


100% of state schools and 30% of municipal schools

100% of state schools and 30% of municipal schools have implemented PDE/GIDE


During implementation, School Development Plans were replaced by GIDE (explained in text)

Indicator 12 : Capacity building of approximately 10.000 managers and state and municipal

vii

staff, and specialization for 4.000


capacity building for 10,000 managers, and specialization for 4,000

9,996 managers trained and 3,654 completed specialization


Indicator 13 : SPAECE coverage


100% of state schools 30% of municipal schools



Indicator 14 : SPAECE coverage



Date achieved 12/30/2006 Comments (incl. % achievement)

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/01/2001 Satisfactory Satisfactory 0.00 2 11/29/2001 Satisfactory Satisfactory 0.90 3 05/28/2002 Satisfactory Satisfactory 1.76 4 12/09/2002 Satisfactory Satisfactory 4.55 5 06/04/2003 Satisfactory Unsatisfactory 6.05 6 12/19/2003 Satisfactory Satisfactory 10.99 7 06/04/2004 Satisfactory Satisfactory 10.99 8 12/15/2004 Unsatisfactory Unsatisfactory 18.68 9 04/27/2005 Unsatisfactory Unsatisfactory 28.82

10 09/16/2005 Unsatisfactory Moderately Unsatisfactory 36.45

11 12/20/2005 Moderately Satisfactory Moderately Satisfactory 39.94 12 05/09/2006 Moderately Satisfactory Moderately Satisfactory 52.54

viii

13 11/22/2006 Moderately Satisfactory Moderately Satisfactory 68.02 14 05/11/2007 Moderately Satisfactory Moderately Satisfactory 76.32

15 12/21/2007 Moderately Unsatisfactory

Moderately Unsatisfactory 87.84

16 04/03/2008 Moderately Unsatisfactory Moderately Satisfactory 88.66

H. Restructuring (if any)

ISR Ratings at RestructuringRestructuring

Date(s)

Board Approved

PDO Change DO IP

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

12/16/2003 S S 10.99 02/09/2007 N MS MS 69.31

I. Disbursement Profile

7

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) Poverty has been historically acute in the State of Ceará (the State). At the time of appraisal of the Ceará Basic Education Quality Improvement Project (the Project), 49.4 % of the State’s 6.8 million population was considered poor using a “food only” poverty line of US$34 (R$65) per month per capita, compared with a rate of 23 % for Brazil as a whole. A World Bank study had indicated that the level of education was the most important determinant of poverty in the State, where 84 % of the poor lived in homes where the head of household had less than four years of schooling. Improvements in education were considered a top priority to reduce poverty in Ceará. At the time, the State Government had been pursuing programs to reduce poverty and underdevelopment, with encouraging results. From 1970 to 1997 the State’s GDP had increased at an average annual rate of 5.8 %, and the coverage rates for fundamental1 education (net enrollment rates) had increased from 56.8 % in 1990 to 98.3 % in 1999. Despite this progress, the Government of Ceará faced important challenges to ensure the State’s sustainable economic development. The most important of these challenges was to improve the quality of its human resources. Improving the low quality of its education system, and reducing inefficiencies and inequalities were considered fundamental to address this challenge to the State’s development. Specifically, the issues described below posed particular challenges: High Repetition and Age-Grade Distortion. Repetition and dropout rates in primary education were still very high and only 46% of the students completed the 8th grade within the expected eight years of schooling. In 2000, about 56% of students were two or more years older than they should be for their respective grades. Age-grade distortion was the cumulative result of late entry to school and chronic grade repetition. Low Level of Teacher Qualification and Skills. A 1998 census carried out by SEDUC indicated that, of a total of 39,225 active fundamental teachers in Ceará, 11,406, or 29.1%, were considered “lay teachers” who did not meet the National Education Law (LDB) requirements. Approximately 10% of them had not completed the eight-year fundamental education level. Despite previous efforts to improve this situation, low teacher qualifications was still a problem, especially at the municipal level, and was considered to be an important factor explaining low student achievement and high repetition and dropout rates.

1 The Brazilian education system offers non-mandatory early childhood (0 to 3 initial and 4 to 6 preschool) education programs (Educação Infantil); a mandatory eight-year “fundamental” education segment (Ensino Fundamental); and a three-year secondary school level (Ensino Médio). Brazilian law defines “basic education” as meaning Educação Infantil, Ensino Fundamental and Ensino Medio.

8

Low Student Achievement. Results of the 1997 National SAEB evaluation showed that in Ceará the average achievement level in Portuguese was 46.5% for 4th graders and 59.4% for 8th graders, compared with 49.4% and 65.8%, respectively, at the national level. In Mathematics, the average achievement level for 4th graders was 27.1%, and 32.2% for 8th graders, while national levels were 29.5% and 35.8%, respectively. The State Government was concerned that these low rates of student achievement could affect negatively the State’s prospects for economic development, especially outside the capital. Low Enrollment and Poor Quality of Initial and Preschool Education (0 to 6 years old). There was increasing awareness of the crucial role played by early childhood education in reducing repetition and evasion, and increasing student achievement. Yet, the early childhood education programs in Ceará fell short of desired levels of coverage, quality and management. As of 1999, overall enrollment of children aged 4 to 6 totaled 355,161, or 72.4% of the population group; the corresponding figure for children aged 0 to 3 was 56,371, or 11.4%, respectively. The low quality was mainly due to insufficient training of teachers and lack of standards and materials. Of the total 18,858 preschool teachers, 7,298, or 38.7% had not completed secondary education. Inadequate Municipal-State System Coordination. Municipal education systems functioned in parallel to the State’s education system, complicating the process of modernization and advancement of education in Ceará. At the 1st to 8th grade level, 184 municipal systems attended to approximately 1.1 million students, representing 68% of public school enrollment. State schools served approximately 520,000 students, or about 32% of total enrollment. Both state and municipal systems operated independently with municipalities, contributing to inefficiencies in the use of resources, inequity of opportunities and difficulties in improving quality. SEDUC had begun several joint initiatives with the municipalities, and the Regional Education Development Centers (CREDEs) were expected to play a crucial role in improving state-municipal coordination. Low Enrollment Rates and Poor Quality of Secondary Education. In 1999, enrollment in fundamental education approached universal coverage, while only 30% of the 15 to 19 year old age group was enrolled in secondary education. The distribution of secondary school places was highly unequal, resulting in striking disparities among and between income groups, the capital city and the rest of the state, and urban and rural areas. There was also an important mismatch between secondary school curricula and labor market demand. Despite efforts to improve their qualifications, 17% of secondary teachers did not meet the National Education Law (LDB) requirements. Poverty-Related Factors. Several studies had demonstrated that poverty strongly affected children’s chance of successfully completing an eight grade education. Parents’ demand for schooling for their children correlated with their own level of education, and approximately 84% of Ceará’s poor (poverty, according to a “food only” poverty line, was estimated at almost half of the State’s population) households were headed by someone with less than four years of schooling. Poor children generally had access to poorer quality schools, whether quality was measure by the adequacy of the facility itself, the availability of textbooks and other learning materials, or by teachers’ qualifications. Poor school quality was directly linked with high repetition and low attendance, and there was strong evidence that poverty and parents’ level of education were also key determinants of learning outcomes.

9

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Project’s main objective was to promote greater quality, efficiency and equity in the provision of education services by: (a) improving education quality in both academic achievement (output) and learning environment (education inputs); (b) expanding access to drop-outs and excluded youth through the use of alternative methodologies; (c) fostering equity in the provision of education services in the poorest municipalities; and (d) strengthening the managerial and administrative capacity of the central, regional and municipal levels to deliver public education services efficiently. The impact of the Project’s activities was to be measured by State program and project-specific indicators. The principle performance indicator for the Project was the percentage of 4th and 8th grade students that performed at a satisfactory level in Portuguese and Mathematics, as measured by the standardized tests of the nationwide Basic Education Evaluation System (SAEB). It was expected that the Project would result in a 15 percentage increase for this indicator by December 2005. Other key project-specific indicators were:

• Completion rates in fundamental education (Grades 1-8) which were to be increased from 72 percent to 80 %, on average, in state and municipal schools;

• Net enrollment rates in early childhood development programs which were to be raised from 11.4 % to 22 % for the 0-3 age group and from 72.4 % to 80 % for the 4-6 age group, including the non-formal programs;

• Age-grade distortion in fundamental education which was to be reduced from 56 % to 35 %; and

• Access to fundamental education for drop-outs and younger workers (15- to 39-year olds) which was to be increased from 36 % to 41 % so that 25 % (or 90,000) of this population would have completed at least the first four grades of schooling.

The targets for these indicators were expected to be achieved by December 2005.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The Project Development Objectives were not revised throughout implementation. The Project was, however, restructured in a Second Order Project Restructuring approved by the Regional Vice President on February 9, 2007 to expand the State’s Literacy Program, eliminate the television mode of distance education and revise some outcome and performance indicators and targets (see Annex 2).

1.4 Main Beneficiaries The Project’s main beneficiaries were to be the school-aged children enrolled in or expected to be enrolled in public fundamental schools, who would benefit from a more efficient, equitable and higher quality education system. In 1998, this included 1.1 million children 0 to 6 years old and 1.3 million children 7 to 14 years old. Of these, the Project through the activities described in Section 1.5 below was expected to benefit about 122,000 children 0 to 3 years old, 188,000 children 4 to 6 years old and 672,000 children 7 to 14 years old who would count on access to better managed schools, adequate learning materials and programs, better prepared

10

and motivated teachers and higher quality education. The Project was to specifically address the needs of native ethnic groups by providing group-appropriate construction and rehabilitation of small schools, including equipment and materials tailored to their needs. The Project was to address the needs of the State’s over-aged students which at the time represented 58 % of its basic education population. Through accelerated programs, the Project would benefit over-aged students in Grades 1 to 4. A piloted Telecurso program was to benefit about 150,000 older students in Grades 5 to 8. Expanded TV Education (Telensino) and Tailored Education Programs were to benefit about 90,000 dropouts, parents and young workers aged 15 to 39 that had recently become literate. Telensino would also provide increased access to students in Grades 1 to 8. The Project was to target Ceará’s 54 poorest municipalities selected on the basis of: (a) the Municipal Development Index (IDM), developed by the Planning Institute of Ceará and based on 32 indicators used to rank the development standing of the municipalities; and (b) the UNICEF typology, that identifies Ceará municipalities with the poorest education and other social indicators. To achieve its objectives, the Project would also provide training for 1st to 8th grade teachers, and community Health Agents and parents and community workers that would support non-formal initial education programs.

1.5 Original Components (as approved) The Project included five components, as follows: Component 1: Improving School Quality and Efficiency (US$65.7 million project cost, without contigencies) supported activities that aimed to improve school quality and efficiency by: (a) carrying out in-service and continuing teacher development programs through different modalities to raise the academic qualifications of teachers in both state and municipal schools and to introduce more effective teaching practices; (b) supporting non-formal early childhood development (ECD) programs by upgrading the Health Agents’ (State Health Secretariat) skills to promote these activities at the local level; (c) assuring basic operational standards (BOS) for state schools; (d) developing “accelerated programs” to help over-aged students accomplish two or three grades in a single school year; and (e) supporting and upgrading the existing Pedagogical Monitoring and Support System (SAP). The objectives of Component 1 were to be achieved through four subcomponents, as described below. Subcomponent 1.1, Enhancing Teacher Development and Early Childhood Development (ECD) was designed to upgrade teachers’ professional performance and ECD programs by providing: (a) higher education programs for approximately 6,700 5th to 8th grade teachers; (b) secondary education programs for approximately 6,400 teachers responsible for early/preschool and 1st to 4th grade education; (c) in-service training programs to update, expand and enrich teachers’ performance; and (d) the expansion of non-formal ECD activities through training of the community Health Agents, who would then train parents and community workers to provide non-formal initial education. Subcomponent 1.2, Assuring Basic Operational Standards for Schools was designed to help upgrade state schools to attain basic operational standards (Padrões Básicos de Funcionamento-PBF), to ensure that each school could count on the resources, conditions and inputs that are conducive to better teaching and learning processes and outcomes. Building on the Ministry of Education’s (MEC) minimum operational standards for schools and incorporating lessons from implementation of the Federal Government’s FUNDESCOLA

11

(School Improvement) Program, SEDUC developed a set of Basic Operational Standards (BOS or PBF). SEDUC’s PBF model would ensure that schools had qualified teachers, adequate physical facilities, appropriate furniture and equipment, functional learning-teaching materials, a school or classroom library, and library books and pedagogical equipment, all adjusted to the size, education modalities and programs of each school. It was expected to be introduced in all of the 709 state schools by the end of the Project. Municipal systems were to be encouraged to meet PBF by using resources from the Municipal Education Development Program (PRODEM) under Component 3. Special attention was to be given to the construction/rehabilitation of small schools, including equipment and materials, for Portuguese-speaking local ethnic groups, which were to define their own standards. Subcomponent 1.3, Reducing Age-grade Distortions through Accelerated Programs, was designed to help the State’s basic education students who were overage due to either late entry into formal schooling or repetition, dropout and reentry problems catch up academically thereby reducing the State’s serious age-grade distortion in fundamental education. Following successful experiences of other states in Brazil, this subcomponent would support the implementation of innovative accelerated programs and TV education programs that the State was developing to meet the needs of older students, and the training for teachers in more effective and supportive pedagogical techniques that were designed to build students’ self-esteem and encourage the performance of over-age students. The programs and methodologies were designed to allow motivated over-age students to accomplish two or three grades in one year. The Project included several activities that were needed to implement an accelerated program for over-age students in Grades 1 to 4 and an accelerated program for students in Grades 5 to 8 as well as secondary education. Subcomponent 1.4, Strengthening the Pedagogical Monitoring and Support System (SAP), was designed to increase education and school quality, efficiency, and accountability through the introduction of a new, integrated system of school supervision called the Pedagogical Monitoring and Support System (SAP), which SEDUC had begun developing in 1996. The SAP (later changed to SADRE) was expected to improve school supervision arrangements in several aspects: (a) logistics, i.e., timing for visits and reviews; (b) results-based orientation through the development and tracking of objective measures of school performance; (c) “whole school” development feedback, involving detailed performance feedback from interdisciplinary teams on pedagogical and managerial issues by SEDUC, directors, teachers, parents, students and School Councils, which would then feed into the development of schools’ development plans (PDE) setting out clear goals for improvement. Component 2: Expanding Access (US$22.1 million project cost, without contingencies) supported activities that aimed at: (a) increasing access through TV education programs (Telensino)for students in Grades 5 to 8 where no schooling at this level was available; and (b) increasing access to education for dropouts, parents and young workers in the 15 to 39 age group who had recently become literate through an 18-month program, including television modules, especially designed to combine general basic education content with activities that would help students to develop skills and abilities useful for entering the labor market. Component 3: Upgrading Education in Poorer Municipalities through a Municipal Education Development Program (PRODEM) (US$20.6 million project cost, without contingencies) supported activities that aimed to improve educational results in the 54 poorest municipalities in Ceará by supporting the implementation of two education programs: (a) non-

12

formal initial education; and (b) innovative fundamental education activities to increase the learning achievements of students in municipal schools. A major objective of this component was to leverage desirable policy changes at the municipal level, where school performance was considered weakest and the state system could not exert direct control. The PRODEM was expected to provide a strong incentive for municipalities to modernize their education management process and bring key policies (governing, recruitment, student assessment, basic operational standards and school autonomy, etc.) into line with those of the state system, thereby leading to a more integrated, consistent and equitable education system across state and municipal systems in the State. Component 4: Promoting Decentralization and Institutional Development (US$27.6 million project cost, without contingencies) supported activities to further the then ongoing decentralization process at the school, regional, municipal and central levels by: (a) enhancing school-based management activities to increase school autonomy; (b) modernizing the processes, procedures and monitoring and evaluation capacity at all levels of the education sector to increase accountability and efficiency in service delivery; and (c) establishing a systematic and effective communication strategy to obtain public support for education reform and innovation, and receive feedback from parents, teachers, students and other key stakeholders. The objectives of Component 4 were to be achieved through four subcomponents, as described below. Subcomponent 4.1, Fostering School Autonomy Strategies, supported activities aimed at improving school management and efficiency through: (a) the design and implementation of School Development Plans (PDE) which embodied the needs, demands and expectations of the school and its community members; School Innovation Projects (SIPs) based on the PDEs would then be eligible for financing under the Project. And (b) a concerted effort to redesign and reinvigorate school councils, parent associations and student organizations through the design and delivery of leadership training programs, design of respective operations manuals and the establishment of mechanisms to mobilize school and community support to school efforts, including the development of their PDEs. Subcomponent 4.2, Strengthening Institutional Modernization, supported activities aimed to carry out institutional modernization and strengthening of the education sector at the central, regional and municipal levels. Specifically, it aimed to: (a) upgrade and consolidate the State Education Evaluation System (SPAECE); (b) improve the planning and information systems to support the reorganization of the public (both state and municipal) school systems; (c) establish and implement staff development programs at the central, regional and municipal levels; and (d) develop evaluation studies to determine the impact of the Project and of selected project interventions. Subcomponent 4.3, Building up Social Participation and Mobilization, supported activities aimed at: (a) raising social awareness of the socio-economic and developmental value of basic education and local schools; and (b) promoting key stakeholder participation in and support to education and school-associated reform initiatives.

13

Component 5: Project Management US$9.0 million project cost, without contingencies) was to finance: (a) the Project’s administrative expenses; and (b) equipment, vehicles, office supplies and technical assistance for supporting project administration.

1.6 Revised Components Although the Project’s objectives remained valid throughout implementation, and its components were not revised, there were several adjustments in the Project’s focus and its intermediate targets that were reflected in three loan amendments, and two reallocations of loan funds. These amendments are described below. The first amendment to the Loan Agreement was signed on December 16, 2003, a little over two years after loan effectiveness. At the time, the Project had been under implementation for about 18 months, but was already facing serious problems. Due to limited counterpart funding, implementation focused mostly on activities that were financed entirely by the Bank’s loan and/or those included in the Accelerated Results Together (ART) exercise2. The Project was also facing problems with project management, disbursements and monitoring and evaluation, as described in Section 2.2. The amendment included several modifications aimed at improving implementation arrangements and adjusting the Project towards the State’s changing priorities, as follows: (a) an increase of the overall percentage of Bank financing from 60% to 80%, with a parallel decrease in project costs from $150 million to $112.5 million; (b) adjustments to the Project’s output indicators and corresponding amendment to the Implementation Letter (see Table 1 and Annex 2); (c) replacement of the Project’s Executive Group responsible for its coordination with a Project Implementation Unit (PIU) with staff experienced in implementing Bank-financed projects; (d) an expansion of the tailored programs to support the Literacy Program under Component 2, which had become a high priority for the State, and an increased allocation of funds for this program; (e) the inclusion of the School of Public Health as a co-implementing agency that would train teachers to complement the training of the community Health Agents, both aimed at expanding non-formal ECD activities under Component 1; and (f) an extension of the loan’s Closing Date by one year, from June 30, 2006 to June 30, 2007. The second amendment to the Loan Agreement was signed on March 30, 2005. Despite expectations that project implementation would improve after signature of the first amendment, implementation had deteriorated significantly towards the end of 2004. Along with other actions taken to improve implementation (Section 2.2), the second amendment aimed to allow more flexible implementation arrangements for the Literacy Program under Component 2 and for the Municipal Education Development Program (PRODEM) under Component 3, by allowing other institutions to participate as implementing agencies for these activities. These two amendments did not address serious structural institutional constraints to project implementation, particularly the lack of coordination between the PIU and the rest of SEDUC, the weak capacity of SEDUC to guide project activities or even draft TORs and technical

2 The Accelerated Results Together (ART) was an exercise delivered jointly by the State, with the participation of all sectors, and the Bank. As a result of the ART, the State’s priority areas were identified. In order to address those priorities, ART participants determined the need for an integrated effort by the State Government to improve the performance of its social programs.

14

specifications to procure consultants and educational infrastructure and goods, the long bureaucratic chain of approvals and clearances that often did not add value to the process, but certainly added processing time. Finally, these amendments did not take into account weaknesses in the decision making process within SEDUC, which meant that even when a problem was identified and a solution encountered, it could take many months before it was acted upon. In 2005, these constraints were identified and corrective actions were taken successfully. Simultaneously, a state-wide SWAP operation was being prepared, which included support for programs in the education sector, thus providing the right incentives for swifter decision making. The third amendment to the Loan Agreement processed as a Second Order Project Restructuring was approved by the Regional Vice President on February 9, 2007. By then, project implementation had improved. The restructuring focused on revising project outcome and output indicators and targets to reflect the State’s priorities and what was considered achievable during implementation. The main modifications were: (a) a further expansion of the Literacy Program under Component 2; (b) the elimination of the television mode of distance education also under Component 2; (c) the adjustment of selected outcome and output indicators and targets (Table 1 and Annex 2); (d) the adoption of the Bank’s May 2004 revised Procurement Guidelines; and (e) a second extension of the Loan’s Closing Date from June 30, 2007 to June 30, 2008.

1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) A reallocation of loan proceeds was approved on December 28, 2007 which increased allocations to Works, Literacy Grants, Consultants’ Services, Training and PRODEM Grants, and reducing allocations to Goods, School Grants, and Incremental Operating Expenses. This reallocation was mainly to reflect actual disbursements claimed until then, and projected requests based on activities then under implementation. A second loan reallocation was processed in the Project’s final stages, to reflect a final allocation of loan proceeds.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) Project preparation was lengthy, with the Bank’s support undergoing several changes in task management. Throughout this process many expectations had been created as to activities that were to be included. In the final phases of project preparation and loan approval, it became difficult to exclude many of the already identified activities to streamline the Project’s design with the result that the Project became overly complex. A formal Quality at Entry Assessment was not carried out for this Project. The factors described below are relevant in evaluating the Project’s Preparation, Design and Quality at Entry. The Project’s quality at entry was positively influenced by several criteria: its’ strategic relevance, its poverty focus and participatory design, and its focus on activities aimed at improving educational monitoring and evaluation. However, because of the overwhelming importance of issues surrounding the Project’s design, risk assessment and readiness for implementation on the Project’s subsequent

15

implementation, the Bank’s performance in ensuring quality at entry is rated moderately unsatisfactory: (i) Strategic Relevance and Approach - The Project responded directly to the goals of the existing Country Assistance Strategy (CAS) and to recommendations of the Bank’s Education Sector Strategy for Brazil3. The CAS aimed to promote sustainable long-term growth, alleviate poverty and reduce income inequity in Brazil, and gave special emphasis to support for education. The Project responded directly to the CAS’ education sector goals of achieving universal completion of the eight-year fundamental education cycle by 2007 and improving educational performance in basic subjects, particularly Portuguese and Mathematics. The Bank’s Education Sector Strategy for Brazil further emphasized focusing efforts to address these goals, especially among the poor, and recommended investments in pre-school, fundamental and middle education in the poorest states of Brazil. The Project responded directly to these goals by: (a) focusing on interventions to improve the quality and efficiency of basic education, both at the state and municipal level; and (b) supporting efforts aimed at poverty alleviation and the reduction of income inequality by targeting one of the poorest states in Brazil, and especially 54 of the state’s poorest municipalities. Project design incorporated lessons learned through a long partnership of support to Brazil’s education sector, both at the federal and state level. Specific lessons from the design and implementation of three Northeast Basic Education Projects (Loans 1867-BR, 3604-BR, and 3663-BR), two federal FUNDESCOLA Projects (Loans 4311-BR and 4487-BR), and several other Bank-financed projects implemented by other states guided the Project’s preparation, including the need to: (a) place emphasis on school development strategies; (b) promote the use of Basic Standards (BOS); (c) incorporate the use of participatory processes for the elaboration of School Development Plans; (d) promote strong cooperation between state and municipal education systems; (e) strengthen monitoring and evaluation activities; and (f) place emphasis on social mobilization and communications among involved parties. Instead of focusing only on improvements at the primary level as a means of decreasing repetition and reducing dropout rates (as previous projects had been designed), the Project addressed directly the issue of late entry to school and lack of adequate preparation through activities to promote and improve early childhood and preschool activities in order to reduce the number of overage students. Project design placed a special importance on the institutional strengthening of SEDUC in several key areas, as experience from previous projects highlighted the importance of strong institutional capacity as a determinant of project success. (ii) Project Design, Risk Assessment and Readiness for Implementation – Despite its attempt to incorporate lessons from previous operations, the Project’s design was overly ambitious. While several project alternatives were considered in the Project Appraisal Document, the alternatives weighed the value of different strategies for addressing proposed activities as opposed to whether to include only some or all of the activities in the same project. The Project’s multiple activities included in five components (with various subcomponents), its complex institutional arrangements involving implementation across multiple municipalities (whose capacity was usually less than that of the State) and different levels of education, and its ambitious objectives resulted in the need for intensive supervision to adjust the Project to the Borrower’s actual implementation capacity and priorities. The issue of the Project’s

3 World Bank Education Sector Strategy for Brazil, dated September 1998

16

complexity had been identified before Appraisal, as reflected in the Minutes of the PAD Review Meeting on January 28, 2000, but even the resulting project design remained overly ambitious. The Project’s complexity was especially complicating in an environment where the importance of strong institutional capacity had been identified as a key determinant of success. The Project’s risk assessment, as outlined in the PAD’s table of critical risks failed to highlight the main risks that in the end affected the Project throughout its implementation: counterpart funding, implementation capacity, staffing changes and lack of capacity at the State level, procurement and financial management issues, the State’s bureaucratic processes, and the macroeconomic factor, i.e. the appreciation of the Real vis-à-vis the US$, which resulted in lack of loan funding for the completion of some project activities. As a result, overall, the Project’s risk assessment, at moderate, was underestimated. The Project was not ready for immediate implementation when it was approved, especially given its ambitious design and the perceived weak capacity of the implementing agency. This was reflected in the early PSRs (now ISRs). As an example, the Summary Issues and Actions section of the first PSR dated June 1, 2001 (six months after Board approval), highlights the need for the Borrower to: contract with a project administration agency, contract for procurement support and review staffing needs. These actions were likely pre-conditions to get the Project off to a smooth start. Terms of reference for key activities needed to get project implementation off the ground had yet to be prepared. The second PSR dated November 29, 2001 mentioned that the “action plan for the next school year will be sent to the Bank no later than December 30”, more than one year after approval of the loan by the Bank’s Board. By the time the third PSR was prepared, Project Management was already rated as Unsatisfactory. This was likely the direct result of the Project’s implementation team not counting on the technical assistance and skills needed to get implementation underway. Furthermore, the Borrower was already questioning the future of the Telensino program under Component 2 since an evaluation study on the program had just been completed. Ideally, project preparation should have counted on the finding of that evaluation, as had been suggested in the PAD Review Meeting. Finally, preparation of the Operational Manual containing criteria for the selection of the municipalities and the operational mechanism for PRODEM only began in 2002. (iii) Poverty, Social Development and Environmental Aspects – Almost by definition, the Project was a Poverty Targeted Intervention. While the Project’s benefits were to include the entire school-age population, some of its interventions were to be targeted to the 54 poorest municipalities in the State. These included: school Basic Operational Standards, implementation of school development plans, expansion of early childhood development activities, innovation in rural education and other activities supported by PRODEM. When the Project was prepared, at least 49.4% of the population of Ceará lived below the “food only” poverty line. Studies had demonstrated that poverty strongly affected children’s chances of successfully completing an eight-grade education. Parents’ demand for schooling for their children also correlated with their own level of education and 84% of the poor households in Ceará were headed by someone with less than four years of schooling. The 1998 PNAD Household Survey had revealed that 29.6% of the population over 15 years of age was illiterate, with a higher percentage among men and a heavy concentration in rural areas. There was also strong evidence that poverty and parent’s level of education were also key determinants of learning outcomes. Hence, the Project’s focus on education access for

17

excluded population segments such as young workers that dropped out of school was an important complement to its other activities. The Project was also designed to focus on the needs of indigenous peoples of Ceará. They were to receive special attention to address their special needs, including: culturally-relevant courses, 20 school buildings and equipment, training of 90 indigenous teachers at the middle school level and the provision of 24 multi-media centers in their schools. A beneficiary assessment based on interviews with community leaders, parents, young people, teachers and principals guided project preparation and design. The assessment confirmed the value of education and revealed that the poor quality of education was seen as the primary reason for school failure and dropouts (and not economic pressures). It also showed that the links between the school and the community needed to be strengthened. Finally, the assessment identified areas that required further study such as the use of television programs that were already generating mixed reviews. The results of the beneficiary assessment and its background studies were incorporated in the Project’s design. Since the Project did not envisage the construction of major infrastructure, it was not seen to pose a significant environmental risk. Hence, new school construction and rehabilitation was to follow state and municipal construction guidelines. Despite this, the Project eventually faced delays in construction of indigenous schools since problems arose regarding the titling of land for those schools which eventually required the approval of architectural designs by a federal agency responsible for indigenous matters (FUNAI). (iv) Monitoring and Evaluation - The Project’s design incorporated several activities aimed at improving monitoring and evaluation. The Project included the development of a new, integrated system of school supervision called the Pedagogical Monitoring and Support System (SAP), which SEDUC had begun developing in 1996. The SAP was meant to improve on previous school supervision arrangement by improving the logistics for school supervision, promoting an increasingly results-based orientation that focused on student achievement, repetition, dropout and completion rates, and organizing interdisciplinary teams that would visit and observe the school and provide comprehensive and detailed performance feedback. The Project also provided for the upgrading and statewide implementation of SPAECE, the State’s Education Evaluation System. SPAECE was to include cognitive achievement by subject matter, to be measured across variable such as age, type of school and education, modality, grade, gender and geographical area. Data relative to school inputs, family and community characteristics, and social and economic environment was to control for school and non-school factors. SPAECE was to be implemented in close collaboration with the Federal Ministry of Education which had developed SAEB, a national system of evaluation based on a national sample of schools and students. SPAECE was expected to be applied in years in which the SAEB was not carried out. Project design also contemplated the development of impact evaluation studies of selected activities and of the Project. Monitoring and evaluation of project activities is described in Section 2.3 below.

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) Mostly because of its ambitious design, complex institutional environment and lack of readiness, the Project’s implementation was difficult. Project Development Objective and Implementation Performance ratings, as well as those of individual components, fluctuated

18

constantly throughout implementation ranging from Satisfactory to Unsatisfactory. In reading the Project Status Reports, it is apparent that implementation did not begin in earnest until late-2005, or nearly five years after loan approval. By then, the Project was already facing issues with project management, with procurement processes, with the State Government’s bureaucratic processes that required other Secretariats’ approval for acquisition of goods, and with the timely provision of adequate counterpart funds. A change in state administration in January 2003 only aggravated implementation problems when existing project staff was replaced with staff that lacked experience in implementing projects financed by the Bank. Still, some progress was being made in selected activities, especially, teacher development, building and upgrading state schools, reducing age-grade distortion and the literacy program (which had been made a priority in the ART exercise), the implementation of PDEs, and training in school management strategies. In 2003, the Bank addressed some of the early implementation obstacles, by processing the first amendment to the Loan Agreement (Section 1.6). This amendment also adjusted the Project to the State’s revised priorities. Despite this amendment and a second amendment in 2004 the Project continued to face important challenges. The second loan amendment (Section 1.6) was intended to make implementation more flexible, inter alia, by including other institutions as possible executing agencies for PRODEM and the Literacy Program. However, implementation had not accelerated, and only the teacher development and literacy programs were making notable progress. The Literacy Program was noteworthy in that its design provided for payment based on the number of students that obtained their certification. In 2005, the Bank carried out an in-depth review of procedural steps taken in each major activity as a statistical tool to support the existing institutional analysis. That review identified significant institutional impediments to implementation, in which procurement required 87 days in SEDUC, another 50 days in other State-level agencies, before returning to SEDUC for procurement. A detailed analysis of each of the Project’s 28 main activities found a generalized state of inertia, mostly due to non-observance of the Project’s Operations Manual and/or lack of institutional capacity. In response to these findings, the Bank recommended: (a) the establishment of standards of process performance within SEDUC; (b) a review of functions necessary for good project implementation relative to existing staff; (c) differentiated treatment of the 28 main project activities; (d) direct approval of the entire annual work program (PTA) by the State Secretariat of Administration, instead of piecemeal approvals; (d) contracting support for project administration and procurement; and (e) intensive supervision until supervision ratings improved. Around that time, given the still low pace of implementation relative to disbursement projections, the Bank mission even recommended that the Borrower consider whether partial cancellation of loan proceeds would be desirable. PRODEM was finally launched and the first 60 agreements were signed at the end of 2005, six months before the loan’s original Closing Date. This, together with progress in other activities, led to a significant improvement in the pace of implementation, despite ongoing issues in the fiduciary area. Disbursements began to pick up markedly, and SEDUC’s success in streamlining its internal procedures led to an increase in the procurement processes under way. Several project activities were included in the trigger matrices of the Ceará Multi-Sector Social Inclusion Development APL, approved in 2005 (Report No. 32487-BR dated June 20, 2005. The Ceará SWAp, as it is known, aimed to support social inclusion objectives across six sectors, while preserving fiscal sustainability. The Ceará SWAp included efforts aimed at

19

ensuring prudent fiscal management and institutional strengthening both in central and line secretaries through results-based management and several activities in the education and other sectors. In education, the Ceará SWAp conditioned loan disbursements on satisfactory implementation of the Ceará Basic Education Development Project, and established annual disbursement-linked indicators to pace disbursements for education. These indicators included to: (a) improve learning achievement through the implementation of a testing (SPAECE/SAEB) cycle building on diagnostics, action plans and implementation; (b) improve cost effectiveness of SEDUC’s operations focusing on administration, staffing, regional units (CREDEs) beginning with studies and elaboration of recommendations and legal frameworks followed by 2005-6 implementation of recommendations, monitoring and evaluation of results; and (c) reduce illiterate population 15 years and older by 100,000/year for 2005-2006. Many of the implementation difficulties the Project faced were seen to be outside of SEDUC, and it was expected that the SWAp could assist in bringing about broad institutional improvements, as well as support sectoral improvements that would also benefit implementation of the Project. By the time of the Mid-Term Review in March, 2006, implementation had improved significantly, and disbursements were accelerating. The Review noted the determination of a new, more dynamic Project Implementation Unit (PIU) Director, as well as the strengthening of the procurement and financial management function in the PIU . Yet, the Review also highlighted that SEDUC faced important challenges, especially, the execution of a large number of complex activities, involving time consuming procurement and legal issues (e.g., land titling for indigenous schools). Given the improved implementation performance, and the need to reflect what the Project would be able to accomplish in its final phase of implementation, the Mid-Term Review mission agreed to consider a Project Restructuring which would, inter alia, extend the Loan’s Closing Date by one additional year and adjust project indicators to that which could be reasonably accomplished during the remaining implementation period (Section 1.6). Despite the serious implementation problems it faced throughout its life, the Project made significant progress in its final phase. However, SEDUC still faced additional challenges, even during the Project’s final phase of implementation: (a) a significant appreciation of the Real vis-à-vis the US$ which reduced that activities that could be financed by the Bank’s loan, especially given the higher cost-sharing that had been allowed in the third Loan Amendment; (b) a temporary suspension of all State procurement processes until they could be analyzed by officials of the new State Administration that had taken office in January 2007; (c) the replacement of project staff by the new Administration; and (d) the need to conclude ongoing construction of schools that were already in process; and (e) selection of consultants to carry out assessments of key project-financed activities. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The Project’s design incorporated several activities aimed at improving the State’s capacity for monitoring and evaluation of its education programs (Section 2.2). This included redesigning the Pedagogical Monitoring and Support System (SAP) which was part of an integrated monitoring and evaluation system, including both qualitative and quantitative indicators to better assess/monitor schools’ performance. The Project Executive Group, later replaced by a PIU, was responsible for monitoring and evaluation of the Project, in accordance with a set of State-level program, outcome indicators and project-specific intermediate outcome indicators (Section 1.2). This dual approach was designed in response to concern raised during the PAD Review Meeting regarding the use of overall state program targets and indicators to measure

20

the Project’s implementation progress and impact. SEDUC was to present the Bank with Annual Progress Reports on project implementation, including the targets achieved and the proposed budgetary requirements for implementation in the subsequent year. The Project’s ISRs show that project indicators were updated periodically. They were carefully reviewed during the Mid-Term Review, since it was important to have up-to-date information to prepare the Project Restructuring which included a revision of the Project’s expected targets. However, despite progress in strengthening SEDUC’s monitoring, the Bank’s supervision team reported continuous difficulties with obtaining up-to-date monitoring information on project activities. Even upon completion of the Project, it was difficult to obtain reliable data and information to include in this ICR. The State’s Student Evaluation System (SPAECE) that was to be implemented state-wide under the Project was carried out in 2005, complementing the Federal SAEB which had been carried out in 1999, 2001, 2003 and 2005.

2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable)

The Project’s fiduciary arrangements were problematic practically from the very start of its implementation. More often than not, both procurement and financial management were rated Unsatisfactory or Moderately Satisfactory. Problems were due mostly to the absence of staff with the knowledge of Bank procurement guidelines and financial management requirements, including disbursement procedures. To give an example, at one point a Bank mission found that despite continuous problems with counterpart funding, the State had spent about $10 million of its own resources on project activities without seeking reimbursement from the Bank. These problems were exacerbated by the frequent turn-over of staff in the Project Executive Group (later the PIU). The numerous procedural steps required to procure goods, works and consultants’ services were identified in the procedural review carried out by the Bank in 2004, and included issues within SEDUC, between the PIU and its other units responsible for implementation. Unfortunately, SEDUC’s very success in streamlining its internal procedures led to an increase in the number of procurement processes ready to be carried out by the PIU, creating a bottleneck of qualified staff to carry out the procurement. At one point a supervision mission found that only 15 out of 71 procurement processes in an annual procurement plan had been concluded, and those that were concluded were seriously delayed. In the end, procurement and financial management functions improved, but not without significant external support, including the hiring of external consultants of international caliber.

Safeguards compliance, specifically with respect to the Project’s focus on the indigenous was relatively smooth. The Project financed teacher training and materials tailored for the Indigenous schools. The main issues that constrained implementation concerned the construction of indigenous schools which was delayed due to the need for approval of land titling and to a lesser extent, as negotiations with FUNAI on architectural and engineering details of the works.

2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) A follow-on project is not being considered. Bank assistance to the State of Ceará is being channeled almost exclusively through the Ceará Multi-Sector Development APL. The first phase of the APL closed on December 31, 2007, and the achievement of its development objectives was considered Satisfactory. In education, the Implementation Completion Report

21

for the Ceará SWAp notes that: (a) implementation of national and state education testing cycles was completed; and (b) the illiterate population aged 15 and over was reduced by over 100,000. The second phase of the Ceará SWAp was approved on September 30, 2008 (Report No. 45084-BR, dated August 29, 2008). Like the first, the second phase of this APL includes support for expenditure programs related to basic school standards (BOS); basic educational quality in rural and urban school zones; and cooperation between the state and municipalities, and included disbursement-linked indicators conditioning sectoral assistance to progress in carrying out annual student exams in the 2nd, 5th, 9th and 12th grades and expanding literacy program to municipalities.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) The Project’s objectives were relevant at the time it was prepared, and continue to be very relevant today. They are fully consistent with the Bank’s 2008 Country Partnership Strategy (CPS) for Brazil which aims to sharpen the focus on human development contribution to the “equity agenda” and broaden that focus to concurrently recognize the importance of human capital and innovations for the “growth agenda”. Specifically, the CPS mentions: “challenges remain on the equity agenda, in terms of expanding job and income opportunities for the poor by strengthening their human capital and in terms of strengthening the affordability, quality and efficiency of service delivery. Significant regional disparities also remain, in poverty, educational attainment, and infant- and child-mortality. Similarly, strengthening human capital crucial to the growth agenda, in terms of developing a higher-skilled, healthier and more agile labor force that is able to innovate and adapt innovations to boost total factor productivity”. Specifically with respect to education, the CPS noted “despite progress in expanding coverage, Brazil must face the challenges of improving quality, performance and outcomes---or miss the opportunities for growth and poverty reduction”. It noted that Brazil suffered from the highest repetition rate in Latin America (some 20 % of students repeated), and low educational attainment---the population over 15 years of age had on average only 4.8 years of schooling. The CPS also highlighted that the situation was worse for the poor who entered the system late, left early and repeated more, citing specifically that by age 15, poor youths had completed three years less schooling than the non-poor. Future Bank assistance for the education sector is to focus on two levels: (a) at the strategic and policy level in the Federal Government; and (b) at the state level (most likely through multi-sectoral SWAps) to help strengthen the implementation of the education strategy and quality improvements in service delivery. The Project addressed, throughout its implementation, most of the issues that are still today seen as priorities, and the Government and the Bank showed flexibility in adjusting the Project to those activities that had produced results. Perhaps the best example of this was the Government’s decision, and the Bank’s support, to scale up substantially the tailored education programs for adults and youth between 15 and 39 years of age, while dropping the Telensino mode of delivery. This adjustment alone allowed more than four times originally planned number of adults and youth to pass literacy tests.

22

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) The Project had mixed results in the achievement of its Development Objectives, despite important progress towards meeting most of its output targets. Table 1 presents a summary of the Project’s outcome indicators, targets, adjustments in targets and final progress towards outcomes. For its broadest outcome indicator, educational achievement for students in fundamental education, the Project showed generally good progress, except for achievement of 4th grade students in Portuguese. Progress in this subject/grade level actually deteriorated. Results for achievement of Mathematics for both 4th and 8th graders, and of Portuguese for 8th graders greatly exceeded expectations. It should be noted that while the Project’s scope was broad throughout the State, the achievement indicators based on results of the 2005 SAEB reflect state-wide results which include results for schools that did not benefit directly from the Project. Furthermore, SAEB has been undergoing a series of institutional and methodological changes since its creation in 1990. Since 1997, the test measurement scale has been in a process of improvement. This, along with significant increases in primary school enrollment and greater effort on behalf of local governments to promote underachieving students in the three target regions have forced SAEB to interpret student achievement across years. Given the above than expected improvements in learning achievements in some areas and the marked deterioration in learning achievement in another, the methodological changes call into question the reliability of the results for purposes of comparison. Progress towards other outcome indicators reflects activities that benefited directly from the Project. Again, however, results are mixed, and in some cases, even contradictory. By far the Project’s most successful outcome was that of improving the literacy rate through offering youth and adults the opportunity to further their education, which has not only a direct impact on their opportunities but is also expected to influence their demand for education for their children. Dropout rates were affected differently among state and municipal schools, declining somewhat (but not as much as expected) in municipal schools, and actually increasing in state schools. Fundamental education completion rates increased in both state and municipal schools, but not as much as had been expected. Repetition rates increased markedly in both state and municipal schools. Finally, age-grade distortion in fundamental education was reduced both as a result of children entering the early years of Fundamental Education at the proper age. The results for age-grade distortion, which improved less than expected, were impacted by the inclusion of overage students in the Tempo de Avançar accelerated program in the 8th grade of fundamental education.

23

Table 1: Ceará Basic Education Development Project: Outcome Indicators

Indicator PAD Annex 1 1st Amendment Project Restructuring 3rd Amendment Outcome

State Program Indicators to be achieved by 2005: Fifteen % (15%) increase over SAEB/1997 results in the number of Ceará Fundamental Education students obtaining basic levels of proficiency in Portuguese, Mathematics and Science4

4th grade: Portuguese From 42.5% to 49% Mathematics From 10% to 11.5% Science From 52.4% to 60% 8th grade: Portuguese From 53.4% to 61% Mathematics From 12.3% to 14% Science From 50% to 57.5%

No change

4th grade: Portuguese From 42.5% to 49% Mathematics From 10% to 11.5% 8th grade: Portuguese From 53.4% to 61% Mathematics From 12.3% to 14%

4th grade: Portuguese 28.4% Mathematics 31.2% 8th grade: Portuguese 66.5% Mathematics 27.9%

Increased educational opportunities for the 15-39 age bracket

90,000

120,000 students and age range change to 15 and up

400,000 students

423,500 young adults and adults certified and 42,084 new students enrolled

Reduction of 5% of the dropout rates at state and municipal schools, by 2005

State Schools: from 11.4% (1999) to 6.4% (2005) Municipal Schools: from 10.7% (1999) to 5.7% (2005)

No change

State Schools: from 11.4% to 7.0% Municipal Schools: from 10.7% to 6.4%

State Schools: 12.1% (2005 data) Municipal Schools: 7.5% (2005 data)

Fundamental education (grades 1-8) completion rates raised

From 73% in 2000 to 80% in 2005

No change

No change

82.0% (2007 data) State Schools: 81.2% Municipal Schools: 82.1%

Age-grade distortion in fundamental education reduced

From 55.7% in 2000 to 35% in 2005

No change

From 55.7% to 38% 32.6% (2006 data) State Schools: 35.7% Municipal Schools: 32.9%

Repetition rates in the state fundamental education system reduced

State Schools: from 7.6% in 1999 to 5% in 2005 Participating Municipal Schools: from 11% in 1999 to 7%

No change

State Schools: from 7.6% to 5.4% Participating Municipal Schools: from 11% to 7.6%

20.05% (2005 data) State Schools: 20.9% Participating Municipal Schools: 21.2%

4 Since 1999, SAEB has not assessed students’ performance in Science. As a result, this sub-indicator was dropped in the Project Restructuring.

24

in 2005 The Project’s progress towards achievement of its expected output targets is described below, by component. Annex 2, Outputs by Component, provides a breakdown by indicator, original and revised targets, and actual results. COMPONENT 1: Improving School Quality and Efficiency (US$38.89 million total project costs, 33.6 % project cost, and 59% appraisal estimate) ICR Rating: Moderately Satisfactory 1.1 Enhancing Teacher Development and Early Childhood Development (ECD). Due to early delays in the selection of methodologies and contracting, implementation of this subcomponent was delayed and, in the end, SEDUC was unable to recover the time lost. As a result, in the Project’s restructuring, the original targets of teachers and health agents to be trained were revised downwards (Annex 2). SEDUC established a program of university-level training for state teachers that had already completed the secondary level. The program, called Magister, was carried out in partnership with four public universities5 in Ceará. Teachers of several municipal school systems also participated in the program, although their training was not financed by the Project. A total of 4,743 state and municipal teachers were enrolled among the four universities, although only 3,657 graduated from the program. An evaluation of the Magister program, carried out under the Project, revealed that it brought positive results for the teachers that participated, who gave a positive evaluation to the program, and emphasized the theoretical and practical skills that they acquired that influenced their personal and professional qualities. However, project design did not include an evaluation of the impact of the program on actual classroom teaching. Upon completing the program, teachers’ salaries were increased to reflect their higher skills. A study by SEDUC revealed that 65% of the teachers that graduated from this program were between the ages of 36 and 55 years, and counted on over 21 years of teaching experience (i.e., many of these teachers were very close to retirement). Because of this, the high investment cost and the program’s five year lag to produce returns, SEDUC decided to discontinue the program. The in-service training program to upgrade teachers’ pedagogical skills was dropped because of the introduction of a federal program with the same objectives. The Project provided secondary school training to teachers in 102 municipalities under a four module, two year program developed by the Federal Ministry of Education (MEC), called PROFORMAÇÃO, aimed at providing secondary education diplomas to teachers in the early years of fundamental education. A total of 3,974 teachers received their secondary school diplomas. SEDUC also established a program to provide secondary education diplomas to preschool teachers, creating a fifth module based on PROFORMAÇÃO. A total of 692 teachers were qualified under this program of which 331 had already participated in PROFORMAÇÃO and 361 already had their secondary education diploma. MEC later established formally a preschool education module

5 The universities that participated were: Universidade Federal do Ceará, Universidade Estadual do Ceará, Universidade Regional do Cariri and Universidade Vale do Acaraú

25

to PROFORMAÇÃO, and 134 teachers from among 11 of the poorest municipalities in Ceará participated. These were not financed by the Project, however. The Project had anticipated training a total of 9,000 Health Agents in educational strategies designed to upgrade their skills in early childhood development, who would then train parents and community workers to provide non-formal education. This target was later revised when a mapping of the 40 participating municipalities revealed that there were only 1,627 Health Agents and 5,605 preschool teachers in those municipalities. The original target had been based on the overall number of Health Agent and preschool teachers throughout the entire State. In the end, 1,325 Health Agents and 5,397 preschool teachers received training to upgrade their skills in early childhood development. The Project also provided 1,283 scholarships for teachers in secondary education at an average cost of R$4,500. 1.2 Assuring Basic Operational Standards for Schools (BOS). Based upon an exercise carried out by SEDUC that considered population growth, growth in secondary enrollment, and pre-defined criteria of basic parameters, the Project expected to finance a total of 35 schools, of which nine would be indigenous. To date, 21 fundamental and secondary schools have been completed, totaling 180 classrooms, in addition to nine indigenous schools. Construction of two additional schools faced problems with the construction contractor. They are expected to be completed in late 2008 with government funding. In addition the Project financed 16 science and 180 information technology laboratories, furniture and equipment for 27 of the completed schools (of which four were indigenous) and textbooks and other pedagogical materials for schools at the fundamental and secondary levels. Furniture and equipment, laboratories and textbooks for the remaining schools had not been delivered by the Project’s completion date. The Project also financed upgrading 35 schools, either through rehabilitation or expansion, to bring them in compliance with the BOS. Additional activities to assure BOS in state schools were included in the schools’ PDE/GIDE under Subcomponent 4.1. 1.3 Reducing Age-grade Distortion through Accelerated Programs. Due to the delays in early implementation, the accelerated class programs for 1st to 4th grade students, aimed at reducing the age-grade distortion, was not able to achieve its original target, reaching only 121,551 children under the Project. The shortfall was also due to the transfer of a large number of state fundamental education schools to the municipal governments (which were not covered by the Program) and because of delays in the implementation of PRODEM since this program would be used as an incentive to make the municipalities adopt SEDUC’s education policies and programs, such as the accelerated program. Because of this decentralization, age-grade distortion is much higher in municipal schools. In 2004, it is estimated that 9,297 children in the 1st through 4th grades in state schools had an age/grade distortion compared with 198,764 in the same grades in municipal schools. The Tempo de Avançar program, an accelerated program for over-age youth (and adults interested in completing fundamental education) reached 187,734 between 2002 and 2005. Despite having an initial positive impact on age-grade distortion, the Program stopped following the well tested methodology agreed during project design and results deteriorated. This program was discontinued following a change in the State’s administration, when

26

responsibility for this accelerated program was assumed by the municipalities. Still, it was a strong factor in reducing the age-grade distortion in Ceará from 55.7% in 2000 to 38.5% in 2005. 1.4 Strengthening the Pedagogical Monitoring and Support System The Project contemplated the introduction of an integrated school supervision system called SAP (Pedagogical Monitoring and Support System) that SEDUC had begun developing in 1996. A review of the functioning of SAP carried out by SEDUC in 2003 revealed that the system was not producing quantitative improvements or improvements in school performance, especially in view of its high financial and human investments. In late 2003, SEDUC designed a new system based on results-based management. The resulting Sistema de Acompanhamento e Desenvolvimento da Rotina Escolar (SADRE) is a web-based system that provided real-time information on, inter alia, enrollments, completions, repetitions, drop-outs, and class size by grade level to guide decision-making by individual schools, CREDE and SEDUC. Together with the PDEs (and now GIDEs), allow officials to monitor schools efficiently in terms of learning achievement, school management and administrative and financial efficiency. Teams from SEDUC, CREDE and schools were trained in how to use SADRE and all State schools in Ceará now have information available in the system. COMPONENT 2: Expanding Access (US$36.53 million total project costs, 31.6 % project cost and 165.9% appraisal estimate) ICR Rating: Moderately Satisfactory Training through the Telensino Program aimed at students in the 5th to 8th grades was not carried out. SEDUC carried out two different evaluation of the Telensino Program, in 2001 and 2004, which revealed that the materials used for the program, including manuals, televised classes and technological aspects required major upgrading. An impact evaluation showed that any upgrading of these would need to be accompanied by the hiring of qualified teachers to implement the program. The State did not count on sufficient qualified teachers for this purpose. Furthermore, there were issues with respect to the State’s television channel through with the program was aired, and any upgrading of the program would have required technological improvements as well. The Telensino Program was discontinued in its then existing state, and no financing was provided under the Project. Given the State’s high rates of illiteracy, programs to improve literacy became a priority upon the change in the State’s administration in 2003. The program to enhance the marketable abilities and skills of youth and adults in the 15 to 39 age bracket was converted to a Literacy Program covering those aged 15 and over (without an age limit). Under the first amendment to the Loan Agreement and then later under the Project’s restructuring, the target for the number of youth and adults expected to pass a literacy test under this program was increased more that four-fold. In coordination with the municipalities, institutions of higher education, non-governmental organizations and civil society, the Project aimed to certify 400,000 previously illiterate citizens (against an original target of 90,000). The Project financed the educational materials for both student and teachers to support implementation of the Literacy Program. By mid-2007, 382,891 youth and adult had been certified, and another 42,084 were enrolled. In the end, a total of 423,500 youth and adults were certified under the Project. This was perhaps the most successful of the Project’s activities. In addition, this activity should have longer-

27

term benefits as the now larger literate population should hopefully result in a greater demand for education for their children. COMPONENT 3: Upgrading Education in Poorer Municipalities (US$11.8 million total project costs, 10.2 % project cost and 57.6% appraisal estimate) ICR Rating: Moderately Satisfactory This component aimed to improve educational results in the 54 poorest municipalities in Ceará through non-formal initial education and innovative fundamental education activities to increase the learning achievements of students in municipal schools. To participate, municipalities were selected on the basis of eligibility criteria that included: (a) relatively recent emancipation; (b) a relatively small territory; (c) low administrative capacity; (d) poor social infrastructure; (e) scattered rural population and (f) high ingrained poverty. The poorest municipalities were not beneficiaries of most social programs funded by the State or the Federal Government, and PRODEM was a compensatory strategy aimed a strengthening their education systems. To participate, municipalities needed to: (a) recruit and appoint all educational personnel under a transparent process; (b) participate in the SPAECE; (c) establish or enforce the Municipal Council for Children and Youth Protection; (d) provide evidence of compliance with the national children’s rights statute; (e) establish school-level governing councils in all schools, with representation of parent and community members; and (f) use the Education Development and Teacher’s Incentive Fund’s (Fundo de Manutenção e Desenvolvimento do Ensino Fundamental e de Valorização do Magistério or FUNDEF) resources satisfactorily. Once qualified through a process that was to be defined in the PRODEM Guidelines, the municipalities would be eligible for funding for: (a) formal and non-formal initial education and pre-school programs; (b) rehabilitation and furnishing of primary schools to ensure BOS; (c) teacher training and teaching-learning materials for multigrade and accelerated classes; (d) staff development training programs; and (f) technical assistance to implement their School Development Plans (PDE). The implementation of PRODEM got off to a slow start. In part this appeared due to a lack of preparedness when the Loan was approved. The guidelines that would define the operational aspects of PRODEM were finalized, and the program only began implementation after elections in 2004. Once it became operational, the government requested to increase the number of municipalities that would be eligible to 60 (from 54). The CREDEs assumed an important role in promoting the program, representing SEDUC at the regional level. The 60 eligible municipalities were supported with technical assistance and submitted a total of 157 PDEs and 143 projects for financing. Implementation of BOS by these 60 municipalities (against a target of 92, or 50% of municipalities) was an integral part of their PDEs. By 2005, SEDUC had signed the agreements with the 60 participating municipalities. Of these, 55 municipalities had completed their projects by the completion date, and the remaining five had completed 50% of their projects. The Project had expected to increase the number of students in early childhood education programs for children aged 0 to 3 from 11% to 22% (or increase enrolment to an additional 56,000 children). This target was reduced substantially through loan amendments and the Project’s restructuring, as the State decided to prioritize the early childhood programs for children aged 4 to 6. In the end, the Project, through PRODEM, in the 60 eligible

28

municipalities has enrolled about 11,030 children aged 0 to 3 in early childhood programs, bringing the total state enrollment in this age group to 58,489 children. The Project had expected to increase the number of students in early childhood education programs for children aged 4 to 6 from 72% to 80% (or increase enrolment to an additional 39.000 children). As this age group became a priority for the State, the Project, through PRODEM, increased its target to about 70,000 children aged 4 to 6 in early childhood programs. In the end, the Project, through PRODEM, has enrolled 33,871 children aged 4 to 5 in early childhood programs, bringing the total state enrollment in this age group to 181,290 children. It was not possible to evaluate the progress towards accomplishment of the target towards enrolment of children aged 4 to 6 in early childhood programs due to a change in age classification of the programs during implementation. In 2003, children aged 4 to 6 were considered enrolled in early childhood programs whereas in 2007, the year for which data is available to evaluate the Project’s progress towards its objectives, children six years of age are included in the numbers for fundamental education (i.e., the data for 2007 early childhood programs includes only children aged 4 and 5).6 COMPONENT 4: Promoting Decentralization and Institutional Development (US$25.85 million total project costs, 22.4% project cost and 93.6% appraisal estimate) ICR Rating: Moderately Satisfactory 4.1 Fostering School Autonomy Strategies. The Project aimed to support the design and implementation of the School Development Plan (PDE) strategy in approximately 1,370 schools (half state, half municipal, all with at least 250 students), and, in the process, enhance community/parent participation in the decision-making process at the school level. In 2004, SEDUC carried out a study aimed at eliminating the duplication of the existing school management programs. As a result of this study, SEDUC introduced a new, single school management tool, the Gestão Integrada da Escola (GIDE) in December 2004. SEDUC carried out intensive training of 635 school directors, technical staff of the CREDEs (316) and of SEDUC (39), to spearhead the preparation of the 2005-2008 GIDEs. All of the state schools delivered their GIDE by July 2005. Under the new process, schools and other units are to be evaluated annually on the basis of progress towards targets in their respective GIDE and their actual results. By project completion, 100% of state schools and 30% of municipal schools had developed and were implementing their GIDEs. 4.2 Strengthening Institutional Modernization. Several activities were envisaged and implemented under this subcomponent, to help achieve the objectives of Component 4, namely, to consolidate the decentralization process among central, regional, municipal and school levels, while strengthening school autonomy and involving the school and social community in the process. SEDUC’s 2003-2006 Education

6 In 2007, all of the municipalities in Ceará enrolled a total of 181,290 children aged 4 and 5 in early childhood programs against 197,572 children aged 4 to 6 enrolled in 2003. Similarly, the 60 PRODEM municipalities enrolled a total of 33,871 chidlren aged 4 and 5 in early childhood program against 352,222 children aged 4 to 6 enrolled in 2003.

29

Plan, contemplated a training program for the of educational and school administration staff of the CREDEs, and the schools’ administrative units. In all, this program included training for 40 school directors in a pilot program developed by MEC on school administration, for 178 school administrative staff in a program aimed at administrative practices geared towards educational achievement (implemented in coordination with a non-governmental organization), and for 4,308 staff of school administrative units and of the CREDEs in continuous education programs on educational and school administration. Technical and managerial training was provided not only to those to whom responsibility was being transferred (CREDEs and municipal school officials, mainly) but also to SEDUC officials who needed to have clear knowledge on how to support those levels more effectively. State and municipal level technical and managerial staff also received training under PROGESTÃO (Programa de Capacitação a Distância para Gestores Escolares), a program aimed at improving their performance and the quality of services in the educational system. An impact evaluation of PROGESTÃO revealed that there were significant changes in the practices of this group and of their teams, in their understanding of school administration, especially in the public sphere and in coordinated efforts to achieve improvements in school quality and the teaching-learning process. A total of 9,996 managers were trained, and additionally 3,654 concluded their specialization. Seventy school directors in the Fortaleza metropolitan area that had not participated in PROGESTÃO were awarded six month scholarships in school management. The Project also financed scholarships for 19 staff of SEDUC and the CREDEs in public policy at the Universidade Estadual do Ceará, and scholarships for 5 technical staff in curriculum development, educational sociology, and other subjects. To complement the implementation of PRODEM in the 60 targeted municipalities, officials of the municipal secretariats of education were provided with technical assistance and training in technical/administrative and pedagogical areas. The 60 municipalities under PRODEM were provided technical assistance through the CREDEs to develop and implement their PMEs and their early childhood and fundamental education programs. In all, 107 out of 184 municipal Secretaries of Education and 2,805 directors of schools with enrolment over 150 students were trained in educational and school management, through a decentralized program administered by the CREDEs. In 2004, SPAECE, the State’s educational evaluation system, was implemented in all state schools, and for the first time, extended to the municipal level. In all, 2,631 public schools (666 state schools and 1,965 municipal schools) and 187,577 students (72,787 in state schools and 114,790 in municipal schools) were evaluated. In addition, 2,600 school directors and 9,550 teachers responded to questionnaires. Like the national SAEB, the SPAECE evaluated performance of 4th and 8th graders in fundamental education, and those in the 3rd level of secondary education, in both Portuguese and Mathematics. As a follow up, technical reports, school bulletins and dissemination materials were prepared for distribution. The dissemination materials highlighted schools that had the best results. Between 2001 and 2004 SEDUC piloted SPAECE-NET, a large scale evaluation system utilizing computer aided testing. In 2006, the SPAECE was again implemented, this time covering not only all state schools, but virtually all municipal schools as well. The results of this latest test revealed a significant improvement in achievement in 8th grade Portuguese. The strongest improvement was in 4th grade Mathematics. The difference in SAEB and SPAECE results might be due to the smaller universe of SAEB which does not cover rural schools or schools with less than 200 students.

30

The Project also had an important emphasis on evaluation: SEDUC, CREDE and state school directors were trained in evaluation processes, and evaluations of the CREDEs and administrative units of selected state schools were carried out. These sought to evaluate performance on the basis of progress towards the accomplishment of results and targets. The Project also supported the carrying out of evaluations of three of the programs it supported: Magister, the teacher training program; PROGESTÃO, the training program for administrative skills; and Projeto Alfabetização é Cidadania, the literacy program. The evaluation of Magister found that the program brought positive results to the teachers that were trained, especially in view of the way in which the program was designed and organized by SEDUC and the theoretical-methodological strengths brought by the participating universities. It concluded that the program was an effective vehicle for continuous learning programs for teachers at all levels. However, in the early years of implementation, Magister was offered to all teachers regardless of age. Later on, acting on Bank advice, the government began to restrict the program to teachers that would stay teaching long enough to recover the investment. A significant number of beneficiary teachers retired soon after completing Magister (and received a higher pension for having completed it). The evaluation of PROGESTÃO found that participation in the program resulted in greatly improved performance by managers and their teams. This program is considered to have facilitated the process of transformation of the educational system to one centered more on the schools and their social environment. Similarly, the evaluation of the Literacy Program, which focuses on the performance of the various institutions and teams; the carrying out of the program, and the achievement of its targets, was generally positive. An impact evaluation of the Project is currently being carried out. 4.3 Building up Social Participation and Mobilization. In order to involve municipalities and communities in the process of developing their PMEs, the Project supported the dissemination of PRODEM. In June 2005, SEDUC and the CREDEs organized in all 14 regional and macroregional fora involving mayors, Municipal Secretaries of Education and their technical staff, to raise awareness of and promote support for the program. It plans to organize similar events with the participation of community members in participating municipalities to monitor the implementation of the PMEs. An additional seven macroregional fora were organized around the Literacy Program between June and July 2005, involving 146 municipalities and 378 participants. In April and June 2005 two fora on early childhood education were organized for Municipal Secretaries of Education, the CREDEs’ Pedagogical Coordinators and specialists in early childhood programs, for a total of 600 participants. Several activities were carried out aimed at promoting key stakeholder participation in and support to education and school-associated reform initiatives. These included, for example: (a) Mathematics and Portuguese Olympics; (b) school games geared at introducing state-wide school athletics (504 schools in 142 municipalities participated); (c) College preparatory programs targeted at 60 high-performing students; (d) SEDUC publications, to organize and publish institutional publications; (e) green school initiative, aimed at improving the school

31

appearance and integrating the school with the environment; (f) SAEB Learning agents, a program where 228 Portuguese and Mathematics teachers were trained to improve the performance of secondary students in the 2005 SAEB; (g) extended day environments, aiming at developing intersectoral programs, focused on the socio-cultural, economic and environmental aspects, so that the schools could be seen as pleasant environments, conducive to improved learning and strengthened integration between families and the communities; and (h) international seminar on quality school transport, through seminars on the subject, demonstration of products and services through promotional activities and round tables with the participation of the Ministry of Transport, and other public transport agencies. COMPONENT 5: Project Management (US$2.48 million total project costs including front end fee, 2.1% project cost and 25% appraisal estimate including front end fee) ICR Rating: Moderately Satisfactory The loan financed significantly less consultants’ services for Project Management than had been anticipated during preparation. Project management was largely carried out by SEDUC staff. The loan financed mainly consultants to assist and train UGP staff in planning, procurement and contracting of goods and services, and financial management and disbursement processing. The loan also financed the contracting of UNDP to assist in contracting consultants, both individuals and firms. Because of difficulties in procuring consulting firms, UNDP mostly assisted in contracting teachers and inputs for the training programs, paying per diems and providing teaching materials.

3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) The Project’s Appraisal Document presented a detailed economic and financial analysis. The Project was expected to have an economic rate of return of 23% based on several assumptions regarding its costs and benefits. The economic analysis has not been updated for this ICR. However, a qualitative judgment of the Project’s efficiency based on the assumptions in the economic analysis in the PAD is described below. The Project was expected to have several impacts. First, the speeding up of student flows was expected to accelerate the entry of students in the labor market. The Tempo de Avançar accelerated learning program for students in grades 5 to 8, who would complete secondary school in one year was expected to bring important benefits in the short term. In the medium term, the main impact was expected to come from the expansion of access that would be expected to generate a substantial increase in the number of students finishing fundamental and secondary education. In the medium to long term, the Project was expected to produce an improvement in the efficiency of Ceará’s education system through, in particular, reduced repetition and dropout rates, and increased completion rates. The Tempo de Avançar program performed above expectations in terms of enrollments, presumably contributing to the Project’s overall efficiency in the short term. However, it would be necessary to look at the repetition rates of this program, as this variable was one of the most critical variables in the sensitivity analysis. Unfortunately, however, the Project’s actual outcomes on dropout and, especially, repetition rates do not contribute positively to the Project’s overall efficiency.

32

As to the Project’s operational efficiency, expenses for Project Management under Component 5 were significantly below original estimates. Only US$ 2.5 million equivalent, out of the originally projected US$9.0 million were allocated to expenditures for Project Management.

3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) The Project’s overall outcome is rated Moderately Unsatisfactory. The Project’s relevance continues to be strong. It is consistent with the Brazil’s current development priorities, with the Bank’s recently-completed Country Partnership Strategy, with sectoral priorities and with those of the State of Ceará. Although the Project did make important contributions to improve education processes and some outcomes and strengthen SEDUC and its capacity to improve education outcomes in the State, there were important shortcomings in the achievement of its objectives, most notably educational achievement. The less than expected decrease in dropout rates and especially the marked increase in repetition rates in Project-supported schools reduced the Project’s expected efficiency. Although it is easier to track results attributable to the Project’s investments with these indicators (than for example to track educational achievements), dropout, repetition and completion rates are still influenced by a number of factors exogenous to the Project. Rating: Moderately Unsatisfactory

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development See Section 1.4 (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The Project had a significant impact on strengthening the institutional environment for education in Ceará. Slightly over 20% of the Project’s cost was devoted to activities specifically focused on supporting the ongoing decentralization process at the school, regional, municipal and central levels. Its focus on promoting school autonomy while upgrading the systems and procedures, and monitoring and evaluation strengthened the capacity of institutions at all in Ceará to continue pursuing its objectives of providing higher quality, efficient education programs for its citizens. Despite this, continued strong efforts will be needed to continue to improve monitoring and evaluation and ensure that the momentum created by the Project, especially in its final phase, will be continued. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A

33

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) N/A

4. Assessment of Risk to Development Outcome Rating: Moderate The risk to the Project’s Development Outcome is considered Moderate. While education undoubtedly continues to be a high priority for the State, the pace and ability with which the State can continue to address this priority will be a challenge. The Project succeeded in generating stakeholder support at all levels for its implementation modality, and made an important start in introducing and developing a culture of increasing accountability and efficiency in educational service delivery and introducing monitoring and evaluation. Still, as evidenced in the Project’s outcome indicators, a sustained effort over the medium- and long-term will likely be needed before the several activities supported by the Project---and others beyond its scope that undoubtedly impact learning---translate into higher learning achievement and more efficient educational systems. The institutional capacity and resources for sustaining this effort are already stretched. On the positive side, Bank support under the second phase of the Ceará SWAp should provide continued assistance to the State’s education sector. Still, it is not clear that this assistance modality, where the Bank’s hands-on technical assistance through supervision is limited, will be sufficient to maintain the continuous advice and support that had been provided under the Project.

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Unsatisfactory The Bank’s performance in ensuring Quality at Entry is considered Unsatisfactory. Project preparation was supported by a US$280,000 PHRD approved on December 16, 1999. Despite continuous support during its two-year preparation phase, Bank assistance in the preparation process did not ensure that the Project was well designed for the following reasons. First, the Project was over-dimensioned, contained a multitude of components some of which were not directly related, especially in view of the Borrower’s then limited institutional capacity and the multiple levels of government administration and other implementing agencies that would be involved. Preparation did not take into account the high degree of centralization in the State and the multiple bureaucratic processes that this represented for the Project. Second, as mentioned in Section 2.1 (ii), the Project was far from ready for implementation when the Bank’s loan was approved. In fact, it took a good three years from Loan approval for the Project’s implementation to begin in earnest. Third, and also as mentioned in Section 2.1 (ii), the PAD severely underestimated the risks the Project would face; in fact, it never even mentioned the most pressing issues it came to face throughout its implementation such as the change in the administration which adjusted the State’s priorities. Finally, the PAD

34

overestimated the Project’s expected outcomes, especially given the activities that were to be financed—several of which would not have a direct impact on the learning outcomes---and the lag with which the indicators selected could be expected to show improvement. Each of the reasons mentioned above influenced in a different way the Project’s moderately unsatisfactory outcome. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory Given the Project’s unsatisfactory Quality at Entry, the Bank’s supervision effort, especially during the latter years where implementation was in full swing, is considered Satisfactory. After a slow start in which little activity took place and supervision did little more than provide support so that implementation could begin, the Project began facing important challenges (Section 2.2). From then on, Bank supervision teams attempted to play catch-up addressing issues as they arose in an attempt to get implementation underway and on track. Its two first amendments were an attempt to address issues that arose (but that had not been identified as risks at entry), and focus the Project on the State’s changing priorities. Fiduciary problems plagued the Project throughout its implementation, and the Bank strived to address the issues as and when they arose by stepping up the hands-on support provided to the Borrower. A major effort by the Bank’s supervision team at identifying procedural steps at every administrative level of SEDUC, leading to procuring goods and services identified unnecessary steps and bottlenecks that were subsequently eliminated to the Project’s benefit. When implementation was finally underway for most project components, it became clear that reaching the Project’s objectives and expected outcomes was an increasingly difficult challenge. In a last ditch effort to get the Project to maximize its results given the already large implementation delays, the Bank worked closely with the Borrower to restructure and re-scale the Project to address its changing priorities (i.e., emphasis for the Literacy Program) and to limit the expected targets. Unfortunately, even a strong supervision effort was insufficient to compensate the Project’s many design flaws. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory The rating for the Bank’s overall performance is considered Moderately Satisfactory. Despite issue with Bank’s Role in Ensuring Quality at Entry, the strong, focused and pragmatic supervision effort was willing to emphasize project activities on the State’s current priorities, providing hands-on support to get implementation underway and providing realism as to what was realistically achievable. It was this effort that allowed the Project to achieve several of its intermediate, output targets, although it still fell short of several of its expected outcomes.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory The State’s overall performance is considered Moderately Unsatisfactory for the following reasons. First, despite its commitment to the Project, counterpart funding became an issue affecting implementation almost from the very start. It was only with the approval and implementation of the Ceará SWAp, which required satisfactory budgetary allocations to and implementation of the Project, that sectoral programs became adequately funded and

35

implemented. (Even so, by then the Bank had increased its disbursement percentage for most project components.) Similarly, the extensive and bureaucratic requirements for most procurement processes, that involved required approvals by agencies besides SEDUC, placed an unnecessary burden on the Project in its initial phases. These were streamlined significantly after the Bank’s supervision worked in close coordination with the State Government, and after approval of the SWAp which also addressed these issues. Finally, while elections and their results are outside the control of the State’s Executive, the two changes of administration that the State faced during the Project’s implementation resulted in a wide-spread change in staff responsible for implementation of its various activities. This created problems, especially with procurement, for which staff knowledgeable in procurement became scarce at several points during implementation. The main criticism that could be made of SEDUC’s performance relates to the large delays in taking action, once problems had been identified by supervision missions. There was also a lack of capacity in the State Secretariat of Planning and other central agencies that slowed all of the Project’s administrative processes, including procurement. Aide Memoires show that the alarms sounded by these missions, including precise prescriptions, were nearly always observed by the client, but with a very long lag. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory SEDUC’s performance is rated as Moderately Satisfactory. Despite its institutional constraints, SEDUC worked closely with the Bank on the Project’s implementation. The several staffing constraints of the PIU and SEDUC proper and other issues that arose were addressed, albeit often with long delays. SEDUC reached out to the CREDEs to assist in implementation, and worked well with municipalities in a new, decentralized environment. The sheer magnitude of the demands that the Project placed on SEDUC was large, but the implementing agencies worked closely in cooperation also with universities, non-governmental organizations and foundations to carry out project activities. At the same time, the Literacy Program and PRODEM introduced many agencies involved with the implementation of parts of the Project. This imposed an enormous burden on SEDUC’s already weak capacity to monitor its activities. Bank supervision supported SEDUC’s effort to monitor the quality of the various agencies, with the Bank’s fiduciary team often playing a vital role of overseeing activities in situs until SEDUC hired qualified staff to take over. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory Overall Borrower Performance is rated Moderately Satisfactory. Although it faced tough challenges in implementing a complex project in an often adverse environment, SEDUC did manage to pursue many activities well, often exceeding expected output targets.

6. Lessons Learned (both project-specific and of wide general application) Projects should be appropriately dimensioned, taking into account the number of activities, the number of implementing agencies and their implementing, and especially coordinating, capacity. With few exceptions, project preparation should ensure that projects are ready for implementation as soon as they are approved. Needed implementation manuals, inter-agency

36

agreements, and Terms of Reference for important activities, among others, should all be ready before Loan approval. Also, careful preparation of individual activities is important to eliminate inefficiencies that could otherwise be reduced with upfront planning. Projects’ expected outcome indicators should be realistic, and should relate directly to the activities financed by the respective project. They should be reviewed carefully to avoid over-ambitiousness, especially given the lag between intermediate outcomes and their impact on expected outcomes, and the many other exogenous variables that may have an impact upon their achievement. Most activities such as those supported by the Project require time and continued efforts to show results. Program-type operations, like the Ceará SWAp, can be effective vehicles to complement the implementation provided under SILs operating in the same environment, especially insofar as the overall regulatory and budgetary processes are concerned. They can ensure the allocation of budget to priority programs, execution of programmed budget and streamlining processes and taking other administrative actions. They do not, however, replace the hands-on technical advisory support that the Bank can provide to sectoral authorities during the ordinary course of supervision of SILs, or provide an instrument to that will lead to actual outcomes (as opposed to inputs and outputs). Ideally, the Bank should be able to provide both types of support to maximize the complementary nature of both, especially in a weak or changing institutional environment where the Bank’s hands-on technical advice can be considered important. Mono-sectoral SWAps (as SWAPs were originally intended to be) could be an alternative model to satisfy these complementary implementation objectives. When the establishment of a PIU is justified, it is important to ensure very close cooperation among the various agencies involved with a project’s implementation. Part of the difficulties faced in implementing the Project on time had to do with lack of responsiveness of the State institutions to the demands of the Project. This, in turn, resulted in delays in decision making by SEDUC with often disastrous consequences. For instance, delays in taking action to correct the unnecessary bureaucratic steps leading to procurement process delays and in staffing the PIU at critical junctures (e.g., in fiduciary functions as well as in monitoring and evaluation functions and in vital positions of coordination between the Project and SEDUC) were decisive factors in the difficulties in implementation faced by the Project. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The draft ICR was sent to the Government on November 20, 2008. No comments have been received. (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

37

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

IMPROVING SCHOOL QUALITY AND EFFICIENCY 65.7 38.89 59.19

EXPANDING ACCESS 22.1 36.53 165.29 UPGRADING EDUCATION IN POORER MUNICIPALITIES THROUGH A MUNICIPAL EDUCATION DEVELOPMENT PROGRAM (PRODEM)

20.6 11.86 57.57

PROMOTING DECENTRALIZATION AND INSTITUTIONAL DEVELOPMENT

27.6 25.85 93.65

PROJECT MANAGEMENT 9.0 1.58 17.55

Total Baseline Cost 145.0 114.66 79.07

Physical and Price Contingencies 4.1 0.00

0.00

Total Project Costs 149.1 114.66 79.07

Front-end fee IBRD 0.9 0.90 100.00

Total Financing Required 150.0 115.56 77.04

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 22.50 25.56 113.60 International Bank for Reconstruction and Development 90.00 90.00 100.00

38

Annex 2. Outputs by Component Output Indicators

Target: PAD Annex 1

Revised 1st Amendment

Revised Project Restructuring 3rd Amendment

Actual

Component 1: Improving School Quality and Efficiency Teachers completing university level programs

6,700 4,600 No change 3,657 teachers completed

Teachers with a non-education university degree receiving pedagogical training

460 Indicator dropped

No change Indicator dropped

Teachers receiving a secondary education diploma

3,400 preschool And 3,000 basic education teachers qualified

No change No change 692 early preschool and 3,974 basic education teachers qualified

Health Agents trained in educational strategies designed to upgrade their skills in early childhood development

9,000 Health Agents trained

2,000 Health Agents and 16,000 teachers in 40 municipalities (Public Health School Program) trained

1,300 Health Agents and 5,000 teachers in 40 municipalities trained

1,325 Health Agents and 5,397 teachers trained

Basic education schools built

38 schools 50 schools 35 schools 30 schools built and 2 schools under construction

Schools meeting Basic Operational Standards

709 schools No change 686 schools, representing 100% of state schools as of

100% of state schools

39

(BOS) July 30, 2006 Students enrolled in accelerated class programs, reducing the age-grade distortion

395,000 enrolled7

See break down in next two entries

No change

- Students enrolled in accelerated class programs (1st to 4th grades)

240,000 students enrolled


No change 121,551 students enrolled

- Students enrolled in Tempo de Avançar Program (5th to 8th grades)


No change No change 187,734 students enrolled

SAP strengthened in state schools

SAP fully operating in 21 CREDEs, all state schools, and in 20% of the municipal schools in each CREDE’s area

SAP fully operating in 100% of CREDEs

No change SAP fully operating in 100% of CREDEs

Component 2: Expanding Access Tailored education program for students in the 15 to 39 age group


120,000 young adults and adults passed the literacy test8

400,000 young adults and adults passed the literacy test9

423,500 young adults and adults passed the literacy test

New students attending TV mediated

210,000 new students covered

60,000 new students covered

Indicator dropped

Program dropped

7 In the PAD, the total number of students, from 1st to 8th grades, to be enrolled in the accelerated programs was 395,000, but the implementation letter defined 240,000 (1st through 4th grades) and 110,000 (5th through 8th grades), totaling 350,000.

8 The 1st amendment extended the students’ age range for the Tailored Literacy Program

9 450,000 youth and adults covered

40

classes (Telensino) Component 3: Upgrading Education in Poorer Municipalities through a Municipal Education Development Program (PRODEM) Number of municipalities that have implemented PRODEM

54 municipalities

60 municipalities

No change PRODEM implemented in 55 municipalities; an additional 5 municipalities are nearing 50% implementation

Children aged 0 to 3 included in the Early Childhood Education Program10

56,000 children included (in PAD this was stated as 11% to 22%)

30,000 children included


58,489 children included in all municipalities; 11,030 children in PRODEM municipalities

Children aged 4 to 6 included in the Early Childhood Education Program11

39,000 children included (in PAD this was stated as 72% to 80%)



181,290 children included in all municipalities; 33,871 children in PRODEM municipalities

Percent of participating Municipal Education Secretariats that have started to implement Basic Operational Standards

50% (92 of 184 total)

No change No change 55 Municipal Education Secretariats under PRODEM supported with technical assistance and 5 municipalities with implementation exceeding 50%

Component 4: Promoting Decentralization and Institutional Development Schools supported by projects that

1,370 schools No change No change 100% of state schools and 30% of

10 In PRODEM’s participating municipalities.

11 In PRODEM’s participating municipalities.

41

strengthen local autonomy

municipal schools have implemented PDE/GIDE12

PDEs developed in all state and municipal schools with at least 250 students

1,370 schools (approximately 686 state schools13, and 686 municipal schools with at least 250 students

No change No change 100% of state schools and 30% of municipal schools have implemented PDE/GIDE

CREDEs and SEDUC trained and equipped to cooperate with municipalities

Approximately 1,200 managers and technical staff trained to cooperate with the municipalities

Approximately 10,000 managers trained and 4,000 specialized

No change 9,996 managers trained and 3,654 concluded specialization

SPAECE expanded

To 100% of state schools and 30% of municipal schools

No change No change 100% of state schools and 30% of municipal schools

12 The PDEs evolved into the GIDEs (Gestão Integrada de Desenvolvimento da Escola) for the state schools. The PDEs are being implemented under the FUNDESCOLA Program in municipal schools.

13 The PDEs were financed by the then ongoing Bank-financed FUNDESCOLA Project in the remaining state schools.

42

Annex 3. Economic and Financial Analysis (including assumptions in the analysis)

See Section 3.3

43

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members Names Title Unit

Lending

Supervision/ICR Susana Amaral Financial Management Analyst LCSFM Laudo Bernardes Consultant LCSHE Tulio Henrique Lima Correa Financial Management Specialis LCSFM Nicolas Drossos E T Consultant LCSFM

Jason Anthony Hobbs Consultant LCSHS-DPT

Jose C. Janeiro Senior Finance Officer LOAFC Jose Parente Consultant LCSHD Daniela Pena De Lima Operations Officer LCSHH Armando Pinheiro-Castelar Consultant LCSHE Anemarie Guth Proite Procurement Spec. LCSPT Pedro Ravela Consultant WBIHD Luis F. Secco Consultant AFTH1 Sandra Monica Tambucho Perez Team Leader LOADM

Eduardo Velez Bustillo Sector Manager EASHD Tatiana Cristina O. de Abreu Souza Finance Analyst LOADM

Maria Madalena R. dos Santos Consultant LCSHD

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks USD Thousands (including

travel and consultant costs) Lending

FY98 22.61 FY99 113.31 FY00 52 239.96 FY01 10 11.95 FY02 0.00 FY03 0.00 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00 FY08 0.00

Total: 62 387.83 Supervision/ICR

FY98 0.00 FY99 0.00 FY00 8.73

44

FY01 13 35.68 FY02 19 53.09 FY03 23 68.60 FY04 38 101.66 FY05 32 119.83 FY06 43 148.07 FY07 37 124.55 FY08 12 71.53 FY09 1 0.00

Total: 218 731.74

45

Annex 5. Beneficiary Survey Results (if any) N/A

46

Annex 6. Stakeholder Workshop Report and Results (if any) N/A

47

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The draft ICR was sent to the Government of the State of Ceará on November 20, 2008. No comments have been received. An informal translation of the Summary to the Borrower’s Completion Report follows: 1. Project Data

• ORIGINAL PROJECT VALUE: US$ 150.0 million (US$ 90.0 million – IRDB and US$ 60.0 million – State Government)

• REVISED VALUE: US$ 112.5 million (US$ 90.0 million – IRDB and US$ 22.5 million – State Government)

• PROJECT CLOSING DATE: June 30, 2008. • NUMBER OF AMENDMENTS TO THE LOAN AGREEMENT: 3 (three). • SUMMARY OF THE NATURE OF THE AMENDMENTS First Amendment (December 16, 2003): i) Adaptation of the Ceará State Basic Education Quality Upgrade Project to the Ceará State Basic

Education Plan for 2003 - 2006; ii) Modification of the financing percentages, altered from 60% for the Bank and 40% for the State, to 80%

for the Bank and 20% for the State; and iii) Extension of the validity of the Loan Agreement to June 30, 2007. Second Amendment (April 1, 2005): (i) Redefinition of the activities eligible for the Project implementation; (ii) Operating adjustments; (iii) Award of study grants for the ongoing training of civil servants engaged in education. Third Amendment (April 18, 2007): (i) Expansion of the Literacy Training is Citizenship Project; (ii) Exclusion of the distance learning capacity building television course from the Project objectives; (iii) Modification of the financing percentages among the various categories, maintaining 80% for the Bank

and 20% for the State in the total amounts; (iv) Extension of the validity of the Loan Agreement to June 30, 2008

• REALLOCATIONS AMONG LOAN AGREEMENT CATEGORIES: December 28, 2007.

• FINAL REALLOCATIONS AMONG LOAN AGREEMENT CATEGORIES: October 2008.

• DATA ON THE LATEST SUPERVISION MISSIONS (i) March 2006 – Fortaleza, Ceará State (ii) June 2006 – Fortaleza, Ceará State

(iii) October − November 2006 − Washington, DC

(iv) May 2007 − Fortaleza, Ceará State (v) September 2007 – Fortaleza, Ceará State (vi) March 2008 – Fortaleza, Ceará State

48

(vii) September 2008 – Fortaleza, Ceará State.

New Millennium School Project – Executive Summary

49

2. General Remarks

The New Millennium School Project - PENM is the outcome of Loan Agreement Nº 4591 – BR, signed by the International Reconstruction and Development Bank (IRDB) and the Ceará State Government, worth US$ 90.0 million, with a State counterpart of US$ 22.5 million. Beginning in August 2001, its implementation period was extended to June 30, 2008 through the Third Deed of Amendment to the Loan Agreement. In the course of this Project, variations have occurred in the financing provided by the World Bank and the State Government (counterpart). Under the initial terms of the Loan Agreement, the financing provided by the IRDB accounted for 60% of the total amount proposed for the Project (US$ 150.0 million), equivalent to US$ 90.0 million, while the counterpart financing totaled 40% (US$ 60 million). With the advent of the First Amendment and the reduction in the amount corresponding to the State counterpart, the parity split was established at 80% for the IRDB, equivalent to US$ 90.0 million and 20% for the State, representing US$ 22.5 million, of the total value of the Loan, altered to US$ 112.5 million. The State complied and exceeded the proposed counterpart goal well before the termination of the Loan Agreement. The main objectives of the Project were: (a) To assure quality, efficiency and equity standards for rendering education services, in terms of aspects that

will upgrade the learning processes of the pupils, extending to all sectors of the schooling system. (b) To extend access to the Literacy Training Programs and Basic Education for drop-outs, young workers and

excluded young people through the use of alternative teaching and learning methodologies. (c) To foster the equitative supply of good quality services in Basic Education. (d) To strengthen in-school managerial and educational capacities at the central, regional, municipal and levels, in

order to ensure efficient supplies of Basic Education services. 3. Implementation of Components and Sub-Components and the Main Outcomes The processes required to implement these activities were defined in order to attain the objectives established for each Component and Sub-Component, with financing strategies that respected the procedures required for each expenditure in order to ensure the feasibility of acquiring goods, contracting works and commissioning consulting services within the pertinent outlay categories. The implementation of each Project Activity was related to one of the following expenditure categories: (a) Works; (b) Goods; (c) Consulting Services; (d) Training; (e) School Sub-Projects; (f) PRODEM Sub-Projects; (g) Operating Costs or Administrative Expenditures; (h) Literacy Training Sub-Projects. The following Table presents the outcomes attained through the main activities and programs implemented during the Project, inherent to its various Components and Sub-Components.


50

PROJECT COMPONENTS, SUB-COMPONENTS AND ACTIVITIES – MAIN OUTCOMES ATTAINED COMPONENT / SUB-COMPONENT MAIN PROGRAMS / ACTIVITIES OUTCOMES ATTAINED

1. Upgrading School Quality and Efficiency (a) University Level Teacher Training – University Teacher Training Program called Magister – Ceará, which is an initial training program (full teaching qualification) conducted through partnerships with the four public universities in Ceará State (UFC, UECE, UVA and URCA), in order to quality teachers in the State Network with high school educations.

3,657 teachers graduated, reaching 79.5% of the established goal (4,600 teachers graduated).

(b) High School Level Teacher Training – Early Childhood Education teachers – initiative implemented by SEDUC as an additional module of the PROFORMAÇÃO distance learning course offered by the Ministry of Education(MEC), in order to award high school diplomas to lay teachers in the MunicipalNetwork working with the initial grades of Primary School, meaning from first tofourth grades.

692 teachers trained (5th PROFORMAÇÂO module), 929 teachers (PROINFANTIL) and 134 faculty members (PILOT), representing 51.6% of the initial goal of 3,400 teachers qualified.

(c) High School Level Teacher Training - Primary School teachers - initial version of the PROFORMAÇÃO distance learning course whose purpose was to award high school diplomas to lay teachers in the initial grades of Primary School.

3,974 teachers trained.

(d) Capacity Building for Community Health Agents (ACS) and Teachers in forty municipalities – designed to upgrade the Indicators for government health and education policies through inter-sectoral actions that strengthen the competencies of schools and families in Ceará State.

1,325 ACS, representing 102% of the adjusted goal and 5,397 teachers in the initial grades of Primary School, reaching around 108% of the goal established in the Third Amendment to the Loan Agreement.

1.1 Fine-tuning Early Childhood Education and Teacher Development Programs

(e) Ongoing training for teachers at different levels and types of education – the main purpose was to upgrade the professional qualifications of teachers in various areas through specialty courses.

1,074 specialty grants were awarded for classroom studies and 209 for distance learning, totaling 1,283 teachers benefiting from specialty courses.

(a) Implementation of the Basic Operating Standards (PBF) in State schools

Construction of new schools 21 Primary and Secondary Schools completed and two schools with completion scheduled for the second half of 2008, underwritten by funds provided by the State Treasury.

Renovation and expansion of schools 35 schools renovated / extended, underwritten by the Project

1.2 Guarantee of the Basic Operating Standards for State Schools

Acquisition and distribution of equipment and furniture to new schools 4 indigenous schools equipped (completed by 2006); 23 Primary and Secondary Schools equipped and furnished, underwritten by the Project.


51

COMPONENT / SUB-COMPONENT MAIN PROGRAMS / ACTIVITIES OUTCOMES ATTAINED Acquisition of teaching and learning materials: IT Laboratories and

Science Laboratories 180 IT Laboratories and 16 Science Laboratories acquired 1.2 Guarantee of the Basic Operating

Standards for State schools (Continuation) Acquisition of books and other educational and teaching materials for

Basic Education subjects 25,000 books in languages other than Portuguese acquired for pupils in the seventh and eighth grades of Primary School; 2,115 copies of twenty titles related to the literature of Northeast Brazil acquired for the various subjects covered by Basic Education, in Primary and Secondary Schools in the State Network; 400 library collections acquired for the Book Chest (Arca das Letras) Project (semi-educational titles); educational media (videos and software) acquired for 180 IT Laboratories in Secondary Schools .

(a) Catch-Up Classes for first to fourth grades This activity reached 97.1% of the planned goal. 1.3 Reduction in Age x Grade Distortions through Catch-Up Programs

b) Time to Progress (Tempo de Avançar) Program / Catch-up classes from fifth to eighth grades

199,900 young people and adults enrolled in the Program during the 2002 – 2004 period, outstripping the goal established in the Project Outcomes Indicators.

1.4 Strengthening the Educational Monitoring and Support System

(a) Strengthening the SADRE 100% of the CREDE with the SADRE system in place and 100% of schools whose pupils are registered with the SADRE system.

2. Opening up Access (a) Updating and resizing the television education (Telensino) program Activity excluded from the Loan Agreement through the Third

Amendment to the Loan Agreement 2.1 Upgrading Access to Basic Education

for Drop-Outs and Excluded Young People (b) Support for Literacy Training for Young People and Adults – Literacy

Training is Citizenship Program 423,500 young people and adults awarded certificates.

3. Municipal Education Development Program – PRODEM PRODEM 55 municipalities with PRODEM implemented and five

municipalities with 50% of their Sub-Projects implemented, underwritten by the Project

3.1 Municipal Education Development Program - PRODEM

(a) Inclusion of children in the zero to three years age bracket Data from the 2007 School Census show 58,489 places in the zero to three years age bracket. In 2003, enrolment in the zero to three years age bracket reached 47,634 children. For municipalities with PRODEM intervention, in 2007, enrolments in the zero to three years age bracket filled 11,030 places.

(b) Inclusion of children in the four to six years age bracket (According to current Brazilian law, the existing data refer to four and five yearsage bracket, encompassing enrolment in early childhood education)

Data from the 2007 School Census show 81,290 places in the four and five years age bracket. In 2003, services in the four to five years age bracket reached out to 197,573 children. For municipalities with PRODEM intervention, in 2007, enrolments in the four to five years age bracket filled 33,871 places


52

COMPONENT / SUB-COMPONENT MAIN PROGRAMS / ACTIVITIES OUTCOMES ATTAINED 4. Institutional Strengthening and Decentralization 4.1 Strengthening School Autonomy (a) School Development Plan (PDE) / Integrated School Development

Administration (GIDE) implemented in government schools with at least 250 pupils.

100% of the State schools and 30% of the municipal schools with GIDE implemented.

(a) Building up the capacities of civil servants engaged in education

(b) Building up the capacities of State and municipal managers and technicians through extension and specialty courses – Distance Learning Capacity-Building Program for School Administrators (PROGESTÃO)

9,996 administrators and technicians trained through extension courses; 3,654 administrators and technicians trained through specialty courses and seventy administrators awarded study grants for School Management Training.

(c) Grants for Master’s Degree Courses 19 technicians from SEDUC and CREDE awarded Master’s Degree in Public Policies – by the Ceará State University and five technicians from SEDUC and CREDE studying for Master’s Degrees at the University of Minho, Portugal

(d) Minimum Operating Standards (POM) for Municipal Education Bureaus (SME)

SMEs in sixty municipalities encompassed by the PRODEM supported through technical assistance in the educational and administrative areas

(e) Permanent Basic Education Assessment System in Ceará State – SPAECE

100% of State schools and more than 30% of municipal schools encompassed by the SPAECE, exceeding the goal (100% of the State schools and 30% of the municipal schools).

(f) Assessment of the State Schools Administration Centre and the CREDE

State Schools Administration Centers and CREDE set up and assessed; administrators and technicians with SEDUC / headquarterstrained, pursuing institutional modernization and strengthening (ongoing training).

4.2 Strengthening Institutional Modernization

(g) Assessment of Programs and Projects The Magister and Progestão Programs and the Literacy Training is Citizenship Project assessed, underwritten by the Project.

(a) Inter-municipal and micro-regional forums in operation 21 forums established and functioning in each CREDE; 60 State Forums (macro-regional and regional) organized.

4.3 Promotion, Mobilization and Social Participation

(b) Social Exclusion and Participation – designed to encourage the engagement of the entire school community, strengthening State + Municipal Cooperation Schemes and upgrading the educational accomplishments of the pupils.

Many incentives encouraging participation, mobilization and social inclusion were implemented, such as participation in Math Olympics and Portuguese Language events in School Games and Educational Innovation Projects.

5. Project Management 5.1 Project Operating Costs (a) Hiring Consultants for building up the capacities of the Project

Management Unit - UGP Staff from the UGP and SEDUC trained in the fields of planning, acquisitions, financial management and disbursements.


53

4. Indicator Status

Table I A: Implementation Indicator Status at the end of the Project Indicators / Components / Sub-Components Previous Situation

(Third Amendment) Current Situation Goal at the End of the Project

1. Upgrading School Quality and Efficiency 1.1 Fine-Tuning the Early Childhood Education and Teacher Development Project University Level Teacher Training 3,657 teachers graduated 3,657 teachers graduated 4,600 teachers graduated High School Level Teacher Training a) Early Childhood Education teachers 692 teachers trained at high school level 692 teachers trained at high school level 3,400 teachers qualified

b) Primary School teachers 3,974 teachers graduated 3,974 teachers trained at high school level 3,000 teachers qualified

Capacity Building for Community Health Agents (ACS) and Teachers in forty municipalities

1,325 Community Health Agents and 5,397 teachers enrolled,

1,325 Community Health Agents and 5,397 teachers trained

1,300 Health Agents trained and 5,000 teachers trained in 40 municipalities

1.2 Implementation of the Basic Operating Standards (PBF) in State schools Implementation of the Basic Operating Standards (PBF) in State schools

90% of the State schools with PBF implemented



Building new schools 27 schools built and 5 schools in the

completion stage 30 schools built and 2 schools in the

completion stage (funded by the State Treasury)

35 schools built

Acquisition and distribution of equipment (to 35 new schools) 6 schools equipped 27 schools equipped 35 schools equipped 1.3 Reduction in Age x Grade Distortion through Catch-Up Programs a) Catch-up classes for first to fourth grades 121,551 new pupils enrolled 121,551 new pupils enrolled 140,000 new pupils enrolled

b) Time to Progress Program / Catch-Up classes for fifth to eighth grades 187,734 pupils enrolled (2002 - 2005) 187,734 pupils enrolled (2002 - 2005) 110,000 new pupils enrolled

1.4 Strengthening the Educational Oversight System

Strengthening do SAP – (SADRE) 100% of the CREDE with SADRE in

place 100% of the CREDE with SADRE in

place and 100% of the State schools with pupils registered with SADRE

100% of the CREDE with SADRE in place


54

Indicators / Components / Sub-Components Previous Situation

(Third Amendment) Current Situation Goal at the End of the Project

2. Opening up Access 2.1 Updating and resizing television education (Telensino)

Activity cancelled Activity cancelled Activity cancelled

2.2 Support for Literacy training for Young People and Adults 382,891 young people and adults awarded certificates

423,500 young people and adults awarded certificates


3. Municipal Education Development Program (PRODEM)

Municipal Education Development Program (PRODEM) in 60municipalities

55 municipalities with PRODEM implemented and 5 municipalities under

way

55 municipalities with PRODEM implemented and 5 municipalities with 50%

of the projects implemented PRODEM implemented in 60

municipalities

Inclusion of 50,000 new children in Early Childhood Education

a) Inclusion of additional children zero to three years age bracket School Census 2003

47,634 places (all municipalities) 10,607 places (municipalities covered by

the PRODEM)

School Census 2007 58,489 places (all municipalities)

11,030 places (municipalities covered by the PRODEM

15,000 children benefited

b) Inclusion of additional children four to six years age bracket School Census 2003

197,573 places (all municipalities) 35,222 places (municipalities covered by

the PRODEM)

School Census 2007 181,290 places (all municipalities)

33,871 places (municipalities covered by the PRODEM)

70,000 children benefited

4. Institutional Development and Decentralization 4.1 Strengthening School Autonomy School Development Plan (PDE) / Integrated School Development Management (GIDE) implemented in government schools with at least 250 pupils.

100% of the State schools with GIDE implemented and 30% of the municipal

schools with PDE implemented,

100% of the State schools with GIDE implemented and 30% of the municipal

schools with PDE implemented,

GIDE implemented in 100% of the State schools and 30% of the municipal

schools 4.2 Strengthening Institutional Modernization

Building up the capacities of State and municipal managers and technicians through extension and specialty courses (PROGESTÃO)

9,996 administrators and technicians trained through extension courses and

3,038 through specialty courses

9,996 administrators and technicians trained through extension courses and 3,654

through specialty courses

10,000 managers trained through extension courses and 4,000 through

specialty courses

Minimum Operating Standards (POMs) for 92 Municipal Education Bureaus (SMEs): 50% of the 184 Under way

SMEs in the 60 municipalities addressed by the PRODEM supported through technical

assistance (POM), 92 SME with POM


55

Scope of SPAECE 100% of the State schools and 30% of

the municipal schools 100% of the State schools and 30% of the

municipal schools 100% of the State schools and 30% of

the municipal schools assisted 4.3 Promotion of Social Mobilization and Participation

Inter-municipal and micro-regional forums in operation 60 macro-regional and regional State

Forums organized and 21 functioning in each CREDE

60 macro-regional and regional State Forums established and 21 functioning in

each CREDE 21 Forums functioning


56

Table IB: Outcome Indicator Status

Baseline14 Current Situation Goal at the End of the Project Indicators Quantitative or text Date Quantitative or text Date Quantitative or text Date

Increase of 15% over the outcomes of the SAEB 1997 for the number of pupils in Primary School in Ceará State who attained basic ability levels in Portuguese, Mathematics and Sciences

4th Grade Portuguese 42.5% 1997 49.0% 2005 49.0% 2006 Mathematics 10.0% 1997 11.5% 2005 11.5% 2006 Sciences 52.4% 1997 Not applicable Not applicable 2006 8th Grade Portuguese 53.4% 1997 61.0% 2005 61.0% 2006 Mathematics 12.3% 1997 14.0% 2005 14.0% 2006 Sciences 50.0% 1997 Not applicable Not applicable 2006 Increase in educational opportunities for young people and adults between 15 and 39 years old 1,200,000 illiterates 2000


and 42,084 new pupils in the classroom

2007 400,000 young people and adults trained in literacy 2007

Reduction in 5% in the drop-out rates in State and Municipal schools 15

12.4% State Network

11.4% Municipal Network 1999

4.8% 7.5% State Network


9% State Network

6% Municipal Network 2006

Increase in the Primary Schooling pass rate (First to Eighth Grades) 73% (*) 2000


82.1% Municipal Network 2007 80% (*) 2006

Reduction in the age x grade distortions in Primary School 55.7% (*) 2000 13.3% 2007 35% (*) 2006

14 Research sources: SAEB Reports, Report prepared by the National Household Sampling Survey (PNAD) / Brazilian Geography and Statistics Bureau (IBGE)

and SEDUC Reports; School Census Database and the National Institute for Educational Studies and Research (INEP).

15 The available index shows drop-out rates rather than truancy rates.


57

59.1% State Network 54.8% Municipal Network

16.9% State Network 14.3% Municipal Network

40% State Network 32% Municipal Network

Reduction in repeat year rates in Primary Schools for participant Municipal systems 16

6.6% State System

9.4% Municipal System 1999



11% State System

13% Municipal System 2006

16 The available index shows failing grades rather than repeat years.

New Millennium School Project - Final Project Report

58

The following Tables present the status of the Implementation and Outcome Indicators, referenced to the final Project Goals planned in the PAD and in the Third Amendment to the Loan Agreement. Although several goals have been reached from the quantitative standpoint, it is apparent that there are no conclusive data for assessing impacts on the quality of the outcomes attained. For this purpose, a consulting agreement is in place for conducting this assessment, particularly for the main programs implemented.

Project Costs and Financing by Component and Category (IRDB and Counterpart Funds)

Table II: Project Costs by Component (US$ million)

Nº Component IRDB State (Counterpart) Total

1 Upgrading School Quality and Efficiency 38.05 0.84 38.89 2 Opening up Access 14.52 22.01 36.53 3 Municipal Education Development Program

(PRODEM) 9.75 2.11 11.86

4 Institutional Development and Decentralization 25.45 0.40 25.85

5 Project Management 2.29 0.19 2.48 Basket of Currencies -0.06 -0.06

OVERALL TOTAL 90.00 25.56 115.56

Table III: Project Costs by Category (US$ million)

SITUATION AFTER THE THIRD AMENDMENT

CURRENT SITUATION

CATEGORIES IRDB SEDUC TOTAL IRDB SEDUC TOTAL % FINANCING FROM THE IRDB

1- Works (except works on School Sub-Projects and PRODEM Sub-Projects)

18.87 2.52 21.39 19.13 2.53 21.66 90% until the amounts drawn down under this category reach US$ 2.90, 75% up to US$ 9.50 and 100% from then on.

2- Goods (except goods for School Sub-Projects and PRODEM Sub-Projects)

9.45 0.50 9.95 9.75 0.53 10.28 90% until the amounts drawn down under this category reach US$ 4.00 and 100% from then on.

3- Consulting Services (except for PRODEM and School Sub-Projects, and Literacy training)


4-Training (except for School Sub-Projects and PRODEM Sub-Projects)


5- School Subsidies 14.01 3.40 17.41 13.74 3.34 17.08 90% until the amounts drawn down under this category reach US$ 3.00, 75% up to US$ 13.70 and 100% from then on.

6- PRODEM Subsidies 9.94 2.30 12.24 9.30 2.36 11.66 90% until the amounts drawn down under this category reach US$ 2.10, 75% up to US$ 8.40 and 100% from then on.


59

SITUATION AFTER THE THIRD AMENDMENT

CURRENT SITUATION

CATEGORIES IRDB SEDUC TOTAL IRDB SEDUC TOTAL % FINANCING FROM THE IRDB

7- Overhead / Administrative Expenditures

0.29 0.04 0.33 0.29 0.04 0.33 90% until the amounts drawn down under this category reach US$ 0.23 and 100% from then on.

8- Not allocated 0.00 0.00 0.00 0.00 0.00 0.00 distributed by Category according to the % established.

9-Front End Fee 0.90 0.00 0.90 0.90 0.00 0.90 amount due under Section 2.04 of the Agreement.

10- Literacy Training 21.56 11.99 33.55 22.12 15.01 37.13 90% until the amounts drawn down under this category reach US$ 2.50, 75% up to US$ 21,325 and 100% from then on.

90.00 22.50 112.50 90.00 25.57 115.56 5. Disbursements From the effective implementation of the Project on November 6, 2001, through to the expiry of the grace period on October 31, 2008, SEDUC submitted 45 requests for disbursements to the IRDB, totaling US$ 90,000,000.00 for financing under the Loan Agreement. Consequently, the full amount of the loan has already been paid out into the Special Project Account. A proposal was presented by SEDUC on final reallocations among the Project Categories and the substitution of amounts returned, which has already been approved by the Bank as well as by the International Affairs Bureau - SEAIN. The following Table presents the final situation for the placement of the IRDB funds by category, after approval of the above-mentioned final reallocation.


60

Table IV: Final Reallocation and Substitution of Returned Amounts (US$ million)

Nº Category Allocation under Loan Agreement

(Dec 2007)

Amount Disbursed through to July

2008 (Client

Connection)

Current Outstanding

Balance of the Loan

SOE 44 Distribution of Returned Amounts

Value of Returned

Amounts to be Substituted with

Expenditures paid by the

Government (SOE 45)

Total Amount Paid Out by the

End of the Project

Difference between the

Amount Allocated and the Amount

Paid out at the end of the

Project

Reallocation Amount

Difference between amount

Allocated and

Amount Paid Out

(%) A B C D E F G H I J A - B B+D+E+F A - G A - H

1 Works 18,870,000.00 18,113,577,62 756,422.38 1,021,052.27 19,134,629.89 -264,629.89 19,134,629.89 1.402 Goods 9,452,600.00 8,858,137,23 594,462.77 888,536.08 9,746,673.31 -294,073.31 9,746,673.31 3.113 Consulting

Services 6,984,200.00 6,686,451.47 297,748.53 22,113.02 6,708,564.49 275,635.51 6,708,564.49 -3.95

4 Training 8.002,215.00 8,021,785.30 -19,570.30 32,862.41 8,054,647.71 -52,432.71 8,054,647.71 0.665 School Sub-

Projects 14.008,810.00 14.006,921.32 1,888.68 231,344.92 -495,484.01 13,742,782.23 266,027.77 13,742,782.23 -1.90

6 PRODEM 9,936,100.00 9,399,246.38 536,853.62 28,511.04 -123,871.00 9,303,886.42 632,213.58 9,303,886.42 -6.367 Operating Costs 287,000.00 287,763.25 -763.25 287,763.25 -763.25 287,763.25 0.27

9 Administration Fee

900,000.00 900,000.00 0.00 900,000.00 0.00 900,000.00 0.00

10 Literacy training 21,559,075.00 21,390,490.20 168,584.80 111,207.49 -619,355.02 1,238,710.03 22,121,052.70 -561,977.70 22,121,052.70 2.61 TOTAL 90,000,000.00 87,664,372.77 2,335,627.23 2,335,627.23 -1,238,710.03 1,238,710.03 90,000,000.00 0.00 90,000,000.00

1

6. Audits The Audit Report on the Project for each previous financial year must be presented to the Bank by June 30 each year. The final audit of the Project Accounts (2008 financial year) was conducted by the Pelegrini e Rodrigues Auditores Independentes audit firm in late October, with the resulting Report being forwarded in a timely manner to the Bank for its approval by the end of this financial year. 7. Technical Cooperation Project - UNPD A Technical Cooperation Project was signed under the aegis of the PENM and the United Nations Development Program – UNPD: BRA/04/001 Project, coming into effect on August 4, 2004 through to June 30, 2007, worth US$ 8,337,000.00, of which approximately US$ 3.3 million were effectively paid out. The BRA/04/001 Technical Cooperation Project – New Millennium School (UNPD) was signed in order to speed up the implementation of the New Millennium School Project, covered by Loan Agreement Nº 4591-BR, financed by the World Bank, particularly for activities related to the “Consulting” and “Training” categories. The Project Document – PRODOC contained the Outcomes and Resources Matrix, addressing the main lines of the Project (IRDB), meaning the set of activities encompassed by the various Project Components. Due to operating difficulties, particularly with regard to diverging procedures for the acquisition and selection of consultants for these two entities UNPD and IRDB − the SEDUC Management decided to implement activities related only to hiring individual consultants (individual persons) especially capacity-building instructors, as well as all input materials required for the training activities, such as hiring a logistics support company, acquisition of educational materials, allowances and per diems for participants, among other aspects. As the flow of funds released by the UNPD is streamlined, it was convenient and appropriate to undertake these activities. Additionally, many consulting agreements were signed with individual persons for a wide variety of crucial tasks, while also ensuring the feasibility of obtaining the input materials required for the capacity-building activities, underpinning the satisfactory implementation of the PENM activities, which will have positive effects on the general outcomes of the Project / IRDB. 8. Final Remarks – Lessons Learned With the termination of the Project, it is appropriate to examine the lessons learned and difficulties encountered during its implementation. The launch of the Project was hampered by a series of difficulties and misunderstandings, which have been remedied in the course of its activities. A factor warranting particular attention was the attempt to implement the Project with financing similar to that of other projects under way at this Bureau, without setting up a specific management group for its administration. The main idea was to engage the various sectors in charge of implementing the actions, although without setting up a group trained in the management of a Project of this type, with direct financing from the World Bank, as earlier experiences were grounded on the implementation of subsidiary agreements through the Ministry of Education. One of the main difficulties detected during this initial experience of implementing a project with international financing was precisely in the human resources area. The SEDUC did not have any technical staff with experience in implementing this type of project. In order to quality the technical staff of this Bureau, some specific capacity-building actions were undertaken, although too brief and with insufficient detail, lacking the necessary follow-up required to bridge these gaps. On the other hand, the capacity-building experience of the SEDUC personnel developed on the job (from December 2005 through to late 2007) with the

2

consultants hired in the competitive bidding procedure and financial management fields was fully satisfactory, although covering only the last two years when the Project was being implemented. Taking decisions on hiring consultants whose profiles are more managerial and executive in order to provide support and streamlining some procedures related to the Project was a crucial step. The Bank has been conducting ongoing diagnoses of the weak points of the institution and drawing up forecasts of the future needs for technical staff, although the situation had to reach a critical state before the decision on hiring was taken. The composition of the Project Management Unit (UGP) in the organizational model was satisfactory for the Bank was attained only in mid-2003, continuing through to the end of the Project, although with a smaller staff. The main characteristic of this model was the full use of the SEDUC structure, coordinated by a technical and operating staff – UGP, set up close to the center where decisions are taken (Offices of the Secretary, the Deputy Secretary and the Executive Secretary). A noteworthy factor was the bureaucratic system and the slow implementation of activities by SEDUC, meaning that some Project activities required more than 150 days to reach the point of being tendered out. During the last three years of the Project, the hurdles presented by this system were noted, implementing measures designed to enhance its efficiency. In addition to the difficulties already mentioned above, others arose, prompted by alterations in the approach adopted by the State Education Policy. As this Project lasted seven years, changes in the Administration and consequently in the work teams resulted in major alterations to the Loan Agreement, as occurred with the inclusion of the Literacy Training is Citizenship Project, which was among the main changes covered by the First Amendment to the Loan Agreement. The process of allocating the Project content by each group taking over the administration of the SEDUC staff resulted in significant delays in the performance of the activities. Some difficulties were encountered by SEDUC in the progress of some actions at the Bank, especially in the field of competitive bidding procedures, causing delays in signing the Agreements. Outstanding among the matters prompting the widest repercussions was the requirement for outside inspections and testing for equipment procurement (Science and IT Laboratories and Special Education equipment); and resistance in accepting the procedures suggested by the Bureau for acquiring books for school libraries (procurement by batch rather than by title, as stipulated by the Bank), which resulted in a failed competitive bidding procedure. The supervision missions were rated as extremely important by SEDUC for tailoring the implementation of the actions, although acknowledging that neither the Bank nor SEDUC assigned high priority to establishing more efficient oversight and monitoring strategies for the actions, especially those implemented in a decentralized manner. It is suggested that new teams be set up (or hired) by SEDUC for future projects in order to oversee and assess actions under way, monitoring their outcomes. In fact, in terms of the difficulties encountered, it must be acknowledged that the PENM reached a satisfactory level of implementation for the planned actions. In many cases, the Government provided funding or drew on other sources of income in order to attain the objectives established by the Project, when it was not possible to finance the planned activities by the Bank, for some reason. The level of commitment to the Project in the Government, rating issues related to its operationalization as top priority, also contributed to the scope of the outcomes resulting from its implementation. The Project Impact Assessment Report, which is now in its final phase, will pinpoint the educational improvements arising from these investments, listing the main social benefits attained and modifications to the Education Indicators for Ceará State. Finally, the implementation of projects with foreign funding handled directly by the State may be considered as quite positive, especially in sector specific terms, as this enriches the expertise of the working teams at the central and regional levels, despite the difficulties encountered.

4

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A

5

Annex 9. List of Supporting Documents