document of the world bank...percentage of pronabes beneficiaries from poor households (income less...

Document of The World Bank

Report No: ICR00002186

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-73460 IBRD-73490)

ON

LOANS

IN THE AMOUNT OF US$171 MILLION TO THE

UNITED MEXICAN STATES

AND IN THE AMOUNT OF US$9 MILLION

TO BANOBRAS

FOR A

TERTIARY EDUCATION STUDENT ASSISTANCE PROJECT

IN SUPPORT OF

THE FIRST PHASE OF THE TERTIARY EDUCATION STUDENT ASSISTANCE PROGRAM

June 29, 2012

Human Development Sector Management Unit Colombia and Mexico Country Management Unit Latin America and the Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective April 11, 2012) Currency Unit = Peso (MXN$)

MXN1.0 = US$0.08 US$1.0 = MXN13.11

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ANUIES National Association of Universities and Higher Education Institutions (Asociación Nacional de Universidades e Instituciones de Educación Superior)

APL Adaptable Program LoanBANOBRAS

CIDE

CNPRONABES

CONEVAL

National Bank of Public Services and Works (Banco Nacional de Obras y Servicios Públicos, S.N.C.) Center for Research and Teaching in Economics (Centro de Investigación y Docencia Económicas) National Coordination of PRONABES (Coordinación Nacional del PRONABES) National Council for the Evaluation of Social Development Policy (Consejo Nacional de Evaluación de la Política de Desarrollo Social)

CURP DGAJ

National Identification Number (Clave Única de Registro de Población) Directorate General for Legal Affairs at SEP (Dirección General de Asuntos Jurídicos de la SEP)

DGAPRF Directorate General for Budget Administration and Financial Resources (Dirección General de Administración Presupuestal y Recursos Financieros) at SEP

DGENAM

DGEST

DGESU

GPA

Directorate General of Teacher Education and Training (Dirección General de Educación Normal y Actualización del Magisterio) Directorate General of Technological Higher Education (Dirección General de Educación Superior Tecnológica) Directorate General for Higher University Education at SEP (Dirección General de Educación Superior Universitaria) Grade Point Average

IADB Inter-American Development BankICEEQROO

IPDP

Student Loan Institute of the State of Quintana Roo (Instituto de Crédito Educativo del Estado de Quintana Roo) Indigenous Peoples Development Plan

MIS MOP

NAFIN

Management Information SystemProject Operational Manual (Manual de Operaciones) National Financier (Nacional Financiera)

PAD Project Appraisal DocumentPAEES Tertiary Education Student Assistance Project (Programa de Asistencia a

Estudiantes de Educación Superior)PAEIIES Program for Assistance to Indigenous Students in Tertiary Education Institutions

(Programa de Apoyo a Estudiantes Indígenas en Instituciones de Educación

ii

Superior) PHRD Program for Human Resources Development, grants financed by the Government

of Japan PND

PRONABES

PSE

National Development Plan (Plan Nacional de Desarrollo 2007-2012) National Scholarship Program for Higher Education (Programa Nacional de Becas para la Educación Superior) Education Strategy (Programa Sectorial de Educación 2007-2012)

SEMS Under-Secretariat for Upper Secondary Education at SEP (Sub-Secretaría de Educación Media Superior)

SEP Secretariat of Education (Secretaría de Educación Pública) SFP Secretariat of Public Service (Secretaría de la Función Pública)

SHCP

SOE

Secretariat of Finance and Public Credit (Secretaría de Hacienda y Crédito Público) Statement of Expenditures

STPS Secretariat of Labor and Social Protection (Secretaría del Trabajo y Previsión Social)

Vice President: Hasan A. Tuluy

Country Director: Gloria M. Grandolini

Sector Director: Keith E. Hansen

Sector Manager: Reema Nayar

Project Team Leader: Erik A. Bloom

ICR Team Leader: Erik A. Bloom

iii

UNITED MEXICAN STATES Tertiary Education Student Assistance Project in Support of

The First Phase of the Tertiary Education Student Assistance Program

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 12. Key Factors Affecting Implementation and Outcomes .............................................. 43. Assessment of Outcomes .......................................................................................... 154. Assessment of Risk to Development Outcomes ....................................................... 225. Assessment of Bank and Borrower Performance ..................................................... 236. Lessons Learned ....................................................................................................... 257. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 26Annex 1. Project Costs and Financing .......................................................................... 28Annex 2. Outputs by Component ................................................................................. 29Annex 3. Economic and Financial Analysis ................................................................. 31Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34Annex 5. Beneficiary Survey Results ........................................................................... 36Annex 6. Stakeholder Workshop Report and Results ................................................... 36Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 37Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ...................... 45Annex 9. List of Supporting Documents ...................................................................... 46

i

A. Basic Information

Country: Mexico Project Name: MX (APL I) Tertiary Educ Student Ass

Project ID: P085593 L/C/TF Number(s): IBRD-73460,IBRD-73490

ICR Date: 06/29/2012 ICR Type: Core ICR

Lending Instrument: APL Borrower: UNITED MEXICAN STATES

Original Total Commitment:

USD 171.00M Disbursed Amount: USD 171.00M

Revised Amount: USD 171.00M

Environmental Category: C

Implementing Agencies: Banco Nacional de Obras y Servicios Publicos S.N.C. (BANOBRAS) Secretaria de Educacion Publica (SEP)

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 03/14/2004 Effectiveness: 10/24/2006 10/24/2006

Appraisal: 04/15/2005 Restructuring(s): 03/01/2010

Approval: 12/13/2005 Mid-term Review: 4/15/2008 06/23/09

Closing: 03/01/2010 12/31/2011 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Moderately Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Moderately Satisfactory

Borrower Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory

Government: Satisfactory

Quality of Supervision: Moderately SatisfactoryImplementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Moderately SatisfactoryOverall Borrower Performance:


ii

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

Yes Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 5 5

Secondary education 5 5

Sub-national government administration 10 10

Tertiary education 80 80

Theme Code (as % of total Bank financing)

Administrative and civil service reform 17 15

Education for the knowledge economy 33 50

Export development and competitiveness 17 5

Indigenous peoples 17 10

Poverty strategy, analysis and monitoring 16 20 E. Bank Staff

Positions At ICR At Approval

Vice President: Hasan A. Tuluy Pamela Cox

Country Director: Gloria M. Grandolini Isabel M. Guerrero

Sector Manager: Chingboon Lee Evangeline Javier

Project Team Leader: Erik A. Bloom Andreas Blom

ICR Team Leader: Erik A. Bloom

ICR Primary Author: Todd W. Crawford

Abril Alicia Ibarra Castaneda

iii

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The objective of the Project is to support the Borrower's strategy to foster the sustainable and equitable and efficient expansion of tertiary education through student assistance through development of a coherent student assistance system consisting of student loan and grant programs, compensatory interventions for disadvantaged students, and supporting the national tertiary education policy. Revised Project Development Objectives (as approved by original approving authority) The revised objective of the Project was to support the Borrower's strategy to foster the sustainable, equitable and efficient expansion of tertiary education through student assistance through the development of a coherent student assistance system consisting of grant programs and compensatory interventions for disadvantaged students and supporting the national tertiary education policy. Project Indicators: Adjustments to some indicators were agreed with the Government based on the availability of new data but these were not included in a formal restructuring process (see para. 36). The following table discusses progress against these informally revised targets as well. (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Number of graduates from tertiary education in Mexico Value quantitative or Qualitative)

296650 432000 475584

Date achieved 10/01/2003 01/03/2009 12/31/2011 Comments (incl. % achievement)

End of Project target informally revised to 478000. Original target met. Informally revised target almost met.

Indicator 2 : Share of 18-24 year-old students in tertiary education from households classified at the two lowest quintiles of the income distribution

Value quantitative or Qualitative)

5.5 7.5 20.6


Baseline informally revised from 5.5% to 10% and end of Project target informally revised from 7.5% to 20%. Target met.

iv

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of PRONABES beneficiaries from poor households (income less than 1 minimum salary)

Value (quantitative or Qualitative)

16.64 20 18.8

Date achieved 10/01/2004 01/03/2009 12/31/2011

Comments (incl. % achievement)

Note: While targets were specified for ind 1-4, benef. came from households with below 4 min. salaries. PRONABES successfully increased the proportion of scholarships to students with income under 1 min salary, but fell short of the targeted proportion.

Indicator 2 : Percentage of PRONABES beneficiaries from poor households (income less than 2 minimum salaries)


44 40 40.2


PRONABES allocated 40 percent of scholarships to students with income between 1 and 2 minimum salaries thus achieving the targeted proportion.



28.1 33 31


PRONABES successfully increased the proportion of scholarships to students with income between 2 and 3 minimum salaries, but fell short of the targeted proportion.



6.22 7 10


PRONABES allocated a higher proportion than targeted of scholarships to students with income between 3 and 4 minimum salaries.

Indicator 5 : Percentage of PRONABES beneficiaries that were in the OPORTUNIDADES program during upper secondary education


8.72 10 17.9

v


Informally revised end of Project target: 20%. Original target met. Informally revised target not met.

Indicator 6 : Percentage of PRONABES beneficiaries that are female Value (quantitative or Qualitative)

55.8 56 56.71


Target exceeded.

Indicator 7 : Percentage of PRONABES beneficiaries that are Indigenous Value (quantitative or Qualitative)

5.2 10 12.55


Baseline informally revised from 5.2% to 6%. Target exceeded.

Indicator 8 : Terminal efficiency Value (quantitative or Qualitative)

n/a n/a

Date achieved 10/01/2004 12/31/2011


This indicator could not be measured during the Project's time frame as data on PRONABES beneficiaries was available after MIS was implemented-2008-09 school year. Will be measured starting 2013 (ftnt 13, para 37. See Table 1 for an alternate estimate)

Indicator 9 : Retention rate of PRONABES beneficiaries Value (quantitative or Qualitative)

65.3 75 87.3


The retention rate is defined as the sum of the number of grants renewed in t and the beneficiaries that graduated in t divided by the number of grants in t -1. Target exceeded.

Indicator 10 : Number of studies Value (quantitative or Qualitative)

0 4 2+

Date achieved 09/01/2004 01/03/2009 12/31/2011


Informally revised target of 3 largely met. Studies: perceived returns to secondary schooling, PRONABES impact eval. design. Also design of FM, MIS, revised PRONABES application, improved targeting to Oportunidades recipients and Indigenous students.

Indicator 11 : Number of PAEIIES beneficiaries (academic and non academic support to indigenous students to enable completion of their degree)

vi


n/a 3500 11918

Date achieved 09/01/2004 01/03/2009 12/31/2011


When Project funds first became available for PAEIIES in 2009, it had approx 9,600 beneficiaries. End of Project target informally revised to 10,000. The Project continued to support the existing beneficiaries but also enabled program expansion. Original

Indicator 12 : Change in retention rate of PAEIIES beneficiaries Value (quantitative or Qualitative)

n/a 25 20


End of Project target informally revised to 20 percent. Original target not met. Informally revised target met.

Indicator 13 : Average change in GPA of Beneficiaries (scale from 1 to 10) Value (quantitative or Qualitative)

n/a 0.5 -0.1


Target informally revised to 0.3. The GPA fluctuated marginally for each year of the Project. The -0.1 value reflects a marginal decline between 2010 and 2011. Target not met.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 12/29/2005 Satisfactory Satisfactory 0.00 2 05/01/2006 Satisfactory Moderately Satisfactory 0.00 3 11/22/2006 Satisfactory Moderately Satisfactory 0.43 4 05/15/2007 Satisfactory Moderately Satisfactory 0.43 5 07/08/2007 Satisfactory Moderately Satisfactory 0.43 6 11/27/2007 Satisfactory Moderately Satisfactory 9.21 7 12/21/2007 Moderately Satisfactory Moderately Satisfactory 14.54 8 03/20/2008 Moderately Satisfactory Moderately Satisfactory 36.26 9 12/14/2008 Moderately Satisfactory Moderately Satisfactory 47.02

10 03/28/2009 Moderately SatisfactoryModerately

Unsatisfactory 47.02

11 12/13/2009 Moderately SatisfactoryModerately

Unsatisfactory 82.41

12 12/20/2009 Moderately Satisfactory Moderately Satisfactory 82.41 13 05/25/2010 Satisfactory Moderately Satisfactory 119.65 14 02/15/2011 Satisfactory Moderately Satisfactory 167.14 15 08/15/2011 Satisfactory Moderately Satisfactory 167.14 16 12/28/2011 Moderately Satisfactory Moderately Satisfactory 167.14

vii

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

03/01/2010 Y MS MS 119.65

The Project was largely restructured to reflect the Government's focus on scholarships by eliminating components 4 and 5 and parts of component 1, and reallocating among components. The closing date was extended by 22 months and the APL structure dropped.

If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Satisfactory Against Formally Revised PDO/Targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal 1. The competitiveness of the Mexican economy was declining at the beginning of the 2000s. Despite entering the North American Free Trade Agreement, by 2003 Mexico lost its position as the United States’ second largest trading partner. On the World Economic Forum’s Global Competitiveness Index, Mexico slid from 34 in 1998 to 48 in 2004 (out of 104 countries). 2. A well-educated labor force was seen as one of the factors that would contribute to improved competitiveness by improving labor force productivity and shifting Mexico towards a more knowledge-based economy. The supply of workers with tertiary education was, however, constrained by a number of factors:

Enrollment in tertiary education in Mexico was low, only 21 percent of the 18-24 year-old population, compared with 31 percent in other middle-income countries.

Increased public financing to expand tertiary enrollment was considered to be unsustainable and inequitable over the long term.

Expanded enrollment in private tertiary education institutions was constrained by the cost, with the average tuition equal to 40 percent of average household income.

The availability of student loans was low, with 2 percent coverage. Access to tertiary education in Mexico was unequal, with less than 2 percent of

the 18-24 year-olds from the poorest quintile attending, compared with 26 percent from the wealthiest. Only 3.5 percent of Indigenous youth attended.

1.2 Original Project Development Objectives (PDO) and Key Indicators 3. The PDO from the loan agreement states “the objective of the Project is to support the Borrower’s strategy to foster the sustainable and equitable and efficient expansion of tertiary education through student assistance through development of a coherent student assistance system consisting of student loan and grant programs, compensatory interventions for disadvantaged students, and supporting the national tertiary education policy.” 4. There were inconsistencies within the Project Appraisal Document (PAD) and between the PAD and the Loan Agreement in how the PDO was stated and the version in the Loan Agreement was the legally binding one, and is the PDO used in the ICR to assess the Project’s impact. The main inconsistencies were: (i) whether the objective was to foster the “sustainable, equitable, and efficient” expansion of tertiary education, or more simply, the “sustainable and equitable” expansion; and (ii) whether such expansion

2

was to be achieved through “student assistance” or, more specifically, through such assistance consisting of student loans and grants. 1 5. Key indicators for rating Project outcomes are2:

Expansion of tertiary education as measured by an increase in the number of graduates.

Improvement in the equity of tertiary education as measured by an increase in the share of 18-24 year-olds in tertiary education coming from households in the two lowest income quintiles.

1.3 Revised PDO (as approved by original approving authority), Key Indicators, and Reasons/Justification 6. The revised objective of the Project was to support the Borrower’s strategy to foster the sustainable, equitable and efficient expansion of tertiary education through student assistance through the development of a coherent student assistance system consisting of grant programs and compensatory interventions for disadvantaged students and supporting the national tertiary education policy. The PDO indicators were not changed as part of the restructuring.

1.4 Main Beneficiaries 7. The primary target group was expected to be economically and socially disadvantaged tertiary education students who would benefit from scholarship and student loan programs. Upper secondary education students were expected to benefit from the early intervention program. In addition: (a) The National Scholarship Program, (Programa Nacional de Becas, PRONABES) would build its capacity; (b) The Federal Secretariat of Public Education (Secretaría de Educación Pública, SEP) would strengthen its capacity to make tertiary education policy; (c) the Student Loan Institute of the State of Quintana Roo (Instituto de Crédito Educativo del Estado de Quintana Roo, ICEEQROO) would benefit from financial and technical support; and (d) the National Association of Universities and Higher Education Institutions (Asociación Nacional de Universidades e Instituciones de Educación Superior, ANUIES) would benefit from a broadening of its assistance program to Indigenous students.

1 Within the PAD the versions of the PDO were as follow: (i) Data Sheet, “to foster the sustainable and equitable expansion of tertiary education through student assistance”; (ii) Section B.2, Program Objective and Phases, “strengthening the Government’s student assistance program to foster the sustainable, equitable and efficient expansion of tertiary education … through … [the] student loan and grant system”; (iii) Section B.3, PDO and Key Indicators, “to foster the sustainable and equitable expansion of tertiary education through student assistance … through ... student loan and grant programs”; and (iv) Annex 3, Results Framework and Monitoring, “strengthening the Government’s student assistance program to foster the sustainable, equitable and efficient expansion of tertiary education.” 2 The reference point in the ICR for the Key Indicators is the “Arrangements for Results Monitoring” framework found on pages 27-29 of the PAD. On page 9 of the PAD, some Intermediate Results Indicators are incorrectly identified as Key Outcome Indicators.

3

1.5 Original Components 8. Component 1 was to finance 58 percent of the federal share (29 percent taking into consideration the states’ shares) of the PRONABES scholarship program for academically qualified tertiary education students entering public tertiary institutions and demonstrating financial need. This component also included technical assistance, consultants, incremental operational costs, and coordination of state-level scholarship programs. It envisioned a number of studies aimed at strengthening PRONABES’s management information system (MIS), refining and verifying the application of its pro-poor and pro-Indigenous peoples targeting criteria, developing guidelines for state-level student loan programs, and assessing the feasibility of a reimbursable grant scheme. 9. Component 2 was to support activities carried out by SEP’s Under-Secretariat for Upper Secondary Education (Sub-Secretaría de Educación Media Superior, SEMS) to develop and disseminate a database on labor market opportunities for upper secondary and tertiary education students. This would strengthen the Labor Market Observatory, which the Secretariat of Labor and Social Protection (Secretaría del Trabajo y Previsión Social, STPS) had already developed. It was also to finance a number of studies to strengthen the analytical foundation for tertiary education policy.

10. Component 3 aimed to develop and pilot, possibly in two regions, an early intervention program to encourage talented students from low-income and Indigenous families who were receiving support through the Oportunidades program (a conditional cash transfer program) to complete upper secondary-level education and qualify for enrollment in a tertiary institution. Interventions to be considered (on a pilot-basis) included counseling, mentoring and tutoring, remedial and advanced placement courses, and preparation for tertiary admission tests. The component included support for the existing Program for Assistance to Indigenous Students in Tertiary Education Institutions (Programa de Asistencia a Estudiantes Indígenas en Instituciones de Educación Superior, PAEIIES). Activities included technical assistance to Academic Support Units located in Tertiary Education Institutions and support for coordinators, tutors, and other activities. 11. Component 4 was to be financed through a separate loan to National Bank of Public Services and Works (Banco Nacional de Obras y Servicios Públicos, BANOBRAS) and was intended to enable it to on-lend to ICEEQROO to support the agency’s lending program as well as institutional strengthening activities. The Project was designed to provide support to one state and Quintana Roo was identified as the target state. The size of the loan (US$9 million) reflected the needs of the state. The loan was to be guaranteed by the state and would capitalize a reserve fund to cover losses to put ICEEQROO on sustainable financial footing. 12. Component 5 was planned for later phases of the Adaptable Program Loan (APL) to promote an expansion of private commercial student loan agencies that could mobilize private capital and leverage public resources for tertiary education scholarships. The design of this component was based on a study on alternative financing for higher education that was to be undertaken in phase I under Component 5.

4

1.6 Revised Components 13. The February 2010 restructuring eliminated Components 4 and 5. This reflected the change in the PDO which eliminated student loans as a means to achieve Project objectives while retaining grants as a means. Parts of Component 1 were also eliminated including the development of the guidelines for state-level student loan programs and the assessment of a reimbursable grant scheme.

1.7 Other significant changes 14. The restructuring entailed some reallocation of funding between components and between financing categories. The changes introduced in the restructuring were carried out by changing the amount in each financing category, with PRONABES managing the corresponding changes in the budget for each component. Under Category 3 (Scholarships grants through component 1), the most significant change was the addition of US$3.8 million in IBRD financing for the PRONABES scholarship program. Under category 2 (Consultant services), IBRD financing for the promotion and information dissemination of PRONABES (Component 2.1) was reduced because the Government decided to rely more on its own resources and IDB technical support and successfully completed these activities. Under category 4 (Grants for PAEIIES, through Component 3.2), IBRD financing was reduced given delays in launching activities. The Project closing date was extended by 22 months. Finally, with the elimination of Components 4 and 5, which were to have been central to Phases 2 and 3 of the APL, the Government advised that the APL structure was no longer viable.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry 15. Project preparation, design and quality at entry had a number of strengths. Chief among these:

The Project identified the need to increase funding and improve the targeting of scholarships. Project preparation explored various options and provided significant support to PRONABES with the aim of improving its capacity.

Project preparation built on international experience and lessons learned in Mexico and other situations.3 Key lessons were: (a) scholarships combined with marginally subsidized student loans are necessary but not sufficient to increase enrollment by low and middle-income students; (b) equity in tertiary education required student financing as well as early intervention programs; (c) a public communications campaign was necessary to ensure awareness; and (d) a sustainable student loan program required a well-defined set of rules.

3 Some of the principal projects included the Mexico Higher Education Financing Project (4332-ME), the Jamaica Student Loan Project (4070-JM), and the Venezuela Student Loan Reform Project (3494-VE).

5

The Project developed an innovative Indigenous People’s Development Plan (IPDP). During Project preparation, the Government and the Bank looked at several alternatives and identified an existing functioning Indigenous Peoples Program, the coverage and functioning of which the Project could help improve.

The rationale for the APL instrument was strong. International experience had shown that reforming tertiary education policy and developing a coherent tertiary education student assistance program was a long-term process. During project preparation, it was clear that there was substantial uncertainty and piloting different approaches would be required.

16. At the same time, there were some weaknesses in Project preparation, design and quality at entry. The most significant of which were the following:

Project design did not adequately mitigate against the risk of change in government or other political development that might undermine support for the overall program or lead to its delay (this risk was only rated as Moderate). Change in the federal administration in December 2006 resulted in some overall delays, including in the provision of budget for some Project activities. This risk was also seen at the state level, with the change of the government of Quintana Roo, and a subsequent failure to reach an agreement on the legal and financial terms of on-lending loan proceeds.

Likewise, the Project design did not appropriately assess and mitigate against the risk that a continued economic downturn could adversely alter the landscape for successfully implementing the Project (this risk was rated Low). The possibility of an economic downturn that might derail the student loan components was not considered at all.

The lack of key agreements and understandings at the time of negotiations delayed loan effectiveness of Project implementation. In particular: (i) the PAD indicated that a draft loan agreement between BANOBRAS and Quintana Roo in support of Component 4 would be available by negotiations, this was not the case. Firm indications of both parties’ agreement could have provided an early opportunity to resolve them; (ii) similarly, the PAD indicated that a draft contract between SEP and ANUIES to support implementation of PAEIIES (through support to the Project’s Indigenous Peoples Development Plan), would be available by negotiations, but it was not finalized until 2009; (iii) requiring a clear indication that budget would be provided in a timely fashion for Components 2 and 3 could have smoothed implementation. As it was, such budget was not available until 2009.

The results monitoring framework was not fully developed. Several problems were known to exist with respect to the baselines and the Government’s ability to monitor indicators. These issues could have been resolved during Project preparation. As it was, monitoring remained a weakness through most of the Project.

17. In addition, the Project design included complex institutional arrangements and did not adequately take into account or mitigate against governance or capacity risks.

6

SEP was to implement Components 1, 2 and 3 through PRONABES and SEMS, with additional support from other directorates-generals of SEP. While it had authority to implement some components, it did not have direct authority over others, which were the purview of other departments in SEP or even of other external organizations:

PRONABES was a program under the Directorate-General for Higher University Education (Dirección General de Educación Superior Universitaria, DGESU) in the Sub-secretariat for Higher Education (Sub-Secretaría de Educación Superior, SES), which is part of SEP. The National Coordination of PRONABES, (Coordinación Nacional de PRONABES, CNPRONABES) was to have direct responsibility for the scholarship program under Component 1.1, establishment of the regulatory framework for tertiary education student assistance programs under Component 1.2, the institutional and policy strengthening activities under Components 2.2 and 2.3 and, in collaboration with ANUIES, an independent association of universities, the expansion of the assistance program for Indigenous students, under component 3.2.

SEMS was to be responsible for the design and testing of the early intervention programs for disadvantaged upper secondary students under Component 3.1. In doing so, it was to work in coordination with PRONABES and interested states.

SEP’s Directorate-General for Legal Affairs, (Dirección General de Asuntos Jurídicos, DGAJ) and the Directorate-General for Budget Administration and Financial Resources (Dirección General de Administración Presupuestal y Recursos Financieros, DGAPRF), were also to provide support.

18. In addition, CNPRONABES had a very small staff at the time of Project preparation – only two staff with no specialists. For example, this lack of capacity was the major contributor to the negative audit report on the management of a project preparation grant from the Program for Human Resources Development, PHRD. These governance and capacity risks to successful Project implementation were not considered adequately in Project design. 2.2 Implementation 19. The Project made some important contributions to tertiary education financing and policy in Mexico, by focusing on expanding and improving the PRONABES scholarship program as well as other support to tertiary education students, despite implementation delays in some components. 20. Component 1.1 (Support for the PRONABES National Scholarship Program) was disbursed in a timely manner and was, by far, the largest in terms of IBRD financing with US$165.8 million allocated to it through the restructuring compare to US$162 million originally. As PRONABES was allocated increasing federal and state budget4

4 According to PRONABES, in fiscal year 2001 their budget (including state and federal resources) was of MX$496 million; before the Project started in fiscal year 2005 the budget increased to MX$1.67billion and continue this trend up to MX$3.14 billion in fiscal year 2011.

7

and demonstrated that it could use it effectively, the number of PRONABES beneficiaries steadily grew at the federal and states levels from 44,422 (2001-2002 school year) to 161,787 before the Project started (2005-2006) and up to 310,690 beneficiaries in the 2010-2011 school year. 21. The implementation of Component 1.2 (Development of National Regulatory and Supervisory Framework for the National Financial Aid Program) was delayed pending the necessary budget authority, which was not forthcoming until 2009. However, after the funds were allocated, the Project made significant contributions to the design, development and implementation of PRONABES financing and statistical management systems (Management Information System, MIS), which were nascent and relatively undeveloped prior to the Project. The MIS, as well as Annual Meetings (also supported though the Project) improved, among others, the communication and flow of information between the states and the National Coordination which was better able to follow up and supervise the implementation of PRONABES at the state level. 22. With Project support, PRONABES improved its on-line statistical MIS (Sistema de Información Estadística de la CNPRONABES). This system was designed in 2009 and launched through a web hosting service in the 2009-10 school year. The statistical system has allowed PRONABES to generate systematic information at the national level through collection of 129 variables related to students’ personal and socioeconomic characteristics. The information for the system is collected from 2009 through an on-line national students’ application questionnaire which is comprised of six sections: (i) general characteristics, (ii) scholarship characteristics, (iii) previous studies, (iv) economic characteristics, (v) family characteristics and (vi) dwelling characteristics. This data has helped improve significantly the ability of PRONABES to target and supervise adequate targeting of scholarships at the state level. 23. The Project supported the redesign of the application questionnaire, building on lessons of experience from the Bank supported Oportunidades program, and improving the ability of the scholarship program to give priority, beginning in 2007, to students from the Oportunidades program, students with an Indigenous background, and eventually to students with disabilities5 . Recently, there has also been an important adjustment of eligibility requirements to better target poor students from large households6. The new application also requires PRONABES applicants to provide their national identification number (Clave Única de Registro de Población, CURP) which eliminates duplicate applications or applications from those benefiting from other scholarships and quickly enables PRONABES to reach its targeted groups.

5 The PAD considered as intermediate indicators the share of students from Oportunidades and with indigenous background receiving PRONABES (see page viii), but not the share of students with disabilities. Currently, about 17 percent of PRONABES beneficiaries have special learning needs. 6 As of 2012 the eligibility criteria has been changed from 4 minimum salaries per family (independently of the family size) to 4 minimum salaries per capita.

8

24. With Project support, PRONABES designed a new financial system (Sistema de Información Financiera para el seguimiento del Programa Nacional de Becas y del Programa de Asistencia a Estudiantes de Educación Superior). This system was designed in 2010 and piloted in 2011. The new financial management system has improved the ability of CNPRONABES to monitor the use of federal and state resources (disbursements) by state to supervise implementation at the state level. This is a useful tool to ensure transparency of PRONABES funds. The financial system has also improved PRONABES efficiency by facilitating a better flow of information among CNPRONABES, the states and the universities enabling, for example, the verification of timely release of scholarships to students. All this information is useful for the CNPRONABES to follow up with states that are facing challenges in targeting or delays in Program administration. 25. Component 2.1 (Promotion and Information Dissemination) was included in the Project to improve the information available to youth on the options available in higher education. The Government, with support from the IADB to the Federal Secretariat of Labor and Social Protection (Secretaría del Trabajo y Previsión Social, STPS), made efforts additional to those financed by the Project to strengthen the Labor Market Observatory fully meeting the objectives of component 2.1. In addition, SEP also created its own website, “Decide Your Career” (www.decidetucarrera.ses.sep.gob.mx) based on the Labor Market Observatory that further increased access to information for youth.

26. Component 2.2 (Analysis and Institutional Framework – Regulatory Framework) was severely affected by the change of federal administration in December 2006, just as the Project became effective. Very few activities were implemented under component 2.2. There have been promising recent developments however. In 2011, the Project supported an international conference on the feasibility of developing a student loan system for tertiary education (1er. Foro Internacional sobre Modelos de Crédito Educativo. Experiencias en su Diseño, Operación e Impacto) in Tijuana, Mexico. The objective was to provide an analytical base grounded in international experience to strengthen tertiary education policy through discussing financing alternatives for tertiary education and the pros and cons of an income contingent reimbursable grant program. A few months after the conference took place, Nacional Financiera (NAFIN) expanded its budget for university loans and some states have reinforced their student loan agendas and reoriented education policies consistent with international experience. 27. Component 2.3 (Analysis and Institutional Framework – Evaluation of Student Assistance Programs). Agreement between the government and the Bank on the TOR for an impact evaluation of PRONABES was delayed by the need to resolve methodological difficulties and the study was not completed before Project closing. However, the Project financed the design of the PRONABES impact evaluation and the Government is committed to continue with this study. For example, SEP has allocated budget for the baseline study and got the formal approval from the National Council for the Evaluation of Social Development Policy (Consejo Nacional de Evaluación de la Política de Desarrollo Social, CONEVAL) on the feasibility study and the TORs for continuing with

9

the impact evaluation. The National Coordination of PRONABES is now in the process of hiring a firm for carrying out the baseline study. 28. Component 3.1 (Early Interventions to Assist Talented Disadvantaged Students) Due to complications in working across different sub-secretariats in SEP, funding was not available until 2009 to support a study that focused on encouraging upper secondary school students to continue studying tertiary education. The study was carried out by SEMS in November 2009 and included piloting and data collection to establish a baseline. It followed an experimental methodology where a treatment group of students in their first and fifth semester were provided with information about the returns to upper secondary and higher education mainly through a computerized interactive software. The sample was representative at the national and regional levels. Two follow up studies were planned, one for November 2010 and the second one for November 2011; however, they were not carried out due to changes in the SEMS administration. Complementary funding from a World Bank Research Support Budget grant (USD$100,000) has supported a follow up study in 2012 in the central region of Mexico with those students that were in their first semester of EMS when the baseline was collected and that are now in their sixth and final semester. The follow-up study will help estimate the impact that information on the returns to schooling have on dropout rates and transition to tertiary education. The results are expected to be available in December 2012 and disseminated, including on the SES website.

29. There were significant delays in implementing Component 3.2 (Expansion of the Indigenous Students Assistance Program) due to delays in establishing the contract between SEP and ANUIES. This component helped finance the Support Program for Indigenous Students in Higher Education, PAEIIES, which was originally created in 2001 by a Ford Foundation grant to ANUIES. Following international best practice, PAEIIES supported universities to establish Academic Support Units to provide academic and non-academic support to Indigenous students, including tutoring, counseling, support to apply for financial aid programs (including PRONABES), and orientation to new entrants. While project financing started later than expected (in 2009), PAEIIES received additional financing from the Ford Foundation through the end of 2009, so that there was no financing gap prior to Project funding beginning. By the time Project funds reached ANUIES, PAEIIES reported it had about 9,600 students enrolled. The Project enabled continued support to these existing students, as well as expansion of the program to almost 12,000 students. Disbursements under PAEIIES were slow reflecting both uncertainties by the Bank, SEP, and ANUIES regarding appropriate disbursement arrangements, as well as ANUIES’ weaknesses to handle the large amount of Project funding. About 75 percent of the loan proceeds earmarked for ANUIES were disbursed in the last six months of Project implementation. 30. Several factors contributed to the inability to implement Component 4 (Support for state-level student loan programs) and 5 (Promotion on Private investments in student loans). The change in state administration in Quintana Roo led the State Legislature to re-examine the financial and legal terms of the loan and alter the draft debt decree in a fashion unacceptable to BANOBRAS. For its part, BANOBRAS changed its

10

terms from a fixed interest rate to a rate that would float at disbursement and required a specific source of funding to guarantee repayment, reflecting its concerns about ICEEQROO’s sustainability. When BANOBRAS was unable to reach agreement with Quintana Roo state authorities on the legal and financial terms of on-lending loan proceeds, no other state expressed interest in such a program. At the federal level the government decided to postpone the introduction of the student loans. The economic crisis further reduced interest in student loan programs at both the federal and the state levels7. 31. A number of difficulties emerged in putting institutional arrangements into practice, difficulties that were compounded by the fact that loan effectiveness occurred soon after a change in the federal administration. The PRONABES National Coordinator, who was expected to manage the Project, had no authority outside of PRONABES, making it difficult to coordinate and lead programs in other areas of SEP, other federal programs such as Oportunidades and PAEIIES, ANUIES, and state-level institutions. The initial small size of CNPRONABES added to the difficulties. For instance, the PRONABES team did not have enough capacity to execute the Project and had little knowledge and experience in World Bank Project management. Therefore, it took time for them to understand how budget requests for the Project needed to be processed at the country and Bank levels, contributing to delays in the availability of the funds for executing project activities. Capacity constraints eased as the staff of PRONABES grew to 49 employees, and Project support for capacity building, especially with respect to targeting scholarships, M&E and carrying out fiduciary responsibilities, materialized. Only from 2009, PRONABES was authorized to charge its operational costs to its own budget and therefore to hire its own personnel with specific qualifications to improve the program’s equity, sustainability and efficiency. PRONABES now has staff expert in scholarship programs, socioeconomic analysis, management information systems and financial management. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design 32. The design of the M&E framework had some weaknesses. Only one of the two PDO outcome indicators had a direct bearing on the equity dimension of the PDO. There were no PDO-level indicators pertaining to sustainability and efficiency of the expansion of tertiary education through student assistance. 33. The PDO indicators focused on the total number of graduates from tertiary education8 and, to measure the equity dimension of the PDO, on the share of tertiary

7 Mexico was hit by the 2008-09 global recession due to its reliance on trade with the United States, the

outbreak of pandemic influenza, and its reliance on oil exports. Between 2008 and 2009, GDP fell 6.1 percent and as a result, poverty increased from 44.5 to 46.2 percent of the population. 8 This indicator takes into account higher education graduates from both, private and public institutions offering higher education.

11

students from the two lowest income quintiles. Intermediate Results Indicators that measured improved equity included detailed monitoring of which low income groups (disaggregated by one, two, three and four minimum salaries) benefited from the PRONABES scholarships. This intermediate indicator helped monitor that those in the two lowest quintiles of income distribution benefited most since the income of four minimum salaries has since 2006 been less than the income for the two lowest quintiles on the income distribution9. Other intermediate results indicators measured the percent of PRONABES beneficiaries that were in the Oportunidades program during secondary education, the percent of female PRONABES beneficiaries (although there were no Project activities that focused on gender) and the percent of Indigenous PRONABES beneficiaries. 34. The Results Framework included one intermediate results indicator to measure the sustainability dimension of the PDO: the default rate of student loans which became irrelevant to the Project when the student loan activities were restructured out of the Project in March 2010. Other parts of the PAD, however, made it clear that financial sustainability of PRONABES would depend on continued one-to-one participation between federal and state governments in the financing of all grants, and also highlighted the possible introduction of a reimbursable grant scheme. The PAD also emphasized that political support for PRONABES would depend on rigorous program evaluation and dissemination of outcomes at the federal level. At the state level, consultations with tertiary education institutions in each of the states to ensure that PRONABES growth took into account the needs of tertiary education institutions as well as continuous dissemination of program operations and results through state websites and mass media were considered key. Finally, the PAD envisioned that PAEIIES’ grant financing and Academic Support Units would be institutionalized within supporting tertiary education institutions.

35. The Results Framework included five intermediate results indicators related to the efficiency dimension of the PDO: terminal efficiency of PRONABES beneficiaries10, retention rate of PRONABES beneficiaries11, and efficiency in operational costs of the state-level student loan program (which became irrelevant to the Project following the March 2010 restructuring). An increase in the number of PAEIIES beneficiaries and the change in their retention rate were also included as an intermediate result.

Implementation 36. Some of the baseline and target numbers to measure the indicators in the PAD were inconsistent through the text and in the Arrangements for Results Monitoring. In

9 For instance in 2010-11 four minimum salaries per capita were equivalent to MX$1,720 (around US$130) whereas the income in the two lowest quintiles on the income distribution for the same period was MX$1,920 (around US$145). 10 Terminal efficiency is defined on page 28 of the PAD: the ratio of the number of students in cohort t completing their degree and the size of the same cohort of students that began their studies 5 years earlier. 11 Retention is defined on page 28 of the PAD as the sum of the number of grants renewed in t and the beneficiaries that graduated in t divided by the number of grants in t-1.

12

addition, some of the indicators were hard to measure at the beginning of the Project since PRONABES did not have the human or technical capacity to provide accurate data. During the June 2009 mid-term review, PRONABES calculated revised baselines and targets for some indictors, primarily at the intermediate level. The revisions were not included in the March 2010 restructuring since further revisions to the MIS were being made. Once the MIS system was developed the quality and completeness of data, monitoring, and reporting gradually improved. Some baselines and targets were again informally revised and agreed between CNPRONABES and the Bank team in 2011, and reflected in the revised Project Operational Manual (Manual de Operaciones, MOP)12. At that time, most Project funds had been disbursed and the revisions were not submitted for formal Bank approval through a Project restructuring.

37. Making the Project M&E framework a useful tool was a protracted process. Due to the late processing of national annual budget approvals for conducting Project activities, and a range of design complexities, work to strengthen the MIS of PRONABES did not near completion until late in 2009. The data reported by ANUIES was usually inconsistent given the high turnover in PAEIIES staff and delayed and sometimes inconsistent data reported from universities. ANUIES gradually improved the quality of its M&E and increased its reporting once Project resources became available. 38. Terminal efficiency of PRONABES beneficiaries proved particularly difficult to measure since it requires tracking PRONABES students for all 5 years to degree completion and data collection has only been available at the national level since the 2009-10 school year. Therefore, it will only be possible to measure terminal efficiency for the cohorts that started their studies on or after the 2009-10 school year and the data for this first cohort will only be available in 2013-14. Instead, PRONABES has been monitoring how many PRONABES beneficiaries stay in their programs of study from one school year to another and through the end of the program.13 39. The ICR team constructed a set of PDO Indicators to measure the equity dimension of the PDO for purposes of the ICR, building on the Intermediate Results Indicators in the PAD that measure outcomes directly attributable to Project activities. Based on references in the PAD to the sustainability and efficiency objectives as described above, as well as intermediate results indicators that measure Project impact, the ICR team also constructed a set of indicators against which achievement of sustainable and efficient expansion of tertiary education through student assistance could be measured (See Section 3, Table 1 for a list of PDO indicators used for purposes of the ICR). PRONABES National Coordination is also now able to measure the direct contribution of the program to the transition rate from upper secondary to public higher education as well as the percentage in which the program has contributed to the number of students graduating from higher education which are also used to complement the discussion in Section 3.

12 Aide Memoire signed between CNPRONABES and the Bank team in May 25, 2011. 13 This is complicated by the fact that students occasionally leave PRONABES but continue studying.

13

Utilization 40. While targets were never formally revised, the informal revisions served as useful benchmarks and goals against which Project progress could be meaningfully monitored. In addition, CONEVAL has prepared and PRONABES has published on its external website 5 evaluations of this scholarship program. The more recent of these evaluations Consistency and Results Evaluation of PRONABES (2011-2012) (Evaluación de Consistencia y Resultados de PRONABES, 2011-2012.), for example, draws on data from the new financial and statistical systems supported by the Project. In addition, two impact evaluations were initiated under the Project: one on PRONABES and one study on an early intervention program to assist talented disadvantaged students.

2.4 Safeguard and Fiduciary Compliance

Safeguard Compliance

41. The sole Safeguard policy triggered by the Project was O.P./B.P. 4.10, Indigenous Peoples. Indigenous youth in Mexico faced a number of disadvantages, including lower income, poor academic preparation, and limited information. The Indigenous Peoples Development Plan (IPDP) sought to address these disadvantages by targeting financial assistance to Indigenous students and providing them with non-financial assistance such as tutoring, health and family life education, academic support, remedial courses, and institution-based coordinators for Indigenous students with the goal to raise the retention rate and their level of academic performance. The PAEIIES program was to be the main channel for this assistance. 42. As part of the IPDP, PRONABES committed to provide a scholarship to all students being supported through the PAEIIES program. The Project originally budgeted US$4.69 million for Component 3.2 to expand the PAEIIES program to establish new Academic Support Units at 8 new tertiary education institutions and to assist at least 3,500 Indigenous students. 43. Implementation of the IPDP through the Project was poor until the later stages of Project implementation because Project funds were slow to reach ANUIES for Component 3.2. The chief impediment was the delay in signature of the agreement to transfer loan proceeds from SEP to ANUIES for PAEIIES in the form of a grant. Although the Bank continued to underscore in its dialogue with its counterparts the urgent need to proceed with this component, the delay reflected country legal and administrative complexities, which were beyond the scope of the Bank, and ANUIES did not receive Project funds until late in 2009, roughly three years after loan effectiveness. 44. The Project complied with the Indigenous people safeguard through PAEIIES which specifically targets Indigenous students in tertiary education. PAEIIES subcomponent exceeded its original target of 3,500 students under the program as well as its informally revised target of 10,000. For its part, PRONABES continued to expand its scholarship support to Indigenous students and exceeded its end-of-Project target of 10

14

percent of such students among its recipients. By December 2011, 12.5 percent of PRONABES beneficiaries were Indigenous students.

Fiduciary Compliance

45. There was a limited amount of procurement under the Project. With training and support from BANOBRAS and the Bank specialists to SEP and PRONABES, this procurement was done in accordance with Bank policies, although sometimes with delay, for a variety of reasons including delays in updating procurement plans and tight submissions by the implementing agency and queries from the Bank. There were, however, persistent financial management problems throughout Project implementation. These included qualified audits of use of loan proceeds, slow submission of disbursement requests, delayed financial management reports, and the need for reclassification of some statements of expenditures (SOEs). The lack of electronic submission of procurement claims on a rolling basis contributed to a lag of approximately two weeks between the request for disbursement by the government and actual disbursement of the last payment. 46. Lack of familiarity with the Bank’s financial management policies was a factor contributing to the financial management problems. The need for training to strengthen the capacity of PRONABES in this regard was recognized from the outset. Training in Bank financial management, procurement, and disbursements was provided in June 2007 following a specialized financial management supervision mission. An action plan was agreed to address the continuing concerns about weaknesses in financial management, including gaps in record keeping, the lack of timely and reliable data, and an unfavorable external audit of management of the PHRD project preparation grant. With support from Bank and BANOBRAS specialists, the action plan was substantially implemented by May 2010.

2.5 Post-completion Operation/Next Phase 47. The government has shown persistent interest in a follow-on project to support its efforts to strengthen access to tertiary education. Discussions of a possible project are centering on consolidation of improvements to date through further refinements in targeting of PRONABES financial support and additional steps to improve efficiency. A new project might also focus on greater coordination between PRONABES and other more specialized scholarship14 and loan programs which might include a reimbursable grant scheme. Consideration is being given to including as part of the project renewed support for developing the policy, regulatory, and financial framework for student loans from both public and private sources as discussed at the September 2011 conference that reviewed international experience of student loan programs and reimbursable grant schemes. In addition, both impact evaluations initiated under the current Project are expected to be completed in December 2012.

14 There are different tertiary education scholarship programs run by the Sub-secretariat of Higher Education. For instance scholarships for: training, social service, high achievement, transport, food, and for graduation.

15

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation Rating: Substantial 48. The objective of supporting the government’s strategy to foster sustainable, efficient, and equitable expansion of tertiary education through student assistance continues to have substantial relevance to Mexico’s development needs. Support for low-income students to enter tertiary education is a central part of both the Government’s National Development Plan (Plan Nacional de Desarrollo 2007-2012, PND) and its Education Strategy (Programa Sectorial de Educación 2007-2012, PSE). The Project is relevant to the third pillar of the PND “Equality of Opportunities” which aims to provide support to the population with greater needs to improve their health and education. The Project is also relevant to the second objective of the PSE which aims to increase education opportunities to reduce inequality among social groups, close gaps and improve equity. In addition, the strategy explicitly mentions the intention “to increase the coverage of PRONABES in all school systems to support economically disadvantaged students with federal and state resources.”

49. The March 2010 Project restructuring responded to the government’s changing approach to higher education financing, which put a greater emphasis on scholarships and largely put exploration into student loans and a reimbursable grant scheme on hold during the economic downturn and the lack of agreement with the state of Quintana Roo on its loan program. The Government is now planning the introduction of a federal student loan program. The Project’s original design, which incorporated both grant and loan programs as a long-term means to foster sustainable, equitable and efficient expansion of tertiary education, remains highly relevant. 3.2 Achievement of Project Development Objectives Rating: Substantial 50. Table 1 summarizes the achievement of results by PDO and relies on evidence from the Project’s PDO Indicators but also on the several supplementary PDO indicators introduced at the ICR stage that are linked to Project activities. This ICR section addresses only the equitable, sustainable and efficient expansion of tertiary education through grant assistance and not through student loan assistance since Project activities to support a student loan program were restructured out of the Project in March 2010.

16

Table 1: Summary of Project Outcomes Indicator Baseline

2004-05 b/ End

Targetb/

2010-11

Current2010-11

PDO 1: The equitable expansion of tertiary education through student assistance1. Number of graduates from tertiary education 359,635 478,000 475,584 2. The share of 18-24 year old students in tertiary education from households classified at the two lowest quintiles of the income distribution

10 20.0 20.6

3. Percentage of PRONABES beneficiaries that were in the Oportunidades program during upper secondary educationa/

8.7 20 17.9

4. Percentage of Indigenous PRONABES beneficiaries a/

6 10.0 12.6

PDO 2: The sustainable expansion of tertiary education through student assistance5. Continued 1:1 participation between federal and state governments in the financing of all grants a/

Yes

Yes

Yes

6. Rigorous program evaluation carried out at the federal level and outcomes systematically disseminated

a/

n/a Yes Partial

7. Number of tertiary education institutions that have institutionalized Academic Support Units for Indigenous students a/

n/a 8 12

PDO 3: The efficient expansion of tertiary education through student assistance8. Estimated percentage of PRONABES students that remain in a tertiary education institution until the end of their program of study a/ (calculated based on the PRONABES retention rates from 2009-2010 to 2010-2011 school year).

n/a n/a 56.9%

9. Retention rate of PRONABES beneficiaries a/ 65.3c/ 75 87.3 10. Administrative costs of PRONABES remain at less than 1 percent of budget for scholarships a/

n/a Yes Yes

11. National on-line application process is in place for all states a/

No Yes Yes

a/ Outcome Indicator added at ICR stage; see ICR Section 2.3. b/ Refers to informally revised baselines and targets, if any. c/Baseline for retention rate of PRONABES beneficiaries is from 2003-04. The Equitable Expansion of Tertiary Education 51. The increase in the number of graduates from tertiary education in Mexico was dramatic – from 359,635 in the 2004-05 school year to 475,584 in 2010-11 – almost achieving the informally revised target of 478,000. The share of 18-24 year-old students enrolled in tertiary education from households classified in the two lowest quintiles of the

17

income distribution also increased substantially, from the informally revised baseline of 10 percent 15 in 2004-05 to nearly 21 percent in 2010-11, slightly exceeding the informally revised end-of-Project target of 20 percent. The total number of PRONABES beneficiaries increased considerably from 137,852 in 2004-05 to 320,828 in 2011-12. 52. The Project contributed to the equitable expansion of tertiary education by financing approximately 390,000 PRONABES scholarships16 and supporting improved targeting of all PRONABES students through improvements in targeting criteria and PRONABES’ application questionnaire. Starting in 2007, targeting criteria were revised to give priority to Oportunidades recipients and Indigenous students. Since 2009, all PRONABES’ applicants have been required to fill in the improved questionnaire on-line, which requests information on an applicant’s family and socioeconomic background to determine and track eligibility, and understand better the applicants’ pool. Also PRONABES has improved dissemination of calls for applications to reach those students more in need. These changes were agreed during the PRONABES Annual Meetings (supported through the Project) which brought together federal and state officials managing PRONABES with technical specialists and representatives from other programs (e.g. Oportundidades, CONEVAL). The improved targeting criteria were incorporated into PRONABES’ Operational Rules (Reglas de Operación) that govern its entire program. 53. In addition, the establishment of the statistical MIS facilitated PRONABES ability to monitor how scholarships are allocated at the state level which also contributed to the ability to improve targeting based on evidence. With the MIS system, CNPRONABES has been able to collect data from all applicants at the national level and compare it with the final selection made by the institutions’ and states’ technical boards and recommend improvements.17 54. Data collected through the application show that the Project has helped PRONABES to better reach students from poor households (with income below 4 minimum salaries). Before the Project started (2004-2005 school year) only 96 percent of scholarships were awarded to these poor students whereas by the Project’s closing date, 100 percent were awarded to these students18. This is an important improvement in targeting, in compliance with PRONABES rules.

15 The original baseline for 2004-05 was 5.5 percent. 16 This number is calculated based on an annual average scholarship of MX$10,333 equivalent to US$727 (June 4, 2012 currency exchange rate). 17 All students applying to a PRONABES scholarship need first to apply on-line (national application) and then compile and submit to the university proof of their on-line application and the necessary documentation. The universities validate the information, identify eligible applicants and send the list of eligible students to the State/Institutional Technical Boards (Comités Técnicos Estatales o Institucionales) that are responsible for the final selection of PRONABES beneficiaries. 18 The intermediate outcome indicators specified disaggregated targets for the distribution of beneficiaries by the following household income categories: 0-1, 1-2, 2-3, 3-4 minimum salaries.

18

55. The improved targeting criteria and application enabled PRONABES, starting in 2007, to give priority to scholarship applicants with an Indigenous background and applicants from the Oportunidades program. With the help of the improved application, PRONABES was able to exceed its targets for these target groups. It increased its coverage of students from an Indigenous background from 6 percent in 2004-05 to 12.6 percent in 2010-11, above the target value of 10 percent. At the same time, the percentage of beneficiaries from Oportunidades households (generally from rural and peri-urban backgrounds) increased from 8.7 percent in 2004-05 to 17.9 percent in 2010-11 above the original target, but slightly below the target informally revised in May 2011.

The Sustainable Expansion of Tertiary Education 56. The Project supported financial and political sustainability of PRONABES and sustainability of the Academic Support Units supported by PAEIIES. This occurred in the broader context of a commitment to the sustainable expansion of tertiary education as reflected in the doubling of the total budget for tertiary education from MX$43.2 billion to MX$80.9 billion between 2006 and 2011. 57. Financial sustainability of PRONABES. Since the creation of PRONABES in 2001, the federal government has made federal funding contingent on state governments matching the federal contribution. Resources from the federal government are sent to each state to be administered through state PRONABES committees. 19 Even with PRONABES’ substantial growth, the states have maintained a one-to-one contribution. During the Project period, from 2006 to 2011, the federal government contributed MX$7.8 billion, or 51.3 percent, and the states contributed MX$7.4 billion, or 48.7 percent. During this period, the federal contribution to PRONABES increased by 12.9 percent annually; the contributions of the states increased by 14 percent annually.

58. The Project contributed to improved communication and compliance between the federal government and the states through its support to PRONABES’ Annual Meetings. It also contributed through the design and implementation of the PRONABES financial system which enables transparency and the use of timely financial data for CNPRONABES to follow up with the states in cases where disbursements of state scholarship resources are lagging. 59. Political sustainability of PRONABES. Political support for PRONABES has increased substantially under the Project. During Project implementation, there was a substantial increase in federal government support for PRONABES from around MX$248 million in 2001 to MX$849.8 million in 2006 (the first year of the Project) to MX$1.6 billion in 2011. For the federal government, this represents a major shift towards demand-side financing, which increased from less than 0.1 percent of public spending on tertiary education in 2001 to 3.6 percent in 2006 and 4.6 percent in 2010. PRONABES was one of the few federal programs that saw funding increase during the financial crisis.

19 Although state financing plays an important role in financing PRONABES, it was never formally included in the Project’s financing plan for legal reasons.

19

In addition, through the years, PRONABES has seen more higher education institutions joining the program. For instance, in the school year 2008-09 the Directorate General of Teacher Education and Training (Dirección General de Educación Normal y Actualización del Magisterio, DGENAM) and the Directorate General of Technological Higher Education (Dirección General de Educación Superior Tecnológica, DGEST) joined the program. These steady increases in budget and in the number of higher education institutions joining PRONABES bear witness to the perception of the important role that PRONABES is playing in expanding opportunities in tertiary education. Importantly in the PRONABES Sustainability Plan (Plan del Ejector para el Mantenimiento futuro del Proyecto 2011), the Government has also committed to continue promoting increased financing to sustain and expand PRONABES. As mentioned earlier, the current Education Strategy explicitly mentions the governments’ intention to increase the coverage of PRONABES in all tertiary education subsystems. 60. The Project has contributed to increased political support of the Program through several activities including through its sponsorship of the Annual Meetings, and through the improved MIS which enabled dissemination of PRONABES strong results through the CNPRONABES website which includes, for example, CONEVAL’s Consistency and Results Evaluation of PRONABES (2011-2012). In addition, several states such as Veracruz, Guanajuato, Jalisco and Nuevo Leon have good websites with PRONABES results. The results of the PRONABES impact evaluation that is underway and the study of the effects of the early intervention to encourage talented upper secondary education students to enroll in tertiary education will be made public to help maintain public support for PRONABES. 61. The Project design aimed to mainstream the Academic Support Units supported by PAEIIES in participating tertiary education institutions with financing through their budgets after Project financing ended. At present, 17 of the 24 universities that participated in PAEIIES have indicated that they will maintain the program with their own resources. ANUIES has confirmed that 12 of these tertiary education institutions have agreed to provide at least 70 percent of the financing received under the Project. ANUIES is also working with the other universities to get a similar level of commitment. The Efficient Expansion of Tertiary Education 62. Through its support to PRONABES scholarships and to PAEIIES, the Project contributed to an increase in the number of higher education students that enter and remain in university until the end of their program of studies. According to CNPRONABES the program contributed, in 2009-10 and 2010-11, 15.8 and 16 percent of the total number of students transitioning from upper secondary to public tertiary education and 24 percent of the total number of graduates from public tertiary education in the same years. The Project also financed improvements that helped PRONABES maintain administrative costs at less than 1 percent of the budget for its scholarships.

63. The annual retention rate of PRONABES beneficiaries in the 2010-2011 school year was of 87.3 percent, exceeding the 75 percent target, and a considerable increase

20

from the amount estimated for the 2003-04 school year (65.3 percent). This suggests that PRONABES is relatively efficient in keeping the poorest students in higher education. While terminal efficiency as defined in the PAD cannot be measured yet, it is possible to estimate the proportion of PRONABES beneficiaries that finish tertiary education with the new PRONABES data. In 2009-10, CNPRONABES started to collect data at a student level, allowing the calculation of the retention rate from one year to the next. These data show that an estimated 56.9 percent of students that started their first years of studies with a PRONABES scholarship are likely to complete their studies20.

64. In addition to the scholarships, the Project contributed to an efficient expansion of tertiary education through its support to PAEIIES. The number of tertiary education institutions participating in PAEIIES increased from 16 to 24 and the number of Indigenous tertiary students benefiting from the PAEIIES program increased to almost 12,000 in 2010-11. 21 Seven out of the eight additional universities participating in PAEIIES were in states with a high Indigenous population. Interviews during supervision missions and in the final report from ANUIES revealed PAEIIES played an important role in helping students adapt and stay in university. The change in retention rate of students enrolled in PAEIIES increased from an estimated 10 percent in 2005-06 to 20 percent in 2010-11, a substantial increase although short of the 25 percent target. The GPA of indigenous students in PAEIIES went down by -0.1, a modest decline that fell short of the adjusted end-of-project target of a 0.3 point improvement.22 65. Overall, the administrative costs (including both state and federal expenditures) for PRONABES are less than one percent of the total budget for scholarships23. This compares favorably to other scholarship programs. For example, the Pell Grant program in the United States has operational costs equivalent to 2.9 percent in 2011. Improved efficiency was also seen in the switch to an on-line national application process for all states and public institutions of higher education. 66. The mechanisms that the Project financed that contributed to PRONABES keeping its administrative costs low in spite of a rapidly growing scholarship program are: (i) a new financial system that provides accurate and up to date information to CNPRONABES allowing it to improve fiduciary management and to follow up with the states rapidly when there are delays, (ii) training of staff in the financial system, (iii) the modernization of the CNPRONABES website which permits students to fill in a national application form on-line, and thus lowers PRONABES and student transaction costs, and (iv) a strengthened statistical MIS with data gathered through the new application, which

20 The actual figure is likely to be higher because this estimate is based on a 5 year program. In fact, PRONABES beneficiaries study diverse programs in terms of content and length. There are higher education programs that can last three years or less (the percentage of PRONABES students in this programs is around 17 percent). The equivalent figure for students in three year programs is 66.4 percent. 21 The PAD indicated that there were 10 participating institutions. By 2003, 11 institutions had joined the program. Another 5 joined in 2005, as the Project was being prepared. 22 However, it is difficult to attach much significance to these minor changes in the GPA in either direction form an already fairly high GPA over 8. 23 Reglas de Operación de PRONABES 2012.

21

has improved communications and collaboration with state offices and with Oportunidades, and helps monitor applicants to eliminate duplicate applications and quickly identify those from targeted groups. 67. In addition, a number of measures to improve PRONABES efficiency were agreed at the PRONABES Annual Meetings. For example, in 2009, an agreement (through a Memorandum of Understanding) with the Oportunidades program to disseminate information on PRONABES and coordinate activities was reached. And, in 2011, a reduction of the paperwork requirement for applicants with income from the informal sector was agreed, reducing the transaction costs for rural students. 3.3 Efficiency Rating: Modest 68. The Project was a good investment. Based on the assumption in the economic analysis, the rate ranged from 3.6 percent to 4.5 percent (Annex 3). This does not include the value of increasing equity and improving the Project’s administration, which is part of the Project’s economic contribution that was not accounted for in the economic analysis.

3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 69. The Project’s design and development objectives remain highly relevant to Mexico’s current development challenges and the priorities of the Bank’s partnership with Mexico. The Project’s objectives with respect to equity were largely achieved through the key role that the Project played in supporting PRONABES scholarships and improving their targeting. In addition, the Project contributed to important advances in PRONABES, including improved communication with public tertiary education institutions and the states, and more efficient student application through an on-line national system available in all states. It also contributed to improved financial management. The Project contributed to sustainability of the expansion of tertiary education through its support to PRONABES’ improved systems which help monitor continued one-to-one federal state contributions to the scholarship program and provide timely and accurate data which, among others, has provided inputs to the last CONEVAL 2011-12 evaluation of the program now available to the public. Since only the means to achieve Project objectives were revised in the March 2010 restructuring, and not the Project objectives themselves, the ICR does not assess separately outcomes before and after the restructuring. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 70. The expansion of the number of PRONABES beneficiaries coming from households earning less than four times the minimum salary and the increase in the share of Indigenous people, Oportunidades recipients and females have been positive in terms of supporting poverty reduction, gender equality, and social development.

22

(b) Institutional Change/Strengthening 71. As explained, the Project supported important institutional changes occurred at PRONABES to strengthen its capacity. PRONABES expanded its professional staff, improved its management information systems, clarified the division of labor between its national and state programs, improved its financial management, and sharpened the pro-poor targeting of its scholarships. (c) Other Unintended Outcomes and Impacts (positive or negative) N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 72. Most supervision missions included interviews and surveys of recipients. The students contributed a number of suggestions that were then relayed back to PRONABES, including ideas for reducing the transaction costs for the application and renewal of scholarships and ideas for outreach to upper secondary students. In general, the students reported that while they greatly appreciated the support, the support paid only for part of their studies. This problem was especially severe for students from larger families. This finding led to a debate on the size of the scholarship and how to treat students from larger families. A survey to state operators conducted in 2011 and Annual Meetings showed that in some states there were problems with the Banking system. This led to sharing of experience among the states and changes in the banking practices in many states.

4. Assessment of Risk to Development Outcomes Rating: Moderate 73. The overall risk to the Project’s development outcomes is moderate. There is widespread support at both the federal and state level of the PRONABES program. Federal funding is contingent on a one-to-one state contribution, and states have consistently increased their support to match the increasing federal contribution. In addition, public dissemination of CONEVAL’s positive evaluations of PRONABES are expected to help maintain support. CNPRONABES has its own budget for operational costs which has enabled it to hire specialized staff, including in scholarship programs, socioeconomic analysis, management information systems, financial management, and MIS, that have substantially increased it capacity, including to coordinate the program with the states. In addition, targeting and financial management improvements have been institutionalized throughout PRONABES through improved information embedded in the MIS and financial system, and have been incorporated in PRONABES’ operational rules. 74. Targeting criteria continue to be refined and improved. For example, starting in 2012, targeting criteria and their application were further modified to give priority also to students with disabilities. In 2012, PRONABES also made further adjustments to

23

eligibility requirements to better target poor students from large households. It also changed the way family income is calculated; it is now divided by the number of household members using a per capita approach, whereas before the household income was taken independently of the number of family members. 75. In addition, PRONABES continues to improve its efficiency. In 2012, for example, a reduction in the transaction costs for returning beneficiaries was put in place and general administrative and financial formats must now be used by all states.

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 76. While the design of Component 1.1, support for PRONABES, was sound, with hindsight there were significant shortcomings in overall Project preparation and design. There was a discrepancy within the PAD and between the PAD and the Loan Agreement in the statement of the PDO. The M&E framework had significant weaknesses as previously discussed. Due diligence with respect to implementing arrangements was inadequate, resulting in a mismatch between the responsibilities of PRONABES and its authorities. Adequate due diligence was also lacking with respect to the interest of states in Project support to strengthen their student lending programs. Other risks to Project implementation, notably that government priorities might shift following the change in government and that an economic downturn might render development of student loan programs infeasible, were not adequately assessed and mitigated. The lack of agreed drafts of key documents at the time of negotiations created a fertile terrain for significant implementation delays, as did the lack of a clear understanding and commitment regarding the timing for provision of budget for all Project components. Finally, Project design was too ambitious and complex, including policy and institutional issues that would require extensive analytical work and evaluation along with elements that were ready for implementation. This was particularly worrisome given the small number of PRONABES staff during preparation and effectiveness. (b) Quality of Supervision Rating: Moderately Satisfactory 77. Supervision was conducted in at least two missions per year. The Bank consistently drew to the attention of the implementing agencies the need to complete the requirements for effectiveness, provide budget for all components, resolve the legal and administrative complexities that were holding up the transfer of resources from SEP to ANUIES, reach agreement on a loan to a state under Component 4, accelerate progress on evaluations and other analytical work, adjust outcome baselines in light of new information, and provide timely and complete monitoring data on progress towards Project outcomes. Supervision of financial management issues intensified in 2007-09

24

when, in addition to the full missions, several specialized missions were conducted which agreed and then followed up on detailed action plans to resolve disbursement, audit, and other problems. The Bank provided training to staff in both fiduciary and technical areas. Most missions included field visits and meetings with students, state authorities, and tertiary education institutions. 78. With hindsight, however, supervision might have been improved in certain areas. An opportunity was missed at the time of Project restructuring to strengthen implementation arrangements and revise formally Project outcome indicators. Although this was discussed with the Government, it was felt that that the M&E system, while improved, was likely to be subject to further change. It is also acknowledged that procurement and disbursement processes could have benefited from greater efficiency across the board, which would have limited some of the delays discussed earlier. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 79. While there were significant shortcomings during preparation leading to quality at entry problems, the Bank made a concerted effort to address these shortcomings through supervision and technical support. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 80. The government’s performance was satisfactory. Provision of budget to PRONABES for the scholarship program under Component 1.1 was timely and increased significantly during Project implementation. Loan disbursements were initially slow. As of end-2007, they had reached only US$14.5 million, or 8.5 percent of the IBRD loan amount, notwithstanding the scope for retroactive finance. Even though PRONABES continued to execute its budget in a timely fashion, disbursements only reached 48.2 percent of the loan amount, or US$82.4 million by the end of 2009. This was a minor shortcoming as the loan was fully disbursed by its revised closing date. (b) Implementing Agency’s or Agencies’ Performance Rating: Moderately Satisfactory 81. PRONABES executed its steadily increasing budget in a timely fashion. It allocated scholarships in accordance with the targeted distribution among household income levels. It also achieved its targets with respect to shares of Indigenous students and beneficiaries of the Oportunidades program. In addition, it made improvements that reduced transactions costs, clarified the division of labor between the federal and state levels, and strengthened internal controls. Likewise, PRONABES could have been more proactive in seeking budget authority for Project components early in implementation, and in resolving the legal and administrative complexities of granting loan proceeds to ANUIES. Also, more attention to designing and executing the studies envisioned under

25

Components 2.2 and 2.3 would have improved SEP’s performance, as would greater attention to the piloting of early interventions under Component 3.1. 82. BANOBRAS performed well in its role as fiduciary agent for the loan to the government. With reference to the separate US$9 million loan that the Bank offered it for the purpose of supporting state student loan programs, BANOBRAS might have been able to improve its performance by enhancing communications with the State of Quintana Roo and/or other states in order to address their concerns. This is a moderate shortcoming; reaching agreement with any state was the responsibility of BANOBRAS but other factors, including the economic downturn and changes in government at the state level, also contributed to the outcome. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory

83. There were both strengths in Borrower performance and moderate shortcomings. Delays and missed opportunities should not obscure the substantial achievements of the Project.

6. Lessons Learned

The Project was excessively complex both in terms of its activity and its management structure, in particular covering activities that were out of the control and influence of the Project manager. Institutional arrangements for projects should take into account capacity as well as span of control of the implementing agency(s).

For a complex project, it is important to address legal issues during project preparation. Implementation would have been smoother and more timely if agreed versions of the grant agreement between SEP to ANUIES and the loan agreement between BANOBRAS and ICEEQROO had been ready at the time of Project negotiation, as well as clear evidence that Project-supported activities would be included in the budget. In the specific case where legal agreement between federal and state governments are required for some loan components, consideration could be given to preparing separate simpler loans.

In a sophisticated middle-income country like Mexico, it is not possible to ring-fence a Bank-financed project from the broader, more complex government program. In the case of this Project, as the size of the PRONABES program expanded, Project restructuring could have included a revision in the Project financing plan to include all government financing to PRONABES. Even without this inclusion, however, Project impact went beyond Project financing since technical assistance influenced all of PRONABES operations and not just those financed by the Project.

Where Project finance is not additional, but rather a source of financing for the overall fiscal budget, a clear understanding about how budget for project-supported activities is provided is essential. In the case of this Project, activities in

26

Components 2 and 3 were not part of the budget until 2009, delaying implementation.

It is important to ensure that objectives and targets are realistic and carefully set at the onset. In the absence of an adequately designed M&E framework, early adjustment of baselines and targets is desirable.

It is also important to ensure that the M&E framework is aligned with the implementing agency’s capacity. Where capacity is insufficient, a concerted effort should be undertaken in the first year of project implementation to strengthen management information systems, refine the baselines and monitoring indicators, and determine the methodology for assessing them. For this Project, there were widely acknowledged issues regarding the precision and reliability of data that were not addressed until late during implementation of the Project.

More care is needed to mitigate against the impact on projects due to a change in administration (as happened with respect to this Project in 2006) that is often accompanied by changes that reach far down into the executive. Rather than rush to ensure that projects are effective before a new administration takes office without attention to risk mitigating measures, project design should anticipate and mitigate against capacity disruptions that are often inevitable with a change in government.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 84. The Borrower through BANOBRAS submitted written comments on the ICR. These have been included in Annex 7. The comments emphasized the efforts of the government to ensure that the Project met its development objectives. The borrower also expressed concern with the slow pace of issuing some procurement No Objections and the delay in the final loan disbursement. Changes were made to the following sections to incorporate this comment: (i) Fiduciary Compliance and (ii) Bank Performance (quality of supervision). In response to further communication with BANOBRAS several smaller changes were made to the ICR including: (i) referencing the Aide Memoire where revisions to the indicators were agreed between de government and the Bank and, (ii) adding to the lessons learned the possibility of separate simpler loans. 85. SEP (PRONABES) also provided several very useful clarifications which were incorporated into this ICR. These include (i) minor revisions to some numerical facts/data taking into account most recent officially published information; (ii) suggestion to add to lessons learned on the importance of setting realistic objectives and targets in the M&E framework, and adjusting in the course of implementation as needed.

27

(b) Cofinanciers N/A (c) Other partners and stakeholders N/A

28

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1. Support for PRONABES national scholarship program and for the development of the national regulatory framework for tertiary education

281.23 284.09 1.01

2. Analysis and Institutional Strengthening 1.73 0.76 43.9

3. Support for Disadvantaged Students 5.92 4.03 68.1

4. Support for State-level Student Loan Programs 10.58 0.00 0.00

5. Promotion of private investments in students loans 0.12 0.00 0.00

Total Baseline Cost 299.58 289.88 96.8

Physical Contingencies 0.00

0.00

0.00

Price Contingencies 0.00

0.00

0.00

Total Project Costs 299.58 289.88 96.8 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.42 0.42 100.00

Total Financing Required 300.00 290.30 96.8

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Borrower 119.57 119.30 99.80 International Bank for Reconstruction and Development 180.00 171.00 95.00

29

Annex 2. Outputs by Component Component/Subcomponent Project Outputs Component 1 Subcomponent 1.1 Support for the PRONABES National Scholarship Program

Financial support provided to deliver 1,276,053 PRONABES scholarships from 2006-2011. PAEES financed approximately 390,000 PRONABES scholarships from 2006-2011. Intensive technical assistance through 2009, 2010 and 2011 provided by procurement and financial management specialists to manage adequately the Project and the expanding budget of PRONABES. Development of Administrative Manuals required to operate PRONABES in 2009.

Subcomponent 1.2 Development of a national regulatory framework for tertiary education student assistance.

Design, development and implementation of the PRONABES improved Statistical on-line System (Sistema de Información Estadística) by school year 2009-2010. Design of the Financial System (Sistema de Información Financiera para el seguimiento del Programa Nacional de Becas y del Programa de Asistencia a Estudiantes de Educación Superior) in 2010 and piloting of the same system in 2011. Financial and technical support to conduct PRONABES Annual Meetings that brought together federal and state officials managing PRONABES with technical specialists and representatives from other programs (e.g. Oportundiades, CONEVAL) to take informed collective decisions affecting, among others the selection criteria of PRONABES students.

Component 2 Subcomponent 2.1 Promotion and Information Dissemination

With additional support from the IADB the Federal Secretariat of Labor and Social Protection (STPS) strengthened the Labor Market Observatory. In addition, SEP also created its own website, “Decide Your Career”

Subcomponent 2.2 Analysis and Institutional Framework – Regulatory Framework

International Conference on the feasibility of developing a student loan system for tertiary education (1er. Foro Internacional sobre Modelos de Crédito Educativo. Experiencias en su Diseño, Operación e Impacto) organized in 2011 in Tijuana, Mexico to provide an analytical base to strength tertiary education policy through discussing

30

Component/Subcomponent Project Outputs financing alternatives of tertiary education and the feasibility for a reimbursable grant program.

Subcomponent 2.3 Analysis and Institutional Framework – Evaluation of Student Assistance Programs

Design of the PRONABES impact evaluation on equity of, access to, and completion of tertiary education

Component 3

Component 3.1 Early Interventions to Assist Talented Disadvantaged Students

A baseline study (with an experimental design) was carried out to explore how student guidance and measures to promote higher expectations among students affect access to tertiary education for poor students.

Component 3.2 Expansion of the Indigenous Students Assistance Program

Financing to support universities to establish Academic Support Units to provide academic and non-academic support to Indigenous students, including tutoring, counseling, support to apply for financial aid programs (including PRONABES), and orientation to new entrants.

Component 4 Support for state-level student loan programs

None

Component 5 Promotion on Private investments in student loan

None

31

Annex 3. Economic and Financial Analysis This Annex reviews the quality and relevance economic analysis carried out during project preparation. It also provides an ex-post review of the project’s economic impact by estimating the cost and benefit of the project with final data to evaluate the appropriateness of the ex-ante estimates. Project Preparation During project preparation, the World Bank carried out an extensive review of the economic literature on the value of tertiary education. This review included the equity impact of the higher education to help understand how different groups benefit from higher education and study of how returns to education have changed with time. The economic analysis also considered the value of higher education to Indigenous students. It is clear that the economic analysis (and the thinking behind it) played an important role in the dialogue with the Government to design the Project and to identify alternatives. In this sense, the economic analysis is better practice and made a real difference in the preparation of the Project and the Government’s policy towards higher education financing. In addition, it is clear that the economic analysis contributed to the development of the Indigenous People’s Plan. It played an important role in the decision to develop a non-financial program to develop compensatory programs to help Indigenous students “catch-up” with the rest of the population. Cost-Benefit Analysis The economic analysis also included a cost-analysis to justify the intervention in the sector and the expected rates of return on investment in PRONABES. Both of these analyses draw heavily on the previously reviewed economic literature and use this as a source for the underlying assumption.

A. Sectorial Rate of Return Prior to project approval, the estimated individual financial rate of return to tertiary education in Mexico was estimated to around 16 percent; the individual economic rate of return (i.e. netting out the public subsidy for education) is around 8 percent.24 While there have not been any recent published studies on the individual returns to higher education in Mexico, a recent review in Europe showed economic rates of return for higher education around 8 percent.25 This is supported by studies that show the growing contribution of education, particularly higher levels of education to economic growth.26

24 This is identified in the PAD as the “social rate of return” but it clearly does not include any social benefits to higher education or capture any positive externalities. 25 Pscharopoulos, 2009. “Returns to Education in Higher Education: A European Survey.” Unpublished mimeo. 26 For example, see Hanushek and Wossmann 2007. “Education Quality and Economic Growth. Washington: World Bank.

32

Given these direct and social benefits, investing in higher education remains a good investment and the assumptions made during the preparation of the project remain valid.

B. Project Rate of Return The Project’s cost-benefit analysis focused exclusively on the impact of PRONABES. Given the large role of PRONABES in the Project (estimated at around 90 percent at the time of project preparation), this decision made sense. Likewise, given the lack of data on the performance of PRONABES (it was only four years old at the time of Project preparation) and the lack of previous evaluation, the cost-benefit analysis was prospective, based on the assumptions and data available at the time. This approach made sense, given the availability of information. The analysis estimated a range of economic rates of returns to the Project, ranging from 2.9 percent to 10.3 percent depending on the Project’s impact on enrollment and graduation rates. The primary underlying assumptions include:

PRONABES will primarily impact on the number of students staying in higher education and improve graduation rates. This appears to be a reasonable assumption and is confirmed by numerous interviews with students during project supervision. PRONABES has initiated an impact evaluation that will provide further evidence on this point.

The entire impact of PRONABES is explained by the increase in income caused by the higher graduation rate. While it is important to take into the personal benefits of the scholarship program in the project benefits calculations, there are at least two other impacts that could have been quantified. First, the impact of the Project on reducing poverty and improving equity. Second, the positive externality associated with an increased of tertiary graduates. The estimates here include a calculation of the number of poor people lifted from poverty.

The individual rate of return remains constant over the length of the Project. Although this assumption was not explicitly state in the PAD, it was used for estimation of the Project benefits. Although the rates of returns would have dropped during the economic crisis, the analysis should be based on the long-term rate of return of education. Based on this, the cost-benefit calculation here is calculated for an individual economic rate of returns ranging from 7 percent to 10 percent. The higher estimates in the range are probably more realistic since there is a growing trend for higher levels of education. The range also provides an easy way to consider the social externalities associated with having more tertiary graduates.

33

The Project had a positive rate of return, ranging from 3.6 percent to 4.5 percent. This estimate is based on a terminal efficiency rate of 63.3%, which is based on the observed graduation rate for PRONABES students. While this tends to be on the lower end of the estimates presented in the PAD it is positive and justifies the Project from an efficient use of capital perspective. The Project plays a major role in breaking the intergenerational cycle of poverty, allowing an estimated 115,000 to 175,000 people to leave poverty27. This based on the assumption that a certain percentage of otherwise poor beneficiaries permanently poverty for the rest of their lives. While the individual benefits of this increase in income is accounted for above, this has a substantial impact on the government budget. Mexico provides poor households with a substantial range of services, including conditional cash transfers (Oportunidades), health insurance (Seguro Popular), among others. Using conservative estimates of the future cost of these programs and assumptions about future family sizes, this would generate an additional rate of return between 4.7 percent and 11.9 percent. Based on this analysis, the ICR concludes that the Project was a good investment. The overall returns to the investment, appears to be in the range of 8.3 percent to 16.4 percent, taking into account the direct benefits to scholarship beneficiaries and the savings to the government from future social programs.

27 These calculations are based on the total benefits to the PRONABES during the Project period. Scaling back the benefits to cover the beneficiaries financed by IBRD has no effect on the estimated rate of return.

34

Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members

Names Title Unit

Lending Andreas Blom Sr Education Econ. SASED Lea D. Braslavsky Consultant LCSPT Maria E. Colchao Senior Program Assistant LCSHE Victor Manuel Ordonez Conde Finance Officer CTRLN Vicente B. Paqueo Consultant EASHD Harry Anthony Patrinos Lead Education Economist HDNED Mariangeles Sabella Senior Counsel LEGLA Supervision/ICR Eric P. Bettinger Consultant LCSHE Andreas Blom Sr Education Econ. SASED Erik A. Bloom Senior Economist LCSHE Maria E. Castro-Munoz Consultant LCSHH Maria E. Colchao Senior Program Assistant LCSHE Alina Garduno Lozano Temporary LCSHD Dmitri Gourfinkel Financial Management Analyst LCSFM Mark V. Hagerstrom Country Program Coordinator EACIQ Martha Belem Hernandez Zavala

Consultant LCSHE

Efraim Jimenez Consultant LCSUW Jose M. Martinez Senior Procurement Specialist ECSO2 Jose Carlos Martinez Gonzalez

LCSFR

Victor Manuel Ordonez Conde Finance Officer CTRLN Gabriel Penaloza Procurement Analyst LCSPT Felix Prieto Arbelaez Senior Procurement Specialist LCSPT Domenec Ruiz Devesa Junior Professional Associate LCSHE Anna Maria Sant'Anna Consultant LCSHE Tomas Socias Senior Procurement Specialist LCSPT Aracelly G. Woodall Senior Program Assistant LCSTR Antonella Novali Program Assistant LCSHE

35

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending FY04 17.75 94.86 FY05 52.45 276.44 FY06 18.02 88.48

Subtotal: 88.22 459.78 Supervision/ICR

FY06 4.06 27.88 FY07 21.57 95.76 FY08 31.61 120.47 FY09 25.66 120.88 FY10 28.26 133.77 FY11 24.34 103.58 FY12 14.63 99.09

Subtotal: 150.13 701.43 Total: 238.35 1,161.21

36

Annex 5. Beneficiary Survey Results (If any) N/A

Annex 6. Stakeholder Workshop Report and Results (if any) N/A

37

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

BANOBRAS OPINION ON THE IMPLEMENTATION OF THE TERTIARY EDUCATION STUDENT ASSISTANCE PROJECT (PAEES), PARTIALLY FUNDED WITH LOAN RESOURCES 7346-ME OF THE WORLD BANK

(Informal translation) BACKGROUND: The World Bank lent US$171,000,000 to the Mexican government (SHCP) with a loan agreement signed between the two bodies on April 7, 2006, to finance part of the Secretariat of Education’s (SEP) Tertiary Education Student Assistance Project, PAEES, whose closing date was scheduled for March 1, 2010. The project’s objective was to support the federal government's strategy to promote the sustainable, equitable and efficient expansion of higher education through student aid through the development of a coherent system of educational assistance, consisting of student loan programs and scholarship, compensatory interventions for disadvantaged students and support to the national higher education policy. At the request of SEP, and with the approval of the Secretariat of Finance and Public Credit (Secretaría de Hacienda y Crédito Público, SHCP), BANOBRAS as the Federal Government’s financial agent negotiated with the World Bank an amendment to the loan agreement to: (1) eliminate the student loan program, (2) modify the project objectives, (3) relocate resources available by category of expenditure, and (4) extend the closing date of the project until December 31, 2011, to allow time to the executing agency to finish Project activities to achieve the Project objectives. To carry out the implementation of the loan, SHCP, BANOBRAS, and SEP signed a Project Execution Contract (Contrato de Mandato y de Ejecuación del Proyecto) on September 1, 2006, which remained in force until the obligations under the loan agreement were finalized. PROCUREMENT AND CONTRACTING PROCESSES: During the execution of the loan, BANOBRAS trained and supported staff from the implementing agency (SEP) in the preparation of Procurement Plan (PA) and its inclusion in the World Bank’s Procurement Plan Execution System (SEPA). Also, BANOBRAS managed and obtained No Objections from the World Bank in the tendering and contracting process for consulting firms that carried out technical studies, as well as for specialists who were members of the Project Coordination Unit (PCU). During the recruitment process and the specialists’ and consulting firms’ performance period, the executing agency had difficulty making payments for contracted services due

38

to SEP restrictions in the use of the annual budget; this circumstance led to delays in the delivery of information to BANOBRAS. This situation was resolved, as SHCP authorized the necessary resources. DISBURSEMENT: BANOBRAS processed the disbursement requests received from the SEP and submitted them to the World Bank. This led to a full disbursement (100 percent) of the loan. As mentioned earlier, the project implementation period ended on December 31st, 2011 and the World Bank granted the Government an administrative extension (grace period) of four months (until April 30, 2012) to submit the final request for payment, based on resources included in the financial statements for the year 2011. All disbursed funds were deposited in a timely fashion in the Federal Treasury as directed by the SHCP. PROJECT PROGRESS REPORTS: BANOBRAS reviewed the biannual Financial Monitoring Reports (FMRs) received from SEP as part of its monitoring role in implementing the project. It made arrangements with the implementing agency to make changes and sent the final FMR to World Bank to comply with contractual commitments in a timely fashion. Similarly, BANOBRAS prepared and delivered monthly and biannual reports in a timely fashion to SHCP on the implementation of the project to comply with the commitments made in the Project Execution Contract. To comply with the commitments established in the loan agreement, BANOBRAS requested that SEP develop a Plan to Ensure the Future Maintenance of the Project. BANOBRAS delivered this Plan to the World Bank six months before the closing date of the Project, as required. This was approved by the World Bank, in due course. AUDIT OF THE FINANCIAL RESOURCES: BANOBRAS supported SEP in the delivery of information to the Internal Control Body of SEP (OIC-SEP) to conduct annual audits of the Project. During project implementation, BANOBRAS received from OIC-SEP annual audit reports that were used to obtain agreement from the World Bank within the established time as agreed in the loan agreement. (It should be noted that the review of the Audit Report of the Project undertaken by the OIC-SEP for the year 2011 is still pending. This should be sent to BANOBRAS in mid-June 2012, for review; and if appropriate, delivered to the World Bank no later than June 30, 2012).

39

CONCLUSIONS: The implementation of the project by the SEP, with support provided by the financial agent, was satisfactory as this operation was closed to the satisfaction of the World Bank. The execution of this loan is considered satisfactory, because in general terms an important number of beneficiaries were granted a PRONABES scholarship. PRONABES beneficiaries increased from 161,787 in the 2005-2006 school year at project start, to 320, 828 during the 2011-2012 school year to end of the Project. It is important to note that the Project provided financial support and assistance to indigenous students around the country, through an agreement between SEP and the National Association of Universities and Institutions of Higher Education (ANUIES) to finance its Program for Assistance to Indigenous Students in Tertiary Education Institutions (PAEIIES), which supported a large number of students. For example, when the Program began in 2009 had 9,600 students enrolled and in 2011 it has reached to nearly 12,000 students. The World Bank's performance was not ideal, because, there were delays in the process of obtaining the No Objection for tendering and contracting specialists, consultants and service companies. Moreover, during the final disbursement of the loan, there was a delay since the World Bank did not make timely arrangements for recording contracts in their internal system, which delayed the World Bank’s Disbursement Unit in Brazil to make the payment requested by BANOBRAS. The Project had the timely intervention of the Internal Control Body of SEP, which followed the Project’s results, and intervened as required to correct the executing agency and to improve its performance. Also, it is important to note the timely participation of the Directorate General for External Audits of the Secretariat of Public Service (Secretaría de la Función Pública, SFP), to ensure that reports submitted to the World Bank complied with the provisions laid down in agreements between the Government and the World Bank (Memorandum of Understanding). The valuable support of the International Affairs Unit of the SHCP, should be mentioned as borrower and to resolve institutional difficulties that arose during the Project implementation. BANOBRAS’ performance as financial agent in this operation can be rated as satisfactory, since it supported the executing agency at all times in compliance with its commitments. BANOBRAS carried out its role as intermediary between the executing agency and the World Bank, and carried out a timely and on-going monitoring and review of the Project’s performance, which was reported to the International Affairs Unit at SHCP in a timely manner.

40

It was a major challenge for BANOBRAS to have been the Federal Government’s financial agent in this complex operation. BANOBRAS took on this challenge with institutional responsibility and used all its human and technical-professional capacity, to ensure at all times, that the executing agency fulfilled in an efficient and timely manner the external commitments established with the World Bank in the loan agreement, and with the Mexican government in the Project Execution Contract between SHCP and BANOBRAS. With the implementation of this operation, BANOBRAS gained extraordinary experience, to add to its over 25 years of experience in managing international operations which will be very valuable to continue administrating other operations as entrusted by Ministry of Finance LESSONS LEARNED: This project is an excellent example for the World Bank and the Ministry of Finance to learn lessons that no doubt will serve them when preparing other projects in education, and will provide the Federal Government’s with evidence to select the financial agent to administer the resources for a new loan.

41

Spanish version received from the Government:

OPINION DE BANOBRAS SOBRE LA EJECUCION DEL “PROYECTO DE ASISTENCIA A ESTUDIANTES DE EDUCACION SUPERIOR” (PAEES),

FINANCIADO PARCIALMENTE CON RECURSOS DEL PRESTAMO 7346-ME DEL BANCO MUNDIAL

ANEXO No. 7 DEL IMPLEMENTATION COMPLETION AND RESULTS REPORT (ICR). ANTECEDENTES: El Banco Mundial otorgó al gobierno mexicano (SHCP) un préstamo por USD171´000,000 cuyo contrato se firmó entre ambas instancias el 7 de abril de 2006, destinado a financiar parcialmente a la Secretaría de Educación Pública (SEP)el “Proyecto de Asistencia a Estudiantes de Educación Superior” (PAEES), cuya fecha de cierre se programó para el 1 de marzo de 2010. El objetivo del proyecto fue apoyar la estrategia del Gobierno Federal para fomentar la expansión sostenible, equitativa y eficiente de la educación superior, a través de ayuda a estudiantes por medio del desarrollo de un sistema coherente de asistencia educativa, consistente en programas de crédito educativo y de becas, intervenciones compensatorias para estudiantes en desventaja y apoyo a la política nacional de educación superior. A petición de la SEP, y con la aprobación de la SHCP, BANOBRAS como agente financiero del Gobierno Federal, gestionó ante el Banco Mundial una enmienda al contrato de préstamo, para: eliminar el programa de préstamos estudiantiles, modificar el objetivo del proyecto, reubicar recursos disponibles por categoría de desembolso, y extender la fecha de cierre del proyecto hasta el 31 de diciembre de 2011, para dar tiempo al ejecutor a concluir las acciones que le permitieran lograr los objetivos del proyecto. Para la ejecución del préstamo, la SHCP, BANOBRAS y la SEP firmaron un “Contrato de Mandato y de Ejecución del Proyecto” el 1 de septiembre de 2006, quedando vigente hasta que las obligaciones derivadas del mismo y de sus partes integrantes, se hayan extinguido. PROCESOS DE ADQUISICIONES Y CONTRATACIONES. Durante la ejecución del préstamo, BANOBRAS capacitó y apoyó al personal del ejecutor (SEP) en la preparación del Plan de Adquisiciones (PA) y su inclusión en el “Sistema de Ejecución del Plan de Adquisiciones” (SEPA) del Banco Mundial. Así mismo, BANOBRAS gestionó ante el Banco Mundial la obtención de las No Objeciones en los procesos de licitaciones y contrataciones de las empresas consultoras que realizaron los estudios, como los especialistas que integraron la Unidad Coordinadora del Proyecto (UCP).

42

Durante los procesos de contratación y ejecución de acciones de los especialistas y de las empresas consultoras; el ejecutor tuvo dificultades para realizar los pagos por los servicios contratados, debido a restricciones en el ejercicio presupuestal de la SEP; circunstancia que generó retrasos en la entrega de información a BANOBRAS; situación que se resolvió, en la medida en que el área de presupuesto de la SHCP liberó los recursos. DESEMBOLSO DE RECURSOS. BANOBRAS gestionó ante el Banco Mundial las solicitudes de desembolso que recibió de la SEP, habiendo logrado desembolsar el 100% de los recursos del préstamo. Como se mencionó inicialmente, el proyecto concluyó su periodo de ejecución el 31 de diciembre de 2011; motivo por el que, el Banco Mundial otorgó al ejecutor una prórroga administrativa de cuatro meses (hasta el 30 de abril de 2012), para que le presentara la última solicitud de desembolso, cuyo monto de recursos fue incluido en los estados financieros del ejercicio 2011. Los recursos desembolsados del préstamo fueron depositados en la Tesorería de la Federación, en su momento, conforme a las indicaciones de la SHCP. INFORMES DE AVANCE DEL PROYECTO. BANOBRAS revisó los Reportes de Monitoreo Financiero (FMR´s) semestrales que recibió de la SEP, como parte de los avances en la ejecución del proyecto, hizo las gestiones necesarias con el ejecutor para que hiciera los cambios procedentes, y los envió al Banco Mundial, para dar cumplimiento oportuno a los compromisos contractuales. De igual manera, BANOBRAS elaboró y entregó a la SHCP los informes mensuales y semestrales, sobre el comportamiento del proyecto, para dar cumplimiento oportuno a los compromisos asumidos en el contrato de mandato. Para dar cumplimiento a los compromisos establecidos en el contrato de préstamo, BANOBRAS solicitó a la SEP con el tiempo debido, la elaboración de un “Plan del ejecutor para asegurar el mantenimiento futuro del proyecto”, BANOBRAS lo entregó al Banco Mundial seis meses antes de la fecha de cierre del proyecto, conforme a lo requerido; mismo que fue aprobado por ese Banco y comunicado al ejecutor, oportunamente. AUDITORIA AL EJERCICIO DE LOS RECURSOS. BANOBRAS apoyó a la SEP en la entrega de información, para que el Órgano Interno de Control (OIC) en la SEP realizara la auditoría al proyecto, de manera anual. Durante la ejecución del proyecto, BANOBRAS recibió del OIC-SEP los informes de auditoría anual, mismos que gestionó y obtuvo la conformidad del Banco Mundial, en los tiempos establecidos en el contrato de préstamo.

43

(Cabe señalar, que está pendiente la revisión del Informe de Auditoría del proyecto, que está realizando el OIC-SEP por el ejercicio 2011, mismo que deberá remitir a BANOBRAS, a mediados del mes de junio del año en curso, para su revisión; y en caso procedente, entregarlo al Banco Mundial a más tardar el 30 de junio de 2012). CONCLUSIONES. Se considera que la ejecución del proyecto por parte de la SEP, con el apoyo proporcionado por el agente financiero, fue satisfactoria; habiendo cerrado esta operación a entera satisfacción del Banco Mundial. La ejecución de este préstamo se considera satisfactoria, debido a que en términos generales se logró el otorgamiento de becas a un número importante de beneficiarios, al pasar de 161,787 en el ciclo escolar 2005-2006 al inicio del proyecto, a 320, 828 durante el ciclo escolar 2011-2012 al final del mismo. De los resultados obtenidos cabe destacar, el apoyo económico y asistencial que se dio a los estudiantes indígenas en todo el país, a través del convenio que la SEP celebró con la “Asociación Nacional de Universidades e Instituciones de Educación Superior” (ANUIES), para financiar su “Programa de Apoyo a Estudiantes Indígenas en Instituciones de Educación Superior” (PAEIIES)”, con el que se logró apoyar a un número importante de estudiantes; por ejemplo, al inicio de este programa en 2009 se tenían 9,600 estudiantes inscritos en el programa, habiendo llegado a finales de 2011 a cerca de 12,000 estudiantes. El desempeño del Banco Mundial no fue de los mejores en esta operación; debido a que, se presentaron retrasos en las gestiones para obtener las No Objeciones a los procesos de licitación y contratación de especialistas, de empresas consultoras y de servicios; además, en el último desembolso del préstamo, se generó un retraso atribuido a que el Banco Mundial no realizó a tiempo gestiones para registrar los contratos en su sistema interno, que permitiera al área de desembolsos del Banco Mundial en Brasil realizar el desembolso solicitado por BANOBRAS. El proyecto contó con la intervención oportuna del Órgano Interno de Control en la SEP, instancia que estuvo al pendiente de los resultados del proyecto, e intervino en los casos que ameritó corregir al ejecutor,y encausarle el rumbo de su actuación. Así mismo, cabe mencionar la oportuna participación de la Dirección General de Auditorías Externas de la Secretaría de la Función Pública, para asegurar que los informes que se entregan al Banco Mundial cumplieran con las disposiciones establecidas en los convenios celebrados entre ambas instancias (Memorandum de Entendimiento). Cabe mencionar, que siempre se contó con el valioso apoyo de la Unidad de Asuntos Internacionales de la SHCP, como prestatario del préstamo, para

44

resolver institucionalmente las dificultades que se presentaron durante la ejecución del proyecto. El desempeño de BANOBRAS como agente financiero en esta operación, se puede calificar de satisfactoria, al haber apoyado al ejecutor en todo momento en el cumplimiento de sus compromisos. BANOBRAS desempeñó su función de intermediario entre el ejecutor y el Banco Mundial, y llevó un seguimiento puntual y permanente sobre el comportamiento del proyecto; situación que reportó a la Unidad de Asuntos Internacionales de la SHCP, de manera oportuna. Para BANOBRAS, el haber sido agente financiero del Gobierno Federal en esta compleja operación, le significó un gran reto que asumió con responsabilidad institucional, en el que puso en práctica toda su capacidad profesional humana y técnica, para asegurar en todo momento, que el ejecutor cumpliera de manera eficiente y oportuna con los compromisos externos establecidos con el Banco Mundial en el contrato de préstamo; y con el gobierno mexicano, en el contrato de mandato formalizado con la SHCP y BANOBRAS. Con la ejecución de esta operación, BANOBRAS ganó una experiencia extraordinaria, que sumada a la que ya tiene en la administración de operaciones internacionales durante los últimos 25 años, le será de gran utilidad en la administración de otras operaciones que la SHCP le encomiende, conforme a las funciones naturales de su ramo. LECCIONES POR APRENDER. Este proyecto es un excelente ejemplo, del que el Banco Mundial y la SHCP pueden obtener valiosas lecciones; que sin duda, les servirán al momento de preparar otro proyecto en materia educativa, y se elija con elementos de juicio sólidos al agente financiero del Gobierno Federal que administre los recursos de un nuevo préstamo.

45

Annex 8. Comments of Co-financiers and Other Partners/Stakeholders N/A

46

Annex 9. List of Supporting Documents CNPRONABES. 2012. PRONABES data and indicators. (Not published) CONEVAL. 2012. Evaluación de Consistencia y resultados 2011-2012. Programa Nacional de Becas y Financiamiento, PRONABES. Mexico. Government of Mexico. 2007. Plan Nacional de Desarrollo, 2007-2012. Poder Ejecutivo Federal (Presidencia de la República). Mexico. Instituto Nacional de Estadística y Geografía. 2008. Encuesta Nacional de Ingresos y Gastos de los Hogares, ENIGH. Mexico. Instituto Nacional de Estadística y Geografía. 2010. Encuesta Nacional de Ingresos y Gastos de los Hogares, ENIGH. Mexico. PRONABES.2011. Plan del Ejecutor para el Mantenimiento futuro del Proyecto, 2011. Mexico. Secretaría de Educación Pública (Diario Oficial de la Federación). December, 2007. ACUERDO número 418 por el que se emiten las Reglas de Operación del Programa Nacional de Becas y Financiamiento (PRONABES). Mexico. Secretaría de Educación Pública (Diario Oficial de la Federación). December, 2009. ACUERDO número 500 por el que se emiten las Reglas de Operación del Programa Nacional de Becas y Financiamiento (PRONABES). Mexico. Secretaría de Educación Pública (Diario Oficial de la Federación). December, 2010. ACUERDO número 575 por el que se emiten las Reglas de Operación del Programa Nacional de Becas y Financiamiento (PRONABES). Mexico. Secretaría de Educación Pública (Diario Oficial de la Federación). December, 2011. ACUERDO número 601 por el que se emiten las Reglas de Operación del Programa Nacional de Becas y Financiamiento (PRONABES). Mexico. Secretaría de Educación Pública. 2007. Programa Sectorial de Educación 2007-2012. Mexico. World Bank. May 1, 2005. Social Assessment and Indigenous Peoples Development Plan. Washington, DC. World Bank. November 2005. Project Appraisal Document for a Tertiary Education Student Assistance Project. Report No. 31727-ME. Washington, DC. World Bank. April 7, 2006. Loan Agreement. Loan Number 7346-ME. Washington, DC.

47

World Bank. February 10, 2010. Restructuring Paper on a Proposed Restructuring of Tertiary Education Student Assistance Project (7346-ME and 7349-ME). Report No. 53003-MX. Washington, DC. World Bank. March 28, 2011. Amendment to the Loan Agreement (7346-ME). Washington, DC. World Bank. Terms of Reference for the Pre-appraisal and Appraisal Missions. World Bank. Aide Memoires and Back-to-Office reports for Preparation and Supervision missions. World Bank. Implementation Status Reports (ISRs) for the Tertiary Education Student Assistance Project (7346-ME).