PISA FOR DEVELOPMENTInternational Advisory Group MeetingClosing Session 27-28 May 2014

OECD Secretariat

1

PISA FOR DEVELOPMENT

Conclusions

Strong representation at the meeting from • Countries signed up or committed to participation in

PISA for Development (Cambodia, Ecuador, Guatemala, Rwanda, Senegal, Tanzania, Zambia),

• Development partners (France, Germany (BMZ/GIZ), IADB, Ireland (Irish Aid), Japan(JICA),Korea, Norway, UK (DFID), World Bank),

• International agencies (UNESCO, UIS, UNICEF, EFA GMR, CONFEMEN and PASEC, LLECE), and

• Independent technicians in the field.• Observers from Luxembourg and the United States

PISA for DevelopmentInternational Advisory Group - Conclusions

• Reconfirmed unanimous support for the PISA for Development project; and

• Acknowledged again the unique value that PISA has as an international benchmarking tool for quality and equity in schooling and for guiding policies for system improvement.

• Appreciated the actions taken since June 2013• Noted the indicative timeline and

implementation schedule for the project and noted that this is subject to change as country schedules are established

PISA for DevelopmentInternational Advisory Group - Conclusions

• Celebrated the signing of the Participation Agreement between OECD and Ecuador and the remarks of Harvey Sanchez , Executive Director of INEVAL, who explained how participation in PISA for Development will help drive up the quality and inclusiveness of the school system in Ecuador.

PISA for DevelopmentInternational Advisory Group - Conclusions

6

• Adopted the ToR for the IAG and the content of the Annual Report, subject to certain amendments, including clarifying:– the respective roles and expectations of the

constituent groups of the IAG– the roles between the IAG and the TAG– the overall governance of the project

• Noted that the IAG will have a chance to further review through written consultation

• Agreed to nominate co-chairs who would be elected at the next meeting of the IAG scheduled for March 2015

PISA for DevelopmentInternational Advisory Group - Conclusions

• Endorsed the issues and challenges identified in the expert paper on enhancing the PISA cognitive instruments and

• Agreed on the options for adapting the cognitive instruments to fit the contexts of developing countries including in the four key aspects of review of assessment framework and items, test design, review of proficiency levels and review of scaling models.

• Noted that sampling and target population comparability was also an issue to seek bidders proposals on.

• Noted the need to ensure sufficient flexibility in order to serve national policy needs, while at the same time providing valid international benchmarking.

PISA for DevelopmentInternational Advisory Group - Conclusions

• Endorsed the issues and challenges identified in the expert paper on enhancing the PISA context questionnaires; and

• Agreed on the options for adapting the questionnaires to fit the contexts of developing countries including measures of early learning opportunities, language at home and school, family and community support, quality of instruction, learning time, student socioeconomic status and school resources.

• Noted the constraint of the length of the student questionnaire and that national optional items could be included to meet specific national needs but suggested that a common core of questionnaire items be identified to permit international comparisons

• Discussed the potential value and resource implications of teacher and parent questionnaires and suggested that bidders are asked to comment on the value added of including these in PISA for development.

PISA for DevelopmentInternational Advisory Group - Conclusions

• Agreed the importance of seeking approaches for including out of school 15-year-olds in the study and recognised the significant challenges involved and the possible options for addressing these.

• Reviewed and endorsed the first draft of the expert paper on out of school 15 year olds and noted the next steps in discussing and finalising this by October 2014.

• Recognised this as a distinct strand of the project and that this provided the opportunity for researching different approaches that may vary to meet country-specific needs.

PISA for DevelopmentInternational Advisory Group - Conclusions

10

Agreed on the draft principles and components of Terms of Reference for the International Contractor(s) related to the cognitive instruments and contextual questionnaires with certain modifications, amplifications and additions, including:• greater flexibility for bidders to respond to

issues and questions and less prescription• need to have a sufficient framework on which

bids can be equally judged• expert papers will be provided to bidders

PISA for DevelopmentInternational Advisory Group - Conclusions

11

• bidders should be open to working with other existing assessment organisations and programmes

• bidders should be asked to clarify the human and financial resource implications of their proposals

• bidders should make use of local contractors in all aspects of the work alongside the international contractor(s)

• ensure there are adequate national structures in place to effectively implement PISA for Development

PISA for DevelopmentInternational Advisory Group - Conclusions

12

• For contextual questionnaires: - need to have clear evaluation

criteria with an emphasis on cross-cultural validity

- bidders should make use of and reference the literature that exists on factors affecting learning outcomes in each of the regions

PISA for DevelopmentInternational Advisory Group - Conclusions

13

• For contextual questionnaires: - bidders should propose the best

respondents to target in order to get robust information on the key features

- bidders should be asked to provide separately: (1) human and financial resource implications for student and school questionnaires and (2) human and financial resource implications for additional questionnaires that may be proposed

PISA for DevelopmentInternational Advisory Group - Conclusions

14

• For cognitive instruments, Secretariat to examine the potential for including in the pool of secure items those that were rejected for main PISA

Noted OECD’s plans for completing and consulting the IAG on the full ToR for the International Contractor(s) and the tendering process

PISA for DevelopmentInternational Advisory Group Conclusions

• Noted the important role that PISA for Development is going to play in building capacity in the participating countries, including – for developing and conducting national and

international large scale assessments, and – for using performance data to diagnose

strengths and weaknesses in the education system, and for supporting school improvement efforts.

PISA for DevelopmentInternational Advisory Group - Conclusions

• Endorsed the plans for completing the capacity needs analysis and designing capacity building programmes for the participating countries

• Noted how the capacity building plans will be incorporated within the countries’ project implementation plans and coordinated with other strategies and support for assessment in the participating countries.

• Requested that the responsibilities of the International Contractor(s) for capacity building are fully reflected in the ToR.

PISA for DevelopmentInternational Advisory Group - Conclusions

• Noted and supported the plans for peer-to-peer learning, especially the high level strategic report

• Noted the important role that PISA for Development and PISA can play in contributing to and measuring post-2015 global educational targets that are focussed on learning.

PISA for DevelopmentInternational Advisory Group - Conclusions

PISA FOR DEVELOPMENT

Next Steps

19

• OECD to complete the remaining planned expert papers and hold third technical workshop (OOS)

• OECD to prepare ToR and tendering documents for international contractor(s) and to finalise these in consultation with the IAG members

• OECD to sign participation agreements with all the countries participating in the project

• Participating countries to nominate NCs and NPMs and to prepare project implementation plans

Next Steps

20

• Participating countries finalise agreements with development partners regarding contributions and support (e.g. international costs, in-country costs and activities)

• OECD to complete capacity needs analysis and to design capacity-building programmes for all participating countries

• OECD to finalise with development partners outstanding agreements for support to the project – general contributions and country-specific contributions

• OECD to confirm with other agencies arrangements for technical partnerships and collaboration

• First meeting of Technical Advisory Group after awarding of international contract

Next steps continued

21

• Complete tendering process to commission international contractor (s) by OECD

• Project implementation:- Design, planning and coordination (including

capacity assessment and capacity building planning for participating countries);

- Technical development- Field trials and main data collection;- Analysis and reporting- Post-pilot governance

Next steps continued