education outcomes and contextual data: the potential of international surveys like pisa
DESCRIPTION
UNDP/OECD/UNESCO IIEP Presented by Simone Bloem, OECD Directorate for Education, Programme for Co-operation with Non-member economies at the Regional Conference on Fighting Corruption in Education in Eastern Europe and Central Asia, 24-25 November 2011, Bratislava, SlovakiaTRANSCRIPT
![Page 1: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/1.jpg)
Education outcomes and contextual data: the potential of international surveys like PISA
Simone BloemOECD Directorate for Education
Programme for Co-operation with Non-member economies
![Page 2: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/2.jpg)
Outline
• Overview of international surveys:
PISA TIMMS PIRLS
• Relevance of PISA for assessing integrity of education systems
• School system characteristics that are positively associated with performance, equity and integrity
• Next steps
2
![Page 3: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/3.jpg)
International Surveys
3
•OECD’s Programme for International Student Assessment
PISA
•IAE’s Trends in International Mathematics and Science Study
TIMMS
•IAE’s Progress in International Reading Literacy Study
PIRLS
![Page 4: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/4.jpg)
OECD: PISA
Triennial survey of key competencies of 15-year-old students
Main subjects tested: reading (2000, 2009), mathematics (2003) and science (2006)
Tool to compare the efficiency and equity of countries’ education systems in an international perspective
![Page 5: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/5.jpg)
A map of PISA countries and economies
5
![Page 6: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/6.jpg)
IAE: TIMMS
Assessment of mathematics and science competencies of 4th- and 8th-grade students
Survey conducted every four years: 1995, 1999, 2003, and 2007
Participation of more than 60 countries, diverse in terms of geographical location and economic development
![Page 7: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/7.jpg)
IAE: PIRLS
Assessment of students’ reading literacy at the end of primary school, in some countries students are assessed in fifth and sixth grade
PIRLS is conducted every five years : 2001, 2006 and 2011
55 countries from all around the world have participated in PIRLS 2011
PIRLS 2011 has been extended to include prePIRLS —less difficult and designed to test basic reading skills that are a prerequisite for PIRLS
![Page 8: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/8.jpg)
Reading Mathematics Science
Mean score Mean score Mean scoreMean S.E. Mean S.E. Mean S.E.
OECD
Estonia 501 (2.6) 512 (2.6) 528 (2.7)Poland 500 (2.6) 495 (2.8) 508 (2.4)Hungary 494 (3.2) 490 (3.5) 503 (3.1)OECD average 493 (0.5) 496 (0.5) 501 (0.5)Latvia 484 (3.0) 482 (3.1) 494 (3.1)Slovenia 483 (1.0) 501 (1.2) 512 (1.1)Czech Republic 478 (2.9) 493 (2.8) 500 (3.0)Slovak Republic 477 (2.5) 497 (3.1) 490 (3.0)Croatia 476 (2.9) 460 (3.1) 486 (2.8)Lithuania 468 (2.4) 477 (2.6) 491 (2.9)Russian Federation 459 (3.3) 468 (3.3) 478 (3.3)Serbia 442 (2.4) 442 (2.9) 443 (2.4)Bulgaria 429 (6.7) 428 (5.9) 439 (5.9)Romania 424 (4.1) 427 (3.4) 428 (3.4)Montenegro 408 (1.7) 403 (2.0) 401 (2.0)Albania 385 (4.0) 377 (4.0) 391 (3.9)
![Page 9: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/9.jpg)
Background data
Students background questionnaire• socio-economic background, learning habits, attitudes towards reading, involvement and
motivation
School principals questionnaire• demographic school characteristics, school policies, quality of learning environment
Parents questionnaire• PISA for selected countries (optional), PIRLS• home learning environment, parental support, school choice
Teachers Questionnaire• For TIMMS and PIRLS only• Professional background, teaching methods, training and professional development
Curriculum questionnaires for National Research Coordinators• TIMMS and PIRLS only
PIRLS/TIMMS 2011 Encyclopedia• provides a profile of each country’s education system
![Page 10: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/10.jpg)
Evidence-based, effective
education policies
Analysis in the international
context
International indicators and benchmarks
Analysis in the national and
regional context
International surveys deliver…
10
![Page 11: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/11.jpg)
Integrity and PISA
• Starting point to tackle corruptive practices is to evaluate the integrity of an educational system
• We are assuming that systems with high level of integrity are in general those systems that perform well – well in terms of student achievement and providing equal opportunities.
Integrity
• In PISA, successful school systems are defined as those that perform above the OECD average in reading and in which students’ socio-economic background has a smaller impact on reading performance than it is the case in a typical OECD country.
• The term “successful school system” refers o two distinct areas: achievement and equity
Successful school system
• Allows identifying school characteristics that are shared by successful school systems
• PISA is used as a first point of reference in order to assign education policy areas of integrity
PISA
![Page 12: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/12.jpg)
School characteristics related to performance and/or equity
Low levels of differentiation
Grade repetition
Student transference
Ability grouping
![Page 13: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/13.jpg)
Low differentiation of the school system
• Unequal access to schools and classrooms creates demand situations for scarce places which may lead to corruption, in particular when perceived quality of schools differs greatly
Differentiation : Access
![Page 14: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/14.jpg)
School characteristics related to performance and/or equity
Low levels of differentiation
Grade repetition
Student transference
Ability grouping
Accountability
The existence of standards-based
external examinations
And how they are used
Monitoring of lessons by inspectors
![Page 15: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/15.jpg)
Accountability
• Educational achievement is made observable and transparent• Facilitate the monitoring of performance of students, teachers,
and schools• Can be considered as best practice to avoid opportunistic
behaviour by teachers and principals• Teachers cannot “get away” by skipping content areas or
employ very easy tests to have a positive teacher performance evaluation
• Since teachers do not know exam questions and areas covered in the test beforehand, incentives for corruption are nonexistent, e.g. selling exam questions in advance
Standards-based external examinations
• Improve the signal effect of school leave certificates and final examinations marks which are of particularly importance for higher educational institutions or future employers and may reduce corruption, e.g. fraud at university admission or favouritism, political affiliation in employment
External final exams
![Page 16: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/16.jpg)
School characteristics related to performance and/or equity
Low levels of differentiation
Grade repetition
Student transference
Ability grouping
Accountability
The existence of standards-based
external examinations
And how they are used
Monitoring of lessons by inspectors
School autonomy
Autonomy to make decisions on curricula and assessments
![Page 17: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/17.jpg)
School autonomy
• School autonomy has a positive effect on student achievement, but only when appropriate accountability practices are in place
• Schools with more autonomy may have more incentives to opportunistic behaviour, in particular when there is a high degree of information asymmetry about school behaviour, e.g. parents, governing boards or central planning authorities have insufficient information on what is happing in the school
• “Without external exams, the advantage of superior local knowledge must weighed against the disadvantage of opportunistic behaviour, and the net effect of school autonomy depends on the relative size of these two partial effects.” Wößmann et al. (2008: 34)
School autonomy
![Page 18: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/18.jpg)
School characteristics related to performance and/or equity
Low levels of differentiation
Grade repetition
Student transference
Ability grouping
Accountability
The existence of standards-based
external examinations
And how they are used
Monitoring of lessons by inspectors
School autonomy
Autonomy to make decisions on curricula and
assessments
Spending on education
Teacher salaries
Minimum resource requirements
![Page 19: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/19.jpg)
Spending on education
• Low teacher salaries are considered as one reason of corruption• Reflect the social status of the teaching profession in a society• Suggests commitment to the rules and guidelines of teaching in a country
and satisfaction of profession• Reduced incentives for corrupt and unethical behaviour, e.g. private tutoring
or absenteeism
High teacher salaries
• The eventuality of corruptive behaviour is greater in schools that lack material, e.g. teachers might feel compelled to ask for money to provide students with learning materials, such as textbooks and laboratory equipment or principals may illegally rent out school facilities
• Great inequity between schools in terms of resources may lead to high demand for scarce places in “better” schools. This demand may lead to corruption occurrence
Availability of sufficient human and material resources
![Page 20: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/20.jpg)
School characteristics related to performance and/or equity
Low levels of differentiation
Grade repetition
Student transference
Ability grouping
Accountability
The existence of standards-based
external examinations
And how they are used
Monitoring of lessons by inspectors
School autonomy
Autonomy to make
decisions on curricula and assessments
Spending on education
Teacher salaries
Minimum resource
requirements
Learning environment
Disciplinary climate
Student and teacher
behaviour
Teacher-student relations
![Page 21: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/21.jpg)
Learning environment
• High quality of the learning environment, approximated by indicators such as good teacher-student relations, positive student and teacher behaviour and good disciplinary climate, are important to ensure a high level of moral and commitment in schools and classrooms.
• This can be considered as crucial elements of an intact school system that prevent various corruptive or unethical practices.
Quality of learning environment
![Page 22: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/22.jpg)
Next steps
• Integration of corruption-related items in the national PISA students’, parents’ and/or school questionnaire, e.g. questions related to private tutoring, gifts to teachers, cheating
PISA and corruption items
• Raise awareness of the problem of corruption in the education sector on an international level
• Open up new opportunities to research corruption empirically
• Make corruption in education empirically measurable and allow to relate corruption to educational outcomes
![Page 23: Education Outcomes and Contextual Data: the Potential of International Surveys like PISA](https://reader036.vdocument.in/reader036/viewer/2022070303/549029c9b47959605a8b45eb/html5/thumbnails/23.jpg)
Thank you for your attention!