what makes schools and school systems successful … · what makes schools and school systems...

OECD EMPLOYER

BRAND

Playbook

1

What Makes Schools and

School Systems

Successful

Lessons for the

GCC States

from PISA 2012

Tue Halgreen 4 March 2015

2 PISA in brief

• Over half a million students… – representing 28 million 15-year-olds in 65 countries/economies

… took an internationally agreed 2-hour test… – Goes beyond testing whether students can

reproduce what they were taught…

… to assess students’ capacity to extrapolate from what they know and creatively apply their knowledge in novel situations

– Mathematics, reading, science, problem-solving, financial literacy

– Total of 390 minutes of assessment material

… and responded to questions on… – their personal background, their schools

and their engagement with learning and school

• Parents, principals and system leaders provided data on… – school policies, practices, resources and institutional factors that

help explain performance differences .

3 The structure of the PISA assessment

2000 2003 2006 2009 2012

Reading Reading Reading Reading Reading

Mathematics Mathematics Mathematics Mathematics Mathematics

Science Science Science Science Science

Problem Solving Digital Reading

Problem Solving, Financial literacy,

Digital Math, Digital reading

Singapore

Hong Kong-China Chinese Taipei

Korea

Colombia

Japan

Peru

Switzerland

Netherlands Estonia Finland Canada

Poland Belgium

Germany Viet Nam

Austria Australia Ireland Slovenia

Brazil

New Zealand

Qatar

France United Kingdom

Argentina Tunisia

Indonesia

Norway Portugal Italy Spain

Russian Fed.

Jordan

United States Lithuania Sweden Hungary

Croatia Israel

Greece Serbia Turkey

Romania

Bulgaria U.A.E. Kazakhstan Thailand

Chile Malaysia

Mexico

350

360

370

380

390

400

410

420

430

440

450

460

470

480

490

500

510

520

530

540

550

560

570

580

Mean score

Fig I.2.13 3

High mathematics performance

Low mathematics performance

… Shanghai-China is above this level (613)

Mean performance in mathematics – PISA 2012

Mathematics 2006 2009 2012

Qatar 318 368 376

United Arab Emirates 411 423

6 Qatar and UAE – Trends in PISA

Science 2006 2009 2012

Qatar 349 379 384


Reading 2006 2009 2012

Qatar 312 372 388


Figures in bold indicate a statistically significant annualised change

100

80

60

40

20

0

20

40

60

80

100A

lban

iaF

inla

nd

Icela

nd

Sw

ede

nN

orw

ay

Denm

ark

Esto

nia

Ire

land

Spa

inC

ana

da

Pola

nd

Latv

iaK

aza

kh

sta

nU

nited

Sta

tes

Me

xic

oC

olo

mb

iaC

osta

Ric

aR

ussia

n F

ed.

Ma

laysia

Jo

rdan

New

Ze

ala

nd

Lithu

ania

Gre

ece

Mo

nte

neg

roU

nited

Kin

gd

om

Arg

en

tin

aA

ustr

alia

Bra

zil

Port

ug

al

Ind

one

sia

Chile

Thaila

nd

Rom

ania

Tunis

iaS

witze

rla

nd

Peru

Uru

gua

yC

roa

tia

U.A

.E.

Ma

ca

o-C

hin

aS

erb

iaV

iet N

am

Kore

aH

ong

Kon

g-C

hin

aS

inga

po

reA

ustr

iaIt

aly

Luxe

mb

ou

rgC

zech R

ep

ub

licJa

pa

nB

ulg

aria

Isra

el

Qata

rS

ha

ngh

ai-

Ch

ina

Germ

any

Slo

ven

iaS

lovak R

epu

blic

Turk

ey

Belg

ium

Hung

ary

Lie

ch

tenste

inN

eth

erla

nds

Chin

ese

Taip

ei

Variability in student mathematics performance

between and within schools Variation in s

tudent

perf

orm

ance

as

% o

f O

ECD

avera

ge v

ariation

Fig II.2.7

OECD average

OECD average

7

Performance variation of

students within schools

Performance differences

between schools

200

494

-3 -2 -1 0 1 2 3

School performance and socio-economic background:

Finland 8

Advantage PISA Index of socio-economic background Disadvantage

Student performance and students’ socio-economic background

School performance and schools’ socio-economic background

Student performance and students’ socio-economic background within schools

Stu

dent

perf

orm

ance

700

200

494

-3 -2 -1 0 1 2 3


Qatar 9





Stu

dent

perf

orm

ance

700

200

494

-3 -2 -1 0 1 2 3


United Arab Emirates 10





Stu

dent

perf

orm

ance

700

200

494

-3 -2 -1 0 1 2 3


United Arab Emirates 11





Stu

dent

perf

orm

ance

700

-100

-50

0

50

100

150

Qata

rU

.A.E

.A

ustr

alia

Th

aila

nd

Ma

ca

o-C

hin

aM

on

ten

eg

roIs

rael

Jo

rdan

Un

ited

Sta

tes

Serb

iaH

ung

ary

Ho

ng

Kon

g-C

hin

aS

inga

po

reS

lovak R

epu

blic

Cana

da

Ne

w Z

eala

nd

Ch

inese

Taip

ei

Lithu

ania

Kaza

kh

sta

nM

ala

ysia

Irela

nd

Turk

ey

Un

ited

Kin

gd

om

Cro

atia

Luxe

mb

ou

rgL

atv

iaC

hile

Co

sta

Ric

aA

rgen

tin

aR

ussia

n F

ed.

Cze

ch R

ep

ub

licO

EC

D a

ve

rag

eG

erm

any

Slo

ven

iaG

reece

No

rwa

yE

sto

nia

Icela

nd

Italy

Austr

iaN

eth

erla

nds

Spa

inF

ran

ce

Port

ug

al

Sw

ede

nL

iech

tenste

inD

enm

ark

Sw

itze

rla

nd

Belg

ium

Bra

zil

Fin

land

Me

xic

oC

olo

mb

iaP

eru

Sha

ngh

ai-

Ch

ina

Sco

re p

oin

t d

iffe

ren

ce

before accounting for students' socio-economic status

after accounting for students' socio-economic status

Differences in mathematics performance between

students without and with an immigrant background

Students without an immigrant

background perform better

Students with an immigrant

background perform better

Fig II.3.4 12

math teaching ≠ math teaching PISA = reason mathematically and understand, formulate, employ

and interpret mathematical concepts, facts and procedures

13

0.00

0.50

1.00

1.50

2.00

2.50V

iet N

am

Ma

ca

o-C

hin

aS

ha

ngh

ai-

Ch

ina

Turk

ey

Uru

gua

yG

reece

Hong

Kon

g-C

hin

aC

hin

ese

Taip

ei

Port

ug

al

Bra

zil

Serb

iaB

ulg

aria

Sin

ga

po

reN

eth

erla

nds

Ja

pa

nA

rgen

tin

aC

osta

Ric

aL

ithu

ania

Tunis

iaN

ew

Ze

ala

nd

Czech R

ep

ub

licIs

rael

Kore

aL

atv

iaQ

ata

rIt

aly

United

Sta

tes

Esto

nia

Irela

nd

Austr

alia

Me

xic

oU

nited

Ara

b E

mira

tes

Norw

ay

Ma

laysia

Kaza

kh

sta

nU

nited

Kin

gd

om

Ro

ma

nia

OE

CD

ave

rag

eA

lban

iaC

olo

mb

iaIn

do

ne

sia

Sw

ede

nB

elg

ium

Peru

Thaila

nd

Denm

ark

Ru

ssia

n F

ede

ratio

nC

ana

da

Slo

vak R

epu

blic

Hung

ary

Germ

any

Cro

atia

Luxe

mb

ou

rgM

on

ten

eg

roC

hile

Pola

nd

Fin

land

Austr

iaS

loven

iaF

ran

ce

Sw

itze

rla

nd

Jo

rdan

Lie

ch

tenste

inS

pa

inIc

ela

nd

Ind

ex

of

ex

po

su

re t

o w

ord

pro

ble

ms

Students' exposure to word problems Fig I.3.1a 14

Formal math situated in a word problem, where it is obvious to

students what mathematical knowledge and skills are needed

0.00

0.50

1.00

1.50

2.00

2.50S

wede

nIc

ela

nd

Tu

nis

iaA

rgen

tin

aS

witze

rla

nd

Bra

zil

Luxe

mb

ou

rgIr

ela

nd

Neth

erla

nds

New

Ze

ala

nd

Costa

Ric

aA

ustr

iaL

iech

tenste

inM

ala

ysia

Ind

one

sia

Denm

ark

Un

ited

Kin

gd

om

Uru

gua

yL

ithu

ania

Germ

any

Austr

alia

Chile

OE

CD

ave

rag

eS

lovak R

epu

blic

Thaila

nd

Qata

rF

inla

nd

Port

ug

al

Colo

mb

iaM

exic

oP

eru

Czech R

ep

ub

licIs

rael

Italy

Belg

ium

Hong

Kon

g-C

hin

aP

ola

nd

Fra

nce

Spa

inM

on

ten

eg

roG

reece

Turk

ey

Slo

ven

iaV

iet N

am

Hung

ary

Bulg

aria

Kaza

kh

sta

nC

hin

ese

Taip

ei

Cana

da

United

Sta

tes

Esto

nia

Rom

ania

Latv

iaS

erb

iaJa

pa

nK

ore

aC

roa

tia

Alb

an

iaR

ussia

n F

ede

ratio

nU

nited

Ara

b E

mira

tes

Jo

rdan

Ma

ca

o-C

hin

aS

inga

po

reS

ha

ngh

ai-

Ch

ina

Icela

nd

Ind

ex

of

ex

po

su

re t

o f

orm

al m

ath

em

ati

cs

Students' exposure to conceptual understanding Fig I.3.1b 15

0.00

0.50

1.00

1.50

2.00

2.50C

zech R

ep

ub

licM

aca

o-C

hin

aS

ha

ngh

ai-

Ch

ina

Vie

t N

am

Uru

gua

yF

inla

nd

Costa

Ric

aS

we

de

nJa

pa

nC

hin

ese

Taip

ei

Italy

Isra

el

Norw

ay

Esto

nia

Ho

ng

Kon

g-C

hin

aA

ustr

iaS

erb

iaK

ore

aC

roa

tia

Latv

iaS

lovak R

epu

blic

Gre

ece

United

Kin

gd

om

Irela

nd

Luxe

mb

ou

rgB

elg

ium

Mo

nte

neg

roA

rgen

tin

aS

loven

iaB

ulg

aria

OE

CD

ave

rag

eL

ithu

ania

Hung

ary

Sw

itze

rla

nd

New

Ze

ala

nd

Germ

any

Tu

rke

yD

enm

ark

Ru

ssia

n F

ede

ratio

nS

inga

po

reIc

ela

nd

United

Sta

tes

Spa

inQ

ata

rL

iech

tenste

inP

ola

nd

Austr

alia

Fra

nce

Bra

zil

Ma

laysia

Peru

Cana

da

Chile

United

Ara

b E

mira

tes

Rom

ania

Tunis

iaN

eth

erla

nds

Port

ug

al

Colo

mb

iaA

lban

iaK

aza

kh

sta

nJo

rdan

Me

xic

oIn

do

ne

sia

Thaila

nd

Ind

ex

of

ex

po

su

re t

o a

pp

lie

d m

ath

em

ati

cs

Students' exposure to applied mathematics Fig I.3.1c 16

430

450

470

490

510

0.0 0.5 1.0 1.5 2.0 2.5 3.0

Me

an

sc

ore

in

ma

the

ma

tics

Index of exposure to applied mathematics

rarely sometimes frequently never

Relationship between mathematics performance

and students' exposure to applied mathematics Fig I.3.2 17

Hong Kong-China

Brazil

Uruguay

Albania

Croatia

Latvia

Lithuania

Chinese Taipei

Thailand Bulgaria

Jordan

Macao-China

UAE Argentina

Indonesia

Kazakhstan

Peru

Costa Rica

Tunisia

Qatar

Singapore

Colombia

Malaysia

Serbia

Romania

Viet Nam

Shanghai-China

USA

Poland

New Zealand

Greece

UK

Estonia

Finland

Slovak Rep.

Luxembourg

Germany Austria

Czech Rep.

France

Japan

Turkey

Sweden

Hungary Australia

Israel

Canada

Chile

Belgium

Netherlands Spain

Denmark

Switzerland

Iceland

Slovenia

Portugal

Norway

Korea

Italy

R² = 0.13

300

350

400

450

500

550

600

650

-1.5 -1 -0.5 0 0.5 1 1.5

Ma

the

ma

tic

s p

erf

orm

an

ce

(s

co

re p

oin

ts)

Index of school responsibility for curriculum and assessment (index points)

Countries that grant schools autonomy over curricula and

assessments tend to perform better in mathematics Fig IV.1.15

Schools with more autonomy perform better than schools with

less autonomy in systems with more accountability arrangements

School data not public

School data public464

466

468

470

472

474

476

478

Less school autonomy

More school autonomy

Score points

School autonomy for curriculum and assessment

x system's level of posting achievement data publicly

Fig IV.1.16

Money makes a difference – but only up to a point

Slovak Republic

Czech Republic Estonia

Israel

Poland

Korea

Portugal

New Zealand

Canada Germany

Spain

France

Italy

Singapore

Finland

Japan

Slovenia Ireland

Iceland

Netherlands

Sweden

Belgium

UK

Australia Denmark

United States

Austria

Norway

Switzerland

Luxembourg

Viet Nam

Jordan

Peru

Thailand

Malaysia

Uruguay

Turkey

Colombia

Tunisia

Mexico Montenegro

Brazil

Bulgaria

Chile

Croatia Lithuania

Latvia

Hungary

Shanghai-China

R² = 0.01

R² = 0.37

300

350

400

450

500

550

600

650

0 20 000 40 000 60 000 80 000 100 000 120 000 140 000 160 000 180 000 200 000

Ma

the

ma

tic

s p

erf

orm

an

ce

(sc

ore

po

ints

)

Average spending per student from the age of 6 to 15 (USD, PPPs)

Cumulative expenditure per student less than USD 50 000

Cumulative expenditure per student USD 50 000 or more

Fig IV.1.8

Hong Kong-China

Brazil

Uruguay

Croatia

Latvia

Chinese Taipei

Thailand

Bulgaria

Jordan

Macao-China

UAE

Argentina

Indonesia

Kazakhstan

Peru

Costa Rica Montenegro

Tunisia

Qatar

Singapore

Colombia

Malaysia Serbia

Romania

Viet Nam

Shanghai-China

USA

Poland

New Zealand

Greece

UK

Estonia

Finland

Slovak Rep.

Luxembourg

Germany

Austria France

Japan

Turkey Sweden Hungary

Australia Israel

Canada

Ireland

Chile

Belgium

Spain Denmark

Switzerland

Iceland

Slovenia

Portugal Norway

Mexico

Korea

Italy

R² = 0.19

300

350

400

450

500

550

600

650

700

-0.500.511.5

Ma

the

ma

tics

perf

orm

an

ce

(sc

ore

po

ints

)

Equity in resource allocation (index points)

Countries with better performance in mathematics tend

to allocate educational resources more equitably

Greater

equity Less

equity

Adjusted by per capita GDP

Fig IV.1.11

30% of the variation in math performance across OECD countries is explained by the degree of similarity of

educational resources between advantaged and disadvantaged schools

PISA-Based Test for Schools Overview

Based on international PISA test of 15-year olds. All results are comparable to international PISA scales

Can be used by schools, networks of schools and districts

Goes beyond testing whether students can reproduce what they were taught to assess students’ capacity to apply their knowledge in novel situations

Provides information on students’ engagement and the learning environment at the school

PISA-Based Test for Schools What does the assessment look like?

• Experience for students similar to that of the main PISA tests: 2h test +30min questionnaire

• Three assessment domains: reading, maths, science

• Student sample size per school: 85

• A comprehensive (150 pages) school report for each participating school

Thank you !

Find out more about PISA at www.pisa.oecd.org

• All national and international publications

• The complete micro-level database

Email: [email protected]

what makes schools and school systems successful … · what makes schools and school systems...

Documents