Maintaining Momentum in Primary School: messages from research and evaluation

 Presentation Prepared for the Social Mobility and Life

Chances Forum Maintaining Momentum, Milton Hill Oxford

December 3-4 2004

 

Pam Sammons & Kathy Sylva

University of Nottingham/University of Oxford

What is the impact of multiple disadvantage and does pre-school promote better child outcomes at primary school? - Evidence from the EPPE study

Does the primary school a child attends ‘make a difference’ to

their educational outcomes? - Evidence from school effectiveness research

What is the evidence of ‘improvement through inspection’ and has this benefited disadvantaged groups? - Evidence from an evaluation of the impact of Ofsted

What kinds of specific interventions promote better outcomes for disadvantaged groups of children? – Evidence from the SPOKES study

 

Content of Presentation

Attainments of EPPE Sample at end of Year 1 by Multiple Disadvantage

Multiple Disadvantage Index

Primary Reading standardised score

Maths 6 standardised score

N mean sd N mean sd

0 588 104.1 13.7 587 106.7 14.3

1 679 102.7 14.2 680 102.7 15.0

2 532 98.8 15.0 531 99.8 14.9

3 336 96.2 13.7 333 96.1 14.7

4 221 92.1 15.0 219 90.0 13.6

5 plus 167 89.9 13.9 165 89.9 14.6

All pupils 2532 99.6 15.0 2515 100.2 15.6

Pre-reading at school entry

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

Low duration High duration

Effe

ct s

ize

Low quality High quality

Impact of quality and duration

Effect of pre-school (v. no pre-school) on social-behavioural outcomes at school entry

0

0.1

0.2

0.3

0.4

0.5

0.6

Cooperation andconformity

Independence andconcentration

Peer sociability

Effe

ct s

ize

READING at key stage 1, social class and pre-school experience

WRITING at key stage 1, social class and pre-school experience

The contribution of social class and pre-school to literacy attainment (age 7)

1.8

2.0

2.2

2.4

2.6

2.8

Professional Skilled Un/semi skilled

Social class by occupation

Me

an

ye

ar

2 r

ea

din

g le

vel

Pre-school

Expected minimum

No pre-school

1.6

1.8

2.0

2.2

2.4

2.6

Professional Skilled Un/semi skilled

Social class by occupation

Me

an

ye

ar

2 w

ritin

g le

vel

Pre-school

Expected minimum

No pre-school

● Higher quality & longer duration of pre-school

● Integrated settings and Nursery schools

● Good home learning environment and employed parent(s)

Pre-school reduced proportion of children ‘at risk of SEN from 1:3 to 1:5

What reduces the risk of SEN?

Focus of SER

The central focus a belief in the potency of social institutions

‘the idea that schools matter, that schools do have major effects upon children’s development and that, to put it simply, schools do make a difference’ (Reynolds & Creemers, 1990)

‘Effectiveness is not a neutral term. Defining the effectiveness of a particular school always requires choices among competing values … the criteria of effectiveness will be the subject of political debate’ (Firestone, 1990)

Aims & Goals of Early SER

to promote Equity and ExcellenceClientele - poor/ethnic minority

studentsSubject matter - basic skills reading &

mathsEquity - children of urban poor should

achieve at same level as those of middle classes

Focus on Student Outcomes

‘For us the ‘touchstone’ criteria to be applied to all educational matters concern whether children learn more or less because of the policy or practice’ Reynolds 1997

‘An effective school is one in which students progress further than might be expected from consideration of its intake’ Mortimore 1991

SER seeks to identify the ‘Value Added’ by schools to student outcomes

Methodology

mainly quantitative, but case studies important

values reliability and replicabilityseeks to make generalisationsworks in partnership with

practitionersvalues the views and perceptions of

teachers, students and parents

The Impact of Intake

‘ ‘Natural justice demands that schools are held accountable only for those things they can influence (for good or ill) and not for all the existing differences between their intakes’ (Nuttall 1990)

SER seeks to disentangle the impact of prior attainment and background characteristics from the impact of school and classes/teachers on students’ progress/social or affective outcomes

Example of More Effective or Less Effective School Profiles at Key Stage 1

Progress Effectiveness

Category English Maths Science

positive* X positive X X

As expected negative O negative* O O

*value added statistical outlier, p<0.05 X more effective profile O less effective profile

Example of value added feedback from Improving School Effectiveness Project: Primary Schools’ AAP

Results

N of primary schools= 44 * p<0.05 , MacBeath & Mortimore, 2000

Value added effectiveness category

AAP Mathematics

AAP Reading

  n % n %

Positive Outlier (p<0.05) *   

10 23 5 11

Positive (non-significant) As expected

7 16 17 39

Negative (non-significant) As expected

15 35 19 43

Negative Outlier (P<0.05) * 11 26 3 7

Defining Consistency Within school comparisons focus on

internal variation in effects

For different cognitive & non-cognitive outcomes

By different year groups within each school, including variations in class or teacher effects

For different pupil groups - boys/girls - initial low/high attainers, - low SES/high SES

Effectiveness is a relative concept which is time and outcome specific

Effective in promoting which outcomes?

the what of effectivenessEffective for which student groups?

the who of effectivenessEffective over what time period?

the when of effectiveness

Differential Effectiveness

The size of school effects for black students were almost twice as large as for white students in the US (Coleman et al 1966)

Differences between public and private schools almost twice as large for low SES students as for middle class students, differences between schools for high SES students small in US (Bryk & Raudenbush, 1992)

School effects vary for students by race and low prior attainment in England. School effects larger for initially low attaining and for black Caribbean students (Nuttall et al 1989)

Primary school effects vary for students with low compared with high initial attainment in England, being larger for low initial attainers (Sammons et al 1993)

Equity Implications

Dutch primary schools are highly stable in effectiveness across grades for low SES students, less stable in effectiveness across grades for high SES students (Bosker 1995)

‘Schools matter most for underprivileged and/or initially low achieving students. Effective or ineffective schools are especially effective or ineffective for these students’

After Scheerens & Bosker 1997

1. The processes of effective leadership 2. The processes of effective teaching 3. Developing & maintaining a pervasive

focus on learning 4. Producing a positive school culture 5. Creating high (& appropriate expectations

for all) 6. Emphasising student responsibilities &

rights 7. Monitoring progress at all levels 8. Developing staff skills at the school site 9. Involving parents in productive

& appropriate ways

The Processes of Effective Schools

After Teddlie & Reynolds 2000

The ‘ineffective’ school (Reynolds 1995)

Non-rational approach to evidence

fear of outsiders dread of change capacity for blaming

external conditions set of internal cliques lack of competencies for

improvement

..may have inside itself multiple schools formed around cliques and friendship groups .. There will be none of the organisational, social, cultural and symbolic tightness of the effective school

Empirical Confirmation of SE : Meta-Analyses

cooperation school climate monitoring at school and class level opportunity to learn (content coverage - homework -

time) parental involvement pressure to achieve school leadership After Scheerens & Bosker 1997

The most powerful factors are located at the classroom level. Schools should address ‘proximal variables’ like curriculum, instruction and assessment which emphasis student outcomes Wang et al 1993

Processes for School Improvement

Clear leadership Developing a shared vision & goals Staff development & teacher learning Involving pupils, parents & community Using an evolutionary development planning process Redefining structures, frameworks, roles & responsibilities Emphasis on teaching & learning Monitoring, problem-solving & evaluation Celebration of success External support, networking & partnership

Significance of School EffectsAlthough the differences in scholastic attainmentachieved by the same student in contrastingschools is unlikely to be great, in many instances itrepresents the difference between success andfailure and operates as a facilitating or inhibitingfactor in higher education.

When coupled with the promotion of other pro-social attitudes and behaviours, and the inculcation of a positive self-image,the potential of the school to improve the life chances of students is considerable.

Mortimore 1998:143

Impact of Inspection: Outcomes of special measures over 10 years

Primary

N %

Special

N %

Secondary

N %

PRUs

N %

Total

N %

Removed

from

special

measures

799 89.6 114 77.0 167 76.6 18 60.0 1098 85.3

Closed 93 10.4 34 23.0 51 23.4 12 40.0 190 14.6

Total 892 148 218 30 1288

Perceptions of benefits of inspections: 2002/03; comparison of head teachers’ and teachers’ views

Benefits outweigh

negative effects

%

Benefits and negative

effects equally

balanced %

Negative effects

outweigh benefits

%

Head

Teachers

N = 2801

60

25

14

Teachers

N = 2436

45

32

22

Judgements of extent of Improvement of primary and secondary schools since their last inspection (2002/03 Annual Report)

24

23

43

44

25

26

8

8

0% 20% 40% 60% 80% 100%

Secondary schools

Primary schools

Excellent/very good

Good Satisfactory

Unsatisfactory/poor

Primary schools: change of inspection judgements from first to second inspection (percentage of schools)

9

24

35

32

42

27

12

13

2

4

0%

20%

40%

60%

80%

100%

Teaching

Leadership and management

Significant improvement

Improvement No change

Deterioration Significant deterioration

Percentage of 11 year-old pupils reaching level 4 and above in English, mathematics and science

5763 65

7175 75 75 75

54

6259

6972 71 73 73

62

69 69

78

85 87 86 87

0

20

40

60

80

100

1996 1997 1998 1999 2000 2001 2002 2003

Pe

rce

nta

ge

of

pu

pil

s

English Maths Science

Government target for English and mathematics for 2006

The proportion of good or better teaching in primaryschools

4 5

5 5

6 2

7 3 7 4 7 6 7 5

0

2 0

4 0

6 0

8 0

1 0 0

1 9 9 6 /9 7 1 9 9 7 /9 8 1 9 9 8 /9 9 1 9 9 9 /0 0 2 0 0 0 /0 1 2 0 0 1 /0 2 2 0 0 2 /0 3

Per

cen

tag

e o

f sc

ho

ols

Distribution of Reading Achievement in 9-10 year olds in 2001

300

325

350

375

400

425

450

475

500

525

550

575

Sw

eden

Net

herl

ands

Eng

land

Bul

gari

a

Latv

ia

Can

ada

(Ont

ario

,Que

bec)

Lith

uani

a

Hun

gary

Uni

ted

Sta

tes

Italy

Ger

man

y

Cze

ch R

epub

lic

New

Zea

land

Sco

tland

Sin

gapo

re

Rus

sian

Fed

erat

ion

Hon

g K

ong

SA

R

Fran

ce

Gre

ece

Slo

vak

Rep

ublic

Icel

and

Rom

ania

Isra

el

Slo

veni

a

Inte

rnat

iona

l Avg

.

Nor

way

Cyp

rus

Mol

dova

, Rep

of

Turk

ey

Mac

edon

ia, R

ep o

f

Col

ombi

a

Arg

entin

a

Iran

, Isl

amic

Rep

of

Kuw

ait

Mor

occo

Bel

ize

Source: PIRLS 2001 International Report: IEA’s Study of Reading Literacy Achievement in Primary Schools

International Comparisons of Reading Attainment 2001:IEA

Is improvement greater in schools facing challenging circumstances?

P r o g r e s s i n c l o s i n g t h e s o c i o - e c o n o m i c a t t a i n m e n t g a p i n p r i m a r y s c h o o l s 1 9 9 6 - 2 0 0 1

S O U R C E : D f E S S c h o o l l e v e l d a t a

K e y S t a g e 1 R e a d i n g

K e y S t a g e 2 E n g l i s h

F S M b a n d 1 9 9 6 2 0 0 1 1 9 9 6 2 0 0 1 8 % o r l e s s 9 0 9 2 7 4 8 7 8 + t o 2 0 % 8 3 8 7 6 4 7 8 2 0 + t o 3 5 % 7 5 8 1 5 1 6 9 3 5 + t o 5 0 % 6 7 7 5 4 1 6 1 A b o v e 5 0 % 6 3 7 1 3 4 5 7 T o t a l 8 2 8 7 6 0 7 8

M E D I A N S C H O O L A C H I E V E M E N T I N T E R M S O F P R O P O R T I O N R E A C H I N G E X P E C T E D L E V E L , G R O U P E D B Y E L I G B I L I T Y F O R F R E E S C H O O L M E A L S

Percentage of unsatisfactory /poor lessons in primary schools going into special measures and two years after coming out (2002/03)

-10

0

10

20

30

40

50

60

70

80

Lessons from S10 inspections when schools went into special measures

Lessons from S10 inspections two years after coming out of special measures

Percentage of lessons unsatisfactory or poor

Percentage of primary schools entering SM within their

FSM bands compared against National distribution- 2002/03

010

2030

4050

1 2 3 4 5

Free School Meals (FSM) Bands

Per

cen

tag

e (%

)

% placed in Special Measures National % in each FSM Band

Disadvantaged pupils are over-represented in schools judged to require special measures

Improving City Schools: key features of teaching

a high degree of consistency across the school

high expectations of pupils, matched by well planned support to help them meet the challenges of the work

skilful management of pupils in classrooms and effective use of time and resources

motivating teaching methods & materials, planned with the improvement of basic skills in mind

Ofsted 2000

Challenges for 21st century

Pupils from disadvantaged backgrounds are more likely than others to experience educational failure

Reasons for eradicating school failure philosophical/ethical - to promote fairness improvement in

quality of life and opportunities for all groups, to encourage positive attitudes to future learning and self-esteem

political - to promote social cohesion and inclusion and empower young people as citizens to participate in a successful democracy

economic - to promote future prosperity & prevent waste of talent & avoid social/economic burden on Governments

Maintaining Momentum in the Primary Phase : messages from research & evaluation Pre-school provides children with a better start to school and is particularly

important in improving attainment for low SES pupils, the impact is still evident at age 7 years

Schools vary in their effectiveness. For disadvantaged groups the effectiveness

of the primary school attended is particularly important.

SER provides an important evidence-base on the correlates of effective schools and teachers and has stimulated school improvement initiatives at national and local level.

Inspection, has helped raise overall attainment levels and improved the quality of teaching in primary schools.

Inspection has acted as a powerful catalyst for improvement of weaker schools and this has benefitted disadvantaged pupil groups because they are over represented in such schools.

For the most vulnerable groups of pupils intensive, structured and targetted interventions are needed at an early stage.

The EPPE teamKathy Sylva University of Oxford

Edward Melhuish Birkbeck, University of London

Pam Sammons University of Nottingham

Iram Siraj-Blatchford Institute of Education, University of London

Brenda Taggart Institute of Education, University of London

http://www.ioe.ac.uk/projects/eppehttp://www.dfes.gov.uk/research/

EPPE is an ESRC TLRP (Affiliate) project