uncovering indicators of effective school management in south africa using the national school...

Programme to Support Pro-Poor Policy DevelopmentA partnership between the Presidency, Republic of South Africa and the European Union

Uncovering indicators of effective school management in South Africa using the National School Effectiveness Study

Stephen TaylorDepartment of Economics, Stellenbosch University

PSPPD Project – April 2011


2

Motivation (the problem)• Low quality education a poverty trap to

many children in historically disadvantaged schools

• Question: Poverty itself or the characteristics of schools in poor communities?• SACMEQ II and III:

Poor South African children performing worse than equally poor children in other African countries

• This despite substantial resource shifts to correct for apartheid inequalities• Historically disadvantaged schools have been

largely unresponsive to additional resources• Consequence: Perpetuation of a “2 systems”

system• How does the literature explain this?


3

Motivation (the literature)• Resources do not necessarily make a difference:

• the ability of schools to convert resources into outcomes is the crucial factor (Van der Berg, 2008)

• Socio-economic status (SES) has a dominant impact on the distribution of achievement

• Studies based on large sample surveys have typically struggled to identify specific aspects of effective management and teaching practice that explain performance.• Crouch and Mabogoane (1998): 50% of variance

explained by “management efficiency”• Van der Berg and Burger (2002): 2/3 variance explained

by SES, racial composition & school resources; remainder probably due to unobserved “management efficiency”.

• Largely due to data limitations most large surveys


4

Data• National School Effectiveness Study

(NSES)• JET Education Services & RNE

• Literacy and numeracy testing:• Grade 3 (2007)• Grade 4 (2008)• Grade 5 (2009)

• Principal questionnaires (2007, 2008, 2009)

• Teacher instruments (2008, 2009)• Teacher comprehension and maths test• Extensive review of learner workbooks

• Greater potential to uncover indicators of effective management an teaching

same individuals


5

Results – overall scores

Literacy Numeracy

2007 (grade 3) 20.15 29.38

2008 (grade 4) 29.59 35.50

2009 (grade 5) 37.73 47.04

Gain 2007 - 2008 9.43 6.12

Gain 2008 - 2009 8.14 11.54

2-year gain 17.57 17.66


6

Results: Numeracy scores by province

020

4060

8010

0N

umer

acy

scor

e (p

erce

ntag

e)

EAS

TER

N C

APE

FREE

STA

TE

KW

AZU

LU-N

ATAL

LIM

PO

PO

MP

UM

ALA

NG

A

NO

RTH

WES

T

NO

RTH

ER

N C

APE

WE

STE

RN

CAP

E

Numeracy 2007 Numeracy 2009


7

Results: Literacy achievement by SES

0.0

1.0

2.0

3.0

4K

erne

l den

sity

0 20 40 60 80 100Literacy score 2009 (grade 5)

Quintile 1 Quintile 2Quintile 3 Quintile 4Quintile 5


8

Results: Literacy achievement by SES

1020

3040

5060

Lite

racy

sco

re (%

)

0 1 2 3 4SES (min = 0, std dev = 1)

Literacy 2007 (grade 3)Literacy 2008 (grade 4)Literacy 2009 (grade 5)


9

Results: Numeracy achievement by ex-department

0.0

1.0

2.0

3.0

4.0

5K

erne

l den

sity

0 20 40 60 80 100Numeracy score (%)

Numeracy grade 3 (DET) Numeracy grade 3 (HOA)Numeracy grade 4 (DET) Numeracy grade 4 (HOA)Numeracy grade 5 (DET) Numeracy grade 5 (HOA)


10

Results: Numeracy achievement of African language students by ex-department

0.00

5.01

.015

.02.02

5D

ensit

y

0 20 40 60 80 100Numeracy score 2008

Ex-DET/ Homelands schools Historically white schools


11

Results: Indicators of effective management and teaching

Ex-department Percentage > 25 topics Number of students DET (B) 26% 6306 HOR (C) 25% 849 HOD (I) 38% 86 HOA (W) 75% 591 Total 29% 7832

Percentage of students in schools where more than 25 maths topics were covered


12

Results: Indicators of effective management and teaching

Mean number of literacy exercises found in the “best” learner’s book

ex-department Mean number of exercises Number of students DET (B) 33.43 6478 HOR (C) 62.40 837 HOD (I) 72.44 102 HOA (W) 75.21 580 Total 39.58 7997


13

Results: Extended writing

No exe

rcise

s with

paragr

aphs

1 or 2

exer

cises w

ith par

agraph

s

3 to 9 ex

ercis

es with

para

graph

s

More th

an 10 exe

rcise

s with

para

graphs

Unspec

ified

0102030405060708090

10085

70

88

19 23

Num

ber o

f Eng

lish

clas

ses


14

Results: Maths teacher knowledge

Teacher score Number of students % Cumulative % Mean Numeracy

2008

0 210 2.12 2.12 37.27

1 2130 21.52 23.64 33.04

2 2774 28.02 51.66 33.50

3 2168 21.9 73.56 34.14

4 1408 14.22 87.79 34.77

5 1209 12.21 100 46.92

Total 9899 100 100 35.44

10 days 75 hours can be written as .... days .... hours


15

Results: Multivariate analysis

• Are teachers with better subject knowledge located in more affluent schools?

• And is it this affluence driving the association of student achievement with teacher knowledge?

• The need for multivariate analysis to disentangle this.

• After accounting for the influence of SES, what school and teacher characteristics are associated with student achievement?

• What distinguishes better and worse-performing schools within poor communities?


16

Results: Multivariate analysis

• 4 multivariate regression models estimated in the education production function tradition:• OLS regression predicting Literacy

achievement in grade 4• OLS regression predicting Numeracy

achievement in grade 4• 2 more sophisticated techniques to model

gain scores


17

Results: Multivariate analysisExplanatory variables Student characteristics Student SES 0.39* (0.18) Male -2.48*** (0.26) Young -0.40 (0.46) Old -2.84*** (0.33) Household size: large -1.89*** (0.37) Read 1 to 3 times a week 1.37** (0.44) Read more than 3 times 2.39*** (0.62) Books at home: 1 to 10 0.60 (0.39) Books at home > 10 1.17* (0.48) Home language English 8.42*** (1.52) Speak English 1-3 times 1.75*** (0.38) Speak English 4+ 1.86** (0.68) English on TV 1-3 times 0.85* (0.39) English on TV 4+ 3.35*** (0.44) School characteristics Mean School SES -9.13*** (1.77) Mean School SES squared 3.35*** (0.45) Pupil-teacher ratio -0.18** (0.07) Teacher absenteeism zero 1.93* (0.81) LTSM Inventory good 1.66* (0.80) Problems with students index -0.96* (0.43) Curriculum planned using year schedule 1.46~ (0.81) Teacher characteristics Full year learning programme 1.55~ (0.87) Constant 29.69*** (3.45) R-squared statistic 0.4591 N 10 860


18

Results: Multivariate analysis• Literacy grade 4 (2008)• Estimated effects of change in characteristics on the

literacy national average (Original sample mean = 26.57%)

Predicted new mean Gain Teacher absenteeism zero 27.84 1.27 LTSM Inventory good 27.36 0.79 Curriculum planned using year schedule 27.18 0.61 Full year learning programme 27.18 0.61 Combined effect of improved characteristics 29.85 3.29


19

Results: Multivariate analysis• Numeracy grade 4 (2008)• Estimated effects of change in characteristics on the

numeracy national average (Original sample mean = 34.21%)

Predicted new mean Gain Assessment record keeping 35.08 0.87 No timetable available 34.45 0.24 Teacher absenteeism zero 36.01 1.80 Maths teacher test score: 100% 36.38 2.17 Maths topics covered: 25 plus 37.20 3.00 Combined effect of improved characteristics 42.29 8.08


20

Results: Multivariate analysis:Modelling the literacy gain scores (Historically black schools only)Explanatory variables [A] Pooled gains step 2 [B] 2-year literacy gains Mean School SES 0.39 (0.35) 1.37* (0.63) Facilities index (2008) 0.14~ (0.08) 0.27~ (0.15) Monitoring through class visits 2.16* (0.90) No timetable available (2008) -2.72 (1.93) Principal absent -1.67** (0.65) -4.03*** (1.13) Teacher punctuality good 0.94~ (0.53) 3.03*** (0.91) More than 2 English mark records 1.44* (0.64) 3.76*** (1.13) Paragraph writing: none -1.72** (0.57) -4.12*** (1.01) Literacy exercises: more than 27 1.34* (0.55) 2.35* (0.96) Years teaching: 4 to 9 1.03 (1.87) Years teaching: 10 to 19 2.64 (1.61) Years teaching: 20 plus 3.83* (1.67) Time dummy (1st year) 0.40 (0.51) Constant -5.33*** 6.10** R-squared 0.1214 0.3976 N 390 195

~ p<0.10, * p<0.05, ** p<0.01, *** p<0.001 Standard errors in parentheses


21

Conclusions and Policy Implications

• Resource variables were not amongst the most important factors predicting achievement

• Several indicators of effective school management and teacher practice that are associated with student achievement have been identified• even within the large historically disadvantaged

section of the school system. • This is an advance on earlier analyses

• An organised learning environment:• curriculum planning for the full year, a functional

timetable, good-quality inventories for LTSM, low teacher absenteeism and up-to-date assessment records

• Extensive coverage of curriculum and exercises


22

Conclusions and Policy Implications• Policies should empower teachers to

cover curriculum and administer exercises:• At the top: clearly communicated curriculum

requirements• Also, textbooks and workbooks that make

worked examples easier for both teachers and students to implement.

• Command and control measures to enforce adherence to best practices?• Probably not…

• Explore ways to attract, train and support better principals, and to replace those at the head of dysfunctional schools.

uncovering indicators of effective school management in south africa using the national school...

Documents

republic of south africa

european union programme

poor children

poor south african children

poor communities

province6 programme

ses8 programme

management efficiencyvan