Schools, fundingand performance:lessons from NSW

NP

Overview

2

Today’s discussion will look at:

1. Recent changes in school funding

2. Evidence of impact of funding

3. Evidence from evaluations of NSW low SES National Partnerships

4. Conditions for ensuring success

Recent changes inschool funding

Former funding arrangements

Resources allocated to schools through three mechanisms: 1. A school staffing allocation using an enrollment-based

multiplier adjusted for type of school, stage of learning and school size

2. Base and equity-adjusted recurrent operationalfunding allocations

3. Special purpose payments

• All children have access to a level of resources which enables them to achieve a high standard of learning

• Shift from providing inputs according to a staffing formula tolooking at the resources needed to have good outcomes

• Resources to promote school improvement• Important to address differences in learning needs• Focus on fairness and effectiveness (quality and equity) • Transparent • Provide flexibility• Promote accountability

Features of new funding models

• Core funding provided to all schools via a per-student price

• Price adjusted for stages of schooling

• Base allocation to assist smaller schools(tapered by school size)

• Core price adjustments for equity(to address additional student needs)

• Additional lines of funding ‘bundled’ to provide maximum flexibility, built into per-student price and stage weights (components in core).

Needs-based funding model

• What student and school needs should we fund?

• How much should we fund?

• What evidence is there money mattersto improving student outcomes?

Emerging issues for the new model

1. Developmentally ready at pointof entry to school

2. Meets or exceeds achievement benchmark inacademic skills at Year 7

3. Attains a Year 12 or equivalent certificate by Age 19

4. Engaged fully in education, training or work at Age 24.

Key milestones of opportunity

Student and school challenges

Student and school challenges

Early years Developmentally on track in all AEDC domains on entry to school

78.0% 241,185 entrants

22.0% 68,026 entrants

Middle yearsMeets or exceeds achievement benchmark in academic skills at

Year 7

71.6% 198,780 students

28.4% 78,853 students

Senior school yearsAttains a Year 12 certificate or

equivalent by Age 1974.0%

231,106 learners

26.0% 81,199 learners

Early adulthoodEngaged fully in employment,

education or training at Age 2473.5%

258,746 young adults

26.5% 93,289 young adults

58.2% 15.8% 15.8% 10.2%

54.8% 15.6% 16.8% 12.8%

60.0% 11.6% 18.7% 9.7%

Australia

MalesFemales

New South WalesVictoria

QueenslandSouth Australia

Western AustraliaTasmania

Northern TerritoryAustralian Capital Territory

Major CitiesInner RegionalOuter Regional

RemoteVery remote

Low23456789

High

EnglishLBOTE

Northern EuropeanSouthern European

Eastern EuropeanSouthwest and Central Asian

Southern AsianSoutheast Asian

Eastern AsianAustralian Indigenous

Other

Non-IndigenousIndigenousAboriginal

Torres Strait IslanderBoth

20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0 100.0

74

69.578.5

72.995100015885276.717481987106676.413330400352

70.99787900212170.9013251560618

59.542640990371447.1561113352158

84.7850678733032

78.22785334673263.9126855210164

61.62032448223756.3585171281089

43.4037692747002

60.605732242509661.3558955506298

64.653037719550865.8995371746423

69.504554543434871.8286111019275

75.656994744042178.6948380030385

81.795340961848389.1374648783332

71.970296568741683.3634642788779

87.769784172661883.4690399545541

87.715765247410874.4438415626696

93.145057766367185.5889876964307

89.643362370635125.1923076923077

71.7178314826911

75.207006750133543.5617860851506

41.942078364565662.6582278481013

58.4045584045584

Gender

State/Territory

Location

SES deciles (Low to High)

Language back-ground

Indigenous status

Year 12 completion

Year 3 Year 9

Within school

Between school

Within school

Between school 82.0 18.0 60.1 39.9 Student Characteristics Socioeconomic

Status 7.7 55.2 6.4 27.2

Female 0.1 0.0 0.5 0.0 ATSI 0.4 2.5 0.6 2.1 LBOTE 0.1 0.1 0.4 4.6

School characteristics

Non-city location 1.6 3.0 School SES 16.2 29.7 ATSI/LBOTE 0.5 7.5 Size of school 0.4 0.0 Selective entry 10.8 Total Variance

explained 8.2 76.4

8.0 84.9

School Effects: Numeracy Year 3, Year 9

Australia New South Wales

Early years Developmentally on track in all AEDC domains on entry to school 22.0% 19.9%

Middle years Meets or exceeds achievement benchmark in academic skills at Year 7 28.4% 28.3%

Senior school years Attains a Year 12 certificate or equivalent by Age 19 26.0% 27.0%

Early adulthood Engaged fully in employment, education or training at Age 24 26.5% 26.4%

Young people missing out

• Task: neutralise the effects of povertyand disadvantage

• 5 billion dollars?

• Based on the numbers missing out = $6.1 billion

• Based on costs of effective interventions = ?

How much funding?

Evidence on impact

• Some researchers (such as Eric Hanushek, Coleman)argue that there is no systematic connection between funding level and student outcomes

• Research on the topic can be confounding- Murnane and Levy.

Does additional money make any difference?

Murnane and Levy conducted a study of 15 low performing schools in Texas serving low income, minority children. Schools were givenan additional $300,000 annually for five years to address low achievement.

Four years later, 13 of these schools still showed poor student attendance and achievement. Yet two of the schools had raised test scores to the state average and had some of the best attendance records in Texas.

The 13 schools invested in smaller class sizes and extra teachers,but did not transform practice or programs

The two schools also lowered class sizes, but also moved special education students into mainstream classes, changed curriculum, brought in health services and used a large portion of their funds to get parents involved.

Murnane and Levy

Does additional money make any difference?

• Some researchers (such as Eric Hanushek, Coleman)argue that there is no systematic connection between funding level and student outcomes

• Research on the topic can be confounding- Murnane and Levy

• System level versus school level.

Review of 24 large-scale studies reporting effect sizes suggeststhat at a system level:

• aggregate measures of per pupil spending are positively associatedwith improved or higher student outcomes

• a positive relationship between funding and outcomes ismore likely to hold:

• For disadvantaged cohorts (primarily low SES students).

• Where schools have greater (and appropriately supported)autonomy over in-school expenditure.

• For mathematics and science subjects (across all cohorts, relative to English/literacy/reading)

• Funding has impact if it is sustained and there is certainty.

Review of evaluation studies

Paper Jurisdiction Key findings Effect size Key variables, conditions or context

Heinesen and Graversen (2005), The effect of school resources on educational attainment: evidence from Denmark.

Denmark • Expenditure per pupil has a small but significant positive effect on educational attainment

• Larger effect for students from disadvantaged backgrounds

Increase in expenditure per pupil by 1 percent increases the probability of obtaining an additional year of education by between 0.097 and 0.112 percentage points

Greenwald, Hedges and Laine (1996), The effect of school resources on student achievement

US, meta-analysis

• Moderate increases in spending may be associated with significant increases in achievement

$1000 increase in per pupil expenditure associated with 0.14 standard deviation increase in student outcomes

Holmlund, McNally and Viarengo (2010), Does money matter for schools?

UK • Expenditure has small positive effect on outcomes at end of primary school

• Higher effects for low SES students

£1000 increase in funding per pupil • English: 0.035 standard

deviations• Maths: 0.029 standard

deviations• Science: 0.034 standard

deviations

Local Education Authority (LEA) schools• Central government

provides funds to LEAs who each use their own funding formulas to allocate money to schools

Cobb-Clark and Jha (2013), Educational achievement and the allocation of school resources .

Victoria, NAPLAN data

• Modest positive relationship between funding and outcomes

• Spending patterns are broadly efficient (ie. Additional expenditure would have similar impact regardless of how it was spent)

$1000 increase in funding per pupil• Grade 5 writing: 0.08

standard deviations• Grade 9 numeracy: 0.05

standard deviations

• Principals have control over school operations

Examples of evaluation studies

Evaluations of school effects and effective practices whichcost money, including class size reduction, welfare support,or case management, suggest that some strategies arepositively associated with student outcomes.

Across studies:

• Some reported effects are larger than others

• there is also variation by student population andschool contextual variables.

Review of evaluation studies: Effective practices

Raising achievement strategies

Dropout prevention strategies

Evidence from NSW

1. Incentives to attract high-performing teachers and principals

2. Adoption of best practice performance management and staffing arrangements that articulate a clear role for principals

3. School operational arrangements that encourage innovationand flexibility

4. Provide innovative and tailored learning opportunities

5. Strengthen school accountability

6. External partnerships with parents, other schools, businesses and communities and the provision of access.

Low SES NP evaluation: Reform areas

Key initiativesReform Area 1: Attract high performance teachers • Establish leadership and strategic positions• Provide incentives to attract high performing

early career teachers• Provide mentoring support to teachers• Provide attractive terms/conditions outside

standard entitlements

Reform Areas 2-3: Staffing & performancemanagement Implement staff performance review procedures• Support early career teachers professionally• Manage the staffing mix and succession planning • Implement professional learning plans for staff• Employ additional paraprofessionals in learning

support roles • Employ additional paraprofessionals in other

support roles

Reform Area 4: Student Learning opportunity• Use assessment and other data to identify

student needs

• Implement targeted approaches to improveoutcomes of students with identified needs

• Implement differentiated teaching methodsto better meet the needs of all students

Reform Area 5: Strengthening school accountability • Use the school plan to drive change to improve

student, teacher and school performance • Use evidence from a range of sources to inform

decision making • Make planning and/or reporting processes

within the school more publicly available

Reform Area 6: External partnerships • Parents and carers generally • Aboriginal and Torres Strait Islander parents and

communities• Employers• Wider community

• A key question for the evaluation was whether or not participation in the Low SES NP has helped improve the education and life opportunities of students from low SES backgrounds through improvements in student outcomes.

• A range of analyses were undertaken using available data across a range of measures including NAPLAN results

• Principal and teacher surveys

• Case studies.

School views impact

9 in 10 of principals who participated in the 2014 survey reported that as a result of the NP their school had been able to:

• better develop their staff capacity,

• develop operational arrangements that encouraged innovationand flexibility in staffing,

• support improved student learning outcomes, and

• build teachers’ and students’ access to more innovativeand tailored learning opportunities

• Improve school culture.

But the acid test is in looking at whether or not it led to improvedstudent outcomes

Results suggest that Low SES NP has had a significant positive effect onstudent NAPLAN achievement.

• Reading scores: 1.98**

• Spelling scores: 1.79**

• Grammar: 3.64**

• Numeracy scores: 2.94**

Trend analysis

Low SES NP was estimated to have a significant positive impact on average student scores with each additional year in place. So, over a period of four years student scores may be expected to increase by four points on average, holding all other student characteristics constant.

Duration of LSNP Reading scores 1.41** Spelling scores 0.88** Grammar 0.37* Numeracy scores 0.54**

Trend analysis

Variance analysis using NAPLAN: Numeracy, Year 3

2009 2013

Within school

Between school Estimate Within

school Between

school Estimate

Constant 82.7 17.3 396.0** 80.7 19.3 391.8**

Student Characteristics

Socioeconomic Status 6.2 52.0 22.9** 7.7 55.2 22.3**

Female 0.1 -0.1 -2.5** 0.1 0.0 -1.9**

ATSI 0.4 2.9 -5.8** 0.4 2.5 -4.3**

LBOTE 0.0 0.0 0.9 0.1 0.1 2.2**

Student Characteristics

Non-city location 0.3 0.3 1.6 -0.9

School SES 14.4 11.3** 16.2 13.9**

LBOTE (%) 0.6 -2.5* 0.4 -2.5**

ATSI (%) 1.7 -2.5** 0.0 -1.3*

Disabilities (%) 1.0 -0.1 0.1 0.6

Females (%) 0.1 3.0 0.0 -0.8

Size of school 0.0 -0.6 0.4 2.0**

Low SES NP 0.0 0.8 0.6 2.1**

Total Variance explained 6.7 72.8 8.2 77.0

2009 2013

Within school

Between school

Estimate

Within school

Between school Estimate

Constant 79.4 20.6 493.4** 79.2 20.8 483.3**

Student Characteristics

Socioeconomic Status 4.1 40.9 22.8** 7.1 49.6 24.1**

Female 0.7 0.1 -5.8** 0.6 0.1 -5.7**

ATSI 0.5 2.6 -5.3** 0.4 2.5 -4.4**

LBOTE 0.4 2.3 5.1** 0.3 2.1 5.5**

School characteristics

Non-city location 0.6 1.4 1.8 0.0

School SES 16.9 15.6** 15.4 16.7**

LBOTE (%) 0.3 0.0 0.4 0.4

ATSI (%) 2.5 -0.9 0.0 -0.9

Disabilities (%) 0.8 1.0 0.3 0.7

Females (%) 0.4 0.5 0.1 -2.1

Size of school 0.0 2.5** 0.7 3.9**

Low SES NP 0.1 1.9* 0.4 2.6**

Total Variance explained 5.7 67.6 8.4 73.2

Variance analysis using NAPLAN: Numeracy, Year 5

Analysis of students in Low SES NP participating schools who remain at or above benchmark standards in NAPLAN across testing points (Year 3 to Year 5, and Year 7 to Year 9). The results are compared to students in schools not participating in Low SES NP.

Measures Description

Dependant variable

Student Outcome

NAPLAN Results are on track

For Year 5 and Year 9 cohorts in Reading and Numeracy respectively

Explanatory variables

Student factors

Prior performance Whether NAPLAN performance on track at Year 3 for Year 5 cohort and at Year 5 for Year 9 cohorts

Family SES Derived based on parental occupation and education levels

ATSI status

LBOTE status Students with Language Background Other Than English (LBOTE)

Gender

School factors

School SES Median of student SES indexes within the school.

ATSI concentration % of students being ATSI LBOTE concentration % of LBOTE students

School Sector Government, Independent or Catholic sector

Size of school Total enrolment School Location ARIA groups

Low SES NP schools Schools participating in Low SES National Partnership programs

NAPLAN analysis based on benchmarks

NAPLAN analysis based on benchmarks

At YR3 At YR5 At YR3 At YR5 At YR7 At YR9 At YR7 At YR9Reading Numeracy Reading Numeracy

YR3-5 match cohort YR7-9 match cohort

79% 79% 76%67% 69% 65% 66%

59%

21% 21% 24% 33% 31% 35% 34% 41%At or Above Bench Mark Below Bench Mark

Year 5 Outcomes Year 9 Outcomes

Explanatory variables Reading Numeracy Reading Numeracy

Student factor

Prior performance 12.50 (+*) 12.50 (+*) 21.30 (+ *) 34.50 (+*)

Family SES 1.02 (+*) 1.02 (+*) 1.02 (+ *) 1.02 (+*)

ATSI status 0.72 (-*) 0.58 (-*) 0.69 (- *) 0.53 (-*)

Gender (Male) 0.80 ( -*) 1.30 (+*) 0.82 (- *) 1.33 (+*)

School factor

School SES 1.02 (+*) 1.02 (+*) 1.04(+*) 1.04 (+*)

School sector

School size

School Location

All Low SES NP schools (VS Non-NP schools) 1.16 (+*) 1.14 (+**) 1.41 (+*)

2011 Low SES NP schools (VS 2011 Non-NP schools) 1.14 (+**) 1.45 (+*) 1.53 (+*) 1.80 (+*)

NAPLAN analysis based on benchmarks

Results of analyses using a repeated measures design for a range of governmentschool performance measures other than NAPLAN revealed that on four measures

• HSC attainment, • HSC achievement, • ATAR scores, and • Attendance

schools participating in Low SES NP had incremental rises in performance relativeto non-NP schools.

Within Low SES NP schools, those commencing in 2009 display relative improvement over time in HSC attainment and apparent retention (both Year 7 to 12 and Year 10to 12). This may reflect an impact of participation of this group of schools (which had been in the program for the longest period) on retaining students in school.

Analysis based on other outcomes

Ensuring success

Aggregate effects conceal marked differences across schoolsin terms of impact.

Based on results we identified three types of schools:

• Schools displaying strong improvement in NAPLAN results,

• Schools displaying positive, but modest improvement in NAPLAN results, and

• Schools displaying no improvement or little change.

Not all schools exhibit improvement

Low Medium High

Reading

Primary 76 202 69 Secondary 26 35 33 Combined 2 39 6

Numeracy Primary 61 164 121 Secondary 14 52 28 Combined 0 34 13

Categories of schools based on impact

Chi-square Probability

School SES-EOI index 1.25 0.74 Percent of LBOTE students 11.01 0.01* School Location 2.21 0.53 Attendance rate 5.83 0.12***

Staff profile Staff Mobility (turnover rate) 9.12 0.03*

Age profile Index 0.79 0.85 Male to female staff ratio 1.39 0.71 Length of current position 4.32 0.23

Program/funding type NP participation year 12.22 0.01*

Improving Teacher Quality programs 5.11 0.02* Literacy and Numeracy program (NP 2009) 1.08 0.3

Priority School Program (2010) 0.22 0.64

Not all schools exhibit improvement

Attracting high performing

teachers

Staffing & performance management

Improving student Learning

opportunity

Strengthening school

accountability External

partnerships

NAPLAN impact category Numeracy

High 1.6 7.6 4.8 3.5 3.3

Medium 1.9 7.7 4.3 3.2 2.9

Low 2.2 7.8 4.1 2.9 3.1

Reading

High 1.9 7.9 4.8 3.5 3.3

Medium 1.7 7.2 4.4 3.2 3.1

Low 2.2 8.5 4.2 3.2 2.7

Number of strategies

Comparing high and low impact schools

Estim Std Error Wald Prob

Attract high performance teachers

Provided mentoring support to teachers -5.56 1.80 9.51 0.00 * Provided attractive terms/conditions outside standard entitlements -3.79 1.79 4.47 0.03 *

Provided opportunities for professional learning and development 3.00 1.62 3.44 0.06 **

Staffing & performance management Introduced greater cooperation with other schools to share resources 2.34 1.35 3.01 0.08 **

Made more use of new technologies in teaching -2.91 1.60 3.33 0.07 ** Provided professional development on a range of behaviour management theories and approaches -2.57 1.59 2.64 0.11 ***

Student Learning opportunity Provided professional learning for teachers on meeting individual learning needs 1.31 0.77 2.88 0.09 **

Provided students with access to learning support services -2.84 1.37 4.28 0.04 *

Provided ILPs for students needing assistance 6.12 1.77 12.03 0.00 *

Strengthening school accountability Used evidence from a range of sources to inform decision making and/or strategic direction setting 2.96 1.72 2.94 0.09 **

Strategies of greatest impact

• They had made marked improvements in attendance rates• They had participated in the Improving Teacher Quality program• They had lower teacher turnover or mobility rates • They had placed greater emphasis than other schools on innovations in

teaching practice and addressing student needs • They had provided stronger welfare and learning support for disadvantaged

students through providing teachers with training around individual learning needs, strengthening the use of ILPs for students, and improving learner support services

• Stronger use of evidence to support decision-making and strategic planning • Stronger emphasis on developing staff through providing opportunities for

professional learning and development

High impact schools

• Some had a stronger emphasis on strategies to attract and retain teachers, particularly through leadership positions, which may itself reflect greater need in this area and schools struggling to recruit and retain quality teachers and leaders

• Less focus on use of evidence for strategic planning and decision-making • Tendency to spread their effort too broadly, rather than in a more focused and

strategic way• Less focus on innovation in teaching, addressing student needs, and

strengthening school accountability and more on staffing and performance management

• Pursued strategies around teacher and leadership recruitment, retention and development suggesting that staff stability was a major issue for them.

Low impact schools

Differences in focus and emphasis reflect the challenges that different schools face and to which they are responding, rather than the effects of the strategies themselves. For example, schools that recorded little change or impact of their reform efforts on student NAPLAN results and placed greater emphasis on attracting high performing teachers and improving the staffing and performance management role of the principal may have issues in staffing and management practices that need (and the schools recognise that need) to be addressed and that seriously undermine their performance. High impact schools, on the other hand, may have more stable staffing profiles (suggested in the results in Table 5.3) and it is their staffing and management stability that has enabled these schools to target areas of improvement in teaching practice and in addressing student needs that has delivered real traction from the NP initiatives they have implemented.

Differences in impact linked to context

• Organisational health of schools critical

• Funding needs to be sustained

• Sufficient resources to overcome the effects of educational disadvantage

• Schools have greater, and appropriately supported, autonomy, combined with accountability and capable school leadership

Conditions for funding impact