Methods and Approaches to investigate the UK Education System

Sandra McNally,University of Surrey and

Centre for Economic Performance, London School of Economics

Overview

• Some questions addressed in the CEP Education Programme

• Data• Core methodological issues

Some current themes in the CEP Education Research Programme

• What works in schools…• Teachers• Peers, neighbours and compositional effects• Parental preferences and admissions• Higher education in the UK

What works (or not) in schools: CEP research

Policy Sector Project Positive Little or no effect

Additional comments

Choice and competition

Primary Gibbons, Machin, Silva (2006, 2009)

√

Suggestion that small effect might be possible in schools with greater autonomy

Increasing overall school expenditure

Primary Holmlund, McNally,Viarengo (2009) Gibbons, McNally,Viarengo (2011)

√

Holmlund et al. find modest effects. Estimated effects (for urban schools) much larger in Gibbons et al.

Literacy and Numeracy Strategies

Primary

Machin and McNally (2008; 2010)

√

Moderate effects at very low cost.

Synthetic phonics

Primary Machin, McNally, Viarengo Work-in-progress….

Special Educational Needs

Primary Keslair, Maurin, McNally (2012) √ No effect (or even a negative effect)

Academies

Secondary Machin and Wilson (2009) Machin and Vernoit (2011) Machin and Silva (2013)

√

Machin and Wilson (2009) found no effect in short-term Machin and Vernoit (2011) find positive effects for schools that have been in programme for some time. But Machin and Silva (2013) find no effects in the tail of the distribution

Excellence in Cities

Secondary Machin, McNally and Meghir (2010) √ Moderate effects but cost-effective; Highest effect for most able pupils in most disadvantaged schools

Most important data sets

• The National Pupil Database (potentially linkable with UCAS and HE data)

• Longitudinal data sets which can be linked with administrative data, e.g. Longitudinal Survey of Young People in England; Millennium Cohort Study

Core methodological issues

1. What is an appropriate counter-factual? – and therefore can we establish causality?

2. For whom is the causal effect identified?

3. Can we look at longer-term effects?

4. Can we do a cost-benefit analysis?

5. Can we extrapolate outside the study context?

Approaches

• Randomised Control Trials

(Education Endowment Fund)• Difference-in-difference approaches• Instrumental Variable Approaches• Regression Discontinuity Approaches• OLS/Propensity Score Matching

Randomised Control Trials

• ‘Treatment’ is randomly assigned amongst target population

• An example:

Design of an ‘information campaign’ about the costs and benefits of staying-on in education targeted to Year 10 students in London schools (McGuigan, McNally, Wyness, 2012)

Difference-in-Difference Approaches

• Compare the outcomes of a treatment and control group before and after the policy.

• An example:

Structured way to teach reading (‘literacy hour’) was piloted in schools in some Local Authorities before it was rolled out to the rest of the UK (Machin and McNally, 2008)

Instrumental Variable Approaches

• Trying to find a variable that predicts the outcome variable only via the impact it has on the ‘treatment’ variable.

• An example

Using changes in the minimum school-leaving age to estimate the impact of additional education on future earnings (Harmon and Walker, 1995).

Regression Discontinuity Designs• Making use of a ‘cut-off’ for eligibility to a

programme/policy. Compare the outcomes of people either side of this ‘cut-off’.

• An example

Using administrative cut-offs in school admissions policies to identify the relationship between when children start school and their later outcomes. Crawford, Dearden and Greaves (2013).

OLS/Propensity Score matching

• Trying to measure the effect of a ‘treatment’ while controlling for all observable characteristics.

• Example

Looking at whether the increase in ‘non-native’ English speakers has an influence on the educational attainment of native English speakers (Geay, McNally and Telhaj, 2012)