championing young people’s learning ypla strategic analysis & research team lat value added...

Championing Young People’s Learning

YPLA Strategic Analysis & Research team

LAT Value AddedKnowledge SharePart 2



The Methodology



Value Added Specification• It’s long and it’s got some complex equations• Producing a version with a bit more plain English in it

• Methodology independently reviewed by NfER• Underlying concepts are complex…

…but individual steps are not so bad


Multi Level Modelling: More detail• MLM applied at qualification & subject level (e.g. A level History)• Attempting to fit a polynomial equation• Complexity of equation based on number of records & insts• Where there are very few records things are grouped together at

SSA level


Multi Level Modelling: Polynomials

Quadratic (order 2)

A + Bx + Cx2

Cubic (order 3)

A + Bx + Cx2 + Dx3

Quartic (order 4)

A + Bx + Cx2 + Dx3 + Ex4

80 to 500 cases 500 to 5000 cases 5000+ cases


Multi Level Modelling: Polynomials

Quadratic (order 2)

A + Bx + Cx2

Cubic (order 3)

A + Bx + Cx2 + Dx3

Quartic (order 4)

A + Bx + Cx2 + Dx3 + Ex4

80 to 500 cases 500 to 5000 cases 5000+ cases

A, B, C, D and E are known as ‘gamma’ coefficients


Multi Level Modelling: Provider Lines


Multi Level Modelling: Provider LinesNational Line = 10 – 5x + 0.5x2 + 0.01x3

Provider A Line = 12 + 10x + 0.5x2 + 0.01x3

Provider B Line = 1000 – 3x + 0.5x2 + 0.01x3

Provider C Line = -400 + 10x + 0.5x2 + 0.01x3


Multi Level Modelling: Equations

111

0

oo

33

221 xxxy o

Error

2

2101

0120

1

0

:,0~

:,0~

ij

j

j

National Line

Provider Line Variation

Distribution Of Results(Variance)


Multi Level Modelling (S+ and R)• Cannot be done with SQL or version of SPSS we have• R & S+ are programmes that support MLM calculation• Data Service use S+ but has licence issues• YPLA getting open source programme R onto estate• Both use the same programming language

• Code used works in a similar way to SPSS


Multi Level Modelling Basic R Syntax

base<-read.table('C:\\DriveD\\LAT VA\\Q_111_S12330.dat',

header = TRUE, fill = TRUE)

library (lme4)

a<-lmer(POINTS~PRIORC+PRIOR2C+PRIOR3C+PRIOR4C

+(PRIORC+PRIOR2C|LAESTAB),data=base)

fixef(a)


The Calculation Process



Step 1: Data from FFT• Received in SPSS format• FFT can advise on issues with the data• Includes data fields on:• Provider code (UPIN & LAESTAB)• Qualification type (A09 and LAT VA qual codes)• Prior attainment and outcome attainment• Learner Names


Step 2: Check and sort data• Data service do a range of checks on the data• Drop any qualifications that are too small to include• Check all providers we expect to be included are in the data• Produce “centred variables”• They then split into lots of small files (one per qual & subject)


Centred Variables• Many equations include quartic term (i.e. x to the power 4)• For a prior attainment score of 58 this is a big number

58 x 58 x 58 x 58 = 11,316,496• To make the MLM calculation run quicker the variables are centred• This means smaller numbers are used

• Involves some fiddly calculations but just basic maths• If you hear reference to “Prior C”, “catalyst file” or “beta variables”

these are interim steps used in this centring process


Step 3: Apply MLM• Data is fed into S+ programme one file after another• This will fit the national line and give details on distribution of results• Gives an output as text file• These text files are then grouped together using a compiler routine


Step 4: Check solutions• The individual solutions are fed into a spreadsheet to check whether

they look reasonable• This spreadsheet is known as the “Batch LAT”• Original FfE version was very complex (Over 6000 lines of code)• For 2009/10 we will be using a simplified version (1000 lines of

code)

• General checking by eye• Some mathematical checks too (positive definite matrix)• Produces “decentered coefficients”


Step 5: Re-apply MLM• If the solution does work then apply a lower order equation

…or group data up to SSA level• Then re-check solution• End point of this is national lines for all of the qualifications


Step 6: Upload data to Online LAT• The national lines and individual provider data are uploaded into

the Online LAT• This undertakes the same calculations as the Batch LAT• It also generates reports for all providers• These are viewable through the provider gateway


Step 7: Extract result files• Bulk data files can be exported from the Online LAT• The “new style” reports will be generated using these files• Results files sent to OfSted to be used in their reports• Results files uploaded to SQL for use by YPLA

• Results for 2007/08 and 2008/09 are on MISVS001 in LSC_MI_DB_PUB (the filenames start LATVA_)


The ReadyReckoner


Ready Reckoner• Providers were keen to get an early view of LAT VA scores• Excel spreadsheet that allows them to model their own data• Due for release in early October• Uses data from LAT VA 08/09 amended data release

• First line support by Data Service’s Service Desk• Second line support by YPLA

Input Data

Output Data


Work scheduled for the next 6 months


Timescales

• Unamended run mid November• Updated LAT Handbook and communications at same time• Amended run planned for mid January

Conclusion


Average GCSE grade

ABCDE

A le

vel r

esul

t

A

B

C

D

ENatio

nal average achievement

Remember the basics

championing young people’s learning ypla strategic analysis & research team lat value added...

Documents