leaving cert maths and performance in universityteaching.sociology.ul.ie/bhalpin/mathssem.pdf ·...

Leaving Cert Maths and Performance inUniversity

Brendan HalpinDepartment of Sociology

December 1, 2010

What good is Maths?

I What do we want of it?I An important part of a rounded education?I A feed-stock for the knowledge economy

I Should we be worried about performance?

I Anxiety and resistance

Declining LC performance of UL entrants

35

40

45

50

55

60

1998 2000 2002 2004 2006 2008Year first observed

Maths Points Pct Higher

What do we want to know about it?

Key research question: Will encouraging LC HL maths improvethings in university?

I Do (where do) people who perform better at LC Mathsperform better in University?

I How much of this is due to prior ability, and how much is itamenable to policy intervention?

Maths bonus points as a policy lever

I Effective?I Will bonus points raise LC learning?I Will they raise student quality?

I Unfair?I Further penalise disadvantaged students?I Penalise female students?I Penalise the non-mathematical?

Nature of the bonus

0

20

40

60

80

100

120

140

D3 D2 D1 C3 C2 C1 B3 B2 B1 A2 A1

UL bonusOrdinary

HigherNew bonus

Nature of the bonus

I A flat 25 points for D3 plus, rather than 5-40 rising from C3to A1

I A good incentive for average students, but it doesn’t reallyreward excellence

I In so far as candidates respond to the incentive, the 25 pointswill be partly consumed by higher entry points

Structural disadvantage

I Different types of school have different provisionI Disadvantaged schools more likely not to offer higher level

mathsI Single-sex female schools more likely not to offer higher level

maths

I De-facto or innate gender differences in maths aptitude?

Gender and maths: How it Works . . .

Gender and maths: How it Works . . .

Credit: Randall Munroe; http://xkcd.com/385/

Leaving Cert Points and gender differences

40

45

50

55

60

65

70

75

80

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Female IrishMale Irish


40

45

50

55

60

65

70

75

80

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Female EnglishMale English


40

45

50

55

60

65

70

75

80

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

Female MathsMale Maths


40

45

50

55

60

65

70

75

80

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008

MAS gender 1MAS gender 2

Gender and maths

I LC performance is poor relative to boysI University performance differs by department:

I In MAS, females do systematically better (somewhat maledept)

I In MAE, females do better than males recently (very maledept)

I In PHY, a tie (somewhat male dept)I In ECO, a tie (close to balanced)I In CES, females do better than males recently (somethat male

dept)

I In other words, no evidence of a debilitating mathematicaldisability

Maths and gender, LC grades

0.0

2.0

4.0

6.0

8

F E D3D2D1C3C2C1B3 B2 B1 A2 A1 F E D3D2D1C3C2C1B3 B2 B1 A2 A1

Ordinary Honours

F M

Density

Analysis: the research question

I The specific question I now address is, how much extra doesLC maths performance help us predict third level performance?

I This addresses a part of the larger question, but since LCperformance is due in part to innate ability and in part to the2nd level educational experience, it is not clear that changesin the educational process at 2nd level will cause changes at3rd level

Data and method

I The data:I From the Student Records SystemI Ten years of undergraduate grade data: 1999/2000 to 2008/9I All Autumn and Spring XY4000 grades that are normally

graded (no pass/fail, no Co-op)I Information about age, gender, LC performance, module size

etc.I Thanks to the VPAR, Student Services, QSU and Research

Office

I The method: Multi-level regression analysis with grade QCVas the dependent variable

Multi-level modelling of QCV

I Why analyse the determinants of grade?

I Retention and final award level are much more significantoutcomes

I However, the grade is the “atom” of performance – linksdirectly to the the module, the department (rather thanprogramme) as well as to the individual

I Repeated measurement at individual and module/departmentlevel is extremely useful – assess individual and structuralvariability

Nested structure of the data

Dataset

G1 G2 G3 . . . Gn


Dataset

M1

G1 . . . Gk

M2

Gk+1 . . . Gl

M3

Gl+1 . . . Gn


Dataset

D1

M

G . . . G

M

G . . . G

D2

M

G . . . G

M

G . . . Gn


Dataset

F1

D1

M1

G1 G2

M2

G3 G4

D2

M3

G5 G6

M4

G7 G8

FN

D3

M5

G9 G10

M6

G11 G12

D4

M7

G13 G14

M8

G15 G16

Nesting as structure of teaching and learning

I Modules (within departments) are where the actual work ofteaching and learning and assessing goes on

I Module instances share much context: average level of ability,the specific curriculum, module size etc.

I Departments affect standards, resources, the identity of theteachers, etc.

I MLM that takes this nested structure into account dealscorrectly with higher level observed variables (e.g., modulesize)

I Also deals with higher-level unobserved heterogeneity – e.g.,the extent to modules differ from each other (and arerelatively homogeneous within) due to unobserved factors

Multiple observations per individual also

Dataset

F1

D1

M1

G1 G2 G3

M2

G4 G5

D2

M3

G6 G7 G8

M4

G9 G10 G11

F...

. . .

MLM structure at the individual level too

I Repeated measurement at the individual level allows us totake account of individual variables correctly (e.g., same LCpoints for 40 different grades)

I It also accounts for individual-level unobserved heterogeneity,e.g., socio-economic status, motivation, aptitudes, bar bill

I This yields a “cross-classified” multi-level model

I A very important side effect: since individuals crossdepartments we get much more meaningful estimates ofdepartmental effects

Excursus: A web of departments

I Comparing disparate departments is hard: how to compareanalysing a marketing plan, and 18th century French text, orthe vibration pattern of a car exhaust?

I Experimentally we could assign matched samples of studentsto different programmes and compare results

I Observationally, we “control for” age, gender, LC points butdepartment differences could still be accounted for byunobserved factors

Grade pairs

I But since we observe the same students in differentdepartments we can control for this too

I But to what extent is the whole university linked by students?

I We can assess this by looking at each grade and linking it tothe same and other departments by pairing it with thatstudent’s other grades

Grade Pairs – an example

��

��

��

��

~

LAW

PPALCS

LCS

Grade Pairs – an example

��

��

��

��

~

LAW

PPALCS

LCS

LCS LAW PPA

LCS 2 2 2LAW 2 0 1PPA 2 1 0

Web of grades

LC

S

LA

W

SO

C

PP

A

HIS

AC

F

MM

A

PE

R

EC

O

PH

I

PS

Y

NM

I

PE

S

EP

S

LS

C

MS

T

AR

C

CS

I

EC

E

MO

E

PH

Y

MA

E

CE

S

MA

S

AR

T

PL

E

PLE

ART

MAS

CES

MAE

PHY

MOE

ECE

CSI

ARC

MST

LSC

EPS

PES

NMI

PSY

PHI

ECO

PER

MMA

ACF

HIS

PPA

SOC

LAW

LCS

0

2000

4000

6000

8000

10000

12000

14000

Web of grades (rate)

LC

S

LA

W

SO

C

PP

A

HIS

AC

F

MM

A

PE

R

EC

O

PH

I

PS

Y

NM

I

PE

S

EP

S

LS

C

MS

T

AR

C

CS

I

EC

E

MO

E

PH

Y

MA

E

CE

S

MA

S

AR

T

PL

E

PLE

ART

MAS

CES

MAE

PHY

MOE

ECE

CSI

ARC

MST

LSC

EPS

PES

NMI

PSY

PHI

ECO

PER

MMA

ACF

HIS

PPA

SOC

LAW

LCS

0

0.05

0.1

0.15

0.2

0.25

Tightly coupled

I AHSS and KBS are extremely tightly linked

I SEN and EHS are tightly linked but with some structure

I MIC’s two departments are, naturally, detached

I Within UL, only ARC stands out as being isolated

Numbers in the analysis

I 728,590 grades in the working data set, with LC points data

I Where Irish, English and Maths results are known:

I Total number of grades: 614,747I Number of individuals: 22,125I Number of module instances: 10,947I Number of departments: 26

Predicting QCVBase model Plus core LC

subjectsPlus

gender×maths

Intercept 1.270***LC points/100 0.595***

Calendar year -0.009***Male 0.055***Male by year interaction -0.022***Age at entry -0.004Years since entry

1 0.113***2 0.244***3 0.384***4 0.562***5 0.718***

Spring semester -0.014*Module size/100 -0.004Mean module points/100 -0.300***Dept proportion female 0.174

Note: ***: p < 0.005; **: p < 0.01; *: p < 0.05


subjectsPlus

gender×maths

Intercept 1.270*** 1.348***LC points/100 0.595*** 0.536***Extra effect, Irish 0.003Extra effect, English -0.062*Extra effect, Maths 0.331***

Calendar year -0.009*** -0.006***Male 0.055*** 0.034***Male by year interaction -0.022*** -0.022***Age at entry -0.004 -0.002Years since entry

1 0.113*** 0.110***2 0.244*** 0.238***3 0.384*** 0.376***4 0.562*** 0.550***5 0.718*** 0.700***

Spring semester -0.014* -0.013*Module size/100 -0.004 -0.004Mean module points/100 -0.300*** -0.302***Dept proportion female 0.174 0.206

Note: ***: p < 0.005; **: p < 0.01; *: p < 0.05


subjectsPlus

gender×maths

Intercept 1.270*** 1.348*** 1.362***LC points/100 0.595*** 0.536*** 0.535***Extra effect, Irish 0.003 0.004Extra effect, English -0.062* -0.060*Extra effect, Maths 0.331*** 0.311***Maths/gender interaction 0.049Calendar year -0.009*** -0.006*** -0.006***Male 0.055*** 0.034*** 0.006Male by year interaction -0.022*** -0.022*** -0.022***Age at entry -0.004 -0.002 -0.002Years since entry

1 0.113*** 0.110*** 0.110***2 0.244*** 0.238*** 0.238***3 0.384*** 0.376*** 0.376***4 0.562*** 0.550*** 0.550***5 0.718*** 0.700*** 0.700***

Spring semester -0.014* -0.013* -0.013*Module size/100 -0.004 -0.004 -0.003Mean module points/100 -0.300*** -0.302*** -0.302***Dept proportion female 0.174 0.206 0.206

Note: ***: p < 0.005; **: p < 0.01; *: p < 0.05

Modelling by departmentNo random slope Departmental random

slope

Intercept 1.3478***LC points / 100 0.5362***Extra effect, Irish 0.0028Extra effect, English -0.0618*Extra effect, Maths 0.3305***Calendar year -0.0063***Male 0.0339***Male by year interaction -0.0218***Age at entry -0.0024Years since entry

1 0.1103***2 0.2383***3 0.3759***4 0.5499***5 0.7000***

Spring semester -0.0132*Module size / 100 -0.0035Mean module points / 100 -0.3021***Dept proportion female 0.2064

Note: ***: p < 0.005; **: p < 0.01; *: p < 0.05

Modelling by departmentNo random slope Departmental random

slope

Intercept 1.3478*** 1.6829***LC points / 100 0.5362*** 0.4941***Extra effect, Irish 0.0028 0.0235Extra effect, English -0.0618* -0.0401Extra effect, Maths 0.3305*** 0.3792***Calendar year -0.0063*** -0.0049**Male 0.0339*** 0.1635***Male by year interaction -0.0218*** -0.0217***Age at entry -0.0024 -0.0040Years since entry

1 0.1103*** 0.1127***2 0.2383*** 0.2399***3 0.3759*** 0.3705***4 0.5499*** 0.5382***5 0.7000*** 0.6772***

Spring semester -0.0132* -0.0156***Module size / 100 -0.0035 0.0052Mean module points / 100 -0.3021*** -0.3209***Dept proportion female 0.2064 0.0338

Note: ***: p < 0.005; **: p < 0.01; *: p < 0.05

Departmental difference in maths effect

-1.5

-1

-0.5

0

0.5

1

1.5

2Dept random effect plus maths fixed effect

99% confidence interval

ARCEPS

PSYPHI

NMIHIS

LAWSOC

MMAPPA

PESMST

PLEPER

LSCMOE

ARTLCS

CSIECE

ACFCES

ECOPHY

MAEMAS


-1.5

-1

-0.5

0

0.5

1

1.5



ARCEPS

PSYPHI

NMIHIS

LAWSOC

MMAPPA

PESMST

PLEPER

LSCMOE

ARTLCS

CSIECE

ACFCES

ECOPHY

MAEMAS

Protection against failure


-1.5

-1

-0.5

0

0.5

1

1.5



ARCEPS

PSYPHI

NMIHIS

LAWSOC

MMAPPA

PESMST

PLEPER

LSCMOE

ARTLCS

CSIECE

ACFCES

ECOPHY

MAEMAS

Protection against failurePromoting excellence

Conclusion

I Maths performance at LC is a very important predictor ofperformance in UL

I This varies by department, but in half the effect is positiveand sometimes very large, and in nearly all the rest it isneutral in effect

I That is, maths ability makes an important contribution mostof the time

I Whether policies that improve performance at 2nd level willhave consequences for 3rd level is another question!

leaving cert maths and performance in universityteaching.sociology.ul.ie/bhalpin/mathssem.pdf ·...

Documents