presented by dr clare hourigan university planning & statistics, office of planning &...
TRANSCRIPT
Presented by Dr Clare Hourigan University Planning & Statistics, Office of Planning & Quality
The use of the general achievement test in student
selection at Monash University:
A preliminary evaluation
The Current Higher Ed Environment
• 2008 Review of Higher Education– Increase bachelor qualification attainment (40%
of 25-34 year olds by 2025)
– Increase participation of low SES students (20% of undergrad population by 2020).
• TEQSA– New round of quality audits are likely to focus on
student performance
Role of Admission Policies
• Admission policies will play important role in meeting these challenges.
• Aim is to grow and diversify without compromising on quality.– Important to continue to select students that
are likely to succeed
– Must aim for a student population diverse in social background but not academic ability.
Use of Year 12 results in selection
• Year 12 results (ENTER in Vic) current primary tool used for undergrad student selection.
• Best predictor of university performance that we have but there is room for improvement.- Monash internal analysis shows it can account for
35.4% of variance in first year marks but varies by course/field of education (Hyndman & Akram, 2006)
Use of Year 12 results in selection
- Reliance on Year 12 results may be part of the reason why low SES students are under-represented at university.- Year 12 completion rates lower (James, Anderson, Bexley,
Devlin, Garnett, Marginson & Maxwell, 2008)
- and ENTER scores are lower among low SES students (Birrell, Rapson, Dobson, Edwards & Smith, 2002)
Use of Year 12 results in selection
• Why do Low SES students obtain lower year 12 results?– Aspirations/encouragement from
peers/family/teachers (James, 2002)
– School resources (Edwards, Birrell & Smith, 2005)
Use of Year 12 results in selection
• Less prepared for university or do their marks under-reflect their ability/potential?– Study of Monash students found, when
ENTER was controlled for, students from non-selective Govt schools performed best (Dobson & Skuja, 2002)
• Challenge for universities is to compensate for disadvantage to the extent that it has resulted in Year 12 results that under-reflect likelihood of success at uni.
Current policies to compensate for disadvantage• Special Entry Access Scheme in Vic
– Bonus points given to applicants for different equity categories including socio-economic disadvantage.
– Systematic approach but qualitative in judgement of extent to which Year 12 results have been affected.
Current policies to compensate for disadvantage
• Under-represented schools– Assumes all students from particular
schools were disadvantaged in the same way.
- 2004 Admissions to Higher Education Review UK- Recommended aptitude tests be considered
by universities in student selection to ‘help uncover hidden talent’.
Aptitude Tests
• UK recommendation based on pilot of Scholastic Assessment Test (SAT).• A levels and SAT both influenced by SES (high SES
students do better) but measure different aspects of academic ability because some students do well on one but not the other.
• Students from schools with lower A levels least likely to score well on both tests (21% compared to 44%).
• If uni selection used both A levels and SAT results, uni offer rates for students from low attaining schools would double.
Aptitude Tests
• Monash started considering using admission tests to enhance selection.– In 2007 trialed the UniTest at Berwick
Campus but can be administratively costly and places extra burden on students.
– GAT examined because it’s undertaken by all VCE students
Aptitude Tests
• Monash undertook analysis into predictive validity of the GAT on 1st year marks.– ENTER is best predictor (35.4% of the
variance)
– GAT also strong predictor (32.3% of the variance)
– Using both GAT and ENTER can improve predictability by 3% and predictive nature of GAT is stronger at lower ENTER ranges.
General Achievement Test
- In 2008, Monash commenced 3 year pilot to use the GAT in middle band selection of VCE applicants with the aim to:- Differentiate between middle band applicants to
select those most likely to succeed
- Identify and select students likely to succeed at university but had underperformed in year 12 due to personal circumstances or educational disadvantage
Use of GAT in selection at Monash
- In 2009, GAT predicted ENTERS (GPENTERS) were used in middle band selection.- Middle Band: Approx 20% of places set aside for
assessment on range of extra criteria- Performance in selected VCE subjects- SEAS bonuses
- Bonus aggregate points (2 to 9 points) where applicants GPENTER > ENTER
Use of GAT in selection at Monash
GAT Preliminary Evaluation
• Based on students who commenced in 2009 as selection method changed.
• Did use of the GAT assist in:– selecting applicants likely to succeed who
otherwise would not have been selected?
– diversifying Monash’s commencing student cohort with regards to socio-economic status?
Performance of GAT offer students
Type of Offer Av. Mark
N Sig. Progress Rate
N Sig
GAT based offer (A) 60.4 118 82.9 121
Standard – app above clearly in (B) 67.1 2733 A C 91.6 2754 A C
Middle band – app below clearly in (C) 61.9 424 87.2 428
Range of Critiera/no clearly-in (D) 69.8 548 A B C
95.8 551 A B C
Total 66.7 3823 91.4 3854
First semester average marks and progress rates among domestic VCE school leaver students commencing a bachelor’s pass degree at Monash University by offer type, 2009
Can GAT mediate for effect of SES on ENTER?• Examined relationship between SES and
ENTER and SES and GPENTER
Can GAT mediate for effect of SES on ENTER?• Perhaps differences in ENTER/GPENTER due
to true differences in potential to succeed.– If true then uni performance should not differ by
SES status when ENTER/GPENTER is controlled for.
• Used linear regression model to examine relationship between ENTER/GPENTER and SES status and uni performance (first semester marks).
Can GAT mediate for effect of SES on ENTER?
• R Square = 0.21, Standard Error = 11.32• Extent to which year 12 results under-reflect
uni performance similar for low and medium SES students
• Perhaps Year 12 results of high SES students are inflated
ENTER MODEL: Average Mark = 0.96 + 0.74xENTER + 2.28xLow SES + 1.95xMedium SES + e1 (2.1) (0.02) (0.65) (0.41)
Can GAT mediate for effect of SES on ENTER?
• When GPENTER used: variation across SES status was not significant.
• GPENTER was not particularly strong predictor of uni peformance but less likely to under-reflect likelihood of success of low and medium SES students
• R Square = 0.08, Standard Error = 12.18
GPENTER MODEL: Average Mark = 41.1+ 0.31xGPENTER + 0.82xLow SES + 0.41xMedium SES + e1 (1.41) (0.02) (0.7) (0.44)
Did the GAT help Monash select more low SES students?
• 20,830 VCE school leavers applied to Monash in 2009
• Examined eligibility for GAT bonus• Middle band offer rates by SES status
Did the GAT help Monash select more low SES students?
• Overall, low (and medium) SES students more likely to be qualify for a GAT bonus
• However, many of the low SES applicants did not meet the eligibility score ruling.
• Among applicants above eligibility score, qualification for GAT bonus almost same for low and high SES applicants
Did the GAT help Monash select more low SES students?
• GAT offer rates among middle band applicants were:– Low: 2%
– Medium: 2.3%
– High: 1.6%
• GAT did not assist Monash in diversifying the student population with regards to low SES status predominately due to eligibility score ruling.
Conclusion• Findings are preliminary and we will be
monitoring this as the pilot progresses.• Results are promising
– 163 students who would not otherwise have been selected were given an offer
– These students performed as well as other middle band students.
Conclusion• The use of the GAT did not help diversify
Monash’s student population with regards to SES.
• This was predominantly due to the eligibility score ruling. Many of the low SES applicants did not meet the eligibility score.– Results may differ for universities with wider
range of eligibility scores