project team
DESCRIPTION
Higher Education Academy NTF Project: 2010/13 The Forgotten Year: Tackling the ‘Sophomore Slump’. Project team Clare Milsom, Martyn Stewart, Sue Thompson, Wayne Turnbull, Margaret Williams, Mantz Yorke , Elena Zaitseva. The Forgotten Year: is there a second year slump? Schedule - PowerPoint PPT PresentationTRANSCRIPT
Project teamClare Milsom, Martyn Stewart, Sue Thompson, Wayne Turnbull, Margaret Williams, Mantz Yorke, Elena Zaitseva
Higher Education Academy NTF Project: 2010/13
The Forgotten Year: Tackling the ‘Sophomore Slump’
The Forgotten Year: is there a second year slump?
Schedule
1200 – 1220 Lunch1220 – 1315 Welcome and Overview 1315 – 1400 Discussion in groups1400 – 1430 Moving forward
Our vision is for students in UK higher education to enjoy the highest quality learning experience in the world
Our vision is for students in UK higher education to enjoy the highest quality learning experience in the world
Our vision is for students in UK higher education to enjoy the highest quality learning experience in the world
UMF Repositioning: evidence - student record data
Undergraduate performance (2008) on 24 credit modules across the institution (7131 modules)
Work with Ruth Ogden
% Students achieving ‘good honours’
Other findings:
Levels 1 and 2: students perform significantly higher on Semester 1 modules
Level 3: students perform best on 36 credit modules (usually dissertation modules) and least well on 12 credit modules.
Level 1 Level 2 Level 320
30
40
50
Level 2 performance dip
‘Sophomore slump’ coined by Freedman (1956)
Extends beyond education to a second effort not living up to former expectations:
“second season syndrome”.......
Difficult second album syndrome
Period of developmental confusion and uncertainty?
Characteristics of the Second Year Experience
Strengthened programme focus
Low(er) academic self efficacy (Stuart Hunter et al. 2010)
Less informal social integration and involvement (Foubert and Grainger 2006)
Increased absenteeism (Wilder 1993) – Faculty of Science
Intense period of personal development
NSS 2010‘Big increase in the amount of work between years 2 and 3 so not as prepared as possible.’
‘Took me nearly half my time into second year before I realised I need to adjust my style of writing.’
‘Lack of information on how course works will be assessed, especially at Level 2’
‘ Some of the lab work, in particular in 2nd year is not overly relevant’
Project aims:
1. Characterise the dip: pervasive or local. Discipline effect?
2. Investigate causes and develop strategies for enhancing Y2 experience.
3. Develop a model for analysing institutional datasets worthy of transfer.
Year-on-year increase
Second-year slump
Second-year peak
Year-on-year decrease
Mean grade 22 (44.9%)No. ‘good’ degrees 23
Mean grade 6 (12.2%)No. ‘good’ degrees 6
Mean grade 1 (2.0%)No. ‘good’ degrees 1
Mean grade 20 (40.8%)No. ‘good’ degrees 19
L1 L2 L3
L1 L2 L3
L1 L2 L3
L1 L2 L3
No. Good degrees most reliable indicator
Year-on-year increase (n=22) 59% (13) SA, 27% (6) SP, 9% (2) HP, 5% (1) HA
Second-year slump (n=20) 55% (11) SA, 25% (5) SP, 15% (3) HA, 5% (1) HP
Second-year peak (n=6) 50% (3) SP, 33% (2) HA, 17% (1) SA,
Year-on-year decrease (n=1) 1 HA
‘Soft Applied’ (applied social sciences, health) 24 49% SA
‘Soft Pure’ (humanities and pure social sciences) 15 31% SP
‘Hard Applied’ (e.g. technology & engineering) 7 14% HA
‘Hard Pure’ (e.g.physical & natural sciences) 3 6% HP
L1
Two patterns dominate.
In both cases L3 marks tend to be significantly higher even after L3 formula removed.
Yearly increase in performanceCharacterised by normal distribution of grades across levels (77% of cases)
Second-year slumpGreater tendency for skewed mark distributions, particularly at L1 & L3
L1 L2 L3
L1 L2 L3
Indicative second year slump pattern tends to be:L1 & L3 = negative skew (weighted in upper grades)
L2 = positive skew (weighted in lower grades)
Signals an issue with marking practices?
Percentage of good degrees awardedL1 L2 L3
62% -5 57% +5 62%66% -14 52% +7 59%48% -9 39% +31 70%41% -2 39% +10 49%38% -3 35% +10 45%52% -19 33% +45 78%33% -5 28% +8 36%43% -16 27% +8 35%30% -6 24% +34 58%39% -15 24% +25 49%25% -3 22% +28 50%45% -27 18% +23 41%19% -2 17% +41 58%47% -32 15% +6 21%14% -3 11% +14 25%28% -18 10% +35 45%30% -20 10% +47 57%21% -14 7% +22 29%6% -1 5% +22 27%
An analysis of selected Level 1 modules
Mantz Yorke
23 February 2011
NOTE
This analysis is limited to Level 1 modulesin which there were 30 or more results from
students who took the module only oncein Academic Year 2008-09
It is the first phase of a sequence that is intended to span Levels 1 – 3,
and will include all first-time attempts.This should eliminate the upward bias
in the present results
Owning Org Unit
N Modules
Overall Mod Mean
Max Mod Mean
Min Mod Mean
CMP 5 60.3 65.5 58.2LSS 31 57.0 64.8 48.0ENR 8 57.0 68.5 49.4PBS 19 56.2 70.1 48.5NSP 29 55.7 67.2 45.4ECL 75 55.3 65.3 45.6BUE 16 55.3 65.6 47.1LSA 26 55.1 63.7 48.2SPS 11 55.1 60.7 48.5HSS 37 53.5 64.0 45.0HEA 16 53.1 62.8 40.6LBS 34 52.7 70.3 43.9LAW 10 49.2 54.5 42.0[ALL] 317 54.9
Module means from 13 Owning Organisational Units
So why the variation?
Lots of variables may have exerted influence, including:
• Student calibre (entry qualifications; commitment)• Nature of the subject (hard/soft; pure/applied)• Curriculum design• Pedagogic quality• Resourcing• Expected standards (intended learning outcomes)• Mode of assessment• Nature of the assessment demand• Marker variability (in some cases, due to differences in School)
Disentangling the effects of these is very difficult!
So why the variation?
Lots of variables may have exerted influence, including:
• Student calibre (entry qualifications; commitment)• Nature of the subject (hard/soft; pure/applied)• Curriculum design• Pedagogic quality• Resourcing• Expected standards (intended learning outcomes)• Mode of assessment• Nature of the assessment demand• Marker variability (in some cases, due to differences in School)
Nature of the subject: hard/soft; pure/applied
Judgements regarding the categorisation of modules are rough and ready
24 Standard Deviation
20
16
12
8
4
40 45 50 55 60 65 70 75
Hard Pure N=38Hard Applied N=29Soft Pure N=104Soft Applied N=146
Module Mean
Mode of assessment
In-class tests are treated as exams
65+
60-64.99
55-59.99
50-54.99
<50
Mean CW = <35% CW = 35-75% CW = >75%
BUE HSS LBS LSA LSS NSP
Civil engineering surveying 1, CW=30%
Nature of the assessment demand
What about equivalence in assessment demand?
12 credit modules at level 1 specify, for example,
• CW 100% (proj docs 60%; oral pres 25%; use e-portfolio for PDP exercise 15%)• CW 100% (essay 1500w)• CW 100% (portfolio 3000w)• CW 100% (group annotated bibliography 20%; portfolio 3000w 80%)• CW 100% (portfolio of resources based on workshop, 2000w equiv)• CW 100% (info retrieval exercise 20%; PPT pres 40%; group pres 40%)• CW 100% (discuss 3 poems 30%; 40-50 lines + comment 50%; wkshp part 20%)• CW 100% (2 phase tests @25%ea; practical report 25%; fieldwork report 25%)• CW 50% Ex 50% (CW = seminar contrib’n 10%; essay 1200w 40%; Ex = 1hr)• CW 50% Ex 50% (CW = seminar presentation; Ex = 1hr)• CW 50% E 50% (CW = 1500-2000w report and presentation; E=1hr unseen)• Ex 100% (Ex = seen exam 2hrs)• Ex 100% (Ex = 1 question based on a case study 1.5hrs)• Ex 100% (Ex = 2hrs)• Ex 100% (Ex = 1.5hrs)
Causes of slump – qualitative research
Enriched insights into the causes and finely grained understanding of interplay between agency and structure
More informed recommendations
Main sources of data : Level 2 and 3 student focus groups ; longitudinal qualitative enquiry; staff interviews
Additional data sources: Mock NSS (L2) – qualitative comments; other University-wide cross sectional surveys; ad-hoc events and observations
Two strands: Curriculum and Student Experience
staff students
Leximancer (semantic content analysis) tool
Outlines main themes
Identifies key concepts and how
they are connected
Explores likelihood of a concept being
mentioned in favourable or unfavourable
context
Sentiment analysis
Attendance data analysis:
Semester Attended Absent Classes %
1 23,309 8,009 483 74.4
2 9,345 4,527 200 67.4
• Throughout this presentation, average % attendances are calculated using total attended and total absent data: Not by simply averaging over individual % class attendances.
How to get the most out of Reggie: An analysis of attendance data since Sept 2009.Phil Denton Faculty of Science
% Attendance vs. Level
• A contribution to the ‘second year slump’?
How to get the most out of Reggie: An analysis of attendance data since Sept 2009.Phil Denton Faculty of Science
1315 – 1400 Discussion in groups
What might be institutional implications?What might be implications for curriculum and assessment?What might be implications for student experience?
One person from the project team will make notes