Exploring student non-completion in higher education using electronic footprint analysis

Dr John Buglear

Nottingham Business School

This work was supported by funding from the Staff and Educational Development Association (SEDA)

20 April 2023 2

‘The Origin of the Thesis’

• Retention matters but institutional retention data is unreliable

• Why students leave is related to when they leave

• Virtual learning environments are an intrinsic part of the modern undergraduate experience

• From an academic management perspective tracking electronic engagement is more robust than physical registers of attendance

• Electronic engagement data is an information resource capability for developing retention strategies

20 April 2023 3

The studyBuilding on pilot research of business students (Buglear, 2009)

• Final electronic engagement of first year undergraduates leaving their course in 2008/9 by type of leaver

• The final electronic engagement by each leaver, the last login – The last visit to the university electronic environment as a registered user

• Why first years?– Most students who leave prematurely do so in their first year

• Defining types of leaver– Notifiers; the ‘decided’, those giving formal notification of their

departure, recorded as e.g. ‘Transferred to other institution’, ‘Gone into employment’, ‘Other withdrawn’.

– Non-notifiers; the ‘drifters’, those giving no such notification, recorded as e.g. ‘Written off after lapse of time’, ‘Dormant’. ‘Academic failure’ is included in this category as the last logins preceded the examination period.

20 April 2023 4

The case study

• Nottingham Trent University (NTU), UK

• Student population of approximately 25,000 in 2008/9

• In 2008/9 nine schools located on three campuses

20 April 2023 5

Results

• Total last logins to May 2009 = 435

• 217 last logins in the first half year (October to January)

• 228 last logins in the second half year (February to May)

• Notifiers = 257 (59.1%)

• Non-notifiers = 178 (40.9%)

20 April 2023 6

Last logins over time

MayAprilMarchFebruaryJ anuaryDecemberNovemberOctober

80

70

60

50

40

30

20

10

0

2008/ 9 academic year

Last

login

s

20 April 2023 7

Last logins over time by notificationYes = notification of departureNo = no notification of departure

80

70

60

50

40

30

20

10

0

Last

login

s

YesNo

Variable

20 April 2023 8

Last logins by school and notificationTotal Yes = notified departures from the schoolTotal No = departures from the school not notified

School

90

80

70

60

50

40

30

20

10

0

Last

login

s

Total YesTotal No

Variable

20 April 2023 9

First half year: 71/217 last logins were non-notifiers (32.7%)Second half year: 108/218 were non-notifiers (49.5%)

Test for difference in proportions = 0, P-Value=0.000Difference is significant

Half year Feb-MayOct-Jan

250

200

150

100

50

0

Last

login

s

YesNo

Variable

20 April 2023 10

Animal, Rural and Environmental SciencesDifference in proportions is not significant(Fisher’s exact test P = 1.000)

Feb-MayOct-Jan

14

12

10

8

6

4

2

0

Last

login

YesNo

Rural & EnvironmNotify_Animal,

20 April 2023 11

Architecture, Design and the Built Environment Difference in proportions is significant at 5%(Fisher’s exact test P = 0.016)

Feb-MayOct-Jan

40

30

20

10

0

Last

login

s

YesNo

Design andNotify_Architecture

20 April 2023 12

Art and DesignDifference in proportions is significant at 10%(Fisher’s exact test P = 0.098)

Feb-MayOct-Jan

50

40

30

20

10

0

Last

login

s

YesNo

& DesignNotify_Art

20 April 2023 13

Arts and HumanitiesDifference in proportions is not significant (Fisher’s exact test P = 0.747)

Feb-MayOct-Jan

25

20

15

10

5

0

Last

login

s

YesNo

HumanitiesNotify_Arts &

20 April 2023 14

Education Difference in proportions is not significant (Fisher’s exact test P = 0.384)

Feb-MayOct-Jan

30

25

20

15

10

5

0

Last

login

s

YesNo

Notify_Education

20 April 2023 15

Nottingham Business SchoolDifference in proportions is not significant (Fisher’s exact test P = 0.286)

Feb-MayOct-Jan

40

30

20

10

0

Last

login

s

YesNo

Business SchoNotify_Nottingham

20 April 2023 16

Nottingham Law SchoolDifference in proportions is not significant (Fisher’s exact test P = 1.000)

Feb-MayOct-Jan

30

25

20

15

10

5

0

Last

login

s

YesNo

Law SchoolNotify_Nottingham

20 April 2023 17

Science and TechnologyDifference in proportions is significant at 10%(Fisher’s exact test P = 0.099)

Feb-MayOct-Jan

30

25

20

15

10

5

0

Last

login

s

YesNo

& TechnologyNotify_Science

20 April 2023 18

Social SciencesDifference in proportions is not significant (Fisher’s exact test P = 0.282)

Feb-MayOct-Jan

50

40

30

20

10

0

Last

login

s

YesNo

SciencesNotify_Social

20 April 2023 19

Discussion

• Financial aspect– Approximately 180 first year students drifted out of NTU programmes in

2008/9. – Consequent loss of tuition fee revenue ≈ £2m.

• Pedagogical aspects– first-semester decisions to exit […] are most aptly characterised as driven

by external factors’ (Peel et al., 2004)– ‘second semester [leavers] seemed more disillusioned and unhappy, […]

expressing feelings of loneliness, isolation, and lack of recognition’, feeling that ‘lecturers were “never there” or “always regard failure with disdain” or “never gave me the help I needed” ’ (Peel et al., 2004)

20 April 2023 20

Discussion

•The Fitzgibbon and Prior (2003) timeline model– ‘Zone 1: enrolment, induction and the first two weeks of teaching’,– ‘Zone 2: late enrolment, late induction and early weeks of

teaching’,– ‘Zone 3: middle to end of teaching period, first/second

assessments’,– ‘Zone 4: final assessment period, revision and examination or

assessment’ – Zone 3 is when ‘students who have poorly established […] study

habits, really come under pressure’ and ‘students […] receive feedback from their first assignment [;] constructive feedback and reassurance is […] crucial’

– Yet by this stage ‘staff assume students have settled […] but this is frequently not the case [,] students are still seeking significant levels of contact with their tutors for a whole range of issues’

20 April 2023 21

DiscussionRetention strategies

•The Beatty-Guenter four-stage retention strategies model (1994)– Sorting students ‘into meaningful subsets […] to create strata that can be

matched with appropriate targeted retention strategies’ – Supporting, ‘making it more likely that they will be able to maintain their

status as students’ – Connecting, ‘bonding between a student and the institution’ – Transforming ‘students from uncommitted to committed, from uninvolved

to involved, from passive to active, or from failure threatened to achievement motivated’

•How did we do?– Sorting – partially applied e.g. international students– Supporting – Welcome weeks, induction– Connecting and Transforming – assumed to be intrinsic

20 April 2023 22

Conclusions

• A significantly greater proportion of second half year leavers than first half year leavers didn’t tell us they were going

• Considerable variation between schools

• The majority, 60% of last logins before the examination period were by students who told us they were going, the ‘decided’ – The notification suggests some form of dialogue about their departure

• The remaining 40% were by the ‘drifters’. – The lack of notification suggests an absence of dialogue about their

departure

• The extent of non-notified departure is the scope for pay-off from Zone 3 Connecting and Transforming strategies

• Not the whole retention picture, but another perspective of it

20 April 2023 23

References

• Beatty-Guenter, P. (1994) Sorting, supporting, connecting, and transforming: Retention strategies at community colleges. Community College Journal of Research and Practice, 18, 113-129.

• Buglear, J. (2009) Logging in and dropping out: exploring student non-completion in higher education using electronic footprint analysis. Journal of Further and Higher Education, 33, 381-393

• Fitzgibbon, K and Prior, J. (2003) Student expectations and university interventions – a timeline to aid undergraduate student retention [online]. BEST Conference: Creativity and Innovation in Academic Practice, Brighton, 9-11 April 2003.

• Peel, M., Powell, S., and Tracey, M. (2004) Student Perspectives on Temporary and Permanent Exit from University: A Case Study from Monash University. Journal of Higher Education Policy and Management 26 (2), 239-249.