HEA-TIS June 16th 2011

Dr Jackie Riley jri@gcu.ac.uk

Director of International Development (Engineering and Computing)

Glasgow Caledonian University

Exams are Exams! Aren’t they? International Students’ experiences

BBackgroundackground

•Funded by the Centre for Sociology, Anthropology and Politics Subject Centre (part of the Higher Education Academy)

•Specific call on Internationalisation, targeted at the Scottish sector

Investigate international students first experiences of exams from three key questions:

1.‘What experiences of exams do ‘international students’ arrive with?’

2.‘What are their expectations of exams?’

3.‘What was their experience here of exams?’

Dr Karen Smith Karen.Smith2@gcu.ac.uk Dr Nick Pilcher N.Pilcher@napier.ac.uk

Caledonian Academy Lecturer in in-sessional EAP ENUBS

Glasgow Caledonian University Edinburgh Napier University

Importance of assessmentImportance of assessment

• Assessment important in higher education• Different types of assessment task (presentations;

case studies; portfolios; exams) (Toohey, 1999)• Movements towards ‘innovative’ assessment

(Hounsell et al, 2007)• BUT - examinations still play major role in

students’ assessment diet – (and may be increasing again)

Existing literatureExisting literature

• Much assessment related-research– E.g. preparation for exams; exam room practices;

different types of exam (e.g. Tal et al, 2008)• Exams and international students

– Mainly just around level of English (e.g. Smith, 2011)• Much student-focused guidance

– Little of which is aimed at international students specifically - except Palgrave’s International Student Handbook (Reinders et al, 2008)

Method

•Interviews (21 students) – before and after – tool designed with aims of giving control to interviewees (Foucault 1973), made participatory (Fontana and Frey 2005; Christians 2005)

•Survey: 168 students (88 ENU 80 GCU) 56% male; 44% female

•Based on areas from general exam advice texts (e.g. Blass 2009, McMillan and Weyers 2010;)

Findings

•Exam Format•Exam Environment•Preparation and Revision

Exam format – back at home

Language – back at home

Exam format – back at home

“exams lasted 4 to 5 hours... [I – So you were 4 or 5 hours in a room writing?] Yeh”(France). “the biggest things with

the exams is the teacher gets to be as creative as they want, they have a lot of freedom to do whatever they want” (USA). “for example if you have a

different point of view from the one stated in the book and you write it you might not have any marks”(Ukraine).

Exam format – in the UK

“sometimes you need to think the structure of the sentence in English but in Chinese you don't need to think” (China).

“I cannot get used to... maybe the question arrangement…in China just one part one point but here one part several points... the answer so I have to divide it in several parts and explain many points... it’s much difficult to get higher points” (China).“what was surprising… was the

referencing work that we had to do because in France we don’t have to refer to a source or stuff like that that much. If you don’t read it’s okay” (France).

Exam format – in the UK (cont.)

“One thing that was new that I’ve never done before and… I think it hurt me on my exams it was to source... I didn’t realise I had to read a document and then think about who wrote that and remember that and source it while writing” (USA).

“lecturers make a close connection between exams and what is taught. It was not like that in Nigeria” (Nigeria).

Environment – back at home

In your home country were any of the following tolerated in the exam room?

Answer Options Always % Sometimes % Never %

Use of mobile phones 2.5 2.5 94.9

Use of MP3 players 2.5 2.5 94.9

General talking 2.5 6.3 91.1

Talking about the exam 2.5 3.8 93.7

Passing notes to each other 2.5 1.3 96.2

Passing stationery to each other 3.8 35.4 60.8

Passing calculators to each other 3.8 44.3 51.9

Leaving the room for a break 12.7 21.5 64.6

Going to the rest rooms 39.2 26.6 32.9

Environment – back at home

In your home country were you aware of any of the following happening?

Answer Options Always % Sometimes % Never %

Buying the exam paper prior to the exam 0% 17% 83%

Taking notes into the exam 5% 39% 56%

Gaining knowledge of the questions in advance 4% 34% 62%

Influencing the marker 0% 17% 83%

Environment – back at home

“95% of students will bribe their lecturers to get a better mark. Male students pay in cash, female students in cash and kind” (Nigeria).

“they keep the classrooms colder usually because it helps you stay awake”(USA).

“they want you to be quiet and that is understandable but no-one is quiet… they’re all chatting away and waiting and some people get their exam… you hear people whispering the answers to each other” (Holland)

Environment – back at home (cont.)

“they have some GPS or some police they call technology police… test if anyone… [makes a]… text message or something very like spy things I don’t know…” (Taiwan)

“Well I got away a few times… but I suppose it was only... maybe key words to remember parts of things but no proper cheating... I wouldn’t do that… [I – Right OK so there’s a scale of cheating] Yeh I suppose so (laughs)” (Spain)

“I think it would be easier to cheat in the States because I did it all the time” (USA)

Preparation - sources

Sources used Pearson PG and UG usage

Use of Recommended text books – in home country

P=0.001 PG students made sig. more use of these than UG students

Use of Recommended text books – in Scotland

P=0.062* *Not statistically significant at 5% level but still PG student made more use of these than UG students

Other text books – in home country

P=0.025 PG students made sig. more use of these than UG students

Other text books – in Scotland

P=0.024 PG students made sig. more use of these than UG students

Recommended web sources in Scotland

P=0.02 PG students made sig. more use of these than UG students

Recommended web sources – in home country

P=0.01 PG students made sig. more use of these than UG students

Preparation – sources – at home

“it’s usually just textbook material and like some slides from lecturers... I just read through that a couple of times” (Slovenia).

“if teacher want to give you surprise you can’t see the question... but if teacher is good enough the exam will not let you surprise it’s all you know” (China)

“using an analogy, supposing that you were supposed to have covered say 100 metres in a course and then you could only cover, or the lecturer decides that you should cover 40 and then he expects you to go on and get the other 60 on your own… when you get to the exam you discover that most of the questions are from that 60 as against the 40 that you’ve been taught” (Nigeria)

Preparation – sources – in UK

“I regret... next semester I will revise” (China)

“should have gone through the coursework and look for similar questions in the textbooks… will do this in the next semester” (Nigeria)

“I realised really late the night before or the day before the test only that our teacher is expecting a lot from us like… to submit the notes or any references… he was like read more and more of this stuff... if we knew this before we would work really hard… I did my best in it but I do feel really guilty” (Oman)

Preparation - Strategies

When comparing their revision behavior, again at home and in Scotland via Chi-Square Tests, there were no significant factors found at the 5% level. However the tendency to revise with friends (P=0.067) and to memorize large amounts of information (P=0.09) both showed increases in Scotland.

Preparation Strategies Prefer to revise with friends Prefer to revise on your own Attempt past papers Review your answers against past paper solutions Attend revision classes. Seek individual assistance from faculty members Memorise large amounts of information

Preparation – strategies – at home

“Chinese people always study alone they seldom talk or discuss in groups I think it doesn’t help too much because... the personal conditions is not same [I – So group work’s not effective] Yeh”(China).

Preparation – strategies – in UK

“we lead the teamwork... [I – your experience was in China you worked individually] Yeh [I – Whereas here you worked as part of a team] Yeh” (China).

“the only thing I was surprised was with the… open book exam... the tutor he told us that we can like make notes and prepare notes… but the other guys… just wrote answers which they thought they will appear… and then they just copied the their essays on to those sheets… one English guy he write down 14 pages through the exam I managed to write 8 [I – What mark did he get?] [Distinction]…so I actually thought so next time when I know there will be an open exam… I will think about the answers and prepare the answers” (Slovakia)

Preparation – strategies – in UK (cont.)

“in Nigeria when exams approach... you would typically wake up in the middle of the night and put in an hour or two and then sleep back. It’s difficult here you know as when you wake up it’s cold you wouldn’t want to leave that bed so you lie down with your book and you spend like 10/15 minutes and you realise you are just deceiving yourself and close the book (laughs)” (Nigeria)

Where are Exams effective? Overall students feel their home exam system is

more effective:

Where are Exams effective?

Results: at one institution most students were disappointed their results had not been higher.

At the other most students were generally pleased, although one student said that they “would definitely change the advice I would give to other students. The way they want an answer to the question is really different here in the UK than in the Netherlands. Here they want you to write a complete essay as an answer to every question while in Holland they just want the answer and that's it. I think this might be where I went wrong” (Holland)

Exam questions and preparationExam questions and preparation

• The data showed that exam question types were similar around the world; familiarity with questions types led most students to prepare as they had done at home

• Some students:– Increased amount of study time– Revised in groups

• External factors (e.g. weather) impacted on some study regimes

Exam answersExam answers• BUT – what was required in the exams differed

– Purpose– Use of references– Use of examples– Introducing own opinion– Level of detail– Links between course content and exam

• Challenges of academic English and timing• Some students realised too late, often after the exam, that these

differences existed– Explains why some were disappointed with their results

• Issue of lack of time for acculturation combined with ‘high stakes’ assessment

exam room environmentexam room environment

• Stress increased due to unfamiliarity with exam room environment– Different expectations of regulations and protocol– Different understandings of cheating– Pressures of time– Challenge of academic English

conclusionsconclusions

• Previous experiences clearly impacted on students’ expectations of how to prepare for and sit UK exams

• Cannot expect lecturers to be aware of different exams systems, so:

– Give early feedback on exam-like tasks (Yorke, 2001)

– Encourage active participation in assessment matters (Ridley, 2009)– Take time early in semester to discuss what is expected in an exam (Price et al,

2010)

We found that

The term ‘exam’ can encompass many things and for international students, the format could be very different, the environment could be very different, and what is expected in an answer could also be very different. Not only this, but people’s approaches to studying and understanding of the purpose of exams could also be different. Awareness and exploration or these issues very early on after arrival would, we believe, help overcome these differences and help international students perform better in their exams.

How?..... http://www2.napier.ac.uk/depts/support/index.html

References

• Blass, E. (2009). Succeeding in exams & assessments. Harlow: Pearson Education. British Council http://www.britishcouncil.org/eumd-pmi2.htm accessed January 2010

• Christians, C.G (2005) Ethics and Politics in Qualitative Research. In Denzin, N. K., & Lincoln, Y. S. The SAGE handbook of qualitative research. Thousand Oaks: Sage Publications.

• Fontana, A and Frey, J.H. (2005) The interview. From neutral stance to political involvement. In Denzin, N. K., & Lincoln, Y. S. The SAGE handbook of qualitative research. Thousand Oaks: Sage Publications.

• Foucault, M (1973) Truth and Juridical Forms, In Faubion, J.D (Ed) (2000) Essential Works of Foucault. Volume 3. Power. New York: the New Press

• Hounsell, D. et al (2007) Innovative assessment across the disciplines: an analytical review of literature, York: HEA

References continued• McMillan, K., & Weyers, J. (2007). How to succeed in exams and assessments. Harlow:

Prentice Hall. • Price, M. et al (2010) If I was going there I wouldn’t start from here: a critical

commentary on current assessment practices, Assessment and Evaluation in Higher Education, ifirst

• Reinders, H. et al. (2008) The International Student Handbook, Basingstoke: Palgrave Macmillan

• Ridley, D. (2004) Puzzling experiences in higher education: critical moments for conversation, Studies in Higher Education, 29(1), 91-107

• Smith, C. (2011) Examinations and the ESL student – more evidence of particular disadvantages, Assessment and Evaluation in Higher Education, 36(1), 13-25

• Tal, I. R. et al (2008) Effect of paper colour and question order on exam performance, Teaching of Psychology, 35(1), 26-28

• Toohey, S. (1999) Designing courses for higher education, Buckingham: SRHE & OUP• Yorke, M. (2001) Formative assessment and its relevance to retention, Higher

Education Research and Development, 20(2), 115-126