school of medicine faculty of medical and health sciences › education › pdf › jhc-symposium...

School of Medicine

Faculty of Medical and Health Sciences

Objective Structured Clinical Examination (OSCE)

Neck Examination

Inguinal Hernia

Abdominal

Examination

Trauma and

Resuscitation

Peripheral Vascular

Disease

Gallstone Disease

Year 4 Surgical OSCE

3 clinical stations involve peer physical

examination:

1. Acute abdominal pain

2. Acute leg pain

3. Neck lump

Year 4 Surgical OSCE

STUDY OBJECTIVES

• To investigate validity and reliability of OSCE grades awarded by Year 6 students during the Year 4 surgical OSCE

• To investigate the perceptions and attitudes of Year 6 students towards OSCE examining

University of AucklandMBChB Programme

YEAR

1

2

3

4

5

6

PHASE

Pre-Clinical

Clinical

OSCE Grading

History-Taking

1. Appropriate and professional manner

2. Logical progression of questions related to

presenting complaint

3. Appropriate depth and range of questioning

4. Asks about other important related areas

5. Summary and working diagnosis is succinct and

appropriate

OSCE Grading

Physical Examination

1. Appropriate and professional manner

2. Logical progression of the examination

3. Inspection

4. Palpation

5. Percussion and Auscultation

OSCE Grading

Year 4 Students are graded against expected

standards of a Final-Year Medical Student

GRADE SCORE DEFINITION

Mastery 4 Standard has been fully met

Clear Pass 3 A few minor mistakes/omissions were made

Flagged

Pass

2 Some major mistakes and/or important

omissions were made

Fail1 Many major and significant mistakes or

omissions were made

Year 4 Surgical OSCE

• Externally Valid

• Internally Reliable

Yu TC, Wheeler BR, Hill AG. “Effectiveness of standardised clerkship teaching across multiple sites.” Journal of Surgical Research, 2011;168:e17-e23.

STUDY PARTICIPANTS

• 124 x Year 4 medical students

• 28 x Year 6 medical students

• 8 x Faculty academic clinicians

Example:

Peripheral Vascular DiseaseYear 4 Surgical OSCE Evaluation Form

I enjoyed being an OSCE Examiner.

From my experience today ….

a) I learnt new clinical information

b) I reviewed my current knowledge

c) I learnt how to be a better teacher

d) I learnt how to provide feedback

e) I learnt what makes a better OSCE station

I will incorporate what I learnt today into my future clinical practice.

Participating as an OSCE Examiner has improved my confidence to teach.

As an OSCE Examiner, I gained insight into student learning processes.

I will volunteer to be an OSCE examiner in the future because ….

a) I will enjoy the opportunity to teach and provide feedback

b) Of a sense of duty to clinical teaching

c) I am likely to improve my clinical knowledge and skills

d) I will enjoy the social interactions with students and other examiners

e) Of the great food

The Clinical School values my contributions today.

My willingness to participate would increase if my contributions were acknowledged by clinical supervising consultants.

STUDY RESULTS

Internal Reliability – Cronbach’s alpha coefficients

StudentExaminers

Faculty Examiners

Acute Abdomen History-taking 0.80 0.69

Examination 0.80 0.83

PeripheralVascular Disease

History-taking 0.76 0.75

Examination 0.80 0.82

Surgical Neck Examination 0.82 0.80

STUDY RESULTS

Acute Abdomen: Student vs Faculty Exmainers

SEmean

FEmean

Paired T-test Pearson’s Correlation

History-taking(20 marks) 15.9 16.3

t= -1.49 (173)p=0.139

0.25p=0.001

Examination(20 marks) 15.8 16.1

t= -0.99 (172)p=0.323

0.24p=0.001

Total Grade(40 marks) 31.6 32.4

t= -1.43 (173)p=0.155

0.27p<0.001

STUDY RESULTS

Peripheral Vascular Disease: Student vs Faculty Examiners

SEmean

FEmean

Paired T-test Pearson’s Correlation

History-Taking(20 marks) 14.8 16.0

t= -2.77 (173)

p=0.006p=0.687

Examination(20 marks) 14.6 15.8

t= -2.62 (173)

p=0.010p=0.441

Total Grade(40 marks) 29.4 31.8

t= -2.76 (173)

p=0.006p=0.770

STUDY RESULTS

Surgical Neck Examination: Student vs Faculty Examiners

SEmean

FEmean

Paired T-test Pearson’s Correlation

Total Grade(20 marks) 15.8 15.6

t = 2.21 (173)p=0.029

0.82p<0.01

TI Perceptions and Attitudes Mean

Enjoyed being an OSCE Examiner 4.36

From today’s experience ....

a) Learnt new clinical information 3.63

b) Reviewed my current knowledge 4.33

c) Learnt how to be a better teacher 4.14

d) Learnt how to provide feedback 4.00

e) learning what makes a better OSCE station 4.14

Incorporate today’s learning into future clinical practice 4.29

Participating has improved my confidence to teach 4.14

As an OSCE examiner, I gained insight into student learning processes 4.30

In the future, I will volunteer to be an OSCE examiner because ....

a) Enjoy opportunity to teach and provide feedback 4.32

b) Sense of duty to clinical teaching 4.18

c) Am likely to improve my clinical knowledge and skills 4.32

d) Enjoy social interactions with students and other examiners 4.43

e) Of the great food 4.11

The Clinical School values my teaching and examining contributions 4.04

Willingness to participate would increase if contributions acknowledged by consultants 3.68

Strongly agree = 5, Agree = 4, Unclear = 3, Disagree = 2, Strongly disagree = 1

STUDY CONCLUSION

Validity of OSCE grades given by Year 6

student examiners varied between different OSCE stations

Year 6 student examiners find OSCE examining to be a positive learning and social experience

STUDY DISCUSSION POINTS

Possible Explanations:

Features of Individual OSCE stations

Examiner versus Patient Perspectives

Clinical Competency of Year 6 students

Performance Expectations of Year 4 students

ACKNOWLEDGEMENTS

South Auckland Clinical SchoolSchool of MedicineFaculty of Medical and Health SciencesUniversity of Auckland

Study Participants, MBChB StudentsFaculty of Medical and Health SciencesUniversity of Auckland

A Pilot Study of Final-Year Medical Students as

Surgical OSCE Examiners

Dr Tzu-Chieh Wendy Yu

Associate Professor Andrew G Hill

Department of Surgery

South Auckland Clinical School, Middlemore Hospital

Faculty of Medical and Health Sciences

University of Auckland

wendy.yu@auckland.ac.nz