bond university research repository a snapshot of osce ...•a/prof jane smith •prof clare heal...
TRANSCRIPT
![Page 1: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/1.jpg)
Bond UniversityResearch Repository
A snapshot of OSCE practices at Australian Medical Schools
Smith, Jane W; Heal, Clare; D’Souza, Karen; Malau-Aduli, Bunmi S.; Turner, Richard
Published in:Journal of Islamic International Medical College
Published: 01/03/2018
Document Version:Peer reviewed version
Link to publication in Bond University research repository.
Recommended citation(APA):Smith, J. W., Heal, C., D’Souza, K., Malau-Aduli, B. S., & Turner, R. (2018). A snapshot of OSCE practices atAustralian Medical Schools. Journal of Islamic International Medical College, 13(Supplement), 123-124.
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
For more information, or if you believe that this document breaches copyright, please contact the Bond University research repositorycoordinator.
Download date: 09 Oct 2020
![Page 2: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/2.jpg)
• A/Prof Jane Smith
• Prof Clare Heal
• Dr. Karen d’Souza
• Dr. Bunmi Malau-Aduli
• Prof Richard Turner
• Dr Jenny Banks
A Snapshot of OSCE practices in Australian Medical schools
![Page 3: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/3.jpg)
![Page 4: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/4.jpg)
BackgroundACCLAIM started in 2011
• Firstly: James Cook University (JCU) and University of Tasmania(UTAS)
• Was 15 of 19, now 15 of 21 Australian Medical Schools
ACCLAiM AIMS:
• Collaboratively develop shared OSCE stations
• Online examiner training with discussion and feedback
• Analysis of student performance data
• A framework to facilitate national benchmarking
![Page 5: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/5.jpg)
“SNAP SHOT” Methods
• Ideas from ACCLAIM meeting August 2016
• Questionnaire designed Nov 2016
• Ethics, in order to send to ALL medical schools
• Email survey monkey link sent early 2017
Aims: Measure and compare OSCE practices with “Gold Standard”
![Page 6: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/6.jpg)
Results
• 18/19 Australian Medical Schools participated (2 new ones since survey)
• All 15 ACCLAIM members, and 3 non-members
• 71 questions were asked
![Page 7: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/7.jpg)
How many OSCE stations are in your exam?
0
1
2
3
4
5
6
7
8
9 10 12 13 16
Nu
mb
er o
f M
edic
al S
cho
ols
Number of OSCE stations
![Page 8: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/8.jpg)
How long is each station?
0
1
2
3
4
5
6
7
8
9
6 7 8 9 10 >10
Nu
mb
er o
f M
edic
al S
cho
ols
Length in Minutes
![Page 9: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/9.jpg)
How long is perusal time?
0
2
4
6
8
10
12
1 2 3 4 5
Nu
mb
er
of
Med
ical
Sch
oo
ls
Length of Perusal time in minutes
![Page 10: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/10.jpg)
May notes be written in perusal time?
9
• 50% allow notes
• 50% DON’T allow notes
• Australian Medical Council DO NOT allow notes
![Page 11: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/11.jpg)
Devices NOT permitted in OSCE
0 2 4 6 8 10 12 14 16 18
Watches
Pens
Fit -Bit bands
Smart watches
Recording device
Mobile phone
Number of Medical Schools applying restrictions
![Page 12: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/12.jpg)
11
![Page 13: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/13.jpg)
Blueprinting
• ALL use their own curriculum
+/- Graduate outcomes from• Australian Medical Council(AMC)• Medical Deans of Australia & New
Zealand (MDANZ )
![Page 14: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/14.jpg)
Procedural skills tested in OSCEs
14 do, and 4 don’t
0 1 2 3 4 5 6
Suturing
PAP smear
CPR/life support
Injection/ABG
Urinary catheter
STI screen
Number of Medical schools using specified procedural skill
![Page 15: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/15.jpg)
Types of rating scales used to score marks
0 1 2 3 4 5 6 7
Competency- based
Checklist
Checklist & Competency
Marks added from Global rating
Number of Medical Schools using
![Page 16: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/16.jpg)
Standard setting used for OSCE
0 2 4 6 8 10 12
Borderline Regression
Borderline Groups
Angoff
Other
Hofstee
Ebel
Number of Medical Schools using each
![Page 17: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/17.jpg)
How much time until re-assessment OSCE?
0 1 2 3 4 5 6
0 to 2
3 to 4
5 to 6
up to 16
Number of Medical Schools
Weeks until Re-assessment OSCE
![Page 18: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/18.jpg)
The multiple sources of simulated Patients
0 2 4 6 8 10 12 14 16
Actors
General Public
Staff
Other
MedicalStudents
Number of Medical schools using each type of simulated Patient
Other: real patients, elective international students, community volunteers, general public who have been trained
![Page 19: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/19.jpg)
Use of REAL PATIENTS in OSCEs
EXAMPLES:• Abnormal heart sounds• Abnormal respiratory sounds eg pulmonary fibrosis• Musculoskeletal deformities• Diabetic limb• Neurological signs eg Parkinsons, eg peripheral
neuropathy • Splenomegaly
Just under half use real patients in OSCE physical examination stations
![Page 20: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/20.jpg)
Simulated Patients marking the student
SCORING of• rapport and communication with patient • whether they were respectful • showed empathy • how they felt treated as a patient
In 20% of the responding Medical schools Simulated patient give 10% of the station marks
![Page 21: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/21.jpg)
Rates of pay to simulated patients
0 1 2 3 4 5 6
Transport
Gift card $ 30-50 total
$20/hr
$30/hr
$40/hr
$50/hr
$800/day
Number of Medical schools
![Page 22: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/22.jpg)
WHO are the EXAMINERS?(ALL had multiple answers)
0 2 4 6 8 10 12 14 16 18
General Practitioners
Specialists
Registrars
Junior Doctors
Retired and Alumni
Medical Students
![Page 23: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/23.jpg)
Examiner Training ?
• Orientation to OSCE & calibration
• Training workshops
• Briefing before the OSCE
• Documents to read prior to the OSCE
• Orientation to examiner’s OSCE station
• Online training prior to the OSCE
![Page 24: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/24.jpg)
Do medical schools pay examiners?
1/3 DO, 2/3 DONT
![Page 25: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/25.jpg)
“Gold Standard” AMEE guide 81
• Examiner training reduces examiner variation in scoring
• Poorly standardized simulated patients = poor reliability
• Checklist v competency based v global rating scale
• Standard setting varies from recommended
• Quarantine recommended & practiced
• Devices = many more than phones
![Page 26: Bond University Research Repository A snapshot of OSCE ...•A/Prof Jane Smith •Prof Clare Heal •Dr. Karen d’Souza •Dr. Bunmi Malau-Aduli •Prof Richard Turner •Dr Jenny](https://reader036.vdocument.in/reader036/viewer/2022071409/61034aaf4d76ed0327314cc6/html5/thumbnails/26.jpg)
Conclusions
• ACCLAiM is a great example of collaboration
• Variation between medical schools
• Shared information is of use to all
• Many future research possibilities
• ACCLAiM contact [email protected]
• My email [email protected]