training the osce examiners katharine boursicot trudie roberts
TRANSCRIPT
![Page 1: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/1.jpg)
Training the OSCE Examiners
Katharine BoursicotTrudie Roberts
![Page 2: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/2.jpg)
Programme
• Principles of OSCEs for examiners• Video marking• Marking live stations• Strategies for enhancing examiner
participation in training
![Page 3: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/3.jpg)
Academic principles of OSCEs
• The basics• What is an OSCE?
• More academic detail• Why use OSCEs?
• The role of examiners• Examiners in OSCEs
![Page 4: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/4.jpg)
The basics
• For examiners who don’t know about OSCEs
• A brief reminder for those who are familiar with OSCEs
![Page 5: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/5.jpg)
What is an OSCE?
•Objective •Structured •Clinical •Examination
![Page 6: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/6.jpg)
Station
OSCE test design
![Page 7: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/7.jpg)
OSCEs - Objective
• All the candidates are presented with the same test
![Page 8: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/8.jpg)
OSCEs - Structured
• The marking scheme for each station is structured
• Specific skill modalities are tested at each station• History taking• Explanation• Clinical examination• Procedures
![Page 9: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/9.jpg)
OSCEs – Clinical Examination
• Test of performance of clinical skills: not a test of knowledge• the candidates have to demonstrate
their skills
![Page 10: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/10.jpg)
More academic detail
• Why use OSCEs in clinical assessment?
• Improved reliability• Fairer test of candidate’s clinical abilities
![Page 11: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/11.jpg)
Why use OSCEs in clinical assessment?
• Careful specification of content• Observation of wide sample of
activities • Structured interaction between
examiner and student• Structured marking schedule• Each student has to perform the
same tasks
![Page 12: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/12.jpg)
Characteristics of assessment instruments•Utility =
• Reliability • Validity• Educational impact• Acceptability• Feasibility
ReferenceVan der Vleuten, C. The assessment of professional competence: developments,research and practical implications Advances in Health Science Education 1996, Vol 1: 41-67
![Page 13: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/13.jpg)
Test characteristics
• Reliability of a test/ measure• reproducibility of scores
across raters, questions, cases, occasions
• capability of differentiating consistently between good and poor students
![Page 14: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/14.jpg)
Test Sample
Test Sample
Domain of Interest
Sampling
![Page 15: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/15.jpg)
Reliability
• Competencies are highly domain-specific
• broad sampling is required to obtain adequate reliability• across content i.e. range of
cases/situations• across other potential factors that cause
error variance i.e.• testing time, examiners, patients, settings,
facilities
![Page 16: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/16.jpg)
OSCE : blueprintHistory Explan Exam Procedur
e
CVS Chest pain Disch drugs
Cardiac BP
RS Haemoptysis
Smoking Resp Peak flow
GIS Abdo pain Gastroscopy
Abdo PR
Repro Amenorrhoea
Abnormal smear
Cx smear
NS Headache Eyes Ophthalmosc
MS Backache Hip
Generic Pre-op assess
Consent for post mortem
IV cannulationBlood trans rea
![Page 17: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/17.jpg)
Test characteristics
• Validity of a test/measure• the test measures
the characteristic (eg knowledge, skills) that it is intended to measure
![Page 18: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/18.jpg)
Knows
Knows how
Shows how
Behaviour~ skills/attitudes
Cognition~ knowledge
Model of competence
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S67.
Does
Pro
fess
ion
al
auth
enti
city
![Page 19: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/19.jpg)
Validity of testing formats
Knows
Knows how
Professional practice assessment
Shows how
Does
Knowledge assessment: MCQs
Problem-solving assessment: EMQs, SEQs
Performance assessment: OSCEs, long/short cases, OSLERs, etc
![Page 20: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/20.jpg)
Test characteristics: Educational impact
Teacher
Curriculum
Student
Assessment
Relationship between assessment and learning
![Page 21: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/21.jpg)
Test characteristics
• Feasibility• cost• human resource• physical resources
![Page 22: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/22.jpg)
Test characteristics
• Acceptability• tolerable effort• reasonable cost
• Acceptability• doctors• licensing bodies• employers• patients/consumer groups• students• faculty
![Page 23: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/23.jpg)
The role of examiners in OSCEs
• General• Types of stations• Standard setting• Practice at marking
![Page 24: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/24.jpg)
The role of examiners in OSCEs
• To observe the performance of the student at a particular task
• To score according to the marking schedule
• To contribute to the good conduct of the examination
![Page 25: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/25.jpg)
The role of examiners in OSCEs
• It is NOT to:• Conduct a viva voce• Re-write the station• Interfere with the simulated patient’s
role• Design their own marking scheme• Teach
![Page 26: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/26.jpg)
Types of OSCE stations
• History taking• Explanation• Clinical examination• Procedures
![Page 27: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/27.jpg)
Communication skills
• Stations involving patients, simulated patients or volunteers
• Content vs process i.e what the candidate says
vs
how the candidate says it
![Page 28: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/28.jpg)
Clinical skills
• People• Professional behaviour
• Manikins• Describe actions to the examiner
![Page 29: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/29.jpg)
The examiner’s role in standard setting
• Use your clinical expertise to judge the candidate’s performance
• Allocate a global judgement on the candidate’s performance at that station
• Remember the level of the examination
![Page 30: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/30.jpg)
Global scoring
Clear fail
Borderline
Clear pass
Very good pass
Excellent pass
![Page 31: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/31.jpg)
Borderline method
Checklist1.
2.
3.
4.
5.
6.
7.
TOTAL
Passing score
Borderline score distribution
Pass, Fail, Borderline P/B/F
Test score distribution
![Page 32: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/32.jpg)
Regression based standard
Checklist1.
2.
3.
4.
5.
6.
7.
TOTAL
Overall rating 1 2 3 4 5
1 2 3 4 5
ChecklistScore
X
X = passing score
1 = Clear fail2 = Borderline3 = Clear pass4 = V Good pass5 = Excellent pass
Clear Borderline Clear V Good Excellent fail pass pass pass
![Page 33: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/33.jpg)
Practice at marking
• Videos
• Live stations
• Mini-OSCE
![Page 34: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/34.jpg)
![Page 35: Training the OSCE Examiners Katharine Boursicot Trudie Roberts](https://reader030.vdocument.in/reader030/viewer/2022032523/56649d805503460f94a65185/html5/thumbnails/35.jpg)
Strategies for enhancing examiner participation
1. CME2. Job plan/ part of contract3. Specific allocation of SIFT4. Experience for post-graduate
examinations5. Payment