1 assessment of students competence in health professionals education professor hossam hamdy...
Post on 20-Dec-2015
217 views
TRANSCRIPT
![Page 1: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/1.jpg)
1
Assessment of Students Competence in Health Professionals Education
Professor Hossam HamdyUniversity of Sharjah
![Page 2: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/2.jpg)
2
Professional Competence
The Habitual and Judicious use of:
• Communication
• Knowledge
• Technical Skills
• Evidence-based decision-making
• Emotions
• Values and reflection to improve the health of the individual patient and the community
![Page 3: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/3.jpg)
3
WHAT Should Be Assessed?
ACGME Competencies- Patient care- Knowledge- Practice-based learning and
improvement - Interpersonal and communication skills- Ethics & Professionalism- System-based practice
![Page 4: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/4.jpg)
4
Blueprinting
Test content matches objectives
• Assessment matches competencies learnt
• Assessment matches format of learning, “PBL”
![Page 5: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/5.jpg)
5
Aim of Blueprinting
Reduce:A. Construct under-representation:
Biased samplingFew items covering a topic e.g. OSCE
B. Construct irrelevant variance:Flawed items formatInadequate sampling of student Behaviour.
![Page 6: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/6.jpg)
6
Domain of Interest
Test Sample
Test Sample
![Page 7: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/7.jpg)
7
Blueprinting
• Identify clinical problems that examinees should be able to handle.
• For each problem, define the clinical tasks in which the examinee is expected to be competent.
• Define level of performance / resolution
![Page 8: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/8.jpg)
8
Blueprinting
• Prepare a blueprint to guide the selection of problems to be included in the assessment.
• Let the characteristics of the clinical problems define the tasks to be included in the assessment.
![Page 9: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/9.jpg)
A Simple BlueprintCompetenceCategories
CVS RS … GI ….. MSS Endo/Metab
Haem/ Oncology
….
History Taking
Physical Exam
Tests and Procedures
…..
Management
Communication and Patient Education
![Page 10: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/10.jpg)
Pre-Clerkship Phase Theme B.Sc. Exam Blueprint 2003-2004
UNITS
Inheritance,
growth &
development;
degeneration & aging
Nurture,
environment
& nutriti
on
Structure:
function;
positional
relationship
Control, regulation &
communication within the body,
Homeostasis: metabolie, circulatory, acid/base & electrolyte
Response and
adaptation
systems &
mechanisms
Injury, inflammati
on & infection;
loss, repair and substitutio
n; neoplasia
Immunological
& defense mechan
isms
Investigative
procedure &
interpretation
Intervention
prevention &
management
principles
Human behavior, ethical & medico legal
principles
1 2 3 4 5 6 7 8 9 10 11
UNIT VIII:Integrated Multi-System
UNIT VII:
Nervous System, Special Sense & Human Behavior
UNIT VI:
Integumentary & Muscular-Skeletal
UNIT V:
Hematopoietic & Immune Systems
UNIT IV:
Endocrine, Metabolism & Reproductive System
UNIT III:
GIT & Renal System
UNIT II:
Respiratory & CVS
UNIT I:
Concepts & Principles & Community Health
![Page 11: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/11.jpg)
11
![Page 12: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/12.jpg)
12
A Simple Model of Competence
Miller GE. The assessment of clinical skills/competence/performance. Academic Medicine (Supplement) 1990; 65: S63-S7.
Knows
Shows how
Knows how
Does
Pro
fess
iona
l aut
hent
icity
Pro
fess
iona
l aut
hent
icity
Written, Oral orComputer based assessment
Performance or hands on assessment
![Page 13: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/13.jpg)
13
Validity Climbing the Pyramid . . .
Knows
Shows how
Knows how
Does
Knows Factual tests: MCQ, essay type, oral…..
Shows howPerformance assessment in vitro:OSCE, SP-based test…..
DoesPerformance assessment in vivo: Masked SPs, Video, Audits…..
Knows how(Clinical) Context based tests:MCQ, essay type, oral…..
![Page 14: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/14.jpg)
14
Sample Good Question & ‘Cover The Options’ Rule
A 32-yo man has a 4-day history of progressive weakness in his extremities. He has been healthy except for an upper respiratory tract infection 10 days ago. His temperature is 100 F, BP 130/80, pulse 94, respirations 42 and shallow. He has symmetric weakness of both sides of the face and the proximal and distal muscles of the extremities. Sensation is intact. No deep tendon reflexes can be elicited; the plantar responses are flexor. Which of the following is the most likely diagnosis?
A. Acute disseminated encephalomyelitisB. Guillain-Barré syndromeC. Myasthenia gravisD. PoliomyelitisE. Polymyositis
![Page 15: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/15.jpg)
15
Sample Good Question & ‘Cover The Options’ Rule
A 32-yo man has a 4-day history of progressive weakness in his extremities. He has been healthy except for an upper respiratory tract infection 10 days ago. His temperature is 100 F, BP 130/80, pulse 94, respirations 42 and shallow. He has symmetric weakness of both sides of the face and the proximal and distal muscles of the extremities. Sensation is intact. No deep tendon reflexes can be elicited; the plantar responses are flexor. Which of the following is the most likely diagnosis?
A. Acute disseminated encephalomyelitisB. Guillain-Barré syndromeC. Myasthenia gravisD. PoliomyelitisE. Polymyositis
![Page 16: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/16.jpg)
16
Assessment of Doctor’s Performance
Fundamental to “Good Medical Practice”
“What doctors do in controlled representations” of practice e.g. “OSCE”
“What doctors do in real life”
Rethans et al, Med Ed 2002.
![Page 17: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/17.jpg)
17
•Unobserved 30 – 45 minute interview and examination on a selected patient
•Candidates present only their findings to the examiner
The Traditional Long Case Examinations (LCE)
![Page 18: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/18.jpg)
18
• It assesses the integrated interaction between the student/doctor and the real patient
• Authenticity high
• More valid than the task given in an OSCE
• Little is known about the construct validity and consequential validity “Educational Impact”
)LCE (Validity
![Page 19: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/19.jpg)
19
• Poor intercase reliability
• Content specificity is the most crucial issue in the assessment of clinical competence
• Broad sampling across cases is essential “Multiple Biopsies”
• Logistics will be difficult
)LCE (Reliability
![Page 20: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/20.jpg)
20
Clinical Encounter
Method
Mini – CEX “Norcini 2003” components of the encounter observed
Direct Observation Clinical Encounter Examination “DOCE”
Entire encounter observed “Hamdy 2000”
![Page 21: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/21.jpg)
21
• Three hour examination of four 45 minutes observed long cases
• Real patient selected from a predetermined blue print of common diseases
• Check List – clinical encounter domains
• Reliability 0.8 (Cronbach’s alpha)
(Hamdy et al, Med Educ, 2003)
Direct Observation Clinical Encounter Examination
The DOCEE
![Page 22: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/22.jpg)
22
Practical Procedural Skills
Method
Direct Observation Practical Procedure Skills “DOPS”
Whole procedure observed from start to finish
![Page 23: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/23.jpg)
23
Multi-source Feedback (MSF) 360 Degree Assessment
Peers
Supervisors
Nurses
Secretary
Lab Technicians
Patients
![Page 24: 1 Assessment of Students Competence in Health Professionals Education Professor Hossam Hamdy University of Sharjah](https://reader036.vdocument.in/reader036/viewer/2022062421/56649d4a5503460f94a279ac/html5/thumbnails/24.jpg)
24
Problems of Workplace Based Assessment of Performance
Sampling: variability in the complexity of patients problem
Judges errors:
Time: Requires adequate allocation of time and resources.
Management of Data: not easy
Validity and Reliability: variable