Domain Based Communication Skills Assessment Pilot For EU

Doctors

Annie M Cushing1, Jean S Ker2, Paul Kinnersley3,

Anthony N Warrens1, Olwyn M R Westwood1 ,

Pascal McKeown4

European Union Law:Article 53 of Directive 2005/36/EC

‘Persons benefitting from the recognition of professional qualification shall have a knowledge of languages necessary for practicing the profession in the host Member State’

Assessment is NOT a test ofClinical Knowledge

Legal Framework: ‘Proportionate and necessary’ :

1. Applied in a non-discriminatory manner2. Justified by overriding reasons based on the general

interest3. Suitable for securing the attainment of the objective

which they pursue and4. Not go beyond what is necessary to obtain that

objective

Mis-communication is multi-factorial• Language Issues• Cultural differences in concept of roles, systems

and professionalism• Consultation style and Communication skills in

doctor-patient relationship • Cultural aspects of healthcare and team

relationships

Pilotti LS 2007, Watt D, 2007,

Woodward-Kron R 2009, Trafford P 2010 (personal communications)

Development Process for Test of Applicants to the UK Foundation Programme

Validity•Tasks (GMC Tomorrow’s Doctors 2009, Scottish Doctor)

•Domain-based mark scheme (Communication)

•Setting the Standard – Pass mark

Reliability •Training: Assessors Clinicians and

Simulated Patients•Information for Candidates

BLUEPRINT TEMPLATE FOR POPULATING EXAM STATIONSCONTEXTS

TASKSRoutine Oral

Emotional Challenge

Diversity - culture, interpreter/impairment

Telephone Reading/Writing

Partnership Acute Setting

History Station 1 Station 15

Station 13 Station 2 Station 1

Drug History

Explain Condition Station 5

Explain Treatment /Procedure

Station 4 Station 14

Examination/Procedure Station 15 Station 3

Write Notes/Drug Chart Station 2Station 7

Read Notes/Drug Chart Station 7

Team/Handover Station 6 Station 7

Station 7 Station 6 &12

Team Receive Instructions Station 7

Bad News/Sensitive Issues/ Error/Complaint

Station 14 Station 5

3rd Party/Interpreter Station 8 Station 13

Shared Decision Making Station 9

Consent / Ethical Issues Station 10

DOMAINS

A = English language skills1. Listening comprehension: understands patient/colleague2. Spoken English comprehension: patient/colleague understands3. Reading comprehension: demonstrates understanding of written input4. Written comprehension: writing understood to enable appropriate action

B = Clinical communication1. Opening the consultation: establishes relationship with patient2. Professional /humanistic quality: continues to maintain the relationship3. Information gathering: gathers enough information to understand the problem(s)4. Examining the patient: patient is enabled to co-operate with examination5. Explaining/negotiating/consenting: information is understood with appropriate patient involvement6. Communication with colleagues/team members

C = Patient ratings: Patient satisfied with doctor-patient relationship and understanding

OSCE Format

15 Station Exam , 10 Minutes each = 2hrs 30mins

5th year Medical Students (n= 15 at each site)

Results (n = 58)

Raw Percentage Mean = 81.4

Standard Deviation = 6.7

Cronbach’s alpha = 0.78

Standard Error of Measurement = 3.11

Standard Setting by the Borderline Regression Method

Cut Score (pass mark) = 62.44

Cut Score + 1 SEm = 65.62

Cut Score + 2 SEm = 68.80

Cumulative Marks’ Curve (integrated frequency distribution) combined sites scores

Overall Marks vs no. of stations failed, overall cut score, confidence intervals and stations’ criterion

Means of Domains & Variance across All StationsA1-4 = Language, B1-5 = Consultation, B6 = Team, C= Patient Rating

Next Steps

Pilot with Doctors qualifying from

non-English Speaking Countries

Acknowledgements

Sam Leinster Siobhan Fitzpatrick

Pascal McKeown Jonathan Silverman

John Patterson Penny Trafford

Lynne Rustecki Tim Swanick

Katie Petty-Saphon