European Hematology Examand
Progress testDr. Tomás Navarro
Chair Curriculum-Exam CommitteeInstitut Català d’Oncologia-Germans Trias i Pujol Hospital
Josep Carreras Leukaemia Research Institute
Reasons for the project
§ A ‘logical’ next step toward the harmonization of hematology education and training
§ Provide an extra quality stamp to hematologists§ Facilitate mobility§ Contribute to improve patient care throughout Europe
Composition Curriculum-Exam CommitteeChair:Tomás Navarro, SpainVice chairMarielle Wondergem, The NetherlandsMembers:- Antonio Almeida, Portugal- Gunnar Birgegård, Sweden- Janaki Brolin, Sweden- Carlos Fernández de Larrea, Spain- Mahesh Prahladan, UK- Alicia Rovó, Switzerland
EHA office:- Nathalie van Havre- Wietske Hollegien- Judith Lewis-Stride- Manon Saris- Enrica Orsini
Target group
§ Hematologists with an interest in self-assessment and education. Independent of their career phase
§ Main target group are hematologists who just finalized their specialty training
§ Having completed the Online Curriculum Passport
Benchmarking:§ National Hematology Societies § European Medical Specialty Societies
§ European Society Anaesthesiology § European Society of Cardiology § European Society/Board of Radiology§ European Board of Urology§ European Respiratory Society
Exam development
European knowledge test
Multiple choice questionnaire§ Allows a large sample size of test items§ Testing knowledge in several areas with high reproducibility§ One single test covering the full European Curriculum§ Can be used in a web-based format§ Automatic objective correcting and scoring
Exam development
European knowledge test
§ Multiple choice questionnaire in English§ official language of EHA§ recognized scientific language worldwide§ Wording in different languages can cause different
interpretations § Mobility requires good level of English
Exam development
EvaluationDiscuss thresholds for high cost (JAK2 inhibition) treatment in PV
SynthesisSuggest a treatment algorithm for myelofibrosis
AnalysisCompare recent risk score models in ET
ApplicationDiscuss how symptom profile influences therapy choice in PV
UnderstandingHow does JAK2V617F mutation cause uninhibited erythropoiesis?
RecallWhich mutation is most common in PV?
The taxonomy ladder
What did we decide?
§ The European Hematology Curriculum (version 3) is our basis and must be reflected in the examination
§ All taxonomic levels cannot be tested but we should aim at the highest level possible
§ The context should be as ”real” as possible§ Must include multiple samples to cover the Curriculum as reliably as
possible
Ergo:§ Well written MCQs that test understanding and its application
Multiple choice question writing
Guidelines for the development of Questions & answers for the European Exam in Hematology
Writing questions§ First: what do you want to test?
(Find common clinical misunderstandings, knowledge gaps in residents)
§ Find a clinical scenario, preferably authentic, reflecting normal clinical pratice.
§ Check the evidence base for the correct answer§ Find 4 distractors if you can, but don’t use unrealistic ones, rather
use 3§ Consider the stem for a 2-step question
Each question consists of three parts: ‒ The stem ‒ The question ‒ The answer options
The Stem
a). A clinical scenario (improves validity, allows higher taxonomic level)
§ Clinically plausible (authentic if possible)
§ Short§ Enough info to anticipate the
right answer
b). Stem without clinical scenario
§ Fact control§ Used when clinical stem is
difficult/impossible to match with the question
§ Low taxonomic level, should be rather few
The Question
§ Direct, unambiguous§ Preferably address single issue§ Knowledgeable student can anticipate correct answer before seeing
options§ Should ask for the best answer: most likely/most appropriate/most
predictive/most reliable/most important§ Avoid negative questions, never use double negatives
The answer options
§ Plausible and realistic§ Clearly related to the stem and question§ One option must be clearly the best§ Generally relate to the same modality§ 5 options if you can find 4 plausible distractors, otherwise 3
distractors
The Exam Format
Test Format
§ Preferably questions with a “real scenario” stem§ Low number of “fact questions”§ Number of questions: 100§ Exam duration: 2.5 hours
Exam question development
§ Recruitment of question writers: 20 European hematologists SWGs as an important source
§ Preparation of guidelines and workshops for question writers (question development training)
§ In pairs, these writers work on their questions and review each other’s questions
The Question Review Group
§ The Question review group consists of 7 hematologists.§ This committee is responsible for the final round of review of all the
questions.§ This group selects the final 100 questions to be included in the
exam, representing all Curriculum sections.§ The review ensures all sections of the curriculum are addressed in
balance, that the questions are relevant and, well written, according to the Question Writing Guidelines.
Quality Assurance
§ To guarantee independent judgment of the quality of the exam and the passing score definition, EHA collaborates with CITO, an internationally recognized professional research and knowledge institute in the field of educational measurement and testing.
§ CITO’s Psychometric Research- and Knowledge Center performs the psychometric analysis of the results.
Candidate ExperienceAt the end of the exam, the candidates are asked to complete a short evaluation survey. Results of 2020 survey:
Exam sessions
2017: 1st exam at the EHA Congress - 64 candidates
2018, 2nd exam at the EHA Congress & parallel session in Bern* - 80 candidates in total
2019, 3rd exam at the EHA Congress & 5 parallel sessions - 149 candidates (incl 87 at main session)
2020, although there was no main session, still 127 participants in 12 parallel sessions!*As of 2019, European Exam is the official knowledge exam for Swiss hematologists
2021 Exam§ Collaborating with National Societies (NS) for organizing
parallel sessions§ Hold the exam on Thursday, June 3, one week before
the Congress§ The Italian Society of Hematology is also very welcome
to host a session!
Progress test
Progress test
What is the goal?
§ To see what level you have achieved in different areas of hematology compared to your European colleagues
§ To pinpoint your own “knowledge gaps” and be provided with suggestions to improve them
§ Monitor your progress through the years§ To prepare for the exam
Progress test
§ The Progress test consists of almost 100 multiple choice questions and is modelled on previous European Hematology Exams.
§ On completion, you receive immediate feedback (on your correct & incorrect answers), including references for more in-depth reading and studying.
§ 3 weeks after the end of the testing period, you will receive comprehensive results per curriculum section, together with feedback on your overall performance in relation to the passing grade.
Progress test
Format: feedback for participant§ Achieved level of competence in
every section of the European Hematology Curriculum: “expectation for end-of-training” level: below or at/above
§ Indicates progress
Progress test
§ The progress test is available twice per year
§ The next progress test will be available in March 2021
Thank you!Gràcies
Gracias
Merci
Danke
Obrigado Grazie mille!!
Tak Dzieki
ευχαριστίες
Mila esker
Bedank
teşekkür ederim