The effects of a simulation game on skills and motivation of doctors and students:
the expertise-reversal effect in medical practice
Games for Health 2016
Mary Dankbaar PhD,
Jelmer Alsma MD, Jan van Saase PhD, Stephanie Schuit PhD,
Erasmus University Medical Center Rotterdam,
Jeroen van Merrienboer PhD, Maastricht University
The Netherlands.
Background: training complex cognitive skills
Simulation games offer challenging & engaging learning tasks. Games are becoming increasingly popular in medical education.
More insight in the effectiveness of simulation games for skills training and their critical design features is needed (1,2, 3).
Erasmus University MC developed a simulation game for residents to train complex cognitive emergency care skills.
(1)Clark, Mayer E-Learning and the science of instruction, 2008. (2) Akl , Pretorius, Sackett et al. BEME Guide No 14,.2010.
(3) Garris, Ahlers, Driskell, Games, Motivation and Learning: a research practice model, 2002, Dec, 33 (4).
Background (2)
A previous study with residents showed positive effects of the simulation game on residents’ cognitive emergency care skills and engagement (4).
For novice learners, high-fidelity cases may cause ‘cognitive overload’ (5) .
Highly realistic tasks with feedback are engaging, but expensive.
Research question: Are cognitive skills and motivation of medical students more improved by adding a simulation game or text-based cases to an instructional e-module on emergency care?
(4) Dankbaar et al, in Games for Health 2014, Schouten, Fedtke, Schijven, Vosmeer, Gekker (eds)
(5) Van Merrienboer, Kirschner, 2012
Simulation game (high fidelity cases)
- Authentic tasks (6 patients cases)- High level of interactivity & feedbackabcdeSIM on emergency care
Text-based cases (low fidelity)
Ee-module-m
Using the same 6 cases
Motivation questionnaire
Control group, n=16
E-module
Game group, n=25 Cases group, n=20
Questionnaire
Knowledge test24 items
Open Cases program
Questionnaire
Questionnaires
Text based cases (low fidelity)Game-based cases (high fidelity)
Cognitive load e-module Cognitive load e-module Cognitive load e-module
Knowledge test Knowledge test Knowledge test
E-module E-module E- module
Cognitive loadCognitive load
Motivation questionnaireMotivation questionnaire
Skill assessment(2 cases)
Mannequin-based assessment
Mannequin-based assessment
Mannequin-based assessment
Study-designRandomized study-design
E-module on emergency care
Text-based cases (low fidelity)
Mannequin-based skills assesment
Clinical competency rating scale (6 items, scale 1-7), ‘uses ABCDE method on treatment’
Communication competency scale (3 items, scale 1-7), ‘communicates with patient effectively’
Global performance scale (single scale 1-10) ‘independent function in caring for acutely ill patient’
The Assessment instrument was validated in a separate study showed good validity and moderate interrater reliability (1)
(1) Dankbaar et al, Assessing the Assessment in Emergency Care Training. Plos-One, Dec. ‘14.
2 scenario-based assessments with blinded assessors; on diabetic coma and urosepsis with delirium (15 min.)
Questionnaires
Cognitive Load (10 items, scale 0-11) Intrinsic load ‘the content of the game/cases was very complex’ (3 items)
Extraneus load ‘the explanations were very unclear’ (3 items)
Germane load ‘the game/cases really enhanced my understanding’ (4 items)
Evaluation (9 items, scale 1-5) Engagement ‘it was fun to work through the material’ (6 items)
Feedback ‘during learning, I could tell whether I was doing well’ (3 items)
Results (n= 61)
No differences between groups in age, clinical experience, GPA bachelor
No differences in knowledge scores between groups (M= 20, SD=3.7)
Self-reported learning time in minutes, mean (SD) Logged game time
E-module Game or Cases Total
Control group 133 (78) - 133
Game group 146 (98) 95 (53) 241 83 (54)
Cases group 177 (118) 70 (46) 247
Performance in Clinical skills
game group cases group control group1
2
3
4
5
6
7
5.4 5.2 5.6
P = 0.13 N=61
Cognitive load for game- and text-based cases
Intrinsic Load Extraneous Load Germane Load0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
6.0
3.3
8.4
5.0
3.4
7.2
Game groepCases groep
P=0.03ES=0.8
P=0.01ES=1.0
P=0.75
N=54
Motivation
Engagement Feedback1
1.5
2
2.5
3
3.5
4
4.5
5
4.3
3.43.5
3.1
3.6
2.7
Game groupCases groupControl group
P < 0.001*, ES =1.4 P < 0.05**, ES =1.0
* Game vs cases+ control** Game vs. control
N=59
Conclusions
For initial skills acquisition, an e-module with a video demonstration is a powerful instructional format.
Novice students do not profit from working on additional open cases, which appeared to be too complex, but nonetheless challenged them to study longer.
High fidelity game-based cases enhance motivation and cognitive effort, but also increases complexity and may impede learning for novices.
The same game improved clinical skills among residents, with limited self-study time.
‘Expertise-reversal effect’:A rich learning environment may benefit experts, but is counter-productive for novice learners (interaction prior knowledge - complexity of formats)
Take-home message
Serious games often are engaging, but careful alignment of case complexity and fidelity in games with students’ proficiency level is essential.
Complexity in cases may be reduced by offering guidance (hints, worked cases) and gradual increase in fidelity
More research is needed on the relation between case-complexity, case-fidelity, motivation and skills development.
Questions?
Contact information [email protected] Mary Dankbaar Erasmus University Medical Center Rotterdam