The effects of a simulation game on skills and motivation of doctors and students:

the expertise-reversal effect in medical practice

Games for Health 2016

Mary Dankbaar PhD,

Jelmer Alsma MD, Jan van Saase PhD, Stephanie Schuit PhD,

Erasmus University Medical Center Rotterdam,

Jeroen van Merrienboer PhD, Maastricht University

The Netherlands.

Background: training complex cognitive skills

Simulation games offer challenging & engaging learning tasks. Games are becoming increasingly popular in medical education.

More insight in the effectiveness of simulation games for skills training and their critical design features is needed (1,2, 3).

Erasmus University MC developed a simulation game for residents to train complex cognitive emergency care skills.

(1)Clark, Mayer E-Learning and the science of instruction, 2008. (2) Akl , Pretorius, Sackett et al. BEME Guide No 14,.2010.

(3) Garris, Ahlers, Driskell, Games, Motivation and Learning: a research practice model, 2002, Dec, 33 (4).

Background (2)

A previous study with residents showed positive effects of the simulation game on residents’ cognitive emergency care skills and engagement (4).

For novice learners, high-fidelity cases may cause ‘cognitive overload’ (5) .

Highly realistic tasks with feedback are engaging, but expensive.

Research question: Are cognitive skills and motivation of medical students more improved by adding a simulation game or text-based cases to an instructional e-module on emergency care?

(4) Dankbaar et al, in Games for Health 2014, Schouten, Fedtke, Schijven, Vosmeer, Gekker (eds)

(5) Van Merrienboer, Kirschner, 2012

Simulation game (high fidelity cases)

- Authentic tasks (6 patients cases)- High level of interactivity & feedbackabcdeSIM on emergency care

Text-based cases (low fidelity)

Ee-module-m

Using the same 6 cases

Motivation questionnaire

Control group, n=16

E-module

Game group, n=25 Cases group, n=20

Questionnaire

Knowledge test24 items

Open Cases program

Questionnaire

Questionnaires

Text based cases (low fidelity)Game-based cases (high fidelity)

Cognitive load e-module Cognitive load e-module Cognitive load e-module

Knowledge test Knowledge test Knowledge test

E-module E-module E- module

Cognitive loadCognitive load

Motivation questionnaireMotivation questionnaire

Skill assessment(2 cases)

Mannequin-based assessment

Mannequin-based assessment

Mannequin-based assessment

Study-designRandomized study-design

E-module on emergency care

Text-based cases (low fidelity)

Mannequin-based skills assesment

Clinical competency rating scale (6 items, scale 1-7), ‘uses ABCDE method on treatment’

Communication competency scale (3 items, scale 1-7), ‘communicates with patient effectively’

Global performance scale (single scale 1-10) ‘independent function in caring for acutely ill patient’

The Assessment instrument was validated in a separate study showed good validity and moderate interrater reliability (1)

(1) Dankbaar et al, Assessing the Assessment in Emergency Care Training. Plos-One, Dec. ‘14.

2 scenario-based assessments with blinded assessors; on diabetic coma and urosepsis with delirium (15 min.)

Questionnaires

Cognitive Load (10 items, scale 0-11) Intrinsic load ‘the content of the game/cases was very complex’ (3 items)

Extraneus load ‘the explanations were very unclear’ (3 items)

Germane load ‘the game/cases really enhanced my understanding’ (4 items)

Evaluation (9 items, scale 1-5) Engagement ‘it was fun to work through the material’ (6 items)

Feedback ‘during learning, I could tell whether I was doing well’ (3 items)

Results (n= 61)

No differences between groups in age, clinical experience, GPA bachelor

No differences in knowledge scores between groups (M= 20, SD=3.7)

Self-reported learning time in minutes, mean (SD) Logged game time

E-module Game or Cases Total

Control group 133 (78) - 133

Game group 146 (98) 95 (53) 241 83 (54)

Cases group 177 (118) 70 (46) 247

Performance in Clinical skills

game group cases group control group1

2

3

4

5

6

7

5.4 5.2 5.6

P = 0.13 N=61

Cognitive load for game- and text-based cases

Intrinsic Load Extraneous Load Germane Load0.0

1.0

2.0

3.0

4.0

5.0

6.0

7.0

8.0

9.0

6.0

3.3

8.4

5.0

3.4

7.2

Game groepCases groep

P=0.03ES=0.8

P=0.01ES=1.0

P=0.75

N=54

Motivation

Engagement Feedback1

1.5

2

2.5

3

3.5

4

4.5

5

4.3

3.43.5

3.1

3.6

2.7

Game groupCases groupControl group

P < 0.001*, ES =1.4 P < 0.05**, ES =1.0

* Game vs cases+ control** Game vs. control

N=59

Conclusions

For initial skills acquisition, an e-module with a video demonstration is a powerful instructional format.

Novice students do not profit from working on additional open cases, which appeared to be too complex, but nonetheless challenged them to study longer.

High fidelity game-based cases enhance motivation and cognitive effort, but also increases complexity and may impede learning for novices.

The same game improved clinical skills among residents, with limited self-study time.

‘Expertise-reversal effect’:A rich learning environment may benefit experts, but is counter-productive for novice learners (interaction prior knowledge - complexity of formats)

Take-home message

Serious games often are engaging, but careful alignment of case complexity and fidelity in games with students’ proficiency level is essential.

Complexity in cases may be reduced by offering guidance (hints, worked cases) and gradual increase in fidelity

More research is needed on the relation between case-complexity, case-fidelity, motivation and skills development.

Questions?

Contact information m.dankbaar@erasmusmc.nl Mary Dankbaar Erasmus University Medical Center Rotterdam