Gaming as a training tool

for cognitive skills in

emergency care:

how effective is it?

Mary Dankbaar, Jan van Saase, Stephanie Schuit, Erasmus MC

Maartje Bahhuys Roozeboom, Esther Oprins, TNO

Frans Rutten, Schola Medica, SBOH

Jeroen van Merrienboer, Maastricht University

The Netherlands

Training in emergency care skills; background

Critical for patient safety & an essential part of medical education.

Obligatory for professionals, working at Emergency Department.

Training costs are substantial.

Erasmus MC has developed a serious game, abcdeSIM, to train cognitive

emergency care skills for residents more efficiently (blended).

ABCDE approach is an international standard for emergency skills

A=Airway, B=Breathing, C=Circulation, D=Disability, E=Exposure; ‘Treat first what kills first’

Design abcdeSIM game on emergengy care skills

- Authentic tasks (6 patients cases)

- Functional realistic simulation

- High level of interactivity & feedback

- Score system

Training setting: Blended design

Tutorial Knowledge

test

Game

Face-to-

face

Training

Assessment

Research on serious games

Serious games have been propagated to learn complex skills in an active,

intrinsically motivating way1; research on the cognitive and motivational

effects of serious games shows mixed results; only few are assessed for

there effectiveness2,3,4.

Systematic reviews on technology enhanced simulations show that, in

comparison with no intervention, simulations have large effects on

knowledge and skills5.

1 Clark Ruth C, Mayer RE. E-Learning and the science of instruction, Pfeiffer, editor. San Fransisco; 2008.

2 Akl et al.The effect of educational games on medical students’ learning outcomes: a systematic review: BEME Guide 14.

3 Sitzmann, a Meta-Analytic Examination of the Instruct. Effectiveness of Computer-Based Simulation Games. PersPsych. 11

4 Graafland et al, Systematic review of serious games for medical education and surgical skills training. Br J Surg. 2012

5 Cook et al, Technology-Enhanced Simulation for Health Professions Education; A systematic review, JAMA. 2011.

Research questions

1. Show residents, after having played the game, better emergency care

skills before training than residents who did not play the game?

2. Are game playing residents feeling engaged with the scenarios in the

game and are they more motivated for the course than residents who

did not play the game?

Study design

Preparation Pre-training

assessment

Training Post-training

assessment

Non game

group

Study material n= 18

Questionnaire on

motivation for course

2 weeks n = 52

Game

groep

Study material

abcdeSIM gameQuestionnaire on

engagement

n= 24

Questionnaire on

motivation for course

2 weeks n= 107

Skills assessment

Clinical competency rating scale

6 items, 7 point scale (7=excellent)

“uses ABCDE method on treatment”

Communication competency rating scale

3 items, 7 point scale (7=excellent)

“communicates with patient effectively”

Global performance scale ‘independent

function in caring for acutely ill patient in ER’

Single 10 point scale (10=perfect)

Assessment instrument was validated in a separate study1:

- Clinical competency showed good validity and moderate interrater reliability

- Communication competency scale showed poor interrater reliability

- Global performance scale showed moderate interrater reliability

-1. Dankbaar et al, Assessing the assessment: in emergency care. In Production.

3.46

4.85

4.25

4.71

1

2

3

4

5

6

7

Clinical competency Communication competency

Nongame group

Game group

p=.03; Cohen’s d= 0.62 SD= 1.27/0.75

p >.5

SD= 0.76/0.92

Show residents, after having played the game,

better emergency care skills (before training) than

residents who did not play the game?

- No difference in

communication

competency level

Groups are comparable on relevant characteristics (n=18/24)

- Game group

showed higher and

more homogenous

clinical competency

level than nongame

group

No difference in global performance between

groups, before training

4.92 5.04

1

2

3

4

5

6

7

8

9

10

Overall Performance

Nongame group

Game group

p > .5

Is the positive effect of the game maintained

after 2 weeks training?

5.75.7 5.8

1

2

3

4

5

6

7

Clinical competency Communicationcompetency

Control group

Intervention group

7.47.2

1

2

3

4

5

6

7

8

9

10

Overall Perferformance

Control group

Intervention group

Global performanceClinical and communication

competencies

After 2 weeks: no longer differences between groups

Are game playing residents feeling engaged

with the scenarios in the game?

Game time Mean = ± 2.5 hours

Engagement/motivation questionnaire (9 items)

“I felt actively involved with the pt. scenarios” (5 pt scale; 5=fully agree)

Mean score: 3.9

Items above mean:

- “I liked playing the game” (M=4.2)

- “I felt actively involved with the patient scenarios” (M=4.2)

- “My attention was completely drawn to the scenarios” (M=4.2)

“ Very instructive, very real and good preparation. I felt the stress”

“I would have liked to get more feedback on specific wrong choices”

Are game playing residents more motivated towards

the course than residents who did not play the game?

Questionnaires before training (MSLQ)

- Task value 7 pt. scale (7=very true of me)

9 items (e.g. “I think that what I’m learning in this course is useful for me”)

No difference between groups in motivation for the course

Control group (n=47) Intervention group (n=102)

Task value M= 6.2, SD=0.43 M=6.2, SD=0.39

Conclusions

After 2.5 hours playing the abcdeSIM game, residents showed a substantial

higher pre-training clinical competency level.

After 2 weeks training, there no longer was a difference between groups.

(effect of 2.5 hr game training overshadowed by 2 weeks training)

The residents felt engaged with patient scenarios and liked playing the game.

Game playing residents were not more motivated for the course.

Take home message

* A serious game can improve emergency care skills with residents in an

effective, attractive and flexible way.

* In combination with the game, the training can probably be shortened.

More research is needed on the blended design and game characteristics.

Thank you for your attention!

Contact information

m.dankbaar@erasmusmc.nl

Mary Dankbaar

Erasmus University Medical Center Rotterdam