MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

GaLAGame and Learning Alliance

The European Network of Excellence on Serious Games

SIG3.5 – SGs for Humanities & Heritageuser studies

Michela Mortara, Chiara Eva Catalano, CNR – IMATIJeffrey Earp, CNR-ITD

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Overview

Icura [ongoing, 50%]

• Playing History – The Plague [started, 20%]

Playing History – The Plague [started, 10%]

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

USER STUDY #1: ICURA

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Game description

• Educational objective: cultural awareness - Japanese etiquette

• Genre: adventure• Environment: 3D • Platform: PC• Intended educational context: informal (at home)• Developer: TU-Wien• Duration: less than 1 h• Target: secondary school and higher• Previous studies: evaluation done by the developers at

the Wien university on a group of 20 students. Reports on papers and PhD. thesis

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Previous evaluation

• Pre-questionnaire (demographic data & game habits)• Pre-test with 12 questions ( on language basics, behaviour

& etiquette, culture and society)• Game session• Post-test (same 12 questions)• Final questionnaire about user satisfactionObservations:• On average from +5 to +10 correct answers btw pre and

post– Selective attention bias

• Learning gain linked to the way the content is communicated (information agent vs quest)

• Lacking estimate of retention

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Our setting

• Same phases but different questions in the pre-test to avoid the selective bias

• Higher participant number (CNR, RWTH, ORT, DIBE)• Open answers to assess higher level in Bloom’s

taxonomy (post questionnaire) e.g.:• What have you learnt from the game and how (through what mechanics)? • Did the game increase your interest towards Japan/ Japanese culture?

Would you read more about Japan? Would you visit it?• How would you summarise Japanese culture in 3 adjectives?• What can you deduce about Japanese culture from the facts you learnt in

the game? • How would you compare Japanese and Western culture?

• Plan to repeat the post test after a few months to evaluate retention

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Current status

• CNR session– 10 university students– Colleagues&friends

• RWTH session– 17 university students

• ORT session– 12 people– students

• DIBE session

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Partial outcomes

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Partial outcomes

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Partial outcomes

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

PLAYING HISTORY – THE PLAGUE

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

The game

• Educational objective: history, historical reconstruction • Genre: adventure• Environment: 3D • Platform: browser (Unity Web player)• Intended educational context: formal (at school)• Developer: Serious Games Interactive• Duration: nearly 3 hours• Target: 8-13• Previous studies: evaluation done by a master student

with two primary school classes in Denmark. Report in her Master thesis (in Danish).

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

The experience: a Joint Research Action

• JRA involving CNR-IMATI (SIG3.5), CNR-ITD (TC2.8, T6.3), SGI and DIS (Deledda International School) in Genova

• Objectives:– explore the suitability and effectiveness of serious games in the

field of humanities and cultural heritage in schools;– evaluate the educational efficacy of the game and the level of

student engagement and enjoyment triggered by the learning activities;

– investigate critical aspects of SG deployment in formal ed./school contexts, especially teacher’s role

– examine cultural/educational issues related to game’s adoption in Italy (game produced in northern Europe and set in medieval Italy)

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

The context

• School: English-speaking school in Genoa following International Baccalaureate & IB Middle Years programmes

• Classes: 2 classes Year 7 (11-12 year olds). Approx. 35 pupils

• Deployment settings: computer lab, classroom, home access

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Research framework

• Approach: qualitative, design based research – DBR Collective

• Tools:

– question matrix for gathering teachers’ GBL/ICT experience & attitudes, plus their expectations, intentions and goals for the experience

– pedagogical planning tool (teachers document intentions/goals of intervention and plan learning activity sequence)

– researchers’ observation sheets for monitoring classroom activities

– pupils’ pre/post tests (in teacher’s resource pack from SGI)

– mind map recording pupils’ collective pre/post knowledge/awareness

– teachers’ monitored focus group

– teacher questionnaire

– reflection sheet for recording researchers’ attitudes, intentions, goals and interventions in the experience

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Activity Phases 1 & 2…

1. Teacher preparation– introduction to the experiment, the game and its teaching resource pack– teachers’ group discussion of matrix questions (researcher compiled)– tech. training session (game’s teacher functions; ITD pedagogical

planner)– teachers consolidate pedagogical plans

2. Deployment – pupil’s individual pre-test & collective brainstorming of pre-knowledge

(making collective mind map)– class briefing: game presentation + play– gameplay sessions in pairs (plus possible home play)– pupil summative test & debriefing (re-examining collective mind map)

= total 4/5 school periods with researcher monitoring & support

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

… Activity phases 3 - 6

3. Teacher debriefing– focus group (inc. re-examination of matrix questions and pre-

deployment responses, and of pedagogical plan/s)– final questionnaire

4. Analysis of data gathered from phases 1-3

5. Follow up – Repeat of post test to evaluate pupil retention

6. Final data analysis and reporting

MadridApril 15-17, 2013

European Commission Information Society and

Media

General Assembly

Current status

• Series of organisational meetings held involving CNR, DIS & SGI

• Activity phase 1 commenced– introduction to teachers– discussion / compiled of teachers’ of question matrix

Issues with running game on DIS network have delayed commencement of deployment sessions with pupils• Solution: DIS agreed to move students to CNR (2

sessions in May)