supporting cognitive adaptability through game design working 100412
DESCRIPTION
Adaptability is a metacompetency critically important to the United States Department of Defense and is considered a key component of 21st Century skills by the U.S. Department of Labor and the U.S. Department of Education). Video games are seen as learning environments supporting the acquisition of 21st century skills. Can games, then, be used as components of an effective learning environment that support the development of adaptability? Initially this paper describes the metacompetency of adaptability. Next is how adaptability can be functionally and discretely measured by focusing on its most granular or micromomentary level which we describe as cognitive adaptability. Finally, the authors examine both the nature of cognitive adaptability, interventions that support its development, and how those interventions might be translated into game design features. Toward this end, the paper will also discuss how these features are exhibited in a popular commercially available video game and how it could be employed to test the hypothesis that a play frame of 12 consecutive hours, using a video game meeting the design criteria and example previously discussed, will increase cognitive adaptability in the playersTRANSCRIPT
Supporting Cognitive Adaptability Through Game Design
P. Shane Gallagher, Ph.D Learning Analysis Reasearch Corp [email protected] [email protected] Shenan Prestwich Advanced Distributed Learning (ADL) [email protected]
Adaptability?
• Adaptability: identified by U.S. DoD, DoL, DoEd as important metacompetency.
• Ability to use existing knowledge to create innovative problem solutions
• Repeatedly trying new/different strategies to solve problems while reflecting on actions and incorporating feedback
2
• Scale of macro to micro
– Macro: adaptive stance, operational adaptability
– Mid: individual adaptive behavior
– Micro: micromomentary cognitive processes, i.e., cognitive adaptability
3
How can we frame it?
• Micromomentary adaptive cognitive processes & functions
• Components
– Cognitive Openness/Creativity
– Focused Attention
– Cognitive Flexbility/Metacognition
– Critical Thinking & Problem-Solving
4
Cognitive adaptability?
Requires
Supports
Fostering cognitive adaptability
• Metacompetency?
• Learning environment design?
– Conceptual framework
– Intersection with CA = research void
• Had to draw from other areas of research and posit how these findings can be translated into design
– Cognitive remediation therapy (CRT)
– Feature overlap theory
– Game design - MDA 6
Intersecting domains
• Cognitive Remediation Therapy (CRT) – Neurocognitive psychotherapy technique aimed at
improving cognitive flexibility, working memory, and planning in sub-normal populations
– Emphasizes practicing cognitive microskills
– Wisconsin Card Sorting Test, Stroop Color Word Test, interventions (verbalizing, scaffolding, errorless learning)
7
Intersecting domains
• Feature Overlap Theory (Halpern, Hansen, & Riefer, 1990) – If training is too similar on a surface level to the actual
event, students will reach for superficial connections
– If training teaches a deep, causal understanding of the material but differs or varies on a surface level, students exercise ability to make deep connections and adapt knowledge.
8
“…Hunicke, LeBlanc, and Zubek (2002), where mechanics are the components of a game at the level of data representation and algorithms, dynamics are how the game components interact with the player and vice-versa, and aesthetics comprise the emotional response evoked by the mechanics and dynamics…”
MDA
10
CRT
FOT MDA
Others…
Conceptual Framework for Developing CA
A variation on MDA: FFMDA
11
12
FFMDA
13
FFMDA
5 Features To Increase C A
• Unstated/Implicit Rules
• Unstated/Implicit Shifting of Rules
• Dynamic Shifting Environments
• Open-Ended Gameplay
• Implicit Reinforcement for Individual Actions/Choices to Achieve Final Goal
14
The Study
Will playing a commercial off-the-shelf game with the five identified features will increase cognitive adaptability?
(Q1) Will 12 hours of consecutive gameplay of a video
game with the design features previously described
(Portal 2) cause an increase in cognitive adaptability as
measured by cognitive function tests (CANTAB)? Ind var
=Portal 2 playtime
(Q2) Will those playing Portal 2 be more likely to be
metacognitively aware during play than those not playing
Portal 2? Ind var Portal 2 play history
(Q3) Will those possessing higher metacognitive
awareness levels (MAL) also score higher on CANTAB
after playing Portal 2 than those who didn’t? Ind var
Portal 2 playtime
(Q4) Are there differences of CA between high MAL
(HMAL) and low MAL (LMAL)? Ind var MAL
(Q5) Is a prior history of playing Portal 2 or like games
positively correlated with higher metacognitive
awareness levels? Ind vars Portal 2 play history and MAL
(Q6) Will MA change over time during game play? Ind
var Portal 2 playtime
Research Questions:
The Game
How we did it…
June 8-17 2012 Provided:
• 39 Airmen • touchscreen
laptops • facilities • tech support on
site
Used FFMDA • Portal 2
Measures • CANTAB • Metacognitive Awareness
Inventory Protocol
• Experimental • 2 groups
Sample
Population
N=39
18 – Control Group (MS Games)
21 – Experimental Group (Portal 2)
Age 17-24, Ave age = 21
Ave years in AF – 1.22
Rank – mostly E1-E3
Education – mostly HS Diploma
Gender – 5.1% Female, 94.9% Male
Measures
• Cambridge Cognition CANTAB Eclipse • http://www.cambridgecognition.com/cantab-tests.asp
• Metacognitve Awareness Inventory • (Haynie, 2005)
• Game History Questionnaire • Active MA Survey • FRAPS recorded game sessions (3)
June 8 CANTAB baseline administration Used for mitigating practice effect in repeated measures Rugged touchscreen laptops in computer lab
Procedure
Procedure
June 14 CANTAB pre-test Metacognitive Awareness Inventory
Metacognitive Awareness Inventory
AMA
June 15-16 Airmen played either Portal II (Intervention)
Has the 5 features that should promote CA
Play for 6 hours over 2 days (12 hours total) with breaks every 1.5 hours
AMA questionnaire each break
N=21
OR
Solitaire, Minesweeper, FreeCell, and Mahjong (Control)
Do not have the 5 features that should promote CA
Commonly found in MS Windows 7
Puzzle/Logic games
Play each for 1.5 hours a day over 2 days (12 hours total)
AMA questionnaire each break
N=18
Procedure
Procedure
June 17 (final day)
• CANTAB posttest
• Game History Questionnaire
• Average plays 6-9 hrs/wk
– True for Exp and Control
• Most have never played Portal or Portal 2 before or have spent little time if they did
– True for Exp and Control
• Majority indicated that time spent playing video games has decreased in the past year and in the past 5 years
Metacognitive Awareness Inventory
• Average Score = 78%, Range 65%-95%
Results
Significant difference (increase) on RVPS between the experimental (Portal) and control groups (n=39, F=6.126, p=.018). Part of variance explained by high/low gaming
Tests of Within-Subjects Effects
Measure:RVPA
Source
Type III Sum of
Squares df Mean Square F Sig.
Trial * GRP Sphericity Assumed .002 1 .002 6.126 .018
Greenhouse-Geisser .002 1.000 .002 6.126 .018
Huynh-Feldt .002 1.000 .002 6.126 .018
Lower-bound .002 1.000 .002 6.126 .018
CANTAB Results
Spatial Span (SSP) -Computerized version of the Corsi Blocks task • Assesses working memory capacity • measure of frontal lobe functioning Previous Portal 2 Play Significant (p =.012) difference (decrease in errors) both pre and post measure on SSP between those that have played Portal 2 in the past six months and those that didn’t.
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Source
Type III Sum of
Squares df Mean Square F Sig.
Intercept 4075.283 1 4075.283 1702.932 .000
PRVPORTL2_NUM 16.756 1 16.756 7.002 .012
Error 86.152 36 2.393
CANTAB Results
High/Low Gaming Experience Analysis by high/low gamers shows a large and significant difference (F=16.778, p=0).
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Source
Type III Sum of
Squares df Mean Square F Sig.
Intercept 4565.978 1 4565.978 2341.745 .000
TIME_WEEK_BI 32.714 1 32.714 16.778 .000
Error 70.193 36 1.950
CANTAB Results
Analysis of SSP by high and low gamers controlling for previous Portal play shows significant portion of variance is most likely due to gaming in general rather than Portal.
Tests of Between-Subjects Effects
Measure:SSP
Transformed Variable:Average
Source
Type III Sum of
Squares df Mean Square F Sig.
Intercept 183.460 1 183.460 92.447 .000
TIME_WEEK_BI 16.694 1 16.694 8.412 .006
PRVPORTL2_NUM .736 1 .736 .371 .546
Error 69.457 35 1.984
CANTAB Results
One Touch Stockings of Cambridge (OTS) • Tests executive planning • Critical component for strategic thinking
Analysis of the number of problems correct on the first try (OTS) by high/low gamers shows strong significant difference between groups. High game play experience scored higher significantly on both the pre and post-tests (F= 10.335, p=.003). Controlling for previous Portal 2 play the significance was still at the .006 level suggesting that most of the variance occurred because of prior game play in general and not due to previous Portal 2 play. Tests of Between-Subjects Effects
Measure:OTS
Transformed Variable:Average
Source
Type III Sum of
Squares df Mean Square F Sig.
Intercept 10938.936 1 10938.936 2700.817 .000
TIME_WEEK_BI 41.859 1 41.859 10.335 .003
Error 149.859 37 4.050
CANTAB Results
Spatial Working Memory (SWM • Tests the ability to retain spatial information • manipulate remembered items in working memory • It is a self-ordered task suggesting heuristic strategies
Analysis by high and low gamers shows large significant decrease in errors made by high gamers (F=11,872, p=.001
Tests of Between-Subjects Effects
Measure:SWM
Transformed Variable:Average
Source
Type III Sum of
Squares df Mean Square F Sig.
Intercept 228139.508 1 228139.508 109.646 .000
TIME_WEEK_BI 24702.892 1 24702.892 11.872 .001
Error 76985.287 37 2080.683
CANTAB Results
Analysis of CANTAB by high/low gamers and previous Portal 2 play taking into account areas of game interest shows four measures of interest. • spatial working memory (SWM) • latency or response for focused attention (RVML*) • spatial span (SSP) • execute planning (OTS) Interactions SWM (spatial working memory)- significant interaction effect over both trials between interest in MMOG type games (F=8.965, p=.006). RVPML - significant interaction for interest in Card games/Simple puzzles (F=5.622, p=.026). Between Subjects Significant effect on SWM scores between interest in Racing games. *latency measure on the focused attention task and is one of the most sensitive measures on the CANTAB. It is a significant factor in the score differences of high/low gamers and previous Portal 2 play.
Differences by Areas of
Game Interest
Implications
Design • Cognitive focused attention was enhanced in Portal 2
players verses MS Games players • Potential cause is the open ended game play present in the
design of Portal 2. Gaming • High frequency gamers scored higher on cognitive tests than
low frequency gamers. • Design characteristics presents in MMOGs, card games,
puzzles, racing, strategy, and sports games may lead to these increases and should be explored.
Playing video games may actually lead to cognitive enhancement.
Further Research
• Cognitive Task Analysis of Portal 2 and others - present
• Further data collection on Portal 2/non-Portal 2 players and CANTAB measures Fall 12/Spr 13
• Testing for metacognitive monitoring on math for adaptability performance evidence and transfer tasks. Fall12/Spr 13
Questions?
For more information contact:
703-575-3718