CHALLENGES IN ASSESSING TRANSFER

FROM BURNING CUPCAKES TO WOMEN IN SCIENCE

NSF video game transfer conferenceOctober 7-10 2010

Susan M BarnettCornell University

Ceci & Bronfenbrenner (1985) cupcakes study

Children bake cupcakes and must keep track of baking time (30 minutes)

Distractor task to play PacMan Can’t watch clock whilst playing game so

must turn to see clock – experimenter records clock checking surreptitiously

One group at home, one in lab No explicit training phase

Effect of physical context on clock checking: cupcake baking task

From Ceci & Bronfenbrenner (1985)

Effect of physical context on clock checking: battery charging task

From Ceci & Bronfenbrenner (1985)

How and when does transfer happen?

Disagreement at the beginning of the 20th century   “…every experience has in it the possibilities of generalization” (Judd,

1908)   “…there is no inner necessity for improvement of one function to

improve others closely similar to it, due to a subtle transfer of practice effect.” (Thorndike and Woodworth, 1901b)

  …and disagreement at the end   “Numerous studies have shown that critical thinking… can be

learned in ways that promote transfer to novel contexts.” (Halpern, 1998)

  “Reviewers are in almost total agreement that little transfer occurs”

(Detterman, 1993).

Why does it matter?

In his classic educational psychology text, Klausmeier (1961) asserted: “A main reason for formal education is to facilitate learning in situations outside school” (p. 352).

So, if critics are correct in asserting that transfer very rarely happens, then the justification for educational and training expenditures may need to be reevaluated, as Detterman goes on to urge: “Cognitive psychologists, and other people who should know better, continue to advocate a philosophy of education that is totally lacking in empirical support” (Detterman, 1993, p. 16).

Why such profound disagreement?

Debate about what counts as transfer For some, only “far” transfer matters, but

what counts as far transfer? For some, transfer is only useful if the

learner can transfer when they do not realise they are supposed to be doing so. In the real world, no one is going to stand over the learner and point out the connections with their prior learning

Some have tried to define transfer

“The carrying over of an act or way of acting from one performance to another” (Woodworth & Schlosberg, 1954, p. 734)

“The ability to extend what has been learned in one context to new contexts” (Bransford, Brown, & Cocking, 1999, p. 39)

But there is no clear, agreed-upon definition of what constitutes ‘carrying over’ or ‘a new context’

A lot of the disagreement comes from this – essentially from comparing apples and oranges

Specifying dimensions of transfer

Specify a framework of relevant dimensions

Identify where along these dimensions the pivotal questions regarding the success or failure of transfer lie

Map the findings of the conflicting studies against this taxonomic framework

From Barnett & Ceci (2002)

Transfer taxonomy applied to cricket

From Müller and Rosalie (2010)

Chen & Klahr’s (1999) study of the Control of Variables strategy

Teaching elementary school children to design unconfounded tests

Short term transfer testing compared different training conditions, but for long term transfer test (discussed here) just compared those exposed to the concepts earlier to new students

Children given descriptions of various test combinations (e.g., whether or not a plant got lots of water, food and sunlight) and asked to circle whether a test was a good test or a bad test

Tested 7 months later, in a different format, and in a variety of domains

Transfer context: Chen & Klahr (1999) remote transfer test

From Barnett & Ceci (2002)

Fong, Krantz & Nisbett (1986) study of transfer of statistics coursework to

thinking about sports The transfer test was given in the guise of a survey about sports, e.g.,

“In general, the major league baseball player who wins Rookie of the Year does not perform as well in his second year. This is clear in major league baseball in the past 10 years. In the American League, eight Rookies of the Year have done worse in their second year; only two have done better. In the National League, the Rookie of the Year has done worse the second year 9 times out of 10. Why do you suppose the Rookie of the Year tends not to do as well his second year?”

Responses to this regression question were coded for the presence and quality of statistical reasoning. A typical nonstatistical response for this question would be, “The Rookie of the Year doesn’t do as well because he’s resting on his laurels; he’s not trying as hard in his second year.” A good statistical response would be, “A player’s performance varies from year to year. Sometimes you have good years and sometimes you have bad years. The player who won the Rookie-of- the-Year award had an exceptional year. He’ll probably do better than average in his second year, but not as well as he did when he was a rookie.”

Transfer context: Fong et al (1986; Expt. 4)

Context matters – to make sense of studies often need to be conscious of context

Application to video game studies – the relationship between 3D mental rotation and

women’s presence in STEM fields

Casey, Nuttal, & Pezaris (1997, 2001) studied the top third of the US college-bound sample on the SAT math test. The relationship between gender and test scores was explained roughly two-thirds by mental rotation ability

Casey and her colleagues have provided evidence of sex differences in 3-D mental rotation among kindergarteners (Casey, Andrews, Schindler, Kersh, & Samper, 2008).

Hyde (2005) synthesized 128 effect sizes from 47 published metaanalyses and found somewhat large effects for mental rotation and mechanical reasoning favoring males (d’s between 0.56 and .76)

Yet, after one semester of computer videogame training (Terlecki & Newcombe, 2005), college women almost caught up with untrained men on mental rotation. And merely playing action video games as part of a experiment can narrow gender differences in mental rotation (Feng et al, 2007)

Feng, Spence & Pratt’s (2007) study of video game transfer to spatial tasks

Training was playing a 3D first person shooter game (Medal of Honor) for several sessions in the lab

Control was playing another 3D computer game that did not involve focussed attenation on a target (a maze puzzle game)

Transfer tests were the Uniform Field of View task (UFOV) and an AutoCAD version of the Vandenberg & Kuse (1978) 3D mental rotation tasks.

Transfer tests in Feng et al video game study

From Feng, Spence & Pratt (2007)

Transfer context: Feng, Spence & Pratt (2007)

Context: where transferred from/to

Near ---------------------------------------------------------- Far

Knowledge domain

Medal of Honor to

3D mental rotation

Physical same room at lab

Temporal same session

months later

Functional both clearly

research

research vs. at play

(earlier study)

Social both individual

Modality both computer

based

So what does this mean?

Given the importance placed on 3D mental rotation ability as an explanation for females’ weaker performance on crucial STEM gatekeeper tests such as the SAT-M, is a little video game play all that’s required to remedy years of male-female disparity in spatial skills and STEM careers?

It’s important to understand the robustness and transferability of the improvement in 3D mental rotation across different contexts and content dimensions