Linking Cognitive Science to Education: Generation and Interleaving Effects

XXVII Annual Conference of the Cognitive Science SocietyStresa, Italy

July 23, 2005

Lindsey E. RichlandRobert A. BjorkJason R. Finley

University of California, Los Angeles

Marcia C. Linn University of California, Berkeley

Introducing Desirable Difficulties for Educational

Applications in Science

www.psych.

University of California, Los Angeles University of California, Berkeley

Robert A. Bjork Marcia C. Linn

Lindsey Richland Britte Cheng

Jason Finley

Matthew Hays

Cognition and Student Learning program:

Institute of Education Sciences Award # R305H020113

“Desirable Difficulties” (Bjork, 1994)

• Design principles that have been found, in laboratory research, to impair performance during training but enhance performance at a delay

“Desirable Difficulties” (Bjork, 1994)

• Spacing rather than massing study• Interleaving rather than blocking practice

on separate topics or tasks• Varying contextual cues• Reducing feedback to the learner• Testing rather than re-presenting

• Laboratory studies using simple material

Learning versus performance• What we can observe is performance, what

we must infer is learning…– and the former is an unreliable index of the

latter

• Instructors are, therefore, susceptible to choosing less-effective conditions of learning over more-effective conditions

• And as learners, we, too, are susceptible to confusing performance with learning

Goals of the IDDEAS project

• Do such findings extend to realistic educational materials and contexts?

• And, more broadly, what design principles are fundamental in optimizing educational materials and practices?

WISE (Web-based Inquiry Science Environment)

http://wise.berkeley.edu

• A software system for science instruction• Advantages as a tool for teachers

– Supports authoring and customization

– Contains a library of tested projects

– Transportable

– Gathers embedded assessments of student progress

• Advantages as an IDDEAS research tool

On-going studies

WISE Platform

• Laboratory studies, UCLA

• Classroom studies, UCB

Design Principles

• Laboratory studies, UCLA

Presentation Order

Example Performance during training

Performance at a delay

Blocked A,A,A,A,A,B,B,B,B,B better worse

Interleaved A,B,B,A,B,A,A,B,A,B worse better

Interleaving Effect

(e.g. Shea & Morgan 1978)

Interleaving

• Motor tasks: patterns, force production, bank machine transactions (Lee & Magill, 1983, Simon & Bjork, 1990 Charles et. al, 1990, Jamieson & Robers, 2000)

• Sports: badminton, volleyball, baseball (Bortoli et al, 1992, Goode & Magill, 1986, Hall et al, 1994)

• Abstract learning tasks: mazes, tracking (Carleson et.al, 1989, Jelsma & Van Merrienboer, 1989, Jelsma & Pieters, 1989)

• Logic rules, boolean operators (Schneider et al, 1995, Carleson & Yaure, 1990)

Generation effects: Example with simple Laboratory materials

(Hirshman & Bjork, 1988)• Read condition:

– Presented: Bread: Butter – Participant responds: “Bread Butter”

• Generation condition:– Presented: Bread: B_tt__– Participant responds: “Bread Butter”

• Read condition: at 30 min. delay– Presented: Bread: Butter

– Participant responds: “Bread Butter”…………13%

• Generation condition:– Presented: Bread: B_tt__

– Participant responds: “Bread Butter”…………41%

Recall of “Butter”

Undergraduate Laboratory Experiment 1

• Explore interleaving and generation effects with realistic educational material

• In a controlled laboratory environment

Undergraduate Laboratory Experiment 1

• Participants: 83 UCLA undergraduates• Material

– Adapted an existing WISE module about habitability of other planets

– 2 Sets of Information to be learned in WISE module (divided into single slides):

• Mass of a Planet

• Distance of a Planet from its Sun

• Procedure:– 1 hour spent on WISE module

– Post-test at 2-day delay

Undergraduate Laboratory Experiment 1

• 2 x 2 Between-subjects

• IV1: Presentation Order: Blocked vs. Interleaved– MMMMMDDDDD vs.– MDDMDMMDMD

• IV2: Embedded Study Events: Read vs. Generate– “Mercury’s mass is less than Earth’s mass.” vs.– “Mercury’s mass is ___ than Earth’s mass.”

Exp. 1 Post-test, 2 Day Delay• Simple sentence-completion: Information that was

presented and re-studied via either generation or reading– E. g. “The amount of heat and light emitted by the sun in our solar

system has increased by ____% since the beginning of earth’s history.”

• Integration questions within a topic (mass or distance)– E. g., “Would an object weigh more on the planets in our solar system

made mostly of gas or made mostly of rock? Why?”

• Integration questions across both topics (mass and distance)– E. g., “Imagine a planet that is smaller than Earth and that was located

1.5 AUs from its sun, which is the same strength as the Earth’s sun. How would this planet’s potential for life compare to Earth’s?”

Exp. 1: Read vs. generate: performance on sentence completions

50

55

60

65

70

75

80

85

90

95

100

Instruction Memory Posttest

% Correct

Read Condition

Generate Condition

Exp. 1: Blocked vs. Interleaved by type of Posttest Integration Question

Undergraduate Laboratory Experiment 2

• Participants: 44 UCLA undergraduates

• Same material

• All interleaved order

• Sentence level generation questions– E.g., “Describe in a sentence how the size of one

planet's mass can affect another planet.”

Undergraduate Laboratory Experiment 2

• Between-subjects• IV: Knowledge required for successful

generation:– Within-topic integration [e.g. Mass only] vs.– Between-topic integration [Mass + Distance]

• DV: Post-test at 2 day delay

0

10

20

30

40

50

60

Instruction Transfer to NewSingle Concept

Questions

Transfer to NewConcept Integration

Questions

Mean Performance

Single Concept Generation

Concept Integration Generation

Single Topic Generation

Topic Integration Generation

Transfer to NewSingle TopicQuestions

Transfer to NewTopic Integration

Questions

Exp. 2: Single-topic vs. Topic-integration generation

Undergraduate laboratory studies Reflections

• Findings promising but not straightforward– Possible benefits of interleaving– Definite benefits of generation for a specific item– Possible broader benefits of generation

• Many questions remain– Repetitions– Relatedness of topics– Feasibility

• Extending the chain of evidence• Parallel Studies: laboratory & classroom

Undergraduate laboratory studies Reflections

• Advantages of Collaboration (complex systems approach)– What is realistic?– What is the goal of our research?

• Technology as bridge & Catalyst

For More Information

• IDDEAS : http://iddeas.psych.ucla.edu• WISE : http://wise.berkeley.edu

• Jason Finley: jfinley@ucla.edu

Exp. 2: Illustrative responses

Single idea Generation

Single concept generation

Concept integration generation

Prompt: The range of distances from the sun where the temperature allows water to be liquid are called the _____ _____.Student: “habitable zone”

Prompt: Scientists often use a single measurement to talk about a planet's distance from its sun, but why is this practice misleading? Use a distance listed in the table you saw to explain.Student: “The distance for Mars would be misleading because Mars travels in an elliptical orbit and is different distances from the sun at different times.”

Prompt: Using Jupiter's distance from the sun as an example, explain how the measure of an object's weight can shift when it is in different locations, even if that object is a planet.Student: “An object's weight can shift when its in different locations because it's weight depends upon the strongest pull of gravity. People weigh more on Jupiter than they do on Earth because Jupiter's gravitational pull is stronger. If the object is a planet, then changing the distance it is from the sun will change it's weight because it will either feel a strong gravitational pull (if close to the sun) or a weak gravitational pull (if further away from the sun).”