April 19, 2023

Using a Multi-representational Design Framework to Develop

and Evaluate a Dynamic Simulation Environment

Shaaron AINSWORTH & Nicolas VAN LABEKEUniversity of Nottingham

{sea,nvl}@psychology.nottingham.ac.uk

2

Why do we need a framework?Many multi-representational systems (e.g. FunctionProbe, StatPlay, spreadsheets, www, multi-media).Tabachneck, et al (1994) found that students who used more than one rep were twice as successful at algebra.Ainsworth et al (1998) found that presenting children with a place value and a table improved maths performance.Mayer & Anderson (1991) paired animations with narrations and text to improve performance.

Yerushalmy (1991) taught 14 yr olds functions. Only 12% of students’ answers involved both visual and numerical reps. Resnick & Omanson (1987) taught children to subtract using Dienes blocks and conventional symbols. It did not help eradicate bugs.Van Somerman & Tabbers (1998) found that qualitative reps did not help learners solve quantitative physic problems.Gruber et al (1995) found that adding multiple perspectives to an economics simulation was harmed learners’ performance.

3

The DeFT Framework

DeFT (Design, Functions, Tasks): Provides a conceptual framework for describing the issues unique to learning with more than one ER. Three aspects of learning with MER Cognitive tasks Functions of MERs Design Parameters

Aims To describe systems To explain conflicting results To guide experimentation To design systems To develop design principles

4

DeFT - Tasks

When learning with presented given ERs1. the properties of the ER2. the relation between the ERs and the domain

When learning with a choice ERs3. how to select appropriate ERs

When learning with self-constructed ERs 1 & 2 & (3) +4. how construct an appropriate ER

When learning with multiple ERs 1 & 2 & (3) & (4)5. how to translate between ERs

DeFT - Functions

StrategiesIndividual

Differences

Tasks

Complementary Roles

Constrain Interpretation

Construct Deeper Understanding

Different Processes

Different Information

Constrain byFamiliarity

Constrain by Inherent Properties

AbstractionExtension

Relation

FUNCTIONS

6

DeFT – Design Parameters: Information and Form

Information. Information can be distributed in different ways between the ERs which influences the complexity of the ER and the redundancy of the system. Many studies have shown its not wise to unnecessarily split

information across MERs (e.g. split attention studies) but sometimes a single ER can become very complex or contain information which is best expressed in different ways.

Form: A multi-representational system can contain representations of different computational properties (e.g. heterogeneous systems, multi-modality systems, multi-dimensional systems). Particular benefits may accrue from different approaches

(e.g. Barwise & Etchemendy 1992; Schnotz, 2001, Mayer, 1997) but also particular problems (e.g. Ainsworth et al, 2002; Moher et al, 1999)

7

DeFT – Design Parameters: Information and Form: Translation and Number

Translation: The degree of support provided for mapping between two representations, ranging from no support through to highlighting and on to full dyna-linking where behaviour on one representation is reflected onto another. Some people recommend dyna-linking (e.g. Kaput, 1992). Ploetzner, Bodemer, & Feuerlein (2001) proposed an approach

based on structure mapping where learners are encouraged to map familiar aspects of an ER onto an unfamiliar ER.

Van-Labeke & Ainsworth (2001) base their approach on scaffolding theory (contingent translation) which fades the degree of system support as the learner experiences grows (supported by Seufert, 15 minutes time).

Number: By definition, a multi-representational environment uses at least two ERs, but many systems use more than that. A related issue is how many ERs to use simultaneously?

8

DeFT – Design Parameters: Sequence

Sequence: Many systems present only a subset of their ERs at a time; consequently further decisions must be made. The order in which the ERs should be presented. e.g. teach integration before differentiation and so velocity-time

before position-time). e.g. qualitative representations to guide subsequent interpretation

of quantitative (Plötzner, Fehse, Kneser, & Spada (1999) e.g. concrete -> abstract or Verdi, Johnson, Stock, Kulhavy, & Ahern,

(1997) graphical before textualWhen to add a new ER Before knowledge has become proceduralised (Resnick & Omanson,

1997) but not so early that learners become overwhelmedWhen to switch between the ERs e.g. when a learner understands the relations between ERs e.g. judicious switching not thrashing (Cox, 1996, Anzai, Tabachneck

et al, 1994).

9

DEMIST

DEMIST is a simulation learning environment in the area of population dynamicsIt provides full flexibility for manipulating the design parameters of DeFTDEMIST supports additional activities Hypothesis on future values, action on the

current values

11

Pilot Study

Experiment on 3 models of population dynamicsParticipants 18 University UGs – no biologists or

mathematiciansMultiple-choice Pre-test and Post-test Conceptual Single Representation Multi Representations

Procedure One hour to explore the 3 models

12

Example (Concept – SSUG)

One of the following types of population will double in a fixed amount of time. Is it

A Prey in the predator-prey model

B Predators in the predator-prey model

C A single species showing unlimited growth

D A single species showing limited growth

13

Example (Single – SSLG)

Given this graph of population growth rate against population density (dN/dT v N), on which point is population growing fastest ?

14

A B

C D

Example (MERs – TSP)

Three of these graphs were generated from the same predator and prey model and one was not. Which one is it?

15

Design Decision

InformationRepresentations created with 1 to 3 dimensions of information. Pairs of representations could therefore have full, partial or no redundancy

FormLarge representational system (8 - 10 ERs for each units), with different computational properties, selected to vary in their relevance and ease of interpretation.

Sequence Learner choice of sequence of ERs and when to swap.

NumberA maximum of 5 co-present representations. A small number of representations selected to be initially displayed.

Translation Dyna-linking so learners can reflect actions onto all ERs.

Design Decisions

16

Pre-test / Post-Test Results

Average pre-test score above chance (p<0.001) but MERs below chance (p=.024)Significant performance increase (p<0.008)

0

10

20

30

40

50

60

70

Pre-Test(11 items)

Post-Test(10+12 items)

% of correct answer

Concept

Single ER

MERs

0

10

20

30

40

50

60

70

Pre-Test(11 items)

Post-Test(10+12 items)

% of correct answer

Concept

Single ER

MERs

17

Categories of ERs in DEMIST

Description No.

X v Time Graph Line graph of data across time 15

X v Time Graph (log)

Logarithmic scaled line graph 2

XY Graph Line graph that plots two dimensions of data where one is not time.

11

Chart Two-dimensional bar chart 10

Pie Chart Proportions of two or more values 3

Concrete Animation

Dynamic ER with a pictorial element 7

Table Tabular representation 11

Dynamic Equation Dynamic ER that contains explicit mathematical expressions 9

Terms Dynamic ER with explanatory text and often a current value 3

Value A very simple representation that provides only a data label and value

9

80

18

Users’ Traces

Unit 1 – 08:34

00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00

Controller

Value: N

Chart: N

Graph: N v Time

Table: N

New Terms

Dynamic Equations

Graph: Ln(N) V T

Graph: N V T (Log)

Graph: N v (dN/dT)

Controller

Map Relation

Action & Hypothesis

Experimental Set

19

Which Representations are used?

Large exploration of the representational space (73 out of 80 ERs available) but unequal use of ERs

0

10

20

30

40

50

60

70

80

X v TimeGraph

Terms Value Chart XYGraph

ConcreteAnimation

Table DynamicEquation

PieChart

X v TimeGraph(log)

% of potential use

0

10

20

30

40

50

60

70

80

X v TimeGraph

Terms Value Chart XYGraph

ConcreteAnimation

Table DynamicEquation

PieChart

X v TimeGraph(log)

% of potential use

Striking correlation between our provision of ERs and the learners’ preferred ones (p < 0.02 )

20

Acting on Representations

Representations are used for display to request translation or predict a value at some future point

0

100

200

300

400

500

600

X v TimeGraph

Terms Value Chart XYGraph

ConcreteAnimation

Table DynamicEquation

PieChart

X v TimeGraph(log)

Total number of events

Translation

Hypothesis

0

100

200

300

400

500

600

X v TimeGraph

Terms Value Chart XYGraph

ConcreteAnimation

Table DynamicEquation

PieChart

X v TimeGraph(log)

Total number of events

Translation

Hypothesis

Hypothesis only from the X-Time Graph 59 Translation requests, more than expected from XY, Log and Table

21

Relationship between ER use & performance

No significant relationship between use of representations (number seen, number co-present, time spent with a particular representation) and; Pre-test scores Post-test scores Prior experience with maths/biology Stated preference as to visualiser/verbaliser

22

DEMIST One:Conclusions and next steps

Need for fine-grained protocols to gain insight into the processes involved in learning with multiple representations.

In particular, how do learners’ goals, decisions and strategies influence their use of representation. E.g. Does spending a long time working with an ER indicate

knowledge or ignorance

Systematic variation of some of the design parameters (e.g. 5 co-present ERs v 1 ER of the 5 at a time)

Keep on reading all of your papers to see if your results support my hypotheses! (describe your system according to DeFT at http://www.psychology.nottingham.ac.uk/research/credit/projects/multiple_representations/deft_systems/)