Screen design: the effects of background color on posttest performance

Roy B. Clariana (RClariana@psu.edu)

Penn State University

A poster presented at the Annual Meeting of the Association for Educational Communications and Technology (AECT) in

Atlanta, GA on November 15, 2002

Slides at: http://www.personal.psu.edu/rbc4

Underlying principle . . .

• Lesson color becomes associated with lesson content, thus becoming part of the posttest retrieval mix

• “…color facilitates information processing by providing structures for the storage of new information and by providing categories by which familiar or already existing information can be stored.” (Dwyer, 1978, p.142)

• In other words, color (and other context variables) may influence the structure of the stored knowledge, and thus also retrieval of TBR content

memorize a list of words, AI

words a b c d e f g h Ia 1 0 0 0 0 0 0 0 0b 1 1 0 0 0 0 0 0 0c 0 1 1 0 0 0 0 0 0d 0 0 1 1 0 0 0 0 0e 0 0 0 1 1 0 0 0 0f 0 0 0 0 1 1 0 0 0g 0 0 0 0 0 1 1 0 0h 0 0 0 0 0 0 1 1 0I 0 0 0 0 0 0 0 1 1

Link Array(asymetric)

I

H

G

F

D

C

B

A

E

MDS

Connectivity Matrix (Kintsch, 1998)

My feedback research describes how to turn a 0 into a 1 (delta rule) 0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.40 0.50 0.60 0.70 0.80 0.90

Lesson item difficulty (p)

Post

test

item

dif

ficu

lty (

p)

InferentialVerbatim

1.00

Lesso

n = Pos

ttest

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.40 0.50 0.60 0.70 0.80 0.90

Lesson item difficulty (p)

Post

test

item

dif

ficu

lty (

p)

InferentialVerbatim

1.00

Lesso

n = Pos

ttest

ih

g

f

e

d

c

ba

i

h

g

f

d

c

b

a

e

MDS of asymetric vs. symmetric arrays

A side issue, if you learn the list forwards and backwards, the resulting node distances are nearly the same

Learn same list AI but add color

words a b c d e f g h Ia 1 0 0 0 0 0 0 0 0b 1 1 0 0 0 0 0 0 0c 0 1 1 0 0 0 0 0 0d 0 0 1 1 0 0 0 0 0e 0 0 0 1 1 0 0 0 0f 0 0 0 0 1 1 0 0 0g 0 0 0 0 0 1 1 0 0h 0 0 0 0 0 0 1 1 0I 0 0 0 0 0 0 0 1 1

blue 1 1 1 0 0 0 0 0 0red 0 0 0 1 1 1 0 0 0

green 0 0 0 0 0 0 1 1 1

Link Array(asymmetric)

I

H

G

F

E D

C

BA

MDS

blue red green

blue

red

green

Compare effects of color

I

H

G

F

E D

C

BA

I

H

G

F

D

C

B

A

ENo color Color

blue

red

green

?…. color alters memory structure in a regular way,and we can think about it visually.

Planned investigations

• Study 1 (submitted to BJET) – Do participants remember the lesson color scheme at posttest (MC lesson items compared to CR lesson items)?

• Study 2 (presented at ICCNS) – Does lesson feedback on error influence participants memory of the lesson color scheme at posttest?

• Study 3 (this presentation, IJIM 31(4) ) – Does matching lesson color at posttest influence memory of TBR content (CR vs. MC)?

• Study 4 (in progress) – Can any effects of lesson color be shown in semantic maps of structural knowledge?

Study 1 – An interaction of screen color and lesson task *

• Graduate students (n = 68) completed a computer-based vocabulary lesson that included either multiple-choice or constructed-response study tasks with feedback. All study tasks used identical color and each lesson section used a different color theme. Posttests also included both CR and MC, but the posttest did not use color. (a 2 x 2)

• Specifically, do participants remember lesson color at posttest?

(*submitted to the British Journal of Educational Technology)

Why use CR and MC?

• Recall and recognition are likely to be distinct declarative knowledge learning outcomes (Jonassen & Tessmer, 1996).

• McDaniel and Mason (1985) have reported that recall tasks (CR) elaborate existing memory traces of the to-be-remembered (TBR) content, providing richer and more meaningful integration with existing semantic memory. In contrast, recognition tasks (MC) strengthen existing TBR traces including non-semantic and contextual information.

Why CR and MC?

• Similarly, Berg and Smith (1994) suggest that multiple-choice questions tend to elicit surface-level, superficial first reactions that often limits further interpretations, while constructed response questions force students to think through various alternate interpretations before responding.

• Thus, constructed-response study tasks may be more likely to elicit elaboration of the TBR content, while multiple-choice study tasks tend to elicit verbatim encoding including co-occurring surface-level (e.g., perceptual) contextual information.

Content – five sections

• Overview – items 1 through 9 (orange, hexadecimal RGB value #FF6531),

• Instructional Analysis – items 10 through 16 (yellow, #FFFF00),

• Learner Analysis – items 17 through 22 (blue, #0030FF), • Objectives and Tests – items 23 through 27 (green,

#00FF00), and • Development and Evaluation – items 28 through 36

(purple, #9C0063). These color hues are not ordinarily associated with these terms or sections (such as using red to indicate a concept such as “stop”).

(Dick & Carey, 1996)

Lesson and posttest screen

Lesson screen Posttest screen

Study 1 – results

• Posttest memory of the lesson color scheme was significantly better for the MC study task treatment, with a mean of 9.8 (27%) compared to the CR study task mean of 8.1 (23%), chance guessing = 7.2 or 20%

• Further, the TBR intervening posttest format also significantly impacted posttest memory of color (MC +, CR -), even though the color was not re-presented…

• Apparently, constructed-response and multiple-choice tasks form distinctly different memory traces, with context variables such as color more likely to be explicitly encoded or strengthened under MC study task an intervening posttest conditions.

Study 1 – explicit/implicit

• Apparently, any exposure to constructed response, whether during the lesson or at posttest, inhibited explicit memory of the lesson color scheme.

• However, later investigations suggest that color has an implicit effect with constructed response. Specifically, with CR, the learner can’t explicitly remember the colors at postest, but the presence of lesson color at posttest nevertheless boosts TBR posttest performance.

• Petrich and Chiesi (1976)

reported significantly greater

free recall for red over green

background color in a P-A task.

• TBR memory at posttest

• Color hue memory at posttest

Hue as a confounding variable

yellow (0.27) = blue (0.27) > orange (0.25) > purple (0.24) > green (0.20)

yellow (0.59) < blue (0.70) < orange (0.76) < purple (0.78) < green (0.86)

(note: lesson item difficulty in each hue was not controlled)

0.21

0.23

0.25

0.27

0.29

0.5 0.6 0.7 0.8 0.9

r = -0.88

Hue/TBR correlations by treatments

0.15

0.20

0.25

0.30

0.35

0.40

0.30 0.40 0.50 0.60 0.70 0.80 0.90

MC st r = -0.40CR st r = 0.09

CR posttest

0.15

0.20

0.25

0.30

0.35

0.40

0.70 0.80 0.90 1.00

MC st r = -0.74

CR st r = -0.99

MC posttest

TBR average TBR average

Hue

sco

res

Hue

sco

res

It looks like the ‘negative’ effect of explicit posttest memory of hue is most obvious on MC posttests

So…

• Following McDaniel & Mason, Study 1 shows the “kind” of activity that occurs in the lesson influences memory of color context (MC better).

– But many of the previous color context studies require only reading. Just reading may not elicit the right kind of mental activity for including color in the memory mix.

– Thus, previous research using reading study tasks can only generalize to other reading settings, and should not be generalized to the kinds of interactions that are available in computer- and web-based lessons.

• MC TBR posttest Qs elicit memories of color while CR TBR psotest Qs do not.

• Simply thinking about MC TBR Qs boosts later color memory, even if the color is not present

• Some weird inverse relationship between color and TBR, and hue may be important too

Study 2 – The effects of feedback on memory of lesson color context *

• Does feedback on error only impact TBR content, or does feedback impact all contiguously active traces? This has rather important connectionist implications, since back-propagation requires specific assignment of error while a generic regional model would allow for ‘regional’ error assignment.

• Graduate students (n = 35) completed the same computer-based vocabulary lesson as Study 1, etc…

(*Presented at the 6th International Conference on Cognitive and Neural Systems (ICCNS), May 30, 2002, at Boston College, Boston, MA)

0.000.100.200.300.400.500.60

0.00 0.20 0.40 0.60 0.80 1.00

0.000.100.200.300.400.500.60

0.00 0.20 0.40 0.60 0.80 1.00

CR posttest

CRst CRt

MCst CRt

0.000.100.200.300.400.50

0.60

0.00 0.20 0.40 0.60 0.80 1.00

MCst MCt

CRst MCt

MC posttest

0.000.100.200.300.400.500.60

0.00 0.20 0.40 0.60 0.80 1.00

L=1 (lesson correct)L=0 (lesson error)

Lesson p Lesson p

Lesson p Lesson p

Hue

mem

ory

Hue

mem

ory

Hue

mem

ory

Hue

mem

ory

Study 2

• Posttest explicit memory of lesson color was better on the MC posttest, for lesson errors

• Does feedback have a “specific” or a “regional” effect? This data suggests that feedback has a regional effect during encoding; during a lesson error, the TBR content and the color context information are both strengthened, BUT more importantly, explicit color context memory may not be encoded when the lesson item is correct.

• Discuss connectionist implications for backprop and for ISD feedback research…

Study 3 – The instructional effects of matching or mismatching lesson and posttest screen color

• Graduate students (n=34) completed the same MC (no CR this time) vocabulary study task with the same screen color-coding of lesson sections.

• Multiple-choice and constructed response posttests included either color that matched the lesson color, or no color.

• Including matching color during posttest enhanced constructed-response and inhibited multiple-choice posttest performance for difficult items relative to the no-color posttest. (*In press, the International Journal of Instructional Media 31(4))

Mixed ANOVA

• 2 (Context, color or no color) X 2 (Format, constructed-response and multiple-choice) X 2 (Difficulty, easy and difficult)

• …of most interest, the 3-way interaction of Context, Format, and Difficulty, F(1, 32) = 4.224, MSe = 0.921, p = .048 was significant.

Study 3 – chart of the significant three-way interaction

3.0

4.0

5.0

6.0

7.0

8.0

9.0

colorgroup

no colorgroup

CR (difficult)

CR (easy)

MC (difficult)MC (easy)

Pos

ttest

(9

max

imum

)

Providing matching color at posttest had a positive effect on posttest memory of TBR content for difficult CR posttest items, but not on MC nor easy CR… somewhat consistent with the outshining hypothesis

Lesson is onlyMC with color

Same data, combined easy and difficult

0.50

0.60

0.70

0.80

0.90

color

no color

0.58

0.77

0.55

0.86

CR test MC test

Again, when posttest color matches lesson color, it helped CR posttest scores a little and hurt MC posttest scores (relative)

Lesson is onlyMC with color

Pos

ttes

t (as

dec

imal

)

(p = 0.08)

Huh?

• Roediger and Guynn (1996) suggest that recognition (MC) involves only item specific processing, also called “familiarity”, while recall (CR) involves item specific as well as relational processing.

• Apparently, color context mainly supports relational processing, and so positively affects recall, but with recognition retrieval, color traces are also activated and then compete with TBR traces in some way.

Study 4

• Can we measure the effects of lesson color using semantic maps of structural knowledge?

• If different color hues are better remembered than other hues, then the “strongest” color memory will have the greatest effect on TBR content …

• 2 (MC lesson with color or no color) X 2 (CR and MC test with color or no color)

• Posttest data – TBR MC and CR scores and semantic maps

Weak effects of color?

I

H

G

F

E D

C

BA

I

H

G

F

D

C

B

A

E

IH

G

F

D

C

A

E

B

(color = 0.10 strength)

No color Color

Strong effect

Weak effect

So color and no color treatments MDS may look something like this

I

H

G

C

BA

blue

red

green

F

DE

‘strong’‘strong’

‘weak’

See if hue memory strength can predict MDS strong or weak clustering

I

H

G

F

D

C

B

A

E

No color Color

How to collect the MAP data

• ConceptMapper software (link)

• www.personal.psu.edu/rbc4

36

35

34

33

32

31

30

29

28

27

26

25

24

23

22

21

20

19

18

17

16

15

14

13

12

11

10

9

8

7

6

5 4

3

2

1

Groups:1 – 1-7 orange2 – 8-16 yellow3 – 17-22 blue4 – 23-27 green5 – 28-36 purple

no color group average MDS

yellow (0.27) > blue (0.27) > orange (0.25) > purple (0.24) > green (0.20)

36

35

34

33

32

3130

29

28

27

26

25

24

23

2221

20

19

1817

16

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

Groups:1 – 1-7 orange2 – 8-16 yellow3 – 17-22 blue4 – 23-27 green5 – 28-36 purple

color group average MDS

yellow (0.27) > blue (0.27) > orange (0.25) > purple (0.24) > green (0.20)

The problem with MDS

• MDS lets us “see” the associations, but the inferential statistics to say something is significant or not isn’t available

• We’ll probably use average distance within each hue, that gives us a good old-fashion data set that we can use MANOVA etc. on

Summary of the main points

• Study 1 (submitted to BJET) – lesson color scheme is better remembered (explicit) at posttest with MC lesson items compared to CR lesson items, and with MC TBR posttest

• Study 2 (presented at ICCNS) – explicit lesson color is only remembered when lesson MC response is an error

• Study 3 (this presentation, IJIM 31(4) ) – lesson color re-presented at posttest supports CR posttest performance and inhibits MC posttest performance (memory context effect)

• Study 4 (in progress) – ??

Conclusions

• Color context research is important to the instructional design practitioner. But most previous research in this area involved only reading tasks, and did not require other types of lesson interactions that are common in computer-and web-based instruction.

• at posttest, color memory traces may be competing with or inhibiting TBR content traces (MC), or vice-a-versa

• The use of color in screen design is common practice, yet much works remains to specify the effects of color on both encoding and retrieval.