CS6604 Spring 2012Notes on

Algorithm Visualization

Clifford A. ShafferDepartment of Computer Science

Virginia Tech

“State of the Field”

• Hundreds of visualizations are freely available on the Internet

• Studies on the effectiveness of AVs– Many studies show no significant difference– But AVs have been shown to help in some

implementations– One conclusion is that creating/using effective AVs

is possible but not easy

• Many faculty wish to use Avs – but there is not as much use as this would indicate.

What AVs are Available?

• A collection of links available at http://algoviz.org• Links to over 500 visualizations• Nearly all AVs now written in Java

– Applets vs. applications• Stand-alone vs. collections

Who Makes Them?

• Single authors, one-off implementations (1-5)– 30%

• Small shops, sustained over a few years– Typically a faculty member and a few students– 5-10 visualizations– 10%

• Larger teams, longer term investment– Team built, maybe funded– 25%

• Major Projects– integrated package or shared look-and-feel– 35%

Is There Adequate Coverage?• No

– Sorting, search trees, and linear structures overwhelmingly dominate

– Coverage for more advanced topics is spotty

134

60

4638

29

14 13 12

4

0

20

40

60

80

100

120

140

Sort A

lgor

ithm

s

Searc

h Stru

ctur

es

Linea

r Stru

ctur

es

Graph

Algo

rithm

s

Other

Alg

orith

ms

Other

Dat

a Stru

cture

s

Compr

essio

n Alg

s

Searc

h Algo

rithm

s

Mem

ory M

anag

emen

t

What Is Their Quality?

• A majority have no pedagogical value– These give the user no understanding of how the data

structure or algorithm works– Will be of little use in the classroom

• We would recommend less than one quarter of what we have seen for any purpose

• Even the better visualizations usually have serious deficiencies– Animation only: Users are passive observers– Tree structure visualizations tend to show what

happens, but not how– Limited interactivity

Is the Field Improving?

• Pros:– A growing body of literature on best practices

to create effective AVs– Community starting to organize (AlgoViz)

• Cons:– Recent projects are no more in tune with

coverage gaps than old projects– No apparent movement in creating

repositories

Is the Field Active?• Appears to be a reduction in “one-off”

development. (Drop in student projects)– Fewer CS students– Less interest in Java– But these trends might reverse

• But steady activity in the larger groups.

AVs: The Problem• AVs have high faculty and student

favorability ratings

• But most faculty don’t use them much in courses

Informal Survey Results• Warning: Self-selected responders

• Are AVs useful?– Strongly Agree: 12– Agree: 17– Neutral: 1

• A (bare) majority indicated that they used some sort of visualization with class

Survey: Impediments to Use• Lack of knowledge/time to find good

AVs: 13

Survey: Impediments to Use• Lack of knowledge/time to find good

AVs: 13

• Time to make good AVs: 2

• Difficulty integrating in class: 9

• Lack of time within class constraints: 2

• Uncertainty about quality outcomes: 1

• Content not relevant to my classes: 1

Overcoming Impediments

• Reassurance about what AVs are good

• Ideas on how to use AVs

• Reassurance about how a given AV can be used successfully in class

• Ability to connect to developers

AVs: The Solution is Community• http://algoviz.org/

– Build a community of users/developers– Better disseminate best practices

information

• Project Support– NSF CCLI grant– NSF NSDL grant– Connections to NSDL/Ensemble project

AlgoViz.org• A collection of links to over 500 AVs

• Annotated bibliography of over 500 research papers

• Forums, field reports

• OpenAlgoViz

Are AVs of Pedagogical Value?

• Instructors generally think so

• Students usually say they “like” them

16

Metastudy: 2002

• Reviewed 24 prior studies on pedagogical effectiveness related to AVs– Generally of an individual system or AV

• Results of 24 studies:– 11 found significant (positive) results– 10 did not find a significant result– 2 entangled prediction with visualization– 1 study found a negative result!

17

Epistemic Fidelity Model

• There is an “objective truth”

• Experts carry a model of this truth in their heads

• For data structures, graphics are especially helpful in representing this model

• Therefore AVs should be especially helpful in transferring this model to students.

18

Cognitive Constructivism

• Individuals construct their own knowledge from subjective experiences

• When they become engaged in learning, they actively construct new understandings from new experiences

• Therefore, passively watching AVs won’t have much effect– Students must become actively engaged– The technology should be a tool for knowledge

construction.19

Classification

• The studies represented a wide range of activities and methods

• Looking deeper, reclassify the independent variables:– Epistemic Fidelity: 10– Cognitive Constructivism: 14– (others too few to measure)

• CC has the highest percentage of positive studies

20

Results

• CC: 71% statistically signficant

• EF: 30% statistically significant

21

CC Activities

• Construct own input sets

• Make predictions about future states

• Program the algorithm

• Answer questions about the algorithm

• Construct own visualization

22

Level of Effort

• Compared whether the two treatments required similar “cognitive effort” vs. different levels of effort– Equivalent effort: 33% significant– Not equivalent: 71% significant

• Construct algorithm/visualization takes time• Note that just taking time need not correlate to

learning

23

Procedural vs. Conceptual Knowledge

• Procedural only: 67% [10/15]

• Procedural and Conceptual: 67% [2/3]

• Conceptual only: 38% [3/8]

24

Study Measures

• Post-test only: 54%

• Pre- to Post-test difference: 78%– But most of these studies came from one

source

25

Study Conclusions

• How students use AV is more important than what they see

• Pre-test/post-test experiments on procedural knowledge show most improvement

• Technology is effective when it is used for active engagment

26

Bloom’s Taxonomy

• Knowledge (facts)

• Comprehension (of the facts)

• Application (mechanically use the facts)

• Analysis (interpreting the facts)

• Synthesis (using facts at higher level)

• Evaluation (ability to make judgments)

27

Engagement Taxonomy

• Naps Working Group 2002– No viewing– Viewing– Responding– Changing– Constructing– Presenting

• Relates to Bloom’s Taxonomy28

Extended Engagement Taxonomy

• Myller, et al.– No viewing* (textbook)– Viewing* (video)– Controlled Viewing (slideshow)– Entering Input (Define the input to execute)– Responding* (answer questions)– Changing* (direct manipulation)– Modifying (Modify existing AV)– Constructing* (create the AV)– Presenting* (Teach the material)– Reviewing (Give a review of AV)

29

2009 EvaluationUrquiza-Fuentes/Velazquez-Iturbide

• Analyzed 33 successful evaluations• Evaluation:

– Usability (half of evaluations – often shallow)

– Learning outcomes (other half)

• Many studies compared Viewing, Changing, or Constructing vs. Non-Viewing

• A few compared Changing or Constructing vs. Viewing

• Learning improvements in 75% of studies30