A Temporal Model for Multilevel Undo and Redo

W. Keith Edwards……….Xerox PARCTakeo Igarashi……...Brown University*Elizabeth D. Mynatt…….Georgia Tech*Anthony LaMarca………………Yahoo!*

* PARC expatriate

The Setting

How to create applications with a rich, flexible, and interactive model of time?– Simple example: undo/redo– Not so simple: modifiable time (e.g.,

Chimera)– Not so simple: divergent time (e.g.,

Timewarp)

This Research

Deals with two issues:• Representing causal effects in

application timelines• Representing timelines that can be

“decomposed” into multiple constituent parts

StableTime

EditableTime

SimpleUndo/Redo

Chimera

Putting This Work in Context

Putting This Work in Context

StableTime

EditableTime

SimpleUndo/Redo

Chimera

Linear Divergent

VersionControl

Timewarp

Divergent

SimpleUndo/Redo

Kurlander

VersionControl

Timewarp

Linear

Stable

Editable

This Work

? ?

VersionControl

Single-level

Multi-level

Putting This Work in Context

Background: Representing TimeWith the Command Pattern

• Each node represents an atomic operation– Can be rolled forward or back– States are the “spaces” between nodes

• Anecdotally: much of the trouble of building apps in this pattern comes from the “atomic” part…

Draw A at X, Y

Move Ato X’, Y’

Resize A to W, H

0 1 2

Case Study: Flatland

Atomicity was a showstopper• User inputs “primitive” ops

– drawing and erasing strokes

• “Behaviors” act on stroke input– An example: map behavior

• The user-visible operations implicitly occur as side effects to the original input– Operations of the behaviors should be hidden

An Example: Map Behavior

Representing Cascading Operations—Three Choices

• Represent all operations in the history– …including operations put there by behaviors

• Only store original input– Requires “replaying” history to rest of application– Not self-describing, expensive, difficult search

• Create new commands representing all possible combinations– Combinatorial explosion…

We took the first approach, but…

Side Effects Become “Foreground” Effects

So now what we have is:

• “Implicit” operations are now foreground.• Atomic movement no longer gives us what we want• Need to explicitly “chunk” the timeline to represent

causality– Maintain benefits of command pattern, but hide implicit

operations

Draw NewStroke A’’

EraseIntersected

Stroke

3 4

DrawOriginalStroke A

EraseStroke

A

Draw NewStroke A’

0 1 2

Transaction Scopes

• Group related operations into causal groups

• Each original user input begins a new scope

• Transactions can be arbitrarily nested• Completion of “side effect” operations

signals the end of a transaction scope• Transactions—not individual operations—

are now the new unit of atomicity

NestingDepth

0

1

2

Start TA

Logical Timeline

End TA

Start TB Start TCEnd TB End TC

AB CTransaction

Stack

Related Work: Hierarchical Events

Some similarity to work by Kosbie and Myers (1993), but:– Difference in goals:

• Hierarchical events impose interpretation• Transactions reflect causality

– Difference in mechanism:• “Higher level” events are assembled bottom-up• Transactions collected top-down

– Difference in experience:• Hierarchical events facilitate undo at any level in the hierarchy

(user visible operations)• Causal model side effects are “implementation details” and

should not be exposed individually

…And all was fine, until we decided to add a new feature…

Boards and Segments

• Flatland UI consists of– A “root” surface– Individual “segments”

containing strokes andassociated behaviors

• Segment specific time– Traverse time in one

segment, while othersstay fixed

• Global time– Traversing global time “snaps” all segments to the

current global time

“Multilevel” Models of Time

• Each segment maintains its own “local” notion of history– Traversing time in a segment does not affect any

other segments– The local segment is in a “bubble,” separated from

other changes– Useful for long-term work in a given segment

• The board as a whole maintains a “global” notion of history– Changes to the global history “snap” all segment

timelines to the new global time– Useful for reverting to past whole-board states

Local versus Global Timelines

Local Timeline 1

Local Timeline 2

Local Timeline 3

A local history comprises only the operationsfrom a single timeline.

The global history comprises all operationsfrom all local timelines.

Multilevel Time Interacts with Transactions

• In atomic models of time, interleaving can be arbitrary– Can “jump between” local and global timelines

freely

• But this isn’t the case with richer, causal models of time– Don’t want to jump in partially completed

transactions– Some ops may have side effects in other

timelines– Potentially arbitrary interleaving of timelines

An Example

• Consider an operation in segment S1 that changes segment S2– A transaction is still open in S1…– …but a new transaction is started in S2

• Locally, timelines consist of sequential toplevel transactions

• Globally (“real” time), transactions are interwoven!

• Not unique to Flatland– Version control operations to move files to other

subprojects (atomic, in this case)

Why is This a Problem?

• While the local timelines align on transactional boundaries, the global timeline does not– Top-level transactions in distinct timelines overlap– “Real time” does not consist of sequentially ordered transactions!

Global

Local 1

Local 2

A Solution

• What “should” happen? (Example)• Within a local timeline, we’ve already got

sequential, consistent transactions• Need to “repair” global timeline in a way

that preserves causality– Turn segment-spanning operations into a single

transaction, globally– If an operation in T1 is the cause of any effects

in T2, T2 is causally dependent on T1 (globally)– (Even though, from T2’s perspective, it is a

standalone op)

A Solution (cont’d)

• Globally modify timelines to “push” side effect operations that span timelines into a higher nesting depth in the global timeline

• Reflects the sense that the “global” timeline captures cross-segment changes as single operations

But the global history is misaligned.

Without Transformation:Local timelines contain aligned,toplevel transactions.

With Transformation:Local timelines still containaligned, toplevel transactions.

AssemblyAssembly

&Transform

Contained transactions arere-nested. The global historyis now aligned.

T1

Tglobal

T2

0

0

1

0

1

1

Tglobal

0

1

T1

T2

0

0

1

1

2

3

The “Push” Transformation

Flatland Implementation• All navigation machinery stays the same

to accommodate these new semantics• Only change is a transformation executed

during the construction of the global timeline– Identify and “push” segment-spanning

causality– Global timeline derived from local timelines

• Accommodates local and global timeline manipulation

Search Over Time

• Flatland provides two time-search tools– Visual scan via scrolling– Query-based search

• Much to be gained by representing all operations explicitly in the timeline:– A stable representation of all changes– No need to “replay” computationally; just reset

the application’s state– Can pattern-match the timeline for interesting

prior states– Nodes can cache thumbnails or other

representations of state– Time-based semantic snapping

Conclusions

• Extensions to the command pattern for representing application time– Explicit representation of causal effects– Transformational model for applications with a

“multilevel” model of time

• Explicit representations afford search• Work still to be done on multilevel time:

– Editing of multilevel timelines– Divergent multilevel timelines

Local versus Global Timeline Management

In this implementation,– Traversing local timelines does not affect other

local timelines• Local timelines can be manipulated in a

“bubble”

– Traversing global timeline “snaps” all local timelines to current global time

• No isolation of individual timelines from the global time—only from each other

– The ability to freeze a segment while reverting the rest of the board could be a useful feature…but our model does not support it