Mik Kersten Thesis defense

December 15, 2006

Focusing Knowledge Work with Task Context

Problems

Information overload• Many knowledge work tasks cut across system structure• Browsing and query tools overload users with irrelevant

information

Context loss• Tools burden users with finding artifacts relevant to the task• Context is lost whenever a task switch occurs• Users to waste time repeatedly recreating their context

Thesis

A model of task context that automatically weights the relevance of system information to a task by monitoring interaction can focus a programmer's work and improve productivity.

This task context model is robust to both structured and semi-structured information, and thus applies to other kinds of knowledge work.

Approach

Memory• Episodic memory: one-shot, only single exposure required• Semantic memory: multiple exposures required

Our approach• Leverage episodic memory, offload semantic memory

Tasks: episodes

Context: weighting of relevant semantics to task

Related Work

Memory• Episodic memory: one-shot, only single exposure required• Semantic memory: multiple exposures required

Our approach• Leverage episodic memory, offload semantic memory

Tasks: episodes• Explicit tasks (UMEA, TaskTracer): flat model, lack of

fine-grained structure

Context: weighting of relevant semantics• Slices (Weiser) and searches (MasterScope): structure only• Interaction-based (Wear-based filtering): no explicit tasks

Model & Operations

Interaction

Task context

Degree-of-interest (DOI) weighting• Frequency and recency of interaction with element• Both direct and indirect interaction

Model

interest

Task context graph• Edges added for relations between elements• Scaling factors determine shape, e.g. decay rate

Thresholds define interest levels

Topology

[l, ∞] Landmark

(0, ∞] Interesting

[-∞, 0] Uninteresting

Operations

Once task context is explicit• Can treat subsets relevant to the task as a unit• Can project this subset onto the UI• Perform operations on these subsets

Composition• See context of two tasks

simultaneously

Slicing • Unit test suite can be slow to run on large project• Find all interesting subtypes of TestCase

cd

b

c

ba

T

T

More operations

Propagation• Interacting with method propagates

to containing elements

Prediction• Structurally related elements of potential

interest automatically added to task context

Only interaction stored

Implementation: programming

Implementation: knowledge work

Validation

Validation

Questions• Does task context impact the productivity of programmers?• Does it generalize to other kinds of knowledge work?

Problems• Knowledge work environment hard to reproduce in the lab• No evidence that non-experts are a good approximation of

experts• Measure long-term effects to account for diversity of tasks

Approach• Longitudinal field studies• Voluntary use of prototypes• Monitoring framework for observation

Study 1: feasibility

Productivity metric• Approximate productivity with edit ratio (edits /selections) • Programmers are more productive when coding than when

browsing, searching, scrolling, and navigating

Subjects• Six professional programmers at IBM

Results• Promising edit ratio improvement

Study 2: programmers

Subjects• Advertised study at EclipseCon 2005 conference• 99 registered, 16 passed treatment threshold

Method and study framework• User study framework sent interaction histories to UBC

server• Baseline period of 1500 events (approx 2 weeks)• Treatment period of 3000 events, to address learning curve

Study 2: results

Statistically significant increase in edit ratio• Within-subjects paired t-test of edit ratio (p = 0.003)

Model accuracy• 84% of selections were of elements with a positive DOI• 5% predicted or propagated DOI• 2% negative DOI

Task activity• Most subjects switched tasks regularly

Surprises• Scaling factors roughly tuned for study, but still unchanged

Study 3: knowledge workers

Subjects• 8 total, ranged from CTO to administrative assistant

Method and study framework• Same framework as previous, monitor interaction with files

and web• No reliable measure of productivity, gathered detailed

usage data

Study 3: results

Task Activity• Users voluntarily activate tasks when provided with task

context• Activations/day ranged from 1 to 11, average is 5.8

Task context contents• Long paths are common• Density over system structure is low• Tagging did not provide a reliable mechanism for retrieval

Task context sizes• Non-trivial sizes, some large (hundreds of elements)• Many tasks had both file and web context

Model accuracy• Decay works, most elements get filtered

Summary

Contributions

Generic task context model• Weighted based on the frequency and recency of interaction • Supports structured and semi-structured data• Weighting is key to reducing information overload• Capturing per-task reduces loss of context when multi-tasking

Task context operations• Support growing and shrinking the model to tailor to activities• Integrate model with existing tools

Instantiation of the model • For Java, XML, generic file and web document structure• Can be extended to other kinds of domains and application

platforms

Monitoring and framework • Reusable for studying knowledge work

Conclusion

Tools’ point of view has not scaled• Complexity continues to grow, our ability to deal with it

doesn’t

Task context takes users’ point of view• Offloads semantic memory, leverages episodic memory

Impact on researchers• University of Victoria group extended it to ontology browsing

Users• “Mylar is the next killer feature for all IDEs” Willian Mitsuda

Industry• “…it’ll ultimately become as common as tools like JUnit, if

not more so.” Carey Schwaber, Forrester analyst