Results Synthesis

Phuoc Lam

Schedule

What is it? How it works Benefits & Drawbacks Activity

What is it?

Paper & pencil based Stage within Heuristic Evaluation Heuristic Evaluation in 4 stages:

– Preparing for inspection– Inspection itself– Results synthesis– Communication to developers & designers

Definition

“The process of transforming the raw inspection data from Heuristic Evaluation into a complete, concise, and coherent statement of the problems in the evaluated interface as well as recommended actions to address the problems identified.”

– Cox, 1998

Developed by Donald Cox in his MSc thesis, 1998 under supervision of Dr. Saul Greenberg

Prior to this:– Approaches were haphazard– Problem reports produced often unhelpful

How it works

Collaboratively in small groups Four stages:

1. Preparation

2. Familiarizing

3. Emergence

4. Finalizing

Preparation

Ensure raw data in suitable form:– One written per piece of paper/sticky note– Paper easily stuck to work surface, grasped

(singly and in small cluster), and repositioned– Problem and heuristic stated – Readable at distance of one meter– Able to identify different inspectors

Preparation (cont)

Create workspace suitable for activity:– Whiteboard covered with newsprint– Table and chairs for sit-down discussion and work– Scissors– masking tape– note paper– sticky notes– index cards– markers– restickable glue sticks

Familiarizing

Group raw data by heuristic Stick on work surface in these groups Obvious duplicate problems stuck together Discussion strongly encouraged Review entire data set

Emergence

“the notion that fully formed, robust ideas and interpretations are not immediately obvious from raw data, but come about only after working with and discussing the data”

– Cox, 1998

Emergence (cont)

Goal: SINGLE view of interface problems that is complete, coherent, and concise

Continuously reorganizing data into groups– Label groups and subgroups

Through discussion and exploration E.g. group by “interface component”

Finalizing

Create written problem reports From each group in workspace Good problem reports:

– Detailed description in terms of users and tasks– Description of severity– Recommended solution

Problem Reports: recommendations

Problem & proposed solution, but describe separately

Provide justification (both problem & solution) Assess severity Think about trade-offs explicitly Careful consideration when adding new

functionality is proposed in solution– flag new functionality & indicate how critical it is for

immediate release of product or whether it can be held off until later release

Be aware of your evaluator/inspector bias

Benefits

Produces interface problem reports that are:– Detailed– Complete– Coherent– Concise– useful to developers & designers

Takes large number of raw problems & synthesizes into problem reports that report on underlying root problems in interface

Benefits (cont)

Eliminates duplicates Is “natural” process; only need high level

description & emergence occurs naturally Groups of varying sizes & compositions

Drawbacks & Problems

spatial environment with enough room Duplicates in data occur & can be non-trivial

to identify -> missed social process requiring privacy Large groups with differing backgrounds

can be problematic -> very time consuming

ActivityTaking the raw data produced in the heuristic inspection done last class, we will

perform a results synthesis. Because of time constraints, I have prepared the workspace prior to class. We will follow these steps in conducting our results synthesis:

• Split into three groups. Each group will have a whiteboard as their workspace.

• Familiarize:• Group all raw data cards by heuristic and stick onto workspace,

discussing with other group members. Obvious duplicates can be stuck together.

• Get a global understanding of entire data set, with group discussion.• Emergence: reorganize and regroup the data into groups that make sense

to your group. Discuss ideas, explore possibilities, and move the data around. Try to come up with a final organization where all problems have been placed in a grouping or subgrouping.

• Interpretation: write a problem report for one of the groups or subgroups in your workspace to see how this is done. Problem report should include:

– Detailed description in terms of users and tasks– Description of severity– Recommended solution

Thank You!!