the end of pointing and clicking: improving computer access with goal crossing

1

The End of Pointing and Clicking: Improving Computer Access with Goal Crossing

June 30, 2009

2

People

3

Design CriteriaEase of motor-impaired performanceHow easy is it motor-wise to acquire the target? Efficiency?No tNo multiple stepsSpeed for able-bodied?Speed for motor-impaired? Safety (Unwanted targets should not be accidentally triggered)How easy/difficult is it to make a false selection?Tolerance to movement error?Is the design reversible? Is it cancelable? Can the user abort? ScalabilityDoes it handle high-density layouts?How does it scale to hundreds of menu items, links, buttons, and icons?Is it compatible with conventional point-and-click widgets? Can both coexist? Etc.Are the benefits of crossing preserved (e.g., “follow-through,” “aggregation”)?Is it easy to learn? Does it require preexisting knowledge? How hard will it be to implement?Is heavy animation required for the technique to work? Can it lend itself to a nice visual design?

4

1. Basic Crossing Ideas

1.1 Cross While Off-hand Button Held Down1.2 Cross and Click Anywhere1.3 Drag and Cross1.4 Cross and Cross-back1.5 Delta t

5

1.1 Cross While Off-hand Button Held Down

OK

6

1.2 Cross and Click Anywhere

OK

Click!

Click!

7

1.3 Drag and Cross

OK

Drag & Cross

8

OK

OK

OK

OK

OK

1.4 Cross and Cross-back

OK

The target gives visual feedback

Cross goal to activate

Cross back toconfirm selection

Cross back toconfirm selection



Activated Selected Double clicked (or)Drag (or)Right clicked…

9

1.4 Cross and Cross-back (cont.)

OK

OK

Then back over, turn 90-degrees and then cross the two bars

Activate an icon by crossing over icon and initial bar

Byron McMullen . Jacob Nelson . Tara Tarjoto . Tim Wibowo

10

1.5 Delta t

OK

Cross and stop for N seconds to confirm the selection

11

1.5 Delta t (cont.)

Cross and Stop

12

2. Advanced Crossing Ideas

2.1 Secondary Goal Crossing2.2 Gesture2.3 Corners and Edges2.4 Moving Targets2.5 Features of Crossing Event

13

OK OK OK OKOK

OK OKOK

OK

• BUTTON

first goal: button second goal (g2): crossing the g1 initiates g2 to appear

g2

Length-to-the-g2 variation angle variation

G2 design variation: how to make it look like a goal?

2.1 Secondary Goal Crossing

14

OKCrossing the first target activates the arc

2.1 Secondary Goal u Arc-shape

Location of the 2nd goal changes depending on the approaching angle

(a)

15

g2 fades out if they are not being selected during a certain amount of time

Right-handed, left-handed

2.1 Secondary Goal v L-Shape

16

• MULTI-LAYERED MENU: Red bar appears only when the menu has no underlying menu items

2.1 Secondary Goal w I-Shape

17

GO

2.1 Secondary Goal x ‘dontclickit’ idea

OK GOOK

ANIMATION

18

2.1 Secondary Goal y Cluster

T1 T2 T3

T4

T7 T8 T9

T6

Interactive prototype

Targets are clustered (3*3 or 2*2) and the secondary goals appear at the fixed location

1 2

3 4

https://students.washington.edu/eunky/agc/Practice_14.html

19

2.2 Gesture u Zig-zag / Scribble

OK OKOK

Nope, it’s not selected

Yes

Yes

20

2.2 Gesture v Encircling Targets

ANIMATION

21

2.2 Gesture w Pigtail

ANIMATION

22

2.2 Gesture x Hover Widget

cross

cancel

Turning 90° activates the most recently crossed goal. When the mouse crosses the end of the tunnel, the crossed goal is confirmed

23

2.2 Gesture y Shivering Around a Target

Sparse layout, one small target

24

2.2 Gesture Rectangle Lasso

Philip Phung, Colin Booth, Mark Vitazko, Chris Claiborne

Public DownloadsFile Folder

To open

25

2.2 Gesture Color In



26

2.2 Gesture Letter gesture

Chris Sinco, Rishi Talwar, Justin Wilbourne, Ken Wong

27

OK

OK

2.2 Gesture

Alexei Bespalov, Kevin Merritt, James Rodgers

28

2.3 Corners and Edges u

Constraining mouse movement with edges

29

OK

2.3 Corners and Edges v

Only Listen to Y-Movements

Now only Listen to X-Movements

• Basic Idea: Person can make the computer ignore x or y movements, creating an edge to travel along.

• The edges (or guidelines, or whatever we call them) could be activated with a keyboard/mouse button? Any other options?

• Maybe both can be activated to keep the from moving during a click?

• Could be used for crossing too.

30

Click!

Select

Skip

Select All

Corners are easy to hit.Make a context menu out of a square, where corners are options.

2.3 Corners and Edges w Box

31

2.3 Corners and Edges x

OK OK

Cancel

Open C.M

32

2.3 Corners and Edges y Diamond

Context Menu (C.M.)

Open

Cancel

Crossing one object

Previous

Open

Cancel

Crossing more than one object

Open all

Previous

Open

Cancel

Context Menu (C.M.)

Open

Cancel(Cancel)

33

2.3 Corners and Edges z Hover Widget + Diamond

Crossing one object Crossing more than one object

Context Menu

Open current

Openmultiple

XX

Open multiple

Open current

• Hitting the corner triggers the queued up targets (aggregation)

• Select or delete the targets that are in the queue

Context Menu

C.MOpen

C.M

Open

34

2.3 Corners and Edges 90-degree

OK OK

35

2.3 Corners and Edges

TARGET

36

2.3 Corners and Edges

The mouse can travel through either of the entrances/exits to begin activating the button, but the cursor must travel all the way through to the other side in order to fully trigger the button’s action.

OK

OK

Once the cursor enters the button, barriers at the top and bottom of the button prevent the passage out of the button. These barriers exist only once the mouse moves in a vertical direction.

OK


37

2.3 Corners and Edges Bounding Buttons

Nathan – Peter - Kaitlyn

Certain elements have un-mousable “screen edges” affixed to them to avoid pointer overshoot.

Ballistic movement can now hit and stick to targets.

38

2.4 Moving target u Coming Toward

Closest target is triggered, and then the target is coming toward the cursor

39

2.4 Moving target v Stepping Backward

Closest target is triggered, and then the target is stepping backward from the cursor

40

2.4 Moving target w Rearranging Targets

41

2.5 Features of Crossing Event u Velocity

Not selected Selected

Needs to cross above certain speed in order to select the target

Too slow

42

2.5 Features of Crossing Event v Acceleration

Not selected Selected

Stop and then accelerate the crossing speed (A>0)

a= 0

43

2.5 Features of Crossing Event Angle

Selected Not selected

Orthogonal crossing: selected

44

3. Other Crossing Ideas3.1 Kinematic Template3.2 Accessible Bubble Cursor3.3 Switch3.4 The Clock3.5 Guideline3.6 Trapezoid3.7 Extended Crossing Line3.8 Plough / Force Field Cursor3.9 Edge.Point3.10 Area Crosser3.11 Stabbing3.12 The Casual Fan3.13 Triple Cross3.14 Cross and Cross Back with Hover Widget3.15 Corner Pocket

45

3.1 Kinematic Template

Passive template: the cursor goes through the button with the help of the (passive) template

OK

OK

46

OK Cancel

EditOptions

Menu

Open

OK Cancel

EditOptions

Menu

Open

Cancel

Options Edit

Tab through targets!

OK MenuOK

OpenEdit

Cancel Menu

3.2 Accessible Bubble Cursor

47

3.3 Switch

Gravity effect + visual feedback

OK

OK

OK

activated

Switching on

48

3.4 The Clock

In this concept, a lever appears inside a circular overlay on the target when the cursor gets close to a goal. To trigger the selection of a target, the user “pushes” the lever (like a clock hand) in a circle until a full rotation is made. The cursor “sticks” to lever, facilitating rotation like the hand on a clock. Once the rotation movement has begun, the user can escape the “stickiness” by sliding the curser off the outer edge of the lever towards the outside of the circle.

Clarke Freeman, Alex Jansen, Kristofer Martin, Josh Rakita

49

3.5 Guideline

If the cursor follows the guideline for a certain amount of time and then cross, the target is selected

Selected

Not Selected

Stop for a while -> guideline appears

Selected

50

3.6 Trapezoid

OK OK OK

51

3.7 Extended Crossing Line u

Extended Crossing Goal

Voronoi tessellation

52

3.7 Extended Crossing Line v

Extended Crossing Goal

Voronoi tessellation Clustering targets around the cursor

53

3.8 Plough / Force Field Cursor

Clarke Freeman, Alex Jansen, Kristofer Martin, Josh Rakita

Cross with acceleration

Inspired by specifically considering the difficult circumstance of a densely arranged set of targets, the force field cursor effectively pushes targets around a circle, or force field, which remains centered on the cursor during tracking movement. Multiple targets in contact with the force field are distributed around the entire circumference. Selection is made by accelerating towards one of the acquired potential targets, achieving an “escape velocity” that allows the cursor leave the center of the field and cross over the desired target.

54

3.9 Edge.Point


As you move your mouse around the screen all Icons in the range shown by the area cursor will visually “tunnel” to your cursor. To toggle between freezing and releasing the area cursor press space bar. Freezing the cursor will also freeze all tunnels that are displayed. While tunnels are frozen they will “catch” your mouse cursor and bound it within the tunnel. This allows for easy and quick navigation to all available targets. As a reminder the tunnels and their bounding properties can be escaped from at any time by hitting space bar.

55

3.10 Area Crosser


An area cursor that traces a path from the mouse cursor to each target. Instead of traveling down the entire path, a goal is presented at the very beginning to cross and thus activate the button.

Pros: Pure crossing goal, reduces cursor travel distance.

Cons: Hard to implement actions other than "cross."

CLICK!

56

3.11 Stabbing


A user moves to a group of items; as the cursor approaches them, the items move around the cursor in a fan motion allowing for the user to select one by moving their mouse backwards over the target. The user is then presented with a menu of possible selection, which they can thereby make their selection by passing back over the icon.

Select Delete

57

3.12 The Casual Fan


45° arch to commit to selection

45° arch to commit to selection

90° arch for escape selection

Cancel

Activate the target

Activate the target

58

3.13 Triple Cross

Alexei Bespalov, Kevin Verritt, James Rodgers

The technique works by having the user cross over a goal three times, before he crosses over any other goals three times within additional crosses of the original goal. This technique is similar to the cross and cross-back technique, however it adds both safety and flexibility to the technique by adding a third cross of the goal.

59


3.14 Cross & Cross-back with Hover Widget

Alexei Bespalov, Kevin Verritt, James Rodgers

The technique works by having the user cross over a goal three times, before he crosses over any other goals three times within additional crosses of the original goal. This technique is similar to the cross and cross-back technique, however it adds both safety and flexibility to the technique by adding a third cross of the goal.

Copy Properties

Rename

Open

Delete

Cut

Send

60

3.15 Corner Pocket

Greg Bickford, Don Bushell, Gavin Elster, Andy Boer

61

4. Designing Actions and Context Menu

4.1 Accessing context menu with Hover Widget (list)4.2 Accessing context menu with Hover Widget (circular)4.3 Accessing context menu with Hover Widget + Diamond (list)4.4 Accessing context menu with Hover Widget + Diamond (circular)4.5 Accessing context menu with Secondary goal crossing

62

4.1 Accessing Context Menu with Hover Widget (list)• Right turn: Access context menu (= right click)

MoveCopyDeleteOpen with…CutSelectGroupProperties


• List context menu can handle sub-context menu well

OpenOffice.orgMicrosoft PowerpointKeynote

• ‘Select’ corresponds to the single left click in Windows

Confirm


• Relative turn• Absolute 90-degree turn

63

4.2 Accessing Context Menu with Hover Widget (circular)

Cancel

Move Copy

Select

Open with…

Cancel

Copy

Select

Open with…

Cancel

Cut

Move

Group

Properties

Move Copy

Select

Open with…

Cancel

Move Copy

Select

Open with…

Cancel

• Right turn: Access context menu

64

4.3 Accessing Context Menu with Hover Widget + Diamond (list)

UpDown

OpenCutCopyMoveDeleteSelectRenameProperties

Open

MoveCopySelect allOpen with…CutDeleteGroupPropertiesBack

‘Select all’ becomes one of the context menu items

• Turn right • Left turn, and at the right corner of the diamond

65

4.4 Accessing Context Menu with Hover Widget + Diamond (circular)

C.MPrevious

C.M

Delete

Copy Select

Open with…

Cancel

Group

C.M

Open

Previous

Cancel

Open

66

4.5 Accessing Context Menu with Secondary Goal Crossing

Move

Copy

Select

Open with…

CancelOpen

Move

Copy

Select

Open with…

Cancel

Open

• Sequencecross -> secondary goal appears -> turn 90 degree

• Cancel1) timeout2) menu item3) cross back4) loophole

merge left-click, right-click, single-click and double-click

Move

Copy

Select

Open with…

Open

67

The End

the end of pointing and clicking: improving computer access with goal crossing

Documents