Freehand Drawing Systembased on Geometric Constraints

Kaisuke Nakajima

Joint work with Takeo Igarashi

User Interface Research GroupThe University of Tokyo

Two Parts of the Talk

Introduction to Pegasus [Igarashi, UIST 1997 & CHI 1998]

Improvement attempts My current study Any comments greatly appreciated!

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

Problem

Diagrams In documents, slides, … By drawing editors

Difficult to satisfy geometric relations!

Example

Copy Flip Move

Example (contd.)

In this case, grids works well, but…

Geometric constraints

Another Example

The user attempts todraw a perpendicular line,

but grid prevents desired placements.

GRID

He draws a box separately, and...

rotates it, but fails to get precise angle.

ROTATE

Better strategy:copy the slope rotate 90 degrees

COPY

ROTATE

The user must plan detailed strategy.

Copy

Flip

Move

RotateGrid

Planning overhead

Why Difficult?

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

Our Idea

Sketching is easy!

Sketch Beautify Interactive Beautification [Igarashi, UIST 1997]

Sketch + Beautify

Draw and...

Interactive Beautification

Beautify.

Satisfied Constraints

Draw and...

Interactive Beautification

Beautify. Draw and...

Interactive Beautification

Beautify.

Interactive Beautification

But free strokes are ambiguous.

Ambiguity Handling

Generate multiple candidates Let the user select

Ambiguity Handling

In this way,the user can construct

precise geometric diagramswithout any editing commands!!

Prototype System Pegasus

Live Demo

Diagrams Drawn

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

[Pavlidis 1985] ...

Beautification Systems

Batch-based

too many errors!

More Interaction (+ multiple candidates)

Beautification Systems (contd.)

[Apple Newton] [SmartSketch] ...

Free Stroke Vectorization

Local context

Free Stroke Vectorization (contd.)

Global contextLocal context

Free Stroke Vectorization (contd.)

Drawing Systems based on Geometric Constraints

[Sutherland 1963] [Nelson 1985] ...

Drawing Systems based on Geometric Constraints (contd.) Previous work

Explicitly specify constraints

Our approach Infer constraints

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

Beautification Algorithm

Constraint Inference

Segment coordinates

Multiple constraints

Constraint Solver

Multiple candidates

Candidate Evaluation

Primary candidate

Constraint Inference

Input Reference segments

(array of (X0, Y0, X1, Y1))

New stroke(x0, y0, x1, y1)

Output Possible constraints

(linear equations of (x0, y0, x1, y1))

Perpendicular const * (x1 – x0) + const * (y1 – y0) = 0

Edge Connection x0 = const, y0 = const

etc.(x0, y0)

(x1, y1)

Constraint Inference (contd.)

How?

Supported constraints

if (distance(point1, point2) < 30) infer constraint(point1 == point2);

Constraint Solver

Input Possible constraints

(linear equations of (x0, y0, x1, y1))

Output Beautification candidates

(array of (x0, y0, x1, y1))

Perpendicular

Edge Connection

Same Length

Horizontal

Constraint Solver (contd.)

Candidate Evaluation

Candidate nearest to the original stroke Primary candidate

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

To confirm the intuition !

“It’s difficult for novice usersto use commands”

“Interactive beautification is good”

Purpose

RapidnessHow fast?

PrecisionBeautiful?

Two Measures

Experimental Settings

18 subjects 3 diagrams, 3 editors (9 sessions / person)

Systems

CAD ... AutoSketch (AutoDesk), CAD Software.

Draw ... SmartSketch (Future Wave), Drawing Editor.

Pegasus…Our Prototype system.On AMiTY (a pen computer)

Diagrams and Constraints

A) B) C)

Parallel-1

Parallel-2

Perpendicular

Vertical and Horizontal

Connected (all Vertices)

Symmetric-1

Symmetric-2Connected (all Vertices)

Parallel-1

Parallel-2

Equal Interval

Connected (all Vertices)

Results

Rapidness PrecisionAveraged drawing time for three diagrams

0100200300400500600

CAD Draw Pegasus020406080100

CAD Draw Pegasus

The ratio of sessions where all constraints are satisfied

Pegasus outperformed CAD and Draw,

in both rapidness and precision.

Evaluation: Conclusion

Pegasus cannot do everything

But: certainly useful in its target domain

CAD Paint

Geometric diagrams,rapidly & precisely!

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

Ongoing Work

Address limitations of Pegasus Only lines Too many candidates Forgets constraints

Limitation: Only Lines

Support curves Circles, ellipses, arcs,

Bézier curves (Experimental demo)

Limitation: Only Lines (contd.)

Challenges with curves More ambiguity

Shape recognition Constraint inference

Complex constraints: how to represent? e.g. line tangent to piecewise Bézier curve

Complex constraints: how to solve efficiently? Exponential growth of candidate solutions

Limitation: Too Many Candidates

Overlapping…

Constraint switching interface Visualize conflicting constraints Enable manual switching

Challenges How to detect conflicting constraints efficiently?

Limitation: Forgets Constraints

Remember constraints Maintain them during manipulation Chorus [Hosobe 2001]

Stretch!

Stretch!

Skewed

Outline

1. Problem

2. Our Idea & Demonstration

3. Related Work

4. Algorithm

5. Evaluation

6. Ongoing Study

7. Summary

Summary

Pegasus: freehand drawing system User study showed promising results

Geometric constraints play an important role in sketch beautification

Further improvements to be made

Thank you

Papers & demos available athttp://www-ui.is.s.u-tokyo.ac.jp/