Latent Doodle SpaceLatent Doodle Space

William Baxter1, Ken-ichi Anjyo2

OLM Digital, Inc.EUROGRAPHICS,2006

Presented by C.M. Hsu

OLM Digital, Inc.

EUROGRAPHICS,2006

AbstractAbstract

AbstractAbstract

Major tech. A heuristic algorithm to match strokes

between the inputs. Extract a low dimensional latent doodle

space from the inputs.

ApplicationsApplicationsThe Randomized Stamp ToolThe Randomized Stamp Tool

InputInput

OutputOutput

ApplicationsApplicationsHandwriting synthesisHandwriting synthesis

OutputOutput

InputInput

OutlineOutline

Overview Related Work Stroke Matching Algorithm Building the Latent Doodle Space Conclusions and Future Work Demo

OverviewOverview

Overview

1. Assignments of stroke correspondence

2. Resample corresponding strokes with the samesame number

of sample points

3. Reverse the parameterization of strokes to improve the point-to-point

correspondence

4. Build the latent control space

1. Similar to Computer-assisted in-between algorithms2. N-way correspondences, not pair3. Competitive-learning algorithm

•K-means-like•Match stokes based on Kuhn-Munkres method•O(N3)

Synthesis

PCA

thin plate spline RBF

PCA

GP

Feature vector

GPLVM

Related WorkRelated Work

Related Work – Stroke Correspondences

The order of strokes between two images are identical, [Burtnyk N., SIG75]

Closed shape only, [Sederberg, SIG92]

Related Work – A low-dimensional latent space

Eigenfaces by PCAPCA, [Turk, Nerosci91] Multidimensional motion interpolation b

y Radial basis functions (RBFRBF) [Rose et al.,IEEE98]

The Gaussian Process Latent Variable Model (GPLVMGPLVM), [Lawrence, NIPS04]

Create keyframe from ex. by GPLVMGPLVM, [Grochow K., CV04]

Related Work Create many drawings from a few ex., [Ko

var, UIST01]

Stroke Matching Stroke Matching AlgorithmAlgorithm

1. Finding Stroke Correspondences2. The Assignment Cost Matrix

3. Stroke Re-sampling and Alignment

Finding Stroke Correspondences

K-means like1. Initialize stroke-to-cluster assignment

Clustering by the drawn order of strokes simply

2. Update the cost matrix3. Reassign stroke based on new clusters

Linear assignment problem (strokes clusters) Constrain: one stroke per drawing to each cluster Kuhn-Munkres algorithm, (N3), N as number of strokes

4. If reassignments made, goto 2

The Assignment Cost Matrix E=ed+ec+et

Translation error, ed The mean of the stroke differs from the mean of t

he cluster Orientation and eccentricity error,ec

The covariance axe of the stroke differs from those of the cluster

Topological matching cost, et The connectivity of the stroke differs from the con

nectivity of the members of the cluster

Translation error

ed: The mean of a stroke differs form the mean of the cluster

2csd -e

stroke theofpoint mean the:s cluter theofpoint mean the:c

Orientation and eccentricity error

ec: The covariance axe of the stroke differing from those of the cluster

f2cs

cc A-Awe

Stroke 1Stroke 2

2212

2111

ssss

sssssC

c2

c1

s 2,

s1

cs

cs

cs

1 1j

2

ijf2

f2

c

4s2

s1

s2

s1c

clusters. and strokes of eeigen valu the:],[

eeigen valuby ordering

C, of orseigen vect ofmatrix the:],[

clusters and strokes ofmatrix covarince the:]C,[C

b norm, Frobenius :

16w1,,/avg,maxw

AA

Bm

i

n

Topological matching cost et: Connectivity cost of the stroke differing from

the connectivity of the members of the cluster

CCssjj: the number of strokes the ith stroke in the same drawing.

CCccjj : the number of clusters the ith cluster by average distance.

2ci

sjt c-c0.2e

S2S1

S3

S2 is connected to S1, cs2=1 C2 is connected to C1, cc

2=1

C2

C1

Stroke Re-sampling and Alignment Same number of points on the correspon

ding strokes for RBF, Gaussian process regression

Reverse backward strokes The total distance error between two stroke

s is lower when the point ordering is reversed

Building the Latent Doodle Space

PCA2 principal

components

thin plate spline RBF

PCA2 principal

components

GP

Feature Vectorm = {p1,p2,..pn} of Strokem

GPLVM

RBF•Gaussian•Thin plate spline, r4logr

bestinput output

Conclusions and Future Work Using machine learning

Find good assignment weights in cost functionEx: S.T. like a support-vector classifier could be trainned to assign strokes to clusters.

Allow user to appraise the products form as latent space. [Kovar, UIST01]

Accept scanned drawing Accept completely free-form hand-drawn sketc

h without the line constrain of uniform width.

EndEnd