study image vectorization using optimized gradeint meshes
DESCRIPTION
studyTRANSCRIPT
![Page 1: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/1.jpg)
IMAGE VECTORIZATION USING GRADIENT MESHES
Jian Sun, Lin Liang, Fang Wen, Heung-Yeung Shum
Microsoft Research Asia
ACM SIGGRAPH 2007
![Page 2: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/2.jpg)
Input image Initial mesh
Optimized gradient meshReconstruction
Cutout tool
![Page 3: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/3.jpg)
Outline
Introduction Background Gradient Mesh Optimized Gradient Mesh Result Conclusions
![Page 4: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/4.jpg)
Introduction
![Page 5: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/5.jpg)
Image Vectorization
Goal : convert a raster image into a vector graphics Compact Scalable Easy to animate
Requirements Vector-based contents (eg. Flash or SVG) on
the Internet Vector-based GUIs used in Windows Vista
![Page 6: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/6.jpg)
Gradient mesh
Gradient mesh, a drawing tool of commercial vector graphics editors
http://www.creativebush.com/tutorials/mesh_tutorial.php
Selecting mid value skin tone
Tracing photograph
Sampling colors from photo
Sampling colors from within the mesh
Start adding mesh points Sampling colors from face mesh to hide seam
Finished eye/eye socket
![Page 7: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/7.jpg)
gradient mesh Image represented by gradient mesh
http://www.creativebush.com/tutorials/mesh_tutorial.php
![Page 8: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/8.jpg)
Image vectorization tools
Adobe Illustrator, “Live Trace” Corel CoreDraw, “CorelTrace” AutoTrace, “AutoTrace”
Input image Adobe, Live Trace
![Page 9: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/9.jpg)
Optimized gradient mesh Blend surface colors according to the
control points color as constructing surface by the control points
Optimize the gradient mesh as an energy minimization problem
Advantages Efficiency of use Easy to edit – modify, animation Scalability Compact representation
JPEG, 37.5 KB
Optimized, 7.7KB
![Page 10: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/10.jpg)
Background
![Page 11: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/11.jpg)
Object-based vectorization
Object-based vectorization [Price and Barrett 06] Hierarchically segmentation of object and
sub-objects by a recursive graph cut algorithm
Subdivide meshes until the reconstruct error is below a threshold
Input image Subdivision mesh
![Page 12: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/12.jpg)
RaveGrid [Swaminarayan and Prasad 06]
Constrained Delaunay triangulation of the edge contour set
![Page 13: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/13.jpg)
cont.
![Page 14: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/14.jpg)
Ardeco Automatic Region Detection and
Conversion algorithm [Lecot and Levy 06] Cubic splines Each region filled with a constant color, or a linear or
circular gradient
![Page 15: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/15.jpg)
cont.
![Page 16: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/16.jpg)
Gradient Mesh
![Page 17: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/17.jpg)
Overview
Input raster image
Initial mesh(Coons mesh)
Optimized gradient mesh
(Ferguson patch)
Reconstruction image
min arg Energy(Mesh) non-linear least squares (NULL) problem
Cutout tool
Levenberg-Marquardt (LM) algorithm
Color fitting: monotonic cubic spline interpolation
Coherent matting method: sample object boundary color from the estimated foreground colors
E(M) Constrains:1. Smoothness2. User guided vector3. Boundary
![Page 18: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/18.jpg)
Mesh initialization (manually) Decompose the input image into sub-
objects using Cutout tool [Li et al. 04]
Divide each sub-objects with 4 segments manually as Coons patches
m(u,v): the position vector of a point (u,v) Q : control points el al. F : basis functions
![Page 19: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/19.jpg)
Ferguson patch [Ferguson 1964]
P(0)=A P(1)=B
TA TB
B
A
T
T
B
A
uuuuP
1122
1233
0100
0001
1)( 32
Basic Curve Segmentation
![Page 20: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/20.jpg)
Ferguson patch
Basic Surface Segmentation
0
![Page 21: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/21.jpg)
Ferguson patch
Basic Surface Segmentation
0
![Page 22: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/22.jpg)
Gradient mesh
A gradient mesh consists of topologically planar rectangular Ferguson patches with mesh-lines
For each point q Position: {xq,yq}
Derivatives: {mqu,mq
v, αqumq
u, αqvmq
v} RGB color: cq = {cq(r), cq (g), cq (b)}
![Page 23: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/23.jpg)
TTf VCUCQvuf ),(
1122
1233
0100
0001
C
321 uuuU 321 vvvV
00
0031
20
3131
2020
uu
uu
vv
vv
f
CC
CC
CCCC
CCCC
Q
Ferguson patches are lack of Cv and Cu !
Color interpolation
![Page 24: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/24.jpg)
[Wolberg and Alfy 99] Determine the smoothest possible curve that
passes through its control points and satisfy monotonic constraint
The seven data points are monotonically increasing in f(xi) for 0 i 6, the cubic spline is not monotonic≦ ≦
Monotonic cubic spline
![Page 25: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/25.jpg)
Rendering Ferguson patches
1. Sample color of control points2. Estimate Cu, Cv by Monotoic Cubic
Spline algorithm3. Render Ferguson patches
TTf VCUCQvuf ),(
![Page 26: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/26.jpg)
Scalability
A gradient mesh• original resolution (x
1)
Scaling result (x8)• shape edges are well
preserved
Bi-cubic raster scaling (x8)• Blocky artifacts
appear
![Page 27: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/27.jpg)
Optimized gradient mesh
![Page 28: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/28.jpg)
Minimize E(M)
min arg.
![Page 29: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/29.jpg)
Solve NULL problem using LM algorithm
Minimizing E(M) is a non-linear least squares (NULL) problem Energy function
z: vector form of unknowns in M
Levenberg-Marquardt (LM) algorithm is the most successful solver for NULL [Levenberg 44], [Nocedal and Wright 99]
![Page 30: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/30.jpg)
cont.
Gaussian pyramid from the input image and apply coarse-to-fine optimization for LM
![Page 31: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/31.jpg)
Optimization
Optimized gradient mesh• 0.7/pixel reconstruction
error• 40 iterations
Gradient mesh of Adobe Illustrator
![Page 32: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/32.jpg)
Smooth constraint
Input image Opt. gradient mesh without smooth(err. 1.8/pixel)
Opt. gradient mesh with smooth (err. 0.9 pixel)
vtus 5,5,50 Smooth neighboring patches also,mp(- s, t) = m△ p-1(1- s, t) △
![Page 33: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/33.jpg)
Vector line guided optimized gradient mesh
Initial mesh
User guided vector, V
Opt. gradient mesh with user guided vector(err. 0.5/pixel)
Directly optimized gradient mesh (err. 2.5/pixel)
![Page 34: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/34.jpg)
Vector line guided optimized gradient mesh
Initial mesh Opt. gradient mesh with V
w = 1/5 L
3wv
![Page 35: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/35.jpg)
Boundary constraint
The boundary of a gradient – one or more cubic Bezier spline
The control points on the boundary only move along the splineEx: control point q on the spline S in u direction
![Page 36: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/36.jpg)
Results
![Page 37: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/37.jpg)
Red pepper
Gradient meshes by an artist (354
patches)
Initial gradient meshes
Optimized the highlight and
shadow regions are reconstructed
![Page 38: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/38.jpg)
Sculpture
Input image ReconstructionOptimized
gradient mesh
![Page 39: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/39.jpg)
Face
Input image ReconstructionOptimized
gradient mesh
![Page 40: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/40.jpg)
Conclusions
Introduce the gradient mesh as an image representation tool first
Present optimized gradient mesh
Limitations A fine image details and
highly textured image Boundaries or topologies are
too complicated
Input image
Reconstructed image
Optimized gradient meshes
![Page 41: study Image Vectorization using Optimized Gradeint Meshes](https://reader036.vdocument.in/reader036/viewer/2022062319/554a3947b4c905293a8b4887/html5/thumbnails/41.jpg)
END