painterly rendering with content-dependent natural paint...

Painterly rendering with content-dependent natural paint strokes

Kim, Wook

2015. 3. 31

Computer Graphics @ Korea University

Vis Comput 2011Hua Huang · Tian-Nan Fu · Chen-Feng Li

Kim, Wook | 2015. 3. 31 | # 2Computer Graphics @ Korea University

Abstract

• Content-dependent painterly rendering method

• New stroke layout strategy

• to enhance the contrast between large and small paint strokes

Input image Result image

3 contributions


Abstract

• Anisotropic digital brush

• multiple colors, naturally varied textures

• Lighting effects

• improve the quality of rendering

Input image Result image


• Traditional painting approach

a painting is built up in a series of layers

paint strokes have large sizes on lower layersand smaller sizes on upper layers

• Large strokes on lower layer contribute nothing to the final result

• Using small strokes to refine the result will damage the abstract impression generated by the large strokes

Introduction

2 problems


Introduction (cont)

Portrait of Caspar Goodrich- face region : small strokes- background region : large strokes

• Motivation --strokes with different sizes are applied not only on different layersbut also in different regions on the canvas

• Paint strokes often have a larger size in smooth regionsand a smaller size in regions full of details


• [Hertzmann 1998]

larger brushes were used to draw a rough sketchand smaller brushes were used to refine the details

curved brush strokes were introduced

• single color

• cubic B-spline

• [Hertzmann and Perlin 2000]

extend videos

painting process was performed only in the regions with significant changes to improve temporal coherence

Related work


• [Hertzmann 2002]

add lighting effects to painterly rendered images

opacity and height maps were assigned to each stroketo construct the height map of the whole image

• [Hays and Essa 2004]

the orientation of each stroke was computedusing RBF interpolation of key-point orientations

• [Zhen et al 2009]

object-based method

try to paint different objects with different styles

Related work (cont)


• Importance driven stroke layout strategy

paints smooth regions with larger strokes while painting details with smaller strokes

• Three steps..

computing the importance map

partitioning the canvas into grids according to local importance

assigning different stroke sizes to different grids

Layout of strokes


• A bilateral filtering is first performed on the input image

remove noises

• The gradients are calculated using Sobel operator

areas close to edges are often considered more important

different measures can be adopted to adjust the importance map

• ex) face detection

Computing the importance map

𝐸(𝑥, 𝑦) = (𝜕𝐼(𝑥, 𝑦)

𝜕𝑥)2+(

𝜕𝐼(𝑥, 𝑦)

𝜕𝑦)2

grayscale intensity


• Divide the canvas into a set of rectangular grids

cutting through the longer edge

division process is repeated until the total importance energy in the resulting grid is less than a threshold

• To find the optimal cutting point, we solve the following optimization problem

Partitioning the canvas into grids

index of valid cutting points

the smaller and larger areas of the two resulting grids

the smaller and larger averageenergy of the two resulting grids


Partitioning the canvas into grids (cont)

Input image Importance map Result of canvas partition

• Smooth regions

• the partition operation quickly ends after a few divisions

• leads to larger grids

• Detailed regions

• more divisions are performed


Assigning stroke property (position)

• Sort the grids in descending order according to their area

• Within each grid, two potential anchor points are defined• pmin – the pixel with the minimum coverage

• pmax – the pixel whose color has the greatest differencefrom the corresponding pixel on the reference image

• If the coverage of pmin is smaller than a given threshold• a stroke will be placed at this point

• Otherwise, the color difference is greater than a threshold• a stroke will be placed at pmax

• If none of these two condition is met• then no stroke will be placed


Assigning stroke property (size)

• The size of the stroke is selected from a pre-defined listaccording to the area of the grid

• The layout of strokes is completed for a layerwhen every grid is checked

• 3 to 5 layers of strokes are sufficient


Orientation of strokes

• Use a method similar to Litwinowicz [16]

• Smooth regions

• stroke direction is mainly determined by the RBF interpolation

• Detailed regions

• stroke direction is mainly determined by local gradient

orientation field obtained from the normals of local gradients

orientation field computed from RBF interpolation

importance energy [0,1]


• Introduce an anisotropic brush model using brush map

• Each visible pixel in the brush map represents a bristle

white means completely touched and black means separated

a stroke is rendered by specifying a color to every bristleand dragging the brush map on the canvas

Stroke rendering

brush map

Stroke rendered by an anti-aliased line

Stroke rendered by the anisotropic brush


Stroke rendering (cont)

• How to define pigment colors?

• The center of brush map is placed at the anchor point of the stroke

• The color of each bristle is sampled from the corresponding point on the reference image

• The color of each bristle is adjusted by

color sampled from the reference imageat the anchor point

optional random color


Stroke rendering (cont)

• After initializing the brush, the stroke is painted along a path consisting of a set of ordered points

intensity value of the associated bristle

color of each bristle color of the canvas


• The pixel intensity on the height map is assignedas the average intensity of bristles that pass through the corresponding point

• The opacity map is obtained by rendering stroke path using a white anti-aliased line

• The canvas height map is constructed using height & opacity map

Lighting effect

the height map the opacity map


• For each pixel (x,y), its intensity adjustment is proportional to height difference D(x,y)

the intensity of pixel (x,y) is increased if D(x,y) is positive

vice versa

Lighting effect (cont)

orientation

pixel height


Results

Our result Hertzmann [6] Zhao and Zhu [26]

• Produces a better contrast between larger and smaller strokes

• The clear contrast between the sophisticated foreground and the flat background emphasizes the message of the painting


Results (cont)

Our resultHays and Essa [5]Litwinowicz [16]

• Produces a better contrast between larger and smaller strokes

• Edges are successfully preserved

• Because of importance map generated by edge detection


Results (cont)

Hertzmann [6] Lighting effect produced by Hertzmann [7]

Our resultLighting effect produced by our approach,using the same height fieldgenerated by Hertzmann [7]

• More realistic

• Easier to implement


Results (cont)

Full version of the new method

without stroke layout strategy

without lighting effect

• Middle - edges in the image are not well preserved and many fine features have been lost

• Right - strokes rendered by using anti-aliased lines appear to be very mechanical


• An effective and robust algorithm

Conclusion

painterly rendering with content-dependent natural paint...

Documents