Empathic Painting: Interactive stylization through observed

emotional stateMaria Shugrina , Margrit Betke , John CollomosseComputer Science Department, Boston University, Boston MA, USA.Computer Science Department, Boston University, Boston MA, USA.Department of Computer Science, University of Bath, Bath, England.NPAR 2006

Russell's 2D pleasure-arousal space

Facial action units (AUs)

Stroke placement

We begin by creating a colour band-pass pyramid segmentation of the source image using the EDISON algorithm.

For each layer, we first render the “interior” strokes of all regions, then the “boundary” strokes of all regions.

Brush strokes are formed using Catmull-Rom piecewise cubic splines, the control points of which are computed from the binary image of each region.

Interior Strokes

Scan-lines are then traversed, and strokes are started and terminated as region boundaries are encountered; the vertical interval between scan-lines during processing is proportional to stroke thickness.

Stroke control points are distributed uniformly over the stroke's length, and jittered via small translations to disguise the regularity of the stroke placement process.

Stroke colour is computed using the mean colour of the original image.

Stroke thickness is set on a per region basis, in proportion to area.

Boundary Strokes

run-length compressed Freeman (chain) code working set the angle between adjacent control points rises

above 50o, stroke is terminated

control points (magenta)

chain code (cyan)

Rendering Parameters

Rendering parameters were: p1 = 0:3, p2 = (-0.3, 0.0); p3 = 0.31, p4 = 0.0, p5 = 0.5 (left image), p1 = 0.0, p2 = (0.0), p3 = 0.0, p4 = 0.0, p5 = 0.0 (right image).

Region Turbulence(p1)

The effect of the turbulence (p1) parameter on interior stroke regions

Tonal Variation (p2)

Happiness (pleasantness)---brightness (luninance), saturation

Arousal---saturation or hue Calmness---blue [Wright] Depression, cold---blue [Mahnke]

Tonal Variation (p2)

G(x): greyingU(x): un-greyingD(x): darkeningL(x): lightening

T1(x): red colors are reddened and green(calm) is reducedT2(x): increasethe blue

Stroke Denotation Style(p3-p5)

A single brush stroke rendered with a range of undulation (p3) and jaggedness (p4) parameters. From left to right: (a) p3 = 0.0, p4 = 0.0, (b) p3 = 0.0, p4 = 1.0, (c) p3 = 0.7, p4 = 0.0, (d) p3 = 0.7, p4 = 0.8.We introduced p5 to dampen the effects of undulation (p3) on interior strokes.

Result- angerRussell's pleasure-arousal space

p{1…5}= [0.64, (-0.86, 0.73), 0.62, 0.85, 0.58]

The painterly output mirrors this through a general red-shift and luminance reduction of colours used in the rendering.Strokes become moderately undulated, and noticeably jagged and chaotic.

Result- sadness/despair

p{1…5}= [0.33, (-0.96, -0.22), 0.31, 0.52, 0.24]The presence of user state in the third quad rant generates a colour shift towards the calmer blues and greens, with low arousal also manifesting itself through both low saturation and stroke undulation, as well as a much calmer, less chaotic background.

Result- jubilation

p{1…5}= [0.87, (0.67, 0.73), 0.85, 0.47, 0.91]

We see highly agitated and chaotic stroke placement, again reflecting high arousal scores.

Control data

A neutral expression corresponding to approximately (0,0) in the Russell space (top-right) and the resulting neutral painting (bottom).