tone reproduction - rochester institute of technology

Tone ReproductionTone ReproductionTone ReproductionErik Reinhard

University of Central FloridaSchool of Computer Sciencereinhard@cs.ucf.edu

Preparing Images for DisplayPreparing Images for Display

•• Need to map floating point numbers to Need to map floating point numbers to bytesbytes

•• Need to preserve some aspect of the image:Need to preserve some aspect of the image:–– ContrastContrast–– VisibilityVisibility–– BrightnessBrightness–– ……

Global vs. LocalGlobal vs. Local

•• GlobalGlobal–– Scale each pixel according to a fixed curveScale each pixel according to a fixed curve–– Key issue: shape of curveKey issue: shape of curve

•• LocalLocal–– Scale each pixel by a curve that is modulated by Scale each pixel by a curve that is modulated by

a local averagea local average–– Key issue: size of local neighborhoodKey issue: size of local neighborhood

Global OperatorsGlobal Operators

Ward Tumblin

Ferwerda

Global OperatorsGlobal Operators

TumblinWard

Ferwerda

Local OperatorLocal Operator

Pattanaik

Spatial ProcessingSpatial Processing

•• Circularly symmetric Circularly symmetric receptive fieldsreceptive fields

•• CenterCenter--surround mechanismssurround mechanisms–– LaplacianLaplacian of Gaussianof Gaussian–– Difference of GaussiansDifference of Gaussians–– BlommaertBlommaert

•• Scale space modelScale space model

Scale Space Scale Space (Histogram Equalized Images)(Histogram Equalized Images)

Tone Reproduction IdeaTone Reproduction Idea

•• Modify existing global Modify existing global operator to be a local operator to be a local operator, e.g. Greg Ward’soperator, e.g. Greg Ward’s

•• Use spatial processing to Use spatial processing to determine a local determine a local adaptation level for each adaptation level for each pixelpixel

5.2

4.0

4.0max

max

5.2

4.0

4.0max

max

),(219.12219.1),(

219.12219.1),(

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎟⎠⎞⎜

⎝⎛+

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎟⎠⎞⎜

⎝⎛+

=

yxL

L

LyxLL

L

L

LyxLL

a

d

doutput

w

d

doutput

BlommaertBlommaert Brightness ModelBrightness Model

22

2

22

1 skr

ii

iesk

R−

=π

ii RvuLsyxV ⊗= ),(),,(

),,(2),,(),,()(),,(

12

21

syxVs

syxVsyxVsWsyxV+

−= ϕ

∑=ns

ssyxVyxB

0

),,(),(

Gaussian filter

Neural response

Center/surround

Brightness

BrightnessBrightness

∑=ns

ssyxVyxB

0

),,(),(

Scale Selection AlternativesScale Selection Alternatives

How large should a local neighborhood be?

0

)),(,,(),(ss

yxsyxVyxBn

m

−=Mean value

ε<)),(,,(: yxsyxVs mmThresholded

Mean ValueMean Value

0

)),(,,(),(ss

yxsyxVyxBn

m

−=

ThresholdedThresholded

ε<),,(: mm syxVs

ToneTone--MappingMapping

Local adaptation5.2

4.0

4.0max

max

1

),(219.12219.1),(

)),(,,(),(

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎟⎠⎞⎜

⎝⎛+

=

=

yxL

L

LyxLL

yxsyxVyxL

a

d

doutput

ma

Greg Ward’s tone-mapping with local adaptation

ResultsResults

•• Good results, but something odd about scale Good results, but something odd about scale selection:selection:

•• For most pixels, a large scale was selectedFor most pixels, a large scale was selected

•• Implication: a simpler algorithm should be Implication: a simpler algorithm should be possiblepossible

Simplify AlgorithmSimplify Algorithm

),(),(

),(1),(

),(219.12219.1),(

5.2

4.0

4.0max

max

yxLLayxL

yxLyxLL

yxL

L

LyxLL

ww

output

a

d

doutput

=

+≈

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

+

⎟⎠⎞⎜

⎝⎛+

=Greg Ward’s tone-mapping with local adaptation

Simplify

Fix overall lightness of image

Global Operator ResultsGlobal Operator Results

Our method Ward

Global Operator ResultsGlobal Operator Results

WardOur method

Global Global LocalLocal

)),(,,(1),(

),(1),(

1 yxsyxVyxLL

yxLyxLL

moutput

output

+=

+=Global operator

Local operator

Local Operator ResultsLocal Operator Results

Global

Local

Local Operator ResultsLocal Operator Results

Global Local Pattanaik

Lesson LearnedLesson Learned

•• A good local average is obtained by using a A good local average is obtained by using a (Gaussian) blurred pixel.(Gaussian) blurred pixel.

•• The amount of blur to apply varies per pixelThe amount of blur to apply varies per pixel

•• The filter kernel should as large as possible, The filter kernel should as large as possible, without crossing sharp contrast boundarieswithout crossing sharp contrast boundaries

•• This is why the scale selection mechanism This is why the scale selection mechanism works!works!

Global OperatorsGlobal Operators

•• Usually a sigmoid:Usually a sigmoid:

•• Sigma is the semiSigma is the semi--saturation constantsaturation constant•• Most global operators can be rewritten into Most global operators can be rewritten into

a form similar to thisa form similar to this

na

nw

nw

d LLLL

)(σ+=

Global Operators Global Operators -- SigmoidSigmoid

Drago

LaLa

•• Adaptation luminance computed asAdaptation luminance computed as–– Global averageGlobal average

•• Log average luminanceLog average luminance•• Interpolation schemeInterpolation scheme

–– Local averageLocal average•• Use four neighbors (good enough in practice)Use four neighbors (good enough in practice)•• Scale selection mechanismScale selection mechanism•• Bilateral filterBilateral filter

•• Typically small filter kernelsTypically small filter kernels

PhotoreceptorPhotoreceptor--basedbased

naa

pixelwa

aw

wd

aw

wd

aw

wd

fLL

LaaLL

LBBB

LGGG

LRRR

=

−+=

+=

+=

+=

)(

)1(

)(

)(

)(

σ

σ

σ

σ

na

nw

nw

d LLLL

)(σ+=

Also interpolate between R,G,B values and pixel luminance

Interpolation Scheme for LaInterpolation Scheme for La

Luminance interpolation

Interpolation Scheme for LaInterpolation Scheme for La

Between R,G,B and luminance

Bilateral filterBilateral filter

Local LaLocal La

Local bilateral

GlobalLocal SUSAN

ConclusionsConclusions

•• Most tone reproduction operators are Most tone reproduction operators are sigmoidalsigmoidal

•• Can be either local or globalCan be either local or global

•• Filter kernels (local) should be small, but in Filter kernels (local) should be small, but in any case should not cross sharp contrast any case should not cross sharp contrast boundariesboundaries

ConclusionsConclusions

•• SigmoidsSigmoids also part of most color appearance also part of most color appearance modelsmodels

•• Color appearance and tone reproduction will Color appearance and tone reproduction will convergeconverge