tone reproduction - rochester institute of technology
TRANSCRIPT
Tone ReproductionTone ReproductionTone ReproductionErik Reinhard
University of Central FloridaSchool of Computer [email protected]
Preparing Images for DisplayPreparing Images for Display
•• Need to map floating point numbers to Need to map floating point numbers to bytesbytes
•• Need to preserve some aspect of the image:Need to preserve some aspect of the image:–– ContrastContrast–– VisibilityVisibility–– BrightnessBrightness–– ……
Global vs. LocalGlobal vs. Local
•• GlobalGlobal–– Scale each pixel according to a fixed curveScale each pixel according to a fixed curve–– Key issue: shape of curveKey issue: shape of curve
•• LocalLocal–– Scale each pixel by a curve that is modulated by Scale each pixel by a curve that is modulated by
a local averagea local average–– Key issue: size of local neighborhoodKey issue: size of local neighborhood
Global OperatorsGlobal Operators
Ward Tumblin
Ferwerda
Global OperatorsGlobal Operators
TumblinWard
Ferwerda
Local OperatorLocal Operator
Pattanaik
Spatial ProcessingSpatial Processing
•• Circularly symmetric Circularly symmetric receptive fieldsreceptive fields
•• CenterCenter--surround mechanismssurround mechanisms–– LaplacianLaplacian of Gaussianof Gaussian–– Difference of GaussiansDifference of Gaussians–– BlommaertBlommaert
•• Scale space modelScale space model
Scale Space Scale Space (Histogram Equalized Images)(Histogram Equalized Images)
Tone Reproduction IdeaTone Reproduction Idea
•• Modify existing global Modify existing global operator to be a local operator to be a local operator, e.g. Greg Ward’soperator, e.g. Greg Ward’s
•• Use spatial processing to Use spatial processing to determine a local determine a local adaptation level for each adaptation level for each pixelpixel
5.2
4.0
4.0max
max
5.2
4.0
4.0max
max
),(219.12219.1),(
219.12219.1),(
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
+
⎟⎠⎞⎜
⎝⎛+
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
+
⎟⎠⎞⎜
⎝⎛+
=
yxL
L
LyxLL
L
L
LyxLL
a
d
doutput
w
d
doutput
BlommaertBlommaert Brightness ModelBrightness Model
22
2
22
1 skr
ii
iesk
R−
=π
ii RvuLsyxV ⊗= ),(),,(
),,(2),,(),,()(),,(
12
21
syxVs
syxVsyxVsWsyxV+
−= ϕ
∑=ns
ssyxVyxB
0
),,(),(
Gaussian filter
Neural response
Center/surround
Brightness
BrightnessBrightness
∑=ns
ssyxVyxB
0
),,(),(
Scale Selection AlternativesScale Selection Alternatives
How large should a local neighborhood be?
0
)),(,,(),(ss
yxsyxVyxBn
m
−=Mean value
ε<)),(,,(: yxsyxVs mmThresholded
Mean ValueMean Value
0
)),(,,(),(ss
yxsyxVyxBn
m
−=
ThresholdedThresholded
ε<),,(: mm syxVs
ToneTone--MappingMapping
Local adaptation5.2
4.0
4.0max
max
1
),(219.12219.1),(
)),(,,(),(
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
+
⎟⎠⎞⎜
⎝⎛+
=
=
yxL
L
LyxLL
yxsyxVyxL
a
d
doutput
ma
Greg Ward’s tone-mapping with local adaptation
ResultsResults
•• Good results, but something odd about scale Good results, but something odd about scale selection:selection:
•• For most pixels, a large scale was selectedFor most pixels, a large scale was selected
•• Implication: a simpler algorithm should be Implication: a simpler algorithm should be possiblepossible
Simplify AlgorithmSimplify Algorithm
),(),(
),(1),(
),(219.12219.1),(
5.2
4.0
4.0max
max
yxLLayxL
yxLyxLL
yxL
L
LyxLL
ww
output
a
d
doutput
=
+≈
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
+
⎟⎠⎞⎜
⎝⎛+
=Greg Ward’s tone-mapping with local adaptation
Simplify
Fix overall lightness of image
Global Operator ResultsGlobal Operator Results
Our method Ward
Global Operator ResultsGlobal Operator Results
WardOur method
Global Global LocalLocal
)),(,,(1),(
),(1),(
1 yxsyxVyxLL
yxLyxLL
moutput
output
+=
+=Global operator
Local operator
Local Operator ResultsLocal Operator Results
Global
Local
Local Operator ResultsLocal Operator Results
Global Local Pattanaik
Lesson LearnedLesson Learned
•• A good local average is obtained by using a A good local average is obtained by using a (Gaussian) blurred pixel.(Gaussian) blurred pixel.
•• The amount of blur to apply varies per pixelThe amount of blur to apply varies per pixel
•• The filter kernel should as large as possible, The filter kernel should as large as possible, without crossing sharp contrast boundarieswithout crossing sharp contrast boundaries
•• This is why the scale selection mechanism This is why the scale selection mechanism works!works!
Global OperatorsGlobal Operators
•• Usually a sigmoid:Usually a sigmoid:
•• Sigma is the semiSigma is the semi--saturation constantsaturation constant•• Most global operators can be rewritten into Most global operators can be rewritten into
a form similar to thisa form similar to this
na
nw
nw
d LLLL
)(σ+=
Global Operators Global Operators -- SigmoidSigmoid
Drago
LaLa
•• Adaptation luminance computed asAdaptation luminance computed as–– Global averageGlobal average
•• Log average luminanceLog average luminance•• Interpolation schemeInterpolation scheme
–– Local averageLocal average•• Use four neighbors (good enough in practice)Use four neighbors (good enough in practice)•• Scale selection mechanismScale selection mechanism•• Bilateral filterBilateral filter
•• Typically small filter kernelsTypically small filter kernels
PhotoreceptorPhotoreceptor--basedbased
naa
pixelwa
aw
wd
aw
wd
aw
wd
fLL
LaaLL
LBBB
LGGG
LRRR
=
−+=
+=
+=
+=
)(
)1(
)(
)(
)(
σ
σ
σ
σ
na
nw
nw
d LLLL
)(σ+=
Also interpolate between R,G,B values and pixel luminance
Interpolation Scheme for LaInterpolation Scheme for La
Luminance interpolation
Interpolation Scheme for LaInterpolation Scheme for La
Between R,G,B and luminance
Bilateral filterBilateral filter
Local LaLocal La
Local bilateral
GlobalLocal SUSAN
ConclusionsConclusions
•• Most tone reproduction operators are Most tone reproduction operators are sigmoidalsigmoidal
•• Can be either local or globalCan be either local or global
•• Filter kernels (local) should be small, but in Filter kernels (local) should be small, but in any case should not cross sharp contrast any case should not cross sharp contrast boundariesboundaries
ConclusionsConclusions
•• SigmoidsSigmoids also part of most color appearance also part of most color appearance modelsmodels
•• Color appearance and tone reproduction will Color appearance and tone reproduction will convergeconverge