Download - CS 395/495-25: Spring 2004
![Page 1: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/1.jpg)
CS 395/495-25: Spring 2004CS 395/495-25: Spring 2004
IBMR:IBMR:Poisson SolversPoisson Solvers
Can Reconstruct Images Can Reconstruct Images from their Changesfrom their Changes
Jack TumblinJack [email protected]@cs.northwestern.edu
![Page 2: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/2.jpg)
Do pixels describe what we see?Do pixels describe what we see?
What We WantWhat We Want What We GetWhat We Get
![Page 3: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/3.jpg)
What do you see?What do you see?
What part has constant intensity?What part has constant intensity?
AA
BB
![Page 4: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/4.jpg)
What do you see?What do you see?Humans don’t sense intensities reliably, but infer them from changesHumans don’t sense intensities reliably, but infer them from changes
BB intensity is intensity is constantconstant, , AA is is darker on rightdarker on right
AA
BB
![Page 5: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/5.jpg)
What do you see?What do you see?Humans don’t sense intensities reliably, but infer them from changesHumans don’t sense intensities reliably, but infer them from changes
(tol’djah!)(tol’djah!)
AA
BB
![Page 6: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/6.jpg)
What do you see?What do you see?
What part has constant intensity?What part has constant intensity?
XX
YY
![Page 7: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/7.jpg)
What do you see?What do you see?
What part has constant intensity?What part has constant intensity? NEITHER! NEITHER!
XX
YY
Humans don’t sense intensities reliably, but infer them from changesHumans don’t sense intensities reliably, but infer them from changes
![Page 8: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/8.jpg)
What do you see?What do you see?
What part has constant intensity?What part has constant intensity? NEITHER! NEITHER!
XX
YY
ConstantConstant
Humans don’t sense intensities reliably, but infer them from changesHumans don’t sense intensities reliably, but infer them from changes
![Page 9: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/9.jpg)
What do you see?What do you see?
Example: aren’t all the dots white? (http://udel.edu/~jgephart/fun2.htm)
Humans don’t sense intensities reliably, but infer them from changesHumans don’t sense intensities reliably, but infer them from changes
![Page 10: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/10.jpg)
Why Pixels Could be Improved:Why Pixels Could be Improved:• People see (or think they see) People see (or think they see) changeschanges
finite features that may have infinite bandwidthfinite features that may have infinite bandwidth occlusion, depth, collision time, trajectory changes, occlusion, depth, collision time, trajectory changes,
corner, cone tip, boundaries, edges, occlusions, corner, cone tip, boundaries, edges, occlusions, shadow details, contact points, velocity & direction shadow details, contact points, velocity & direction changes...changes...
Pixels only Pixels only approximateapproximate changes, changes, and approximate discontinuous changes poorly;and approximate discontinuous changes poorly;
object boundaries, silhouettes, etc.object boundaries, silhouettes, etc.They force indirect estimation...They force indirect estimation...
![Page 11: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/11.jpg)
How? Retinal Receptive Fields…How? Retinal Receptive Fields…• 130M Photoreceptors130M Photoreceptors1M optic nerve fibers1M optic nerve fibers
• Center-Surround Antagonism:Center-Surround Antagonism:Out Out Center - (avg surround) Center - (avg surround)
• Complementary Complementary ON-center, OFF-center typesON-center, OFF-center types
• Center responds quickly;Center responds quickly; Surround responds more slowlySurround responds more slowly
• Output: ‘recent local change’Output: ‘recent local change’
++ -- -- -- -- --
-- -- --
++++++ ++++
++ ++ ++--
![Page 12: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/12.jpg)
Complementary Receptive FieldsComplementary Receptive Fields• Retina is ~differential for small signalsRetina is ~differential for small signals
– Better SNRBetter SNR– Can signal ambiguity (eyes closed, etc)Can signal ambiguity (eyes closed, etc)– Allows quality/fault detectionAllows quality/fault detection
++ -- -- -- -- --
-- -- --
++++++ ++++
++ ++ ++--
ctr/surrctr/surr
Firing Rate (Hz)Firing Rate (Hz)
1010
100100
5050
1010110.10.1ctr/surrctr/surr
1010
100100
5050Firing Firing Rate (Hz)Rate (Hz)
![Page 13: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/13.jpg)
Yarbus (1950s): Pioneer of Yarbus (1950s): Pioneer of Retinal Stabilization ExperimentsRetinal Stabilization Experiments
(inspired a flood of others…)(inspired a flood of others…)
![Page 14: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/14.jpg)
![Page 15: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/15.jpg)
‘‘BUT BUT HEREHERE is a is a Big ring of Big ring of VERY strong change!’VERY strong change!’
‘Not much to see.(pink-ish?)’
‘mm, nothing much. (green-ish?)’
Strongly ImpliesStrongly Implies‘‘Filling In’ requiresFilling In’ requiresNystagmus forNystagmus fortemporal transients...temporal transients...
![Page 16: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/16.jpg)
‘Not much to see.(pink-ish?)’
‘mm, Not much to see. (green-ish?)’
![Page 17: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/17.jpg)
What ‘Changes’ do we Sense?What ‘Changes’ do we Sense?• Intensity (luminance) vs. local positionIntensity (luminance) vs. local position• Color (chrominance) vs. local positionColor (chrominance) vs. local position• Intensity vs. time (‘flicker’)Intensity vs. time (‘flicker’)• Color vs. timeColor vs. time• VERY weak, low-res: overall intensityVERY weak, low-res: overall intensity• Inertial changes: movement, velocity…Inertial changes: movement, velocity…
Compensated eye moves Compensated eye moves (saccade, glissade, smooth-pursuit…(saccade, glissade, smooth-pursuit…
• Higher-level attributes? Umm, er, uh,….Higher-level attributes? Umm, er, uh,….
![Page 18: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/18.jpg)
‘‘Digital’ Image: a 2D Grid of NumbersDigital’ Image: a 2D Grid of Numbers• NO intrinsic meaning—use it for NO intrinsic meaning—use it for anything:anything: reflectance, transparency, illumination, normal reflectance, transparency, illumination, normal
direction, material, velocity. BUT usually direction, material, velocity. BUT usually ‘intensity’‘intensity’
xx
yy
xx
yy
![Page 19: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/19.jpg)
2D Images Described by Change?2D Images Described by Change?• Image intensity as height field Image intensity as height field f(x,y)f(x,y)::• 11stst derivative— derivative—
Gradient: the ‘uphill’ vector at point x,y Gradient: the ‘uphill’ vector at point x,y = = f(x,y) = (f(x,y) = (f(x,y)f(x,y)//xx, , f(x,y)f(x,y)//yy) = ) = ff
xx
yy
f(x,y)
f(x,y)
![Page 20: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/20.jpg)
2D Images Described by Change?2D Images Described by Change?• Image intensity as height field Image intensity as height field f(x,y)f(x,y)::• 22stst derivative— derivative—
Gradient: the ‘uphill’ vector at point x,y Gradient: the ‘uphill’ vector at point x,y = = f(x,y) = (f(x,y) = (f(x,y)f(x,y)//xx, , f(x,y)f(x,y)//yy) = ) = ff
xx
yy
f(x,y)
f(x,y)
![Page 21: CS 395/495-25: Spring 2004](https://reader036.vdocument.in/reader036/viewer/2022070502/56814af9550346895db80a8f/html5/thumbnails/21.jpg)
Review: Div, Grad and CurlReview: Div, Grad and Curl• Formalized, computable ‘Local Change’Formalized, computable ‘Local Change’