think-pair-share · 2017-11-13 · think-pair-share what visual or physiological cues help us to...
TRANSCRIPT
![Page 1: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/1.jpg)
![Page 2: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/2.jpg)
Think-Pair-Share
What visual or physiological cues help us to
perceive 3D shape and depth?
![Page 3: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/3.jpg)
Shading
[Figure from Prados & Faugeras 2006]
![Page 4: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/4.jpg)
Focus/defocus
[figs from H. Jin and P. Favaro, 2002]
Images from
same point of
view, different
camera
parameters
3d shape / depth
estimates
![Page 5: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/5.jpg)
Texture
[From A.M. Loh. The recovery of 3-D structure using visual texture patterns. PhD thesis]
![Page 6: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/6.jpg)
Perspective effects
Image credit: S. Seitz
![Page 7: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/7.jpg)
Motion
Figures from L. Zhang http://www.brainconnection.com/teasers/?main=illusion/motion-shape
![Page 8: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/8.jpg)
Occlusion
Rene Magritt'e famous painting Le Blanc-Seing (literal translation: "The Blank Signature") roughly translates as "free hand" or
"free rein".
![Page 9: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/9.jpg)
Stereo
Slides: James Hays and Kristen Grauman
![Page 10: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/10.jpg)
If stereo were critical for depth
perception, navigation, recognition, etc.,
then rabbits would never have evolved.
![Page 11: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/11.jpg)
![Page 12: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/12.jpg)
Devin Montes
![Page 13: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/13.jpg)
Human stereopsisHuman eyes fixate on point in space – rotate so that corresponding images form in centers of fovea.
![Page 14: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/14.jpg)
Disparity occurs when
eyes fixate on one object;
others appear at different
visual angles.
Disparity is distance from
b1 to b2 along retina.
Human stereopsis: disparity
![Page 15: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/15.jpg)
Yes, you can be stereoblind.
![Page 16: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/16.jpg)
Random dot stereograms
• Julesz 1960:
Do we identify local brightness patterns before fusion (monocular process) or after (binocular)?
• Think Pair Share – yes / no? how to test?
![Page 17: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/17.jpg)
Random dot stereograms
• Julesz 1960:
Do we identify local brightness patterns before fusion (monocular process) or after (binocular)?
• To test: pair of synthetic images obtained by randomly spraying black dots on white objects
![Page 18: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/18.jpg)
Random dot stereograms
Forsyth & Ponce
![Page 19: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/19.jpg)
Random dot stereograms
![Page 20: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/20.jpg)
1. Create an image of suitable size. Fill it with random dots. Duplicate the image.
2. Select a region in one image.
CC BY-SA 3.0, https://en.wikipedia.org/wiki/Random_dot_stereogram
3. Shift this region horizontally by a small amount. The stereogram is complete.
![Page 21: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/21.jpg)
Random dot stereograms
• When viewed monocularly, they appear random; when viewed stereoscopically, see 3d structure.
• Human binocular fusion not directly associated with the physical retinas; must involve the central nervous system (V2, for instance).
• Imaginary “cyclopean retina” that combines the left and right image stimuli as a single unit.
• High level scene understanding not required for stereo…but, high level scene understanding is arguably better than stereo.
![Page 22: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/22.jpg)
Autostereograms – ‘Magic Eye’
Images from magiceye.com
Exploit disparity as
depth cue using single
image.
(Single image random
dot stereogram, Single
image stereogram)
![Page 23: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/23.jpg)
Images from magiceye.com
Autostereograms
![Page 24: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/24.jpg)
Stereo photography and stereo viewers
Invented by Sir Charles Wheatstone, 1838Image from fisher-price.com
Take two pictures of the same subject from two slightly
different viewpoints and display so that each eye sees
only one of the images.
![Page 25: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/25.jpg)
Anaglyph stereo
http://www.johnsonshawmuseum.org
![Page 26: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/26.jpg)
http://www.johnsonshawmuseum.org
![Page 27: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/27.jpg)
Wiggle images
http://www.well.com/~jimg/stereo/stereo_list.html
![Page 28: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/28.jpg)
Two cameras, simultaneous
views
Single moving camera and
static scene
Stereo vision
![Page 29: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/29.jpg)
Why multiple views?
Structure and depth can be ambiguous from single views...
Images from Lana Lazebnik
![Page 30: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/30.jpg)
Why multiple views?
Points at different depths along a line project to a single point
Optical center
P1
P2
P1’=P2’
![Page 31: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/31.jpg)
Multiple views
Hartley and Zisserman
Lowe
Stereo vision
Structure from motion
Optical flow
![Page 32: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/32.jpg)
Multi-view geometry problems
• Stereo correspondence: Given a point in one of the
images, where could its corresponding points be in the
other images?
Camera 3
R3,t3
Camera 1Camera 2
R1,t1 R2,t2Slide credit:
Noah Snavely
![Page 33: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/33.jpg)
Multi-view geometry problems
• Structure: Given projections of the same 3D point in two
or more images, compute the 3D coordinates of that point
Camera 3
R3,t3 Slide credit:
Noah Snavely
?
Camera 1Camera 2
R1,t1 R2,t2
![Page 34: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/34.jpg)
Multi-view geometry problems
• Motion: Given a set of corresponding points in two or
more images, compute the camera parameters
Camera 1Camera 2 Camera 3
R1,t1 R2,t2R3,t3
? ? ? Slide credit:
Noah Snavely
![Page 35: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/35.jpg)
Multi-view geometry problems
• Optical flow: Given two images, find the location of a world
point in a second close-by image with no camera info.
Camera 1
Camera 2
![Page 36: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/36.jpg)
Multiple views - Dogception
![Page 37: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/37.jpg)
Estimating depth with stereo
• Stereo: shape from “motion” between two views
• We’ll need to consider:
• Info on camera pose (“calibration”)
• Image point correspondences
scene point
optical
center
image plane
James Hays
![Page 38: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/38.jpg)
Geometry for a simple stereo system
• Let’s look at a simple stereo system first.
• Assume:
– parallel optical axes,
– known camera parameters (i.e., calibrated cameras):
![Page 39: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/39.jpg)
baseline
optical
center
(left)
optical
center
(right)
Focal
length
World
point
image point
(left)
image point
(right)
Depth of p
![Page 40: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/40.jpg)
• Assume parallel optical axes, known camera parameters
(i.e., calibrated cameras). What is expression for Z?
Similar triangles (pl, P, pr) and
(Ol, P, Or):
Geometry for a simple stereo system
Z
T
fZ
xxT rl
lr xx
TfZ
disparity
![Page 41: Think-Pair-Share · 2017-11-13 · Think-Pair-Share What visual or physiological cues help us to perceive 3D shape and depth? Shading [Figure from Prados & Faugeras 2006] Focus/defocus](https://reader033.vdocument.in/reader033/viewer/2022043002/5f7db3632ba38311ee1090e1/html5/thumbnails/41.jpg)
Depth from disparity
image I(x,y) image I´(x´,y´)Disparity map D(x,y)
(x´,y´)=(x+D(x,y), y)
So if we could find the corresponding points in two images,
we could estimate relative depth…