what have we leaned so far? camera structure eye structure project 1: high dynamic range imaging
TRANSCRIPT
![Page 1: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/1.jpg)
What have we leaned so far?• Camera structure• Eye structure
Project 1: High Dynamic Range Imaging
![Page 2: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/2.jpg)
What have we learned so far?• Image Filtering• Image Warping• Camera Projection Model
Project 2: Panoramic Image Stitching
![Page 3: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/3.jpg)
What have we learned so far?• Projective Geometry• Single View Modeling• Shading Model
Project 3: Photometric Stereo
![Page 4: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/4.jpg)
Today• 3D modeling from two images – Stereo
![Page 5: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/5.jpg)
![Page 6: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/6.jpg)
Public Library, Stereoscopic Looking Room, Chicago, by Phillips, 1923
![Page 7: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/7.jpg)
Inventor: Sir Charles Wheatstone, 1802 - 1875 http://en.wikipedia.org/wiki/Sir_Charles_Wheatstone
![Page 8: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/8.jpg)
Inventor: Sir Charles Wheatstone, 1802 - 1875 http://en.wikipedia.org/wiki/Wheatstone_bridge
![Page 9: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/9.jpg)
Stereograms online• UCR stereographs
• http://www.cmp.ucr.edu/site/exhibitions/stereo/• The Art of Stereo Photography
• http://www.photostuff.co.uk/stereo.htm• History of Stereo Photography
• http://www.rpi.edu/~ruiz/stereo_history/text/historystereog.html• Double Exposure
• http://home.centurytel.net/s3dcor/index.html• Stereo Photography
• http://www.shortcourses.com/book01/chapter09.htm• 3D Photography links
• http://www.studyweb.com/links/5243.html• National Stereoscopic Association
• http://204.248.144.203/3dLibrary/welcome.html• Books on Stereo Photography
• http://userwww.sfsu.edu/~hl/3d.biblio.html
A free pair of red-blue stereo glasses can be ordered from Rainbow Symphony Inc• http://www.rainbowsymphony.com/freestuff.html
![Page 10: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/10.jpg)
FUJIFILM, September 23, 2008
Fuji 3D printing
![Page 11: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/11.jpg)
Stereo
scene pointscene point
optical centeroptical center
image planeimage plane
![Page 12: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/12.jpg)
Stereo
Basic Principle: Triangulation• Gives reconstruction as intersection of two rays
• Requires – calibration
– point correspondence
![Page 13: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/13.jpg)
Stereo correspondence• Determine Pixel Correspondence
• Pairs of points that correspond to same scene point
Epipolar Constraint• Reduces correspondence problem to 1D search along conjugate
epipolar lines• Java demo: http://www.ai.sri.com/~luong/research/Meta3DViewer/EpipolarGeo.html
epipolar planeepipolar lineepipolar lineepipolar lineepipolar line
![Page 14: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/14.jpg)
Epipolar Line Example
courtesy of Marc Pollefeys
![Page 15: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/15.jpg)
Stereo image rectification
![Page 16: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/16.jpg)
Stereo image rectification
• reproject image planes onto a common• plane parallel to the line between optical
centers• pixel motion is horizontal after this transformation• two homographies (3x3 transform), one for each input
image reprojection C. Loop and Z. Zhang. Computing Rectifying Homographies for Stereo
Vision. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
![Page 17: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/17.jpg)
Epipolar Line Example
courtesy of Marc Pollefeys
![Page 18: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/18.jpg)
Epipolar Line Example
courtesy of Marc Pollefeys
![Page 19: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/19.jpg)
Stereo matching algorithms
• Match Pixels in Conjugate Epipolar Lines• Assume brightness constancy• This is a tough problem• Numerous approaches
– A good survey and evaluation: http://www.middlebury.edu/stereo/
![Page 20: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/20.jpg)
Basic stereo algorithm
For each epipolar line
For each pixel in the left image• compare with every pixel on same epipolar line in right image
• pick pixel with minimum match cost
Improvement: match windows
![Page 21: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/21.jpg)
Basic stereo algorithm
• For each pixel• For each disparity
– For each pixel in window» Compute difference
• Find disparity with minimum SSD
![Page 22: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/22.jpg)
Reverse order of loops• For each disparity
• For each pixel
– For each pixel in window» Compute difference
• Find disparity with minimum SSD at each pixel
![Page 23: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/23.jpg)
Incremental computation
• Given SSD of a window, at some disparity
Image 1Image 1
Image 2Image 2
![Page 24: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/24.jpg)
Incremental computation
• Want: SSD at next location
Image 1Image 1
Image 2Image 2
![Page 25: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/25.jpg)
Incremental computation
• Subtract contributions from leftmost column, add contributions from rightmost column
Image 1Image 1
Image 2Image 2
++++++++++
----------
----------
++++++++++
![Page 26: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/26.jpg)
Selecting window size
• Small window: more detail, but more noise• Large window: more robustness, less detail• Example:
![Page 27: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/27.jpg)
Selecting window size
3 pixel window3 pixel window 20 pixel window20 pixel window
Why?
![Page 28: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/28.jpg)
Non-square windows
• Compromise: have a large window, but higher weight near the center
• Example: Gaussian• Example: Shifted windows (computation cost?)
![Page 29: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/29.jpg)
Problems with window matching
• No guarantee that the matching is one-to-one• Hard to balance window size and smoothness
![Page 30: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/30.jpg)
A global approach
• Finding correspondence between a pair of epipolar lines for all pixels simultaneously
![Page 31: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/31.jpg)
A global approach
left
right
left
right
left
right
Define an evaluation score for each configuration, choose the best matching configuration
![Page 32: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/32.jpg)
A global approach• How to define the evaluation score?
• How about the sum of corresponding pixel difference?
![Page 33: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/33.jpg)
Ordering constraint
• Order of matching features usually the samein both images
• But not always: occlusion
![Page 34: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/34.jpg)
Dynamic programming
• Treat pixel correspondence as graph problem
Left imageLeft imagepixelspixels
Right image pixelsRight image pixels
11 22 33 4411
22
33
44
![Page 35: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/35.jpg)
Dynamic programming
• Find min-cost path through graph
Left imageLeft imagepixelspixels
Right image pixelsRight image pixels
11 22 33 4411
22
33
44
11
3344
11
222233
44
![Page 36: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/36.jpg)
Dynamic Programming Results
![Page 37: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/37.jpg)
Energy minimization
• Another global approach to improve quality of correspondences
• Assumption: disparities vary (mostly) smoothly
• Minimize energy function:Edata+Esmoothness
• Edata: how well does disparity match data
• Esmoothness: how well does disparity match
that of neighbors – regularization
![Page 38: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/38.jpg)
Stereo as energy minimization• Matching Cost Formulated as Energy
• “data” term penalizing bad matches
• “neighborhood term” encouraging spatial smoothness
),(),(),,( ydxyxdyxD JI
similar)something(or
d2 and d1 labels with pixelsadjacentofcost
21
21 ),(
dd
ddV
)2,2(),1,1(
2,21,1),(
, ),(),,(})({yxyxneighbors
yxyxyx
yx ddVdyxDdE
![Page 39: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/39.jpg)
Energy minimization
• Many local minimum• Why?• Gradient descent doesn’t work well
• In practice, disparities only piecewise smooth• Design smoothness function that doesn’t penalize
large jumps too much• Example: V()=min(||, K)
– Non-convex
)2,2(),1,1(
2,21,1),(
, ),(),,(})({yxyxneighbors
yxyxyx
yx ddVdyxDdE
![Page 40: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/40.jpg)
Energy minimization
• Hard to find global minima of non-smooth functions• Many local minima• Provably NP-hard
• Practical algorithms look for approximate minima (e.g., simulated annealing)
![Page 41: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/41.jpg)
Energy minimization via graph cuts
Labels
(disparities)
d1
d2
d3
edge weight
edge weight
),,( 3dyxD
),( 11 ddV
![Page 42: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/42.jpg)
• Graph Cost• Matching cost between images• Neighborhood matching term• Goal: figure out which labels are connected to which pixels
d1
d2
d3
Energy minimization via graph cuts
![Page 43: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/43.jpg)
Energy minimization via graph cuts
d1
d2
d3
• Graph Cut• Delete enough edges so that
– each pixel is connected to exactly one label node • Cost of a cut: sum of deleted edge weights• Finding min cost cut equivalent to finding global minimum of
energy function
![Page 44: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/44.jpg)
Computing a multiway cut
• With 2 labels: classical min-cut problem• Solvable by standard flow algorithms
– polynomial time in theory, nearly linear in practice• More than 2 terminals: NP-hard
[Dahlhaus et al., STOC ‘92]• Efficient approximation algorithms exist
• Yuri Boykov, Olga Veksler and Ramin Zabih, Fast Approximate Energy Minimization via Graph Cuts, International Conference on Computer Vision, September 1999.
• Within a factor of 2 of optimal• Computes local minimum in a strong sense
– even very large moves will not improve the energy
![Page 45: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/45.jpg)
Move examples
Starting point
Red-blue swap move
Green expansion move
![Page 46: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/46.jpg)
The swap move algorithm1. Start with an arbitrary labeling
2. Cycle through every label pair (A,B) in some order
2.1 Find the lowest E labeling within a single AB-swap
2.2 Go there if it’s lower E than the current labeling3. If E did not decrease in the cycle, we’re done
Otherwise, go to step 2
Original graph
A
B
AB subgraph(run min-cut on this graph)
B
A
![Page 47: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/47.jpg)
The expansion move algorithm
1. Start with an arbitrary labeling
2. Cycle through every label A in some order
2.1 Find the lowest E labeling within a single A-expansion
2.2 Go there if it’s lower E than the current labeling3. If E did not decrease in the cycle, we’re done Otherwise, go to
step 2
Multi-way cut A sequence of binary optimization problems
![Page 48: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/48.jpg)
Stereo results
ground truthscene
• Data from University of Tsukuba
http://cat.middlebury.edu/stereo/
![Page 49: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/49.jpg)
Results with window correlation
normalized correlation(best window size)
ground truth
![Page 50: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/50.jpg)
Results with graph cuts
ground truthgraph cuts(Potts model E,expansion move algorithm)
![Page 51: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/51.jpg)
Depth from disparity
f
x x’
baseline
z
C C’
X
f
input image (1 of 2) [Szeliski & Kang ‘95]
depth map 3D rendering
![Page 52: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/52.jpg)
Real-time stereo
• Used for robot navigation (and other tasks)• Several software-based real-time stereo techniques have
been developed (most based on simple discrete search)
Nomad robot searches for meteorites in Antarticahttp://www.frc.ri.cmu.edu/projects/meteorobot/index.html
![Page 53: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/53.jpg)
• Camera calibration errors• Poor image resolution• Occlusions• Violations of brightness constancy (specular reflections)• Large motions• Low-contrast image regions
Stereo reconstruction pipeline• Steps
• Calibrate cameras• Rectify images• Compute disparity• Estimate depth
What will cause errors?
![Page 54: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/54.jpg)
Spacetime Stereo
Li Zhang, Noah Snavely, Brian Curless, Steven SeitzCVPR 2003, SIGGRAPH 2004
![Page 55: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/55.jpg)
Stereo
![Page 56: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/56.jpg)
Stereo
???
![Page 57: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/57.jpg)
Marker-based Face Capture
The Polar Express, 2004
“The largest intractable problem with ‘The Polar Express’ is that the motion-capture technology used to create the human figures has resulted in a film filled with creepily unlifelike beings.”
New York Times Review, Nov 2004
![Page 58: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/58.jpg)
Stereo
![Page 59: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/59.jpg)
Stereo
Frame-by-Frame Stereo WH = 1515 Window
A Pair of Videos 640480@60fps Each
Inaccurate & Jittering
![Page 60: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/60.jpg)
3D Surface
Spacetime Stereo
![Page 61: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/61.jpg)
Spacetime Stereo
Time
3D Surface
![Page 62: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/62.jpg)
Spacetime Stereo
Time
3D Surface
![Page 63: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/63.jpg)
Spacetime Stereo
3D Surface
Time
![Page 64: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/64.jpg)
Spacetime Stereo
Surface Motion
Time
![Page 65: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/65.jpg)
Spacetime Stereo
Surface Motion
Time=0
![Page 66: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/66.jpg)
Spacetime Stereo
Surface Motion
Time=1
![Page 67: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/67.jpg)
Spacetime Stereo
Surface Motion
Time=2
![Page 68: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/68.jpg)
Spacetime Stereo
Surface Motion
Time=3
![Page 69: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/69.jpg)
Spacetime Stereo
Surface Motion
Time=4
![Page 70: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/70.jpg)
Surface Motion
• Matching Volumetric Window
• Affine Window Deformation
Key ideas:
Spacetime Stereo
Time
![Page 71: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/71.jpg)
Spacetime Stereo
Time
![Page 72: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/72.jpg)
Spacetime Stereo
Time
![Page 73: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/73.jpg)
Spacetime Stereo
![Page 74: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/74.jpg)
Spacetime Stereo
A Pair of Videos 640480@60fps Each
Spacetime Stereo WHT = 955 Window
![Page 75: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/75.jpg)
Frame-by-Frame vs. Spacetime Stereo
Spacetime Stereo WHT = 955 Window
Frame-by-Frame WH = 1515 Window
Spatially More AccurateTemporally More Stable
![Page 76: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/76.jpg)
Video Projectors
Color Cameras
Black & White Cameras
Spacetime Face Capture System
![Page 77: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/77.jpg)
System in Action
![Page 78: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/78.jpg)
Input Videos (640480, 60fps)
![Page 79: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/79.jpg)
Spacetime Stereo Reconstruction
![Page 80: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/80.jpg)
Creating a Face Database
![Page 81: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/81.jpg)
Creating a Face Database
[Zhang et al. SIGGRAPH’04]
…
![Page 82: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/82.jpg)
Application 1: Expression Synthesis
[Zhang et al. SIGGRAPH’04]
…
A New Expression:
![Page 83: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/83.jpg)
Application 2: Facial Animation
[Zhang et al. SIGGRAPH’04]
…
![Page 84: What have we leaned so far? Camera structure Eye structure Project 1: High Dynamic Range Imaging](https://reader038.vdocument.in/reader038/viewer/2022102718/56649dbe5503460f94ab0f1d/html5/thumbnails/84.jpg)
Keyframe Animation