informatik i for d-mavt€¦ · institute of visual computing . assignment 6 3 tasks: disparity...
TRANSCRIPT
Institute of Visual Computing
Computer Vision
Exercise Session 6 – Stereo matching
Institute of Visual Computing
Assignment 6
3 Tasks: Disparity computation Winner-takes-all Graph-cut
Textured 3D model
Institute of Visual Computing
Stereo Setup
Bring two views to standard stereo setup Epipoles are at infinity Epipolar lines are parallel
Institute of Visual Computing
Planar Rectification
Code provided
Institute of Visual Computing
Disparity – SSD, Winner-Takes-All
Match windows along scanline
Remember offset of best matching window as disparity value
Institute of Visual Computing
Disparity – SSD Slow Version
Find the offset d(x, y) of matching pixels
Search algorithm (convert to gray scale rgb2gray) For each pixel (x, y), for each disparity d SSD = 0 For each pixel (i, j) in window SSD = SSD + (I1(x+i, y+j) - I2(x+i+d, y+j+d)).^2
Remember disparity with smallest SSD
SLOW!
Institute of Visual Computing
Disparity – faster version for Matlab
For each disparity d Shift entire image by d (code provided (shiftImage)) Compute image difference (SSD, SAD) Convolve with box filter Use conv2(..., ‘same’) and fspecial(‘average’,…)
Remember best disparity for each pixel mask = Idiff < bestDiff
Resize images if your stereo is taking too long
Institute of Visual Computing
Disparity result
Depth should be continuous
Depth discontinuities
Institute of Visual Computing
Disparity – Graph-Cut
Stereo is a labeling problem Assign each pixel the corresponding disparity (label) Matching pixels should have similar intensities Most nearby pixels should have similar disparities
Institute of Visual Computing
Disparity – Graph-Cut
Familiarize yourself with the sample code See the gc_example() file on color segmentation compile_gc
Adapt the code to compute the disparity Change the data cost (Dc) Compute for each pixel the SSD at each disparity Store SSD values in a m x n x r matrix, where m x n is
the image size and r is the number of disparities (labels) The rest remains unchanged
You may need to change the weighting of the terms
Institute of Visual Computing
Graph-Cut - Results
Result with simple cost function
winner-takes -a ll graphcut
Institute of Visual Computing
For each pixel find the corresponding 3D point using
Disparity maps
Camera parameters, rectifying homographies
Generate textured 3D model (code provided)
.obj-file
.mtl-file
Image file
Textured 3D model
Put everything in the same folder, load .obj-file with Meshlab.
Institute of Visual Computing
3D Point Cloud
One 3D point per pixel
Two possible approaches Standard stereo geometry Triangulation
Do not forget that homographies were applied to the images in order to rectify them and maybe also scale them!
Institute of Visual Computing
3D Point Cloud
Standard stereo geometry formula given on the slides requires Equal intrinsics for both cameras Zero skew, principal point (0,0)
Do not forget that applying homographies to the images changes the camera matrices for the respective images
Institute of Visual Computing
3D Point Cloud
Triangulation For each pixel in image 1 get the corresponding pixel
in image 2 using the disparity map Triangulate using the camera matrices
Do not forget that applying homographies to the images changes the camera matrices for the respective images
Institute of Visual Computing
Textured 3D model from an image pair
Institute of Visual Computing
Framework
Functions that need to be completed/implemented (you can add functions, of course): stereoDisparity.m diffsGC.m gcDisparity.m generatePointCloudFromDisps.m exercise6.m
Institute of Visual Computing
Framework
Several functions provided for convenience linTriang: linear triangulation leftRightCheck: see if disparity calulated for left
image agrees with disparity calculated for right image Decompose: decompose camera matrix P into K, R
and COP shiftImage: shift entire image along x-axis …