Stanford CS223B Computer Vision, Winter 2005

Lecture 5: Stereo I

Sebastian Thrun, Stanford

Rick Szeliski, Microsoft

Hendrik Dahlkamp and Dan Morris, Stanford

StereoStereo

Sebastian Thrun Stanford University CS223B Computer Vision

Stereo Vision: Illustration

http://www.well.com/user/jimg/stereo/stereo_list.html

Sebastian Thrun Stanford University CS223B Computer Vision

Stereo Vision: Outline

Basic Equations Epipolar Geometry Image Rectification Reconstruction Correspondence Dense and Layered Stereo (Active Range Imaging Techniques)

Sebastian Thrun Stanford University CS223B Computer Vision

Pinhole Camera Model

Imageplane Focal length f

Center ofprojection

Sebastian Thrun Stanford University CS223B Computer Vision

Pinhole Camera Model

Imageplane

),,( ZYXP

),,( ZYXP

f

Oy

x

z

Z

Z

Y

Y

X

X

ZZ

YY

XX

OPPO

Sebastian Thrun Stanford University CS223B Computer Vision

Pinhole Camera Model

Imageplane

),,( ZYXP

),,( ZYXP

f

Oy

x

z

)1,,()1,,(),,(Z

Yf

Z

XfyxZYX

YyXxZ

YfY

Z

XfXfZ

Sebastian Thrun Stanford University CS223B Computer Vision

Basic Stereo Derivations

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

B

BfxxZ ,,, offunction a as for expression Derive 21

1p

2p

Sebastian Thrun Stanford University CS223B Computer Vision

Basic Stereo Derivations

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

B

211

11

1

12

1

11 ,

xx

BfZ

Z

Bfx

Z

BXfx

Z

Xfx

Sebastian Thrun Stanford University CS223B Computer Vision

What If…?

),,(1 ZYXP 1Oy

x

z

f

2Oy

x

z

B

1p

2p

),,(1 ZYXP 1Oy

x

z

1p

f2O

y

x

z

2p

Sebastian Thrun Stanford University CS223B Computer Vision

Epipolar Geometry

pl pr

P

Ol Or

Xl

Xr

Pl Pr

fl fr

Zl

Yl

Zr

Yr

Rrotation Tontranslati

Sebastian Thrun Stanford University CS223B Computer Vision

Epipolar Geometry

plp

r

P

Ol Orel er

Pl Pr

Epipolar Plane

Epipolar Lines

Epipoles

Sebastian Thrun Stanford University CS223B Computer Vision

Epipolar Geometry

Epipolar plane: plane going through point P and the centers of projection (COPs) of the two cameras

Epipoles: The image in one camera of the COP of the other

Epipolar Constraint: Corresponding points must lie on epipolar lines

Sebastian Thrun Stanford University CS223B Computer Vision

Essential Matrix

pl pr

P

Ol Orel er

Pl Pr

Orthogonality T, Pl, PlT: 0)( lT

l PTTP

)( TPRP lr Coordinate Transformation:

0

0

0

xy

xz

yz

TT

TT

TT

S

ll SPPT

0)( lT

rT SPPR

0lT

r RSPP

0)( lT

rT PTPRResolves to

RSE Essential Matrix 0lT

r EPP

Sebastian Thrun Stanford University CS223B Computer Vision

Essential Matrix

pl pr

P

Ol Orel er

Pl Pr

0

0

0

xy

xz

yz

TT

TT

TT

SRSE Essential Matrix

0 lTr Epp0l

Tr EPP

Projective Line: lr Epu

Sebastian Thrun Stanford University CS223B Computer Vision

Fundamental Matrix

Same as Essential Matrix in Camera Pixel Coordinates

0lTr pFp

0lTr Epp

Pixel coordinates 1 lT

r EMMF

Intrinsic parameters

Sebastian Thrun Stanford University CS223B Computer Vision

Computing F: The Eight-Point Algorithm

Input: n point correspondences ( n >= 8)– Construct homogeneous system Ax= 0 from

• x = (f11,f12, ,f13, f21,f22,f23 f31,f32, f33) : entries in F• Each correspondence give one equation• A is a nx9 matrix

– Obtain estimate F^ by SVD of A:• x (up to a scale) is column of V corresponding to the least

singular value– Enforce singularity constraint: since Rank (F) = 2

• Compute SVD of F:• Set the smallest singular value to 0: D -> D’• Correct estimate of F :

Output: the estimate of the fundamental matrix F’ Similarly we can compute E given intrinsic

parameters

0lTr pFp

TUDVA

TUDVF ˆ

TVUDF' '

Sebastian Thrun Stanford University CS223B Computer Vision

Recitification

Idea: Align Epipolar Lines with Scan Lines.

Question: What type transformation?

Sebastian Thrun Stanford University CS223B Computer Vision

Locating the Epipoles

pl pr

P

Ol Orel er

Pl Pr

Input: Fundamental Matrix F– Find the SVD of F– The epipole el is the column of V corresponding to the

null singular value (as shown above)– The epipole er is the column of U corresponding to the

null singular value (similar treatment as for el) Output: Epipole el and er

TUDVF

el lies on all the epipolar lines of the left image

0lTr pFp

0lTr eFp

0leF

Sebastian Thrun Stanford University CS223B Computer Vision

Stereo Rectification (see Trucco)

Stereo System with Parallel Optical AxesEpipoles are at infinity

Horizontal epipolar lines

pl

pr

P

Ol Or

Xl

Xr

Pl Pr

Zl

Yl

Zr

Yr

T

Sebastian Thrun Stanford University CS223B Computer Vision

pl

pr

P

Ol Or

Pl Pr

Reconstruction (3-D): Idealized

Sebastian Thrun Stanford University CS223B Computer Vision

pl

pr

P

Ol Or

Pl Pr

Reconstruction (3-D): Real

See Trucco/Verri, pages 161-171

Sebastian Thrun Stanford University CS223B Computer Vision

Summary Stereo Vision (Class 1)

Epipolar Geometry: Corresponding points lie on epipolar

line

Essential/Fundamental matrix: Defines this line

Eight-Point Algorithm: Recovers Fundamental matrix

Rectification: Epipolar lines parallel to scanlines

Reconstruction: Minimize quadratic distance