Computer Vision April 2004 L1.1© 2004 by Davi Geiger

Binocular Stereo

Binocular Stereo

Left Image Right Image

Computer Vision April 2004 L1.2© 2004 by Davi Geiger

Each potential match is represented by a square. The black ones represent the most likely scene to “explain” the image, but other combinations could have given rise to the same image (e.g., red)

Stereo Correspondence: Ambiguities

What makes the set of black squares preferred/unique is that they have similar disparity values, the ordering constraint is satisfied and there is a unique match for each point. Any other set that could have given rise to the two images would have disparity values varying more, and either the ordering constraint violated or the uniqueness violated. The disparity values are inversely proportional to the depth values

Computer Vision April 2004 L1.3© 2004 by Davi Geiger

Rig

ht

boundary

no m

atc

h

Boundary no matchLeft

depth discontinuity

Surface orientation

discontinuity

A BC

DE F

AB

A

CD

DC

F

FE

Stereo Correspondence: Matching Space

F D C B A

AC

D

E

F

Computer Vision April 2004 L1.4© 2004 by Davi Geiger

Stereo Correspondence: Constraints

Left Epipolar Line

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

j-1 j=3 j+1

t+1

t=5

t-1w

w=2

Right Epipolar LineSmoothness (+Ordering)

oooooo

oooooo

ooooooo

oooooo

oooooo

oooooo

ooooooo

j-1 j=3 j+1

t+1

t=5

t-1w

w=2

Left Epipolar Line

Right Epipolar Line

w=3

w=0

w=-2

Uniqueness

Smoothness : In nature most surfaces are smooth in depth compared to their distance to the observer, but depth discontinuities also occur.  Usually implies an ordering constraint, where points to the right of match point to the right of . Uniqueness: There should be only one disparity value associated to each point.  

jq

tq

2and

2 and

wxj

wxtjtwjtx

Computer Vision April 2004 L1.5© 2004 by Davi Geiger

Stereo Algorithm: Data

C0(e,x,w) Є [0,1] representing how good is a match between a point (e,j) in the

left image and a point (e,t) in the right image (x=t+j, w=t-j is the disparity.) The epipolar lines are indexed by e. We use a correlation technique that computes the “angle” between two vectors representing the window values of the intensity.

)();5,0,,(ˆ)(

cos0001.0)(.)(

)()(,

0001.0)0(.)0(

)0()0(min),,(0

jwjjeIWIwhere

WIWI

WIWI

WIWI

WIWIwxeC

LL

LL

RL

LL

RL

                                                

                                                                       

                                                                                                                       

0,...1,0

)5,0,,(ˆ)5,0,,(ˆ

)0()0(

wi

RL

RL

iteIijeI

WIWI

)0( LWI

)0( RWI

)( LWI

)( RWI

)(LWI

)(RWI

Computer Vision April 2004 L1.6© 2004 by Davi Geiger

We also consider the matching of intensity edges, where x+w is odd, and so we enhance C0(e,x,w) Є [0,1]

Stereo Algorithm: Data (cont.)

We would like to distinguish the “common” cases (i) where low intensity edges match well low intensity edges from the ”rare” cases (ii) where high intensity edges match high intensity edges

So this formula must be modified….

)();5,0,2

,(ˆ)5,0,2

,(ˆ

)5,0,2

,(ˆ.)5,0,2

,(ˆ

)5,0,2

,(ˆ)5,0,2

,(ˆ

),,(

0

0

jwjwx

jeIDwx

jeIDwhere

wxteID

wxjeID

wxteID

wxjeID

wxeC

LL

RL

RL

Computer Vision April 2004 L1.7© 2004 by Davi Geiger

The stereovision algorithm produces a series of matrices Cn, which converges to a

good solution for many cases, with 0 <

 The positive feedback is given by the two neighbors of node (e,j,t) (or (e,x,w)) with matches at the same disparity w=t-j.

Stereo: Smoothing and Limit Disparity

),2,(),2,(2

1)1(),,(),,(),,( 01 wxeCwxeCwxeCwxeCwxeC nnnn

The matrix is updated only within a range of

disparity : 2D+1 , i.e.,

The rational is:

(i) Less computations

(ii) Larger disparity matches imply larger errors in 3D estimation.

Djtw ||||

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

ooooooo

j-1 j=3 j+1

t+1

t=5

t-1w

w=2

Right Epipolar Line

Left Epipolar Line

D=3

D=-3

Dw ||

Computer Vision April 2004 L1.8© 2004 by Davi Geiger

Result

Computer Vision April 2004 L1.9© 2004 by Davi Geiger

Region A

Region B

Region A Left

Region A Right

Region B Left

Region B Right

Junctions and its properties (false matches that reveal information from vertical disparities (see Malik 94, ECCV)

Some Issues in Stereo: