CS 6825: Motion Part 2 – CS 6825: Motion Part 2 – Optical FlowOptical Flow

Recall: Optical FlowRecall: Optical Flow is an approximation of the 2D motion is an approximation of the 2D motion

field. field. Motion in the world usually occurs in 3D, Motion in the world usually occurs in 3D,

but, we have a 2D image sensor. So, we but, we have a 2D image sensor. So, we see the results as movement across the see the results as movement across the 2D image plane. 2D image plane.

Hence we are seeing the projection of the Hence we are seeing the projection of the 3D moving points onto the image plane. 3D moving points onto the image plane.

00

00

10

0

00

ˆby torelated

imagein induces , velocity has point Object

zr

rrrr

rv

rv

vv

f

dt

d

dt

d

P

ii

i

i

Motion Field and Optical Flow Motion Field and Optical Flow FieldField

Motion field: projection of 3D motion vectors on image planeMotion field: projection of 3D motion vectors on image plane

Optical flow field: apparent motion of brightness patternsOptical flow field: apparent motion of brightness patterns We equate motion field with optical flow fieldWe equate motion field with optical flow field

Brightness Constancy EquationBrightness Constancy Equation

Let P be a moving point in 3D:Let P be a moving point in 3D:• At time t, P has coords (X(t),Y(t),Z(t))At time t, P has coords (X(t),Y(t),Z(t))• Let p=(x(t),y(t)) be the coords. of its Let p=(x(t),y(t)) be the coords. of its

image at time t.image at time t.• Let E(x(t),y(t),t) be the brightness at p at Let E(x(t),y(t),t) be the brightness at p at

time t.time t. Brightness Constancy Assumption:Brightness Constancy Assumption:

• As P moves over time, E(x(t),y(t),t) As P moves over time, E(x(t),y(t),t) remains constant.remains constant.

Brightness Constancy Brightness Constancy EquationEquation

Taking derivative wrt time:Taking derivative wrt time:

Brightness Constancy EquationBrightness Constancy Equation

LetLet(Frame spatial gradient)(Frame spatial gradient)

(optical flow)(optical flow)

andand (derivative across frames)(derivative across frames)

Brightness Constancy EquationBrightness Constancy Equation

Becomes:Becomes:

vvxx

vv

yy

rr E E

The Optical Flow The Optical Flow is CONSTRAINED is CONSTRAINED to be on a line !to be on a line !

-E-Ett/|/|rr E| E|Can calculate these: Different techniques to figure these out.

E is the spatial change in brightness in image i

Et is the difference in the brightness at (x,y) between image i and image i+1

We want to calculate v = [dx/dt dy/dt]