lecture 22: camera motionrtc12/cse486/lecture22.pdf · lecture 22: camera motion readings: ......
TRANSCRIPT
![Page 1: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/1.jpg)
CSE486, Penn StateRobert Collins
Lecture 22:
Camera Motion
Readings: T&V Sec 8.1 and 8.2
![Page 2: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/2.jpg)
CSE486, Penn StateRobert Collins
Moving Camera
yaw
pitch
roll
Tx
TyTz
From one time to the next, the camera undergoesrotation (roll, pitch, yaw) and translation (tx,ty,tz)
t t+1
t+2t-1
t-2t-3
t-4
Camera takes a sequence of images (frames)indexed by time t
![Page 3: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/3.jpg)
CSE486, Penn StateRobert Collins
Motion (Displacement) Fields
Time t
Time t+1
Displacement field
![Page 4: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/4.jpg)
CSE486, Penn StateRobert Collins
Motion Field vs Optic Flow
Motion Field: projection of 3D relative velocity vectors onto the 2D image plane.
Optic Flow: observed 2D displacements of brightness patterns in the image.
Motion field is what we want to know.Optic flow is what we can estimate.
![Page 5: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/5.jpg)
CSE486, Penn StateRobert Collins
Motion Field vs Optic FlowSometimes optic flow is a good approximationto the unknown motion flow.
optic flow field
We can then infer relative motion between the camera and objects in the world.
![Page 6: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/6.jpg)
CSE486, Penn StateRobert Collins
Warning: Optic Flow ≠ Motion Field
Consider a moving light source:
MF = 0 since the points on the scene are not movingOF ≠ 0 since there is a moving pattern in the images
![Page 7: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/7.jpg)
CSE486, Penn StateRobert Collins
Motion Field
We are going to derive an equation relating 3D scene structure and velocity to the 2D motion flow field.
![Page 8: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/8.jpg)
CSE486, Penn StateRobert Collins
Motion Field
What is a Field anyways?
Image a vector at each point in space. This is avector field. In 3D space, we will look at the fieldof 3D velocity vectors induced by camera motion.
In 2D, we will be looking at the projections of those3D vectors in the image. There will be a 2D flow vector at each point in the image. This is the 2Dmotion flow field.
![Page 9: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/9.jpg)
CSE486, Penn StateRobert Collins
Recall : General Projection Equation
Relative R,TProjectionInternalparams
Strategy:1) Assume internal params known (set to identity)2) At time t, set R=I, T=03) At time t+1, movement generates a relative R and T simplifying assumption: small motion --> small rotation
4) Compute 3D velocity vector 5) Compute 2D velocity vector a function of X,Y,Z,R,T,f
![Page 10: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/10.jpg)
CSE486, Penn StateRobert Collins
Displacement of 3D World Point
Time t: 3D position PTime t+1: 3D position RP+T
3D Displacement = RP+T - P
Now consider short time period (like time betweentwo video frames = 1/30 sec). Can assume a smallrotation angle in that amount of time. Make a smallangle approximation and rewrite displacement. In thelimit (infinitesimal time period), we will get a velocity.
![Page 11: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/11.jpg)
CSE486, Penn StateRobert Collins Write Rotation matrix in terms of Euler Angles
![Page 12: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/12.jpg)
CSE486, Penn StateRobert Collins
![Page 13: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/13.jpg)
CSE486, Penn StateRobert Collins
Small Angle Approx
for small angle x
xx ≈sin
1cos ≈x
x
0sinsin ≈yx
![Page 14: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/14.jpg)
CSE486, Penn StateRobert Collins
![Page 15: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/15.jpg)
CSE486, Penn StateRobert Collins
3D Velocity
Under small angle approx,displacement = RP+T - P = (I+S)P+T-P = SP+T
Note: SP = [θx, θy, θz]T X P
In limit, displacement becomes a velocity
V = T + ω X P
3D velocity
linear velocityangular velocity
where
Caution: Abuse of notation ==> we are re-using T as a velocity
![Page 16: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/16.jpg)
CSE486, Penn StateRobert Collins
3D Relative Velocity
ff
PPVV
ppvv
ZZ zz
The relative velocity of P The relative velocity of P wrt wrt camera:camera:
Camps, PSU
Note: we change signs of the velocity to beconsistent with the book. This should notcause concern. It just depends on whetheryou want to think of the motion as being dueto the camera or the scene.
![Page 17: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/17.jpg)
CSE486, Penn StateRobert Collins
3D Relative Velocity
ff
PPVV
ppvv
ZZ zz
The relative velocity of P The relative velocity of P wrt wrt camera:camera:
Camps, PSU
![Page 18: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/18.jpg)
CSE486, Penn StateRobert Collins
3D Relative Velocity
ff
PPVV
ppvv
ZZ zz
Where are we in this lecture?
• We just derived an equationrelating R,T and to the 3D velocity vector at each scenepoint.
• We can think of that velocity as a little vector inthe scene.
• Now ask, what does the projection of that vectorlook like in the image? It is a 2D vector. It is oneof the vectors that make up the Motion Field!
![Page 19: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/19.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the 2D velocity of p
ff
PPVV
ppvv
ZZ zz
Taking derivative Taking derivative wrt wrt time:time:
Camps, PSU
Perspectiveprojection
![Page 20: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/20.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the velocity of p
ff
PPVV
ppvv
ZZ zz
Camps, PSU
![Page 21: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/21.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the velocity of p
ff
PPVV
ppvv
ZZ zz
Camps, PSU
![Page 22: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/22.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the velocity of p
ff
PPVV
ppvv
ZZ zz
Camps, PSUTHIS IS THE EQUATION WE WANT!!!!
![Page 23: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/23.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the velocity of p
ff
PPVV
ppvv
ZZ zz
TranslationalTranslationalcomponentcomponent
Camps, PSU
![Page 24: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/24.jpg)
CSE486, Penn StateRobert Collins
Motion Field: the velocity of p
ff
PPVV
ppvv
ZZ zz
RotationalRotationalcomponentcomponent
NOTE: The rotational component is independent of depth Z !NOTE: The rotational component is independent of depth Z !
Camps, PSU
![Page 25: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/25.jpg)
CSE486, Penn StateRobert Collins
Special Case I: Pure Translation
Assume Assume TTzz ≠≠ 0 0
Define:Define:
Camps, PSU
![Page 26: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/26.jpg)
CSE486, Penn StateRobert Collins
Special Case I: Pure Translation
What if What if TTzz == 0 ? 0 ?
All motion field vectors are parallel to each other andAll motion field vectors are parallel to each other andinversely proportional to depth !inversely proportional to depth !
Camps, PSUTIE IN WITH SIMPLE STEREO!
![Page 27: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/27.jpg)
CSE486, Penn StateRobert Collins
Special Case I: Pure Translation
The motion field in this case is RADIAL:The motion field in this case is RADIAL:••It consists of vectors passing through pIt consists of vectors passing through poo = (x = (xoo,,yyoo))••If:If:
•• TTzz > 0, > 0, (camera moving towards object)(camera moving towards object)
•• the vectors point away from p the vectors point away from poo
••ppoo is the is the POINT OF EXPANSIONPOINT OF EXPANSION•• TTzz < 0, < 0, (camera moving away from object)(camera moving away from object)
••the vectors point towards pthe vectors point towards poo
••ppoo is the is the POINT OF CONTRACTIONPOINT OF CONTRACTION
TTzz > 0> 0 TTzz < 0< 0
Camps, PSU
![Page 28: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/28.jpg)
CSE486, Penn StateRobert Collins Pure Translation:
Properties of the MF
• If Tz≠ 0 the MF is RADIAL with all vectorspointing towards (or away from) a singlepoint po. If Tz = 0 the MF is PARALLEL.
• The length of the MF vectors is inverselyproportional to depth Z. If Tz ≠ 0 it is alsodirectly proportional to the distancebetween p and po.
Camps, PSU
![Page 29: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/29.jpg)
CSE486, Penn StateRobert Collins Pure Translation:
Properties of the MF
• po is the vanishing point of the direction oftranslation.
• po is the intersection of the ray parallel tothe translation vector and the image plane.
Camps, PSU
![Page 30: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/30.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
ff
PPVV
ppvv
ZZ zz
nnPlanar surfaces are commonPlanar surfaces are commonin man-made environmentsin man-made environments
Question: How does the MF of a moving plane look like?Question: How does the MF of a moving plane look like?
Camps, PSU
Self-study
![Page 31: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/31.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
Points on the plane must satisfy thePoints on the plane must satisfy theequation describing the plane.equation describing the plane.
LetLet••nn be the unit vector normal to the plane. be the unit vector normal to the plane.••dd be the distance from the plane to the be the distance from the plane to theorigin.origin.••NOTE:NOTE: If the plane is moving If the plane is moving wrtwrtcamera, camera, nn and and dd are functions of timeare functions of time..
Then:Then:
dd PP
ZZ
nn
OOYY
XX
wherewhere
Camps, PSU
Self-study
![Page 32: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/32.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
Camps, PSU
dd PP
ZZ
nn
OOYY
XX
Using the pinholeUsing the pinholeprojection equation:projection equation:pp
LetLet be the image of Pbe the image of P
Using the plane equation:Using the plane equation:
Solving for Z:Solving for Z:
Self-study
![Page 33: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/33.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
And Plug in:And Plug in:
dd PP
ZZ
nn
OOYY
XX
pp
Now consider the MF equations:Now consider the MF equations:
Camps, PSU
Self-study
![Page 34: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/34.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
dd PP
ZZ
nn
OOYY
XX
pp
The MF equations become:The MF equations become:
wherewhere
Camps, PSU
Self-study
![Page 35: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/35.jpg)
CSE486, Penn StateRobert Collins
Special Case II: Moving Plane
dd PP
ZZ
nn
OOYY
XX
pp
MF equations: MF equations:
Q: What is the significance of this?Q: What is the significance of this?
A: The MF vectors are given by low order (second)A: The MF vectors are given by low order (second)polynomials.polynomials.
••Their Their coeffscoeffs. a. a11 to a to a8 8 (only 8 !) are functions of n, d, T and (only 8 !) are functions of n, d, T and ωω..••That is, can estimate 8That is, can estimate 8 param param global flow rather than 2global flow rather than 2 params params per pixel! Huge savings in time and robustness.per pixel! Huge savings in time and robustness.
Camps, PSU
Self-study
![Page 36: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/36.jpg)
CSE486, Penn StateRobert Collins
Application: Time to Collision
ff
LL
vv
LL
DDoo
l(t)l(t)
An object of heightAn object of height L L moves with moves withconstant velocity constant velocity v:v:••At time At time t=0t=0 the object is at: the object is at:
•• D(0) = DD(0) = Doo
••At time At time tt it is at it is at••D(t) = DD(t) = Doo –– vt vt
••It will crash into the camera atIt will crash into the camera attime:time:
•• D(D(ττ) = D) = Doo –– v vττ = 0 = 0•• ττ = D = Doo/v/v
t=0t=0tt
D(t)D(t)
Camps, PSU
Self-study
![Page 37: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/37.jpg)
CSE486, Penn StateRobert Collins
Time to Collision
ff
LL
vv
LL
DDoo
l(t)l(t)
t=0t=0tt
D(t)D(t)
The image of the object has size l(t):The image of the object has size l(t):
Taking derivative Taking derivative wrt wrt time:time:
Camps, PSU
Self-study
![Page 38: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/38.jpg)
CSE486, Penn StateRobert Collins
Time to Collision
ff
LL
vv
LL
DDoo
l(t)l(t)
t=0t=0tt
D(t)D(t)
Camps, PSU
Self-study
![Page 39: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/39.jpg)
CSE486, Penn StateRobert Collins
Time to Collision
Camps, PSU
ff
LL
vv
LL
DDoo
l(t)l(t)
t=0t=0tt
D(t)D(t)
And their ratio is:And their ratio is:
Self-study
![Page 40: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/40.jpg)
CSE486, Penn StateRobert Collins
Time to Collision
Can beCan bedirectlydirectlymeasuredmeasuredfrom imagefrom image
ff
LL
vv
LL
DDoo
l(t)l(t)
t=0t=0tt
D(t)D(t)
And And time to collisiontime to collision::
Can be found, without knowing Can be found, without knowing LL or or DDoo or or vv !! !!
Camps, PSU
Self-study
![Page 41: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/41.jpg)
CSE486, Penn StateRobert Collins
Application: Decomposition of Vehicle Flow Fieldinto Rotational and Translational Components
Motivation•Estimate steering angles•Computation of scene structure simplifies when rotational motion is removed
![Page 42: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/42.jpg)
CSE486, Penn StateRobert Collins
Review: Flow Due to Self-Motion
Rotation Translation
Flow:
Rotation component Translation component
Note: I’ve changed sign again!
![Page 43: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/43.jpg)
CSE486, Penn StateRobert Collins
Approach
Use prior knowledge of car’s likely motion to simplify the problem.
Assume we know approx direction of translation (Tx,Ty,Tz)
[ e.g. if driving forward, we choose (0,0,1) ]
Assume roll angle of motion (wz) is 0
Problem reduces to solving for pitch and yaw angles (wx and wy)
0known
![Page 44: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/44.jpg)
CSE486, Penn StateRobert Collins
Approach (continued)
For each observed flow vector, form a linear constraint on wx and wy
by taking the dot product of the flow equation with a new vector that
is constructed to “annihilate” the translation component of flow.
This annihilation vector is
Verify:
0
Note: these are allknown values!
![Page 45: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/45.jpg)
CSE486, Penn StateRobert Collins
Approach (continued)
For each observed flow vector, we annihilate the translation component
to form one linear constraint of the form (ai wx + bi wy + ci ) = 0.
Given the whole set of n observed flow vectors, we seek wx and wy
that simultaneously satisfy the set of n linear equations
Geometric intuition:
image space solution space
wx
wy
![Page 46: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/46.jpg)
CSE486, Penn StateRobert Collins
Use Robust Estimator
robust solution
least-squares solution
red: outliers blue: inliers
Solving using least-squares will be sensitive to outliers (grossly incorrect data).Instead, we use a robust estimator (Random Sample Consensus – RANSAC).
![Page 47: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/47.jpg)
CSE486, Penn StateRobert Collins
![Page 48: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/48.jpg)
CSE486, Penn StateRobert Collins
![Page 49: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/49.jpg)
CSE486, Penn StateRobert Collins
![Page 50: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/50.jpg)
CSE486, Penn StateRobert Collins
![Page 51: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/51.jpg)
CSE486, Penn StateRobert Collins
![Page 52: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/52.jpg)
CSE486, Penn StateRobert Collins
![Page 53: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/53.jpg)
CSE486, Penn StateRobert Collins
![Page 54: Lecture 22: Camera Motionrtc12/CSE486/lecture22.pdf · Lecture 22: Camera Motion Readings: ... Special Case II: ... Taking derivative wrt time: Camps, PSU Self-study. CSE486, Penn](https://reader031.vdocument.in/reader031/viewer/2022020215/5b8a75b47f8b9a655f8e68b8/html5/thumbnails/54.jpg)
CSE486, Penn StateRobert Collins