image formation, camerascseweb.ucsd.edu/classes/wi07/cse252a/lec4.pdf · • homework 1 – will be...

CS252A, Winter 2007 Computer Vision I

Image Formation, Cameras

Computer Vision ICSE 252ALecture 4


Announcements• Read Chapters 1 & 2 of Forsyth & Ponce• Homework 1 – will be on web page later

today.

• Office Hours– Kriegman: Tues 1:30-2:30– Neil: Wed 11:00-12:00, EBU-3b B250A


Last lecture in a nutshell


Pinhole Camera: Perspective projection

• Abstract camera model - box with a small hole in it

Forsyth&Ponce


Geometric properties of projection

• Points go to points• Lines go to lines• Planes go to whole image

or half-plane• Polygons go to polygons• Angles & distances not preserved

• Degenerate cases:– line through focal point yields point– plane through focal point yields line


Parallel lines meet in the image• vanishing point

Image plane


Projective Geometry• Axioms of Projective Plane

1. Every two distinct points define a line2. Every two distinct lines define a point (intersect

at a point)3. There exists three points, A,B,C such that C

does not lie on the line defined by A and B.• Different than Euclidean (affine) geometry• Projective plane is “bigger” than affine

plane – includes “line at infinity”

ProjectivePlane

AffinePlane= + Line at

Infinity


Homogenous Coordinates & the camera matrix

Homogenous Coordinates– Euclidean -> Homogenous: (x, y) -> k (x,y,1)– Homogenous -> Euclidean: (x, y, z) -> (x/z, y/z)

– Projective transformation– 3 x 3 linear transformation of homogenous coordinates– Points map to points, lines map to lines

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

3

2

1

333231

232221

211211

3

2

1

xxx

aaaaaaaaa

uuu


The equation of projection

Cartesian coordinates:

(x,y, z) → ( f xz

, f yz)

UVW

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

=1 0 0 00 1 0 00 0 1

f 0

⎛

⎝

⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟ ⎟

XYZT

⎛

⎝

⎜ ⎜ ⎜ ⎜ ⎜

⎞

⎠

⎟⎟⎟⎟⎟

Homogenous Coordinates and Camera matrix


Affine Camera Model

• Take Perspective projection equation, and perform Taylor Series Expansion about (some point (x0,y0,z0).

• Drop terms of higher order than linear.• Resulting expression is affine camera model

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

0

0

0

zyx

Appropriate in NeighborhooAbout (x0,y0,z0


• Perspective

• Assume that f=1, and perform a Taylor series expansion about (x0, y0, z0)

• Dropping higher order terms and regrouping.

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

zf

vu

( ) ( )

( ) ( ) L+−⎥⎦

⎤⎢⎣

⎡+−⎥

⎦

⎤⎢⎣

⎡+

−⎥⎦

⎤⎢⎣

⎡+−⎥

⎦

⎤⎢⎣

⎡−⎥

⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡

20

0

030

00

00

00

0200

0

0

221

101

01111

zzyx

zyy

z

xxz

zzyx

zyx

zvu

bAp +=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡

−−

+⎥⎦

⎤⎢⎣

⎡≈⎥

⎦

⎤⎢⎣

⎡

zyx

zyzzxz

yx

zvu

2000

2000

0

0

0 //10/0/11


bAp +=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

⎥⎦

⎤⎢⎣

⎡

−−

+⎥⎦

⎤⎢⎣

⎡≈⎥

⎦

⎤⎢⎣

⎡

zyx

zyzzxz

yx

zvu

2000

2000

0

0

0 //10/0/11

Rewrite Affine camera modelin terms of Homogenous Coordinates

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−−

≈⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

11000///10//0/1

002000

002000

zyx

zyzyzzxzxz

wvu


⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡yx

zvu

0

1

Orthographic projectionStarting with Affine camera mode

Take Taylor series about (0, 0, z0) – a point on optical axis


The projection matrix for orthographic projection

⎟⎟⎟⎟⎟

⎠

⎞

⎜⎜⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

TZYX

zz

WVU

100000/10000/1

0

0

Parallel lines project to parallel linesRatios of distances are preserved under orthographic


Other camera models• Generalized camera – maps points lying on rays

and maps them to points on the image plane.

Omnicam (hemispherical) Light Probe (spherical)


Some Alternative “Cameras”


What if camera coordinate system differs from object coordinate system

{c}

P{W}


Euclidean Coordinate Systems

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=⇔++=⇔

⎪⎩

⎪⎨

⎧

===

zyx

zyxOPOPzOPyOPx

Pkjikji

...


Coordinate Changes: Pure Translations

OBP = OBOA + OAP , BP = AP + BOA


Rotation Matrix

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡=

BABABA

BABABA

BABABABA R

kkkjkijkjjjiikijii

.........

[ ]AB

AB

AB kji=

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=TB

A

TB

A

TB

A

kji


Coordinate Changes: Pure Rotations

[ ] [ ]

PRP

zyx

zyx

OP

ABA

B

B

B

B

BBBA

A

A

AAA

=⇒

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

=⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

= kjikji


Coordinate Changes: Rigid Transformations

ABAB

AB OPRP +=


A convenient notation

ABAB

AB OPRP +=

– Points: AP1

– Leading superscript coordinate system w.r.t.– Subscript – an identifier

– Rotation Matrices– Lower left (Going from this system)– Upper left (Going to this system)

– To add vectors, coordinate systems must agree– To rotate a vector, points coodrinate system must agree with lower left of rotation matrix

RBA


Another Digression

Rotation Matrices SO(3)


Some points about SO(n)• SO(n) = { R ∈ ℜnxn : RTR = I, det(R) = 1}

– SO(2): rotation matrices in plane ℜ2

– SO(3): rotation matrices in 3-space ℜ3

• R-1 = RT

• Bounded Ri,j ∈ [-1, +1]• FYI, if det(R)= -1, there would be a reflection

E.g.

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

−100010001


Some more points about SO(n)• SO(n) = { R ∈ ℜnxn : RTR = I, det(R) = 1}• Forms a Group under matrix product operation:

– Identity– Inverse– Associative– Closure

• Rotations are not commutative in general• Manifold of dimension n(n-1)/2

– Dim(SO(2)) = 1– Dim(SO(3)) = 3


SO(3)• Parameterizations of SO(3)• 3-D manifold, so between 3 parameters and 2n

parameters (Whitney’s Embedding Thm.)– Roll-Pitch-Yaw– Euler Angles– Axis Angle (Rodrigues formula)– Cayley’s formula– Matrix Exponential– Quaternions (four parameters + one constraint)


Rigid Transformations as Mappings: Rotation about the k Axis

= rot(k,θ)


Rotation: Homogenous Coordinates

• About z axis

x'y'z'1

=

xyz1

cos θsin θ

00

-sin θcos θ

00

0010

0001

rot(z,θ) x

y

z

p

p'

θ


Rotation

• About x axis:

• About y axis:

x'y'z'1

=

xyz1

0cos θsin θ

0

0-sin θcos θ

0

1000

0001

x'y'z'1

=

xyz1

cos θ0

-sin θ0

sin θ0

cos θ0

0100

0001


Roll-Pitch-Yaw),ˆ(),ˆ(),ˆ( ϕβα krotjrotirotR =

),ˆ(),'ˆ(),''ˆ( ϕβα krotjrotkrotR =

Euler Angles


A Gimbal• Hardware implementation of Euler angles

(used for mounting gyroscopes and globes)


Rotation• About (kx, ky, kz), a unit

vector on an arbitrary axis(Rodrigues Formula)

x'y'z'1

=

xyz1

kxkx(1-c)+ckykx(1-c)+kzskzkx(1-c)-kys

0

0001

kzkx(1-c)-kzskzkx(1-c)+ckzkx(1-c)-kxs

0

kxkz(1-c)+kyskykz(1-c)-kxskzkz(1-c)+c

0

where c = cos θ & s = sin θ

Rotate(k, θ)

x

y

z

θ

k


Quaternions

q = (a,α)q is a quaternion (generalization of

imaginary numbers)a ∈ RR is its real part (cos angle of rotation)α ∈ RR3 is its imaginary part (axis of rotation)

• Sum of quaternions: ( a,α ) + ( b,β ) ≡ (( a+b ),(α+β )

• Multiplication by a scalar: λ ( a,α ) ≡ ( λa,λα )

• Quaternion product:

( a,α ) ( b,β ) ≡ (( a b – α ⋅ β ) , ( a β + b α + α × β ))

• Conjugate: q = (a,α) q’ ≡ (a, -α)

Operations on quaternions:

• Norm: | q |2 ≡ q q’ = q’ q = a2 + | α |2


Unit Quaternions and Rotations• Let R denote the rotation of angle θ about the unit vector u.

• Define unit quaternion q = (cos θ/2, sin θ/2 u).• Note |q| = 1 (i.e., q lies on unit sphere for any u and θ.

• Then for any vector α, R α = imaginary(q α* q’)

where α*= (0, α)•q and –q define the same rotation matrix.

If q = (a, ( b, c, d )T) is a unit quaternion, the corresponding rotation matrix is:


End of Digression on SO(3)


Block Matrix Multiplication

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡=

2221

1211

2221

1211

BBBB

BAAAA

A

What is AB ?

⎥⎦

⎤⎢⎣

⎡++++

=2222122121221121

2212121121121111

BABABABABABABABA

AB

Homogeneous Representation of Rigid Transformations

⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡ +=⎥

⎦

⎤⎢⎣

⎡⎥⎦

⎤⎢⎣

⎡=⎥

⎦

⎤⎢⎣

⎡

11111P

TOPRPORP A

BA

ABAB

AA

TA

BBA

B

0


Camera parameters• Issue

– camera may not be at the origin, looking down the z-axis

• extrinsic parameters– one unit in camera coordinates may not be the

same as one unit in world coordinates• intrinsic parameters - focal length, principal point,

aspect ratio, angle between axes, etc.

UVW

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟ =

Transformationrepresenting intrinsic parameters

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟

1 0 0 00 1 0 00 0 1 0

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟

Transformationrepresentingextrinsic parameters

⎛

⎝

⎜ ⎜

⎞

⎠ ⎟ ⎟

XYZT

⎛

⎝

⎜ ⎜ ⎜

⎞

⎠

⎟ ⎟ ⎟

3 x 3 4 x 4


, estimate intrinsic and extrinsic camera parameters

• See Text book for how to do it.

Camera Calibration


Beyond the pinhole CameraGetting more light – Bigger Aperture


Pinhole Camera Images with Variable Aperture

1mm

.35 mm

.07 mm

.6 mm

2 mm

.15 mm


Limits for pinhole cameras

image formation, camerascseweb.ucsd.edu/classes/wi07/cse252a/lec4.pdf · • homework 1 – will be...

Documents