CIS580Machine Perception

Spring 2017 Midterm Review Questions

Instructions. All coordinate systems are right handed.

1 Projective Geometry 1

2D Projective Plane, P2, is the set of an equivalence class of vectors in R3 \ 0. Formally,

P2 = {[a] | a ∈ R3 \ 0},

where the equivalence class of an element a is denoted as [a], [a] = {x ∈ R3 \ 0 | a ∼ x}, and theequivalence relation, ∼, is defined as follows:xy

ω

∼x′y′ω′

iff ∃λ 6= 0

xyω

= λ

x′y′ω′

for

xyω

,x′y′ω′

∈ R3 \ 0.

The homogeneous representation of a 2D point or line is an element of P2.

1. Properties of a homogeneous representation of a 2D point and line.

(a) Let a 2D point in an image be x̄. What is the homogeneous representation, x, of this point?

(b) A standard line equation in 2D plane is ax + by + c = 0. What is the homogeneousrepresentation, l, of this line?

(c) What is the intersection of two lines l and l′?

(d) What is the line passing two points x and x′?

(e) Points at infinity

x1x20

pass the line at infinity. What is the form of the line at infinity,

l∞? Why?

2. Finding a projective transformation H.

Some projective transformation H preserves the points

[10

],

[01

], and the origin of the coordinate

system. However, it maps the point

[11

]to the points

[21

], meaning d

211

= H

111

. Compute

H with one free variable d.

1

2 Projective Geometry 2

Figure 1: Facade rectification.

I took an image of a rectangular object, and its image measurement is shown in Figure 1 left. Theoriginal object shape is shown in Figure 1 right. Let the homography from the rectangle drawing(Figure 1 right) to the image measurement (Figure 1 left) be H.

λix′i = Hxi =

h11 h12 h13h21 h22 h23h31 h32 h33

xi,

where λi is a scalar, x′i and xi are homogeneous coordinates for corresponding points on Figure 1 leftand Figure 1 right respectively.

The homography, H, maps

• the origin to

[−1−1

]

•[11

]to

[11

]

• infinity in the x-direciton to the vanishing point

[50

]

• infinity in the y-direciton to the vanishing point

[05

]1. Compute H with h33 normalized to 1.

2

2. Compute the vanishing line, l, in Figure 1 left.

3. Project the vanishing line l in Figure 1 left to Figure 1 right.

Hint: l′ = T−T l, where T is the homography transformation from Figure 1 left to right, l andl′ are line equation on Figure 1 left and right respectively.

3

3 Single View Metrology

Figure 2: The cross-ratio.

In this problem, we will use cross-ratio to determine possible correspondence between a floor plan(Figure 2 left) and a room photograph (Figure 2 right). This is useful for online apartment findingsince we often have a floor plan and photos listed, but no correspondence for matching room photosto those shown on the floor plan.

In this example shown in Figure 2 left, the floor-plan shows the width of the two windows and thewall between them are of the same width W . Let a,b, c,d and e be the 2D points measured in pixelcoordinates in Figure 2 right.

1. Express the image cross-ratio in terms of a,b, c,d and e.

2. Express and compute the corresponded scene cross-ratio.

3. How to use this information to verify if this pair of the floor plan and room photo matches?

4

4 Rotation

(a) Two meanings of rotation matrix (b) Rotation Combination

Figure 3: Camera Rotation

In this problem, we will familiarize ourselves with the concept of rotation matrix R . Mathemati-cally, the rotation matrix R can be used as following (3D example):xbyb

zb

= R

xayaza

This equation has two geometrical meanings for the same rotation action, as illustrated in Figure3(a). Two rotation matrices are given:

R1 =

45 0 − 3

50 1 035 0 4

5

,R2 =

2425 − 7

25 0725

2425 0

0 0 1

R1 measure the rotation of θ1 about y-axis, and R2 measure the rotation of θ2 about z-axis.

1. In the same coordinate system, rotate point from Xa = (−6,−2, 4)T, to point Xb for θ1 abouty-axis. Compute point Xb position.

2. Rotate the coordinate system from U to V for θ2 about z-axis. Given the point position in Ucoordinate XU = (−2, 2, 4)T, compute the point position in V coordinate XV.

3. First, rotate the coordinate system from U to V for θ2 about z-axis. Then in the coordinatesystem V, rotate point from XV

a , to point XVb = (8, 3, 10)T for θ1 about y-axis. Compute the

point position in U coordinate XUa .

5

5 Epipolar Geometry

When two cameras view a 3D scene from two distinct positions / orientations, there are a numberof geometric relations between the 3D points and their projections onto the 2D images that leadto constraints between the image points. In this problem, we will use known camera positions andorientations to build essential matrix, fundamental matrix, epipolar line and epipole.

The intrinsic parameters of the two cameras K1, K2 and the external parameters of the two camerasR1, t1 and R2, t2 are all given, where P1 = K1 [R1|t1] and P2 = K2 [R2|t2]. X is a 3D point.

1. Express the essential matrix 2E1 with respect to R1, R2, t1 and t2.

Note that for a given point X in 3D world, XT22E1X1 = 0, where X1 and X2 are the coordinates

for X in the first and second camera frame, respectively.

2. Express the fundamental matrix 2F1 with respect to R1, R2, t1, t2, K1 and K2.

Note that for a given point X in 3D world,

[x2

1

]T2F1

[x1

1

]= 0, where x1 and x2 are the 2D

projection of X in the first and second camera image plane, respectively.

3. Epipolar line is the intersection line of image plane and epipolar plane. Given the 3D point Xand its 2D projection x1 on image plane 1, it corresponds to an epipolar line on image plane 2.

Express the equation for epipolar line in image 2 of point x1 in image 1 with respect to x1, R1,R2, t1, t2, K1 and K2,

4. Epipole is the 2D projection of one camera center in another camera’s image plane.

Express the coordinate of epipole in image 2 with respect to R1, R2, t1, t2, K1 and K2,

6

6 Parameter Estimation[x1

]= K

[R|t

] [X1

]represents perspective projection, where K is camera calibration matrix, R and

t are the camera extrinsic parameters, X is 3D point in world coordinate and x is the projected pointin image coordinate.

Given part of the parameters in the equation, we can solve for the other parameters using linear andnon-linear optimization. In this problem, we will discuss about triangulation, PnP, Bundle Adjustmentand revisit calibration.

1. Triangulation refers to the process of determining a point in 3D space given its projections ontotwo, or more, images. In this question, given the perspective projection parameters R, t, K andprojected point x, we need to solve point in world coordinate X.

(a) In linear triangulation, we formulate the triangulation problem as a linear least squaresproblem. Explicitly, we combine and reduce the projection equation into the form of

A

[X1

]= 0, where A is build from R, t, K and x, and solve X by SVD.

What’s the minimum number of cameras needed to triangulate one point? Express matrixA with respect to R, t, K and x. Use Ri, ti, Ki and xi for the parameters and 2Dprojection of the ith camera. Note that intermediate steps are needed for this question.

(b) In nonlinear triangulation, we minimize the reprojection error of our estimation X̃ withrespect to its 2D reprojection.

Express reprojection error with respect to X̃ and 2D projection x.

Compute the Jacobian matrix for the cost function, when X̃ = X0.

2. Perspective-n-Point (PnP) refers to the process of determining the camera extrinsic parame-ters given camera calibration matrix, several points in world coordinate and corresponding 2Dprojection.

In linear PnP, we formulate the PnP problem as a linear least squares problem. Explicitly, wecombine and reduce the projection equation into the form of AT = 0, where T is a 12×1 vectorrepresenting elements in R and t, A is build from K, X and x, and solve T by SVD.

What’s the minimum number of points needed to register one camera? Express matrix A withrespect to K, X and x. Use Xi and xi for 3D coordinates and its 2D projection for the ith

point. Note that intermediate steps are needed for this question.

3. Camera calibration can be formulated as a problem of reprojection error minimization. In thisquestion, given the perspective projection parameters R and t, points in world coordinate Xand projected point x, we will solve the camera calibration matrix K.

Using the similar linear least squares method, what’s the minimum number of X and x pairsneeded to determine camera calibration matrix? Express matrix A with respect to R, t, Xand x. Use Xi and xi for 3D coordinates and its 2D projection for the ith point. Note thatintermediate steps are needed for this question.

7

7 Triangulation

Triangulation refers to the process of determining a point in 3D space given its projections onto two,or more, images. In this problem, we will solve coordinate of 3D point X with two known camerasand the 2D projection of X on the two cameras.

The projection matrices of the two cameras P1, P2 are given, where P1 = K1 [R1|t1] and P2 =K2 [R2|t2]. X is a 3D point, and x1, x2 are its 2D projection in image 1 and image 2 respectively.

1. In linear triangulation, we formulate the triangulation problem as a linear least squares problem.

Explicitly, we combine and reduce the projection equation

[x1

1

]= P1

[X1

]and

[x2

1

]= P2

[X1

]into the form of A

[X1

]= 0, where A is build from P1, P2, x1, x2, and solve X by SVD.

Express matrix A with respect to P1, P2, x1 and x2. Note that intermediate steps are neededfor this question.

2. In nonlinear triangulation, we minimize the reprojection error of our estimation X̃ with respectto its 2D reprojection x1 and x2.

Express the reprojection error with P1, P2, x1, x2 and X̃.

3. When X = X0, compute the Jacobian matrix for the cost function.

8