[ieee 2014 ieee winter conference on applications of computer vision (wacv) - steamboat springs, co,...

6
3D Metric Rectification using Angle Regularity Aamer Zaheer and Sohaib Khan Syed Babar Ali School of Science and Engineering Lahore University of Management Sciences, Lahore, Pakistan. cvlab.lums.edu.pk/MetricRectification/ Abstract This paper proposes Automatic Metric Rectification of projectively distorted 3D structures for man-made scenes using Angle Regularity. Man-made scenes, such as build- ings, are characterized by a profusion of mutually orthog- onal planes and lines. Assuming the availability of pla- nar segmentation, we search for the rectifying 3D homog- raphy which maximizes the number of orthogonal plane- pairs in the structure. We formulate the orthogonality con- straints in terms of the Absolute Dual Quadric (ADQ). Us- ing RANSAC, we first estimate the ADQ which maximizes the number of planes meeting at right angles. A rectifying homography recovered from the ADQ is then used as an initial guess for nonlinear refinement. Quantitative experi- ments show that the method is highly robust to the amount of projective distortion, the number of outliers (i.e. non- orthogonal planes) and noise in structure recovery. Unlike previous literature, this method does not rely on any knowl- edge of the cameras or images, and no global model, such as Manhattan World, is imposed. 1. Introduction Geometric regularities are ubiquitous in man-made envi- ronments such as urban architecture, and play an important role in visual perception. Some examples include the pres- ence of planes, lines, corners, curves, smoothness and con- tinuity. In particular, a common characteristic of man-made scenes is an abundance of mutually orthogonal line-pairs and plane-pairs. The 3D structure of a scene can be recov- ered from two or more of its images. When the cameras are uncalibrated, the structure is recovered up to a projec- tive transformation distorting all the angles [1, p. 266]. We consider the problem of recovering a rectifying 3D homog- raphy that corrects all the inter-plane angles. Previous automatic approaches to 3D metric rectifica- tion either use camera calibration constraints on or rely on regularities in the 3D structure. Prominent camera cali- bration constraints assume the knowledge of some intrin- sic parameters, square-pixel camera, or same camera (with potentially varying focal length). The multi-planar nature of man-made structure can be exploited using known im- age to image homographies. Angle regularity of the man- made world is typically modeled globally, such as Manhat- tan World, and exploited through image 2D quantities like vanishing points, vanishing lines and Dual Image of the Ab- solute Conic. In this paper, we propose a method that does not assume any knowledge of the cameras or images and does not impose a global constraint on the 3D structure. Given a projective structure with multi-planar segmenta- tion, we formulate the orthogonality constraints of the scene locally between the two planes in terms of the Absolute Dual Quadric (ADQ) [2] leading to an efficient linear al- gorithm. Assuming that a large number of planes are or- thogonal to each other in the undistorted structure, we use RANSAC to automatically identify the mutually orthognal plane-pairs. The ADQ that maximizes the number of or- thogonal plane-pairs is used to compute a rectifying homog- raphy. The homography is nonlinearly refined using all the RANSAC inliers. The structure can be rectified by applying this homography to all the points, planes and cameras. The proposed method is unique because: a) no knowl- edge of cameras or images is required; b) an automatic solution without assuming a global model for 3D struc- ture is presented; and (c) a linear solution, outlier rejec- tion, and nonlinear refinement are proposed for estimating the rectifying homography. Quantitative evaluation is per- formed on synthetic data, and real data distorted through completely random projective distortions. It includes anal- ysis of robustness to amount of projective distortion, noise in structure recovery, and percentage of outliers, i.e. non- orthogonal plane-pairs. The code and data are shared through our project page for reproducibility [3]. 2. Related Work In human perception, the effect of angle regularities of plane under perspective has been studied [4, 5] but most of such work focused on parallel lines. A recent work [6] shows that both orthogonality as well as parallelism play an 31

Upload: sohaib

Post on 07-Mar-2017

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

3D Metric Rectification using Angle Regularity

Aamer Zaheer and Sohaib KhanSyed Babar Ali School of Science and Engineering

Lahore University of Management Sciences, Lahore, Pakistan.cvlab.lums.edu.pk/MetricRectification/

Abstract

This paper proposes Automatic Metric Rectification ofprojectively distorted 3D structures for man-made scenesusing Angle Regularity. Man-made scenes, such as build-ings, are characterized by a profusion of mutually orthog-onal planes and lines. Assuming the availability of pla-nar segmentation, we search for the rectifying 3D homog-raphy which maximizes the number of orthogonal plane-pairs in the structure. We formulate the orthogonality con-straints in terms of the Absolute Dual Quadric (ADQ). Us-ing RANSAC, we first estimate the ADQ which maximizesthe number of planes meeting at right angles. A rectifyinghomography recovered from the ADQ is then used as aninitial guess for nonlinear refinement. Quantitative experi-ments show that the method is highly robust to the amountof projective distortion, the number of outliers (i.e. non-orthogonal planes) and noise in structure recovery. Unlikeprevious literature, this method does not rely on any knowl-edge of the cameras or images, and no global model, suchas Manhattan World, is imposed.

1. IntroductionGeometric regularities are ubiquitous in man-made envi-

ronments such as urban architecture, and play an importantrole in visual perception. Some examples include the pres-ence of planes, lines, corners, curves, smoothness and con-tinuity. In particular, a common characteristic of man-madescenes is an abundance of mutually orthogonal line-pairsand plane-pairs. The 3D structure of a scene can be recov-ered from two or more of its images. When the camerasare uncalibrated, the structure is recovered up to a projec-tive transformation distorting all the angles [1, p. 266]. Weconsider the problem of recovering a rectifying 3D homog-raphy that corrects all the inter-plane angles.

Previous automatic approaches to 3D metric rectifica-tion either use camera calibration constraints on or rely onregularities in the 3D structure. Prominent camera cali-bration constraints assume the knowledge of some intrin-

sic parameters, square-pixel camera, or same camera (withpotentially varying focal length). The multi-planar natureof man-made structure can be exploited using known im-age to image homographies. Angle regularity of the man-made world is typically modeled globally, such as Manhat-tan World, and exploited through image 2D quantities likevanishing points, vanishing lines and Dual Image of the Ab-solute Conic. In this paper, we propose a method that doesnot assume any knowledge of the cameras or images anddoes not impose a global constraint on the 3D structure.

Given a projective structure with multi-planar segmenta-tion, we formulate the orthogonality constraints of the scenelocally between the two planes in terms of the AbsoluteDual Quadric (ADQ) [2] leading to an efficient linear al-gorithm. Assuming that a large number of planes are or-thogonal to each other in the undistorted structure, we useRANSAC to automatically identify the mutually orthognalplane-pairs. The ADQ that maximizes the number of or-thogonal plane-pairs is used to compute a rectifying homog-raphy. The homography is nonlinearly refined using all theRANSAC inliers. The structure can be rectified by applyingthis homography to all the points, planes and cameras.

The proposed method is unique because: a) no knowl-edge of cameras or images is required; b) an automaticsolution without assuming a global model for 3D struc-ture is presented; and (c) a linear solution, outlier rejec-tion, and nonlinear refinement are proposed for estimatingthe rectifying homography. Quantitative evaluation is per-formed on synthetic data, and real data distorted throughcompletely random projective distortions. It includes anal-ysis of robustness to amount of projective distortion, noisein structure recovery, and percentage of outliers, i.e. non-orthogonal plane-pairs. The code and data are sharedthrough our project page for reproducibility [3].

2. Related WorkIn human perception, the effect of angle regularities of

plane under perspective has been studied [4, 5] but mostof such work focused on parallel lines. A recent work [6]shows that both orthogonality as well as parallelism play an

31

Page 2: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

important part in perception of the plane pose. Similarly,the famous Ames Room illusion [7] demonstrates the effectof angle regularities in perception of 3D structure. A pro-jectively distorted room is perceived as a cuboid becausethe cuboid form maximizes the orthogonal angles betweenlines and planes in the world. In this case, Angle regularityseems to be a much stronger cue than even size constancy.

In computer vision, Liebowitz et al. [8] studied 2D met-ric rectification using angles between scene lines. They pro-posed a two step approach based on orthogonal vanishingpoints i.e. the image of a point at infinity. The vanishingpoint based approach was also extended to 3D but its au-tomation typically imposes a global restriction on the lineorientations. They also proposed a direct approach basedon orthogonal line-pairs which uses the Conic Dual to Cir-cular Points C∗∞. A recent work on Shape from Angle Reg-ularity automates rectification without a global orientationrestriction by modeling the regularity as a local constraint[9]. The proposed method can be thought of as a 3D exten-sion of the C∗∞ approach, automated by a 3D version of thelocal Angle Regularity constraint.

The use of Absolute Dual Quadric was first proposed forcamera calibration using known, same or square pixel cam-era constraints in [2]. The seminal work by Pollefeys etal. was the first to introduce automatic camera calibration,and metric rectification, for varying focal length cameras[10]. Later on, Sturm et al. combined multi-planarity ofman-made scenes with the camera constraints to propose ageneral method based on scene homographies [11]. Inter-planar angles were used in the form of parallelepipeds forcalibration and metric reconstruction but their approach isinteractive [12].

Our automatic 3D rectification method does not requireany knowledge of the cameras or images, models the worldin a significantly simpler way, and performs direct one-steprectification. It is applicable to the work in projective multi-planar structure from motion (e.g. [13], [14]) which showsvery stable results from a sparse set of input images. It canalso be used with point-based projective SfM [15] and au-tomatic multi-planar segmentation [16].

3. BackgroundThree dimensional structure computed in the absence of

camera calibration contains projective ambiguity, as statedby the projective reconstruction theorem [1]. This appliesto stereo reconstruction, point based Structure from Motion(SfM) and Multi-planar SfM (MPSfM) methods. Our goalis to compute the rectifying homography H ∈ P3 to removethe projective distortion and estimate the structure up to asimilarity transform. We assume that the structure repre-sents a man-made world and the multi-planar segmentationof points is available, or it can be computed using some ex-isting method such as [16].

3.1. Factorization of 3D Homography

Any homography H ∈ P3 can be factorized into sub-parts corresponding to the Similarity, Affine and Projectiveparts respectively, i.e.

H = HPHAHS , (1)

H =

[I3×3 0vT 1

] [A3×3 00T 1

] [sR3×3 t0T 1

], (2)

where A is an upper diagonal 3× 3 matrix with unit Frobe-nius norm, R is a 3 × 3 rotation matrix while v and t are3-dimensional column vectors [1].

Since the similarity part of H cannot be recovered with-out knowing some world measurements, we will ignore itand only recover the remaining 8 degrees of freedom. In or-der to search for the rectifying homography, we formulatethe plane orthogonality constraint in terms of the AbsoluteDual Quadric.

3.2. Absolute Dual Quadric

The Absolute Dual Quadric (ADQ) was introduced forcamera calibration in [2]. It is a degenerate dual (planesinstead of points) quadric in P3, which represents a collec-tion of planes tangent to the absolute conic lying on planeat infinity [1, p. 83]. It takes a canonical form in the Metricframe i.e.

Q =

[I3×3 03×101×3 0

]. (3)

The ADQ is invariant to similarity transformations sinceHSQHT

S∼= Q (here ∼= means equal up to a homoge-

neous scale). Therefore, when a general 3D homogra-phy is applied (or the structure has been recovered with aprojective ambiguity), the matrix Q takes the form Q′ =HPHAQHT

AHTP . It is a symmetric, singular matrix with

8 dof. Additionally the 3 × 3 subpart containing only firstthree rows and columns, must be positive (or negative, sinceit is homogeneous) semi-definite by construction.

The most relevant property of the ADQ is that any twoplanes p1 and p2 that are orthogonal in the world will always(under any projective ambiguity) follow the constraint

pT1 Q′p2 = 0. (4)

This constraint leads to a simple linear algorithm for esti-mating the Q′ using orthogonal plane-pairs.

3.3. Recovering Rectifying 3D Homography fromADQ

If we know the ADQ, Q′ in a non-metric frame distortedby some homography H , we can recover the distorting ho-mography (and its inverse that rectifies the structure to met-ric frame) by using Cholesky decomposition and simplematrix operations. For details see [1, p. 463]. The difficulty,

32

Page 3: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

however, arises when the ADQ matrix is estimated withnoisy data. With noise, the matrix could be non-singularand its first 3 × 3 block may not be positive (or negative)semi-definite. In such cases, the matrix cannot be decom-posed into the components of the rectifying homography,unless these constraints are imposed during estimation.

4. 3D Metric RectificationEvery orthogonal plane-pair gives us one constraint on

the ADQ leading to a linear estimation algorithm. This al-gorithm is then used with RANSAC in order to automati-cally compute the orthogonal plane-pairs and rectify a pro-jectively distorted structure. Finally, a nonlinear refinementis used to further optimize the recovered homography.

4.1. Linear ADQ Estimation

The input to our approach is N plane pairs (pi, qi) suchthat pi is orthogonal to qi and N >= 9. The output isa singular symmetric matrix Q′ that contains the rectify-ing homography H . This algorithm is highly similar to theeight point algorithm [17] for fundamental matrix. Like theeight point algorithm, we also use one more constraint thanrequired since a minimal parametrization would result in anonlinear solution.

From plane orthogonality constraint in Equation 4, weknow that for two orthogonal planes pi = [ai, bi, ci, di]

T

and qi = [a′i, b′i, c′i, d′i]T :

[ai, bi, ci, di]

q1 q2 q3 q4q2 q5 q6 q7q3 q6 q8 q9q4 q8 q9 q10

a′ib′ic′id′i

= 0, (5)

where qj are elements of Q′. We have N such constraints(for different values of i) which can be rearranaged such thatthe values of qj are expressed as a 10-dimensional vectorwhile all the data is in a single N × 10 matrix A i.e.

Aq = 0, (6)

where, for i = 1→ N

Ai,1 = aia′i , Ai,2 = aib

′i + bia

′i ,

Ai,3 = aic′i + cia

′i , Ai,4 = aid

′i + dia

′i ,

Ai,5 = bib′i , Ai,6 = bic

′i + cib

′i ,

Ai,7 = bid′i + dib

′i , Ai,8 = cic

′i ,

Ai,9 = cid′i + dic

′i , Ai,10 = did

′i ,

and q =[q1 q2 . . . q10

]T. Here q can be computed

as the smallest singular vector of the matrix A and the ma-trix Q′ constructed by a re-arrangement of the vector. Sin-gularity constraint can be imposed by setting the smallestsingular value of Q′ to zero which costs the least error interms of Frobenius norm. The planes are normalized to

unit norm to further aid the stability of the algorithm un-der noise.

The algorithm presented above is a very simple andhighly efficient way of estimating the Absolute DualQuadric, and thus rectifying the structure. However, itdoes not impose the positive semi-definiteness constraintand when Q′ is not positive semi-definite, the rectifyinghomography cannot be factored out. This, however, rarelyhappens with correct constraints in practice and when a badrandom sample is picked by RANSAC, we simply discardthe iteration as degenerate.

4.2. Automatic Rectification using RANSAC

We have a linear method for estimation of the rectify-ing homography if some known orthogonal plane-pairs areavailable. We now present an automatic method to deter-mine orthogonal plane-pairs and the rectifying homographysimultaneously. Since the input structure is a man-made en-vironment, therefore we may assume that planes frequentlymeet at 90◦. We use RANSAC to obtain the rectifyinghomography automatically, like the 2D metric rectificationcounter-part described in [9], which maximizes the numberof orthogonal plane-pairs after rectification.

In the RANSAC part, we repeatedly estimate the ADQthrough the linear method described above using a randomset of nine plane-pairs. From the estimate of ADQ, we com-pute the homography, and count the total number of plane-pairs having a dot-product less than a threshold, and con-sider inliers. This step is repeated a number of times andthe best estimate of the rectifying homography is used asinitial guess to nonlinearly refine the solution (describedlater). Once in a while, the recovered ADQ is not posi-tive semi-definite but it rarely happens with a good sam-ple; we discard such trials as degenerate. We use adaptiveRANSAC in order to automatically compute the number ofrandom trials needed to achieve a good confidence in theresult [1, p. 120].

In man-made environments, adjacent plane-pairs are typ-ically more likely to be orthogonal. Plane adjacencies arecomputed in existing literature using common 3D points ordelineations in plane boundaries in images [16, 14]. If planeadjacencies are available they make RANSAC faster by al-lowing to sample only from adjacent plane-pairs; otherwise,all plane-pairs are assumed adjacent. This algorithm doesnot enforce a global constraint on the structure and does notrequire any knowledge of cameras or images, in contrast tothe existing approaches.

4.3. Nonlinear Refinement

Since our linear algorithm for computing the rectifying3D homography using ADQ is not exact, here we developan algorithm that directly estimates the rectifying homogra-phy given an initial guess and some inter-plane constraints.

33

Page 4: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

Algorithm 1 3D Rectification using Angle Regularity

Input: P , a set of planes in the image; A, the plane adja-cency matrix; and t, the RANSAC thresholdbestinliers← {}bestH← I4×4repeat

pick nine adjacent plane-pairs uniformly at randomfind the rectifying homography H linearly assumingthese pairs are orthogonal (section 4.1)inliers← {}for all pi, qi ∈ P do

if (uTi vi)

2 < t then {(uTi vi)2 defined in Sec-

tion 4.3}add (pi, qi) to inliers

end ifend forif |inliers| > |bestinliers| then

bestH← Hbestinliers← inliers

end ifuntil a high confidence consensus set is foundcompute the plane pairs that have become close to paral-lel after rectificationrefine bestH (section 4.3) using orthogonal as well as par-allel constraints

The initial guess of rectifyig homography and angle con-straints are computed during RANSAC. Both orthogonal aswell as parallel plane-pairs can be used at this stage but atleast five orthogonality constraints are necessary since par-allelism constraints only help in computing the plane at in-finity (i.e. the projective part of the rectifying homography).We use this algorithm to refine the rectifying homographycomputed from RANSAC.

Given a set of N >= 8 plane pairs (pi, qi) and a list Ospecifying whether the planes are orthogonal (Oi = 0) orparallel (Oi = 1), the rectifying homography is computedby minimizing the cost:

cH =

N∑i=1

(Oi −

∣∣uTi vi

∣∣2)2 (7)

where p′i = H−T pi, q′i = H−T qi; ui and vi are computedby normalizing the first three components of p′i and q′i, re-spectively. Note that the summation terms in this equa-tion should be zero for ideal orthogonal as well as paral-lel planes. The minimization can be performed using theLevenberg-Marquardt minimization algorithm. The com-plete algorithm with RANSAC as well as nonlinear refine-ment is explained in Algorithm 1.

5. Relative Scales AmbiguityWhile we have presented the method to recover the struc-

ture up to a similarity by computing the rectifying homogra-phy, in certain cases, the relative scale parameters betweenthe three principal directions cannot be computed. This hap-pens when all the planes are aligned along the three princi-pal directions in the 3D world. In such a case, we haveno constraint on the two relative scale parameters, becauseeach relative scale produces exactly the same cost for theoptimization.

Theorem: Given a set of planes A ∈ P3 and their pairingsthat provide angular constraints (parallel or perpendicular)on them such that every plane in A is involved in some ofthe constraints; if for any plane p ∈ A, all the planes inA\{p} are either perpendicular or parallel to p, then the rel-ative scale in the direction of the normal vector of the planep cannot be computed using only the given information.

Proof: We assume that p is parallel to the principal planeXY without loss of generality because the P3 space can berotated to make p parallel to the XY plane without chang-ing any of the inter-planar constraints or the Absolute DualQuadric. The set A can be partitioned into two sets B andC such that B contains all the planes (including p) that areparallel to the XY plane, and all the planes in C are per-pendicular to XY plane. Any plane pi ∈ B can be writtenas:

pi = [0, 0, 1, di]T , (8)

where−di is the perpendicular distance of the plane pi fromthe origin. Similarly, any plane pj ∈ C can be written as:

pj = [uj , vj , 0, dj ]T , (9)

where at least one uj and vj is not zero, [uj , vj , 0]T is nor-

mal to pj , and −dj

u2j+v2

jis the perpendicular distance of the

plane pj from the origin. Now consider any arbitrary scal-ing in the Z-axis i.e. the direction normal to p:

H =

1 0 0 00 1 0 00 0 s 00 0 0 1

, (10)

where s is an arbitrary non-zero scale. Therefore the trans-formed planes will be:

H−T pi ∼= p′

i = [0, 0, 1, sdi]T (11)

andp

j = H−T pj = [uj , vj , 0, dj ]T . (12)

Since the transformation does not change the planes in Ctherefore their internal angular constraints stand un-altered.

34

Page 5: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

0 10 20 30 40 50 600

5

10

15

20

25

Outliers percentage

RM

S e

rro

r (i

n d

eg

ree

s)

noise in training data

noise in test data

RANSAC rectification error in training data

RANSAC rectification error in test data

nonlinear rectification error in training data

nonlinear rectification error in test data

(a)

0 5 10 15 20 25 30 35 40 45 50

8

10

12

14

16

18

Outliers percentage

RM

S e

rro

r (i

n d

eg

ree

s)

noise in training data

noise in test data

RANSAC rectification error in training data

RANSAC rectification error in test data

nonlinear rectification error in training data

nonlinear rectification error in test data

(b)

Figure 1: Quantitative evaluation of 3D rectification, the plots show angle rectification error for different percentages ofoutliers with (a) 25 input constraints and 2 degree Gaussian noise and (b) 100 input constraints and 10 degree Gaussian noise.The results have been averaged over 60 and 90 random trials per data point respectively. Best viewed in color.

Similarly, all the planes in B remain parallel to each other.And the planes in B are still orthogonal to the planes in Cafter the scaling. Since none of the angle constraints havebeen changed by the transformation H therefore the sameangle constraints are valid for all non-zero scales in the di-rection of Z-axis i.e. normal to the plane p. Hence therelative scale in Z-axis direction cannot be recovered usingthese angular constraints alone. This completes the proof.This scale ambiguity in 3D is directly analogous to the sim-ilar scale ambiguity involved in 2D Metric Rectification us-ing orthogonal lines in [8].

From the proof above, it is obvious that the angle con-straints of any plane that is neither perpendicular nor par-allel to p will not be invariant to an arbitrary scaling byH−T (since their third component cannot be zero). How-ever, there are two degrees of freedom for relative scales andwe need at least two constraints involving planes that areneither perpendicular nor parallel to the rest of the planes inorder to recover the complete rectifying homography.

6. ResultsThe input to our algorithm consists of a projectively dis-

torted structure with a given multi-planar segmentation thatcould be used to fit the planes. Since the algorithms used forreconstructing the input projective structure and segmenta-tion could be highly varied, we model their inaccuracies interms of Gaussian error in the angle between mutually or-thogonal plane-pairs. The algorithm was tested under zeromean Gaussian noise with 0-20◦ standard deviation. Sincewe do not impose a global regularity constraint, we also testagainst outliers i.e. non-orthogonal plane-pairs. Uniformlyrandom angles were used for outliers and the algorithm wastested with 0−60% outliers. The results were averaged over

Table 1: Error Statistics (in degrees) for Rectification with100 noisy constraints (0-60% outliers shown in columnsand 0-20◦ Gaussian error in angles shown in rows)

- 0% 10% 20% 30% 40% 50% 60%0◦ 0.2531 0.3594 0.3547 1.0526 0.9838 0.8929 1.53422◦ 0.8748 0.8737 1.1536 1.1724 1.6261 1.8812 3.55724◦ 2.6266 2.4912 2.6348 3.0633 3.5833 4.3717 6.82756◦ 4.2285 4.3853 4.6049 5.4005 5.9636 7.6886 10.22778◦ 5.4210 6.1341 7.1370 7.4398 8.0200 10.2314 13.467910◦ 7.2488 8.0873 8.2666 9.1230 10.4425 12.5932 14.877712◦ 8.6888 9.7061 10.4679 11.4578 12.8848 14.8009 16.635314◦ 10.4518 10.5650 12.1307 12.4736 15.3280 15.9277 18.562716◦ 11.6304 13.0353 12.2874 14.3241 15.3467 16.6695 20.283718◦ 13.9390 14.4375 14.3310 15.7926 17.9819 17.4549 21.830320◦ 14.6225 14.9180 16.2473 16.7124 17.7220 19.9708 22.2968

60 random trials and they are presented in Table 1. Two rep-resentative examples in Figure 1 demonstrate the effect ofoutliers for fixed noise in orthogonal pairs and total numberof input constraints. These results show that the proposedalgorithm is stable over a wide range of noise and outliers.The algorithm, typically becomes unstable when the num-ber of RANSAC inliers drops to less than 15. Comparingthe two graphs in Figure 1 also demonstrates that numberof input constraints significantly effect the stability of thealgorithm.

We visualize a lab dataset in Figure 2 where the inputstructure is hardly recognizable but the proposed methodrectifies it correctly up to a vertical scale ambiguity. Someestimation algorithms tend to over-fit the model to the pro-vided data but perform worse on unseen data. Therefore,we also test the model accuracy on a separate noise-freeset of orthogonal plane-pairs. Figure 1 shows that our al-gorithm consistently performs better for the noise-free testdata, eliminating the possibility of over-fitting.

35

Page 6: [IEEE 2014 IEEE Winter Conference on Applications of Computer Vision (WACV) - Steamboat Springs, CO, USA (2014.3.24-2014.3.26)] IEEE Winter Conference on Applications of Computer Vision

(a) (b) (c)

Figure 2: 3D metric rectification of a randomly distorted projective structure: a) one view of the actual structure (recoveredusing Tomasi-Kanade algorithm [18]), b) projectively distorted structure (using a random homography), and c) automaticallyrectified structure.

7. AcknowledgementsThe authors gratefully acknowledge helpful comments

from Dr. Yaser Sheikh (CMU). The research was fundedin part by Higher Education Commission of Pakistan andLahore University of Management Sciences.

References[1] R. I. Hartley and A. Zisserman, Multiple View Geometry

in Computer Vision. Cambridge University Press, ISBN:0521540518, second ed., 2004.

[2] B. Triggs, “Autocalibration and the absolute quadric,” incvpr, p. 609, Published by the IEEE Computer Society, 1997.

[3] A. Zaheer, “Project Page - Metric Rectification us-ing Angle Regularity.” cvlab.lums.edu.pk/MetricRectification/, 2014.

[4] Z. Pizlo, “A theory of shape constancy based on perspectiveinvariants,” Vision Research, vol. 34, no. 12, pp. 1637–1658,1994.

[5] Z. Pizlo, 3D Shape: Its unique place in our perception. TheMIT Press, 2008.

[6] J. Saunders and B. Backus, “Both parallelism and orthog-onality are used to perceive 3d slant of rectangles from 2dimages,” Journal of Vision, vol. 7, no. 6, 2007.

[7] V. Ramachandran, “3 visual perception in people and ma-chines,” in AI and the Eye:[proceedings of the 11th Euro-pean Conference on Visual Perception, Held in Bristol, UKin 1988], p. 21, John Wiley & Sons, 1990.

[8] D. Liebowitz and A. Zisserman, “Metric rectification for per-spective images of planes,” in Computer Vision and PatternRecognition, 1998. Proceedings. 1998 IEEE Computer Soci-ety Conference on, pp. 482–488, IEEE, 1998.

[9] A. Zaheer, M. Rashid, and S. Khan, “Shape from angle regu-larity,” in Computer Vision–ECCV 2012, pp. 1–14, Springer,2012.

[10] M. Pollefeys, R. Koch, and L. Van Gool, “Self-calibrationand metric reconstruction inspite of varying and unknownintrinsic camera parameters,” International Journal of Com-puter Vision, vol. 32, no. 1, pp. 7–25, 1999.

[11] P. F. Sturm and S. J. Maybank, “On plane-based camera cal-ibration: A general algorithm, singularities, applications,” inComputer Vision and Pattern Recognition, 1999. IEEE Com-puter Society Conference on., vol. 1, IEEE, 1999.

[12] M. Wilczkowiak, P. Sturm, and E. Boyer, “Using geomet-ric constraints through parallelepipeds for calibration and 3dmodeling,” IEEE transactions on pattern analysis and ma-chine intelligence, pp. 194–207, 2005.

[13] C. Rother, S. Carlsson, and D. Tell, “Projective factorizationof planes and cameras in multiple views,” in Pattern Recog-nition, 2002. Proceedings. 16th International Conference on,vol. 2, pp. 737–740, IEEE, 2002.

[14] A. Bartoli and P. Sturm, “Constrained structure and mo-tion from multiple uncalibrated views of a piecewise planarscene,” International Journal of Computer Vision, vol. 52,no. 1, pp. 45–64, 2003.

[15] P. Sturm and B. Triggs, “A factorization based algorithm formulti-image projective structure and motion,” Computer Vi-sion ECCV 96, pp. 709–720, 1996.

[16] A. Bartoli, “A random sampling strategy for piecewise pla-nar scene segmentation,” Computer Vision and Image Un-derstanding, vol. 105, no. 1, pp. 42–59, 2007.

[17] R. I. Hartley, “In defense of the eight-point algorithm,” IEEETransactions on Pattern Analysis and Machine Intelligence,vol. 19, no. 6, pp. 580–593, 1997.

[18] C. Tomasi, “Shape and motion from image streams underorthography: a factorization method,” International Journalof Computer Vision, vol. 9, pp. 137–154, 1992.

36