new geometric interpretation and analytic solution for quadrilateral reconstruction (icpr-2014)

6
New Geometric Interpretation and Analytic Solution for Quadrilateral Reconstruction Joo-Haeng Lee Convergence Technology Research Lab ETRI Daejeon, 305–777, KOREA Abstract—A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we gen- eralize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilat- eral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples. I. I NTRODUCTION A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadri- laterals. We extend the previous approach, called coupled line camera (CLC), which models a rectangular frustum of a pinhole camera using two line cameras [1], [2]. (A line camera in our context does not refer to a capturing device such as a line-scan camera. Rather, our geometric configuration is more related to modeling approaches based on linear elements for camera calibration [3] or multi-perspective image [4].) Under CLC configuration, geometric relation among the base rectangle, the image quadrilateral and the optical center can be comprehensively described as simple equations of a compact parameter set. Hence, given a single image quadri- lateral, we can uniquely identify the frustum by reconstructing the base rectangle and optical center using a closed-form solution. The solution also contains a determinant that tells if a image quadrilateral is the projection of any rectangle prior to reconstruction. In the CLC-based reconstruction, no explicit form of camera parameters is involved since the formulation is based on pure geometric configuration of a pinhole projection. In application, an image quadrilateral is represented by a set of diagonal parameters (i.e. relative lengths of partial diagonals and the crossing angle) rather than actual pixel coordinates. If re- quired, unknown camera parameters such as the focal length can be computed subsequently using a standard calibration technique [5], [6]. Generally the previous solutions require to reconstruct the camera parameters first [7]. For example, when we apply the IAC (image of the absolute conic) method, the unknown focal length should be found first [5], [8]. Another interesting feature of CLC-based reconstruction is geometric interpretation of the solution space, which leads to an optimized analytic solution [2]. For example, given an im- age quadrilateral, two candidate line cameras are defined over two solution spheres. By the constraint of common principal axis, spheres are confined to two solution circles. Finally, the optical center is found in the intersection of two solutions circles. We believe a similar geometric framework can be applied in other geometric computer vision problems such as investigating the solution space of n-view reconstruction. In this paper, we propose generalized coupled line camera (GCLC) that inherits the key features of CLC and models a projective frustum with a quadrilateral base, which targets on a prospective application of projective reconstruction of an unknown scene quadrilateral. While keeping the same centering constraint of CLC that the principal axis passes through the center of quadrilaterals, we extend the model with additional parameters to describe the lengths of all partial diagonals. In CLC, these parameters need not be specified since they cancel out due to equilateral partial diagonals of a rectangle [1], [2]. The increased number of configuration parameters in GCLC, however, hinders to formulate a closed- form solution for single view reconstruction. We investigate this property and propose an analytic solution that works for single view reconstruction under special conditions, and a method to approximate unknown diagonal parameters from multiple views. For practical application of CLC framework, we need to handle an off-centered case. In this paper, we also propose an improved method composed of simpler operations based on geometric properties, not relying on constrained equation solving or explicit homography as in [1]. This paper is organized as follows. In Section II, we sum- marize the previous work on CLC [1], [2]. In Section III, we generalize CLC and describe reconstruction solution including off-centered cases. In Section IV, we give experimental results on synthetic and real quadrilaterals to demonstrate the perfor- mance. Finally, we conclude with remarks on future work.

Upload: joo-haeng-lee

Post on 03-Jul-2015

470 views

Category:

Technology


0 download

DESCRIPTION

Accepted as poster presentation for ICPR 2014, Stockholm, Sweden on August 24~28, 2014. [Revised Version] Title: New geometric interpretation and analytic solution for quadrilateral reconstruction Author: Joo-Haeng Lee Affiliation: Human-Robot Interaction Research Team, ETRI, KOREA Abstract: A new geometric framework, called generalized coupled line camera (GCLC), is proposed to derive an analytic solution to reconstruct an unknown scene quadrilateral and the relevant projective structure from a single or multiple image quadrilaterals. We extend the previous approach developed for rectangle to handle arbitrary scene quadrilaterals. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. A completely unknown quadrilateral can be reconstructed from four views through non-linear optimization. We also describe a improved method to handle an off-centered case by geometrically inferring a centered proxy quadrilateral, which accelerates a reconstruction process without relying on homography. The proposed method is easy to implement since each step is expressed as a simple analytic equation. We present the experimental results on real and synthetic examples. [Submitted Version] Title: Generalized Coupled Line Cameras and Application in Quadrilateral Reconstruction Abstract: Coupled line camera (CLC) provides a geometric framework to derive an analytic solution to reconstruct an unknown scene rectangle and the relevant projective structure from a single image quadrilateral. We extend this approach as generalized coupled line camera (GCLC) to handle a scene quadrilateral. First, we generalize a single line camera by removing the centering constraint that the principal axis should bisect a scene line. Then, we couple a pair of generalized line cameras to model a frustum with a quadrilateral base. Finally, we show that the scene quadrilateral and the center of projection can be analytically reconstructed from a single view when prior knowledge on the quadrilateral is available. ...

TRANSCRIPT

Page 1: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

New Geometric Interpretation and Analytic Solutionfor Quadrilateral Reconstruction

Joo-Haeng LeeConvergence Technology Research Lab

ETRIDaejeon, 305–777, KOREA

Abstract—A new geometric framework, called generalized

coupled line camera (GCLC), is proposed to derive an analytic

solution to reconstruct an unknown scene quadrilateral and the

relevant projective structure from a single or multiple image

quadrilaterals. We extend the previous approach developed for

rectangle to handle arbitrary scene quadrilaterals. First, we gen-

eralize a single line camera by removing the centering constraint

that the principal axis should bisect a scene line. Then, we couple

a pair of generalized line cameras to model a frustum with a

quadrilateral base. Finally, we show that the scene quadrilateral

and the center of projection can be analytically reconstructed

from a single view when prior knowledge on the quadrilat-

eral is available. A completely unknown quadrilateral can be

reconstructed from four views through non-linear optimization.

We also describe a improved method to handle an off-centered

case by geometrically inferring a centered proxy quadrilateral,

which accelerates a reconstruction process without relying on

homography. The proposed method is easy to implement since

each step is expressed as a simple analytic equation. We present

the experimental results on real and synthetic examples.

I. INTRODUCTION

A new geometric framework, called generalized coupledline camera (GCLC), is proposed to derive an analytic solutionto reconstruct an unknown scene quadrilateral and the relevantprojective structure from a single or multiple image quadri-laterals. We extend the previous approach, called coupledline camera (CLC), which models a rectangular frustum of apinhole camera using two line cameras [1], [2]. (A line camerain our context does not refer to a capturing device such as aline-scan camera. Rather, our geometric configuration is morerelated to modeling approaches based on linear elements forcamera calibration [3] or multi-perspective image [4].)

Under CLC configuration, geometric relation among thebase rectangle, the image quadrilateral and the optical centercan be comprehensively described as simple equations of acompact parameter set. Hence, given a single image quadri-lateral, we can uniquely identify the frustum by reconstructingthe base rectangle and optical center using a closed-formsolution. The solution also contains a determinant that tells ifa image quadrilateral is the projection of any rectangle priorto reconstruction.

In the CLC-based reconstruction, no explicit form of cameraparameters is involved since the formulation is based on puregeometric configuration of a pinhole projection. In application,an image quadrilateral is represented by a set of diagonalparameters (i.e. relative lengths of partial diagonals and the

crossing angle) rather than actual pixel coordinates. If re-quired, unknown camera parameters such as the focal lengthcan be computed subsequently using a standard calibrationtechnique [5], [6]. Generally the previous solutions require toreconstruct the camera parameters first [7]. For example, whenwe apply the IAC (image of the absolute conic) method, theunknown focal length should be found first [5], [8].

Another interesting feature of CLC-based reconstruction isgeometric interpretation of the solution space, which leads toan optimized analytic solution [2]. For example, given an im-age quadrilateral, two candidate line cameras are defined overtwo solution spheres. By the constraint of common principalaxis, spheres are confined to two solution circles. Finally, theoptical center is found in the intersection of two solutionscircles. We believe a similar geometric framework can beapplied in other geometric computer vision problems such asinvestigating the solution space of n-view reconstruction.

In this paper, we propose generalized coupled line camera(GCLC) that inherits the key features of CLC and modelsa projective frustum with a quadrilateral base, which targetson a prospective application of projective reconstruction ofan unknown scene quadrilateral. While keeping the samecentering constraint of CLC that the principal axis passesthrough the center of quadrilaterals, we extend the model withadditional parameters to describe the lengths of all partialdiagonals. In CLC, these parameters need not be specifiedsince they cancel out due to equilateral partial diagonals ofa rectangle [1], [2]. The increased number of configurationparameters in GCLC, however, hinders to formulate a closed-form solution for single view reconstruction. We investigatethis property and propose an analytic solution that works forsingle view reconstruction under special conditions, and amethod to approximate unknown diagonal parameters frommultiple views. For practical application of CLC framework,we need to handle an off-centered case. In this paper, we alsopropose an improved method composed of simpler operationsbased on geometric properties, not relying on constrainedequation solving or explicit homography as in [1].

This paper is organized as follows. In Section II, we sum-marize the previous work on CLC [1], [2]. In Section III, wegeneralize CLC and describe reconstruction solution includingoff-centered cases. In Section IV, we give experimental resultson synthetic and real quadrilaterals to demonstrate the perfor-mance. Finally, we conclude with remarks on future work.

Page 2: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

ds0

s2

q0

y2 y0

u0

u2

v0v2 vm

pc

l0l2

m0m2

(a) Camera pose when d = 1.7.

cv0v2 vm

(b) Circular trajectory of pc for varying d.

Fig. 1. An example of a canonical line camera: m0 = m2 = 1, l0 = 0.6,l2 = 0.4, and ↵ = 0.2.

II. PRELIMINARIES OF COUPLED LINE CAMERAS

A. Line CameraDefinition 1. A line camera captures an image line uiui+2

from a scene line vivi+2

where vi = (mi, 0, 0) and vi+2

=

(�mi+2

, 0, 0) for positive mi and mi+2

. See Figure 1a.

Definition 2. In a centered line camera, the principal axispasses through the center vm of the scene line vivi+2

:

vm = (vi + vi+2

)/2. (1)

Definition 3. A canonical line camera is a centered linecamera with two constraints for simple formulation: vm =

(0, 0, 0)T and equilateral unit division:

kvi � vmk = kvik = kvi+2

k = 1. (2)

For a line camera Ci, let d be the length of the principal axisfrom the center of projection pc to vm. Let ✓i be the orientationangle of the principal axis measured between vmp

c

and vmvi.

Definition 4. For a canonical line camera, its pose equationis expressed as follow:

cos ✓i =

✓li � li+2

li + li+2

◆d = ↵i d (3)

where li = kui�umk is the length of partial diagonals. Let ↵i

be the line division coefficient of the canonical configuration

↵i =li � li+2

li + li+2

(4)

According to Eq.(3), we can observe the relation among ✓i,d and ↵i. Note that when ↵i is fixed, pc is defined along acircular trajectory or on a solution sphere of radius 0.5/|↵|.See Figure 1b.

B. Coupled Line CamerasDefinition 5. Coupled line camera is a pair of line cameras,that share the principal axis and the center of projection.By coupling two canonical line cameras, we can represent aprojective structure with a rectangle base. See Figure 2.

Definition 6. For coupled line camera, we can derive acoupling constraint:

� =

l1

l0

=

tan 1

tan 0

=

sin ✓1

(d� cos ✓0

)

sin ✓0

(d� cos ✓1

)

(5)

v0

v1 v2

v3

f vm

G

(a) Scene rectangle G (b) 1st line camera C0 (c) 2nd line camera C1

(d) Coupling C0 and C1 (e) Projective structure

u0

u1 u2

u3r um

Q

(f) Projection of G to Q

Fig. 2. Coupling of two canonical line cameras to represent a projectivestructure with a rectangle base.

where � is the coupling coefficient defined by the ratio of thelengths, l

0

and l1

, of two partial diagonals of Q. See Figure 2f.

C. Projective Reconstruction

Algorithm 1 (Single View Reconstruction with CLC). Theunknown elements of projective structure, such as the scenerectangle G and the center of projection pc, can be recon-structed from a single image quadrilaterals Q as in the below.

First, the pose equation of Eq.(3) and the coupling constraintof Eq.(5) can be rearranged into a system of equations:

d =

� sin ✓0

cos ✓1

� cos ✓0

sin ✓1

� sin ✓0

� sin ✓1

=

cos ✓0

↵0

=

cos ✓1

↵1

(6)

Then, the length d of the common principal axis can becomputed from the system of equations in Eq.(6) as follows:

d =

pA

0

/A1

(7)

where A0

= (1 � ↵1

)

2�2 � (1 � ↵0

)

2 and A1

= ↵2

0

(1 �↵1

)

2�2 � (1 � ↵0

)

2↵2

1

. Once d is computed, two orientationangles, ✓

0

and ✓1

, can be computed using Eq.(3).The base rectangle G can be reconstructed by computing

its unknown shape parameter, the diagonal angle �:

cos� = cos ⇢ sin ✓0

sin ✓1

+ cos ✓0

cos ✓1

(8)

where ⇢ is the diagonal angle of the image quadrilateral Q.Finally, the projective structure can be reconstructed by

computing the coordinates of a center of projection pc:

pc =d (sin� cos ✓

0

, cos ✓1

� cos� cos ✓0

, sin ⇢ sin ✓0

sin ✓1

)

sin�(9)

D. Determinant Condition

When Eq.(7) has a valid value, two conditions should besatisfied: (1) A

0

and A1

have the same sign; and (2) the lengthd of the common principal axis should not exceed the diameter

Page 3: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

ds0

s2q0

y2 y0

u0

u2

v0v2 vm

pc

l0

l2

m0m2

(a) Camera pose when d = 1.7.

cv0v2 vm

(b) Trajectory of pc when d is not fixed.

Fig. 3. An example of a generalized line camera: m0 = 1, m2 = 1.4,l0 = 0.6, l2 = 0.4, and ↵ = 0.2.

of each solution sphere: d min(1/k↵0

k, 1/k↵1

k). Theseconditions can be combined into Boolean expressions:

D = D0

_D1

(10)

D0

=

✓� � 1� ↵

0

1� ↵1

◆^✓1

����↵0

↵1

����

◆(11)

D1

=

✓� 1� ↵

0

1� ↵1

◆^✓1 �

����↵0

↵1

����

◆(12)

where ^ and _ are Boolean and and or operations, respec-tively. Since ↵

0

, ↵1

and � are the coefficients from a givenimage quadrilateral Q, we can determine if Q is an imageof any scene rectangle before actual reconstruction. Once thedeterminant D is satisfied, Algorithm 1 can be applied.

E. Off-Centered Case

CLC assumes the principal axis passes through the centersof the image quadrilateral Q and the scene rectangle G. Whenhandling an off-centered quadrilateral Q

g

, a centered proxyquadrilateral Q should be found first by solving equationsthat formulate edge parallelism between Q and Q

g

, centeringconstraint of Q, and a vanishing line derived from Q

g

[1].Once Q is found, the centered proxy rectangle G can be

reconstructed using Algorithm 1. Since the inferred Q doesnot guarantee congruency to Q

g

, the target scene rectangleG

g

should be reconstructed using a homography H betweenQ and G: G

g

= HQg

.In this paper, we propose a new method to handle an off-

centered case. First, we derive a centered proxy quadrilateralQ that is perspectively congruent to Q

g

. Then, we show thatthe target scene rectangle G

g

can be geometrically derivedwithout relying on homography. See Section III-E.

III. GENERALIZATION OF COUPLED LINE CAMERAS

As a main contribution of this paper, we generalize a linecamera to support a non-canonical configuration. Then, weshow that a pair of generalized line cameras can be coupled torepresent a projective structure with a quadrilateral base otherthan a rectangle. Finally, we describe how we can reconstructa projective structure from a single view with a sufficient priorknowledge to constrain the solution space. We also describehow to handle off-centered cases.

v0v1

v2 v3

f

vmG

(a) Scene quad. G (b) 1st line camera C0 (c) 2nd line camera C1

(d) Coupling C0 and C1 (e) Projective Structure

u0

u1 u2

u3r um

Q

(f) Projection of G to Q

Fig. 4. Coupling of two generalied line cameras to represent a projectivestructure with a quadrilateral base. A generalized line camera Ci is assignedfor each diagonal of a scene quadrilateral G. actual values of diagonalparameters.

A. Generalized Line Camera

Definition 7. In a general configuration of a line camera,the principal axis may not bisect the scene line: we may notconsider the centering constraints of Eqs.(1)-(2). See Figure 3where m

0

6= m2

.Accordingly, the pose equation of a canonical line camera in

Eq.(3) should be generalized with two additional parameters,m

0

and m2

. Assuming m0

> 0 and m2

> 0, the followinggeometric relation holds:

li : li+2

= mi sin ✓0d

d� ˆdi: mi+2

sin ✓0

d

d+ ˆdi+2

(13)

where ˆd0

= m0

cos ✓0

and ˆd2

= m2

cos ✓0

.

Definition 8. The generalized pose equation can be derivedfrom Eq.(13):

cos ✓i =

✓mi+2

li �mili+2

mimi+2

(li + li+2

)

◆d = ↵

g,i d (14)

where ↵g,i is the generalized division coefficient

↵g,i =

mi+2

li �mili+2

mimi+2

(li + li+2

)

. (15)

For a fixed ↵g,i, the center of projection pc is defined over a

circular trajectory as in Figure 3b, or on a solution sphere [2].

B. Coupling Generalized Line Cameras

By coupling two generalized line cameras, we can representa projective structure with a quadrilateral base G with vertices:v0

= m0

(1, 0), v1

= m1

(cos�, sin�), v2

= �m2

/m0

v0

, andv3

= �m3

/m1

v1

where mi’s are the relative lengths of partialdiagonals or diagonal parameters of G. See Figure 4.

Definition 9. A generalized coupling constraint �g

is definedas follows:

�g

=

l1

l0

=

m1

sin ✓1

m0

sin ✓0

(d�m0

cos ✓0

)

(d�m1

cos ✓1

)

(16)

Page 4: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

C. Projective ReconstructionUsing a trigonometric identity and the pose equation of

Eq.(14), we can derive the equation for �2

g

by squaring boththe sides of Eq.(16):

sin

2 ✓i = 1� cos

2 ✓i = 1� ↵2

g,i d2 (17)

�2

g

=

m2

1

(1�m0

↵g,0)

2

(1� ↵2

g,1d2

)

m2

0

(1�m1

↵g,1)

2

(1� ↵2

g,0d2

)

(18)

From Eq.(18), the length d of the common principal axiscan be expressed with GCLC parameters:

d =

sA

g,0

Ag,1

(19)

where Ag,0 = m2

0

(1 � m1

↵g,1)

2�2

g

� m2

1

(1 � m0

↵g,0)

2 andA

g,1 = m2

0

↵2

g,0(1 � m1

↵g,1)

2�2

g

� m2

1

(1 � m0

↵g,0)

2↵2

g,1.Eq.(19) states that d can be computed from known diagonalparameters, mi and li, of a single pair of scene and imagequadrilaterals, not relying on their diagonal angles, � and ⇢.

Algorithm 2 (Single View Reconstruction with GCLC). Oncethe length d of the common principal axis has been foundusing Eq.(19) with prior knowledge on diagonal parameters,we can compute the orientation angles, ✓

0

and ✓1

, using thepose equation of Eq.(14). Then, the diagonal angle � of a scenequadrilateral and the center of projection pc can be computedusing Eqs.(8) and (9), respectively. ⇤

If we have no prior knowledge on diagonal parameters mi

of G, we can infer them using multiple image quadrilateralsQj from different views. By setting m

0

= 1, the number ofunknown diagonal parameters of G is reduced to three: m

1

,m

2

and m3

. For each Qj , the crossing angle �j of Eq.(8) isexpressed with m

1

, m2

and m3

, and coefficients derived fromknown diagonal parameters li,j of Qj . Since the reconstructed�j’s should be identical regardless of views, the followingidentity should hold: cos�j = cos�j+1

. Hence, if we havefour different views, we can formulate three equations of threeunknowns, m

1

, m2

and m3

:

cos�0

= cos�1

= cos�2

= cos�3

(20)

The number of views are varying according to the degree offreedom in diagonal parameters.

Although an analytic solution for Eq.(20) is not foundyet, the problem can be formulated as minimization of thefollowing objective function:

fobj

=

n�1X

j=0

k cos�j � cos�j+1

k2 (21)

where n is the number of views. Generally, Eq.(21) can besolved using a numerical nonlinear optimization method [9].Since optimization may get stuck in a local minima, we maycheck the validity using determinant of Eq. 24.

Algorithm 3 (n-View Reconstruction with GCLC). WhenAlgorithm 2 cannot be applied due to lack of knowledgeon the scene rectangle G, but we have multiple image

(a) Reference: Gg and Qg (b) Inferring a centered Q in blue

(c) Reconstruction of G and Gg (d) Congruency of G and Gg

Fig. 5. Reconstruction of a synthetic quadrilateral Gg from an off-centeredquadrilateral Qg: m0 = 1, m1 = 0.75, m2 = 1.35, m3 = 1.4 and � =1.35. Diagonal parameters mi and the vanishing line is given.

quadrilaterals Qj from n different views, we can find theunknown mi’s by minimizing the objective function ofEq.(21). Then, we can apply Algorithm 2 for one of theviews to reconstruct the projective structure. ⇤

The number of views required in Algorithm 3 depends onthe number of unknown mi’s. For a general quadrilateral ofthree unknown mi’s except m

0

= 1, at least 4 views arerequired according to Eq.(20). For a parallelogram with knownm

0

= m2

= 1 and unknown m1

= m3

, at least 2 views arerequired to find m

1

. See Section IV for real examples.

D. Determinant ConditionSimilarly as in Section II-D, we can derive, from Eqs.(14)

and (19), a condition Dg

that can determine if Q is projectionof a centered scene quadrilateral G with known mi’s.

Dg

= Dg,0 _D

g,1 (22)

Dg,0 =

⇣� � m1(1�m0↵g,0)

m0(1�m1↵g,1)

⌘^⇣1

���↵g,0

↵g,1

���⌘

(23)

Dg,1 =

⇣� m1(1�m0↵g,0)

m0(1�m1↵g,1)

⌘^⇣1 �

���↵g,0

↵g,1

���⌘

(24)

E. Off-Centered CaseLet an off-centered image quadrilateral Q

g

be projectionof a scene quadrilateral G

g

, which is also off-centered andunknown yet. See Fig. 5a. To apply Algorithms 2 and 3, weprovide a method to find a centered proxy quadrilateral Q thatis an image of a centered scene quadrilateral G. Specially, G isguaranteed to be congruent to G

g

through parallel translationby t. We also show that the translation vector t can becomputed in image space. Hence, we do not need to computehomography H between G and Q to reconstruct G

g

as in CLC.See Section II-E and [1].

Algorithm 4 (Reconstruction from an Off-Centered Quadri-lateral). An off-centered scene quadrilateral G

g

can be recon-structed from its image Q

g

by adding extra steps to the GCLCmethods presented in Section III-C. See Figure 5:

Page 5: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

QgQ

omug,0 ug,1

ug,2ug,3 um

u0 u1

u2u3

w0w1

wd,0

wd,1

wm

Fig. 6. Derivation of a centered proxy quadrilateral Q that is perspectivelycongruent to Qg. Assume the vanishing line w0w1 is given.

1) Infer a centered proxy quadrilateral Q from Qg

suchthat Q is projection of a centered scene quadrilateralG that is congruent to the target quadrilateral G

g

. SeeAlgorithm 5;

2) Apply Algorithm 2 to Q to reconstruct the correspondingcentered quadrilateral G and the center of projection p

c

.If multiple Q

g,j are available, apply Algorithm 3.3) The target scene quadrilateral G

g

can be computed astranslation of G: G

g

= G+ t where t can be computedfrom displacement s = u

m

� om

between centers of Qand Q

g

using Algorithm 6.

Algorithm 5 (Centered Proxy Quadrilateral). Assuming avanishing line w

0

w1

is given, we can find a centered proxyquadrilateral Q by perspectively translating an off-centeredquadrilateral Q

g

. See Figure 6:1) Find the intersection points w

d,i between the vanishingline w

0

w1

and each diagonal ug,iug,i+2

of Gg

.2) Find the intersection point w

m

between the vanishingline w

0

w1

and the line of translation om

um

.3) Find the intersection point u

0

between the line ug,0wm

and the line omwd,0. Similarly, find u

2

from ug,2wm

andomw

d,0.4) Find the intersection point u

1

between the line ug,1wm

and the line omwd,1. Similarly, find u

3

from ug,3wm

andomw

d,1.5) The i-th vertex of Q is ui. ⇤

Note that Algorithm 5 is composed of simple line-line inter-sections rather than geometric constraint solving as in [1].

Algorithm 6 (Perspective-to-Euclidean Vector Transforma-tion). With GCLC defined with known Q and G (as in Fig. 4),we can project an image vector s to a scene vector t. First,we perspectively decompose s along two diagonals of Q:

1) Find the intersection points us,0 between the line u

0

om

and the line um

wd,1. Similarly, find u

s,1 from u1

om

andumw

d,0.2) For each decomposition coefficient si of u

s,i, computethe coefficient ti for vi using Eq. 26.

3) The corresponding scene vector t can be expressed as avector sum of two diagonal vectors, t

0

v0

+ t1

v1

, of Gassuming v

m

= 0. See Fig. 4b. ⇤Algorithm 6 is based on the following property of a general-ized line camera.

um

ug,m

u0 u1us,0

us,1

w0

w1

wd,0

wd,1

v0

vm

vg,m

v1

vt,1

vt,0

G

Gg

Q

Qg

Fig. 7. Perspective-to-Euclidean vector transformation.

d

q0v0v2 vm vt,0vt,2

pc

l0l2

u0

u2

us,0

us,2

m0m2

um

Fig. 8. Scaling transformation in a generalized line camera, which isexplained as a cross ratio between corresponding four points.

Using projective invariance of cross-ratio [8], the followingholds for two sets of collinear points, (v

t,0, v0, vm, v2) and(u

s,0, u0, um, u2), in the scene and images lines, respectively:

sili(li + li+2

)

li(sili + li+2

)

=

timi(mi +mi+2

)

mi(timi +mi+2

)

(25)

where si = kus,i � u

m

k/li and ti = kvt,i � v

m

k/mi. (SeeFig. 8.) By solving Eq.(25) for ti, we get the following relationbetween ti and si:

ti =simi+2

(li + li+2

)

simi+2

li + ((1� si)mi +mi+2

)li+2

(26)

Hence, if a line camera is defined, a scaling factor si of imageline can be mapped to ti of the scene, and vice versa.

IV. EXPERIMENT

We give experimental results on real and synthetic exam-ples. All the experiments were performed in Mathematicaimplementations.

We applied Algorithm 4 to real-world quadrilaterals foundin web images of modern architectures. We assume each imageis independently taken by unknown cameras and not altered(by cropping). Each input quadrilateral Q

g,j is specified inred lines in Fig. 9a and Fig 10a. To infer a centered proxyquadrilateral Qj using Algorithm 5, we find a vanishing lineusing patterns of parallel lines such as window frames [10].Once a set of centered quadrilaterals Qj are found, weestimate unknown diagonal parameters mi that minimize theobjective function f

obj

of Eq.(21). In the experiment, weused NMinimize[] function of Mathematica for non-linearoptimization [9]. With mi known, we can reconstruct thecentered scene quadrilateral Gj which is congruent to thetarget scene quadrilateral G

g,j . See Fig. 9b and Fig 10b. Theresult of reconstructed 3D view frustum is omitted for the pagelimit.

Page 6: New geometric interpretation and analytic solution for quadrilateral reconstruction (ICPR-2014)

#1 #2 #3 #4

(a) Input: web images of Fountain Place in Dallas, Texas.

(b) A reconstructed quadrilateral with different textures of givenimages.

Fig. 9. Reconstruction of a quadrilateral from four views using Algorithm 4.

Input

• Two images from uncalibrated cameras

#1 #2

(a) Input: web images of the Dockland in Hamburg, Germany.

Output

• Reconstructed parallelogram!m1=2.87 (err 2.8%), phi=0.61 (err 0.7%), inc=24. 29 (err 1.2%) using Ref-#2

(b) A reconstructed parallelogram with different textures of givenimages.

Fig. 10. Reconstruction of a parallelogram from two view using Algorithm 4.

For a quadrilateral case of Fig. 9, four images were used.The optimization converges when f

obj

3.7 ⇥ 10

�4 withm

1

= 2.46639,m2

= 0.476389,m3

= 1.25378. The mean� of four �j is 1.77297 with variance 5.9974 ⇥ 10

�5. Theoptimization takes about 3 seconds in 2.6 GHz Intel Core i7.Time for other reconstruction steps is trivial through evaluationof analytic expressions. For a parallelogram of Fig. 10, itconverges when f

obj

10

�30 with m1

= 2.87419 and� = 0.606594 in 0.06 second.

We also applied Algorithm 4 to the synthetic quadrilateralG of Fig. 4 with four different views. The optimization for mi

converges when fobj

< 10

�15 in 3 seconds. The mean errorof reconstructed mi is 1.2⇥10

�7. Timing is similar to the realexample of Fig. 9, but precision is much higher due to absence

of noise sources such as lens distortion or feature detection.When added random noises of 1-pixel radius to vertices ofQj in 1280 ⇥ 1024 image, the precision dropped with errors6.9⇥ 10

�3 and 4.3⇥ 10

�3 in mi and �, respectively.

V. CONCLUSION

We proposed a novel method to reconstruct a scene quadri-lateral and projective structure based on generalized coupledline cameras (GCLC). The method gives an analytic solutionfor a single-view reconstruction when prior knowledge ondiagonal parameters is given. Otherwise, required parameterscan be approximated beforehand from multiple views throughoptimization. We also provide an improved method to handleoff-centered cases by geometrically inferring a centered proxyquadrilateral, which accelerates a 2D reconstruction processwithout relying on homography or calibration. The overallcomputation is quite efficient since each key step is representedas a simple analytic equation. Experiments show a reliableresult on real images from uncalibrated cameras.

To apply the proposed method to a real-world case with anoff-centered quadrilateral, a vanishing line should be availablefor each view. This condition can be easily satisfied in aspecially textured quadrilateral of artifacts [11]. Otherwise,we need other types of prior knowledge to infer a centeredquadrilateral. For example, a predefined parametric polyhedralmodel can be a good candidate [12].

Lastly, coupled line projectors (CLP) [13] is a dual of CLC.We expect that generalized CLP can be combined with GCLCfor a projector-based augmented reality application.

REFERENCES

[1] J.-H. Lee, “Camera calibration from a single image based on coupledline cameras and rectangle constraint,” in ICPR 2012, 2012, pp. 758–762. 1, 3, 4, 5

[2] ——, “A new solution for projective reconstruction based on coupledline cameras,” ETRI Journal, vol. 35, no. 5, pp. 939–942, 2013. 1, 3

[3] Z. Zhang, “Camera calibration with one-dimensional objects,” IEEETrans. on Pattern Analysis and Machine Intelligence, vol. 26, no. 7,pp. 892–899, 2004. 1

[4] J. Yu and L. McMillan, “General linear cameras,” in Computer Vision-ECCV 2004. Springer, 2004, pp. 14–27. 1

[5] P. Sturm and S. Maybank, “On plane-based camera calibration: Ageneral algorithm, singularities, applications,” in CVPR 1999, 1999, pp.432–437. 1

[6] Z. Zhang, “A flexible new technique for camera calibration,” IEEE Trans.on Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330–1334, 2000. 1

[7] Z. Zhang and L.-W. He, “Whiteboard scanning and image enhancement,”Digital Signal Processing, vol. 17, no. 2, pp. 414–432, 2007. 1

[8] R. Hartley and A. Zisserman, Multiple View Geometry in ComputerVision, 2nd ed. Cambridge University Press, 2004. 1, 5

[9] J. A. Nelder and R. Mead, “A simplex method for function minimiza-tion,” The computer journal, vol. 7, no. 4, pp. 308–313, 1965. 4, 5

[10] J.-C. Bazin, Y. Seo, C. Demonceaux, P. Vasseur, K. Ikeuchi, I. Kweon,and M. Pollefeys, “Globally optimal line clustering and vanishing pointestimation in manhattan world,” in CVPR 2012, 2012, pp. 638–645. 5

[11] Z. Zhang, A. Ganesh, X. Liang, and Y. Ma, “Tilt: transform invariantlow-rank textures,” International journal of computer vision, vol. 99,no. 1, pp. 1–24, 2012. 6

[12] P. E. Debevec, C. J. Taylor, and J. Malik, “Modeling and renderingarchitecture from photographs: A hybrid geometry-and image-basedapproach,” in SIGGRAPH 1996. ACM, 1996, pp. 11–20. 6

[13] J.-H. Lee, “An analytic solution to a projector pose estimation problem,”ETRI Journal, vol. 34, no. 6, pp. 978–981, 2012. 6