direct linear transformation & computer vision models · ce 59700: digital photogrammetric...

CE 59700: Digital Photogrammetric Systems Ayman F. Habib1

Direct Linear Transformation & Computer Vision Models

Chapter 7-A4


Photogrammetry Vs. Computer Vision• Conventional Photogrammetry is focusing on precise

geometric information extraction from imagery.– Topographic mapping from space borne and airborne imagery– Metrological information extraction through close-range

photogrammetry (terrestrial photogrammetry)• Object-to-camera distance is less than 100meter

• Computer Vision (CV) is mainly concerned with automated image understanding:– Object recognition,– Navigation and obstacle avoidance, and– Object modeling


Airborne Photogrammetric Mapping


Close-Range Photogrammetric Mapping


CV: Object Recognition


CV: Navigation & Obstacle Avoidance


Photogrammetry Vs. Computer Vision• Photogrammetry is always concerned with precise

geometric information extraction.– Photogrammetric mapping considers potential deviations from

the assumed perspective projection.

• For Computer Vision (CV):– Focus is always on automation.– Object recognition and navigation applications do not require

precise derivation of geometric information.– Depending on the application, object modeling might require

precise geometric information extraction.– CV usually assumes that the collinearity of the object point,

perspective center, and corresponding image point is maintained, even for un-calibrated cameras.


Object-to-Image Coordinate Transformation in Photogrammetry

Collinearity Equations


o

a

A

oa = oA

These vectors should be defined w.r.t.the same coordinate system.



Oi

xc

yczc

A

XA

YA

ZA

a+

(xa, ya)

R( , , )

XG

YG

ZG

OG

pp+

c

(Perspective Center)

XO

ZO

YO




The vector connecting the perspective center to the image point

w.r.t. the image coordinate system

o

a

c

distyydistxx

cyx

distydistx

rv ypa

xpa

p

p

ya

xac

oai

0



The vector connecting the perspective center to the object point

w.r.t. the ground coordinate system

o

A

oA

oA

oA

o

o

o

A

A

Am

oAo

ZZYYXX

ZYX

ZYX

rV


Where: is a scale factor (+ve).


11 12 13

21 22 23

31 32 33

( , , )c c mi oa O m oA

a p x A o

a p y A o

A o

v r M V R rx x dist m m m X Xy y dist m m m Y Y

c m m m Z Z

oAoa


Collinearity EquationscmRM

yoAoAoA

oAoAoApa

xoAoAoA

oAoAoApa

distZZmYYmXXmZZmYYmXXmcyy

distZZmYYmXXmZZmYYmXXmcxx

)()()()()()()()()()()()(

333231

232221

333231

131211

mcRR

yoAoAoA

oAoAoApa

xoAoAoA

oAoAoApa

distZZrYYrXXrZZrYYrXXrcyy

distZZrYYrXXrZZrYYrXXrcxx

)()()()()()()()()()()()(

332313

322212

332313

312111


Object-to-Image Coordinate Transformation

Direct Linear TransformationComputer Vision Model


DLT & Computer Vision Models• The DLT and computer vision models encompass:

– Collinearity Equations,– Non-orthogonality () between the axes of the image/camera

coordinate system, and– Two scale factors (Sx, Sy) along the axes of the image

coordinate system.

• DLT & CV models can directly deal with pixel coordinates.

• We will start with modifying the rotation matrix to consider the impact of the non-orthogonality ().– Primary rotation @ the -axis of the ground coord. system– Secondary rotation @ the -axis– Tertiary rotation & @ the -axis


Primary Rotation ()

X

ZY

& X

YZ


Primary Rotation ()

zyx

Rzyx

zyx

zyx

cossin0sincos0001


Secondary Rotation ()

Y

Z

X

X

Z

& Y


Secondary Rotation ()

zyx

Rzyx

zyx

zyx

cos0sin010

sin0cos


Tertiary Rotation ()

X

Z

YY

X

& Z


Tertiary Rotation ()

zyx

Rzyx

zyx

zyx

1000cossin0sincos


Rotation in Space

zyx

RRRzyx

// to the ground coordinate system // to the image coordinate system


Rotation in Space

coscossinsincoscossin

cossincossinsincossin

sinsinsincoscoscossinsinsincos

sinsincos

coscos:

33

32

31

23

22

21

13

12

11

333231

232221

131211

rrrrrrrrrwhere

rrrrrrrrr

RRRR


X

Z

YY

X

& Z

Consideration of the Non-Orthogonality ()

ZYX

ZYX

1000)cos(sin0)sin(cos


ZYX

ZYX

1000)cos(sin0)sin(cos

ZYX

ZYX

1000sincossin0cossincos


sincossinsincoscos)cos(cossinsincoscossin)sin(

Assuming small non-orthogonality angle ()



10001001

1000cossin0sincos

1000sincossin0cossincos

ZYX

RZYX

RRRZYX

10001001

10001001



ZYX

RZYX

zyx

T

10001001

// to the image coordinate system // to the ground coordinate system

ZYX

RZYX

10001001

Note: 1 00 1 00 0 1

1 00 1 00 0 1


Consideration of the Non-Orthogonality ()• Collinearity Equations while considering the non-

orthogonality () between the axes of the image coordinate system.

O

O

OT

p

p

ZZYYXX

Rcyyxx

10001001


O

O

OT

yp

xp

ZZYYXX

Rc

syysxx

10001001

/)(/)(

O

O

OT

yp

xp

ZZYYXX

Rccsyycsxx

10001001

/1

)/()()/()(

• Divide both sides by (-c).

Consideration of the Scale Factors• Collinearity Equations while considering the non-

orthogonality () between the axes of the image coordinate system & different scale factors.


Consideration of the Scale Factors

O

O

OT

yp

xp

ZZYYXX

Rcyycxx

10001001

1)/()()/()(

O

O

OT

p

p

y

x

ZZYYXX

Ryyxx

cc

10001001

1)()(

1000/1000/1

O

O

OT

y

x

p

p

ZZYYXX

Rcc

yyxx

10001001

1000000

1)()(

• csx→ cx , csy→ cy & -/c → `.


DLT & Computer Vision Models

O

O

OT

y

xx

p

p

ZZYYXX

Rccc

yyxx

100000

1)()(

O

O

O

y

xx

p

p

ZZYYXX

rrrrrrrrr

ccc

yyxx

332313

322212

312111

100000

1)()(

&



′

′

′

′



′

Where:



3

3

'1

1

0 { }0 0 1

{ }

TO

x x p

y p

TO

Xx

Yy K R I X

Z

c c xK c y Calibration Matrix

R I X Exterior Orientation Matrix


DLT & Computer Vision Models• The Direct Linear Transformation (DLT), which has

been developed by the photogrammetric community, is an alternative to the collinearity equations that allows for direct transformation between machine/pixel coordinates and corresponding ground coordinates.– &

• The DLT can be also represented by the following form:

–1 1



1 2 3

5 6 7

9 10 11

' 00 0 1

x x pT

y p

L L L c c xD L L L c y R

L L L

4

8

12

' 00 0 1

x x p OT

y p O

O

L c c x XL c y R YL Z

DLT: Direct Linear Transformation


DLT & CV Models: Pixel Coordinates• The DLT & CV models can also consider the direct

transformation from pixel to ground coordinates.

⁄ _ _⁄ _ _

u

v


DLT & CV Models: Pixel Coordinates⁄ _ _

⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _


DLT & CV Models: Pixel Coordinates_ _ ⁄ _ _

_ _ ⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _

_ _ ⁄ _ __ _ ⁄ _ _

_ _⁄ ⁄_ _⁄ ⁄

_ _⁄ ⁄_ _⁄ ⁄


DLT & CV Models: Pixel Coordinates• Modified Calibration Matrix:

_ _⁄ ⁄_ _⁄ ⁄

_ _⁄ _ _⁄ _ _⁄ ⁄0 _ _⁄ _ _⁄ ⁄0 0 1


DLT & CV Models: Pixel Coordinates• For DLT when working with pixel coordinates, we have

the following model.

–

•

•


Modern Photogrammetry & Computer Vision• Modern Photogrammetry and Computer Vision are

converging fields.

Art and science of tool development for automatic generation of spatial and descriptive information from

multi-sensory data and/or systems


DLT → IOPs & EOPs

Approach # 1


DLT → IOP & EOP

1 2 3

5 6 7

9 10 11

00 0 1

x x pT

y p

L L L c c xD L L L c y R

L L L

4

8

12

00 0 1

x x p OT

y p O

O

L c c x XL c y R YL Z


DLT → IOP & EOP

No Sign Ambiguity

O

O

O

ZYX

LLLLLLLLL

LLL

11109

765

321

12

8

4

• Given:

• Then:

12

8

41

11109

765

321

LLL

LLLLLLLLL

ZYX

O

O

O


DLT → IOP & EOP

1000

10002

pp

yx

x

py

pxxT

yxcc

cycxcc

DD

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT

2211

210

2933)( LLLDD T

}{211

210

29 AmbiguitySignLLL

Then:


DLT → IOP & EOP

pT xLLLLLLDD 2

3112101913)(

)()(

211

210

29

31121019LLL

LLLLLLxp

No Sign Ambiguity

Then:

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT


pT yLLLLLLDD 2

7116105923)(

DLT → IOP & EOP

)()(

211

210

29

71161059LLL

LLLLLLyp

No Sign Ambiguity

Then:

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT


)()( 22227

26

2522 yp

T cyLLLDD

DLT → IOP & EOP

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT

5.02

211

210

29

27

26

25

)(

py yLLL

LLLc

No Sign Ambiguity

Then:


DLT → IOP & EOP

)()( 273625121 ppyx

T yxccLLLLLLDD

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT

ppyx yxLLL

LLLLLLcc )(/1 211

210

29

736251

No Sign Ambiguity

Then:


)()( 2222223

22

2111 pxx

T xccLLLDD

DLT → IOP & EOP

1173

1062

951

11109

765

3212)()(

LLLLLLLLL

LLLLLLLLL

KKRKRKDD TTTTT

5.0222

211

210

29

23

22

21

)(

pxx xcLLL

LLLc

No Sign Ambiguity

Then:


DLT → IOP & EOP• Given:

• Then: sin139 rL

Sign Ambiguity

332313

322212

312111

11109

765

321

1000

rrrrrrrrr

ycxcc

LLLLLLLLL

py

pxx


Collinearity Equations• Objective: Resolve the sign ambiguity in

• Since the scale factor is always +ve

• Assuming that the origin (0, 0, 0) is visible in the imagery

veZZrYYrXXr OOO )()()( 332313

veZrYrXr OOO 332313

)()()()()()()()()(

332313

322212

312111

OOO

OOO

OOO

p

p

ZZrYYrXXrZZrYYrXXrZZrYYrXXr

Scyyxx


DLT → IOP & EOP

• By choosing L12 = 1.

12 13 23 33

13 23 33

13 23 33

( )1 ( )

1( )

O O O

O O O

O O O

L r X r Y r Zr X r Y r Z

r X r Y r Z

is Negative

211

210

29 LLL

O

O

OT

py

pxx

ZYX

Rycxcc

LLL

1000

12

8

4


DLT → IOP & EOP

• No sign Ambiguity

211

210

29

9

139

sin

sin

LLLLrL


DLT → IOP & EOP

coscoscossin

3311

2310

rLrL

11

10tan LL

No Sign Ambiguity

332313

322212

312111

11109

765

321

1000

rrrrrrrrr

ycxcc

LLLLLLLLL

py

pxx


DLT → IOP & EOP

1000

1000

1

11109

765

321

333231

232221

131211

332313

322212

312111

11109

765

321

py

pxx

py

pxx

ycxcc

LLLLLLLLL

rrrrrrrrr

rrrrrrrrr

ycxcc

LLLLLLLLL

• Retrieve

• Note: There is an ambiguity in determination ( cannot be distinguished).

coscos 11r


DLT → IOP & EOP

Approach # 2: Matrix Factorization


DLT → IOP (Factorization # 1)• Conceptual basis: Direct derivation of the calibration matrix

• Cholesky Decomposition of DDT→ K (Calibration Matrix)? Wrong

1000

1000

)()(

2

1173

1062

951

11109

765

3212

pp

yx

x

py

pxxT

TTTTT

yxcc

cycxcc

DD

LLLLLLLLL

LLLLLLLLL

KKRKRKDD


1

1)(

NMMMNCHO

T TMM

TDDN TKK

TT

T

KKMMN

MMN21

1

1

11)]}({[ TDDCHOK

TKK

DLT → IOP (Factorization # 2)


1000

1000

1

11109

765

321

333231

232221

131211

332313

322212

312111

11109

765

321

py

pxx

py

pxx

ycxcc

LLLLLLLLL

rrrrrrrrr

rrrrrrrrr

ycxcc

LLLLLLLLL

• Using the rotation matrix R, one can derive the individual rotation angles , and .

DLT → Rotation Angles


Analysis


Perspective Center

1211109

8765

4321

12109

765

321

12

8

4

LZLYLXLLZLYLXLLZLYLXL

ZYX

LLLLLLLLL

LLL

OOO

OOO

OOO

O

O

O

• (XO, YO , ZO) is the intersection point of three different planes whose surface normals are (L1, L2, L3), (L5, L6, L7) and (L9, L10, L11), respectively.


Perspective Center

332313

322212

312111

11109

765

321

1000

rrrrrrrrr

ycxcc

LLLLLLLLL

py

pxx

• Assuming:– xp ≈ 0.0 and yp ≈ 0.0– -cx ≈ 0.0

332313

322212

312111

11109

765

321

rrrrcrcrcrcrcrc

LLLLLLLLL

yyy

xxx

• The three surfaces are orthogonal to each other.– This would lead to better intersection.


332313

322212

312111

11109

765

321

1000

rrrrrrrrr

ycxcc

LLLLLLLLL

py

pxx

• Assuming:– xp ≠ 0.0 and yp ≠ 0.0– -cx ≈ 0.0

332313

333223221312

333123211311

11109

765

321

rrrryrcryrcryrcrxrcrxrcrxrc

LLLLLLLLL

pypypy

pxpxpx

• As xp and yp increase, the surface normals become almost parallel.– This would lead to weak intersection.

Perspective Center


• The rows of D are not correlated:– They are orthogonal to each other.

• L-1 is well defined.

1000

1

11109

765

321

333231

232221

131211

py

pxx

ycxcc

LLLLLLLLL

rrrrrrrrr

• Assuming:– xp ≈ 0.0 and yp ≈ 0.0– -cx ≈ 0.0

332313

322212

312111

11109

765

321

rrrrcrcrcrcrcrc

LLLLLLLLL

yyy

xxx

Rotation Angles


1000

1

11109

765

321

333231

232221

131211

py

pxx

ycxcc

LLLLLLLLL

rrrrrrrrr

• Assuming:– xp ≠ 0.0 and yp ≠ 0.0– -cx ≈ 0.0

332313

333223221312

333123211311

11109

765

321

rrrryrcryrcryrcrxrcrxrcrxrc

LLLLLLLLL

pypypy

pxpxpx

• The rows of D tend to be highly correlated.• L-1 is not well defined.

Rotation Angles

direct linear transformation & computer vision models · ce 59700: digital photogrammetric...

Documents