ismat elhassan - photogrammetric space resection (3) · church’ solution (1945) first iterative...
TRANSCRIPT
www.geospatialworldforum.org
#GWF2020
7-9 April 2020 /// Amsterdam
• Introduction
• Problem Definition
• Importance of Space Resection• Importance of Space Resection
• Photogrammetry versus Computer Vision
• Mathematical Indirect Solutions• Mathematical Indirect Solutions
• Direct Solutions
• Comparison• Comparison
• Conclusions
2
Photogrammetry is defined as a measurement technique Photogrammetry is defined as a measurement technique where the coordinates of the points in 3D of an object are calculated after measurements made on 2D photographic images taken by metric camera.photographic images taken by metric camera.
The position and attitude of the camera (camera exterior The position and attitude of the camera (camera exterior orientation elements) during the exposure is an important factor in determining the required ground coordinates.
The process of determining the position and attitude of the camera is called Space Resection (SR).the camera is called Space Resection (SR).
3
4
The diagram below shows the SR problemThe diagram below shows the SR problem
Importance:The problem is important for both photogrammetryThe problem is important for both photogrammetryand computer vision disciplines.Some of the photogrammetric applications of the space resection are:space resection are:- Fixing ground coordinates by intersection from single photo after solving the space resection problem.problem.- Photo Triangulation, using multiple photos (Bundle Adjustment)- Camera calibration (Tsai, 1987)- Head-Mounted Tracking System Positioning (Azuma and Ward, 1991)and Ward, 1991)- Object Recognition- Ortho photo rectification 6
Importance of SR in Computer VisionImportance of SR in Computer Vision
SR applications in computer vision include::SR applications in computer vision include::
- Head-Mounted Tracking System Positioning (Azuma and Ward, 1991)(Azuma and Ward, 1991)
- Robot picking and Robot navigation - Robot picking and Robot navigation (Linnainmaa et al. 1988)
- Visual surveying in 3D input devices
- Head pose computation - Head pose computation
7
Space Resection: Space Resection: Photogrammetry-Computer Vision
Space resection is therefore dealt with in both Space resection is therefore dealt with in both photogrammetry & computer vision.
Here is a comparison between the two treatments.Here is a comparison between the two treatments.
Difference is shown in terminology and in processing:
Terms:Terms:
Photogrammetry V Computer Vision
Camera exterior orientationparameters
Camera pose estimation
8
Photogrammetry V Computer VisionPhotogrammetry V Computer VisionTerms, coordinates
Space resection Perspective n-point Space resection
problemPerspective n-point
problem
3D Cartesian Homogenous Coordinates
3D Cartesian Spatial coordinates
Of camera persp centerCamera pose angles
CoordinatesCamera line elements
Camera attitiude
9
Camera pose angles Camera attitiude
Photogrammetry V Computer VisionSolution ConceptsSolution Concepts
Non-linear problemNon-linear problem
Iterative SolutionLinear problemDirect Solution
Initial Approximations No need for initialInitial Approximations needed
No need for initialapproximations
Collinearity ConditionCollinearity Condition Closed Form
10
Approximate SolutionsApproximate SolutionsHere are some approximate solutions of the SR problem:problem:1- The Direct Linear Transformation (DLT), which is a method frequently used in photogrammetry and remote method frequently used in photogrammetry and remote sensing.
2- The Church method proposed as a solution for single image resection (Slama, 1980).image resection (Slama, 1980).
3- A simplified absolute orientation method based on object-distances and vertical lines used when no control object-distances and vertical lines used when no control points are available. This method is largely applied in archaeology and architecture by non-photogrammetrists due to its simplicity.due to its simplicity.
11
Approximate SolutionsApproximate Solutions
Continued:Continued:
4- A method of 3D conformal coordinate transformations (Dewitt, 1996) where a special transformations (Dewitt, 1996) where a special formulation of the rotation matrix as a function of the azimuth and tilt is proposed.the azimuth and tilt is proposed.
5- An approximate solution of the spatial transformation (Kraus, 1997) which is particularly transformation (Kraus, 1997) which is particularly suitable when incomplete control points are used.
12
Church’ solution (1945)First iterative solution was published by Church, First iterative solution was published by Church,
1945, 1948.This is done by linearizing collinearity equations,
(Wolf, 1980; Salama, 1980)This is done by linearizing collinearity equations,
(Wolf, 1980; Salama, 1980)It needs a good starting value which constitutes an
approximate solution. approximate solution. Approximate initial values which can be known to
10% accuracy for scale and distances and to within 15o for rotation angles would then be 10% accuracy for scale and distances and to within 15o for rotation angles would then be adjusted by the solution.
See Seedahmed, 2008 for autonomous initial See Seedahmed, 2008 for autonomous initial values for exterior orientation parameters (EOP)
13
Collinearity Condition ZyL
f xaCollinearity Condition
The exposure stationO
x
aTilted photo
plane
f xa
ya
The exposure station
of a photograph (L), an object
point (A) and its photo A
ZL
Y
point (A) and its photo
image (a) all lie along
a straight line (L, a, A).
AZa
Xa
YaXL
YaXL
YL
X
14
Image & Ground Coordinate SystemsImage & Ground Coordinate Systems
• Ground Coordinate System - X, Y, Z z’y’• Ground Coordinate System - X, Y, Z
In Ground Coordinate System
• Exposure Station Coordinates L( XL, YL, ZL)
Z x’
y’
L
za’
ya’
Xa’
• Object Point (A) Coordinates A( Xa, Ya, Za)
• Image coordinate system (x’, y’, z’ ) parallel to
• ground coordinate system (XYZ) ZL
X
• ground coordinate system (XYZ)
In image Coordinate System
image point (a) coordinates a(xa’, ya’, za’)
Y
A
Za
Xa
ZL
xa’ , ya’ and za’ are related to the measured
photo coordinates xa, ya, focal length (f) and the
three rotation angles omega, phi and kappa.
Ya
YL
XL
15
X
YL
Developed in a sequence of three independent two-dimensional rotations.
• rotation about x’ axisx1 = x’1
y1 = y’Cos + z’Sin
z1 = -y’Sin + z’Cos
• f rotation about y’ axis• f rotation about y’ axisx2 = -z1Sinf + x1Cosf
y2 = y1
z2 = z1Cosf + x1Sinfz2 = z1Cosf + x1Sinf
• rotation about z’ axisx = x2Cos + y2Sin
y = -x2Sin + y2Cosy = -x2Sin + y2Cos
z = z2
16
x = x’(Cosf Cos ) + y’(Sin Sinf Cos + Cos Sin ) + z’(-Cos Sinf Cos + Sin Sin )y = x’(-Cosf Sin ) + y’(-Sin Sinf Sin + Cos Cos ) + z’(Cos Sinf Sin + Sin Cos )z = x’(Sinf ) + y’(-Sin Cosf ) + z’(Cos Cosf )
X = MX’
Rotation Matrixx = m11x’ + m12y’ + m13z’y = m21x’ + m22y’ + m23z’z = m x’ + m y’ + m z’
'131211 xmmmx
The sum of the squares of the three “direction cosines” in any row or in any
z = m31x’ + m32y’ + m33z’'
'
333231
232221
z
y
mmm
mmm
z
y
The sum of the squares of the three “direction cosines” in any row or in any column is unity.
M -1 = MT
X’ = MTX
17
Collinearity Condition EquationsCollinearity Condition Equationsfrom Similar Triangles
Collinearity condition equations developed from similar triangles (Wolf)
AL
a
LA
a
LA
a
ZZ
z
YY
y
XX
x ''' x = m11x’ + m12y’ + m13z’y = m21x’ + m22y’ + m23z’
* Dividing xa and ya by za
ALLALA ZZYYXX y = m21x’ + m22y’ + m23z’z = m31x’ + m32y’ + m33z’
aLA
LAa
LA
LAa
LA
LAa z
ZZ
ZZmz
ZZ
YYmz
ZZ
XXmx ''' 131211
* Dividing xa and ya by za
* Substitute –f for za
* Correcting the offset of Principal
point (xo, yo)aLA
aLA
aLA
a
aLA
LAa
LA
LAa
LA
LAa
zZZ
mzYY
mzXX
mz
zZZ
ZZmz
ZZ
YYmz
ZZ
XXmy
'''
'''
333231
232221
point (xo, yo)aLA
aLA
aLA
a zZZ
mzZZ
mzZZ
mz ''' 333231
)()()(
)()()(
)()()(
232221
333231
131211
LALALA
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXm
ZZmYYmXXmfxx
18
)()()(
)()()(
333231
232221
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXmfyy
Collinearity Condition equations from Vector Collinearity Condition equations from Vector Triangle
I prefer using this approach when teaching to allow students to understand the principle:
XA = XO + xa
XA = 3D object space coordinatesXA = 3D object space coordinates
XO = exposure station space coordinates
= scale= scale
xa=image coordinates
• Nonlinear
• Nine unknowns )()()(
)()()(
333231
131211
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXmfxx
• , f ,
• XA, YA and ZA
• X , Y and Z
)()()(
)()()(
333231
232221
LALALA
LALALAoa
ZZmYYmXXm
ZZmYYmXXmfyy
• XL, YL and ZL
Taylor’s Theorem is used to linearize the nonlinear equations equations
substituting
0
)(
)(!
)(
n
nn
axn
af)()()(
)()()(
131211
333231
LALALA
LALALA
ZZmYYmXXmr
ZZmYYmXXmq
20
0 !n n)()()(
)()()(
232221
131211
LALALA
LALALA
ZZmYYmXXms
ZZmYYmXXmr
Rewriting the Collinearity EquationsRewriting the Collinearity Equations
ao xq
rfxF
LL
LL
LL
o dZZ
FdY
Y
FdX
X
Fd
Fd
Fd
FF
000000
ao
ao
yq
sfyG
xq
fxF
LL
LL
LL
o
aAA
AA
AA
dZZ
GdY
Y
GdX
X
Gd
Gd
Gd
GG
xdZZ
FdY
Y
FdX
X
F
ZYX
000000
000
000000
Taylor’s Theorem
• F0 and G0 are functions F and G evaluated at the initial approximations for the nine unknowns
qaA
AA
AA
AydZ
Z
GdY
Y
GdX
X
G
000
0
approximations for the nine unknowns• d , df , d are the unknown corrections to be applied to the
initial approximations • The rest of the terms are the partial derivatives of F and G wrt • The rest of the terms are the partial derivatives of F and G wrt
to their respective unknowns at the initial approximations
21
• Residual terms must be included in order to make the equations consistent
VJdZbdYbdXbdZbdYbdXbdbdbdb
J = xa – Fo ; K = ya – Go
b terms are coefficients equal to the partial derivatives
yaAAALLL
xaAAALLL
VKdZbdYbdXbdZbdYbdXbdbdbdb
VJdZbdYbdXbdZbdYbdXbdbdbdb
262524262524232221
161514161514131211
b terms are coefficients equal to the partial derivatives
Numerical values for these coefficient terms are obtained by using initial approximations for the unknowns.
The terms must be solved iteratively (computed corrections are added to the initial approximations to obtain revised approximations) until the magnitudes of corrections to initial approximations become negligible.
22
negligible.
Formulate the collinearity equations for a number of control points whose X, Y and Z ground coordinates are known and whose images appear in the tilted photo. The equations are then solved for the six unknown elements of exterior orientation which appear in them.exterior orientation which appear in them.
Space Resection collinearity equations for a point A
xaLLL VJdZbdYbdXbdbdbdb 161514131211
A two dimensional conformal coordinate transformation is used
X = ax’ – by’ + T
yaLLL VKdZbdYbdXbdbdbdb 262524232221
X = ax’ – by’ + Tx X, Y – ground control coordinates for the point
Y = ay’ + bx’ + Ty x’, y’ – ground coordinates from a vertical photograph
a, b, Tx, Ty – transformation parameters
23
This method does not require fiducial marks and can be solved without supplying initial approximations for the parameters
Collinearity equations along with the correction for lens distortion
dx, dy – lens distortion
fx – pd in the x direction
fy – pd in the y direction)()()(
)()()(
)()()(
)()()(
2322210
333231
1312110
LALALAyya
LALALA
LALALAxxa
ZZmYYmXXm
ZZmYYmXXmfxy
ZZmYYmXXm
ZZmYYmXXmfxx
fy – pd in the y direction
Rearranging the above two equations
)()()( 3332310
LALALAyya ZZmYYmXXm
fxy
LZLYLXLLmfmxL
LmfmxL
LmfmxL
x
x
x
/)(
/)(
/)(
133303
123202
113101
1
1
8765
11109
4321
ZLYLXL
LZLYLXLy
ZLYLXL
LZLYLXLx
ya
xa
LmfmyL
LmfmyL
LmfmyL
LZmYmXmfxL
LmfmxL
y
y
LLLx
x
/)(
/)(
/)(
/)(
/)(
223206
213105
13121104
133303
24
111109 ZLYLXLyaLmfmyL y /)( 233307
The resulting equations are solved iteratively using LSM
/
/
/)(
3210
319
23222108 LLLy
LmL
LmL
LZmYmXmfyL
4
1
321 LLLLX L
)(
/
/
333231
3311
3210
LLL ZmYmXmL
LmL
LmL
18
4
11109
765
321
L
L
LLL
LLL
LLL
Z
Y
X
L
L
L
Advantages- No initial approximations are required
for the unknowns.for the unknowns.Limitations- Requirement of atleast six 3D object space control points- Lower accuracy of the solution as compared with a rigorous bundle- Lower accuracy of the solution as compared with a rigorous bundle
adjustment
25
Direct SolutionsDirect Solutions
Since the indirect solution requires initial approximate values for the exterior orientation parameters, and in computer vision problems the parameters, and in computer vision problems the approximate values are not known, a direct solution is a must. solution is a must.
Six approaches of direct solution were presented and tested by Haralick, et al, 1994.and tested by Haralick, et al, 1994.
26
Direct Retrieval of EOP using 2D Projective Direct Retrieval of EOP using 2D Projective Transformation
Seedahmed (2006) presented a direct algorithm to retrieve the EOPs from the 2D projective transformation, based on a direct relationship between transformation, based on a direct relationship between the 2D projective transformation and the collinearity model using a normalization process. This leads to a model using a normalization process. This leads to a direct matrix correspondence between the 2D projection parameters and the collinearity model parameters: hence, a direct matrix factorization to parameters: hence, a direct matrix factorization to retrieve the EOPs.
27
(no linearization, iterations or initial values)(no linearization, iterations or initial values)
Space resection in photogrammetry using Space resection in photogrammetry using collinearity condition without linearisation Said Easa (2010) presented an optimization model for space resection with or without redundancy that space resection with or without redundancy that requires no linearization, iterations, or initial approximate values. approximate values. The model, which is nonlinear and noncovex, is solved using advanced Excel-based optimization software that has been recently developed. that has been recently developed. The proposed model is simple and converges to the global optimal solution very quickly. global optimal solution very quickly.
28
Rodrigues Matrix Solution(no initial values needed)(no initial values needed)
The classical iterative method based on Euler angle using The classical iterative method based on Euler angle using collinearity equations usually fail because the initial values are poor or unknown. are poor or unknown.
For this reason, H. Zeng, (2010) used Rodrigues matrix to represent the rotation matrix, and then established the mathematical model of space resection based on Rodrigues mathematical model of space resection based on Rodrigues matrix, finally, he presented the solution of the model. This algorithm need linearization and iterative process, but has algorithm need linearization and iterative process, but has no initial values problem, regardless of the size of the Euler angle, and is fast and effective
29
Newton-Raphson Search SolutionNewton-Raphson Search Solution(no initial values)
Said Easa (2007) described the geometry of the 3point resection and the difficulties with N-R method.
He presented a new Excel-based method that He presented a new Excel-based method that identifies the four solutions of the quartic polynomial.
The method does not need initial estimates of the The method does not need initial estimates of the roots.
30
Explicit Solution with LSA to Redundant ControlExplicit Solution with LSA to Redundant Control
Smith 2006 suggested discrimination of the four Smith 2006 suggested discrimination of the four resulting solutions by using redundant control and applying least squares adjustment (LSA) for the general case where photography is not nearly vertical.general case where photography is not nearly vertical.
All four solutions will be examined in the light of the All four solutions will be examined in the light of the available redundant control.
The disadvantage of the method is the need for redundant control.redundant control.
Non-Rigid Space Resectionby Parameterized Modelsby Parameterized Models
Wang et al proposed a model based approach to space resection. The method recovers a camera’s location and resection. The method recovers a camera’s location and orientation relative to an object coordinate system up to a scale factor without the use of any GCP or vanishing point.
The mathematical basis for this approach is the equivalence between the vector normal to the interpretation equivalence between the vector normal to the interpretation plane in the image space and the vector normal to the rotated interpretation plane in the object space.
A two-step iterative scheme for recovering camera orientation that, unlike existing methods, does not require orientation that, unlike existing methods, does not require a good initial guess for the rotation.
32
Non-Rigid Approach, continuedNon-Rigid Approach, continuedInstead, the good initial estimate for the rotation is Instead, the good initial estimate for the rotation is computed directly by using coplanarity constraints. A non-linear least squares minimization procedure is then applied to determine camera orientation accurately.to determine camera orientation accurately.
The camera translation and predefined model parameters The camera translation and predefined model parameters are determined based on the calculated rotation through a linear least squares minimization.
Unlike existing methods, this method does not require a model-to-image fitting process, and is more effective and model-to-image fitting process, and is more effective and faster than previous approaches.
33
Non-Rigid Solution
),,(R
Non-Rigid SolutionObject line 6-7 & image line 6-7
zCamera Coordinate
Systemy x
CImage edge 67{(x1, y1, -f), (x2, y2, -f)}
Imageplane
Z
1 1 2 2f)}
t (X0, Y0,Z0 )
56
Y
Model edge 67(v, u)
Object Coordinate System
2 1
3 4
8
567
XO
34
Object Coordinate System
Straight lines intersection approachLagunes & Battle (2009) proposed a technique based on determining the coordinates of the exposure station by intersection of the straight lines through the station by intersection of the straight lines through the three known ground control points.
The required azimuths of these lines are obtained The required azimuths of these lines are obtained from the geometric relationships between two similar triangles.
Numerical solutions that show the good performance and accuracy of the solution were reported.
Recursive Straight Lines Solution Recursive Straight Lines Solution
Tomaselli and Tozzi (1996) developed a space resection Tomaselli and Tozzi (1996) developed a space resection solution using an explicit math model relating straight lines as features, applying Kalman Filtering.
An iterative process using sequential estimated camera An iterative process using sequential estimated camera location parameters to feed back to the feature extraction leading to a gradual reduction of image extraction leading to a gradual reduction of image space fro feature searching.
Results show highly accurate space resection Results show highly accurate space resection parameters are obtained as well as a progressive processing time reduction.
36
Genetic Evolution SolutionUrban and Stroner (2012), Elnima (2013) suggested an optimizaation model based on genetic evolution algorithm to solve the three point space resection algorithm to solve the three point space resection problem
The solution does not need linearization nor The solution does not need linearization nor redundancy.
The proposed model is simple and converges to the The proposed model is simple and converges to the global optimal solution.
The disadvantage is the high computational demand.
38
Automobile Construction
Machine Construction, Metalworking, Quality Control
Mining EngineeringMining Engineering
Objects in Motion
ShipbuildingShipbuilding
Structures and Buildings
Traffic EngineeringTraffic Engineering
Biostereometrics
39
Collinearity condition and collinearity equations are Collinearity condition and collinearity equations are more suitable for photogrammetric students who want to go deep in sight of the problem, considering single photo.photo.
Iteration solution needing initial approximations of parameters give good accuracy suitable for surveying parameters give good accuracy suitable for surveying needs. It makes no problem with the use of computers.computers.
Initial approximations can now be easily obtained either using simple DLT or when GPS is used.
40
41