the singular value decomposition and applications in geodesy

7272019 The Singular Value Decomposition and Applications in Geodesy

httpslidepdfcomreaderfullthe-singular-value-decomposition-and-applications-in-geodesy 13Student professional magazine bull Faculty of Geodesy bull University of Zagreb

mr sc Vida Zadelj Martić

fakultet

The singular value decomposiTion

and applicaTions in geodesy

A983138983155983156983154983137983139983156

T983144983141 983152983137983152983141983154 983139983151983150983155983145983140983141983154983155 983156983144983141 983155983145983150983143983157983148983137983154 983158983137983148983157983141 983140983141983139983151983149983152983151983155983145983156983145983151983150 (SVD) 983151983142 983137 983143983141983150983141983154983137983148 983149983137983156983154983145983160 S983151983149983141 983145983149983149983141983140983145983137983156983141 983137983152983152983148983145983139983137-

983156983145983151983150983155 983155983157983139983144 983137983155 983140983141983156983141983154983149983145983150983145983150983143 983156983144983141 983155983152983141983139983156983154983137983148 983137983150983140 F983154983151983138983141983150983145983157983155 983150983151983154983149 983154983137983150983147 983137983150983140 983152983155983141983157983140983151983145983150983158983141983154983155983141 983151983142 983156983144983141 983149983137983156983154983145983160 983137983154983141

983140983141983155983139983154983145983138983141983140 A983152983152983148983145983139983137983156983145983151983150983155 983137983148983155983151 983145983150983139983148983157983140983141 983137983152983152983154983151983160983145983149983137983156983145983150983143 983156983144983141 983143983145983158983141983150 983149983137983156983154983145983160 983138983161 983137 983149983137983156983154983145983160 983151983142 983137 983148983151983159983141983154 983154983137983150983147 I983156 983145983155 983137983148983155983151

983155983144983151983159983150 983144983151983159 983156983151 983157983155983141 SVD 983142983151983154 983155983151983148983158983145983150983143 983156983144983141 983144983151983149983151983143983141983150983141983151983157983155 983148983145983150983141983137983154 983155983161983155983156983141983149 983137983150983140 983156983144983141 983148983141983137983155983156 983155983153983157983137983154983141983155 983152983154983151983138983148983141983149 T983144983141

983152983137983152983141983154 983139983151983150983155983145983155983156983155 983151983142 983156983144983154983141983141 983152983137983154983156983155

983089) T983144983141 983155983145983150983143983157983148983137983154 983158983137983148983157983141 983140983141983139983151983149983152983151983155983145983156983145983151983150

983090) S983151983149983141 983137983152983152983148983145983139983137983156983145983151983150983155 983151983142 983156983144983141 983155983145983150983143983157983148983137983154 983158983137983148983157983141 983140983141983139983151983149983152983151983155983145983156983145983151983150

983091) A983152983152983148983145983139983137983156983145983151983150983155 983145983150 983143983141983151983140983141983155983161

K983141983161983159983151983154983140983155

Singular value decomposition

Unitary matrices

Frobenius norm

Spectral norm

Pseudoinverse

Homogeneous system of

linear equations

Least squares problemRank

1 THE SINGULAR VALUE

DECOMPOSITION

11 BASIC NOTIONS AND NOTATION

By m times n ( m times n) is denoted the set of m times n

complex (real) matrices

Let Aisin m times n Two most common vector subs-

paces associated with A are the range of A

real( A) = Ax xisin nsube m

and the kernel (null-subspace) of A

N ( A) = x Ax = 0 sube n

e range of A is spanned by the columns of A

so it is sometimes called the column space of A If

A then real( A) and N ( A) are appropriately defined

using the vector spaces m and n

e dimensions of real( A) and N ( A) are the rank

and the defect of A

e spectral radius of a square matrix A

spr(A) = max| λ| λisinσ( A)

is the largest distance of an eigenvalue of A to

zero Here σ( A) is the spectrum of A which is the

set of eigenvalues of A

e trace of A is the sum of its diagonal elements

It can be shown that tr ( A) is the sum of the ei-

genvalues of A

1

( )n

kk

k

tr A a

=

= sum

e diagonal part of A=(ai j

) is the diagonal matrix

If A is not square then diag ( A) has additional zero rows or co-

lumns

e spectral and the Frobenius norm of an m times n matrix are defined

by

and

respectively Here A is the hermitian transpose of A If A is real then

A = Aτ is just the transpose of A

e matrix A is normal if A A = AA e complex (real) matrix U is

unitary (orthogonal) if U U = UU = I

12 THE SINGULAR VALUE DECOMPOSITION

e following theorem serves as the definition of the singular value

decomposition

Teorem 1

Let Aisin m times n en there exist unitary matrices U isin m times m and

V isin n times n

so that

U AV = Σ = diag (σ1 σ

2 σ

minmn )

11

11( )

mn

nm

a

diag a

a

a =

2( ) A spr A A=

( )2

i j F i j

A tr A A a= = sum

Zadelj Marć V (2010) The singular value decomposion and applicaons in geodesy

Ekscentar iss 13 pp 60-62

Institute of Geomatics Faculty of Geodesy University of Zagreb Kačićeva 26 10000 Zagreb vzadeljgeofhr



where σ1

ge σ2

ge ge σminmn

ge 0 e nonnegative numbers

σ1 σ

2 σ

minmn are the singular values of A and the columns of U (V )

are the left (right) singular vectors of A

Matrix Σ is uniquely determined by the matrix A However the ma-

trices U and V are not unique If the singular values are multiple then

U and V can be post-multiplied by an arbitrary block-diagonal unitary

matrix whose diagonal blocks are of appropriate dimensions

e singular value decomposition has hundreds of applications It

is an excellent tool in matrix theory It is often used for solving different

matrix problems which arise in science economy engineering medicine

and even in human sciences Data mining web searching image reco-

gnition just to mention some contemporary problems which use SVD

often of large matrices Its widespread use is enhanced by the fact that

there exist excellent efficient and accurate methods for computing it

ere are several classes of methods the most important are one-sided

Jacobi methods divide and conquer (DC) differential qd (DQD) and QR

methods

We will first list several immediate consequences of this decompo-

sition

Let A = U ΣV be the singular value decomposition of the matrix Aisin m times n so that

σ1 ge σ

2 ge ge σ

r gt σ

r+1 = = σ

p = 0 p = minmn

Let U = [u1 u

m] V = [v

1 v

m] be the column representations

(often called partitions) of U and V respectively

en it is easy to show that the following holds

(i) rank ( A) = r

(ii) N ( A) = spanvr+1vn

(iii) real( A) = spanu1u

r

(iv)

1

r

i i i r r r

I

A u v U V σ =

= = Σsum such that

U r = [u

1 u

r] isin m times r

V r = [v

1 v

r] isin n times r

Σr = diag (σ

1 σ

r ) isin r times r

(v) || A ||F

2 = σ12 + + σ

r 2

(vi) || A ||F = σ

1

It can also be shown that the distance (in spectral norm) of a square

matrix A and the set of singular matrices of the same dimension is equal

to the smallest singular value of Ais follows from a more general re-

sult the Ekhard Young and Mirsky theorem

Teorem 2

Let Aisin m times n and let A = U ΣV be the singular value decomposition

of A so that

σ1 ge σ2 ge ge σr gt σr+1 = = σ p = 0 p = minmn

holds If k lt r = rank ( A) and

1

k

k i i i

i

C u vσ =

= sum then

12 2( )

min k k rank K k

A K C C σ +=

minus = minus =

us the closest rank k approximation of A is obtained by using the

first k singular values and vectors of A

2 SOME APPLICATIONS OF THE SINGULAR VALUE

DECOMPOSITION

Here we will address several applications

21 DETERMINING THE PSEUDOINVERSE OF A GENERAL

MATRIX

Here we will consider the Moore-Penrose pseudoinverse

Let Aisin m times n e matrix X isin n times m is pseudoinverse of Aisin m times n if the

following four conditions are fulfilled

(i) AXA = A

(ii) XAX = X

(iii) ( AX ) = AX

(iv) ( XA) = XA

e conditions (i) - (iv) ensure that the matrix X is unique It is usu-

ally denoted by Adagger

Let A = U ΣV be the singular value decomposition of A en the

pseudoinverse Adagger of the matrix A is given by the expression

A

dagger

= V Σ+

U

where

Σdagger = diag (σ1+ σ+

minmn ) isin n times m

and

10

00

i

ii

i

σ σ σ

σ

+

gt

= =

By using the SVD of A it is easy to prove the following properties of Adagger

1 ( Adagger)dagger = A

2 ( Aτ)dagger = ( Adagger)τ

3 ( A)dagger = ( Adagger)

4 rank ( A) = rank ( Adagger) = rank ( AAdagger) = rank ( Adagger A)

5 If Aisin m times n with rank n then Adagger = ( A A)-1 A and Adagger A = I n

6 If Aisin m times n with rank m then Adagger = A( AA)-1 and AAdagger = I m

In addition one can easily prove that AAdagger

( Adagger

A) is orthogonal projector onto real( A) (real( A)) is fact is important when solving the over-

determined system of the linear equations Ax = b Aisin m times n in the least

squares sense





β = Adagger y = V Σ+ U τ y

Such β is the solution of the least squares problem and among all

solutions it has the minimum Euclidean norm

3 APPLICATIONS IN GEODESY

e singular value decomposition and the pseudoinverse are taught

in the courses Analysis and processing of geodetic measurements and

Special algorithms of geodetic measurement processing at the Faculty of

Geodesy University of Zagreb

Using any web browser one can find many applications of SVD in

Geodesy Such as the following presentation

22 SOLVING A HOMOGENEOUS SYSTEM OF LINEAR

EQUATIONS

Let Aisin m times n be the matrix of rank r en solving the homogeneous

system of linear equations Ax = 0 reduces to determining the null-su-

bspace of the matrix A From item (ii) of immediate consequences of the

SVD one finds out that

N ( A) = spanvr+1

vn

Here vr+1

vn are the last n-r columns of the matrix V from the SVD

of A Since V is unitary these vectors form an orthonormal basis of the

subspace N ( A) Since the SVD of A can be very accurately computed

this approach for solving the homogenous linear system is most com-

monly used

23 SOLVING THE LINEAR LEAST SQUARES PROBLEM

Let us consider an overdetermined system

1

1n

ij j i

J

A y i m β =

= =sum

of m linear equations in n unknown coefficients β1 β

2 β

n with m

gt n In matrix form it reads

Aβ = y

where

11 12 1

1 1

21 22 2

2 2

1 2

n

n

m n n m

n m

m m mn

A A A y

A A A y

A y

y A A A

β

β β

β

times

= isin = isin = isin

Such a system has usually no solution erefore it is reformulated

into the least squares problem

minβ || y - Aβ ||2

Suppose that there exists n times m matrix S so that AS is an orthogonal

projection onto real( A) In that case the solution is given by

β = Sy

because

Aβ = A(Sy) = ( AS ) y

is the orthogonal projection of y onto real( A)

If A = U ΣV τ is the singular value decomposition of A then the pse-

udoinverse Adagger of A given by Adagger = U Σ+V τ has the property that AAdagger is

orthogonal projector on real( A) Indeed we have

AAdagger = U ΣV τV Σ+U τ = UPU τ

where the square matrix P is obtained from Σ by replacing the non-zero diagonal elements with ones Matrices A and Σ have the same rank

and AAdagger = UPU τ is the orthogonal projector onto real( A) us Adagger is just

the wanted matrix S e final conclusion is that unknown βisin n is given

by the expression

FIGURE 1 httpwwwiag2009comarpresentations

E

REFERENCES

∙ GH Golub and CF VanLoan (1989) Matrix Computations e

Johns Hopkins University Press ∙ V Zadelj-Martić (1999) Parallel Algorithms for Butterfly Matrices

MSc esis (Croatian) University of Zagreb

∙ URL-1 httpwwwiag2009comarpresentations

∙ URL-2 httpenwikipediaorgwikiSingular_value_decomposition





where σ1

ge σ2

ge ge σminmn

ge 0 e nonnegative numbers

σ1 σ

2 σ

minmn are the singular values of A and the columns of U (V )

are the left (right) singular vectors of A

Matrix Σ is uniquely determined by the matrix A However the ma-

trices U and V are not unique If the singular values are multiple then

U and V can be post-multiplied by an arbitrary block-diagonal unitary

matrix whose diagonal blocks are of appropriate dimensions

e singular value decomposition has hundreds of applications It

is an excellent tool in matrix theory It is often used for solving different

matrix problems which arise in science economy engineering medicine

and even in human sciences Data mining web searching image reco-

gnition just to mention some contemporary problems which use SVD

often of large matrices Its widespread use is enhanced by the fact that

there exist excellent efficient and accurate methods for computing it

ere are several classes of methods the most important are one-sided

Jacobi methods divide and conquer (DC) differential qd (DQD) and QR

methods

We will first list several immediate consequences of this decompo-

sition

Let A = U ΣV be the singular value decomposition of the matrix Aisin m times n so that

σ1 ge σ

2 ge ge σ

r gt σ

r+1 = = σ

p = 0 p = minmn

Let U = [u1 u

m] V = [v

1 v

m] be the column representations

(often called partitions) of U and V respectively

en it is easy to show that the following holds

(i) rank ( A) = r

(ii) N ( A) = spanvr+1vn

(iii) real( A) = spanu1u

r

(iv)

1

r

i i i r r r

I

A u v U V σ =

= = Σsum such that

U r = [u

1 u

r] isin m times r

V r = [v

1 v

r] isin n times r

Σr = diag (σ

1 σ

r ) isin r times r

(v) || A ||F

2 = σ12 + + σ

r 2

(vi) || A ||F = σ

1

It can also be shown that the distance (in spectral norm) of a square

matrix A and the set of singular matrices of the same dimension is equal

to the smallest singular value of Ais follows from a more general re-

sult the Ekhard Young and Mirsky theorem

Teorem 2

Let Aisin m times n and let A = U ΣV be the singular value decomposition

of A so that

σ1 ge σ2 ge ge σr gt σr+1 = = σ p = 0 p = minmn

holds If k lt r = rank ( A) and

1

k

k i i i

i

C u vσ =

= sum then

12 2( )

min k k rank K k

A K C C σ +=

minus = minus =

us the closest rank k approximation of A is obtained by using the

first k singular values and vectors of A

2 SOME APPLICATIONS OF THE SINGULAR VALUE

DECOMPOSITION

Here we will address several applications

21 DETERMINING THE PSEUDOINVERSE OF A GENERAL

MATRIX

Here we will consider the Moore-Penrose pseudoinverse

Let Aisin m times n e matrix X isin n times m is pseudoinverse of Aisin m times n if the

following four conditions are fulfilled

(i) AXA = A

(ii) XAX = X

(iii) ( AX ) = AX

(iv) ( XA) = XA

e conditions (i) - (iv) ensure that the matrix X is unique It is usu-

ally denoted by Adagger

Let A = U ΣV be the singular value decomposition of A en the

pseudoinverse Adagger of the matrix A is given by the expression

A

dagger

= V Σ+

U

where

Σdagger = diag (σ1+ σ+

minmn ) isin n times m

and

10

00

i

ii

i

σ σ σ

σ

+

gt

= =

By using the SVD of A it is easy to prove the following properties of Adagger

1 ( Adagger)dagger = A

2 ( Aτ)dagger = ( Adagger)τ

3 ( A)dagger = ( Adagger)

4 rank ( A) = rank ( Adagger) = rank ( AAdagger) = rank ( Adagger A)

5 If Aisin m times n with rank n then Adagger = ( A A)-1 A and Adagger A = I n

6 If Aisin m times n with rank m then Adagger = A( AA)-1 and AAdagger = I m

In addition one can easily prove that AAdagger

( Adagger

A) is orthogonal projector onto real( A) (real( A)) is fact is important when solving the over-

determined system of the linear equations Ax = b Aisin m times n in the least

squares sense
















EQUATIONS





N ( A) = spanvr+1

vn

Here vr+1





monly used



1

1n

ij j i

J

A y i m β =

= =sum


2 β

n with m


Aβ = y

where

11 12 1

1 1

21 22 2

2 2

1 2

n

n

m n n m

n m

m m mn

A A A y

A A A y

A y

y A A A

β

β β

β

times




minβ || y - Aβ ||2



β = Sy

because

Aβ = A(Sy) = ( AS ) y









by the expression


E

REFERENCES





















EQUATIONS





N ( A) = spanvr+1

vn

Here vr+1





monly used



1

1n

ij j i

J

A y i m β =

= =sum


2 β

n with m


Aβ = y

where

11 12 1

1 1

21 22 2

2 2

1 2

n

n

m n n m

n m

m m mn

A A A y

A A A y

A y

y A A A

β

β β

β

times




minβ || y - Aβ ||2



β = Sy

because

Aβ = A(Sy) = ( AS ) y









by the expression


E

REFERENCES








the singular value decomposition and applications in geodesy

Documents