singular value decomposition and digital image compression€¦ · singular value decomposition and...

Singular Value Decompositionand Digital Image Compression

Chris Bingham

December 12, 2016

Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 1 / 32

Plan

What is SVD?

Process of SVD

Application to Digital Image Compression


What is Singular Value Decomposition?

Method used to diagonalize a non-square matrix A.

Find the singular values of matrix A

Use the singular values to find an approximation of matrix A that hasa lesser rank that the original matrix.


Theorem of Singular Value Decomposition

Let A be an m × n matrix with rank r . Then there exists an m × n matrixΣ for which the diagonal entries in D are the first r singular values of A,σ1 ≥ σ2 ≥ ... ≥ σr > 0, and there exist an m ×m orthogonal matrix Uand an n × n orthogonal matrix V such that

A = U Σ V T

m × n m ×m m × n n × n


Example Using a 3x2 Matrix

Given the 3x2 matrix below, compute the SVD.

A =

1 11 00 1


Calculate ATA

ATA =

[1 1 01 0 1

]1 11 00 1

=

[2 11 2

]By calculating ATA we obtain a square symmetric matrix through whichwe can find the eigenvalues.


Find the Eigenvalues of ATA

p(λ) =λ2 − Tλ+ D

p(λ) =λ2 − 4λ+ 3

p(λ) =(λ− 3)(λ− 1)

We now have the eigenvalues λ1 = 3 and λ2 = 1


Construct V

λ1 = 3 v1 =

[11

]λ2 = 1 v2 =

[−11

]Convert v1 and v2 to unit vectors and put as columns of matrix V .

V =

[1√2− 1√

21√2

1√2

]


Construct Σ

The format of Σ is

Σ =

[D 0

0 0

]Where

D =

[√3 0

0 1

]We want Σ to be the same dimesions as the original matrix A. ThereforeΣ will be a 3× 2 matrix.

Σ =

√3 00 10 0


Construct U

Using

uk =1

σk(Avk)

Letting k range from 1 to the rank of A, construct the columns of U.


Construct U

u1 =1√3

1 11 00 1

[ 1√21√2

] =

√

23

1√61√6

u2 =1

1

1 11 00 1

[− 1√2

1√2

] =

0− 1√

21√2


Construct U

U =

√

23 0

1√6− 1√

21√6

1√2

Problem! This should be a square matrix by the SVD theorem. To findthe third vector use the Gram-Schmidt Process.


Using the Gram-Schmidt Process

Start by choosing a vector y that is not in the plane of the the two vectorsalready found for the matrix U.

y =

100

We can now use this formula from the Gram-Schmidt process to find ourthird unit vector for U.

u3 = y − y .u1u1.u1

u1 −y .u2u2.u2

u2


Completing Matrix U

By the Gram-Schmidt process we obtain our third vector, all we need todo is make it a unit vector.

u3 =

13−1

3−1

3

u3||u3||

=

1√3

− 1√3

− 1√3

We now have the correct matrix U and we can complete the SVD ofmatrix A.

U =

√

23 0 1√

31√6− 1√

2− 1√

31√6

1√2− 1√

3


A = UΣV T

A = U Σ V T

m × n m ×m m × n n × n1 11 00 1

=

√

23 0 1√

31√6− 1√

2− 1√

31√6

1√2− 1√

3

√3 0

0 10 0

[ 1√2− 1√

21√2

1√2

]T


Check that A = UΣV T

To be able to check that our original matrix is achieved, start bymultiplying the matrix U by each column of Σ

A =[u1 u2 · · · ur · · · um

]σ1 0 0

0. . . 0 0

0 0 σr0 0

vT1...vTrvTr+1

...vTn

A =[σ1u1 σ2u2 · · · σrur 0 · · · 0

]

vT1...vTrvTr+1

...vTn


Check thatA = UΣV T

Partition the matrices and perform column-row multiplication

A =[σ1u1 σ2u2 · · · σrur 0 · · · 0

]

vT1...

vTrvTr+1

...

vTn

A = σ1u1v

T1 + σ2u2v

T2 + ...+ σrurv

Tr + 0vTr+1 + 0vTn

A = σ1u1vT1 + σ2u2v

T2 + ...+ σrurv

Tr

Ak =k∑

i=1

σiuivTi

Letting k be the desired rank approximation of AChris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 17 / 32

Applied to Our 3× 2 Example Matrix A

Our calculated SVD

A =

1 11 00 1

=

√

23 0 1√

31√6− 1√

2− 1√

31√6

1√2− 1√

3

√3 0

0 10 0

[ 1√2− 1√

21√2

1√2

]T

Which has the form

A =[u1 u2 u3

] σ1 00 σ20 0

[vT1vT2

]


T2


Applied to Our 3× 2 Example Matrix A


T2

A =√

3

√

23

1√61√6

[ 1√2

1√2

]+ 1

0− 1√

21√2

[− 1√2

1√2

]

A =√

3

1√3

1√3

1√12

1√12

1√12

1√12

+ 1

0 012 −1

2−1

212

A =

1 112

12

12

12

+

0 012 −1

2−1

212


Application to Digital Image Compression

Digital images are stored as pixels, each pixel representing a cell in amatrix.

Each cell represents the intensity of the pixel and has an associatedvalue.

Pixel data can be extracted from an image and processed throughSVD to gain an compressed version of the image

Visual representation of SVD can be seen by displaying theapproximated pixel data using varying amounts of singular values.


Original Image

Original image that data was extracted from to create matrix A.


Using 1 Singular Value

The image is unrecognizable. While the file size is tiny, this is not a usefulcompression of the original image.


Using 2 Singular Values

Just one rank added to the approximation made a noticable difference. Asthe rank gets larger the difference will be indistinguishable.



Pretty accurate approximation of the original image. Very minor visualchanges from the original image.



Image looks identical to original. Additional ranks added will have novisual changes.



Approximation with same rank as original matrix.


Bibliography

A. Schultz. 2006 SWEET Applications of SVDhttp://palmer.wellesley.edu/ as-chultz/summer06/math103/coursenotes and handouts/mathematica060816/

C. Long. Visualization of Matrix Singular Value Decompositionhttp://faculty.uml.edu/cbyrne/svd.pdf

D. Kalman The Singularly Valuable Decomposition: The SVD of aMatrix http://mth1007.mathappl.polymtl.ca/DanKalman.pdf

D. Lay. 2012 Linear Algebra and its Applications

J. Chen. Image Compression with SVDhttp://fourier.eng.hmc.edu/e161/lectures/svdcompression.html


singular value decomposition and digital image compression€¦ · singular value decomposition and...

Documents