singular value decomposition and digital image compression€¦ · singular value decomposition and...
Post on 04-Jun-2020
8 Views
Preview:
TRANSCRIPT
Singular Value Decompositionand Digital Image Compression
Chris Bingham
December 12, 2016
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 1 / 32
Plan
What is SVD?
Process of SVD
Application to Digital Image Compression
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 2 / 32
What is Singular Value Decomposition?
Method used to diagonalize a non-square matrix A.
Find the singular values of matrix A
Use the singular values to find an approximation of matrix A that hasa lesser rank that the original matrix.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 3 / 32
Theorem of Singular Value Decomposition
Let A be an m × n matrix with rank r . Then there exists an m × n matrixΣ for which the diagonal entries in D are the first r singular values of A,σ1 ≥ σ2 ≥ ... ≥ σr > 0, and there exist an m ×m orthogonal matrix Uand an n × n orthogonal matrix V such that
A = U Σ V T
m × n m ×m m × n n × n
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 4 / 32
Example Using a 3x2 Matrix
Given the 3x2 matrix below, compute the SVD.
A =
1 11 00 1
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 5 / 32
Calculate ATA
ATA =
[1 1 01 0 1
]1 11 00 1
=
[2 11 2
]By calculating ATA we obtain a square symmetric matrix through whichwe can find the eigenvalues.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 6 / 32
Find the Eigenvalues of ATA
p(λ) =λ2 − Tλ+ D
p(λ) =λ2 − 4λ+ 3
p(λ) =(λ− 3)(λ− 1)
We now have the eigenvalues λ1 = 3 and λ2 = 1
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 7 / 32
Construct V
λ1 = 3 v1 =
[11
]λ2 = 1 v2 =
[−11
]Convert v1 and v2 to unit vectors and put as columns of matrix V .
V =
[1√2− 1√
21√2
1√2
]
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 8 / 32
Construct Σ
The format of Σ is
Σ =
[D 0
0 0
]Where
D =
[√3 0
0 1
]We want Σ to be the same dimesions as the original matrix A. ThereforeΣ will be a 3× 2 matrix.
Σ =
√3 00 10 0
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 9 / 32
Construct U
Using
uk =1
σk(Avk)
Letting k range from 1 to the rank of A, construct the columns of U.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 10 / 32
Construct U
u1 =1√3
1 11 00 1
[ 1√21√2
] =
√
23
1√61√6
u2 =1
1
1 11 00 1
[− 1√2
1√2
] =
0− 1√
21√2
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 11 / 32
Construct U
U =
√
23 0
1√6− 1√
21√6
1√2
Problem! This should be a square matrix by the SVD theorem. To findthe third vector use the Gram-Schmidt Process.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 12 / 32
Using the Gram-Schmidt Process
Start by choosing a vector y that is not in the plane of the the two vectorsalready found for the matrix U.
y =
100
We can now use this formula from the Gram-Schmidt process to find ourthird unit vector for U.
u3 = y − y .u1u1.u1
u1 −y .u2u2.u2
u2
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 13 / 32
Completing Matrix U
By the Gram-Schmidt process we obtain our third vector, all we need todo is make it a unit vector.
u3 =
13−1
3−1
3
u3||u3||
=
1√3
− 1√3
− 1√3
We now have the correct matrix U and we can complete the SVD ofmatrix A.
U =
√
23 0 1√
31√6− 1√
2− 1√
31√6
1√2− 1√
3
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 14 / 32
A = UΣV T
A = U Σ V T
m × n m ×m m × n n × n1 11 00 1
=
√
23 0 1√
31√6− 1√
2− 1√
31√6
1√2− 1√
3
√3 0
0 10 0
[ 1√2− 1√
21√2
1√2
]T
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 15 / 32
Check that A = UΣV T
To be able to check that our original matrix is achieved, start bymultiplying the matrix U by each column of Σ
A =[u1 u2 · · · ur · · · um
]σ1 0 0
0. . . 0 0
0 0 σr0 0
vT1...vTrvTr+1
...vTn
A =[σ1u1 σ2u2 · · · σrur 0 · · · 0
]
vT1...vTrvTr+1
...vTn
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 16 / 32
Check thatA = UΣV T
Partition the matrices and perform column-row multiplication
A =[σ1u1 σ2u2 · · · σrur 0 · · · 0
]
vT1...
vTrvTr+1
...
vTn
A = σ1u1v
T1 + σ2u2v
T2 + ...+ σrurv
Tr + 0vTr+1 + 0vTn
A = σ1u1vT1 + σ2u2v
T2 + ...+ σrurv
Tr
Ak =k∑
i=1
σiuivTi
Letting k be the desired rank approximation of AChris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 17 / 32
Applied to Our 3× 2 Example Matrix A
Our calculated SVD
A =
1 11 00 1
=
√
23 0 1√
31√6− 1√
2− 1√
31√6
1√2− 1√
3
√3 0
0 10 0
[ 1√2− 1√
21√2
1√2
]T
Which has the form
A =[u1 u2 u3
] σ1 00 σ20 0
[vT1vT2
]
A = σ1u1vT1 + σ2u2v
T2
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 18 / 32
Applied to Our 3× 2 Example Matrix A
A = σ1u1vT1 + σ2u2v
T2
A =√
3
√
23
1√61√6
[ 1√2
1√2
]+ 1
0− 1√
21√2
[− 1√2
1√2
]
A =√
3
1√3
1√3
1√12
1√12
1√12
1√12
+ 1
0 012 −1
2−1
212
A =
1 112
12
12
12
+
0 012 −1
2−1
212
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 19 / 32
Application to Digital Image Compression
Digital images are stored as pixels, each pixel representing a cell in amatrix.
Each cell represents the intensity of the pixel and has an associatedvalue.
Pixel data can be extracted from an image and processed throughSVD to gain an compressed version of the image
Visual representation of SVD can be seen by displaying theapproximated pixel data using varying amounts of singular values.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 20 / 32
Original Image
Original image that data was extracted from to create matrix A.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 21 / 32
Using 1 Singular Value
The image is unrecognizable. While the file size is tiny, this is not a usefulcompression of the original image.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 22 / 32
Using 2 Singular Values
Just one rank added to the approximation made a noticable difference. Asthe rank gets larger the difference will be indistinguishable.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 23 / 32
Using 5 Singular Values
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 24 / 32
Using 10 Singular Values
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 25 / 32
Using 20 Singular Values
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 26 / 32
Using 40 Singular Values
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 27 / 32
Using 80 Singular Values
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 28 / 32
Using 160 Singular Values
Pretty accurate approximation of the original image. Very minor visualchanges from the original image.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 29 / 32
Using 320 Singular Values
Image looks identical to original. Additional ranks added will have novisual changes.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 30 / 32
Using 480 Singular Values
Approximation with same rank as original matrix.
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 31 / 32
Bibliography
A. Schultz. 2006 SWEET Applications of SVDhttp://palmer.wellesley.edu/ as-chultz/summer06/math103/coursenotes and handouts/mathematica060816/
C. Long. Visualization of Matrix Singular Value Decompositionhttp://faculty.uml.edu/cbyrne/svd.pdf
D. Kalman The Singularly Valuable Decomposition: The SVD of aMatrix http://mth1007.mathappl.polymtl.ca/DanKalman.pdf
D. Lay. 2012 Linear Algebra and its Applications
J. Chen. Image Compression with SVDhttp://fourier.eng.hmc.edu/e161/lectures/svdcompression.html
Chris Bingham Singular Value Decompositionand Digital Image CompressionDecember 12, 2016 32 / 32
top related