some recent advances in nonnegative matrix factorization and … · 2013. 11. 21. · [gg10] g.,...
TRANSCRIPT
![Page 1: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/1.jpg)
Some Recent Advances in Nonnegative MatrixFactorization and their Applications to
Hyperspectral Unmixing
Nicolas Gillis
https://sites.google.com/site/nicolasgillis/
Universite de MonsDepartment of Mathematics and Operational Research
Joint work with Robert Plemmons (Wake Forest U.)and Stephen Vavasis (U. of Waterloo)
International Workshop on Numerical Linear Algebra with Applicationsin honor of the 75th birthday of Prof. Robert Plemmons
November, 2013CUHK Recent Advances in NMF 1
![Page 2: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/2.jpg)
First...
Figure: Bob and I in Lisbon (Third workshop on hyperspectral image and signalprocessing: evolution in remote sensing –WHISPERS, 2011)
CUHK Recent Advances in NMF 2
![Page 3: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/3.jpg)
Outline
1. Nonnegative Matrix Factorization (NMF)
I Definition, motivations & applications
2. Using Underapproximations for NMF
I Solving NMF recursively with underapproximations
I Sparse and spatial underapproximations for hyperspectral unmixing
3. Separable and Near-Separable NMF
I A subclass of efficiently solvable NMF problems
I Robust algorithms for Near-Separable NMF
I Application to hyperspectral unmixing
CUHK Recent Advances in NMF 3
![Page 4: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/4.jpg)
Nonnegative Matrix Factorization (NMF)Given a matrix M ∈ Rm×n+ and a factorization rank r ∈ N, find
U ∈ Rm×rand V ∈ Rr×n such that
minU≥0,V≥0
||M − UV ||2F =∑i,j
(M − UV )2ij . (NMF)
NMF is a linear dimensionality reduction technique for nonnegative data :
M(:, i)︸ ︷︷ ︸≥0
≈r∑
k=1
U(:, k)︸ ︷︷ ︸≥0
V (k, i)︸ ︷︷ ︸≥0
for all i.
Why nonnegativity?
→ Interpretability: Nonnegativity constraints lead to a sparse andpart-based representation.→ Many applications. Text mining, hyperspectral unmixing, imageprocessing, community detection, clustering, etc.
CUHK Recent Advances in NMF 4
![Page 5: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/5.jpg)
Nonnegative Matrix Factorization (NMF)Given a matrix M ∈ Rm×n+ and a factorization rank r ∈ N, find
U ∈ Rm×rand V ∈ Rr×n such that
minU≥0,V≥0
||M − UV ||2F =∑i,j
(M − UV )2ij . (NMF)
NMF is a linear dimensionality reduction technique for nonnegative data :
M(:, i)︸ ︷︷ ︸≥0
≈r∑
k=1
U(:, k)︸ ︷︷ ︸≥0
V (k, i)︸ ︷︷ ︸≥0
for all i.
Why nonnegativity?
→ Interpretability: Nonnegativity constraints lead to a sparse andpart-based representation.→ Many applications. Text mining, hyperspectral unmixing, imageprocessing, community detection, clustering, etc.
CUHK Recent Advances in NMF 4
![Page 6: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/6.jpg)
Nonnegative Matrix Factorization (NMF)Given a matrix M ∈ Rm×n+ and a factorization rank r ∈ N, find
U ∈ Rm×rand V ∈ Rr×n such that
minU≥0,V≥0
||M − UV ||2F =∑i,j
(M − UV )2ij . (NMF)
NMF is a linear dimensionality reduction technique for nonnegative data :
M(:, i)︸ ︷︷ ︸≥0
≈r∑
k=1
U(:, k)︸ ︷︷ ︸≥0
V (k, i)︸ ︷︷ ︸≥0
for all i.
Why nonnegativity?
→ Interpretability: Nonnegativity constraints lead to a sparse andpart-based representation.→ Many applications. Text mining, hyperspectral unmixing, imageprocessing, community detection, clustering, etc.
CUHK Recent Advances in NMF 4
![Page 7: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/7.jpg)
Application 1: image processing
U ≥ 0 constraints the basis elements to be nonnegative.
Moreover V ≥ 0 imposes an additive reconstruction.
The basis elements extract facial features such as eyes, nose and lips.
CUHK Recent Advances in NMF 5
![Page 8: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/8.jpg)
Application 1: image processing
U ≥ 0 constraints the basis elements to be nonnegative.
Moreover V ≥ 0 imposes an additive reconstruction.
The basis elements extract facial features such as eyes, nose and lips.
CUHK Recent Advances in NMF 5
![Page 9: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/9.jpg)
Application 2: text mining
� Basis elements allow to recover the different topics;
� Weights allow to assign each text to its corresponding topics.
CUHK Recent Advances in NMF 6
![Page 10: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/10.jpg)
Application 2: text mining
� Basis elements allow to recover the different topics;
� Weights allow to assign each text to its corresponding topics.
CUHK Recent Advances in NMF 6
![Page 11: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/11.jpg)
Application 3: hyperspectral unmixing
Figure: Hyperspectral image.
CUHK Recent Advances in NMF 7
![Page 12: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/12.jpg)
Application 3: hyperspectral unmixing
� Basis elements allow to recover the different materials;
� Weights allow to know which pixel contains which material.
CUHK Recent Advances in NMF 8
![Page 13: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/13.jpg)
Application 3: hyperspectral unmixing
� Basis elements allow to recover the different materials;
� Weights allow to know which pixel contains which material.
CUHK Recent Advances in NMF 8
![Page 14: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/14.jpg)
Application 3: hyperspectral unmixing
Figure: Urban dataset.
CUHK Recent Advances in NMF 9
![Page 15: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/15.jpg)
Application 3: hyperspectral unmixing
Figure: Urban dataset.
CUHK Recent Advances in NMF 9
![Page 16: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/16.jpg)
Using Underapproximations for NMF
CUHK Recent Advances in NMF 10
![Page 17: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/17.jpg)
Issues of using NMF
1. NMF is NP-hard [V09].
2. The optimal solution is, in most cases, non-unique and the problem isill-posed [G12]. Many variants of NMF impose additional constraints(e.g., sparsity on U , smoothness of V , etc.).
3. In practice, it is difficult to choose the factorization rank (in general,trial and error approach or estimation using the SVD).
A possible way to overcome drawbacks 2. and 3. is to useunderapproximation constraints to solve NMF recursively.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.[G12] G., Sparse and Unique Nonnegative Matrix Factorization Through Data Preprocessing, J.of Machine Learning Research, 2012.
CUHK Recent Advances in NMF 11
![Page 18: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/18.jpg)
Issues of using NMF
1. NMF is NP-hard [V09].
2. The optimal solution is, in most cases, non-unique and the problem isill-posed [G12]. Many variants of NMF impose additional constraints(e.g., sparsity on U , smoothness of V , etc.).
3. In practice, it is difficult to choose the factorization rank (in general,trial and error approach or estimation using the SVD).
A possible way to overcome drawbacks 2. and 3. is to useunderapproximation constraints to solve NMF recursively.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.[G12] G., Sparse and Unique Nonnegative Matrix Factorization Through Data Preprocessing, J.of Machine Learning Research, 2012.
CUHK Recent Advances in NMF 11
![Page 19: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/19.jpg)
Issues of using NMF
1. NMF is NP-hard [V09].
2. The optimal solution is, in most cases, non-unique and the problem isill-posed [G12]. Many variants of NMF impose additional constraints(e.g., sparsity on U , smoothness of V , etc.).
3. In practice, it is difficult to choose the factorization rank (in general,trial and error approach or estimation using the SVD).
A possible way to overcome drawbacks 2. and 3. is to useunderapproximation constraints to solve NMF recursively.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.[G12] G., Sparse and Unique Nonnegative Matrix Factorization Through Data Preprocessing, J.of Machine Learning Research, 2012.
CUHK Recent Advances in NMF 11
![Page 20: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/20.jpg)
Issues of using NMF
1. NMF is NP-hard [V09].
2. The optimal solution is, in most cases, non-unique and the problem isill-posed [G12]. Many variants of NMF impose additional constraints(e.g., sparsity on U , smoothness of V , etc.).
3. In practice, it is difficult to choose the factorization rank (in general,trial and error approach or estimation using the SVD).
A possible way to overcome drawbacks 2. and 3. is to useunderapproximation constraints to solve NMF recursively.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.[G12] G., Sparse and Unique Nonnegative Matrix Factorization Through Data Preprocessing, J.of Machine Learning Research, 2012.
CUHK Recent Advances in NMF 11
![Page 21: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/21.jpg)
Nonnegative Matrix Underapproximation (NMU)It is possible to solve NMF recursively, solving at each step
minu≥0,v≥0
||M − uvT ||2F such that uvT ≤M ⇐⇒ M − uvT ≥ 0.
NMU is yet another linear dimensionality reduction technique.However,
� As PCA, it is computed recursively and is well-posed [GG10].
� As NMF, it leads to a separation by parts. Moreover theadditional underapproximation constraints enhance this property.
� In the presence of pure-pixels, the NMU recursion is able todetect materials individually [GP11].
[GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization,Pattern Recognition, 2010.[GP11] G., Plemmons, Dimensionality Reduction, Classification, and Spectral Mixture Analysisusing Nonnegative Underapproximation, Optical Engineering, 2011.
CUHK Recent Advances in NMF 12
![Page 22: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/22.jpg)
Nonnegative Matrix Underapproximation (NMU)It is possible to solve NMF recursively, solving at each step
minu≥0,v≥0
||M − uvT ||2F such that uvT ≤M ⇐⇒ M − uvT ≥ 0.
NMU is yet another linear dimensionality reduction technique.However,
� As PCA, it is computed recursively and is well-posed [GG10].
� As NMF, it leads to a separation by parts. Moreover theadditional underapproximation constraints enhance this property.
� In the presence of pure-pixels, the NMU recursion is able todetect materials individually [GP11].
[GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization,Pattern Recognition, 2010.[GP11] G., Plemmons, Dimensionality Reduction, Classification, and Spectral Mixture Analysisusing Nonnegative Underapproximation, Optical Engineering, 2011.
CUHK Recent Advances in NMF 12
![Page 23: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/23.jpg)
Nonnegative Matrix Underapproximation (NMU)It is possible to solve NMF recursively, solving at each step
minu≥0,v≥0
||M − uvT ||2F such that uvT ≤M ⇐⇒ M − uvT ≥ 0.
NMU is yet another linear dimensionality reduction technique.However,
� As PCA, it is computed recursively and is well-posed [GG10].
� As NMF, it leads to a separation by parts. Moreover theadditional underapproximation constraints enhance this property.
� In the presence of pure-pixels, the NMU recursion is able todetect materials individually [GP11].
[GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization,Pattern Recognition, 2010.[GP11] G., Plemmons, Dimensionality Reduction, Classification, and Spectral Mixture Analysisusing Nonnegative Underapproximation, Optical Engineering, 2011.
CUHK Recent Advances in NMF 12
![Page 24: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/24.jpg)
Nonnegative Matrix Underapproximation (NMU)It is possible to solve NMF recursively, solving at each step
minu≥0,v≥0
||M − uvT ||2F such that uvT ≤M ⇐⇒ M − uvT ≥ 0.
NMU is yet another linear dimensionality reduction technique.However,
� As PCA, it is computed recursively and is well-posed [GG10].
� As NMF, it leads to a separation by parts. Moreover theadditional underapproximation constraints enhance this property.
� In the presence of pure-pixels, the NMU recursion is able todetect materials individually [GP11].
[GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization,Pattern Recognition, 2010.[GP11] G., Plemmons, Dimensionality Reduction, Classification, and Spectral Mixture Analysisusing Nonnegative Underapproximation, Optical Engineering, 2011.
CUHK Recent Advances in NMF 12
![Page 25: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/25.jpg)
Example of NMU on the Urban dataset
Figure: Hyperspectral image from aircraft - Army Geospatial Center -307× 307× 162.
CUHK Recent Advances in NMF 13
![Page 26: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/26.jpg)
Example of NMU on the Urban dataset
CUHK Recent Advances in NMF 14
![Page 27: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/27.jpg)
Example of NMU on the Urban dataset
CUHK Recent Advances in NMF 14
![Page 28: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/28.jpg)
Example of NMU on the Urban dataset
CUHK Recent Advances in NMF 14
![Page 29: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/29.jpg)
Example of NMU on the Urban dataset
CUHK Recent Advances in NMF 14
![Page 30: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/30.jpg)
Example of NMU on the Urban dataset
CUHK Recent Advances in NMF 14
![Page 31: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/31.jpg)
Example on the San Diego Airport dataset
CUHK Recent Advances in NMF 15
![Page 32: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/32.jpg)
Example on the San Diego Airport dataset
CUHK Recent Advances in NMF 16
![Page 33: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/33.jpg)
Additional Sparsity Constraints
� With more blur and noise, NMU typically fails to detect materialsindividually.
� However, it can still be a useful dimensionality reductiontechnique (e.g., combined with nearest neighbor or k-means).
� In order to detect materials, it is possible to enhance sparsity ofthe abundance matrix V to extract more localized features. Wesolve [GP13]
minu≥0,v≥0
||M − uvT ||2F + µ||v||1
uvT ≤M. (sparse NMU)
[GP13] G., Plemmons, Nonnegative Matrix Underapproximation and its Application forHyperspectral Image Analysis, Lin. Alg. Appl., 2013.
CUHK Recent Advances in NMF 17
![Page 34: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/34.jpg)
Additional Sparsity Constraints
� With more blur and noise, NMU typically fails to detect materialsindividually.
� However, it can still be a useful dimensionality reductiontechnique (e.g., combined with nearest neighbor or k-means).
� In order to detect materials, it is possible to enhance sparsity ofthe abundance matrix V to extract more localized features. Wesolve [GP13]
minu≥0,v≥0
||M − uvT ||2F + µ||v||1
uvT ≤M. (sparse NMU)
[GP13] G., Plemmons, Nonnegative Matrix Underapproximation and its Application forHyperspectral Image Analysis, Lin. Alg. Appl., 2013.
CUHK Recent Advances in NMF 17
![Page 35: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/35.jpg)
San Diego Airport
Figure: First four basis elements of NMU
Figure: First four basis elements of sparse NMU
CUHK Recent Advances in NMF 18
![Page 36: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/36.jpg)
Additional Spatial Constraints
It is also possible to take into account spatial constraints [GPZ12]:neighbor pixels are more likely to contain the same materials.
Figure: NMU vs. spatial NMU on the Cuprite data set.
[GPZ12] G., Plemmons, Zhang, Priors in Sparse Recursive Decompositions of HyperspectralImages, Proc. of SPIE, 2012.
CUHK Recent Advances in NMF 19
![Page 37: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/37.jpg)
Separable and Near-Separable NMF
CUHK Recent Advances in NMF 20
![Page 38: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/38.jpg)
Can we only solve NMF problems?
Given a matrix M ∈ Rm×n+ and a factorization rank r ∈ N, find
U ∈ Rm×rand V ∈ Rr×n such that
minU≥0,V≥0
||M − UV ||2F =∑i,j
(M − UV )2ij . (NMF)
� NMF is NP-hard [V09].
� In practice, it is often satisfactory to use locally optimal solutions forfurther analysis of the data. In other words, heuristics often solve theproblem efficiently with acceptable answers.
� Try to analyze this state of affairs by considering enerative modelsand algorithms that can recover hidden data.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.
CUHK Recent Advances in NMF 21
![Page 39: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/39.jpg)
Can we only solve NMF problems?
Given a matrix M ∈ Rm×n+ and a factorization rank r ∈ N, find
U ∈ Rm×rand V ∈ Rr×n such that
minU≥0,V≥0
||M − UV ||2F =∑i,j
(M − UV )2ij . (NMF)
� NMF is NP-hard [V09].
� In practice, it is often satisfactory to use locally optimal solutions forfurther analysis of the data. In other words, heuristics often solve theproblem efficiently with acceptable answers.
� Try to analyze this state of affairs by considering enerative modelsand algorithms that can recover hidden data.
[V09] Vavasis, On the Complexity of Nonnegative Matrix Factorization, SIAM J. onOptimization, 2009.
CUHK Recent Advances in NMF 21
![Page 40: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/40.jpg)
Separability Assumption
For NMF, it is possible to compute optimal solutions in polynomial time,given that the input data matrix M satisfies a (rather strong) condition:separability [AGKM12].
The nonnegative matrix M is r-separable if and only if
there exists an NMF (U, V ) ≥ 0 of rank r with M = UV whereeach column of U is equal to a column of M .
[AGKM12] Arora, Ge, Kannan, Moitra, Computing a Nonnegative Matrix Factorization –Provably, STOC 2012.
CUHK Recent Advances in NMF 22
![Page 41: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/41.jpg)
Separability Assumption
For NMF, it is possible to compute optimal solutions in polynomial time,given that the input data matrix M satisfies a (rather strong) condition:separability [AGKM12].
The nonnegative matrix M is r-separable if and only if
there exists an NMF (U, V ) ≥ 0 of rank r with M = UV whereeach column of U is equal to a column of M .
[AGKM12] Arora, Ge, Kannan, Moitra, Computing a Nonnegative Matrix Factorization –Provably, STOC 2012.
CUHK Recent Advances in NMF 22
![Page 42: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/42.jpg)
Is separability a reasonable assumption?
� Text mining: for each topic, there is a ‘pure’ document on that topic,or, for each topic, there is a ‘pure’ word used only by that topic.[KSK13] Kumar, Sindhwani, Kambadur, Fast Conical Hull Algorithms for Near-separableNon-Negative Matrix Factorization, ICML 2013.[AG+13] Arora, Ge, Halpern, Mimno, Moitra, Sontag, Wu, Zhu, A Practical Algorithm forTopic Modeling with Provable Guarantees, ICML 2013.[DRIS13] Ding, Rohban, Ishwar, Saligrama, Topic Discovery through Data Dependent andRandom Projections, ICML 2013.
� Hyperspectral unmixing: separability is particularly natural: foreach constitutive material, there is a ‘pure’ pixel containing only thatmaterial. This is the so called pure-pixel assumption which is widelyused in hyperspectral imaging.
� General image processing: No.
CUHK Recent Advances in NMF 23
![Page 43: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/43.jpg)
Is separability a reasonable assumption?
� Text mining: for each topic, there is a ‘pure’ document on that topic,or, for each topic, there is a ‘pure’ word used only by that topic.[KSK13] Kumar, Sindhwani, Kambadur, Fast Conical Hull Algorithms for Near-separableNon-Negative Matrix Factorization, ICML 2013.[AG+13] Arora, Ge, Halpern, Mimno, Moitra, Sontag, Wu, Zhu, A Practical Algorithm forTopic Modeling with Provable Guarantees, ICML 2013.[DRIS13] Ding, Rohban, Ishwar, Saligrama, Topic Discovery through Data Dependent andRandom Projections, ICML 2013.
� Hyperspectral unmixing: separability is particularly natural: foreach constitutive material, there is a ‘pure’ pixel containing only thatmaterial. This is the so called pure-pixel assumption which is widelyused in hyperspectral imaging.
� General image processing: No.
CUHK Recent Advances in NMF 23
![Page 44: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/44.jpg)
Is separability a reasonable assumption?
� Text mining: for each topic, there is a ‘pure’ document on that topic,or, for each topic, there is a ‘pure’ word used only by that topic.[KSK13] Kumar, Sindhwani, Kambadur, Fast Conical Hull Algorithms for Near-separableNon-Negative Matrix Factorization, ICML 2013.[AG+13] Arora, Ge, Halpern, Mimno, Moitra, Sontag, Wu, Zhu, A Practical Algorithm forTopic Modeling with Provable Guarantees, ICML 2013.[DRIS13] Ding, Rohban, Ishwar, Saligrama, Topic Discovery through Data Dependent andRandom Projections, ICML 2013.
� Hyperspectral unmixing: separability is particularly natural: foreach constitutive material, there is a ‘pure’ pixel containing only thatmaterial. This is the so called pure-pixel assumption which is widelyused in hyperspectral imaging.
� General image processing: No.
CUHK Recent Advances in NMF 23
![Page 45: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/45.jpg)
Geometric Interpretation of Separable NMF
After normalization, the columns of M,U and V sum to one: the columnsof U are the vertices of the convex hull of the columns of M .
M is r-separable ⇐⇒ M = U [Ir, V′]Π,
for some V ′ ≥ 0 and some permutation Π.
CUHK Recent Advances in NMF 24
![Page 46: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/46.jpg)
Separable NMF with Noise
M = U [Ir, V′]Π +N, where N is the noise.
CUHK Recent Advances in NMF 25
![Page 47: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/47.jpg)
Near-Separable NMF: Noise and Conditioning
We will assume that the noise is bounded (but otherwise arbitrary):
||N(:, i)||1 ≤ ε, for all i,
and some dependence on some condition number is unavoidable:
CUHK Recent Advances in NMF 26
![Page 48: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/48.jpg)
Near-Separable NMF: Noise and Conditioning
We will assume that the noise is bounded (but otherwise arbitrary):
||N(:, i)||1 ≤ ε, for all i,
and some dependence on some condition number is unavoidable:
CUHK Recent Advances in NMF 26
![Page 49: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/49.jpg)
Near-Separable NMF: Noise and Conditioning
We will assume that the noise is bounded (but otherwise arbitrary):
||N(:, i)||1 ≤ ε, for all i,
and some dependence on some condition number is unavoidable:
CUHK Recent Advances in NMF 26
![Page 50: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/50.jpg)
Fast and Robust Algorithm for Separable NMF
M = UV = U [Ir, V′] = [U,UV ′] Π,
where V ′ ≥ 0 and its column sum to one.
Observation 1. The maximum of a strongly convex function f over apolytope is attained at a vertex:
max1≤i≤n
f(M(:, i)) = max1≤i≤r
f(U(:, i)).
Observation 2. This property is robust: for M = M +N , if M(:, i) is thecolumn of M maximizing f , then there exists p such that
||M(:, i)− U(:, p)||2 ≤ O(ε κ2(U)
).
Observation 3. Pre-multiplying M preserves separability:
PM = (PU) [Ir, V′] Π.
CUHK Recent Advances in NMF 27
![Page 51: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/51.jpg)
Fast and Robust Algorithm for Separable NMF
M = UV = U [Ir, V′] = [U,UV ′] Π,
where V ′ ≥ 0 and its column sum to one.
Observation 1. The maximum of a strongly convex function f over apolytope is attained at a vertex:
max1≤i≤n
f(M(:, i)) = max1≤i≤r
f(U(:, i)).
Observation 2. This property is robust: for M = M +N , if M(:, i) is thecolumn of M maximizing f , then there exists p such that
||M(:, i)− U(:, p)||2 ≤ O(ε κ2(U)
).
Observation 3. Pre-multiplying M preserves separability:
PM = (PU) [Ir, V′] Π.
CUHK Recent Advances in NMF 27
![Page 52: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/52.jpg)
Fast and Robust Algorithm for Separable NMF
M = UV = U [Ir, V′] = [U,UV ′] Π,
where V ′ ≥ 0 and its column sum to one.
Observation 1. The maximum of a strongly convex function f over apolytope is attained at a vertex:
max1≤i≤n
f(M(:, i)) = max1≤i≤r
f(U(:, i)).
Observation 2. This property is robust: for M = M +N , if M(:, i) is thecolumn of M maximizing f , then there exists p such that
||M(:, i)− U(:, p)||2 ≤ O(ε κ2(U)
).
Observation 3. Pre-multiplying M preserves separability:
PM = (PU) [Ir, V′] Π.
CUHK Recent Advances in NMF 27
![Page 53: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/53.jpg)
Successive Projection Algorithm (SPA)0: R = M .For i = 1 : r% Identify the column of R maximizing ||.||2.1: j∗ = argmaxj ||R(:, j)||2 and U(:, i) = M(:, j∗).% Project all columns of R onto its orthogonal complement.
2: R←(I − R(:,j∗)R(:,j∗)T
||R(:,j∗)||22
)R.
endIt is essentially modified Gram-Schmidt with column pivoting.
Theorem ([GV12]). If ε ≤ O(σmin(U)√rκ2(U)
), SPA leads to an NMF (U, V ) s.t.
||M − UV ||2 ≤ O(εκ2(U)
).
Advantages. Extremely fast, no parameter.
Drawbacks. Requires U to be full rank; bound is weak.[GV12] G., Vavasis, Fast and Robust Recursive Algorithms for Separable Nonnegative MatrixFactorization, to appear in IEEE Trans. Patt. Anal. Mach. Intell.
CUHK Recent Advances in NMF 28
![Page 54: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/54.jpg)
Successive Projection Algorithm (SPA)0: R = M .For i = 1 : r% Identify the column of R maximizing ||.||2.1: j∗ = argmaxj ||R(:, j)||2 and U(:, i) = M(:, j∗).% Project all columns of R onto its orthogonal complement.
2: R←(I − R(:,j∗)R(:,j∗)T
||R(:,j∗)||22
)R.
endIt is essentially modified Gram-Schmidt with column pivoting.
Theorem ([GV12]). If ε ≤ O(σmin(U)√rκ2(U)
), SPA leads to an NMF (U, V ) s.t.
||M − UV ||2 ≤ O(εκ2(U)
).
Advantages. Extremely fast, no parameter.
Drawbacks. Requires U to be full rank; bound is weak.[GV12] G., Vavasis, Fast and Robust Recursive Algorithms for Separable Nonnegative MatrixFactorization, to appear in IEEE Trans. Patt. Anal. Mach. Intell.
CUHK Recent Advances in NMF 28
![Page 55: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/55.jpg)
Pre-conditioning for More Robust SPA
Observation 3. Pre-multiplying M preserves separability:
P M = P (U [Ir, V′] +N) = (PU) [I, V ′] + PN.
Ideally, P = U−1 so that κ(PU) = 1.Solving the minimum volume ellipsoid centered at the origin andcontaining all the columns of M (which is SDP representable)
minA∈Sr+
log det(A)−1 s.t. miTAmi ≤ 1 ∀ i,
allows to approximate U−1: in fact, A∗ ≈ U−TU−1.
Theorem ([GV13]). If ε ≤ O(σmin(U)r√r
), preconditioned SPA leads to an
NMF (U, V ) s.t. ||M − UV ||2 ≤ O (εκ(U)).
[GV13] G., Vavasis, SDP-based Preconditioning for More Robust Near-Separable NMF,arXiv:1310.2273.
CUHK Recent Advances in NMF 29
![Page 56: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/56.jpg)
Pre-conditioning for More Robust SPA
Observation 3. Pre-multiplying M preserves separability:
P M = P (U [Ir, V′] +N) = (PU) [I, V ′] + PN.
Ideally, P = U−1 so that κ(PU) = 1.Solving the minimum volume ellipsoid centered at the origin andcontaining all the columns of M (which is SDP representable)
minA∈Sr+
log det(A)−1 s.t. miTAmi ≤ 1 ∀ i,
allows to approximate U−1: in fact, A∗ ≈ U−TU−1.
Theorem ([GV13]). If ε ≤ O(σmin(U)r√r
), preconditioned SPA leads to an
NMF (U, V ) s.t. ||M − UV ||2 ≤ O (εκ(U)).
[GV13] G., Vavasis, SDP-based Preconditioning for More Robust Near-Separable NMF,arXiv:1310.2273.
CUHK Recent Advances in NMF 29
![Page 57: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/57.jpg)
Pre-conditioning for More Robust SPA
Observation 3. Pre-multiplying M preserves separability:
P M = P (U [Ir, V′] +N) = (PU) [I, V ′] + PN.
Ideally, P = U−1 so that κ(PU) = 1.Solving the minimum volume ellipsoid centered at the origin andcontaining all the columns of M (which is SDP representable)
minA∈Sr+
log det(A)−1 s.t. miTAmi ≤ 1 ∀ i,
allows to approximate U−1: in fact, A∗ ≈ U−TU−1.
Theorem ([GV13]). If ε ≤ O(σmin(U)r√r
), preconditioned SPA leads to an
NMF (U, V ) s.t. ||M − UV ||2 ≤ O (εκ(U)).
[GV13] G., Vavasis, SDP-based Preconditioning for More Robust Near-Separable NMF,arXiv:1310.2273.
CUHK Recent Advances in NMF 29
![Page 58: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/58.jpg)
Pre-conditioning for More Robust SPA
Observation 3. Pre-multiplying M preserves separability:
P M = P (U [Ir, V′] +N) = (PU) [I, V ′] + PN.
Ideally, P = U−1 so that κ(PU) = 1.Solving the minimum volume ellipsoid centered at the origin andcontaining all the columns of M (which is SDP representable)
minA∈Sr+
log det(A)−1 s.t. miTAmi ≤ 1 ∀ i,
allows to approximate U−1: in fact, A∗ ≈ U−TU−1.
Theorem ([GV13]). If ε ≤ O(σmin(U)r√r
), preconditioned SPA leads to an
NMF (U, V ) s.t. ||M − UV ||2 ≤ O (εκ(U)).
[GV13] G., Vavasis, SDP-based Preconditioning for More Robust Near-Separable NMF,arXiv:1310.2273.
CUHK Recent Advances in NMF 29
![Page 59: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/59.jpg)
Synthetic data sets� Each entry of U ∈ R20×20
+ uniform in [0, 1]; each column normalized.� The other columns of M are the middle points of the columns of U
(hence there are(202
)= 190).
� The noise moves the middle points toward the outside of the convexhull of the column of U .
Figure: Example for r = 3.
CUHK Recent Advances in NMF 30
![Page 60: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/60.jpg)
Results for the synthetic data sets
Figure: Average of the percentage of columns correctly extracted depending onthe noise level (for each noise level, 10 matrices are generated).
CUHK Recent Advances in NMF 31
![Page 61: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/61.jpg)
Hubble telescope hyperspectral image
Figure: Sample of images for the Hubble telescope hyperspectral image with100 spectral bands and 128× 128 pixels.
CUHK Recent Advances in NMF 32
![Page 62: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/62.jpg)
Hubble telescope hyperspectral image
Figure: Spectral signatures extracted by SPA, corresponding to constitutivematerials (matrix U with κ(U) = 115).
CUHK Recent Advances in NMF 32
![Page 63: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/63.jpg)
Hubble telescope hyperspectral image
Figure: Reconstructed abundance maps (matrix H).
CUHK Recent Advances in NMF 32
![Page 64: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/64.jpg)
Hubble telescope with blur and noise
Figure: Sample of images for the Hubble telescope.
CUHK Recent Advances in NMF 33
![Page 65: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/65.jpg)
Hubble telescope with blur and noise
Figure: Spectral signatures extracted by SPA, corresponding to constitutivematerials (matrix U).
CUHK Recent Advances in NMF 33
![Page 66: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/66.jpg)
Hubble telescope with blur and noise
Figure: Reconstructed abundance maps (matrix V ). With the blur and noise,SPA fails to identify good columns.
CUHK Recent Advances in NMF 33
![Page 67: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/67.jpg)
Hubble telescope with blur and noise
Figure: Spectral signatures extracted by preconditioned SPA, correspondingto constitutive materials (matrix U).
CUHK Recent Advances in NMF 34
![Page 68: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/68.jpg)
Hubble telescope with blur and noise
Figure: Reconstructed abundance maps (matrix V ). With the blur and noise,preconditioned SPA is able to identify the right columns.
CUHK Recent Advances in NMF 34
![Page 69: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/69.jpg)
Conclusion
1. Nonnegative matrix factorization (NMF)
I Easily interpretable linear dimensionality reduction technique fornonnegative data, with many applications
2. Nonnegative matrix underapproximation (NMU)
I Underapproximations allow to solve NMF recursivelyI additional sparsity and regularity constraints leads to better
separation by parts
3. Separable NMF
I Separability makes NMF problems efficiently solvableI Need for fast, practical and robust algorithmsI SPA, a recursive algorithm for separable NMFI SDP preconditionning can be used to make SPA significantly
more robust to noise
CUHK Recent Advances in NMF 35
![Page 70: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/70.jpg)
Conclusion
1. Nonnegative matrix factorization (NMF)
I Easily interpretable linear dimensionality reduction technique fornonnegative data, with many applications
2. Nonnegative matrix underapproximation (NMU)
I Underapproximations allow to solve NMF recursivelyI additional sparsity and regularity constraints leads to better
separation by parts
3. Separable NMF
I Separability makes NMF problems efficiently solvableI Need for fast, practical and robust algorithmsI SPA, a recursive algorithm for separable NMFI SDP preconditionning can be used to make SPA significantly
more robust to noise
CUHK Recent Advances in NMF 35
![Page 71: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/71.jpg)
Conclusion
1. Nonnegative matrix factorization (NMF)
I Easily interpretable linear dimensionality reduction technique fornonnegative data, with many applications
2. Nonnegative matrix underapproximation (NMU)
I Underapproximations allow to solve NMF recursivelyI additional sparsity and regularity constraints leads to better
separation by parts
3. Separable NMF
I Separability makes NMF problems efficiently solvableI Need for fast, practical and robust algorithmsI SPA, a recursive algorithm for separable NMFI SDP preconditionning can be used to make SPA significantly
more robust to noise
CUHK Recent Advances in NMF 35
![Page 72: Some Recent Advances in Nonnegative Matrix Factorization and … · 2013. 11. 21. · [GG10] G., Glineur, Using Underapproximations for Sparse Nonnegative Matrix Factorization, Pattern](https://reader035.vdocument.in/reader035/viewer/2022071109/5fe39a16fd4e890a280aa921/html5/thumbnails/72.jpg)
Thank you for your attention
and Thanks to Bob!
Code and papers available onhttps://sites.google.com/site/nicolasgillis/.
CUHK Recent Advances in NMF 36