non-linear dimensionality reduction
DESCRIPTION
Non-Linear Dimensionality Reduction. Presented by: Johnathan Franck Mentor: Alex Cloninger. Outline. Different Representations 5 Techniques Principal component analysis (PCA)/ Multi-dimensional scaling (MDS) Sammons non-linear mapping Isomap Kernel PCA Locally linear embedding. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/1.jpg)
Non-Linear Dimensionality ReductionPresented by: Johnathan FranckMentor: Alex Cloninger
![Page 2: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/2.jpg)
OutlineDifferent Representations5 Techniques
◦ Principal component analysis (PCA)/Multi-dimensional scaling (MDS)
◦ Sammons non-linear mapping◦ Isomap◦ Kernel PCA◦ Locally linear embedding
![Page 3: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/3.jpg)
Different viewpoints
![Page 4: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/4.jpg)
Principal Component Analysis/Multidimensional Scaling
Singular Value Decomposition of A:
![Page 5: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/5.jpg)
PCA/MDS FormulationX – reduced dimension Y – higher dimensionW - Orthogonal
Distance Preserved Rotation
Preserve Gram Matrix: S
Linear Algebra Optimization:
Singular Value decomposition of Y equivalent toEigenvalue decomposition of gram matrix
![Page 6: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/6.jpg)
Swiss Roll Open Box
![Page 7: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/7.jpg)
Sammons Nonlinear mapping
Weight distances differently
![Page 8: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/8.jpg)
Swiss Roll Open Box
![Page 9: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/9.jpg)
Isomap
• Build graph with K-neighbors/ε-ball
• Weight graph with Euclidean distance
• Compute pairwise distances with Dijkstra’s algorithm
Approximate geodesic
![Page 10: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/10.jpg)
Swiss Roll Open Box
![Page 11: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/11.jpg)
Kernel PCAWhat other distance functions?
Mercer’s theorem
![Page 12: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/12.jpg)
Swiss Roll Open Box
![Page 13: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/13.jpg)
Local Linear EmbeddingPreserve relation to local points
![Page 14: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/14.jpg)
Swiss Roll Open Box
![Page 15: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/15.jpg)
Original 3 dimensions
PCA
Sammon’s nonlinear mapping
Isomap
Kernel PCA
Locally Linear Embedding
![Page 17: Non-Linear Dimensionality Reduction](https://reader035.vdocument.in/reader035/viewer/2022062323/56816245550346895dd281a9/html5/thumbnails/17.jpg)
SourcesImages
◦Nonlinear Dimensionality reduction By John Lee and Michael Verleyson
◦http://blog.pluskid.org/wp-content/uploads/2010/05/isomap-graph.png
◦http://imagineatness.files.wordpress.com/2011/12/gaussian.png
◦http://scikit-learn.org/0.11/_images/plot_kernel_pca_1.png
◦http://www.cs.nyu.edu/~roweis/lle/images/llef2med.gif
◦http://upload.wikimedia.org/wikipedia/commons/b/bb/SOMsPCA.PNG