ece 484 digital image processing lec 16 - transform domain ......ece 484 digital image processing,...
TRANSCRIPT
ECE 484 Digital Image Processing Lec 16 - Transform Domain Image Processing I
Dimension Reduction
Zhu LiDept of CSEE, UMKC
Office: FH560E, Email: [email protected], Ph: x 2346.http://l.web.umkc.edu/lizhu
ECE 484 Digital Image Processing, 2019 p.1
slides created with WPS Office Linux and EqualX equation editor
Outline
Recap: Image Processing tricks so far...
Linear Algebra Refresher SVD Principal Component Analysis (PCA) Laplacian Eigen Map (LEM)
ECE 484 Digital Image Processing, 2019 p.2
Recap of DIP tricks so far
Point Operation (1 x 1)
Linear Filtering (m x m)
Non-Linear Fitlering (m x m)
Freq Domain Filtering (W x H)
ECE 484 Digital Image Processing, 2019 p.3
(much hyped) Deep Leanring
Classificaiton: Pixels to Label
Image Denoising/Enhancement
ECE 484 Digital Image Processing, 2019 p.4
HW-4: Deep Learning Denoising
4 types of noisy images: PSF = Gaussian Blur, Motion Blur Additive Gaussian Noise:
Data Set: NWPU Aerial Image Data Set:
ECE 484 Digital Image Processing, 2019 p.5
HW-4
Task 1: Denoising with VDSR like network prediction: up to you, baseline can just use a gaussian
smoothing residual learning network:
o each conv layer 64 3x3 filterso need to train with the NWPU data set, or any other data set
Task 2: bonus points, Target Decomposition wavelet decomposition (will have a tutorial on this) optimize the wavelet decomposition structure to see which
one works out the best.
ECE 484 Digital Image Processing, 2019 p.6
Image Recognition Problem - Transform
Image Tagging given pixels in Rhxw
predict its label: {airplane, automobiles, bird, cat, dog,....}
Can we find a projection A, s.t.
ECE 484 Digital Image Processing, 2018 p.7
Y AX
=
Outline
Recap: Image Processing tricks so far...
Linear Algebra Refresher SVD Principal Component Analysis (PCA) Laplacian Eigen Map (LEM)
ECE 484 Digital Image Processing, 2019 p.8
Vector and Matrix Notations
Vector
Matrix
p.9ECE 484 Digital Image Processing, 2019
Vector Products
Inner Product
Outer Product
p.10ECE 484 Digital Image Processing, 2019
Matrix-Vector Product
y=Ax
So, y is a linear combination of basis {ak} with weights from x
p.11ECE 484 Digital Image Processing, 2019
Matrix Product
C=AB
Associative: ABC = (AB)C = A(BC)
Distributive: A(B+C) = AB + AC
p.12
A: nxp B: pxm A: nxm=
ECE 484 Digital Image Processing, 2019
Outer Product/Kron
Vector outer product:
Example
ECE 484 Digital Image Processing, 2019 p.13
Matrix Transpose
Transpose
p.14ECE 484 Digital Image Processing, 2019
Matrix Trace and Determinant
Trace:Tr(A): only for nxn square matrix
Determinant: Det(A): The size of volumes spanned by A, All possible linear combinations of a1 and a2
p.15
Det(A) = |2-9| = 7;
ECE 484 Digital Image Processing, 2019
Eigen Values and Eigen Vectors
Definition: for nxn matrix A:
In Matlab: [P, V]=eig(A);
p.16ECE 484 Digital Image Processing, 2019
Eigen Vectors of Symmetric Matrix
p.17ECE 484 Digital Image Processing, 2019
SVD for non square matrix: A mxn:
p.18ECE 484 Digital Image Processing, 2019
SVD as Signal Expansion
p.19
A(mxn) = U(mxm) S(mxn)
V(nxn)
The 1st order SVD approx. of A is:
ECE 484 Digital Image Processing, 2019
SVD approximation of an image
Very easy…function [x]=svd_approx(x0, k)dbg=0;if dbg x0= fix(100*randn(4,6)); k=2;end[u, s, v]=svd(x0);[m, n]=size(s);x = zeros(m, n); sgm = diag(s);for j=1:k x = x + sgm(j)*u(:,j)*v(:,j)'; end
p.20ECE 484 Digital Image Processing, 2019
SVD for Separable Filtering
Take LoG filter for eg. h=fspecial('LoG', 11, 2.0); [u,s,v]=svd(h); h1=s(1,1)*u(:,1)*v(:,1)';
ECE 484 Digital Image Processing, 2019 p.21
h1 is 1-SVD approx of LoG
Many implications forDeep networks acceleration !
NormVector Norm: Length of the vector Euclidean Norm (L2 Norm): norm(x, 2)
Lp norm:
Matrix Norm: Forbenius Norm
p.22ECE 484 Digital Image Processing, 2019
Quadratic Form
Quadratic form f(x)=xTAx in R:
Positive Definite (PD): For non-zero x, xTAx > 0
Positive Semi-Definite (PSD): For non-zero x, xTAx >= 0
Indefinite: Exists x1, x2 non zero, but x1
TAx1 >0, while x2TAx2 < 0;
p.23ECE 484 Digital Image Processing, 2019
Matrix Calculus
Gradient of f(A):
Matrix Gradient Properties
p.24ECE 484 Digital Image Processing, 2019
Hessian of f(X)
p.25ECE 484 Digital Image Processing, 2019
Outline
Recap: About HW-2 Quiz-1
Linear Algebra Refresher SVD Principal Component Analysis (PCA)
ECE 484 Digital Image Processing, 2019 p.26
PCA -Dimension Reduction in Retrieval
A typical image retrieval pipeline
ECE 484 Digital Image Processing, 2019 p.27
Image Formation
Feature Computing
Feature Aggregation Classification
Knowledge/Data Base
e.g, dense SIFT: 12000 x 128 e.g, Fisher Vector: k=64, d=128
Rd -> Rp
Principal Component Analysis
The formulation: for data points {x1, x2,…, } in Rn, find a lower dimensional
representation in Rm, via a projection W,: mxn, s.t., the energy of the data is preserved
ECE 484 Digital Image Processing, 2019 p.28
PCA solution
Take the Lagrangian of the problem
Take the derivative w.r.t. to w, and KKT condition gives us,
This is an Eigen problem, finding projection s.t. it is just a scaling along the scatter matrix eigen vectors.
ECE 484 Digital Image Processing, 2019 p.29
PCA – how to compute
PCA via SVD on the Covariance matrix
ECE 484 Digital Image Processing, 2019 p.30
S: covariance, nxn
2d Data
ECE 484 Digital Image Processing, 2019 p.31
Principal Components
1st principal vector
2nd principal vector
Gives best axis to project Minimum RMS
errorPrincipal vectors
are orthogonal
ECE 484 Digital Image Processing, 2019 p.32
PCA on HoGs
Matlab Implementation of PCA: [A, s, eig_values]=princomp(hogs);
ECE 484 Digital Image Processing, 2019 p.33
HoG basis function
PCA Application in Aggregation
SIFT aggregation Usually a PCA is
done on SIFT features, to reduce the dimension from 128 to say 24, 32. Then a GMM is
trained in R32 space, for FV encoding
Homework-2 Aggregation Fisher Vector
Aggregation of SIFT
ECE 484 Digital Image Processing, 2019 p.34
load../../dataset/cdvs_sift_aggregation_test_data.mat;
[n_sift, kd_sift]=size(gd_sift_cdvs);offs = randperm(n_sift); offs = offs(1:200*2^10);% PCA[A1, s1, lat1]=princomp(double(gd_sift_cdvs(offs,:)));
figure(41); hold on; grid on; stem(lat1, '.'); title('sift pca eigen values');
SIFT PCA
Eigen values
That is why we have kd=[24, 32 48] for SIFT GMM in FV aggregation
ECE 484 Digital Image Processing, 2019 p.35
SIFT PCA Basis Functions
Capturing max variation directions
ECE 484 Digital Image Processing, 2019 p.36
Visualizing SIFT in lower dimensional space
Project SIFTs from 2 images to 2D space
ECE 484 Digital Image Processing, 2019 p.37
Outline
Recap: Part I and Quiz-1
Linear Algebra Refresher SVD Principal Component Analysis (PCA) Laplacian Eigen Map (LEM)
ECE 484 Digital Image Processing, 2019 p.38
Laplacian Eigen Map
Directly compute an embedding {yk} from input {xk} without the explicit projection model A, s.t. Y=AX Objective function
where the nxn affinity matrix W reflects the data points relationship in the original space X.
ECE 484 Digital Image Processing, 2019 p.39
M. Belkin and P. Niyogi. Laplacian Eigenmaps and spectral techniques for embedding and clustering. In Advances in Neural Information Processing Systems, volume 14, pages 585–591, Cambridge, MA, USA, 2002. The MIT Press
Graph Laplacian
Graph Laplacian: L= D - W
p.40ECE 484 Digital Image Processing, 2019
Laplacian Eigenmap
Minimizing the following
Is equivalent to
Where D is the degree matrix (diagonal) with
ECE 484 Digital Image Processing, 2019 p.41
Laplacian Eigen Map Solution
Numerically, solve the eigen problem:
where the first d smallest eigen values’ corresponding eigenvectors, will form a d-dimensional feature of {yk}
ECE 484 Digital Image Processing, 2019 p.42
43
Locally Linear Embedding (LLE) History: S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by Locally Linear
Embedding.Science, 290(5500):2323–2326, 2000Procedure
1. Identify the neighbors of each data point2. Compute weights that best linearly reconstruct the point from its
neighbors
3. Find the low-dimensional embedding vector {yi} which is best reconstructed by the weights determined in Step 2
In matrix form:
ECE 484 Digital Image Processing, 2019
LLE Solution
Similar to Laplacian Eigenmap, the first d eigenvectors provides the best local embedding.
Many successful applications in embedding
ECE 484 Digital Image Processing, 2019 p.44
2d embedding with Laplacian Eigenmap 2d embedding with LLE
Summary
SVD and PCA SVD – non-square matrix decomposition, left transform and
right transform, with scaling in between SVD – as an image decomposition, linear combination of
outer-product basis PCA – eigen values indicate amount of info/energy in each
dimension, PCA – basis are eigen vectors to the covariance matrix LEM – direct data embedding without explicit projection
from graph relationship (expressed as Laplacian) Many applications
ECE 484 Digital Image Processing, 2019 p.45