lecture 18 varimax factors and empircal orthogonal functions
TRANSCRIPT
![Page 1: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/1.jpg)
Lecture 18
Varimax Factorsand
Empircal Orthogonal Functions
![Page 2: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/2.jpg)
SyllabusLecture 01 Describing Inverse ProblemsLecture 02 Probability and Measurement Error, Part 1Lecture 03 Probability and Measurement Error, Part 2 Lecture 04 The L2 Norm and Simple Least SquaresLecture 05 A Priori Information and Weighted Least SquaredLecture 06 Resolution and Generalized InversesLecture 07 Backus-Gilbert Inverse and the Trade Off of Resolution and VarianceLecture 08 The Principle of Maximum LikelihoodLecture 09 Inexact TheoriesLecture 10 Nonuniqueness and Localized AveragesLecture 11 Vector Spaces and Singular Value DecompositionLecture 12 Equality and Inequality ConstraintsLecture 13 L1 , L∞ Norm Problems and Linear ProgrammingLecture 14 Nonlinear Problems: Grid and Monte Carlo Searches Lecture 15 Nonlinear Problems: Newton’s Method Lecture 16 Nonlinear Problems: Simulated Annealing and Bootstrap Confidence Intervals Lecture 17 Factor AnalysisLecture 18 Varimax Factors, Empircal Orthogonal FunctionsLecture 19 Backus-Gilbert Theory for Continuous Problems; Radon’s ProblemLecture 20 Linear Operators and Their AdjointsLecture 21 Fréchet DerivativesLecture 22 Exemplary Inverse Problems, incl. Filter DesignLecture 23 Exemplary Inverse Problems, incl. Earthquake LocationLecture 24 Exemplary Inverse Problems, incl. Vibrational Problems
![Page 3: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/3.jpg)
Purpose of the Lecture
Choose Factors Satisfying A Priori Information of Spikiness
(varimax factors)
Use Factor Analysis to DetectPatterns in data
(EOF’s)
![Page 4: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/4.jpg)
Part 1: Creating Spiky Factors
![Page 5: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/5.jpg)
can we find “better” factors
that those returned by svd()
?
![Page 6: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/6.jpg)
mathematically
S = CF = C’ F’with F’ = M F and C’ = M-1 Cwhere M is any P×P matrix with an inverse
must rely on prior information to choose M
![Page 7: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/7.jpg)
one possible type of prior information
factors should contain mainly just a few elements
![Page 8: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/8.jpg)
example of rock and mineralsrocks contain minerals
minerals contain elements
Mineral Composition
Quartz SiO2
Rutile TiO2
Anorthite CaAl2Si2O8
Fosterite Mg2SiO4
![Page 9: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/9.jpg)
example of rock and mineralsrocks contain minerals
minerals contain elements
Mineral Composition
Quartz SiO2
Rutile TiO2
Anorthite CaAl2Si2O8
Fosterite Mg2SiO4
factors
most of these minerals are “simple” in the sense that each contains just a few elements
![Page 10: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/10.jpg)
spiky factors
factors containing mostly just a few elements
![Page 11: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/11.jpg)
How to quantify spikiness?
![Page 12: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/12.jpg)
variance as a measure of spikiness
![Page 13: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/13.jpg)
modification for factor analysis
![Page 14: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/14.jpg)
modification for factor analysis
depends on the square, so positive and negative values are treated the same
![Page 15: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/15.jpg)
f(1)= [1, 0, 1, 0, 1, 0]T
is much spikier than
f(2)= [1, 1, 1, 1, 1, 1]T
![Page 16: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/16.jpg)
f(2)=[1, 1, 1, 1, 1, 1]T
is just as spiky as
f(3)= [1, -1, 1, -1, -1, 1]T
![Page 17: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/17.jpg)
“varimax” procedure
find spiky factors without changing P
start with P svd() factors
rotate pairs of them in their plane by angle θto maximize the overall spikiness
![Page 18: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/18.jpg)
fB fA
f’B
f’A
q
![Page 19: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/19.jpg)
determine θ by maximizing
![Page 20: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/20.jpg)
after tedious trig the solution can be shown to be
![Page 21: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/21.jpg)
and the new factors are
in this example A=3 and B=5
![Page 22: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/22.jpg)
now one repeats for every pair of factors
and then iterates the whole process several times
until the whole set of factors is as spiky as possible
![Page 23: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/23.jpg)
f2 f3 f4 f5
f2p f3p f4p f5p
Old New
f5f2 f3 f4 f’5f’2f’3 f’4
SiO2
TiO2
Al2O3
FeOtotal
MgO
CaO
Na2O
K2O
example: Atlantic Rock dataset
![Page 24: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/24.jpg)
f2 f3 f4 f5
f2p f3p f4p f5p
Old New
f5f2 f3 f4 f’5f’2f’3 f’4
SiO2
TiO2
Al2O3
FeOtotal
MgO
CaO
Na2O
K2O
example: Atlantic Rock dataset
not so spiky
![Page 25: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/25.jpg)
f2 f3 f4 f5
f2p f3p f4p f5p
Old New
f5f2 f3 f4 f’5f’2f’3 f’4
SiO2
TiO2
Al2O3
FeOtotal
MgO
CaO
Na2O
K2O
example: Atlantic Rock dataset
spiky
![Page 26: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/26.jpg)
Part 2: Empirical Orthogonal Functions
![Page 27: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/27.jpg)
row number in the sample matrix could be meaningful
example: samples collected at a succession of times
time
![Page 28: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/28.jpg)
column number in the sample matrix could be meaningful
example: concentration of the same chemical element at a sequence of positions
distance
![Page 29: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/29.jpg)
S = CFbecomes
![Page 30: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/30.jpg)
S = CFbecomes
distance dependence time dependence
![Page 31: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/31.jpg)
S = CFbecomes
each loading: a temporal pattern of variability of the corresponding factor
each factor:a spatial pattern of variability of the element
![Page 32: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/32.jpg)
S = CFbecomes
there are P patterns and they are sorted into order of importance
![Page 33: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/33.jpg)
S = CFbecomes
factors now called EOF’s (empirical orthogonal functions)
![Page 34: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/34.jpg)
example 1
hypothetical mountain profiles
what are the most important spatial patterns
that characterize mountain profiles
![Page 35: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/35.jpg)
this problem has space but not time
s( xj , i ) = Σk=1p cki f (k)(xj)
![Page 36: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/36.jpg)
this problem has space but not time
s( xj , i ) = Σk=1p cki f (k)(xj )factors are spatial patterns that add together to make mountain profiles
![Page 37: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/37.jpg)
0 5 100
5
i
s 1
0 5 100
5
i
s 2
0 5 100
5
i
s 3
0 5 100
5
i
s 4
0 5 100
5
i
s 5
0 5 100
5
i
s 6
0 5 100
5
i
s 7
0 5 100
5
i
s 80 5 10
0
5
i
s 9
0 5 100
5
i
s 10
0 5 100
5
i
s 11
0 5 100
5
is 12
0 5 100
5
i
s 13
0 5 100
5
i
s 14
0 5 100
5
i
s 15
![Page 38: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/38.jpg)
0 5 100
5
i
s 1
0 5 100
5
i
s 2
0 5 100
5
i
s 3
0 5 100
5
i
s 4
0 5 100
5
i
s 5
0 5 100
5
i
s 6
0 5 100
5
i
s 7
0 5 100
5
i
s 80 5 10
0
5
i
s 9
0 5 100
5
i
s 10
0 5 100
5
i
s 11
0 5 100
5
is 12
0 5 100
5
i
s 13
0 5 100
5
i
s 14
0 5 100
5
i
s 15
elements:elevations ordered by distance along profile
![Page 39: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/39.jpg)
0 5 10-1
0
1
i
F1
0 5 10-1
0
1
i
F2
0 5 10-1
0
1
i
F3
EOF 3EOF 2EOF 1
index, i index, i index, i
f i (1)
f i (2)
f i (3)
λ1 = 38 λ2 = 13 λ3 = 7
![Page 40: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/40.jpg)
factor loading, Ci2
factor loa
ding, C i3
![Page 41: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/41.jpg)
example 2
spatial-temporal patterns
(synthetic data)
![Page 42: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/42.jpg)
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
the data
![Page 43: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/43.jpg)
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
the data
spatial
pattern at
a single
time
x
y
t=1
![Page 44: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/44.jpg)
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
the data
time
![Page 45: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/45.jpg)
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
16 17 18 19 20
21 22 23 24 25
the data
![Page 46: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/46.jpg)
4 6 19 1 21 3 6
461912136s
need to unfold each 2D image into vector
![Page 47: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/47.jpg)
0 5 10 15 20 250
50
100
150
200
index, iλi
p=3
![Page 48: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/48.jpg)
1 2 3EOF 1 EOF 2 EOF 30 5 10 15 20 25
-100
-50
0
0 5 10 15 20 25-20
0
20
0 5 10 15 20 25-50
0
50
load
ng1
load
ng 2lo
adng
3(A)
(B)
time, ttime, ttime, t
![Page 49: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/49.jpg)
example 3
spatial-temporal patterns
(actual data)
sea surface temperature in the Pacific Ocean
![Page 50: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/50.jpg)
29S
29N
124E 290E
latitude
longitude
equatorial Pacific Ocean
sea surface temperature (black = warm)
CAC Sea Surface Temperature
![Page 51: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/51.jpg)
![Page 52: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/52.jpg)
the image is 30 by 84 pixels in size, or 2520 pixels total
to use svd(), the image must be unwrapped into a vector of length 2520
![Page 53: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/53.jpg)
2520 positions in the equatorial Pacific ocean
399
tim
es
“element” means temperature
![Page 54: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/54.jpg)
sing
ular
val
ues,
Sii
index, i
singular values
![Page 55: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/55.jpg)
sing
ular
val
ues,
Sii
index, i
singular values
no clear cutoff for P, but the first 12 singular values are considerably larger than the rest
![Page 56: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/56.jpg)
![Page 57: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/57.jpg)
![Page 58: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/58.jpg)
![Page 59: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/59.jpg)
![Page 60: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/60.jpg)
![Page 61: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/61.jpg)
![Page 62: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/62.jpg)
using SVD to approximate data
![Page 63: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/63.jpg)
S=CMFM
S=CPFP
S≈CP’FP’
With M EOF’s, the data is fit exactly
With P chosen to exclude only zero singular values, the data is fit exactly
With P’<P, small non-zero singular values are excluded too, and the data is fit only approximately
![Page 64: Lecture 18 Varimax Factors and Empircal Orthogonal Functions](https://reader036.vdocument.in/reader036/viewer/2022062320/56649cbb5503460f949834bd/html5/thumbnails/64.jpg)
A) Original B) Based on first 5 EOF’s