an exposition of bootstrap and permutation tests for principal components analyses derek beaton...
TRANSCRIPT
![Page 1: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/1.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
![Page 2: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/2.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
![Page 3: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/3.jpg)
Kinds of Data
9 6 7 4 5 5 2 2 7 5 1 9 3 3 1 2 2
8 5 8 1 1 5 4 2 3 8 2 9 1 5 1 2 2
… … … … … … … … … … … … … … … … …
2 1 2 2 0 0 2 7 2 6 8 3 6 6 2 6 4
2 3 1 4 5 1 3 1 5 6 7 1 3 4 5 7 8
![Page 4: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/4.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
![Page 5: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/5.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
![Page 6: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/6.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
![Page 7: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/7.jpg)
An ExPosition of Bootstrap and Permutation tests for Principal
Components Analyses
Derek Beaton
Joseph Dunlop
Hervé Abdi
Daniel Faso
![Page 8: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/8.jpg)
Outline
• We have a lot to talk about!
– Principal Components Analysis (PCA)
–Multiple Correspondence Analysis (MCA)
– Bootstrap
– Permutation
![Page 9: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/9.jpg)
The SVD
• We have a lot to talk about!
– Principal Components Analysis (PCA)
–Multiple Correspondence Analysis (MCA)
– Bootstrap
– Permutation
![Page 10: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/10.jpg)
Resampling
• We have a lot to talk about!
– Principal Components Analysis (PCA)
–Multiple Correspondence Analysis (MCA)
– Bootstrap
– Permutation
![Page 11: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/11.jpg)
An ExPosition of
• The SVD
• Resampling
![Page 12: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/12.jpg)
An ExPosition of
• The SVD
• Resampling
![Page 13: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/13.jpg)
The SVD
• Root of all evil most multivariate
techniques
• Is just an eigendecomposition*
• Analyses or pre-analyses
![Page 14: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/14.jpg)
Orthogonawesome
• The SVD is for rectangular tables
• Does two things
– Finds the major source of variance
– Finds orthogonal slices of your data
![Page 15: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/15.jpg)
PCA = SVD
• Center & Scale your data
• Then SVD
• = PCA!
• Quick illustration
![Page 16: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/16.jpg)
Data
![Page 17: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/17.jpg)
Centered & Normed
![Page 18: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/18.jpg)
Find variance
![Page 19: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/19.jpg)
How?
![Page 20: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/20.jpg)
How?
![Page 21: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/21.jpg)
How?
![Page 22: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/22.jpg)
That’s a component!
![Page 23: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/23.jpg)
PCA!
![Page 24: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/24.jpg)
And variables
![Page 25: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/25.jpg)
PCA!
![Page 26: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/26.jpg)
And variables
![Page 27: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/27.jpg)
PCA!
![Page 28: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/28.jpg)
PCA!
![Page 29: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/29.jpg)
Usual visual
![Page 30: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/30.jpg)
An ExPosition of
• The SVD
• Resampling
![Page 31: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/31.jpg)
Resampling
• Why?
![Page 32: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/32.jpg)
Resampling
• Why?
– Provides a null
– Provides a distribution
– Provides intervals
![Page 33: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/33.jpg)
First: Folklore
• Require > 200 (Guilford, 1954) or >
250 (Cattell, 1978) observations
• Require 5:1 observations:measures
ratio (Gorsuch, 1983)
![Page 34: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/34.jpg)
More Folklore
• Keep components with eigen values
> 1
• Scree/elbow “tests”
![Page 35: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/35.jpg)
Fixing Folklore
• High dimensional low sample size
can be OK (Jung & Marron, 2009; Chi
2012)
• Power derived like MANOVA (in some
cases; D’Amico et al., 2001)
![Page 36: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/36.jpg)
Fixing Folklore
• Sometimes all eigens < 1
![Page 37: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/37.jpg)
We need a null
• Resampling can do that!
• Bootstrap (Efron & Tibshirani, 1983,
Hesterberg 2011, Chernick 2008)
• Permutation (Berry et al., 2011)
– But really, Fisher & Student did this first.
![Page 38: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/38.jpg)
Permutation
• Scrambles data
• An exact test of the H0
– Tests an omnibus effect
– Tests each component
![Page 39: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/39.jpg)
Permutation
Obs. W Y1 1 162 3 103 4 124 4 45 5 86 7 10
r = -0.5
![Page 40: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/40.jpg)
Permutation
Obs. W Obs. Y1 1 1 162 3 2 103 4 3 124 4 4 45 5 5 86 7 6 10
![Page 41: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/41.jpg)
Permutation
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 42: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/42.jpg)
Permutation
Obs. W1 12 33 44 45 56 7
Obs. Y6 105 83 124 41 162 10
![Page 43: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/43.jpg)
Permutation
Obs. W1 12 33 44 45 56 7
Obs. Y6 105 83 124 41 162 10
![Page 44: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/44.jpg)
Permutation
“Obs.”
W Yperm
1 1 102 3 83 4 124 4 45 5 166 7 10
![Page 45: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/45.jpg)
Permutation
“Obs.”
W Yperm
1 1 102 3 83 4 124 4 45 5 166 7 10
r = 0.2
![Page 46: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/46.jpg)
Permutation in R
• R> sample(1:4,4,FALSE)
2 3 1 4
• R> sample(1:4,4,FALSE)
3 2 1 4
• R> sample(1:4,4,FALSE)
4 3 2 1
• R> sample(1:4,4,FALSE)
3 4 1 2
![Page 47: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/47.jpg)
Bootstrap
• Confidence intervals
–Which measures are different from each
other
• t-like tests
–Which measures are important to
components?
![Page 48: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/48.jpg)
Bootstrap
Obs. W Y1 1 162 3 103 4 124 4 45 5 86 7 10
r = -0.5
![Page 49: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/49.jpg)
Bootstrap
Obs. W Obs. Y1 1 1 162 3 2 103 4 3 124 4 4 45 5 5 86 7 6 10
![Page 50: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/50.jpg)
Bootstrap
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 51: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/51.jpg)
Bootstrap
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 52: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/52.jpg)
Bootstrap
Obs. W1 15 55 56 75 53 4
Obs. Y1 165 85 86 105 83 12
![Page 53: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/53.jpg)
Bootstrap
Obs. W1 15 55 56 75 53 4
Obs. Y1 165 85 86 105 83 12
![Page 54: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/54.jpg)
Bootstrap
Obs.
Wboo
t
Yboot
1 1 165 5 85 5 86 7 105 5 83 4 12
r = -0.79
![Page 55: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/55.jpg)
Bootstrap in R
• R> sample(1:4,4,TRUE)
1 2 4 4
• R> sample(1:4,4,TRUE)
4 4 1 4
• R> sample(1:4,4,TRUE)
4 1 2 1
• R> sample(1:4,4,TRUE)
4 3 2 1
![Page 56: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/56.jpg)
Simple Resampling Examples
• We have permutation and bootstrap
tests of just a correlation
![Page 57: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/57.jpg)
Today’s data
• Simulated Paranoia Scale data
– Some of us have seen it!
• Control group, Social Anxiety,
Psychosis
• 20 questions on sub-clinical paranoia
• 5 responses – none to a lot.
![Page 58: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/58.jpg)
Time for PCA!
• Go to code for most of PCA. Return
here before the “inference battery”
![Page 59: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/59.jpg)
Boot & Perm in PCA
• Permutation of components
![Page 60: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/60.jpg)
Permute for Components
• Scramble up the data
![Page 61: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/61.jpg)
Permute for Components
• Scramble up the data
![Page 62: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/62.jpg)
Permutation
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 63: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/63.jpg)
Permutation
Obs. W1 12 33 44 45 56 7
Obs. Y6 105 83 124 41 162 10
![Page 64: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/64.jpg)
Permute for Components
• Perform the analysis again
• Keep track of singular or eigen
values (variance)
• Keep only the ones that explain more
than chance.
![Page 65: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/65.jpg)
Boot & Perm in PCA
• Bootstrap ratios
![Page 66: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/66.jpg)
Bootstrap for Variables
• Find which are significant
![Page 67: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/67.jpg)
Bootstrap
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 68: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/68.jpg)
Bootstrap
Obs. W1 12 33 44 45 56 7
Obs. Y1 162 103 124 45 86 10
![Page 69: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/69.jpg)
Bootstrap
Obs. W1 15 55 56 75 53 4
Obs. Y1 165 85 86 105 83 12
![Page 70: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/70.jpg)
Bootstrap for Variables
• Perform analysis again
• Keep track of how much variables
change their position
• Compute a t-value
• Keep those above a threshold (e.g.,
1.96).
![Page 71: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/71.jpg)
And back to PCA!
• See the inference results from the
code.
• Return to the slides after PCA and
before MCA
![Page 72: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/72.jpg)
But, Derek Disagrees
• Like always
![Page 73: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/73.jpg)
Are the data categorical?
• If so, how do we “PCA” with
categories?
![Page 74: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/74.jpg)
Today’s data
• Simulated Paranoia Scale data
– Some of us have seen it!
• Control group, Social Anxiety,
Psychosis
• 20 questions on sub-clinical paranoia
• 5 responses – none to a lot.
![Page 75: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/75.jpg)
Today’s data
• Simulated Paranoia Scale data
– Some of us have seen it!
• Control group, Social Anxiety,
Psychosis
• 20 questions on sub-clinical paranoia
• 5 responses – none to a lot.
![Page 76: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/76.jpg)
Multiple Correspondence Analysis
• What is it?
• Why haven’t I heard of it before?
![Page 77: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/77.jpg)
MCA
• What is it?
![Page 78: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/78.jpg)
MCA
Q1 Q21 13 2… …… …… …4 2
![Page 79: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/79.jpg)
MCA
Q1 Q21 13 2… …… …… …4 2
1 2 3 41 0 0 00 0 1 0… … … …… … … …… … … …0 0 0 1
![Page 80: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/80.jpg)
MCA
Q1 Q21 13 2… …… …… …4 2
1 2 3 41 0 0 00 1 0 0… … … …… … … …… … … …0 1 0 0
![Page 81: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/81.jpg)
MCA
1 2 3 41 0 0 00 1 0 0… … … …… … … …… … … …0 1 0 0
1 2 3 41 0 0 00 0 1 0… … … …… … … …… … … …0 0 0 1
Q1 Q21 13 2… …… …… …4 2
![Page 82: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/82.jpg)
MCA
• Many perspectives
• PCA, CA, etc…
![Page 83: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/83.jpg)
MCA
• Short version:
– Compute the marginal probabilities
– Compute an observed and expected
matrix
• Subtract
–Multiply by the marginal probabilities.
![Page 84: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/84.jpg)
That’s familiar!
• χ2 so far!
![Page 85: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/85.jpg)
MCA
• χ2 preprocessed disjunctive table
• Put through SVD
![Page 86: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/86.jpg)
Back to code!
![Page 87: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/87.jpg)
Conclusions
• How many people are “enough”?
• How many variables are “too many”?
• How many iterations are “enough”?
![Page 88: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/88.jpg)
Enough is enough!
• It’s hard to tell, but here are some
suggestions
![Page 89: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/89.jpg)
Conclusions
• When to use PCA
![Page 90: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/90.jpg)
PCA is for quantitative
• Reaction Times
• Hits & False alarms
• Eye tracking
• fMRI
• Surveys
![Page 91: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/91.jpg)
Conclusions
• When to use MCA
![Page 92: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/92.jpg)
MCA
• Demographics data
• Genetics
• Preference
• Surveys
![Page 93: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/93.jpg)
Conclusions
• Why resampling?
![Page 94: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/94.jpg)
We need tests
• Not folklore!
– Some of it’s not bad though
• We need to know what is reliable
![Page 95: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/95.jpg)
Big data can be tough
• Permutation
– Focus on only significant components
• Bootstrap
– Focus on only significant contributors
![Page 96: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/96.jpg)
What about those groups?
• There are between-group (a la,
ANOVA) approaches for PCA & MCA
![Page 97: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/97.jpg)
Barycentric (Discriminant)
• Barycentric Discriminant Analysis
(BADA)
– PCA for between groups
• Discriminant Correspondence
Analysis
–MCA for between groups
![Page 98: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/98.jpg)
Fin
• Questions, comments, complaints?
– If we don’t have time up here, we’ll be
around
– Please feel free!
![Page 99: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/99.jpg)
General wrap up
• We covered a lot in 2.5 hours
• We hope it was worth it!
![Page 100: An ExPosition of Bootstrap and Permutation tests for Principal Components Analyses Derek Beaton Joseph Dunlop Hervé Abdi](https://reader035.vdocument.in/reader035/viewer/2022062500/5697bfea1a28abf838cb71bf/html5/thumbnails/100.jpg)
Fin fin
• Thanks for sticking around
• If you have any questions about
either workshop – please find us
– Or email us!