![Page 1: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/1.jpg)
3. Factor Analysis
Techniques and Applications of Multivariate Analysis
![Page 2: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/2.jpg)
Lecture 3. Factor Analysis (FA)
Lecture3-1 3.1 Introduction 3.2 Orthogonal Factor Model 3.3 Methods of Estimation: PCM vs MLM Lecture3-2 3.4 Factor Rotation 3.5 Factor Scores: WLSM vs RM 3.6 Example : Air-pollution Data 3.7 Strategy for Factor Analysis : Example 9.14
![Page 3: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/3.jpg)
Gaelic English History Arithmetic Algebra Geometry
Gaelic 1.00 . . . . .
English 0.44 1.00 . . . .
History 0.41 0.35 1.00 . . .
Arithmetic 0.29 0.35 0.16 1.00 . .
Algebra 0.33 0.32 0.19 0.59 1.00 .
Geometry 0.25 0.33 0.18 0.47 0.46 1.00
3.1 Introduction of FA
Definition
FA: technique for describing the covariance relationship among many variables in terms of a few factors which are underlying, but unobservable random quantities.
Example (Ex. 9.8, p.502)
Correlation matrix of 6 subjects
Mathematical-ability factor
Verbal-ability factor
![Page 4: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/4.jpg)
3.1 Introduction of FA
History : K. Pearson and Charles Spearman provided beginnings of FA in the early 20th century.
Charles Spearman is known for being the one who coined the term factor analysis
and actually used it to measure children’s cognitive performance.
Spearman, C. (1904). “General intelligence” objectively determined and measured.
"American Journal of Psychology", 15, 201–293.
![Page 5: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/5.jpg)
3.2 Orthogonal Factor Model
Model with m common factors
Properties
Matrix of factor loadings
Vector of specific factors
Assumptions
Common factors decomposition
communality
Specific variance
Loading of the ith the variable on the jth factor
![Page 6: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/6.jpg)
3.3 Methods of Estimation: PCM vs MLM
: Common factor decomposition
with
![Page 7: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/7.jpg)
3.3 Methods of Estimation
[step 3] Obtain the matrix of estimated factor loadings (m<p), the matrix of estimated specific variances and the estimated communalities.
Question : Which communality is estimated very well?
![Page 8: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/8.jpg)
3.3 Methods of Estimation: PCM
How do we select the number of factors m in PCM?
for S, for R
![Page 9: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/9.jpg)
3.3 Methods of Estimation: PCM
Example (Ex. 9.8, p.502)
Correlation matrix of 6 subjects
Program
Gaelic English History Arithmetic Algebra Geometry Gaelic 1.00 . . . . . English 0.44 1.00 . . . . History 0.41 0.35 1.00 . . .
Arithmetic 0.29 0.35 0.16 1.00 . . Algebra 0.33 0.32 0.19 0.59 1.00 .
Geometry 0.25 0.33 0.18 0.47 0.46 1.00
![Page 10: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/10.jpg)
3.3 Methods of Estimation : Results of SAS
They are the only eigenvalues greater than 1.
2 factors account for a cumulative proportion of the total sample variance.
General intelligence factor
Bipolar factor: Half + and half -
All communalities are nearly about 1
All elements are small
![Page 11: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/11.jpg)
3.3 Methods of Estimation: MLM
2) Algorithm for Maximum Likelihood Method
[step 1] Given , consider the likelihood function
[step 3] We have the ml estimators and mles of the communalities
[step 2]With and
obtain the maximization of the likelihood function subject to uniqueness condition
![Page 12: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/12.jpg)
3.3 Methods of Estimation: MLM
How do we select the number of factors m in MLM?
Test the hypotheses with an appropriate m.
: Bartlett’s test statistic based on the chi-square approximation
Residual matrix:
The diagonal elements are zero and the other elements are small: m factor model is appropriate !
![Page 13: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/13.jpg)
3.3 Methods of Estimation: MLM
Program
![Page 14: Techniques and Applications of Multivariate Analysiscontents.kocw.net/KOCW/document/2015/pusan/choiyongseok/... · 2016. 9. 9. · Techniques and Applications of Multivariate Analysis](https://reader034.vdocument.in/reader034/viewer/2022051920/600c6b023640bb1e657fea4d/html5/thumbnails/14.jpg)
3.3 Methods of Estimation: Comparison