1 peter fox data analytics – itws-4963/itws-6965 week 10a, april 7, 2015 factor analysis (v. pca),...
TRANSCRIPT
![Page 1: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/1.jpg)
1
Peter Fox
Data Analytics – ITWS-4963/ITWS-6965
Week 10a, April 7, 2015
Factor Analysis (v. PCA), Fischer Linear Discriminant
![Page 2: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/2.jpg)
Factor Analysis• Exploratory factor analysis (EFA) is a
common technique in qualitative sciences for explaining the (shared) variance among several measured variables as a smaller set of latent (hidden/not observed) variables.
• EFA is often used to consolidate survey data by revealing the groupings (factors) that underlie individual questions.
• A large number of observable variables can be aggregated into a model to represent an underlying concept, making it easier to understand the data.
2
![Page 3: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/3.jpg)
Examples• E.g. business confidence, morale, happiness
and conservatism - variables which cannot be measured directly.
• E.g. Quality of life. Variables from which to “infer” quality of life might include wealth, employment, environment, physical and mental health, education, recreation and leisure time, and social belonging. Others?
• Tests, questionnaires, visual imagery…
3
![Page 4: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/4.jpg)
Relation among factors• “correlated” (oblique) or “orthogonal” factors?
• E.g. wealth, employment, environment, physical and mental health, education, recreation and leisure time, and social belonging
• Relations?
4
![Page 5: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/5.jpg)
Factor Analysis
![Page 6: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/6.jpg)
PCA and FA?• CFA analyzes only the reliable common variance of
data, while PCA analyzes all the variance of data. • An underlying hypothetical process or construct is
involved in CFA but not in PCA. • PCA tends to increase factor loadings especially in
a study with a small number of variables and/or low estimated communality.
• Thus, PCA is not appropriate for examining the structure of data.
6
![Page 7: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/7.jpg)
FA vs. PCA conceptually
• FA produces factors; PCA produces components
• Factors cause variables; components are aggregates of the variables
![Page 8: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/8.jpg)
FA
I1 I3I2
PCA
I1 I3I2
Conceptual FA and PCA
![Page 9: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/9.jpg)
PCA and FA?• If the study purpose is to explain correlations
among variables and to examine the structure of the data, FA provides a more accurate result.
• If the purpose of a study is to summarize data with a smaller number of variables, PCA is the choice.
• PCA can also be used as an initial step in FA because it provides information regarding the maximum number and nature of factors. – Scree plots (Friday)
9
![Page 10: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/10.jpg)
The Relationship between Variables
• Multiple Regression
– Describes the relationship between several variables, expressing one variable as a function of several others, enabling us to predict this variable on the basis of the combination of the other variables
• Factor Analysis
– Also a tool used to investigate the relationship between several variables
– Investigates whether the pattern of correlations between a number of variables can be explained by any underlying dimensions, known as ‘factors’
From: Laura McAvinue School of PsychologyTrinity College Dublin
![Page 11: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/11.jpg)
Uses of Factor Analysis
Test / questionnaire constructiono For example, you wish to design an anxiety questionnaire…o Create 50 items, which you think measure anxietyo Give your questionnaire to a large sample of peopleo Calculate correlations between the 50 items & run a factor
analysis on the correlation matrixo If all 50 items are indeed measuring anxiety…
• All correlations will be high• One underlying factor, ‘anxiety’
Verification of test / questionnaire structureo Hospital Anxiety & Depression Scaleo Expect two factors, ‘anxiety’ & ‘depression’
![Page 12: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/12.jpg)
How does it work?
• Correlation Matrix
– Analyses the pattern of correlations between variables in the correlation matrix
– Which variables tend to correlate highly together?
– If variables are highly correlated, likely that they represent the same underlying dimension
• Factor analysis pinpoints the clusters of high correlations between variables and for each cluster, it will assign a factor
![Page 13: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/13.jpg)
Correlation Matrix
• Q1-3 correlate strongly with each other and hardly at all with 4-6• Q4-6 correlate strongly with each other and hardly at all with 1-3• Two factors!
Q1 Q2 Q3 Q4 Q5 Q6
Q1 1
Q2 .987 1
Q3 .801 .765 1
Q4 -.003 -.088 0 1
Q5 -.051 .044 .213 .968 1
Q6 -.190 -.111 0.102 .789 .864 1
![Page 14: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/14.jpg)
Factor Analysis
• Two main things you want to know…
– How many factors underlie the correlations between the variables?
– What do these factors represent?• Which variables belong to which factors?
![Page 15: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/15.jpg)
Steps of Factor Analysis
1. Suitability of the Dataset
2. Choosing the method of extraction
3. Choosing the number of factors to extract
4. Interpreting the factor solution
![Page 16: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/16.jpg)
1. Suitability of Dataset
Selection of Variables
Sample Characteristics
Statistical Considerations
![Page 17: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/17.jpg)
Selection of Variables
Are the variables meaningful?• Factor analysis can be run on any dataset• ‘Garbage in, garbage out’ (Cooper, 2002)
Psychometrics • The field of measurement of psychological constructs• Good measurement is crucial in Psychology• Indicator approach
• Measurement is often indirect• Can’t measure ‘depression’ directly, infer on the basis of an
indicator, such as questionnaire
Based on some theoretical / conceptual framework, what are these variables measuring?
![Page 18: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/18.jpg)
Selection of Variables, Example
![Page 19: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/19.jpg)
How would you group these faces?
![Page 20: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/20.jpg)
Sample Characteristics
Size At least 100 participants
Participant : Variable Ratio Estimates vary Minimum of 5 : 1, ideal of 10 : 1
Characteristics Representative of the population of interest? Contains different subgroups?
![Page 21: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/21.jpg)
Statistical Considerations
Assumptions of factor analysis regarding data Continuous Normally distributed Linear relationships
These properties affect the correlations between variables
Independence of variables Variables should not be calculated from each other
e.g. Item 4 = Item 1 + 2 + 3
![Page 22: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/22.jpg)
Statistical Considerations
Are there enough significant correlations (> .3) between the variables to merit factor analysis?
Bartlett Test of Sphericity
Tests Ho that all correlations between variables = 0
If p < .05, reject Ho and conclude there are significant correlations between variables so factor analysis is possible
![Page 23: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/23.jpg)
Statistical Considerations
Are there enough significant correlations (> .3) between the variables to merit factor analysis?
Kaiser-Meyer-Olkin Measure of Sampling Adequacy
Quantifies the degree of inter-correlations among variables Value from 0 – 1, 1 meaning that each variable is perfectly
predicted by the others Closer to 1 the better If KMO > .6, conclude there is a sufficient number of
correlations in the matrix to merit factor analysis
![Page 24: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/24.jpg)
Statistical Considerations, Example
• All variables• Continuous• Normally Distributed• Linear relationships• Independent
• Enough correlations?• Bartlett Test of Sphericity (χ2; df ; p < .05)• KMO
![Page 25: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/25.jpg)
2. Choosing the method of extraction
Two methods
Factor AnalysisPrincipal Components AnalysisDiffer in how they analyse the variance in the
correlation matrix
![Page 26: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/26.jpg)
Variable
Specific
Variance
Error
Variance
Common Variance
Variance unique to the variable itself
Variance due to measurement error or some
random, unknown source
Variance that a variable shares
with other variables in a
matrix
When searching for the factors underlying the relationships between a set of variables, we are interested in detecting and explaining the common variance
![Page 27: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/27.jpg)
Principal Components Analysis
• Ignores the distinction between the different sources of variance
• Analyses total variance in the correlation matrix, assuming the components derived can explain all variance
• Result: Any component extracted will include a certain amount of error & specific variance
Factor Analysis
• Separates specific & error variance from common variance
• Attempts to estimate common variance and identify the factors underlying this
Which to choose?
• Different opinions
• Theoretically, factor analysis is more sophisticated but statistical calculations are more complicated, often leading to impossible results
• Often, both techniques yield similar solutions
V
![Page 28: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/28.jpg)
3. Choosing the number of factors to extract
• Statistical Modelling– You can create many solutions using different
numbers of factors
• An important decision– Aim is to determine the smallest number of factors
that adequately explain the variance in the matrix– Too few factors
• Second-order factors – Too many factors
• Factors that explain little variance & may be meaningless
![Page 29: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/29.jpg)
Criteria for determining Extraction
Theory / past experience
Latent Root Criterion
Scree Test
Percentage of Variance Explained by the factors
![Page 30: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/30.jpg)
Latent Root Criterion (Kaiser-Guttman)
• Eigenvalues
– Expression of the amount of variance in the matrix that is explained by the factor
– Factors with eigenvalues > 1 are extracted– Limitations
• Sensitive to the number of variables in the matrix• More variables… eigenvalues inflated… overestimation of
number of underlying factors
![Page 31: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/31.jpg)
Scree Test (Cattell, 1966)
• Scree Plot
– Based on the relative values of the eigenvalues– Plot the eigenvalues of the factors– Cut-off point
• The last component before the slope of the line becomes flat (before the scree)
![Page 32: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/32.jpg)
Elbow in the graph
Take the components above the elbow
![Page 33: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/33.jpg)
Percentage of Variance
• Percentage of variance explained by the factors
– Convention– Components should explain at least 60% of the
variance in the matrix (Hair et al., 1995)
![Page 34: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/34.jpg)
3. Choosing the number of factors to extract
• Three components with eigenvalues > 1
• Explained 67.26% of the variance
Scree Plot
Co mp o n e n t Nu mb e r
987654321
Eig
en
va
lue
4. 0
3. 5
3. 0
2. 5
2. 0
1. 5
1. 0
. 5
0. 0
![Page 35: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/35.jpg)
BFI data in psych (R)
35
![Page 36: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/36.jpg)
4. Interpreting the Factor Solution
• Factor Matrix– Shows the loadings of each of the variables on the
factors that you extracted– Loadings are the correlations between the variables
and the factors– Loadings allow you to interpret the factors
• Sign indicates whether the variable has a positive or negative correlation with the factor
• Size of loading indicates whether a variable makes a significant contribution to a factor
– ≥ .3
![Page 37: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/37.jpg)
Component 1 – Visual imagery tests
Component 2 – Visual imagery questionnaires
Component 3 – ?
Variables Component 1 Component 2 Component 3
Vividness Qu -.198 -.805 .061
Control Qu .173 .751 .306
Preference Qu .353 .577 -.549
Generate Test -.444 .251 .543
Inspect Test -.773 .051 -.051
Maintain .734 -.003 .384
Transform (P&P) Test
.759 -.155 .188
Transform (Comp) Test
-.792 .179 .304
Visual STM Test .792 -.102 .215
![Page 38: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/38.jpg)
Factor Matrix
• Interpret the factors
• Communality of the variables– Percentage of variance in each variable that can be
explained by the factors
• Eigenvalues of the factors– Helps us work out the percentage of variance in the
correlation matrix that the factor explains
![Page 39: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/39.jpg)
Communality of Variable 1 (Vividness Qu) = (-.198)2 + (-.805)2 + (.061)2 = . 69 or 69%
Eigenvalue of Comp 1 = ( [-.198]2 + [.173]2 + [.353]2 + [-.444]2 + [-.773]2 +[.734]2 + [.759]2 + [-.792]2 + [.792]2 ) = 3.36
3.36 / 9 = 37.3%
Variables Component 1 Component 2 Component 3 Communality
Vividness Qu -.198 -.805 .061 69%
Control Qu .173 .751 .306 69%
Preference Qu .353 .577 -.549 76%
Generate Test -.444 .251 .543 55%
Inspect Test -.773 .051 -.051 60%
Maintain .734 -.003 .384 69%
Transform (P&P) Test
.759 -.155 .188 64%
Transform (Comp) Test
-.792 .179 .304 75%
Visual STM Test .792 -.102 .215 69%
Eigenvalues 3.36 1.677 1.018 /
% Variance 37.3% 18.6% 11.3% /
![Page 40: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/40.jpg)
Factor Matrix
• Unrotated Solution– Initial solution– Can be difficult to interpret– Factor axes are arbitrarily aligned with the variables
• Rotated Solution– Easier to interpret– Simple structure– Maximises the number of high and low loadings on
each factor
![Page 41: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/41.jpg)
Factor Analysis through Geometry
• It is possible to represent correlation matrices geometrically
• Variables– Represented by straight lines of equal length– All start from the same point– High correlation between variables, lines positioned
close together– Low correlation between variables, lines positioned
further apart– Correlation = Cosine of the angle between the lines
![Page 42: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/42.jpg)
60º
30º
V1
V2
V3
The smaller the angle, the bigger the cosine and the bigger the correlation
V1 & V3
90º angle
Cosine = 0
No relationship
V1 & V230º angleCosine = .867r = .867
V2 & V360º angleCosine = .5R = .5
![Page 43: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/43.jpg)
V1
V5
V4
Factor Loading
Cosine of the angle between each factor and the variable
Factor Analysis
Fits a factor to each cluster of
variables
Passes a factor line through
the groups of variables
V2 V3
V6
F1
F2
![Page 44: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/44.jpg)
V1
V5
V4
V2 V3
V6
F1
F2
V1
V5
V4
V2 V3
V6
F1
F2
Two Methods of fitting Factors
Orthogonal Solution
Factors are at right angles
Uncorrelated
Oblique Solution
Factors are not at right angles
Correlated
![Page 45: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/45.jpg)
V1
V5
V4
V2 V3
V6
F1
F2
Two Step Process
Factors are fit arbitrarily
Factors are rotated to fit the clusters of variables better
V1
V5
V4
V2 V3
V6
F1
F2
![Page 46: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/46.jpg)
Variables C1 C2 C3
Vividness Qu -.198 -.805 .061
Control Qu .173 .751 .306
Preference Qu .353 .577 -.549
Generate Test -.444 .251 .543
Inspect Test -.773 .051 -.051
Maintain Test .734 -.003 .384
Transform (P&P) Test
.759 -.155 .188
Transform(Comp) Test
-.792 .179 .304
Visual STM Test .792 -.102 .215
Variables C1 C2 C3
Vividness Qu -.029 -.831 .008
Control Qu .174 .744 .323
Preference Qu -.010 .679 -.547
Generate Test -.197 .112 .709
Inspect Test -.717 -.103 .279
Maintain Test .819 .116 .043
Transform (P&P) Test
.779 -.013 -.166
Transform(Comp) Test
-.599 -.01 .626
VisualSTM Test .813 .045 -.147
Unrotated Solution Solution following Orthogonal Rotation
For example…
![Page 47: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/47.jpg)
Factor Rotation
• Changes the position of the factors so that the solution is easier to interpret
• Achieves simple structure
– Factor matrix where variables have either high or low loadings on factors rather than lots of moderate loadings
![Page 48: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/48.jpg)
Evaluating your Factor Solution
• Is the solution interpretable?– Should you re-run and extract a bigger or smaller number of
factors?
• What percentage of variance is explained by the factors?– >60%?
• Are all variables represented by the factors?– If the communality of one variable is very low, suggests it is not
related to the other variables, should re-run and exclude
![Page 49: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/49.jpg)
Variables C1 C2 C3
Vividness Qu -.029 -.831 .008
Control Qu .174 .744 .323
Preference Qu -.010 .679 -.547
Generate Test -.197 .112 .709
Inspect Test -.717 -.103 .279
Maintain Test .819 .116 .043
Transform (P&P) Test
.779 -.013 -.166
Transform(Comp) Test
-.599 -.01 .626
VisualSTM Test .813 .045 -.147
First Solution Second Solution
For example…
Variables Component 1 Component 2
Vividness Qu .013 -.829
Control Qu -.023 .770
Preference Qu .195 .648
Generate Test -.493 .130
Inspect Test -.760 -.146
Maintain Test .711 .183
Transform (P&P) Test .773 .042
Transform (Comp) Test
-.811 -.028
Visual STM Test .792 .103
Component 3 = ?C1 = Efficiency of objective visual imagery
C2 = Self-reported imagery efficacy
![Page 50: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/50.jpg)
![Page 51: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/51.jpg)
![Page 52: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/52.jpg)
![Page 53: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/53.jpg)
![Page 54: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/54.jpg)
We are looking for an eigenvalue above 1.0.
Cumulative percent of variance explained.
![Page 55: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/55.jpg)
![Page 56: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/56.jpg)
![Page 57: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/57.jpg)
![Page 58: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/58.jpg)
Expensive
Exciting
Luxury
Distinctive
Not Conservative
Not Family
Not Basic
Appeals to Others
Attractive Looking
Trend Setting
Reliable
Latest Features
Trust
![Page 59: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/59.jpg)
Expensive
Exciting
Luxury
Distinctive
Not Conservative
Not Family
Not Basic
Appeals to Others
Attractive Looking
Trend Setting
Reliable
Latest Features
Trust
What shall these components be called?
![Page 60: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/60.jpg)
Expensive
Exciting
Luxury
Distinctive
Not Conservative
Not Family
Not Basic
Appeals to Others
Attractive Looking
Trend Setting
Reliable
Latest Features
Trust
EXCLUSIVE TRENDY RELIABLE
![Page 61: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/61.jpg)
= (Expensive + Exciting + Luxury + Distinctive – Conservative – Family – Basic)/7
= (Appeals to Others + Attractive Looking + Trend Setting)/3
= (Reliable + Latest Features + Trust)/3
EXCLUSIVE
TRENDY
RELIABLE
Calculate Component Scores
![Page 62: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/62.jpg)
![Page 63: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/63.jpg)
Exclusive Trendy ReliableBeetle 1.4 6.7 6.9Hummer 3.9 6.2 6.7Lotus 4.1 7.3 6.7Minivan -1.67 4.83 6.5Pick-Up -0.43 4.93 6.3
Not much differing on this dimension.
![Page 64: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/64.jpg)
Exclusive Trendy ReliableBeetle 1.4 6.7 6.9Hummer 3.9 6.2 6.7Lotus 4.1 7.3 6.7Minivan -1.67 4.83 6.5Pick-Up -0.43 4.93 6.3
![Page 65: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/65.jpg)
Vehicle by Component
-3 -2 -1 0 1 2 3 4 5 6 7 8
Beetle
Hummer
Lotus
Minivan
Pick-Up
Exclusive Trendy
![Page 66: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/66.jpg)
References
• Some…• Cooper, C. (1998). Individual differences.
London: Arnold.
• Kline, P. (1994). An easy guide to factor analysis. London: Routledge.
![Page 67: 1 Peter Fox Data Analytics – ITWS-4963/ITWS-6965 Week 10a, April 7, 2015 Factor Analysis (v. PCA), Fischer Linear Discriminant](https://reader035.vdocument.in/reader035/viewer/2022062718/56649e7d5503460f94b804ca/html5/thumbnails/67.jpg)
Assignment to come…
• Assignment 7: Predictive and Prescriptive Analytics. Due ~ week ~12. 20%..
67