Factor Analysis in Individual Differences Research: The Basics

Psych 437

Data from over 1000 people in the National Election Study

   1 2  3  4  5  6  7  8  9  10  11 

 1. Abortion 1 -.086 .007 -.046 -.193 -.245 .272 .055 -.089 -.033 -.097

 2. Against Death Penalty -.086 1 .128 -.179 -.129 -.116 .134 .102 -.008 -.106 -.084

 3. Immigrants .007 .128 1 -.053 -.020 -.090 .122 .025 .065 .018 -.034

4. Decrease welfare spending -.046 -.179 -.053 1 .115 .133 -.217 -.188 -.010 .121 .222

 5. Increase defense spending -.193 -.129 -.020 .115 1 .198 -.096 -.007 .096 .065 .144

6. No Gays in military -.245 -.116 -.090 .133 .198 1 -.242 -.055 .157 .100 .162

7. Women's movement .272 .134 .122 -.217 -.096 -.242 1 .366 -.194 -.182 -.146

8. Labor unions .055 .102 .025 -.188 -.007 -.055 .366 1 -.037 -.117 -.146

9. Environment spending not important

-.089 -.008 .065 -.010 .096 .157 -.194 -.037 1 .032 .066

10. Decrease financial aid students -.033 -.106 .018 .121 .065 .100 -.182 -.117 .032 1 .120

11. Privatize Health care -.097 -.084 -.034 .222 .144 .162 -.146 -.146 .066 .120 1

• To summarize, there are correlations among the social and political attitudes that people hold.

• People who have favorable attitudes towards abortion, for example, are also more likely than others to have favorable attitudes towards the women’s movement, gays in the military, and decreasing defense spending.

• Why do these correlations exist?

• One of the assumptions often made in personality psychology is that these diverse attitudes tend to covary or hang together across people because they are organized by a common trait or attitude (latent factors).

Latent factor

Abortion

Increase defense spending

No gays in the military

Women’s movement

+

--

+

• Thus, one of the goals of many research studies is to investigate the latent factors that help explain these statistical patterns.

• One of the more commonly used tools for doing so is factor analysis.

• In short, factor analysis is a statistical method for explaining the correlations among measured variables with respect to a smaller number of unobserved variables (sometimes called factors or traits).

• Factor analysis is used in many areas of behavioral science.

– To understand the latent factors that underlie political attitudes

– To understand the latent factors that underlie performance in spatial, verbal, and speeded cognitive tests

– To understand the factors underlying people’s musical classifications

– To understand the latent factors that underlie personality descriptors (e.g., the Big Five personality traits)

• One of the big questions that motivates a lot of factor analytic research is: How many latent factors are needed to parsimoniously explain the associations among measured variables?

Var 1 Var 2 Var 3 Var 4Var 1 Var 2 Var 3 Var 4

• Mathematical decomposition of the correlation matrix (eigenvalue-eigenvector decomposition)

• In practice, this amounts to an attempt to reproduce as much of the correlation matrix as possible with as few latent factors as possible.

• Each latent factor has a corresponding eigenvalue. An eigenvalue can be conceptualized as representing information.

Performing the decomposition in SPSS

Select the variables you wish to analyze and shoot them over to the “variables” box.

Next, click the Extraction button.

Check the option labeled “Scree plot” then click Continue.

When you’re back at the main menu, press “ok” to run the analysis.

Notice that the first factor has a large eigenvalue.

The next few have smaller eigenvalues.

Each successive factor buys us less and less information.

Where is the tipping point?

Elbow rule

Methods for optimizing parsimony (using as few factors as possible) with information (being able to reproduce the original correlation matrix)

Eigenvalue > 1 rule

What do the latent factors represent?

• What do the latent factors represent? How should we interpret them?

• To answer this question it is useful to extract the factors and find a useful way to position them in multivariate space.

Variable 1

Variable 2

Peoples’ scores on two variables in a two-dimensional space.

Variable 1

Variable 2

Notice that the sets of scores are highly correlated. In fact, there is one primary factor that underlies these scores.

Variable 1

Variable 2

This specific orientation of the factor maximizes the variance of the data points (or the length of the line) compared to alternatives.

But where do we position that factor? There are clearly many ways to “rotate” an axis is this two-dimensional space.

Variable 1

Variable 2

Notice that it is possible to place another factor through the data points. This factor is smaller than the original one.

Variable 1

Variable 2

The process of extracting additional factors and placing them in the multivariate space can be challenging.

Varimax rotation: Each additional factor is orthogonal (90-degrees) to the previous. Moreover, the axes are placed such that the variance with respect to each factor is maximized.

The variance maximizing property of varimax rotation is better illustrated with a more complex example—a situation in which two factors carry large amounts of information—and by pulling the axes out and laying them side-by-side.

It is also possible to maximize variance while allowing the factors to correlate with one another. These are sometimes called oblique rotations. SPSS uses a method called direct oblimin.

We can represent people in this factor space. In fact, once the location of the factors is specified, we can compute for each person a factor score—his or her score with respect to each factor.

We can also represent the observed variables in this factor space. We do so with factor loadings. Variables that “load” highly on or correlate highly with a factor are typically used to interpret the meaning of the latent factors.

Now let’s instruct the program to specifically select 3 factors.

In the main menu, click the Extraction button.

Select the radio button labeled “fixed number of factors” and enter 3 into the textbox.

Also, for Method, be sure to choose Principal Axis Factoring.

Press continue

From the main menu, click the Rotation button.

Select the option labeled “Varimax.”

Also, check the option called “Loading Plots(s)”

Continue

From the main menu, click the Options button.

Choose “exclude cases pairwise”

Check “Sorted by size”

The first factor seems to reflect individual differences in anti-abortion, against gays in the military, in favor of defense spending, and against environmental spending

The second factor seems to reflect individual differences in support of the women’s movement and labor unions.

The second factor seems to reflect individual differences in favoring the death penalty, decreasing welfare spending, privatizing health care, and decreasing financial aid for college students

Run the analysis again with 2 factors

The first factor seems to reflect individual differences in being against the women’s movement, decreasing welfare spending, opposition to labor unions, favoring the death penalty, favoring the privatization of health care, decreasing student aid.

The second factor seems to reflect individual differences in anti-abortion, anti-gays in the military, increased defense spending, decreased environmental spending

No gays in military

Decrease welfare spending

Abortion

Women’s movement

Labor unions

Against death penalty

Immigrants

Environmental spending

Decrease student aid

Privatize health care

Increase defense spending

Regression slopes where Variable = b1*Factor 1 + b2*Factor 2

Correlations between Factors and Variables

Correlations between factors

Regression slopes where Variable = b1*Factor 1 + b2*Factor 2

Correlations between Factors and Variables

Correlations between factors

Correlated Factors (Direct Oblimin) Uncorrelated Factors (Varimax rotation)

Notice that, in this case, the interpretation of the results is not dependent upon whether we assumed correlated or uncorrelated factors.

That will not always be the case.

Caveats concerning factor analysis

• The method is often used as a method for reducing the number of measured variables into a smaller number of composites. Utilitarian.

• Sometimes the factors are reified—that is, treated as something real. The factors may or may not represent anything real; the answer to that question lies beyond the scope of factor analysis itself.

• There is no correct rotation. Rotation choices often need to be made to balance research needs.