beliefs and good thinking
TRANSCRIPT
1
Stanovich and West’s (2007) Actively Open-Minded Thinking Scale: An Examination of Factor
Structure
Kimberly R. Marsh
Dena A. Pastor
James Madison University
Paper presented at the annual meeting of the Association for Psychological Science, Washington,
D.C., May 2011
2
Abstract
Research examining the divergence of human thinking from what is considered to be the ideal
standard has indicated that human thinking is often prone to error and everyday thinking
processes, judgments, and decision making are negatively affected by our own biased
viewpoints. These cognitive shortcomings have inspired researchers to examine characteristics of
what they deem to be “good thinking” such as actively open-minded thinking. In order to
examine the relationship between characteristics of good thinking and performance on critical
thinking tasks, Stanovich and West (2007) developed their own Actively Open-Minded Thinking
(AOT) scale. The current study examined the factor structure of Stanovich and West’s (2007)
AOT scale by examining multiple factor solutions via exploratory factor analysis. Results
indicated that AOT items did not load on the six subscales originally developed by Stanovich
and West (2007) nor were they unidimensional; instead, a 4-factor solution emerged. Theoretical
and practical implications of these findings are discussed.
3
Stanovich and West’s (2007) Actively Open-Minded Thinking Scale: An Examination of Factor
Structure
There is an extensive body of research examining the divergence of human thinking from
what is considered to be the ideal standard. Perhaps unsurprisingly, this research has indicated
that human thinking is often prone to error and everyday thinking processes, judgments, and
decision making are negatively affected by our own biased viewpoints. Knowledge of these
common cognitive shortcomings has inspired research that seeks to identify aspects of good
thinking practices. Ideally, in identifying aspects of good thinking, researchers in this domain
may identify some individual skills or characteristics that could be cultivated and lend
themselves to the improvement of reasoning.
According to Baron (1991), “good thinking” is a complex process that incorporates,
“optimal search for possibilities, evidence, and goals, and fairness in the search for evidence and
in inference” (p. 172). Conversely, Baron characterizes aspects of “poor thinking” as inadequate
search for evidence and the general tendency of people to allow their own biases to affect their
reasoning (Baron, 1991, p. 172). Baron further delineates good and poor thinking by identifying
what he believes to be the defining aspect of good thinking; that it is actively open-minded. This
actively open-minded thinking component of good thinking incorporates a search for evidence,
the extent to which is determined by the importance of the question, and an objectivity in one’s
consideration and review of the evidence (Baron, 1991).
There has been some research conducted in the area of cognitive psychology that
suggests the distinct nature of cognitive ability (i.e., general intelligence) and individual thinking
dispositions, such as actively open-minded thinking (Baron, 1985, 1988, 1991; Ennis, 1987;
Moshman, 1994; Norris, 1992; Perkins, Faraday, & Bushey, 1991). More specifically, there is
4
some evidence to suggest that each of these variables is able to account for unique variance in
performance on critical thinking tasks, suggesting that cognitive ability and individual thinking
dispositions may address different aspects of what is considered to be good thinking (Stanovich
& West, 1997, 1998, 2007). Stanovich and West have devoted much of their research to
examining the nature of the relationship between cognitive ability, individual thinking
dispositions, such as actively open-minded thinking, and whether these constructs can uniquely
predict performance on a number of cognitive tasks. For instance, Stanovich and West (1997)
evaluated individual performance on an instrument they refer to as the Argument Evaluation
Task. Participants were instructed to objectively evaluate the quality of a number of arguments
crafted by a fictitious individual after previously indicating their own opinions and thoughts on
the subject matter of these arguments (e.g., social and political issues, such as, crime, gun
control). Stanovich and West (1997) found that after controlling for cognitive ability (e.g., self-
reported SAT scores), individual thinking disposition variables, such as actively open-minded
thinking accounted for a significant amount of variance in performance on the Argument
Evaluation Task; ultimately providing more support for the argument that cognitive ability and
individual thinking disposition variables may both contribute to characteristics of critical
thinking.
Stanovich and West (1997, 1998, 2007) have been instrumental in the development and
utilization of multiple versions of their own Actively Open-minded Thinking scale (AOT).
Stanovich and West (1997) conceptualized actively open-minded thinking as the tendency of
people to engage in flexible or open-minded thinking and avoid dogmatic or categorical thinking.
Stemming from this definition of actively open-minded thinking, six subscales were
conceptualized for the current Stanovich and West (2007) 41 item version of the AOT scale:
5
Flexible thinking (Stanovich & West, 1997), Openness-Values (Costa & McCrae, 1992),
Dogmatism (Paulhus & Reid, 1991; Robinson, Shaver, & Wrightsman, 1991; Troldahl & Powell,
1965), Categorical thinking (Epstein & Meier, 1989), Belief identification (Sá et al., 1999), and
Counterfactual thinking (Stanovich & West, 1997). Although there have been multiple versions
of the AOT scale (Sá, West, & Stanovich, 1999; Stanovich & West, 1997) consisting of different
combinations of items and subscales leading up to the most current, 41 item version of Stanovich
and West’s (2007) AOT scale, the rationale behind scale modifications and revisions is unclear.
Purpose of the Current Study
To date, no published factor analysis studies have been conducted on the most current, 41
item version of Stanovich and West’s (2007) AOT scale. Stanovich and West (2007) currently
suggest summing participant item responses for the six subscales, indicating that they believe the
actively open-minded construct to be unidimensional. The purpose of the current study is to
perform exploratory factor analysis (EFA) on the most current version of the Stanovich and West
(2007) AOT scale in order to examine the emergent factor structure. An EFA will inform the
validity of the AOT scores by indicating the extent to which the relationships among items
conform to Stanovich and West’s conceptualization of the construct, aid in identifying items not
functioning as intended, and inform understanding of the actively open-minded thinking
construct itself.
Methods
Participants and Procedures
Undergraduate students from a mid-sized university participated in the current study.
Participants registered to participate through the university subject pool and were awarded class
credit for an introductory psychology course in exchange for their participation in the study.
6
Undergraduate research assistants conducted study sessions in an on-campus computer lab,
running four participants at a time. Participants completed a series of measures in addition to the
Stanovich and West (2007) AOT scale via Qualtrics Survey Software.
A total of 452 students initially completed the survey, however, seven participants were
listwise deleted due to missing data. The remaining sample (N = 445) was screened for both
univariate and multivariate normality. Upon data screening, variable distributions were deemed
univariate normal (skewness and kurtosis values falling within the acceptable range of |2| and |7|,
respectively; Kline, 2005). Using DeCarlo’s (1997) macro to screen for multivariate outliers, 15
cases were flagged as outliers at the .05 significance level. These cases were removed from the
data set, resulting in the final sample used for the subsequent EFA (N = 430). Of the 430
undergraduate students who were included within the analyses for the current study, the majority
were first-years (70.2%), female (70.2%), and Caucasian (85.1%). Mean participant age was
19.00 years (SD = 1.43).
Actively Open-minded Thinking (AOT) Scale. Participants completed Stanovich and
West’s (2007) 41 item version of the AOT. This version of the AOT consists of six subscales
including: Flexible thinking (Stanovich & West, 1997), Openness-Values (Costa & McCrae,
1992), Dogmatism (Paulhus & Reid, 1991; Robinson et al., 1991; Troldahl & Powell, 1965),
Categorical thinking (Epstein & Meier, 1989), Belief identification (Sá et al., 1999), and
Counterfactual thinking (Stanovich & West, 1997). Participants indicated their agreement with
each of the 41 statements on a Likert scale consisting of 6 response options: Disagree Strongly
(1), Disagree Moderately (2), Disagree Slightly (3), Agree Slightly (4), Agree Moderately (5),
and Agree Strongly (6). Items were reflected in such a way that higher AOT scores indicated a
7
higher level of agreement with items that endorsed actively open-minded thinking. Cronbach’s
alpha for the Stanovich and West (2007) AOT scale was .83.
Results
Data Screening
Descriptive statistics for each of the Stanovich and West (2007) AOT scale items are
presented in Table 1. Referring to the estimates provided in Table 1, we can see that, overall, the
distribution of the variables are univariate normal with skewness and kurtosis values falling
within the acceptable range of |2| and |7|, respectively (Kline, 2005). It should be noted that these
descriptive statistics are based upon the reduced sample (N = 430) utilized within the analyses
after the data were screened and cleaned.
AOT inter-item Pearson correlation coefficients ranged from -.21 to .48, indicating that
multicollinearity was not an issue within the current sample. It should be noted that these inter-
item correlations are rather low; suggesting that AOT items are not substantially related to one
another and thus factor analysis may be inappropriate (Nunnally & Bernstein, 1994). However,
Bartlett’s test of sphericity, the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (KMO)
index, and the Measure of Sampling Adequacy (MSA) indices were still examined in order to
determine whether it was appropriate move forward with factor analysis.
Bartlett’s test of sphericity (2(820) = 4088.23, p < .001) indicated that we should reject
the null hypothesis that the AOT variables are completely uncorrelated, thus suggesting that the
variables are inter-correlated and it is appropriate to run a parallel analysis procedure and
conduct factor analysis (Gorsuch, 1983). The KMO index (KMO = .85) approached one,
indicating that the correlation matrix was amenable to factoring. This KMO index value is
considered to be “meritorious” by Kaiser’s (1974) standards, indicating that the AOT variables
8
are likely to share common factors. Based upon the values in the anti-image correlation matrix,
we can conclude that the amenability of the variables to factoring is acceptable. Overall, the
Measure of Sampling Adequacy (MSA) values along the diagonal in the anti-image correlation
matrix were fairly high (approaching a value of one), indicating that each of the AOT variables
share an adequate amount of variance with the remaining AOT variables. Furthermore, the
absolute values on the off-diagonal of the anti-image correlation matrix were rather small (close
to 0). That is, the negative partial correlations amongst the AOT variables were quite small,
indicating that any combination of two AOT variables do not relate to one another after
controlling for how these two variables relate to the remaining AOT variables.
Exploratory Factor Analysis
Both a scree plot of eigenvalues based on the reduced correlation matrix and the results
of a parallel analysis (Horn, 1965) were examined in order to determine the number of factor
solutions for review (see Figure 1). All possible AOT factor eigenvalues were plotted on the
scree plot. Upon visual examination of the scree plot, it appears that plotted eigenvalues begin to
“level off” after the first seven plotted eigenvalues, suggesting that these subsequent factors
account for insignificant amounts of variance in AOT scores above and beyond that accounted
for by the first seven factors. The results of the parallel analysis procedure mirror those of the
visual examination of the scree plot suggesting the extraction of seven factors based upon the
fact that the first seven eigenvalues extracted from the AOT data set are greater than those
eigenvalues calculated based upon randomly generated data sets that have similar characteristics.
Because of the exploratory nature of the analyses, all seven factor solutions (i.e., 1-factor, 2-
factor, 3-factor, etc.) were subsequently examined. However, only the results for the 4-factor
9
solution are presented and discussed in detail due to the fact that this emergent factor solution
displayed what most closely resembles simple structure and was most interpretable.
Table 2 displays the resulting pattern coefficients for the 4-factor EFA solution using
Direct Oblimin rotation with delta set to zero. Pattern coefficients provide the most interpretable
factor loading estimates as they reflect the correlation between each item and the factor after
controlling for the inter-correlations amongst other items. Item loading criteria employed within
the current study was |.40|, thus items were deemed to load on a factor if their pattern coefficient
was greater than or equal to |.40|.
Results indicated that AOT item functioning was problematic. There were nine items that
failed to load on any of the factors across the seven factor solutions examined. Failure of these
items to load on any factor may indicate that these items are confusing for respondents or they
fail to measure similar aspects of the actively open-minded thinking construct. There were nine
additional items that failed to load on any of the four factors in the 4-factor solution. This may
also suggest that these items are problematic for respondents.
There were 13 items that loaded solely on the first factor of the 4-factor solution and
accounted for 15.84% of the total variance in AOT scores. Nine of the 13 items that loaded on
the first factor in the 4-factor solution consistently loaded on the first factor across multiple
solutions. Overall, these items appeared to represent participant proclivity to engage in a kind of
rigid, black and white thinking style that is characterized by dogmatic and categorical thinking.
This suggests that these previously hypothesized aspects of actively open-minded thinking
merged together to form one factor.
There were five items that loaded solely on the second factor of the 4-factor solution and
accounted for 4.33% of the total variance in AOT scores. Overall, items that loaded on this
10
second emergent factor appeared to reflect aspects of flexible, open-minded thinking. Similar to
those items that loaded on factor 1, the most salient item loadings on factor 2 were items that
were originally identified as two aspects of the actively open-minded thinking construct, flexible
thinking and openness- values; thus, it appears as though these two previously hypothesized
aspects of actively open-minded thinking have merged to form the second emergent factor in the
4-factor solution. Items that loaded on factors 3 and 4 were relatively minor, or secondary, in
respect to the first two emergent factors of the 4-factor solution. There were three items that
loaded on factor 3 and accounted for 3.36% of the total variance in AOT scores. Unlike the first
two emergent factors in the 4-factor solution, these items did not appear to be the result of
previously specified aspects of the actively open-minded thinking construct merging to form one
factor. Moreover, these items appear to reflect a “perseverance in your beliefs” factor not
originally hypothesized.
Finally, there were only two items that loaded on factor 4 and accounted for 2.63% of the
total variance in AOT scores. These two items were originally specified as the counterfactual
thinking subscale by Stanovich and West (1997, 2007). Although these items load on their
originally hypothesized factor, they do not account for much variance in AOT scores suggesting
that counterfactual thinking may not be a central component of actively open-minded thinking.
Inter-factor correlations for the 4-factor solution were rather low, ranging from .05 to .23;
providing further evidence to suggest the distinct nature of AOT items statements that load on
each of the 4 emergent factors. AOT item communalities reported in Table 2 range from .06 to
.48. Item communality estimates indicate the amount of common variance in the item explained
by the factor solution. These item communality estimates are quite low. We would hope that
these item communality estimates would be higher, indicating that the set of four emergent
11
factors explains a substantial amount of common variance within a specific item; however, this is
not the case.
The total variance accounted for in AOT score by the four emergent factors combined
was 26.15%. It should be noted that this estimate of total variance reflects overlap amongst the
factors as the rotation method allowed them to inter-correlate, thus it reflects a slight
overestimate of the variance accounted for by the factors. The total variance estimate is rather
low, leaving approximately 74% of the variance in AOT scale scores unaccounted for by the four
emergent factors. The large amount of variance left unaccounted for in AOT scores could have
several implications for the current version of the AOT scale. First, item wording could be such
that it confuses participants, causing them to respond in a contradictory manner to items meant to
address similar aspects of the construct. Secondly, item statements could be measuring distinct
concepts that do not covary, or relate in such a way, to adequately represent Stanovich and
West’s (1997, 2007) original conceptualization of actively open-minded thinking.
Discussion
The purpose of the current study was to examine the emergent factor structure of
Stanovich and West’s (2007) AOT scale in an effort to determine whether scores on the AOT
accurately represent actively open-minded thinking as conceptualized by Stanovich and West
(1997, 2007). Stanovich and West (2007) suggest calculating an AOT scale total score by
summing participant responses across item statements originally developed to address six facets
of the actively open-minded thinking construct, including, dogmatic thinking, categorical
thinking, flexible thinking, openness to values, belief identification, and counterfactual thinking.
The suggested calculation of an AOT total score implies that Stanovich and West believe the
scale to be unidimensional. However, this does not appear to be the case based upon the findings
12
of the current study. AOT items did not load on the six subscales originally developed by
Stanovich and West (2007) nor were they unidimensional; instead, a 4-factor solution emerged.
EFA results not only indicated that calculation of an AOT total score is inappropriate, but
that the current version of the Stanovich and West (2007) AOT scale needs to be revised and
conceptualization of the actively open-minded thinking construct revisited. Examination of the
AOT inter-item correlation matrix revealed that inter-item correlations were quite low, indicating
that the 41 items currently on the AOT scale may be measuring distinct, or seemingly unrelated
concepts. Given the low inter-item correlations, it is not surprising that although the EFA
resulted in an interpretable 4-factor solution, there were a series of problematic items identified
in need of substantial revision or removal (e.g., items that failed to load across factor solutions,
items that failed to load on any of the emergent four factors). In addition, the emergent factors
failed to adequately represent theorized aspects of actively open-minded thinking (e.g., factor 3
which did not appear to represent a theorized aspect of actively open-minded thinking, factor 4
which only consisted of two items). Moreover, estimates regarding the percentage of variance
accounted for in AOT scores by each of the four factors, item communalities, and total variance
accounted for in AOT scores by all four factors further support the distinct nature of AOT item
statements.
Confusing AOT scale item content and/or wording may contribute to the distinctness of
the item statements and, ultimately, the failure of items to form factors, particularly factors that
are able to explain a large percentage of variance in scores. For example, one of the nine items
that failed to load on any of the factors across multiple solutions reads, “Even though freedom of
speech for all groups is a worthwhile goal, it is unfortunately necessary to restrict the freedom of
certain political groups”. The wording for this particular item is confusing and it may be difficult
13
for respondents to decipher the meaning of the statement. Another item statement that failed to
load on any factor solution reads, “My blood boils over whenever a person stubbornly refuses to
admit he’s wrong”. This particular item does not seem to relate to the other items on the scale in
any logical or intuitive way. There appear to be other aspects of AOT items that pose additional
threats to comprehension, such as wording that encourages respondents to misconstrue subject
matter. For example, there are several items that inquire about one’s “beliefs” or opinions about
“lifestyles” (e.g., “It is important to persevere in your beliefs even when evidence is brought to
bear against them”, “I consider myself broad-minded and tolerant of people’s lifestyles”).
Wording such as this may cause the respondent to think of religious beliefs or about one’s
“lifestyle” in regards to sexual orientation, thus causing participants to respond to the items in
differential manners based upon their own interpretation of the item content. There is evidence to
suggest that respondents are interpreting items in different ways. Zilberberg and Pastor (2009)
conducted a study in which they asked respondents to “think aloud” as there were responding to
a subset of items statements from the AOT scale. The think aloud exercise revealed that item
wording was ambiguous and participants had differential conceptualizations of item subject
matter (Zilberberg & Pastor, 2009).
The emergent 4-factor solution offered insight regarding how AOT scores reflect
Stanovich and West’s (2007) conceptualization of actively open-minded thinking. As mentioned
previously, Stanovich and West (1997, 2007) originally drafted items to reflect six facets of
actively open-minded thinking, but two of the four factors within the emergent 4-factor solution
appeared to be the result of a merging of originally hypothesized subscales. More specifically,
factor 1 seemed to represent a merging of Stanovich and West’s (2007) originally specified
dogmatic and categorical thinking subscales, with items from each subscale loading on the
14
factor. These items reflected a similar, concrete or dichotomizing thinking style. Similar to factor
1, factor 2 appeared to be the result of a merging of previously specified subscales. Items loading
on factor 2 appeared to represent the merging of the flexible thinking and openness to values
subscales originally specified by Stanovich and West (2007), with items from both subscales
loading on the factor to represent an emergent factor that reflected a flexible/open-minded
thinking style.
One could make the argument that none of the items loading on the four factors in the
emergent 4-factor solution address central components of actively open-minded thinking
previously detailed by researchers who have defined the construct. However, items loading on
factors 3 and 4 in the 4-factor solution seem to be addressing thinking characteristics particularly
tangential to the actively open-minded thinking construct. More specifically, items loading on
factor 3 appeared to represent a previously unspecified aspect of Stanovich and West’s (2007)
concept of actively open-minded thinking construct, with these items reflecting perseverance in
one’s beliefs. Factor 4 was defined by only two items with salient factor loadings that reflected
the propensity of individuals to engage in counterfactual thinking. Although these two items that
loaded on factor 4 were the two items originally developed to represent Stanovich and West’s
(2007) counterfactual thinking subscale, it is unclear as to whether they should represent core
aspects of actively open-minded thinking. The presence of these two factors (factors 3 and 4) in
particular raises the question of what are the essential components of actively open-minded
thinking.
Stanovich and West (1997) provide one way of thinking about actively open-minded
thinking when they described it as the tendency of people to engage in flexible or open-minded
thinking and avoid dogmatic or categorical thinking. It was this conceptualization of actively
15
open-minded thinking that inspired and informed the development of their AOT scale. Other
researchers, such as Baron, conceptualize actively open-minded thinking differently. Baron
(1991) characterizes actively open-minded thinking as a search for evidence, the extent to which
is determined by the importance of the question, and an objectivity in one’s consideration and
review of the evidence. An instrument measuring actively open-minded thinking as described by
Baron would look quite different than the Stanovich and West (2007) AOT scale. As mentioned
previously, it could be argued that the components of Stanovich and West’s (2007) instrument
might be related to actively open-minded thinking, but are not necessarily essential features of
the construct.
The Stanovich and West (2007) AOT scale has been used in many research endeavors,
however, development and purpose of the scale is relatively unclear. Did Stanovich and West set
out to develop a psychometrically sound (i.e., valid and reliable) measure of respondents’
propensity to engage in actively open-minded thinking? Or was their intent to develop a measure
that evaluates tangentially related aspects of actively open-minded thinking that relates well to
other measures of similar characteristics (e.g., cognitive ability, need for cognition, argument
evaluation)? One way of viewing the current Stanovich and West (2007) AOT instrument,
consistent with the latter question, is that it is comprised of features related to actively open-
minded thinking, but not necessarily essential to the definition of the construct. The development
of the instrument may have been guided by the goal of measuring things related to good
thinking, not necessarily by the intention to reliably and validity measure actively open-minded
thinking itself. It is therefore not surprising that scores on the AOT scale relates to other
constructs in expected ways; however these relationships are between aspects related to actively
open-mined thinking and other constructs, not between the construct and actively open-minded
16
thinking itself. If the AOT scale was developed with the first purpose in mind, we recommend
that researchers review the literature and revisit the conceptualization of the actively open-
minded thinking construct in order to develop a more valid and reliable scale that reflects core,
theoretical components of actively open-minded thinking. This process might begin with
revisiting Baron’s conceptualization and initiating the development of a measure of actively
open-minded thinking that directly reflects this characteristic of good thinking.
17
Table 1
Item Descriptive Statistics
Note. N = 430.
Item M SD Skewness Kurtosis
1 4.07 1.37 -0.20 -0.93
2 3.44 1.19 0.03 -0.43
3 4.67 1.19 -0.80 0.04
4 5.40 0.80 -1.65 3.79
5 4.54 1.28 -0.76 0.04
6 5.26 0.98 -1.56 2.45
7 3.86 1.50 -0.07 -1.10
8 4.82 1.17 -1.04 0.75
9 3.71 1.41 0.10 -0.93
10 4.77 0.94 -0.82 1.14
11 4.30 1.41 -0.44 -0.82
12 3.17 1.08 0.09 -0.13
13 3.16 1.37 0.09 -0.86
14 4.97 1.20 -1.08 0.38
15 3.27 1.27 0.10 -0.61
16 4.15 1.19 -0.15 -0.72
17 4.67 1.26 -0.79 -0.01
18 4.91 0.98 -1.22 2.21
19 3.08 1.28 0.18 -0.51
20 3.94 1.19 -0.18 -0.41
21 3.80 1.15 0.11 -0.50
22 4.66 1.01 -0.51 0.03
23 4.17 1.23 -0.15 -0.58
24 4.87 1.25 -1.24 1.18
25 3.93 1.46 -0.34 -0.84
26 4.43 1.01 -0.30 -0.25
27 4.32 1.02 -0.49 0.40
28 3.78 1.41 -0.29 -0.76
29 4.52 0.95 -0.16 -0.43
30 4.56 1.01 -0.40 0.16
31 2.60 1.13 0.55 0.12
32 3.64 0.95 -0.03 0.82
33 4.65 1.06 -0.68 0.50
34 3.96 1.24 -0.34 -0.33
35 3.76 1.19 0.08 -0.54
36 3.36 1.01 0.30 0.04
37 3.96 1.06 -0.08 0.12
38 4.96 1.15 -1.00 0.33
39 4.89 1.06 -0.86 0.37
40 3.22 1.05 0.49 0.34
41 4.51 0.93 -0.37 0.60
18 Table 2
4-Factor Solution Pattern Coefficients, Item Communalities, and Variance Accounted for by Each Factor
Note. N = 430. (R) indicates that item is reflected.
1 2 3 4
1 Even though freedom of speech for all groups is a worthwhile goal, it is unfortunately necessary
to restrict the freedom of certain political groups. (R)0.22 0.05 -0.04 0.05 0.06
2 What beliefs you hold have more to do with your own personal character than the experiences
that may have given rise to them. (R)0.20 -0.03 0.13 0.18 0.11
3 I tend to classify people as either for me or against me. (R) 0.53 -0.02 -0.03 -0.05 0.27
4 A person should always consider new possibilities. 0.36 0.27 -0.26 -0.02 0.27
5 There are two kinds of people in this world: those who are for the truth and those who are against
the truth. (R)0.55 0.02 0.12 0.06 0.36
6 Changing your mind is a sign of weakness. (R) 0.57 0.07 -0.20 -0.01 0.33
7 I believe we should look to our religious authorities for decisions on moral issues. (R) 0.16 0.28 0.31 0.00 0.25
8 I think there are many wrong ways, but only one right way, to almost anything. (R) 0.50 0.11 0.19 0.04 0.37
9 It makes me happy and proud when someone famous holds the same beliefs that I do. (R) 0.35 -0.01 0.21 -0.06 0.20
10 Difficulties can usually be overcome by thinking about the problem, rather than through waiting
for good fortune.0.07 0.32 -0.19 0.03 0.15
11 There are a number of people I have come to hate because of the things they stand for. (R) 0.53 -0.12 -0.08 -0.23 0.29
12 Abandoning a previous belief is a sign of strong character. -0.32 0.37 0.22 -0.07 0.21
13 No one can talk me out of something I know is right. (R) 0.11 -0.06 0.40 0.00 0.19
14 Basically, I know everything I need to know about the important things in life. (R) 0.45 0.01 0.12 0.15 0.27
15 It is important to persevere in your beliefs even when evidence is brought to bear against them.
(R)0.04 0.21 0.54 0.05 0.38
16 Considering too many different opinions often leads to bad decisions. (R) 0.44 0.07 0.17 0.08 0.29
17 There are basically two kinds of people in this world, good and bad. (R) 0.52 0.07 0.14 -0.08 0.33
18 I consider myself broad-minded and tolerant of other people's lifestyles. 0.24 0.34 -0.15 -0.11 0.23
19 Certain beliefs are just too important to abandon no matter how good a case can be made against
them. (R)0.08 0.12 0.56 -0.07 0.36
20 Most people just don't know what's good for them. (R) 0.49 -0.06 0.17 -0.02 0.29
21 It is a noble thing when someone holds the same beliefs as their parents. (R) 0.36 0.07 0.28 -0.02 0.26
22 Coming to decisions quickly is a sign of wisdom. (R) 0.59 0.02 -0.02 0.08 0.36
23 I believe that loyalty to one's ideals and principles is more important than "open-mindedness." (R)0.42 0.35 0.25 0.02 0.48
24 Of all the different philosophies which exist in the world there is probably only one which is
correct. (R)0.36 0.28 0.22 0.10 0.37
25 My beliefs would not have been very different if I had been raised by a different set of parents. (R)0.03 -0.15 -0.05 0.68 0.47
26 If I think longer about a problem I will be more likely to solve it. 0.03 0.28 -0.08 -0.06 0.09
27 I believe that the different ideas of right and wrong that people in other societies have may be
valid for them.0.09 0.49 0.04 0.07 0.28
28 Even if my environment (family, neighborhood, schools) had been different, I probably would
have the same religious views. (R)0.06 -0.06 0.02 0.54 0.30
29 There is nothing wrong with being undecided about many issues. 0.19 0.26 -0.12 -0.06 0.13
30 I believe that laws and social policies should change to reflect the needs of a changing world. 0.01 0.53 0.05 0.05 0.30
31 My blood boils over whenever a person stubbornly refuses to admit he's wrong. (R) 0.15 -0.17 0.14 -0.26 0.13
32 I believe that the "new morality" of permissiveness is no morality at all. (R) 0.29 0.05 0.16 0.07 0.15
33 One should disregard evidence that conflicts with your established beliefs. (R) 0.35 0.26 0.15 0.23 0.37
34 Someone who attacks my beliefs is not insulting me personally. 0.04 0.42 0.05 -0.09 0.20
35 A group which tolerates too much difference of opinion among its members cannot exist for long.
(R)0.37 0.13 0.09 -0.01 0.20
36 Often, when people criticize me, they don't have their facts straight. (R) 0.25 -0.12 0.12 -0.05 0.09
37 Beliefs should always be revised in response to new information or evidence. -0.15 0.51 0.15 0.05 0.27
38 I think that if people don't know what they believe in by the time they're 25, there's something
wrong with them. (R)0.56 0.00 -0.15 0.04 0.30
39 I believe letting students hear controversial speakers can only confuse and mislead them. (R) 0.50 0.19 -0.04 0.25 0.41
40 Intuition is the best guide in making decisions. (R) 0.08 -0.11 0.25 0.15 0.12
41 People should always take into consideration evidence that goes against their beliefs. 0.06 0.48 0.08 0.05 0.27
Percent of Variance Accounted for by each Factor 15.84 4.33 3.36 2.63
Factor
Item textItem
Item
communalities
19
Figure 1. AOT Scale EFA Parallel Analysis Results
Figure 1. AOT scale EFA parallel analysis results. N = 430.
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Raw Data Eigenvalues
Generated Data Eigenvalues
20
References
Baron, J. (1985). Rationality and intelligence. New York: Cambridge University Press.
Baron, J. (1988). Thinking and deciding. New York: Cambridge University Press.
Baron, J. (1991). Beliefs about thinking. In J. F. Voss, D. N. Perkins, & J. W. Segal (Eds.),
Informal reasoning and education, pp. 169-186. Hillsdale, N. J. : Erlbaum.
Costa, P. T., & McCrae, R. R. (1992). Revised NEO personality inventory. Odessa, FL:
Psychological Assessment Resources.
DeCarlo, L. T. (1997). On the meaning and use of kurtosis. Psychological Methods, 3(2), 292-
307.
Ennis, R. H. (1987). A taxonomy of critical thinking dispositions and abilities. In J. Baron & R.
Sternberg (Eds.), Teaching thinking skills: Theory and practice (pp. 9-26) New York:
Freeman.
Epstein, S., & Meier, P. (1989). Constructive thinking: A broad coping variable with specific
components. Journal of Personality and Social Psychology, 57, 332 – 350.
Gorsuch, R.L. (1983). Factor Analysis, 2nd Edition, Hillsdale, N.J.: Erlbaum.
Horn, J. L. (1965). A rationale and test for the number of factors in factor analysis.
Psychometrika, 30, 179-185.
Kaiser, H.F., 1974. Index of factorial simplicity. Psychometrika 39,
31–36.
Kline, R. B. (2005). Principles and Practice of Structural Equation Modeling (2nd
Edition). New
York: Guilford.
21
Moshman, D. (1994). Reasoning, metareasoning, and the promotion of rationality. In A.
Demetriou & A. Efklides (Eds.), Intelligence, mind, and reasoning: Structure and
development (pp. 135-150), Amsterdam: Elsevier.
Norris, S. P. (1992). Testing for the disposition to think critically. Informal Logic, 14, 157-164.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-
Hill.
Paulhus, D. L., & Reid, D. B. (1991). Enhancement and denial in socially desirable responding.
Journal of Personality and Social Psychology, 60, 307 – 317.
Perkins, D. N., Faraday, M., & Bushey, B. (1991). Everyday reasoning and the roots of
intelligence. In J. Voss, D. Perkins, & J. Segal (Eds.), Informal reasoning and education
(pp. 83-105). Hillsdale, NJ: Erlbaum.
Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (1991). Measures of Personality and Social
Psychological Attitudes (Vol. 1, pp. 560 – 564). San Diego, CA: Academic Press.
Sá, W. C., West, R. F., & Stanovich K. E. (1999). The domain specificity and generality of belief
bias: Searching for generalizable critical thinking skill. Journal of Educational
Psychology, 91, 497-510.
Stanovich, K. E., & West, R. F. (1997). Reasoning independently of prior belief and individual
differences in actively open-minded thinking. Journal of Educational Psychology, 89(2),
342-357.
Stanovich, K. E., & West, R. F. (1998). Individual differences in rational thought. Journal of
Experimental Psychology, 127 (2), 161-188.
Stanovich, K. E. & West, R. F. (2007). Natural myside bias is independent of cognitive ability.
Thinking & Reasoning, 13, 225-247.
22
Troldahl, V., & Powell, F. (1965). A short-form dogmatism scale for use in field studies. Social
Forces, 44, 211 – 215.
Zilberberg, A. & Pastor D. A. (2009). Willingness to consider contradictory evidence scale: A
mixed methods approach. Paper presented at the Northeastern Educational Research
Association, Rocky Hill, CT.