in the eye of the beholder: how information …...(democrats vs. republicans) and ideological...
TRANSCRIPT
In the Eye of the Beholder:
How Information Shortcuts Shape Individual Perceptions of Bias in the Media
Research has shown that humans are biased information processors. This study investigates an
important potential example of biased information processing: when ex ante assessments of a
media outlet’s ideological orientation cause individual’s to perceive bias in balanced news
coverage. We conduct an experiment in which participants evaluated the content of a news
report about the 2004 presidential election identified as originating from CNN, FOX or a
fictional TV station. Our results suggest that in an increasingly fragmented media marketplace,
individuals not only distinguish between media outlets but, more importantly, outlet brand
names, and the reputations they carry, function as heuristics, heavily influencing perceptions of
bias in content. As a result, individuals sometimes create bias, even where none exists. This
suggests that assessments of media content operate on a more nuanced level than has been
captured in previous research.
Matthew A. Baum ([email protected])
Harvard University John F. Kennedy School of Government
79 JFK Street Cambridge, MA 02468 (on leave from UCLA)
and
Phil Gussin
([email protected]) University of California, Los Angeles
Department of Political Science, Box 951472 Los Angeles, CA 90095-1472
1
Are the news media biased? If so, does this imbalance tilt leftward or rightward? The
debate on this topic and its implications has raged for decades without resolution. On one end of
the spectrum, conservatives, like FOX News anchor John Gibson, claim "The mainstream media,
typified by the Big Three newscasts, [is] rooting for the Dems to win and slant coverage to make
it happen." (The Big Story with John Gibson, 11/1/06). Liberals, like New York Times columnist
Paul Krugman, counter “One of the great jokes of American politics is the insistence by
conservatives that the media have a liberal bias” (New York Times 11/1/00).
These discrepancies beg the question of whether or not bias is an objective characteristic of
media content that social scientists can measure with some degree of precision. Certainly, many have
tried. Self-described media watchdog groups such as the Media Research Center (MRC), the Center
for Media and Public Affairs (CMPA), and Fairness and Accuracy In Reporting (FAIR) claim to
objectively analyze media content. Yet they routinely disagree on the incidence, severity, and
direction of bias in the media. Scholarly attempts to assess media bias are similarly inconclusive
(e.g., Efron 1971; Patterson 1993; Sutter 2001, Alterman 2003, Groseclose and Milyo 2005).
An alternative possibility is that bias is, at least in significant measure, a matter of
perception; a consumer’s assessment based more on her own prior beliefs – however accurate or
inaccurate -- than the actual content of information presented by the media at a given point in
time. This raises the possibility that in an increasingly fragmented media marketplace, individuals
distinguish between media outlets. If so, outlet brand names, and the reputations they carry, may
function as heuristics, influencing perceptions of bias in content.
Prior research (Vallone et al. 1985) has shown that, all else equal, ideologues (that is,
liberals or conservatives) tend to view the media as biased against their own views. While
substantial research (e.g., Giner-Sorolla and Chaiken 1994; Gunther and Schmitt 2004; Morehouse
2
Mendez 2004; Peffley, et al. 2001) supports this so-called “Hostile Media Phenomenon” (HMP),
we believe further elaboration is necessary to adequately account for, and determine the political
implications of, individual perceptions of bias in the contemporary media environment.
For instance, most research on the HMP conceptualizes the news media as an
undifferentiated mass of information sources that individuals can (and do) reasonably characterize
as having a uniform political orientation (Eveland and Shah 2003; Giner-Sorolla and Chaiken 1994;
Peffley, et al. 2001). Yet the past two decades have seen a dramatic increase in the number and
variety of news sources. One consequence is that Democrats and Republicans are increasingly
likely to differ systematically in their assessments of specific media outlets.
In this study, we investigate whether individuals do, in fact, create at least some of the bias
they perceive. To do so, we employ a survey experiment involving a nationally representative
population sample provided by Knowledge Networks.1 Our participants evaluated the content of a
single news transcript -- focused on the 2004 presidential election -- variously attributed to the
FOX News Channel (henceforth “FOX”), CNN, or a fictional local TV news station. They
assessed the favorability of the report toward candidates Bush and Kerry. This allows us to
investigate the influence of ex ante beliefs concerning the political orientations of specific media
outlets on perceptions of bias.
THEORY
As many have noted (Hamilton 2003, Baum and Kernell 1999, Webster, et al. 2005), the
media environment has changed dramatically over the past two decades. In today’s fragmented
media marketplace, news consumers have a myriad of outlets from which to choose. These outlets
differ in many ways, such as their relative focus on local vs. national news, or proportions of hard
vs. soft news themes (Baum 2003). Most important for our purposes, contemporary news outlets
3
also vary in providing information that appeals to consumers with different partisan affiliations
(Democrats vs. Republicans) and ideological orientations (liberals vs. conservatives).
For instance, according to a 2006 Harris Poll, CNN attracts far more Democrats (42%)
than Republicans (19%), while far more Republicans (63%) than Democrats (12%) watch FOX
News. In the same survey, 61% of self-described FOX viewers identified themselves as
conservative compared to 35% describing themselves as moderate and only 4% as liberal. In stark
contrast, CNN’s self-reported audience was 61% moderate, 27% liberal and only 12%
conservative.2 Self-described conservatives are thus 49 percentage points more likely to watch
FOX than CNN, while liberals are 23 percentage points more likely to watch CNN.
Substantial evidence also indicates that -- either through first hand experience or in
response to elite cues (Domke et al. 2000; Watts et al. 1999) -- individuals differ systematically in
their assessments of the reliability of specific media outlets. In another recent survey (Pew Center
2004a), 45% of Democratic respondents said they believed “all or most” news on CNN,
compared to only 25% of Republicans. Across seventeen broadcast, cable, and print outlets
addressed in the survey, FOX News Channel was the sole outlet for which Republicans expressed
greater confidence (29%) than Democrats (24%).3
Notwithstanding the significant changes in the media environment, citizens presumably
process information similarly as in the past, with liberals and conservatives still tending to view
as hostile information not explicitly supportive of their political perspective. Moreover, since they
tend to view their position as superior (Vallone et al. 1985), they should be as likely as ever to
view balanced coverage as inappropriate at best and unconscionably biased at worst.
In its original formulation (Vallone et al. 1985), the HMP holds that ideologues tend to
perceive the media as hostile to their own perspective.4 Yet, given the diversity of the current media
4
environment, and the extent to which individuals (rightly or wrongly) perceive some outlets as
reflecting perspectives across the full ideological spectrum, we suspect the utility of conceptualizing
the media as an undifferentiated whole with a uniform political orientation has diminished.
So, while ideologues may have a tendency to view news coverage with suspicion, there is
no reason to believe that, given repeated exposure to a media outlet – again, either through first
hand experience or indirectly, via cues from trusted elites (Lupia and McCubbins 1998, Popkin
1994, Domke et al. 2000; Watts et al. 1999) – ideologues are incapable of updating that assessment
(Giner-Sorolla and Chaiken 1994; Morehouse Mendez 2004). If so, it follows that in the
contemporary media environment ideologues make such determinations primarily on an individual,
per-media-outlet basis. Thus, while ideologues may develop an overall impression of the media as a
whole, they are increasingly likely to form individual media outlet “stereotypes.”5 Moreover, these
stereotypes should affect the manner in which individuals’ process the information each media
outlet provides. In the same way that individuals use partisan stereotypes to simplify choices that
otherwise might be cognitively demanding or complex (Rahn 1993, Ottati 1990), media outlet
stereotypes can be useful when ideologues encounter ambiguous (e.g. balanced) information.
As Vallone et al. (1985) suggest, ideologues are unlikely to be agnostic in their evaluation
of media outlets and the content they provide. From their perspective, outlets that are not friendly
are, by definition, hostile. Given the diversity of the media environment, it is also possible that
ideologues will conclude that an outlet is not hostile if they perceive its content as tending to
cohere with their own ideological orientation.
In other words, we argue that ideologues are not limited to perceiving only hostility.
Rather, by employing a modified either you’re for me or against me heuristic, they are likely to
divide the world into two relatively clearly demarcated camps: those that are hostile to their
5
ideological perspective and those that are not. This raises the possibility that an individual’s ex
ante assessment of the ideological orientation of a specific media outlet conditions subsequent
responses to the information it provides.
Additional research (e.g. Druckman 2001a, 2001b; Lupia and McCubbins 1998, Gunther
and Schmidt 2004) has shown that individual responses to new information frequently hinge on
evaluations of its source. For instance, individuals are less susceptible to persuasion or framing
effects when they believe the information source lacks credibility (Lupia and McCubbins 1998,
Druckman 2001b). One implication is that if an individual believes, ex ante, that a media outlet is
hostile (friendly), she is more (less) likely to perceive information from that outlet as biased in
favor of the opposing ideological perspective.6
This, of course, begs the question of how individuals evaluate the credibility of media
messages. In general, we suspect ideologues base such assessments on the extent to which an outlet
provides information perceived as confirming or challenging their pre-existing beliefs (Popkin
1994, Zaller 2002, Lupia and McCubbins 1998). In short, ideologues tend to view a source as
credible when they perceive it as typically providing confirmatory information and non-credible
when it does not. More importantly, once they make this determination, the media outlet label
functions as a heuristic, affecting how they perceive subsequent information from that outlet.
Research in social cognition theory, in turn, suggests that people use heuristic cues, or
cognitive shortcuts, in order to manage the flood of information that washes over them daily
(Fiske and Taylor 1984; Hastie 1986). Indeed, Popkin (1994) argues that typical individuals can
use heuristics in order to make reasoned political judgments without a great deal of information.
A prior belief that a given media outlet is or is not hostile can act as an important judgmental
heuristic (Giner-Sorolla and Chaiken 1994; Kahneman, et al. 1982).
6
If, as we contend, ideologues distinguish between differing media outlets, this raises the
possibility that they may assess the content of those outlets in part through reliance on a relatively
simple heuristic: the outlet label. Hence, if a conservative ideologue believes that, say, CNN has a
liberal bias, this facilitates the discounting (or rejection) of information from CNN. This also
increases the likelihood of perceiving balanced coverage on CNN as having a liberal slant.
Alternatively, if the same conservative believes FOX has a conservative bias, this facilitates
greater acceptance of information from FOX. It also reduces the likelihood of perceiving balanced
coverage as hostile. Either way, the brand name functions as an information shortcut, allowing
ideologues to assume that the news presented by CNN and FOX is either hostile or sympathetic to
their ideological orientation. By assuming that information from a given outlet is “probably
dissonant” or “probably consonant” through reference to a single, easy-to-use information
shortcut, an individual can save substantial time and cognitive energy.7 A hypothesis follows.
H1: Heuristic Hypothesis. Individuals who believe that an outlet is slanted in a liberal
(conservative) direction will believe that news coverage by that outlet is slanted in a
liberal (conservative) direction.
It also follows from this argument that the stronger an individual’s perception regarding an
outlet’s ideological hostility, or non-hostility, the greater the extent to which the label will
influence that individual’s perceptions of bias in news content. For instance, if a liberal believes
that a specific outlet has, say, a moderately conservative ideological orientation, she is likely to
view information from that outlet with less suspicion than information from an outlet she believes
is extremely conservative. A second hypothesis follows:
H2: Heuristic Intensity Hypothesis: The relationship predicted in H1 will become more
prevalent as the belief that an outlet is ideologically slanted strengthens.
7
Because, in turn, political sophisticates are more likely to perceive ideological valence in
information (Zaller 1992), they are presumably also more likely to ascribe ideological valence to
media outlets. Hence, the outlet labels are likely to hold greater heuristic value for these
individuals. An additional hypothesis follows.
H3: Political Awareness Heuristic Hypothesis: The relationship predicted in H1 will become
more prevalent as political awareness increases.
This argument does not imply that individuals no longer develop assessments of the ideological
orientation of the mass media as a whole. Clearly, they do. Moreover, global assessments of the
media should affect how individuals interpret information from a specific outlet. However, if an
individual has ascribed an ideological orientation to a given outlet, the heuristic value of the outlet
label should have a greater effect on her evaluation of content from that outlet than her global
assessment of the media. A final hypothesis follows.
H4: Familiarity Hypothesis: Individuals’ beliefs regarding the ideological orientations of
specific media outlets will have a stronger influence on their perceptions of news
content than global assessments of the ideological slant of the media as a whole.
DATA AND METHODS
Design. We investigate whether varying the information that identifies specific media
outlets’ influences consumers’ perceptions of literally identical substantive news content. We
implemented this experiment with a national population sample of 1014 individuals who
participated in a series of Web TV-based surveys conducted by Knowledge Networks.8
[Table 1 here]
After completing a pre-experiment survey, all participants read an identical news
transcript reporting on the 2004 presidential election (see Table 3) that varied only in its presumed
8
source.9 We randomly assigned participants to a version of the transcript designed to appear as
having originated on either CNN, FOX, or a fictional TV news station (KNWZ), with all
identifying information, including reporter names and the outlet logo, altered to insure
consistency with the outlet identification (including fictional reporter names for the unfamiliar
station).10 (The outlet logos are presented in Appendix 1.) We divided the transcript into eight
segments, including one quote from each major party candidate, as well as six comments that
were, to the greatest extent possible, balanced (in terms of positive, negative, and neutral
references to the candidates).11 Participants read each comment and evaluated whether and to
what extent it was favorable to or critical of Senator Kerry and President Bush.12
Dependent Variable. After reading each transcript segment, participants read the following
prompt: “The coverage of JOHN KERRY (GEORGE BUSH) in the above section was:...” We then
presented six response options for each candidate: (1) “Very favorable”, (2) “Somewhat favorable”,
(3) “Balanced/Even handed”, (4) “Somewhat unfavorable,” (5) “Very unfavorable”, and “No
coverage of Kerry (Bush).”13 Because, in most cases, we do not have strong prior expectations
regarding differences across transcript segments, for our key dependent variable we employ the sum
of all eight Bush evaluations (each varying from 1 to 5), minus the sum of all eight Kerry
evaluations (each also ranging from 1 to 5). The resulting scale thus theoretically runs from -32 to
+32 with -32 representing maximum relative favorability toward Kerry, 0 representing neutrality or
balance, and +32 representing maximum relative favorability toward Bush.14 In our sample,
however, the actual coding on the summary scale ranges from -28 to +28 (
!
µ =1.37 and
!
" =6.4).
Independent Variables. Our key causal variables are based on the distance between
participants’ self-assessed ideological orientations and their ratings (on identical 7-point scales) of
the ideological orientations of CNN and FOX, as well as, for the unfamiliar outlet, the distance
9
between themselves and their global assessments of the ideological orientation of the mass media
(the latter of which is based on a 5-point scale). These variables are constructed by subtracting the
individual’s rating of a given outlet’s ideology, or their global assessment of the media’s
ideological orientation, from their self-ideological rating. The resulting scales run from -6 to +6 for
the FOX and CNN distance indicators, and -5 to +5 for the overall media distance indicator. In
each case, positive values indicate that the participant rates him or herself as more liberal than she
rates a given outlet, or the media as a whole, while negative values indicate that the participant
rates him or herself as more conservative than a given outlet or the media as a whole. We interact
each ideological distance variable with the treatment conditions, as well as with political
knowledge (based on the number of correct responses to five factual knowledge questions,
normalized to a 0-1 interval, where µ=.49 and σ=.28). (In Appendix 2, we present the questions
comprising the political knowledge scale.) For hypothesis tests involving regression analyses, we
also include controls for participants’ demographic characteristics.15
STATISTICAL RESULTS
Figure 1 presents the distribution of our participants’ ex ante ratings of the ideological
orientations of CNN and FOX, broken out into three ideological groups: liberals, moderates, and
conservatives. It is important to note that while a substantial number of participants considered
FOX significantly more conservative than CNN, few held the opposing position, and none rated
FOX as extremely liberal while rating CNN as extremely conservative. Overall, only 6.6% (N=17)
of liberals (including leaners) rated FOX as liberal or extremely liberal. This compares to 44%
(N=114) of liberals who rated FOX as conservative or extremely conservative. Nearly precise
inverse percentages obtain for conservatives, with 7.1% (N=24) rating CNN as conservative or
extremely conservative, compared to 43% (N=146) rating it as liberal or extremely liberal. The
10
implication is that predictions based on the less common assumptions derive from a quite small
number of cases. Consequently, even though the case where respondents believe, ex ante, that
FOX is liberal and CNN is conservative is associated with a large value on the ideological distance
scale, any results based on this scenario should be interpreted with caution, as they are less reliable
than those based on the other, more common, scenarios.16
[Figure 1 here]
Conversely, we anticipate the strongest and most reliable results for the scenario in which
participants perceive FOX as conservative and CNN as liberal, as this scenario is much more
common and also entails substantial ideological distance between the outlets.17 The other two
scenarios – in which participants perceive the outlets as either similarly liberal or conservative –
should fall in between. While the ideological distance between the outlets is, by definition, small, a
fair number of participants fall into these categories. Hence, we anticipate relatively weak, but
reasonably reliable, results for these two cases.
With these assumptions in mind, we turn to our hypothesis tests. Our four hypotheses
predict that, upon exposure to a media outlet’s brand name, individuals will perceive the content
from that outlet as consistent with their prior beliefs concerning its ideological orientation (H1),
that the stronger the belief that an outlet is ideologically slanted, the greater the extent to which its
label will influence perceptions of bias in news content (H2, the Heuristic Intensity Hypothesis) --
a tendency that will be more prevalent among political sophisticates than political novices (H3, the
Political Awareness Heuristic Hypotheses) -- and that a specific (familiar) network’s outlet label
will influence an individual’s perceptions of media content more than global assessments of the
media (H4, the Familiarity Hypothesis).
To investigate the main treatment effects (H1), we begin with difference of means tests.
11
We then present a more nuanced test of H1 and test our conditional hypotheses (H2-H4) through
multivariate regression analysis. Beginning with difference of means tests of H1, Table 2 compares
the effects of exposure to the CNN, FOX, and KNWZ outlet labels, both in the aggregate and
disaggregated by participants’ ideological orientations. At the bottom of the table, we compare the
difference in ratings of CNN and FOX with each other as well as with KNWZ. A positive
(negative) difference denotes movement in a pro-Bush (pro-Kerry) direction.
[Table 2 here]
The results shown in Table 2 clearly support H1. As noted earlier, Figure 1 indicates that while
liberals rate FOX as considerably more conservative than conservatives, and conservatives rate
CNN as considerably more liberal than liberals, all groups of respondents rate FOX as more
conservative than CNN. Hence, we anticipate that respondents exposed to the FOX variant of the
transcript will code the content as more pro-Bush than will respondents exposed to the CNN
variant. In fact, this is just what we find. Regardless of whether we look at self-described liberals
(3.27, p≤ .001) or conservatives (2.25, p≤ .001), or whether we look at the aggregate sample (1.75,
p≤ .001), those who saw the FOX variant rated the transcript as statistically significantly more
favorable to Bush, relative to Kerry, than those who saw the CNN version. The sole exception is
among moderates, who rated CNN and FOX ex ante as considerably less ideologically slanted than
their liberal and conservative counterparts. Among moderates, the treatment effects, though
similarly signed (.60), are, as one might anticipate, statistically insignificant.
The results also indicate that liberals and conservatives rated KNWZ, a media outlet label
about which they had no prior information, as lying in between FOX and CNN; less conservative
than FOX and less liberal than CNN. In all but one instance (conservative ratings of CNN vs.
12
KNWZ), the differences are statistically significant. Once again, however, as we anticipated,
among moderates none of the differences are significant.
The HMP holds that ideologues tend to perceive balanced coverage as hostile. The results
for conservatives in Table 2 are consistent with that expectation. In all three treatments,
conservatives perceived the coverage as hostile (i.e., pro-Kerry), although significantly less so on
FOX than KNWZ or CNN. Interestingly, however, among liberals, the results are less consistent
with the HMP. While liberals did perceive hostility in the FOX treatment, they did not perceive
hostility in the CNN treatment. In other words, by switching the media outlet labels, we induced a
corresponding reversal in perceptions of valence.
As noted earlier, we have no clear expectations regarding how respondents might code the
various segments of our transcript relative to one another. However, the segments do clearly
differ. Two consist solely of direct candidate statements, one includes no substantive content at all
(by design), while others contain positive or negative information about one or both candidates.
Hence, as a robustness test, we disaggregate the transcript and repeat the CNN vs. FOX difference
of means tests across all eight individual transcript segments. Table 3 presents the results.
[Table 3 here]
Our one clear expectation was that Segment 2, which contains absolutely no substantive
political content, would be least likely to be perceived as slanted. After all, respondents
presumably need some political cue upon which to base perceptions of ideological bias. The results
clearly bear this out with respondents -- both in the aggregate (third column) and disaggregated
into ideological subgroups (fourth and fifth columns) -- perceiving no significant differences.
Segment 2, however, is an exception. In all seven segments containing substantive political
13
content, participants coded the transcript as more favorable to Bush when it had a FOX label than
when it had a CNN label. The difference is statistically significant in five of the seven cases.
The fourth and fifth columns in the table report the results for liberals and conservatives,
respectively. Here, excluding Segment 2, participants coded the FOX-labeled transcripts as more
pro-Bush than the CNN-labeled transcript in 12 out of 14 instances, nine of which are statistically
significant. Once again, as one might expect, no statistically significant differences between FOX
and CNN arose among moderates. That is, moderates effectively coded every segment as
ideologically neutral.18
Our predictions thus far are based on our respondents’ overall average ex ante ratings of
the ideological orientations of FOX and CNN. Of course, individual respondents vary widely in
this regard, even within ideological subgroups. Consequently, a more direct test of our hypotheses
requires that we take into account the meaning for each respondent of the FOX and CNN brand
names. This, more nuanced analysis is more easily accomplished through multiple regression.
Table 4 thus presents three OLS models that test all four hypotheses. In Model 1, which tests H1
and H2, we employ as our key causal variables the aforementioned ideological distance
indicators. In order to determine the effects of ex ante assessments of the ideological orientation
of individual outlets relative to respondents’ self-reported ideologies, we interact our ideological
distance indicators with the treatment to which a given respondent was exposed.
[Table 4 here]
For ease of interpretation, we employ a statistical simulation procedure (King et al. 2000)
to transform the coefficients from Table 4 into expected values of the dependent variable, at
different levels of the key causal variables, as well as to estimate the statistical significance of the
differences in the dependent variable – net perceived favorability of coverage for Bush, relative to
14
Kerry – associated with varying outlet identifying information. Additionally, in order to clarify
the substantive significance of the numerical changes in the dependent variable associated with
variations in treatment conditions, we also describe the results in terms of standard deviation
shifts. One standard deviation (6.4) represents about 11% of the observed range in the candidate
favorability scale (from -28 to +28). This is equivalent to about a one-point shift in relative
favorability on an individual coded segment (based on the -4 to +4 scales).19 In Figure 2 we
graphically illustrate the effects of moving from FOX to CNN outlet identification. (The expected
values and differences underlying Figure 2 are presented in Section A of Appendix Table A-1.)
[Figure 2 here]
Figure 2 presents 12 scenarios (each represented by an individual curve) -- divided across
four graphics -- concerning participants’ ratings of their own ideologies and those of FOX and
CNN.20 Specifically, the graphs separately plot -- for liberal, moderate, and conservative
participants -- the expected transcript ratings among participants who saw the FOX and CNN
identifying information, under four distinct pairs of assumptions concerning the outlets’
ideological orientations, where participants perceive: (1) FOX as extremely conservative and
CNN as extremely liberal (top-left graphic); (2) FOX as liberal and CNN as conservative (top-
right graphic); (3) both FOX and CNN as extremely conservative (bottom-left graphic); and (4)
both FOX and CNN as extremely liberal (bottom-right graphic). This yields the aforementioned
12 distinct scenarios (three ideological groups x four sets of assumptions concerning outlet
ideologies). We also indicate, to the right of each curve, the magnitude and statistical significance
of the difference in expected values associated with moving from FOX to CNN identifying
information. We expect that as subjects move from a conservative to liberal media outlet label,
the sign on the difference will be negative, indicating movement in a pro-Kerry direction.
15
Conversely, as subjects move from liberal to conservative outlet label, we expect a positive
difference, denoting movement in a pro-Bush direction.
To evaluate H1, we focus on the six cases where participants rate the two outlets as having
distinct ideological orientations (shown in the top half of the figure). Here, we see that in every
case (liberals, moderates, and conservatives), participants’ evaluations are consistent with their
prior assumptions concerning the outlets’ ideological orientations. In other words, in each case
where participants rate CNN as more liberal than FOX, ex ante, they subsequently coded the
content of the CNN version of the transcript as more favorable to Kerry, relative to Bush, than the
FOX version. Conversely, in every case where participants rated FOX as more liberal than CNN,
ex ante, they subsequently rated the FOX version of the transcript as more favorable to Kerry than
the CNN version. These differences are statistically significant in five of the six cases, with the
sole exception being the rare case where conservatives view FOX as extremely liberal and CNN
as extremely conservative. Moreover, as expected, the magnitude and significance of the
differences are larger for the three relatively common cases in which (liberal, moderate, and
conservative) participants considered FOX more conservative than CNN, compared to the far less
common cases where they considered CNN more conservative than FOX.
For instance, among liberals who considered FOX extremely conservative and CNN
extremely liberal, those exposed to the FOX treatment rated the transcript as .85 standard deviations
more favorable to Bush, relative to Kerry, than their counterparts who saw the CNN treatment
(3.383 vs. -2.057, for a difference of -5.44 on the summary transcript favorability rating scale,
p≤.01). (Recall that negative signs indicate perceptions of greater relative favorability toward Kerry,
while positive signs indicate perceptions of greater relative favorability toward Bush.) This is
equivalent to a shift of about .85 points on the individual segment favorability scales (each of which
16
run from -4 to +4). The corresponding difference among self-described conservatives was 1.1
standard deviations (-.527 for FOX vs. –7.590 for CNN, for a difference of -7.063, p≤.01), or about
1.1 points on the individual segment favorability scale. In substantive terms, this latter difference
roughly equates to moving from, say, very to somewhat favorable, or from somewhat favorable to
balanced/even handed. These results, shown in the top-left graphic in Figure 2, clearly support H1.
To test the Heuristic Intensity Hypothesis (H2), we contrast the magnitude of the effects of
varying outlet labels for participants who see no ideological difference between the outlets
(shown in the lower half of the figure) with the effects for participants who do perceive the outlets
as ideologically distinct. In the relatively common case where participants perceive CNN as more
liberal than FOX, for each of the six possible comparisons, the effects of varying outlet labels are
larger relative to those for participants who consider both outlets ideologically indistinguishable.
As noted earlier, among liberals who perceive FOX as more conservative than CNN (top-
left graphic in Figure 2), moving from a FOX to a CNN label is associated with a .85 standard
deviation increase in perceived favorability toward Kerry (-5.440, p≤.01), or about .85 points on
an individual segment favorability scale. The corresponding effects among liberals who perceive
both outlets as equally conservative or liberal (bottom-left and bottom-right graphics in Figure 2,
respectively) are .01 (+.093, insig.) and .24 (-1.531, p≤.01) standard deviation differences in
perceived favorability toward Kerry, respectively. Among conservatives, the respective
differences are 1.1 standard deviations – or about 1.1 points on an individual segment scale -- for
the corresponding ideologically distinct case (-7.063, p≤.01), compared to .24 (-1.531, p≤.01) and
.49 (3.154, insig.) standard deviations for the ideologically equivalent cases, respectively. Finally,
among moderates, the corresponding differences are .98 standard deviations (-6.252, p≤.01) –
equivalent to about a one-point shift on an individual segment scale (say from very to somewhat
17
unfavorable) compared to .11 (-.719, insig.) and .36 (-2.342, p≤.05) standard deviations,
respectively. Across all six comparisons, the effects of varying outlet labels are statistically
significantly larger when subjects perceive the outlets as ideologically distinct than when they do
not (p≤.01 in every case).
Despite the far smaller sample, the same pattern arises across five of the six corresponding
comparisons where participants perceive CNN as far more conservative than FOX (top-right graphic
in Figure 2). In other words, across liberals, moderates, and (in one of two cases) conservatives, the
effects of varying outlet labels when participants perceive CNN as more conservative than FOX are
statistically significantly larger (p≤.01) than the corresponding effects when participants perceive
FOX and CNN as either similarly liberal or similarly conservative. Overall, our results support H2 in
11 out of 12 scenarios where participants perceive an ideological gap between FOX and CNN. The
sole exception is among conservatives who perceive FOX as more liberal than CNN. In this
instance, the coefficient is somewhat larger among conservatives who perceive both outlets as
similarly liberal. However, both differences are insignificant, making it difficult to draw a strong
conclusion regarding this case. Regardless, on balance, these results strongly suggest that
perceptions of actual news content are heavily influenced by consumers’ prior expectations
regarding what that content is likely to look like (i.e., the anticipated ideological orientation).
Turning to the Political Awareness Heuristic Hypothesis (H3), which predicts that outlet
labels hold greater heuristic value for political sophisticates, we turn to a second investigation –
shown in Model 2 of Table 4 -- in which we interact our political awareness indicator with the
several treatment conditions and ideological distance measures. This allows us to compare the
magnitude of the gap in perceptions of identical media content between politically sophisticated
liberals, moderates, and conservatives, on the one hand, and their less-sophisticated counterparts,
18
on the other. Figures 3 and 4 graphically illustrate the substantive results from this analysis for
participants one standard deviation above and below the mean level of political knowledge,
respectively. (The data underlying Figures 3 and 4 are presented in Sections C and B,
respectively, of Appendix Table A-1.)
[Figures 3 and 4 here]
The results strongly support H3. The gap between the ratings of FOX and CNN is larger
among high-knowledge participants (Figure 3) than among their low-knowledge counterparts
(Figure 4) in 10 out of 12 cases, and considerably so in most instances. Moreover, among high-
knowledge participants, the differences are correctly signed in every case where participants
perceive the two networks as ideologically distinct, and statistically significant in five out of six
instances. In Figure 3, this is apparent in the reversal of the direction of the effects as we move from
the top-left graphic (that is, the case where participants perceive FOX as extremely conservative
and CNN as extremely liberal, ex ante) to the top-right graphic (where participants perceive FOX as
extremely liberal and CNN as extremely conservative). In the former case, moving from FOX to
CNN is associated with an increase in perceived favorability toward Kerry, while in the latter the
same change in outlet labels is associated with an increase in perceived favorability toward Bush.
None of the differences among low-knowledge participants are statistically significant --
in fact, as is clear in Figure 4, they follow no consistent patterns at all -- while 9 out of 12
differences among high-knowledge participants are significant. In the latter case, all three
insignificant relationships arise for comparisons where, on the surface, meaningful differences
seem least appropriate or reliable: the rare case where conservative participants rate FOX as
extremely liberal and CNN as extremely conservative, as well as among liberals and moderates
who rate both CNN and FOX as extremely conservative.
19
As one would expect, in turn, the largest effects emerge in the most common scenario
overall: among those who rate FOX as conservative and CNN as liberal. Among liberals, the
magnitude of the effect of moving from FOX to CNN outlet identification is .70 standard
deviations larger for high-awareness participants than for their low-awareness counterparts: +6.34
points more favorable to Kerry on the summary favorability scale among high-awareness subjects,
compared to +1.87 points more favorable to Kerry among low-awareness subjects. This 4.47 point
treatment effect gap – equivalent to about a .7 point shift in relative favorability on an individual
segment -- is significant at p≤.05. The corresponding treatment effect gaps among moderates and
conservatives are 1.15 and 1.60 standard deviations, respectively (+7.34 points, p≤.01 and +10.21
points, p≤.01, respectively). The latter, 1.6 standard deviation, change among conservatives
represents nearly a two-point shift in relative favorability on an individual coded segment (say,
from somewhat favorable to Bush to somewhat favorable to Kerry). Overall, of the 10 cases where
we observe larger treatment effects among sophisticates, six of the differences between low- and
high-awareness participants – that is, the differences in the differences -- are themselves
statistically significant, including five out of the six key instances where participants perceived
FOX and CNN as ideologically distinct. Taken together, these results strongly support H3.
Finally, Model 3 of Table 4 tests the Familiarity Hypothesis (H4), which predicts that the
heuristic value of a familiar outlet label will influence an individual’s perceptions of media
content more than a global assessment of the media. For this test, we interact the treatment
conditions with the difference between participants’ self-ideological ratings and their global
assessments of the media’s ideological orientation. We then compare the effects of variations in
such assessments with those from the prior model, where we employed participants’ ratings of the
specific outlets’ ideological orientations.
20
At Table 5, we again transform the coefficients into expected values. The results strongly
support our hypothesis. Comparing the “Difference” column in the top and bottom sections of the
table, we see that for both the FOX and CNN treatments, and among liberals and conservatives,
variations in assessments of the specific outlets’ ideological orientations affect perceptions of
identical media content substantially more, both in magnitude and statistical significance, than
variations in global assessments of the media’s ideological orientation.
[Table 5 here]
Table 5 indicates that, among both liberals and conservatives, the effects on perceptions of
news content of varying assumptions regarding FOX’s ideological orientation (from extremely
conservative to extremely liberal) are dramatically larger than the effects associated with varying
a participant’s global media assessment: .61 standard deviation increases in perceived favorability
toward Kerry for both liberals and conservatives in the former case (p≤.01), compared to .03
standard deviation decreases for both liberals and conservatives in the latter (insig.)
For CNN, again among both liberals and conservatives, the effects of varying ex ante
perceptions from extremely conservative to extremely liberal are far larger than those associated
with varying a participant’s global media assessment: .86 standard deviation increases in perceived
favorability toward Kerry for both liberals and conservatives in the former case (p≤.01), compared
to .05 standard deviation increases, again for liberals and conservatives, in the latter (insig.).21 In
short, as predicted, specific, widely recognized media outlet brand names influence consumers’
perceptions of news content more than global assessments of the media’s ideological orientation.
Table 6 summarizes our findings, as well as identifying the tables where we have
presented the results for each hypothesis. The last column indicates that the proportion of tests
supporting our hypotheses ranges from a low of 83% (for H1 and H3), to a high of 100% (for H1,
21
H2, and H4). Overall, we find support for our theory in 86% of all hypothesis tests (49 out of 57
total possible comparisons). Moreover, this summary is based on an extremely conservative
criterion. Most of the unsupportive results emerge for either the least common scenario (where
FOX is perceived as extremely liberal and CNN as extremely conservative) or for instances and
among participants where we arguably should have found insignificant results, or based on a
narrow definition of a supportive outcome. Consequently, our results could reasonably be
interpreted as almost perfectly supporting our predictions.
[Table 6 here]
CONCLUSION
The question of whether or not the media are ideologically biased remains controversial.
While this study certainly cannot resolve this debate, it has clarified the conditions under which
typical individuals are likely to perceive the media -- or, more precisely, individual media outlets
-- as ideologically hostile.
Our results are broadly consistent with Vallone et al’s (1985) finding of a hostile media
phenomenon, along with those of subsequent studies concerning the importance of ideology and
political sophistication. We extended this argument by applying the implications of recent
research on information processing -- including the roles of source credibility and heuristic cues --
to the questions of whether and when individuals perceive balanced news as hostile. We found
that ex ante assessments concerning the ideological slant of a media outlet matter a great deal.
Merely by varying the identifying information in a news report from that of an outlet perceived as
liberal (conservative) to one perceived as conservative (liberal), we induced participants to
evaluate the report’s content as significantly more conservative (liberal). The implication is that
the heuristic effect of outlet labels extends beyond conditioning whether or not an individual is
22
prone to accept or reject a given message -- the emphasis of most prior research on biased
information processing (e.g., Zaller 1992, Taber and Lodge 2006; but see Morehouse Mendez
2007) -- to influencing how individuals perceive and respond to the actual content of information.
In other words, not only do citizens disproportionately counter-argue dissonant information while
accepting consonant information, but they sometimes also create consonance and dissonance even
where none actually exists. This suggests that the effects of outlet labels on perceptions of bias,
and any consequent effects on political attitudes, emerge through an interaction between
heuristics and biased information processing.
If perceptions of media bias exist, in whole or in part, in the minds of consumers, this
represents a particularly consequential case of biased information processing (e.g., Lord, et al. 1979;
Redlawsk 2002; Zaller 1992, Morehouse Mendez 2007), with potentially profound implications.
From a strictly economic standpoint, the availability of more news choices is a positive development.
After all, individuals are, to a greater extent than in prior decades, able to consume news products
suited to their specific tastes. Liberals can consume liberal news while conservatives can consume
conservative news, thereby presumably making everyone happier. From the standpoint of democratic
theory, however, this trend may have unfortunate consequences. Most notably, if individuals attend
to news sources that present only one side of a story, their willingness to fashion compromises may
diminish. This, in turn, could render effective democratic governance more difficult.
For much of the past century, the mass media, especially television, have served as an
important common civic space, providing citizens with a shared understanding of their culture, as
well as of the major issues and events of the day. An increasingly ideologically fragmented media
may be gradually eroding the common space, and with it the shared cultural understanding, that
the mass media once provided. If news consumers “tune out” outlets deemed hostile in favor of
23
those they consider friendly, the result may be a less tolerant (Mutz 2002) and more politically
polarized (Huckfeldt, et al. 2004; Sunstein, 2001) society. Our findings also raise a second
potentially troubling possibility, that, depending on the prior beliefs of individual consumers,
even exposure to relatively “balanced” coverage may have a similar polarizing effect.
Our experimental design does not allow us to assess whether or not the media are, in fact,
ideologically biased. However, our results do strongly suggest that perceptions of bias are not a
purely objective response to media content. Rather, our findings lend strong support to the notion
that, to a significant extent, perceived bias in the mass media is indeed in the eye of the beholder.
Several recent studies (Groseclose and Milyo 2005, Gentzkow and Shapiro 2006) have
devised sophisticated indices aimed at accurately gauging the true ideological slant of media
outlets and, by implication, the gap between the actual coverage the media provide and the ideal
of balanced coverage. Their results indicate that mainstream (TV and print) media news outlets
are at most modestly slanted in one or the other direction. For example, Groseclose and Milyo
find that CBS Evening News is about as left-of-center as Sen. Joseph Lieberman (I-Ct), who, in
2006, was forced to run for re-election as an Independent after losing a Democratic primary to a
more left-leaning opponent. They also find that FOX’s "Special Report" is about as right-of-
center as Sen. Susan Collins (R-Me), an avowed centrist.
Our research helps account for why substantial portions of the American public perceive
this apparent relatively small objective gap as a vast chasm. More importantly, it suggests that
the significance of media bias for American politics stems not only from the objective truth of its
presence, absence, or magnitude, but also, to at least some extent, from consumers’ prior beliefs.
After all, as we have shown, consumers tend to find bias when they look for it, whether it exists in
a particular news report or not.
24
APPENDIX 1: OUTLET LOGOS FOR EXPERIMENT
APPENDIX 2: POLITICAL KNOWLEDGE SCALE QUESTIONS
(1) Who has the final responsibility to decide if a law is constitutional or not? (Response options:
President, Congress, Supreme Court, Attorney General, or Don’t Know);
(2) Which political party has the most members in the United States Senate? (Response options:
Republican, Democrat, Evenly split, or Don’t Know);
(3) Whose responsibility is it to nominate judges to the United States Supreme Court? (Response
options: President, Congress, Supreme Court, Attorney General, or Don't Know);
(4) In order for an international treaty to become law in the United States, who, other than the
President, must approve it? (Response options: The Senate, The House of Representatives,
Both Houses of Congress [Senate and House of Representatives], The Supreme Court, Only
the President, or Don't know); and
(5) What job or political office is currently held by Dennis Hastert? (Open ended; correct
response: Speaker of the House of Representatives).
Note: Participants were given one point for each correct response (correct responses shown in
bold).
[Table A-1 here]
25
REFERENCES
Alterman, Eric. 2003. What Liberal Media? The Truth About Bias and the News. New York: Basic
Books.
Baum, Matthew A. 2003. Soft News Goes to War: Public Opinion and American Foreign Policy
in the New Media Age. Princeton: Princeton University Press.
Baum, Matthew A. and Samuel Kernell. 1999. Has Cable Ended the Golden Age of Presidential
Television? American Political Science Review 93 (March): 1-16.
Baum, Matthew A. and Phil Gussin. 2004. In the Eye of the Beholder: An Experimental
Investigation into the Foundations of the Hostile Media Phenomenon. Paper presented at 2004
Meeting of the American Political Science Association, Chicago, IL.
Brosius, Hans-Bernd. 1993. The Effects of Emotional Pictures in Television News. Communication
Research 20:105-124.
Domke, David, Taso Lagos, Mark LaPointe, Melissa Meade, and Mike Xenos. 2000. Elite Messages
and Source Cues: Moving Beyond Partisanship. Political Communication 17: 395-402.
Druckman, James. 2001a. Evaluating Framing Effects. Journal of Economic Psychology 22:91-101.
Druckman, James. 2001b. On the Limits of Framing Effects: Who Can Frame? Journal of Politics
63 (4):1041-1066.
Efron, Edith. 1971. The News Twisters. Los Angeles: Nash Pub.
Eveland, William, Jr., and Dhavan Shah. 2003. The Impact of Individual and Interpersonal
Factors on Perceived News Media Bias. Political Psychology 24 (1):101-117.
Fiske, Susan, and Shelley Taylor. 1984. Social Cognition. Reading, Mass.: Addison Wesley.
Gentzkow, Matthew and Jesse M. Shapiro. 2006. "What Drives Media Slant? Evidence from U.S.
Daily Newspapers." Unpublished manuscript. University of Chicago and NBER.
26
Giner-Sorolla, Roger, and Shelly Chaiken. 1994. The Causes of Hostile Media Judgments.
Journal of Experimental Social Psychology 30:165-180.
Graber, Doris A. 2001. Processing Politics : Learning from Television in the Internet Age, Studies
in Communication, Media, and Public Opinion. Chicago: University of Chicago Press.
Groseclose, Tim and Jeff Milyo. 2005. A Measure of Media Bias. Quarterly Journal of
Economics 120(4): 1191-1237.
Gunther, Albert, and Kathleen Schmitt. 2004. Mapping Boundaries of the Hostile Media Effect.
Journal of Communication 54 (1):55-70.
Gussin, Phil. 2005. Views that Matter: How Visual Representations of Protest in Television News
Shape Political Preferences. Paper read at Pre-APSA Political Communication Conference,
August 31, at Washington D.C
Hamilton, D.L., & Trolier, T.K. (1986). Stereotypes and stereotyping: An overview of the
cognitive approach. In J. F. Dovidio & S. L. Gaertner (Eds.), Prejudice, discrimination, and
racism (pp. 127-163). Orlando, FL: Academic Press.
Hastie, Reid. 1986. A Primer of Information-processing Theory for the Political Scientist. In
Political Cognition, edited by R. R. Lau and D. O. Sears. Hillsdale, NJ: Erlbaum.
Huckfeldt, R. Mendez, J. M., & Osborn, T. 2004. Disagreement, ambivalence, and engagement:
The political consequences of heterogeneous networks. Political Psychology, 25(1). 65-95.
Kahneman, Daniel, Paul Slovic, and Amos Tversky. 1982. Judgment Under Uncertainty:
Heuristics and Biases. Cambridge. New York: Cambridge University Press.
King, Gary., Michael Tomz, and Jason Wittenberg. 2000. Making the Most of Statistical Analyses:
Improving Interpretation and Presentation. American Journal of Political Science 44:341-355.
Lord, C.G., L. Ross, and M.R. Lepper. 1979. Biased Assimilation and Attitude Polarization: The
27
effects of prior theories on subsequently considered evidence. Journal of Personality and
Social Psychology 37:2098-2109.
Lupia, Arthur, and Mathew D. McCubbins. 1998. The democratic dilemma: can citizens learn
what they need to know? Political economy of institutions and decisions. Cambridge, U.K. ;
New York: Cambridge University Press.
Morehouse Mendez, Jeanette, 2007. Message Distortion: The Effects of Media Content, Perceptions of
Media Bias, and Partisan Predispositions on Candidate Evaluations. Journal of Media Psychology
12(Spring):
Morehouse Mendez, Jeanette. 2004. Killing the Messenger: An Experimental Analysis of the Hostile
Media Effect. Presented at Southern Political Science Association, New Orleans, LA.
Mutz, Diana. C. 2002a. Cross-cutting social networks: Testing Democratic Theory in Practice.
American Political Science Review, 96(1). 111-126.
Ottati, V. 1990. Determinants of political judgments: The joint influence of normative and
heuristic rules of inference. Political Behavior 12 (2):159-179.
Patterson, Thomas E. 1993. Out of Order, 2nd ed. New York: Knopf.
Paivio, Allen. 1979. Imagery and verbal processes. Hillsdale, NJ: Erlbaum
Peffley, Mark, Jason Glass, and James Avery. 2001. Public Perceptions of Bias in the News
Media: Taking A Closer Look at the Hostile Media Phenomenon. Paper presented at Annual
Meeting of the Midwest Political Science Association, Chicago, Illinois.
Pew Research Center for the People and the Press. 2004a. News Audiences Increasingly
Politicized: Online News Audience Larger, More Diverse. Washington, DC: Pew.
Pew Research Center for the People and the Press. 2004b. Cable and Internet Loom Large in
Fragmented Political News Universe Perceptions of Partisan Bias Seen as Growing,
28
Especially by Democrats. Washington, DC: Pew.
Popkin, Samuel L. 1994. The Reasoning Voter: Communication and Persuasion in Presidential
Campaigns. Second ed. Chicago: The University of Chicago Press.
Rahn, Wendy M. 1993. The Role of Partisan Stereotypes in Information Processing about
Political Candidates. American Journal of Political Science 37 (May):472-496.
Redlawsk, David P. 2002. Hot Cognition or Cool Consideration? Testing the Effects of Motivated
Reasoning on Political Decision Making. Journal of Politics 64:1021-1044.
Sunstein, C. (2001). Republic.com. Princeton: Princeton University Press.
Sutter, Daniel. 2001. Can the Media be so Liberal? The Economics of Media Bias. Cato Journal
20(3): 431-451.
Taber, Charles S., and Milton Lodge. 2006 (forthcoming). Motivated Skepticism in the
Evaluation of Political Beliefs. American Journal of Political Science 50 (3).
Vallone, R., L. Ross, and M.R. Lepper. 1985. The hostile media phenomenon: Biased perception
and perceptions of media bias in coverage of the "Beirut Massacre." Journal of Personality
and Social Psychology 49:577-585.
Watts, Mark, David Domke, Dhavan Shah, and David Fan. 1999. Elite Cues and Media Bias in
Presidential Campaigns: Explaining Public Perceptions of a Liberal Press. Communication
Research 26 (2): 144-175.
Webster, James G., Patricia F. Phalen, and Lawrence W. Lichty. 2005. Ratings Analysis: The
Theory and Practice of Audience Research. Third Edition. Hillsdale, NJ: Erlbaum.
Zaller, John. 1992. The Nature and Origins of Mass Opinion. Cambridge: Cambridge University
Press.
29
TABLE 1. Summary of Participant Characteristics (unweighted) Total Number of Participants 1014 a % Liberal .14/.25 b % Conservative .21/.33 b % Male .50 % African American .10 % White .73 % Hispanic .10 Mean Age 46 Mean Education Level “some” college Mean Annual Family Income $35-$40,000 Mean % Correct of Five Factual Political Knowledge Questions
.49
aDue to missing data the total N in our statistical analysis varies from 928 to 938. bFirst percentage excludes liberal and conservative leaners; second percentage includes leaners.
30
TABLE 2. Difference of Means Tests of Effects of Varying Outlet Labelsa (+ = pro-Bush) Overall Liberals Conservatives Moderates FOX -0.40
(N=315) -1.64 (N=91)
-1.76 (N=112)
-0.39 (N=129)
CNN -2.15 (N=313)
1.64 (N=74)
-4.01 (N=101)
-0.99 (N=120)
KNWZ -1.57 (N=324)
0.42 (N=79)
-3.44 (N=112)
-1.13 (N=132)
Overall FOX-CNN Diff. 1.75*** 3.27*** 2.25** 0.60 Overall FOX-KNWZ Diff. 1.17** 1.22^ 1.68* 0.74 Overall CNN-KNWZ Diff. -0.57 -2.06** -0.57 0.14 ^p≤.10, *p≤.05, *p≤.01, ***p≤.001 aTests based on unweighted data
31
TABLE 3. Net Coded Favorability of News Transcript (CNN Version), by Segment and Coder Ideology (+ = Pro-Bush; – = Pro-Kerry; 0 = Neutral) Transcript Segments
MediaOutlet
Overall
Liberal
Conservative
Moderate
FOX -0.01 0.47 -0.29 -0.04
CNN -0.33 -0.25 -0.82 0.02
[Segment 1] WOLF BLITZER: Good evening. We start tonight with the presidential candidates on the attack. Today with just 13 days left in the campaign, John Kerry was explaining why he believes the president does not understand the problems of ordinary people while, for President Bush, the message was mostly about why Senator Kerry's plans will leave Americans worse off Diff. 0.32** 0.72*** 0.53** -0.06
FOX 0.00 0.00 -0.01 0.01 CNN 0.01 -0.01 -0.08 0.09
[Segment 2] BLITZER: From now on, this race is going to be fought in ten battleground states, four of them are in the Midwest and today the candidates were following each other's footsteps in Wisconsin. We begin with CNN's Jonathan King, traveling with the Bush campaign in Milwaukee. Carl. Diff. -0.01 0.01 0.07 -0.08
FOX -0.06 0.58 -0.4 -0.13 CNN -0.41 -0.41 -0.65 -0.19
[Segment 3] JONATHAN KING REPORTING: In traditionally Democratic Wisconsin, where over 40,000 jobs have been lost since President Bush took office, the President refocused the debate on what polls demonstrate is his strongest appeal: voters belief that he will wage a more effective war on terror than his Democratic rival. Diff. 0.35** 0.99*** 0.25 0.06
FOX 1.07 1.46 0.97 0.94 CNN 0.96 1.06 1.08 0.76
[Segment 4] PRESIDENT GEORGE W. BUSH: In this time of choosing, I want all Americans to know you can count on me to fight our enemies and defend our freedom.
Diff. 0.11 0.40* -0.11 0.18 FOX -0.18 0.21 -0.52 -0.12 CNN -0.36 -0.06 -0.95 -0.11
[Segment 5] KING: Although the president has lead the race in almost every statewide poll taken since the Republican convention, the Kerry campaign is quick to point out that the Senator has closed the gap to just a few percentage points in the last few days. Jonathan King, CNN, Milwaukee. Wolf. Diff. 0.18^ 0.27 0.43** -0.01
FOX 0.03 0.61 -0.25 -0.06
CNN -1.63 -1.75 -1.93 -1.3
[Segment 6] BLITZER: Thank you, Jonathan. CNN's Candy Crowley is traveling with the Kerry campaign. Candy. What's the latest? CANDY CROWLY REPORTING: Wolf. The Kerry campaign spent the day desperately trying to keep the spotlight on domestic issues, repeatedly citing unconfirmed reports in the St. Louis Dispatch that President Bush is privately telling supporters that, if reelected, he plans to privatize Social Security. Diff. 0.37** 0.76** 0.49* 0.09
FOX -1.33 -1.65 -1.24 -1.22
[Segment 7] SENATOR JOHN KERRY: My fellow Americans, on November 2, Social Security is on the ballot and it's a choice between one candidate who will save Social Security and another who will undermine it.
Diff. 0.30** 0.10 0.69** 0.08
FOX 0.33 0.56 0.12 0.37
CNN 0.26 0.56 -0.10 0.34
[Segment 8] CROWLEY: While Kerry’s new focus on domestic issues appears to have reversed his slide among women voters and he now leads President Bush by as much as a 10 points, the Kerry campaign did express concern about recent polls that indicate that President Bush has gained ground among blacks, with 18 percent of black adults saying they would like to see Mr. Bush win in November. Diff. 0.07 0.00 0.22^ 0.03
32 TABLE 4. OLS Analyses of Correlates of Individual Perceptions of Media Bias Independent Variablesa Model 1 Model 2 Model 3
FOX Identification .641 (.542) -.344 (1.010) 1.118 (.512)* CNN Identification -1.070 (.502)* -.774 (1.079) -.995 (.501)* FOX Ideology Distance -.088 (.189) -.065 (.195) .170 (.157) CNN Ideology Distance .478 (.244)* .528 (.241)* .612 (.222)** Media Ideology Distance -.181 (.261) -.157 (.266) ------- FOX ID x Fox Ideology Distance .731 (.285)** .052 (.418) ------- CNN ID x CNN Ideology Distance .439 (.263)^ -1.209 (.555)* ------- KNWZ ID x Media Ideology Distance .569 (.263)* -1.065 (.526)* .098 (.263) FOX ID x Media Ideology Distance ------- ------- -.023 (.273) CNN ID x Media Ideology Distance ------- ------- -.055(.296) FOX ID x Political Knowledge ------- 3.095 (1.172)** ------- CNN ID x Political Knowledge ------- .482 (1.466) ------- KNWZ ID x Political Knowledge ------- .919 (1.465) ------- CNN x Ideology Distance x Knowledge ------- 2.534 (.857)** ------- KNWZ x Ideology Distance x Knowledge ------- 2.453 (.687)*** ------- FOX x Ideology Distance x Knowledge ------- .897 (.497)^ ------- Black -.239 (1.215) -.268 (1.207) -.054 (1.207) White -.147 (.912) -.198 (.912) -.120 (.902) Hispanic .377 (1.128) .415 (1.128) .408 (1.123) Political Knowledge 1.145 (.843) ------- 1.125 (.845) Family Income .032 (.055) .033 (.054) .036 (.056) Age .015 (.014) .015 (.014) .015 (.014) Male -.160 (.408) .007 (.406) -.099 (.406) Southern Resident .063 (.432) .144 (.426) .068 (.434) Internet Access -.818 (.424)* -.831 (.424)* -.789 (.426)^ Education .161 (.135) .143 (.134) .147 (.134) Constant -2.680 (1.377)* -2.632 (1.534)^ 2.918 (1.377)* R2
.10 N=928
.13 N=928
.09 N=928
^p≤.10, *p≤.05, **p≤.01, ***p≤.001
33
TABLE 5. Difference in Perceived Favorability, as Participant Ideology and Global Media Assessment or Individual Outlet Assessments Vary
Individual Outlet Assessments
Treatment Participant
Ideology Outlet Extremely
Conservative Outlet Extremely
Liberal Difference FOX Liberal 3.383 -0.527 -3.910** FOX Conservative -0.527 -4.436 -3.910** CNN Liberal 3.476 -2.057 -5.533** CNN Conservative -2.057 -7.590 -5.533**
Global Media Assessment
Media Extremely
Conservative Media Extremely
Liberal Difference FOX Liberal -0.499 -0.315 +0.184 FOX Conservative -0.315 -0.131 +0.184 CNN Liberal -2.760 -2.443 +0.317 CNN Conservative -2.443 -2.126 +0.317 KNWZ Liberal -.987 -1.333 -0.346 KNWZ Conservative -1.505 -1.851 -0.346
**p≤.01
34
TABLE 6. Summary of Hypotheses, Test Results, and Locations
Hypothesis
Where Tested?
Proportion (%) of Tests Supporting
Hypothesis
H1 (Heuristic Hypothesis) Table 2
Table 3
Figure 2 (Table A-1)
3/3 (100%)
9/14 (64%)a
12/12 (100%) H2 (Heuristic Intensity Hypothesis) Figure 2 (Table A-1) 11/12 (92%)b
H3 (Political Awareness Heuristic Hypothesis) Figures 3 & 4 (Table A-1) 10/12 (83%)c
H4 (Familiarity Hypothesis) Table 5 4/4 (100%)
OVERALL TOTALS 49/57 (86%) a Based on correctly signed and statistically significant differences. If all correctly signed differences
are counted as supportive, the figure changes to 12/14 (86%). b The three tests not supporting hypothesis derive from relatively uncommon scenario (where
participants perceive FOX as liberal and CNN as conservative), resulting in an extremely small number of observations.
c The two tests not supporting hypothesis derive from the scenario (where participants perceive both FOX and CNN as conservative), for which the smaller coefficients for political sophisticates are arguably consistent with our theory.
35
TABLE A-1. Net perceived pro-Bush Content in News Transcript, as outlet label and perception of outlet ideological distance changes (D.V.= Bush average minus Kerry average)
A. Overall B. High Political Awareness C. Low Political Awareness Treatment
(Outlet Ideology)
Diff. Treatment
(Outlet Ideology)
Diff. Treatment
(Outlet Ideology)
Diff. Participant Ideology FOX CNN FOX CNN FOX CNN
Con‡ Lib‡ Con‡ Lib‡ Con‡ Lib‡ Liberal 3.383 -2.057 -5.440** 4.438 -1.906 -6.344** -0.315 -2.184 -1.869 Moderate 1.428 -4.824 -6.252** 2.404 -5.677 -8.082** -0.862 -1.604 -0.742 Conservative -0.527 -7.590 -7.063** 0.371 -9.448 -9.819** -1.410 -1.024 +0.386
Lib‡ Con‡ Lib‡ Con‡ Lib‡ Con‡ Liberal -0.527 3.476 +4.002* 0.371 5.636 +5.265* -1.410 -3.345 -1.935 Moderate -2.482 0.709 +3.191* -1.662 1.865 +3.527^ -1.957 -2.764 -0.807 Conservative -4.436 -2.057 +2.379 -3.695 -1.906 +1.789 -2.505 -2.184 +0.321
Con‡ Con‡ Con‡ Con‡ Con‡ Con‡ Liberal 3.383 3.476 +0.093 4.438 5.636 +1.199 -0.315 -3.345 -3.030 Moderate 1.428 0.709 -0.719 2.404 1.865 -0.539 -0.862 -2.764 -1.902 Conservative -0.527 -2.057 -1.531** 0.371 -1.906 -2.277** -1.410 -2.184 -0.774
Lib‡ Lib‡ Lib‡ Lib‡ Lib‡ Lib‡ Liberal -0.527 -2.057 -1.531** 0.371 -1.906 -2.277** -1.410 -2.184 -0.774 Moderate -2.482 -4.824 -2.342* -1.662 -5.677 -4.015** -1.957 -1.604 +0.354 Conservative -4.436 -7.590 -3.154^^ -3.695 -9.448 -5.753* -2.505 -1.024 +1.481 ^^ p≤.15, ^p≤.10, *p≤.05, **p≤.01, ***p≤.001 ‡ Note: “Con” denotes outlet perceived as “extremely conservative” and “Lib” denotes outlet perceived as “extremely liberal”.
36
NOTES 1 This experiment was funded by TESS (Time-Sharing Experiments for Social Sciences) and
conducted by Knowledge Networks (www.knowledgenetworks.com).
2 This survey is available at: http://www.harrisinteractive.com/harris_poll/index.asp?PID=679
3 Individuals also differ systematically in their assessments of the ideological orientation of the
mass media as a whole. For example, in a 1998 Gallup Poll, Republicans were 34 percentage
points more likely than Democrats to perceive a liberal bias (52 vs. 18%), while Democrats were
12 percentage points more likely to perceive a conservative bias (25 vs. 13%).
4 Vallone et al. (1985) compared the responses of individuals with strong policy preferences
relative to the Israeli-Palestinian conflict. In this study, we compare the responses of individuals
with strong views concerning the ideological orientation of FOX, CNN, and the mass media.
5 Hamilton and Trolier (1986, 133) define social stereotypes as “cognitive structures that contain
the perceiver’s knowledge, beliefs, and expectations about human groups.” We extend this
concept to include media outlets.
6 We do not, in this study, investigate the effects of variations in assessments of
ideological bias in media outlets on individual attitudes. However, if one accepts the fairly
modest assumption that assessments of bias are significantly related to credibility
assessments, then it follows fairly clearly from prior research on the importance of source
credibility for persuasion that assessments of bias ought to affect the extent to which
media content influences attitudes. While we did not ask subjects to rate the “credibility”
of specific media outlets, and therefore cannot directly test that assumption, the survey
data presented at the outset of this paper clearly suggests such a relationship.
37
7 Reliance on outlet labels as an information shortcut -- based either on repeated personal
exposure to a given outlet, or on repeated exposure to trusted elites characterizing the ideological
orientation of a the outlet in a certain manner -- may allow typical individuals to fairly accurately
estimate an outlet’s overall average ideological slant. After all, if reliance on outlet labels led to
excessively inaccurate results – based on a each individual’s personal definition of “excessively
inaccurate” -- it seems unlikely that individuals would continue to rely in this heuristic. However,
individuals may still misjudge any particular segment of news content, which might not perfectly
match the outlet’s overall average ideological slant. Hence, consumers’ prior assumptions may be
relatively accurate on average – or at least accurate enough given whatever personal importance
they place on correctly judging the ideological slant of news content -- yet simultaneously
incorrect in any given specific instance.
8 We replicated the study with a second subject pool consisting of 932 college students, including
UCLA undergraduates and students from Los Angeles area community colleges. The results closely
mirror those from the national sample. Hence, we report only the latter results herein. The student
replication results are available upon request.
9 Since television is primarily a visual medium, the decision to use transcripts could raise
concerns about the external validity of the experiment. It is important to note, however, that our
theoretical argument hinges on subjects’ ex ante assessments of the ideological orientation of
media outlets. As such, using transcripts that only include one visual cue (i.e., the outlet logo)
should reduce, rather than increase, the likelihood of triggering prior assessments (Brosius 1993;
Graber 2001; Gussin 2005; Paivio 1979), thereby creating a more difficult test of our theory.
10 To ensure consistency across treatment conditions, we created a logo for the unfamiliar outlet.
38
11 Out-of-sample pre-testing indicates that ideological moderates perceive the “valence” of the
fictionally labeled transcript as statistically indistinguishable from perfect ideological balance.
12 The candidate quotes are actual quotes from President Bush (ABC World News Tonight,
10/18/04) and John Kerry (Anderson Cooper 360 Degrees, 10/20/04).
13 “No coverage” responses were set to the mean.
14 For example, if a subject rated all eight segments as “Very unfavorable” to Bush (1 x 8=8) and
“Very favorable” to Kerry (5 x 8=40), the formula returns a value of –32.
15 Random assignment ought, theoretically, to account for many potentially confounding causal
factors. However, when we apply the population sample weight provided by Knowledge
Networks – but not when we exclude the weight -- the sample appears slightly less likely to be
ideologically liberal or conservative, and more likely to be moderate, than the national population.
(Moreover, it is always possible to draw a systematically biased sample even when drawing at
random, especially given relatively small treatment subgroups.) Given the centrality of ideology
to our study, we elected to employ the unweighted data. Consequently, to guard against any
demographic biases this might introduce along other dimensions, we include several demographic
controls. Nonetheless, our results are substantively comparable with or without weighting, and
with or without demographic controls. Since, ceteris paribus, it is more difficult to find significant
results with fewer degrees of freedom, while superfluous controls are unlikely to improve our
results, we report what we consider the most appropriate, and in some respects most difficult, test
of our predictions.
16 As one might anticipate, moderates perceived less bias than their more ideologically extreme
counterparts. Moderates were evenly divided with respect to CNN, with 29 (6.9%) rating the
outlet as liberal or extremely liberal and 31 (7.5%) rating it as conservative or extremely
39
conservative. However, they were more than twice as likely to rate FOX as conservative. Overall,
only 22 moderates (5.2%) rated FOX as liberal or extremely liberal, compared to 58 (13.9%) who
rated it as conservative or extremely conservative. Still, large majorities of moderates – 81% for
FOX and 86% for CNN -- rated both outlets as “centrist” or centrist but leaning in one or the
other ideological direction.
17 This scenario also routinely emerges in public opinion surveys (e.g., Pew Center 2004b).
18 These results suggest that we succeeded in creating a balanced transcript, overall and for the
individual segments (excluding candidate statements).
19 If we exclude the four most extreme cases – two at +28 and two at -28 -- the standard deviation
drops to 6.14 and the observed range becomes -21 to +24. Hence, the numerical magnitude of a
one standard deviation change increases to about a 14% shift, or 1.3 points on the individual
segment (-4 to +4) favorability scales. However, in order to avoid any appearance of “stacking the
deck” in favor of our predictions, in the discussion that follows we employ the more conservative
standard deviation estimate, based on the full sample.
20 Because OLS is a linear estimator, the predicted effects if subjects perceive one or both outlets
as moderate necessarily fall in between those where both outlets are perceived as opposing in
ideology. Hence, we omit these results, which are conceptually less interesting.
21 To address potential concerns regarding external validity in our study, we constructed a second
experiment, where we recruited 110 undergraduates to code nearly 400 randomly selected and
unedited news reports from the fall 2000 presidential campaign, from ABC, NBC, CBS, CNN,
FOX, The New York Times, The Wall Street Journal, and USA Today. The results, reported
elsewhere (Baum and Gussin 2004) offer further support for our key hypotheses.
FIGURE 1. Distribution of Participants’ Ratingsof CNN & FOX Ideological Orientations
FIGURE 2. Net Perceived pro-Bush Slant in News Transcripts, asPresumed Outlet Ideology & Participant Ideology Vary (All Participants)
Liberals Moderates Conservatives
*p<.05, **p<.01
Liberals Moderates Conservatives
FIGURE 3. Net Perceived pro-Bush Content in News Transcripts, as PresumedOutlet Ideology & Participant Ideology Vary (High Awareness Participants)
^p<.10, *p<.05, **p<.01
Liberals Moderates Conservatives
FIGURE 4. Net Perceived pro-Bush Content in News Transcripts, as PresumedOutlet Ideology & Participant Ideology Vary (Low Awareness Participants)