Dynamic Estimation of Media Slant∗

Jong Hee ParkDepartment of Political Science and International Relations

Seoul National Universityhttp://jhp.snu.ac.kr

jongheepark@snu.ac.kr

January 4, 2016

Abstract

Existing methods for media slant estimation focus on how to map observedtext data with a low dimensional vector of fixed quantities. In doing so, thesemethods ignore the sequence of news and fail to consider the possibility of changesin media slant. In this paper, we highlight the chronological characteristic of newsreports and develop Bayesian statistical methods that allow a joint estimation oftime-varying public opinion trends and media slant subject to multiple discretechanges. Automated text analysis methods are used to extract relative frequencyof partisan words from news reports. We apply this method to news reports cov-ering the Sewol ferry disaster by nineteen daily newspapers of South Korea.

Keyword : media slant, dynamic linear model, hidden Markov model, automatedtext analysis, South Korea

∗Prepared for the Asian Political Methodology Meeting at Tsinghua University in January 2016.

1

1 Introduction

In a modern democratic society, mass media reflect and shape public opinion. As strategicactors and professionals pursuing independence, journalists and editors wish to be more than“a conveyor belt” for what has happened and what politicians have said (Zaller, 1999, 1-2).Mass media play a critical role in framing, disseminating, and constructing public issues. Onthe other hand, mass media cannot report an “unrepresentative sample” of public opinionconsistently because of market pressure. Irresponsive media, if disfavored by consumers, willquickly lose market shares. As the number and types of media outlets increase, behaviors ofmass media have become increasingly constrained by the market competition. Consideringthe constant and complex interactions between mass media and public opinion, it is essentialto explicate the public opinion-media nexus to properly understand the role of mass mediain a democracy.

Among many important roles of mass media in a democracy, theoretical and empiricalinvestigations of “media slant”1 –“media bias” – have recently received increasing attentionfrom social scientists (Groseclose and Milyo, 2005; Baron, 2006; Matthew Gentzkow, 2006;Bernhardt, Krasa and Polborn, 2008; Gerber and Bergan, 2009; Gentzkow and Shapiro,2010; Larcinese, Puglisi and James M. Snyder, 2007; Stone, 2011; Gans and Leigh, 2012;Stromberg, 2015; Agirdas, 2015). This paper contributes to the literature by developing anew method for dynamic estimation of media slant.

In our dynamic modeling framework, the news content generating process is assumed tobe a stochastic process where public opinion and mass media constantly interact with eachother. We consider media slant – or any quantity we derive from time-ordered news contents– as time-indexed and subject to temporal changes. Such characterization of the newsgenerating process as stochastic marks an important departure from the existing literature.Most existing methods for the media slant estimation focus on how to map observed text datawith a low dimensional vector of fixed quantities, such as sentiments, slants, or ideologies(Groseclose and Milyo, 2005; Gentzkow and Shapiro, 2010; Gans and Leigh, 2012; Taddy,2013). In doing so, these methods treat each document as “an exchangeable collectionof phrase tokens” (Taddy, 2013, 755) and fail to consider the temporal order in the newscontent generating process. For example, Gentzkow and Shapiro (2010) treat their key inputmeasures, phrase frequencies in newspapers and phrase frequencies in the 2005 CongressionalRecord, as samples randomly drawn from a population of all phrases.

There are other reasons to consider the possibility of changes in media slant rather thanthe mere fact that news contents are time-ordered. Epochal changes in a society funda-mentally reshape both positions of mass media and directions of public opinion. If we failto account for those massive and long-lasting changes in the society, our measurement ofmedia slant will be biased in an unknown direction and inconsistent in a statistical sense.Moreover, even though politicians’ use of partisan phrases shows a consistent pattern, move-ments of public opinion and media reports as reflections of public opinion at each time tendto show nonstationary patterns even for a short period of time. Public opinion movementshave periods of a persistent trend and periods of irregular changes. For example, Gentzkowand Shapiro (2010) assume that each media outlet chooses “estate tax” over “death tax,”

1We define media slant as relative ideological leanings of media outlets in news coverage.

2

or vice versa, in a consistent manner throughout the entire year of 2005. The assumptionof time homogeneity needs to be tested because observed phrase frequencies are sensitive topublic opinion changes or effects of events such as the enactment of the Death Tax RepealPermanency Act of 2005.2

In this paper, we propose a new method to jointly estimate public opinion trends andmedia slant that aims to properly account for the time-ordered nature of the news contentgenerating process and public opinion movements. The first step is to extract a relativefrequency of partisan phrases from media reports using automated text analysis methods.This can be done in various ways depending on the availability of other information thathelps us identify ideological dimensions of news content data. Then, we decompose thetime series cross-sectional data of the relative partisan word frequency into time-varyingmovements of public opinion and hidden Markov transitions of media slant using dynamiclinear multilevel model with parametric breaks in the second level parameter. Instead ofimposing dynamics and Markov transitions on the data, we test the modeling assumptionof time-varying movements of public opinion and hidden Markov transitions of media slantusing two simpler models as benchmarks: the multilevel model of media slant and dynamiclinear multilevel model. For model diagnostics and changepoint detection, we utilize theWidely Applicable Information Criterion (WAIC), an intuitive and effective Bayesian modeldiagnostic tool (Watanabe, 2010).

2 The Proposed Method

Our method for a dynamic estimation of media slant consists of three steps. First, we identifypartisan phrases and quantify their relative frequency in media reports. Second, the wordselection on an issue by media is modeled as a function of public opinion toward the issueat each time and a political ideology of the media. We discuss different types of dynamicmodels for the slant estimation. Model diagnostics of competing dynamic models in orderto avoid overfitting is the last step of the proposed method.

2.1 Relative Frequency of Partisan Phrases

Many different methods have been proposed to identify partisan phrases from news contents,mostly in the U.S. context. For example, Groseclose and Milyo (2005) consider the frequencyof citations to think-tank materials by legislators and journalists to indicate a partisanleaning of the medium. Gentzkow and Shapiro (2010) use a hand-coded list of bigram ortrigram phrases that are more frequently used by one party members than the other; therelative frequency of partisan phrases is then regressed on a congressperson’s ideology toobtain a partisan score for the newspaper.

Generally speaking, however, the definition of partisan phrases varies by time and space.It is hard to generalize the identification method of partisan phrases in the literature tonon-US cases because congressional voting and speech data are not always readily available.Also, in some countries, like South Korea and the United Kingdom, party-line voting is

2The bill was introduced by Rep. Hulshof, Kenny C. (R-MO) on Feb. 17, 2005 (https://www.congress.gov/bill/109th-congress/house-bill/8).

3

quite dominant. Therefore, there is a need to develop a creative way to identify ideologicaldimensions of news content data using other types of information.3 In the case of SouthKorea–discussed in Section 3–political parties release press releases almost every day. Suchpress releases can be used to uncover ideological dimensions of the news content data.

2.2 Models

Once we extract the relative frequency of partisan phrases, the next step is to develop anempirical model of the news content generating process in which media slant is one of thekey random variables. In this section, we discuss three modeling options in the order ofmodel complexity: (1) the multilevel model (MLM), (2) the linear dynamic multilevel model(DMLM), and (3) the hidden Markov linear dynamic multilevel model (HMDMLM). Thediscussion of how to choose the “right” model using a Bayesian model diagnostic tool follows.

2.2.1 Model 1: Multilevel Model (MLM) of Media Slant

Using the relative frequency of partisan phrases for newspaper i at t, we model partisanbiases of each newspaper as random samples from an unknown normal distribution:

yit = αi + β + σyεit, εit ∼ N (0, 1) (1)

αi ∼ N (µα, σ2α)

β ∼ N (b0, B0)

σy ∼ IG(c0, d0)

σα ∼ IG(e0, f0)

This is a baseline multilevel model upon which we will build additional complexity. Al-though this model is naive in that it ignores the time-varying nature of the news contentgeneration, there are three major advantages of modeling media slant within a multilevelmodel.

First, the multilevel setup provides consistent estimates of group-level parameters fromtime series cross-sectional data. Media slant is information at a group level that needs tobe gleaned across individual observations. When a group-level parameter is a quantity ofour substantive interest, we cannot use the fixed-effects method that removes group-levelconstants for the consistent estimation of individual level slopes.

Second, the hierarchical structure of the multilevel model allows us to combine complexdynamic models in the context of the time series cross-section data analysis. As it will bediscussed shortly, we can add various time series models, such as an autoregressive model, alinear dynamic model, and a hidden Markov model, to the multilevel model.

Last, and most importantly, the multilevel model summarizes variations across hetero-geneous groups in a way that balances within-group variations with inter-group differences.

3Alternatively, one can develop a sophisticated theoretical model that explains the news content gen-erating process and can estimate theoretical parameters from the news content data. Recently, a groupof scholars are working on deriving ideological measures from a theoretical model of (slanted) word choicein text document (Baron, 2006; Bernhardt, Krasa and Polborn, 2008; Stone, 2011; Kim, Londregan andRatkovic, 2015).

4

This characteristic, known as “partial pooling” in the multilevel literature, ensures us toavoid overfitting the data. Overfitting usually happens in the estimation of media slantwhen we estimate media slant of newspaper i only using data observed from i. In that case,i’s slant estimate captures only the realized i’s information without learning from other sim-ilarly generated data; thus, the resulting estimates tend to be overly confident about theout-of-sample predictive accuracy of i’s slant estimate.

The posterior samples of slant estimates (αi) can be decomposed into the sum of twoparts:

αi =

Tiσ2y

Tiσ2y

+ 1σ2α

No pooling︷ ︸︸ ︷(yi − β) +

1σ2α

Tiσ2y

+ 1σ2α

Complete pooling︷︸︸︷µα︸ ︷︷ ︸

Partial pooling

(2)

where Ti is the number of newspaper i’s observation and yi is a group mean of newspaper i’srelative partisan phrase frequency. Here, i’s slant is a weighted sum of i’s average distancefrom the global mean (yi − β)) and the mean of all media slants (µα).

The Markov chain Monte Carlo (MCMC) sampling algorithms of the MLM are wellknown. Several Bayesian statistical packages in R provide functions to fit the MLM andother Bayesian software like the BUGS (Bayesian inference Using Gibbs Sampling) project,JAGS (Just Another Gibbs Sampler), and Stan (http://mc-stan.org) are also available.However, it is not straightforward to add dynamic components to these packages. Thus, weproceed to discuss the MCMC algorithms.

For efficient sampling of group level parameters, we decompose the posterior density intothree blocks as suggested by Algorithm 2 of Chib and Carlin (1999):

p(β, σ2y , σ

2α|y) =

∫p(β, αi|y)p(σ2

α|y, β, αi)p(σ2y|y, β, αi, σ2

α, αi)dαi.

The full conditional distribution of p(β, αi|y) is decomposed into p(β|y)p(αi|y, β) so that thesampling of β is not directly dependent upon αi. Using conjugate priors, all the samplingsteps can be done by the Gibbs sampler.

2.2.2 Model 2: Linear Dynamic Multilevel Model (DMLM) of Media Slant

Now, we add one layer of dynamics onto the MLM. The global mean of the relative partisanphrase frequency follows a random walk where the current global mean is determined by theprevious global mean and a random shock. The size of a random shock (σβ) is estimated by

5

data. The resulting model takes a following form:

yit = αi + βt + σyεit, εit ∼ N (0, 1) (3)

αi ∼ N (µα, σ2α)

βt = βt−1 + σβεt for t > 1

β1 ∼ N (b0, B0)

σy ∼ IG(c0, d0)

σα ∼ IG(e0, f0)

σβ ∼ IG(g0, h0).

The above model was first introduced by Jackman (2005) who used it to estimate houseeffects in time series cross-sectional polling data.

A natural question arising from this model setup is what the time-varying global mean ofthe relative partisan phrase frequency (βt) substantively means. Technically speaking, βt isthe average level of partisan slant reflected in news reports measured at t. βt follows a randomwalk. Thus, the current βt is highly predictable from βt−1, but not from βt−2 ≡ (βt−2 . . . β1).

We interpret β ≡ (β1 . . . βT ) as movements of public opinion showing the average partisanslant among the public observed by the mass media in the sample. Public opinion changessmoothly over time, mostly reflecting what people thought right before. Yet, it sometimeschanges dramatically in one way or the other in response to external shocks. Such irregularmovements are modeled as a random walk process in our model, and the size of irregularityis estimated by the transition variance (σ2

β).Interpreting β as public opinion movements makes it easy to substantively understand

our estimates of media slant. The posterior samples of slant estimates (αi) from the DMLMconsist of the sum of two parts:

αi =

Tiσ2y+σ2

β

Tiσ2y+σ2

β+ 1

σ2α

i’s average distance from public opinion︷ ︸︸ ︷∑t∈Ti

(yi,t − zitβ) +

1σ2α

Tiσ2y+σ2

β+ 1

σ2α

µα (4)

where Ti is a collection of time indices realized in i’s data and zit is tth row of zi which is aTi × T matrix that identifies specific time indices realized in i’s data.4

As noted by the brace in Equation (4), the DLML constructs newspaper i’s slant as aweighted sum of i’s average distance from public opinion (

∑t∈Ti(yi,t−zitβ)) and the mean of

all media slants (µα). This formula allows us to consider the time-varying nature of publicopinion and anchor our slant measure, accordingly, which makes more sense than anchoring

4For example, if newspaper i has three observations at t = 1, 3, 5 and the maximum range of time seriesin the data set is t = (1, 2, 3, 4, 5), then

zi =

1 0 0 0 00 0 1 0 00 0 0 0 1

, β =

β1β2β3β4β5

.

6

media slant to the constant global mean (β) in Equation (1).

Measuring Slant from the Public Opinion

Rela

tive

Freq

uenc

y

2014−0

6−05

2014−0

6−06

2014−0

6−07

2014−0

6−08

2014−0

6−09

2014−0

6−10

−0.4

−0.2

0.0

0.2

0.4

Distance from the public opinion

Distance from the public opinion

Distance from the public opinion

Distance from the public opinion

Distance from the public opinion

Distance from the public opinion

Figure 1: Media Slant as Average Distance from Public Opinion: Data from 19 South Korean Newspapers,April 2014 - April 2015. Red dots are relative partisan phrase frequencies of Munhwa Ilbo and the blue lineis estimated public opinion from 19 newspapers.

Figure 1 visualizes the idea of estimating media slant as an average distance from publicopinion. The data set used here is the relative partisan phrase frequency data from nineteenSouth Korean newspapers over the Sewol ferry disaster. The blue line is estimated publicopinion (βt) and red dots are relative partisan phrase frequencies of a newspaper calledMunhwa Ilbo. The braces between the red dots and the blue line indicate partisan slants ofMunhwa Ilbo at each time point. It is clear that Munhwa Ilbo’s partisan slants are locatedabove the public opinion except on June 9, 2014, indicating the possibility of Munhwa Ilbo’sconservative bias.

The MCMC sampling algorithm of the DMLM involves two additional steps for β and σ2β.

As the movement of β is modeled as a linear dynamic model, all the MCMC sampling stepscan be done by the Gibbs sampler. The posterior density can be decomposed as follows:

p(β, σ2β, σ

2y, σ

2α|y) =

∫ forward filtering backward sampling︷ ︸︸ ︷p(β|y) p(σ2

β|y,β)p(αi|y,β, σ2β)

p(σ2α|y,β, σ2

β, αi)p(σ2y|y,β, σ2

β, σ2α, αi)dαi.

The sampling of β is done by the forward filtering backward sampling (FFBS) algorithmdeveloped by Carter and Kohn (1994) and Fruhwirth-Schnatter (1994). The sampling of σ2

β

7

is from the following inverse gamma distribution:

σ2β|y,β, σ2

α, αi ∼ IG

(g0 + T

2,h0 +

∑t=1T (βt − βt−1)2

2

).

2.2.3 Model 3: Hidden Markov Linear Dynamic Multilevel Model (HMDMLM)of Media Slant

The DMLM makes an important contribution to the literature by allowing the joint estima-tion of public opinion and constant parameters at the group level. However, it is logicallyincoherent and theoretically unsatisfying to assume that only public opinion changes overtime while media slant remains constant. However strong partisan predispositions of massmedia might be, mass media cannot ignore massive and dramatic changes in public opinion.Important social events – such as disasters, international disputes, economic crisis, politicalscandals, critical elections, and social movements – affect how journalists think and writeabout their society. Moreover, tides of public opinion reshuffle the popularity of mass mediain terms of their responsiveness. As a result, the assumption of constant media slant, how-ever convenient it would be in terms of model estimation, must be tested with data ratherthan to be taken for granted.

In order to let media slant change during the sample period, two assumptions are made.First, changes of media slant are discrete rather than continuous. That is, media slantchanges take place not every time, but once in a while. This is a reasonable assumptionthat reduces the complexity of model and the computational cost. Ideological positions ofmass media are usually consolidated over a long period of time by ownership, governmentregulations, media market trends, recruitment, and promotion. They may change in responseto major changes in public opinion, but not always. Second, changes in media slant arecaused mainly by shifts in public opinion or positions of other media outlets. Changes inmedia slant can be simply passive reflections of changes in public opinion. Or, they can bea result of strategic repositioning by mass media in response to changes in positions of othermedia outlets.

To highlight the difference between the DMLM and the HMDMLM, Figure 2 and Figure 3compares the modeling structure of the two dynamic models. The arrows show the directionof statistical dependence in each model. Observed partisan frequencies of each media (yit)are generated by two stochastic variables: βt and αi,st . Values of αi,st are further dependentupon latent states (st), the transition of which is governed by βt. In plain words, publicopinion (βt) is the main driving force of the news content generation process and changes inideological positions of mass media. We extract information about the transition of hiddenstates by assuming that βt follows a local linear trend until unknown break points. Thatis, the sampling of latent states is not dependent upon the response data in our model:p(s|y,β, σ2

β, σ2α, αi, σ

2y) = p(s|β, σ2

β, σ2α, αi, σ

2y).

8

βt−1 βt βt+1

yi,t−1 yi,t yi,t+1

αi αi αi

βt−1 βt βt+1

yi,t−1 yi,t yi,t+1

αi,(st−1) αi,(st) αi,(st+1)

st−1 st st+1

Figure 2: DMLM Figure 3: HMDMLM

Based on these two assumptions, we introduce the HMDMLM to account for dramaticchanges in media slant and time-varying changes in public opinion. Compared with theDMLM, two new parameters are added to the HMDMLM. The first one is latent state vari-ables (s ≡ (s1, . . . , sT )), and the second one is a transition matrix (P) which summarizes themovement of latent state variables. We adopt Chib (1998)’s non-ergodic (i.e. non-switchingand forward moving Markov chain) design that efficiently identifies multiple changepointsacross various types of response data.

The resulting model can be written as follows:

yit = αi,st + βt + σyεit, εit ∼ N (0, 1)

βt = βt−1 + σβεt for t > 1

st|st−1 ∼ Markov(P, π0)

αi,st ∼

N (µα,1, σ

2α,1) if τ0 < t 6 τ1

...

N (µα,M+1, σ2α,M+1) if τM < t 6 τM+1

β1 ∼ N (b0, B0)

σy ∼ IG(c0, d0)

σα ∼ IG(e0, f0)

σβ ∼ IG(g0, h0)

pii ∼ Beta(a, b) for i = (1, . . . ,M)

where τ0 = 0 and τM+1 = T . Note that the cutpoints (τ ≡ (τ0, . . . , τM+1)) are identifiedby sampled latent state variables (s ≡ (s1, . . . , sT )) at each MCMC simulation step. pii is aprobability of staying in the ith state. Although each row of the transition matrix follows aDirichlet distribution, there are only two non-zero elements in each row except the last row.The sum of these two non-zero elements is 1 by definition. Hence we model only diagonalelements (pii) of the transition matrix as a Beta distribution.

Slant estimates (αi) from the dynamic linear multilevel model with hidden Markov transi-

9

tions are state dependent. Suppose that the hidden state at t be m andMi be the collectionof time indices pertaining to state m for media i. We interpret state dependent slant esti-mates (αi,m) as media i’s average distance from public opinion during state m.

αi,st=m =

miσ2y+σ2

β

miσ2y+σ2

β+ 1

σ2α,m

i’s average distance from public opinion during state m︷ ︸︸ ︷∑t∈Mi

(yi,t − zitβ) +

1σ2α,m

miσ2y+σ2

β+ 1

σ2α,m

µα,m (5)

where mi is the number of media i’s observations during state m.The MCMC sampling algorithm of the dynamic linear multilevel model with hidden

Markov transitions at the group-level parameters involves two additional steps for s and Pcompared with that of the MLM-DLM. The posterior density can be decomposed as follows:

p(β, σ2β, σ

2y, σ

2α,P|y) =

∫ ∫p(β|y)p(σ2

β|y,β)p(αi|y,β, σ2β)p(σ2

α|y,β, σ2β, αi)

p(σ2y|y,β, σ2

β, σ2α, αi)

multi-move sampling︷ ︸︸ ︷p(s|y,β, σ2

β, σ2α, αi, σ

2y)

row-wise sampling from a Beta distribution︷ ︸︸ ︷p(P|y,β, σ2

β, σ2α, αi, σ

2y, s) dαids.

The sampling of pii can be easily done once we have samples of hidden states. Using theBeta prior distribution,

π(pii|s) ∝ f(s|pii)Beta(a, b)

∝ pniiii (1− pii)nijpa−1ii (1− pii)b−1

pii|s ∼ Beta(a+ nii, b+ nij)

where nii be the number of one-step transitions from state i to i and nij be the number ofone-step transitions from state i to j.

The sampling of hidden state variables is done by the multi-move sampling algorithmproposed by Chib (1996) and Chib (1998), and explained in detail by Fruhwirth-Schnatter(2006, 342-346). Note that we do not sample hidden states from response data (y) but fromβ because, as discussed above, we identify the source of discrete changes in media slant fromdramatic changes in public opinion or position changes of other media outlets.

2.3 Model Validity Check

Like so, the paper presents statistical methods to jointly estimate dynamics of public opinionand discrete changes in media slant. However, one important question we need to addresswhen developing complex models is: “is it really necessary?” That is, the models discussedabove will return estimates of dynamic public opinion and/or discrete changes in mediaslant simply due to the structure of the model. Then, how do we know that the estimateddynamics and changes are real? Cross-validation tests and Monte Carlo studies can be usefuland suggestive but they do not provide a definitive answer to the question of model validity.

10

In this Section, we discuss how to check the validity of the above models against data –model diagnostics – and other alternative models – model comparison.

Discussions of the Bayesian model diagnostics and model comparison have been exten-sively covered in the statistics literature (Gelfand and Smith, 1990; Raftery, 1995; Kass andRaftery, 1995; Green, 1995; Chib, 1995; Gelman et al., 2004; Vehtari and Lampinen, 2002;Vehtari and Ojanen, 2012). The goal of the Bayesian model diagnostics and model compari-son is to assess the posterior probability of the model. Assuming a uniform prior probabilityfor each model, the posterior probability of model i is:

p(Mi|y) =p(y|Mi)∑Jj=1 p(y|Mj)

.

An important quantity that needs to be computed from each model to find the posteriormodel probability is the marginal density of data given each model:

p(y|Mi) =

∫p(y|Θi,Mi)p(Θk|Mi)dΘi

where Θi denotes the parameter vector for model i.However, it is very difficult to get a stable estimate of the marginal density of data except

in the case of simple models. Our dynamic linear multilevel models have many parameters,most of which are latent. Thus, even if we manage to compute the marginal density of datausing an approximate method, the likely accuracy of the obtained values can be problematic.

An alternative way to check the model validity without computing the marginal densityof data is to use Watanabe-Akaike information criterion or widely applicable informationcriterion (WAIC) proposed by Watanabe (2010). Gelman, Hwang and Vehtari (2014) providean introduction to the WAIC. The WAIC has several advantages in the Bayesian modeldiagnostics. First, the WAIC approximates leave-one-out cross validation (LOO-CV) andhence can serve as a metric for out-of-sample predictive accuracy of a model. We believethat predictive accuracy of a model is a critical criterion for choosing the “right” model.Second, the WAIC is relatively easy to compute. The computation of the WAIC can bedone at the end of MCMC sampling and does not involve additional MCMC runs. Third,unlike a deviance information criterion (DIC) (Spiegelhalter et al., 2002, 2014), the WAICis fully Bayesian and applicable to both nonsingular and singular models.5 Last, estimatesof the WAIC are numerically stable.

We use Gelman, Hwang and Vehtari (2014)’s formula to compute WAIC. Let Θ denote

5Singular models are statistical models with singularity in their Fisher-information matrix. When statis-tical models are singular, the assumption of large sample approximation in the Akaike information criterion(AIC) or the Bayesian information criterion (BIC) does not hold. Mixture models and hidden Markov modelsare examples of singular models.

11

all the parameters in a model. Then,

log pointwise predictive density =N∑i=1

∑t∈Ti

log

∫p(yit|Θ)ppostdΘ

pWAIC = 2N∑i=1

∑t∈Ti

(log(Epostp(yit|Θ))− Epost(log p(yit|Θ)))

WAIC = −2(log pointwise predictive density− pWAIC).

The expectation and integration in the formula are easily computable using simulated MCMCsamples.

3 Analysis of Korean Media Slant: News Coverage of

the Sewol Ferry Disaster

3.1 Aftermath of the Sewol Ferry Disaster

On April 16, 2014, a South Korean ferry capsized in the Yellow Sea on its way from Incheonto Jeju island. Out of total 476 people in the ferry including passengers and the crew, 295died; 9 are still missing at the time of writing this Paper. Most of the victims were highschool students and teachers on a field trip. The Sewol disaster was considered as the worstpeacetime disaster in South Korean history, leaving a tremendous impact on the collectivememory of the public.

Immediately after the disaster, reactions of South Korean people were homogenous with-out any political divide. People on the left and right were both raged by factors that causedthe disaster. For example, the ferry company, Cheonghaejin Marine, ignored basic safetyprotocols; government regulators blindly approved the condition of the freight before the de-parture; the captain and the crew who were not regular employees of Cheonghaejin Marine,but those of a contractor who did not make an evacuation order even at the moment theyleft the ferry; government rescue agencies and the Ministry of Security and Public Admin-istration wasted critical moments for early rescuing; and the president did not comprehendthe seriousness of the disaster until several hours had passed after the incident. South Ko-rean people were torn by pictures and clips taken by dead high school students in their lastminutes: they were firmly following the last order of stay inside calmly from the crew whoalready deserted the ferry.

However, with the passage of time, media reports on the disaster started to divergefollowing the ideological division. One group of media, usually rendered “progressive” or“left-leaning,” considered the disaster as epitomizing systematic failure and government in-competence. They extensively reported demands by the families of the victims, civil organi-zations, and the opposition party (Saejungchi). Another group of media, usually considered“conservative” or “right-leaning,” focused on the investigation of the ferry company and thecrew. One of the most sensitive issues was whether the Congress should establish its own in-dependent investigation agency and, if so, how much authority and power such agency should

12

have. What made this issue so sensitive is whether the president who did not take any actionfor seven hours after the disaster should be investigated. The president’s party (Saenuri)harshly denounced the demands for the investigation as pointless and ill-intentioned polit-ical attacks. The two electionsthe nationwide gubernatorial election on June 4, 2014 andthe by-election to fill fifteen vacancies in Congress on July 30, 2014 further escalated thepoliticization of the issue.

In this Section, we estimate ideological leanings of nineteen South Korean newspapersbased on their Sewol-related news reports from the date of the incidence, April 16, 2014, toApril 11, 2015. Our quantity of interests is (1) whether South Korean newspapers showedsignificant ideological leanings in their reports related to the Sewol ferry disaster and (2)whether these ideological leanings have changed in response to changes in public opinion.

3.2 Data

For the analysis, we downloaded newspaper reports containing the term, “Sewol-ho” fromApril 16, 2014 to April 11, 2015 on the NAVER News stand (http://news.naver.com).6

Nineteen major newspapers in South Korea are the targets of analysis. The total numberof downloaded newspaper reports containing “Sewol-ho” is 122,317. On average, each news-paper published 6,438 reports on various issues related with the ferry disaster for 360 days.This is equivalent to 18 reports per day for each newspaper. We also downloaded the twomajor political parties’ press releases containing “Sewol-ho” for the same period in order toidentify partisan phrases in the issues related with the disaster. The government party ofSaenuri published 334 press releases while the opposition party of Saejungchi published 577press releases containing the term, “Sewol-ho.” We decompose each newspaper report andpress release by 20 morphemes using KoNLPy, a Python package for natural language pro-cessing of the Korean language (Park and Cho, 2014). We discard meaningless morphemesin the Korean language.

Then, we search bigram or trigram phrases that were used by only one political party (one-party phrases) regarding the Sewol ferry disaster. This sets the method apart from that ofGentzkow and Shapiro (2010). Gentzkow and Shapiro (2010) choose partisan phrases byidentifying a list of partisan phrases that have a predictive power of individual legislators’party identification. This “feature selection” method is not feasible in our case because wedo not have incident-related speech data at the individual legislator level. However, ourmethod of choosing partisan phrases has several advantages. First, our method of one-partyphrase selection is highly intuitive. If we find partisan phrases from a pool of phrases usedby both parties, we need to consider an arbitrary threshold in terms of asymmetric usageto classify partisan words. Yet, there is no doubt that bigram or trigram phrases that areused by only one political party and never used by the other party well satisfy the definitionof “partisan phrases.” Second, political parties choose phrases in their press releases verycautiously and selectively. Unlike individual legislators’ speech, press releases rarely containa cheap talk or a slip of the tongue. They are well planned and sophisticatedly calculatedsignals to outside audience.

6The Naver is the most popular South Korean web search portal and the NAVER News stand is NAVER’sonline news content providing service.

13

After removing some meaningless phrases from a pool of one-party phrases, we haveninety partisan phrases for the opposition party and thirty for the government party. Figure 4shows partisan phrases of the opposition party. Some of the main phrases utilized to criticizethe government for mishandling the disaster are: congressional investigation, irresponsiblegovernment, obstinacy, and deregulation. On the other hand, Figure 5 shows partisan phrasesof the government party in defense of the president. The government party accused theopposition party of inciting conflicts and divisive conflicts; it emphasizes the normalizationof Congress and the bipartisan focus on the economy.

Figure 4: Partisan Words by the Opposition Party ( Saejungchi): Word sizes are adjusted by relativefrequency. Frequent words are located at the center.

Using the chosen partisan phrases, we compute the relative frequency of partisan phrasesfor each media. Let O and G be the total counts of partisan phrases for the opposition partyand the government party, respectively. Let fOpposition

i,o,t be the frequency of the oth oppositionparty phrase within reports by media i at t. rOpposition

o is the weight of the oth oppositionparty phrase measured by the relative frequency of the oth opposition party phras in theopposition party’s press releases. We compute the relative frequency by a ratio of each phrasefrequency to the maximum frequency in each party’s phrases. We define quantities for thegovernment party similarly. Then, the relative frequency of partisan phrases for newspaperi at t is defined as

yit =G∑g=1

fGovernmenti,g,t × rGovernment

g −O∑o=1

fOppositioni,o,t × rOpposition

o . (6)

14

Figure 5: Partisan Words by the Government Party ( Saenuri): Word sizes are adjusted by relative fre-quency. Frequent words are located at the center.

We subtract frequencies of the opposition party phrases from those of the governmentparty phrases so that the negative sign of yit indicates the liberal direction and the positivesign indicates the conservative direction. Since opposition party phrases are more verbosethan government party phrases, the mean of yit is not necessarily close to zero.

Figure 6 shows yit (colored dots) for the nineteen South Korean newspapers over the sam-ple period. The thick solid line in the middle indicates daily averages of yit. Positive signsindicate conservative media reports, and negative signs indicate liberal media reports. Over-all, daily averages of the relative partisan phrase frequency are below zero; the distributionof yit is quite large.

Figure 7 illustrates a close look at yit by displaying data for only two newspapers: MunhwaIlbo and Hangyereh Shinmun. Generally speaking, Munhwa Ilbo is considered as one of mostconservative newspapers while Hangyereh Shinmun is one of most liberal newspapers inSouth Korea. Reflecting this conventional belief, the relative partisan phrase frequencies ofHangyereh Shinmun are almost always located below those of Munhwa Ilbo. The averagerelative partisan phrase frequencies of Hangyereh Shinmun is -3.40 (standard deviation =5.45) and that of Munhwa Ilbo is 0.01 (standard deviation = 2.07).

15

Figure 6: Relative Partisan Phrase Frequencies of Nineteen Newspaper Companies: Colored dots are rel-ative frequencies of partisan phrases for each newspaper and the red thick line is daily averages of relativefrequencies of partisan phrases.

Partisan Word Relative Frequency

Rel

ativ

e F

requ

ency

2014

−04

−16

2014

−04

−26

2014

−05

−06

2014

−05

−16

2014

−05

−26

2014

−06

−05

2014

−06

−15

2014

−06

−25

2014

−07

−05

2014

−07

−15

2014

−07

−25

2014

−08

−04

2014

−08

−14

2014

−08

−24

2014

−09

−03

2014

−09

−13

2014

−09

−23

2014

−10

−03

2014

−10

−13

2014

−10

−23

2014

−11

−02

2014

−11

−12

2014

−11

−22

2014

−12

−02

2014

−12

−12

2014

−12

−22

2015

−01

−01

2015

−01

−11

2015

−01

−21

2015

−01

−31

2015

−02

−10

2015

−02

−20

2015

−03

−02

2015

−03

−12

2015

−03

−22

2015

−04

−01

2015

−04

−11

−25

−20

−15

−10

−5

05

Figure 7: Relative Partisan Phrase Frequencies of Two Newspaper Companies: Munhwa Ilbo and Hangy-ereh Shinmun

Partisan Word Relative Frequency

Rel

ativ

e F

requ

ency

2014

−04

−16

2014

−04

−26

2014

−05

−06

2014

−05

−16

2014

−05

−26

2014

−06

−05

2014

−06

−15

2014

−06

−25

2014

−07

−05

2014

−07

−15

2014

−07

−25

2014

−08

−04

2014

−08

−14

2014

−08

−24

2014

−09

−03

2014

−09

−13

2014

−09

−23

2014

−10

−03

2014

−10

−13

2014

−10

−23

2014

−11

−02

2014

−11

−12

2014

−11

−22

2014

−12

−02

2014

−12

−12

2014

−12

−22

2015

−01

−01

2015

−01

−11

2015

−01

−21

2015

−01

−31

2015

−02

−10

2015

−02

−20

2015

−03

−02

2015

−03

−12

2015

−03

−22

2015

−04

−01

2015

−04

−11

−30

−20

−10

010

20

16

4 Results

4.1 Model Selection Result

Table 1 demonstrates the results of model diagnostics for the nine models. The largest WAICin the second row assures that the static multilevel model has the smallest predictive powerand that it poorly fits the data. Although all the dynamic models show much smaller WAICvalues, the HMDMLMs have better predictive power than the DMLM. Among them, thefive break model (HMDMLM (5)) shows the smallest WAIC; it is analyzed in detail in thefollowing.

Table 1: Model Diagnostics: A linear dynamic multilevel model with 6 parametric breaks has the smallestWAIC.

Model Break Number WAIC log pointwise predictive density pwaic

MLM 0 32965.996 -16442.941 40.057DMLM 0 32801.005 -16353.122 47.381

HMDMLM 1 31358.685 -15642.791 36.551HMDMLM 2 31375.490 -15639.193 48.553HMDMLM 3 31360.990 -15628.587 51.907HMDMLM 4 31372.260 -15624.030 62.100HMDMLM 5 31321.365 -15620.046 40.636HMDMLM 6 31332.379 -15621.829 44.360HMDMLM 7 31363.821 -15618.744 63.166

Figure 8: Public Opinion Trend and 6 Breaks: Expected break points are 2014-05-26, 2014-08-01, 2014-09-19, 2014-12-08, 2015-02-18.

Public Opinion Trend

Sla

nt

2014

−04

−16

2014

−04

−26

2014

−05

−06

2014

−05

−16

2014

−05

−26

2014

−06

−05

2014

−06

−15

2014

−06

−25

2014

−07

−05

2014

−07

−15

2014

−07

−25

2014

−08

−04

2014

−08

−14

2014

−08

−24

2014

−09

−03

2014

−09

−13

2014

−09

−23

2014

−10

−03

2014

−10

−13

2014

−10

−23

2014

−11

−02

2014

−11

−12

2014

−11

−22

2014

−12

−02

2014

−12

−12

2014

−12

−22

2015

−01

−01

2015

−01

−11

2015

−01

−21

2015

−01

−31

2015

−02

−10

2015

−02

−20

2015

−03

−02

2015

−03

−12

2015

−03

−22

2015

−04

−01

2015

−04

−11

−25

−20

−15

−10

−5

05

Trend (no break)

Trend (5 breaks)

Break Timing

Figure 8 shows the estimated public opinion trend and timings of the five breaks. Thefirst regime from April 16, 2014 to May 26, 2014 is distinguished by strongly negative values

17

of the relative frequency, reflecting the immediate impact of the Sewol ferry disaster onpublic opinion. The second regime, which is also distinguished by negative values of therelative frequency, lasts from May 26, 2014 to August 1, 2014. Note that these two regimesare identified by the gubernatorial election of June 4 and the by-election of July 30, 2014.The two elections were significant political events that electrified partisan accusations overwho was to blame. To many’s surprise, the government party managed to win eight out ofseventeen gubernatorial offices in the nationwide gubernatorial election in June and to sweepeleven out of fifteen seats in the by-election in July. After the two elections, especially theJuly one, public opinion quickly moved in the conservative direction.

The fourth regime started from September 19, 2014 and ended at December 8, 2014.The beginning of the fourth regime is likely to be marked by the driver beating incident;on September 17th, a member of the victim’s family and a member of the opposition partygot involved in beating a driver from a driving escort company, intensifying the criticalvoice against the victims’ families and the opposition party. In November 2014, the victim’sfamilies agreed to accept the bipartisan proposal for the special Sewol law that includedplans for the congressional investigation and compensation. The fifth regime from December8, 2014 to February 18, 2015 is very similar to the fourth regime except the fact that thevolatility in the relative frequency is larger in the fifth regime than in the fourth regime.The relatively stable public opinion during the fourth and fifth regime is largely due to thesuccessful bipartisan agreement of the special Sewol law in the National Assembly.

The stable public opinion continued until March 27, 2015, from which public opinion be-gan to sharply move towards the liberal direction. On March 27, the government announcedthe enforcement ordinance of the special Sewol law under which the government can freelyappoint their own bureaucrats to important positions within the special investigation com-mittee. The opposition party and the victims’ families harshly criticized the ordinance; evensome members of the ruling party considered the ordinance as the usurpation of legislativepower. Reflecting these concerns, public opinion turned away from the government and theruling party at the end of March 2015. Unfortunately, this sharp drop in the public opinionis not detected as another break due to the lack of information at the end of the sampleperiod.

4.2 Slant Estimates

Before discussing time-varying slant estimates from the HMDMLM, we first examine time-constant slant estimates from the DMLM for the sake of comparison. Figure 9 shows time-constant measures of media slant from the DMLM. The dots are posterior means, andhorizontal bars are 95% credible intervals. South Korean newspapers are quite selective inreporting partisan phrases of the two parties. We can roughly classify the nineteen news-papers into four groups: strongly liberal, moderately liberal, moderately conservative, andstrongly conservative. Hangyereh Shinmun and Kyunghang Shinmun distinguish themselvesas most liberal. Asia Economy, Hanguk Ilbo, Seoul Shinmun, Money Today, Kukmin Ilbo andSegye Ilbo can be considered as moderately liberal. Herald Economy, Donga Ilbo, HangukEconomy, Seoul Economy and Joongang Ilbo are moderately conservative. Lastly, ChosunIlbo, Mail Economy, Financial News, Digital Times, Junja Shinmun and Munhwa Ilbo aremost conservative in their choice of partisan phrases regarding the Sewol ferry disaster.

18

Figure 9: Time-Constant Media Slants After Controlling for Public Opinion

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

Hangyereh

Kyunghang

Asia Economy

Hanguk

Seoul

Money Today

Kukmin

Segye

Herald Economy

Donga

Hanguk Economy

Seoul Economy

Joongang

Chosun

Mail Economy

Financial News

Digital Times

Junja

Munhwa

−3 −2 −1 0 1Slant

Med

ia

No Break

This classification is quite consistent with previous studies on the ideological identifica-tion of the South Korean newspapers. For example, Lee and Koh measure political ideologiesof five major newspapers in South Korea (Joongang Ilbo, Chosun Ilbo, Donga Ilbo, Hangy-ereh Shinmun, Kyunghang Shinmun) by examining “the valences of information given byvarious sources appearing in US beef imports articles” (Lee and Koh, 2009, 458). They con-cluded that Joongang Ilbo, Chosun Ilbo and Donga Ilbo–moderately or strongly conservativenewspapers according to our measure–are more conservative than Hangyereh Shinmun andKyunghang Shinmun–strongly liberal newspapers according to our measure–in their selec-tion of news sources. Park (N.d.) also echoed this line of ideological division among theSouth Korean newspapers by examining news reports regarding the part time employmentproblem.

The next question is whether the positions of the newspapers have changed as publicopinion shifted in one way or the other during the sample period. Our model diagnostic testillustrated in Table 1 strongly suggests that the positions of the newspapers have changed asthe linear trend of public opinion shifted. The details of the change are reported in Figure10 and Figure 11.

Figure 10 shows regime-specific estimates of media slant.7 Two changes are notablefrom Figure 10. First, the rank order of the slant estimates changes over time. While

7Note that Chosun Ilbo did not provide their news reports to NAVER News stand from April 16 toAugust 29, 2014. Thus, they are missing in our data. Despite the missingness, the HMDMLM providesslant estimates for Chosun Ilbo from April 16 to August 29, 2014 by borrowing information from othernewspapers. Figure 10 reports regime-specific slant estimates and slant estimates for Chosun Ilbo during themissing periods have large variances due to the lack of direct information.

19

Figure 10: Regime Specific Media Slants

●●

●●

●●

●●●●●

●●●

●●●●●

HangyerehHanguk

KyunghangSeoul

Asia EconomyKukmin

DongaChosun

Hanguk EconomySegye

Money TodayFinancial NewsSeoul EconomyMail Economy

JoongangDigital Times

Herald EconomyMunhwa

Junja

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 1

●●●

●●

●●●●●●

●●●●●●●

●

HangyerehKyunghang

Asia EconomyHanguk

Money TodayHerald Economy

KukminSeoulSegye

Hanguk EconomyChosun

Seoul EconomyJoongang

DongaJunja

Mail EconomyDigital Times

Financial NewsMunhwa

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 2

●●●

●●

●●●●●●●●

●●●

●●●

HangyerehKyunghang

HangukSeoul

Money TodayAsia Economy

Herald EconomyDigital Times

JunjaJoongang

DongaSeoul Economy

Hanguk EconomySegye

ChosunFinancial News

KukminMail Economy

Munhwa

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 3

●●

●●●●●●●●●●●●●●●●

●

HangyerehKyunghang

SegyeHangukKukmin

Money TodayHerald Economy

Asia EconomyJoongang

DongaSeoul Economy

SeoulMail Economy

Hanguk EconomyChosun

MunhwaJunja

Digital TimesFinancial News

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 4

●●

●●●●●●●●●●●●●●●●●

HangyerehKyunghang

KukminAsia Economy

JoongangFinancial News

DongaHanguk

SegyeMail Economy

Herald EconomyMoney Today

MunhwaSeoul

Seoul EconomyChosun

Hanguk EconomyDigital Times

Junja

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 5

●●●●●●●●●●●●●●●●●●

●

Money TodayHangyereh

Asia EconomyKyunghang

SeoulKukmin

Herald EconomySegye

Financial NewsDonga

ChosunJoongang

Seoul EconomyHanguk Economy

HangukMail EconomyDigital Times

MunhwaJunja

−5.0 −2.5 0.0 2.5 5.0Slant

Med

ia

Regime 6

Hangyereh Shinmun and Kyunghang Shinmun are consistently located at the far left corner,as we saw in Figure 9, Hanguk Ilbo and Seoul Shinmun are very close to these two liberalnewspapers during the first regime. In the third regime, Hanguk Ilbo is located near the farleft corner with Hangyereh Shinmun and Kyunghang Shinmun. These two regimes – Regime1 and Regime 3 – are periods in which liberal newspapers are well identified from the othernewspapers.

The second notable finding from Figure 10 is changes in the variance of media slantsover time, which is drawn as line graphs in Figure 11 for easier interpretation. Regime 1and Regime 3 stand out in terms of large variances in media slants. The four ideologicalgroupings of the newspapers – strongly liberal, moderately liberal, moderately conservative,and strongly conservative – clearly emerged right after the ferry disaster. The ideologicalgroups became weaker during June and July 2014. Then, they reemerged after the July 30by-election. There must be a multitude of factors that contributed to the turn of publicopinion in the third regime: the surprising victory of the ruling party in the by-election, thediscovery of the death of the ferry company owner, Byung Eon You, and the uncompromisingattitude of the victim’s families.

5 Conclusion

Public opinion changes over time. Mass media try to maximize the market share by closelyreflecting shifts of public opinion. At the same time, journalists, editors, and other partici-pants of news production try to influence public opinion with their own views and agenda.The constant interaction of public opinion and mass media is one of essential characteristicsof modern democracy.

20

Figure 11: Changes in Media Slants Over Time: Colors are scaled based on the rank-order of constantmedia slant. Bright colors indicate conservative slants and dark colors indicate liberal slants. Dim verticallines in the background indicate timings of break in public opinion.

Slant Changes Over Time

Sla

nt

2014

−04

−16

2014

−04

−26

2014

−05

−06

2014

−05

−16

2014

−05

−26

2014

−06

−05

2014

−06

−15

2014

−06

−25

2014

−07

−05

2014

−07

−15

2014

−07

−25

2014

−08

−04

2014

−08

−14

2014

−08

−24

2014

−09

−03

2014

−09

−13

2014

−09

−23

2014

−10

−03

2014

−10

−13

2014

−10

−23

2014

−11

−02

2014

−11

−12

2014

−11

−22

2014

−12

−02

2014

−12

−12

2014

−12

−22

2015

−01

−01

2015

−01

−11

2015

−01

−21

2015

−01

−31

2015

−02

−10

2015

−02

−20

2015

−03

−02

2015

−03

−12

2015

−03

−22

2015

−04

−01

2015

−04

−11

−4

−2

02

4

Kyunghang

Kukmin

Donga

Digital Times

Mail Economy

Money Today

Munhwa

Seoul Economy

Seoul

Segye

Asia Economy

Junja

Chosun

Joongang

Financial News

Hangyereh

Hanguk Economy

Hanguk

Herald Economy

Estimation of media slant becomes an important empirical endeavor to understand strate-gic and political choices of mass media in its market. These choices are fundamentally shapedby public opinion. However, the existing methods of media slant have focused on how tomap observed text data with a low dimensional vector without paying enough attention tothe fact that news reports are time-ordered data and that media slant may change in re-sponse to exogenous shocks or public opinion shifts. This Paper has sought to fill this gapby presenting its own method for dynamic estimation of media slant.

We build our method upon a simple multilevel model that parameterizes media slantas group-level varying intercepts when we observe frequencies of partisan phrases at themedia-time level. Then, we let the global mean of the observed partisan phrase frequencysmoothly vary over time using the linear dynamic model in Bayesian time series literature. Inthe model, the smoothly moving average partisan phrase frequency captures public opinionchanges. In order to check the possibility of changes in media slant, we allow group-levelvarying intercepts to follow hidden Markov transitions identified by shifts in the linear trendof public opinion. We suggest the WAIC as a measure of model diagnostics to choose themost reasonable model out of a pool of our competing models – either static or dynamic.

The paper has illustrated the application of this dynamic method by analyzing the ide-ological positions of nineteen South Korean newspapers in their news reports related withthe Sewol disaster for the period from April 16, 2014 to April 11, 2015. We have uncovereddramatic changes in public opinion and ideological positions of the newspapers. Reflectingthe fluctuations in public opinion, the variance of media slant has changed over time. Wefound that the by-elections of July 30 played an important role in shifting public opinion

21

in favor of the ruling party and the government. Moreover, some newspapers transformedtheir ideological positions greatly in the middle of the sample period. Static models of mediaslant would have failed not only to distinguish public opinion changes from changes in mediaslant, but also to detect short-term changes in ideological positions of mass media.

22

References

Agirdas, Cagdas. 2015. “What Drives Media Bias? New Evidence From Recent NewspaperClosures.” Journal of Media Economics 28:123–141.

Baron, David P. 2006. “Persistent Media Bias.” Journal of Public Economics 90:1–36.

Bernhardt, Dan, Stefan Krasa, and Mattias Polborn. 2008. “Political polarization and theelectoral effects of media bias.” Journal of Public Economics 92:1092 – 1104.

Carter, C. K., and R. Kohn. 1994. “One Gibbs Sampling for State-Space Models.” Biometrika81:541–533.

Chib, Siddhartha. 1995. “Marginal Likelihood From the Gibbs Output.” Journal of theAmerican Statistical Association 90 (December):1313–1321.

Chib, Siddhartha. 1996. “Calculating Posterior Distributions and Modal Estimates inMarkov Mixture Models.” Journal of Econometrics 75:79–98.

Chib, Siddhartha. 1998. “Estimation and Comparison of Multiple Change-Point Models.”Journal of Econometrics 86 (June):221–241.

Chib, Siddhartha, and Bradley P. Carlin. 1999. “On MCMC Sampling in Hierarchical Lon-gitudinal Models.” Statistics and Computing 9:17–26.

Fruhwirth-Schnatter, Sylvia. 1994. “Data Augmentation and Dynamic Linear Models.” Jour-nal of Time Series Analysis 15:183–202.

Fruhwirth-Schnatter, Sylvia. 2006. Finite Mixture and Markov Switching Models. Heidelberg:Springer Verlag.

Gans, Joshua S., and Andrew Leigh. 2012. “How Partisan is the Press? Multiple Measuresof Media Slant.” The Economic Record 88:127–147.

Gelfand, Alan E., and Adrian F. M. Smith. 1990. “Sampling-Based Approaches to Calculat-ing Marginal Densities.” Journal of the American Statistical Association 85 (June):398–409.

Gelman, Andrew, Jessica Hwang, and Aki Vehtari. 2014. “Understanding predictive infor-mation criteria for Bayesian models.” 24:997–1016.

Gelman, Andrew, John B. Carlin, Hal S. Stern, and Donald B. Rubin. 2004. Bayesian DataAnalysis. 2nd ed. New York: Chapman and Hall.

Gentzkow, Matthew, and Jesse M. Shapiro. 2010. “What Drives Media Slant? EvidenceFrom U.S. Daily Newspapers.” Econometrica 78:35–71.

Gerber, Alan S., Dean Karlan, and Daniel Bergan. 2009. “Does the Media Matter? AField Experiment Measuring the Effect of Newspapers on Voting Behavior and PoliticalOpinions.” American Economic Journal: Applied Economics 1:35–52.

23

Green, Peter J. 1995. “Reversible Jum Markov Chain Monte Carlo Computation andBayesian Model Determination.” Biometrika 82:711–732.

Groseclose, Tim, and Jeffrey Milyo. 2005. “A Measure of Media Bias.” Quarterly Journal ofEconomics CXX:1191–1237.

Jackman, Simon. 2005. “Pooling the Polls Over an Election Campaign.” Australian Journalof Political Science 40:499 – 517.

Kass, Robert E., and Adrian E. Raftery. 1995. “Bayes Factors.” Journal of the AmericanStatistical Association 90:773–795.

Kim, In Song, John Londregan, and Marc Ratkovic. 2015. “Voting, Speechmaking, and theDimensions of Conflict in the US Senate.” Presented at the 2015 Asian Political Method-ology Meeting.

Larcinese, Valentino, Riccardo Puglisi, and Jr. James M. Snyder. 2007. “Partisan Bias inEconomic News: Evidence on the Agenda-Setting Behavior of U.S. Newspapers.” Journalof Public Economics 95:1178–1189.

Lee, Gunho, and Heungseok Koh. 2009. “Korean Newspaper’s Political Orientation Featuringin US Beef Imports Articles.” Korean Journal of Journalism Communication Studies53:347–369.

Matthew Gentzkow, Jesse M. Shapiro. 2006. “Media Bias and Reputation.” Journal ofPolitical Economy 114:280–316.

Park, Eunjeong, and Sungzoon Cho. 2014. KoNLPy: Korean natural language processing inPython. In Proceedings of the 26th Annual Conference on Human & Cognitive LanguageTechnology.

Park, Jae-Young. N.d. “Ideological Dimension of South Korean News Media (Hanguk Eolon-sadulye Jungpasung Jihyung).” Seminar on the Jounalism Practice Committee.

Raftery, Adrian E. 1995. “Bayesian Model Selection in Social Research.” Sociological Method-ology 25:111–163.

Spiegelhalter, David J., Nicola G. Best, Bradley P. Carlin, and Angelika Van Der Linde. 2002.“Bayesian measures of model complexity and fit.” J R Stat Soc Series B Stat Methodol64:583–639.

Spiegelhalter, David J., Nicola G. Best, Bradley P. Carlin, and Angelika van der Linde. 2014.“The deviance information criterion: 12 years on.” Journal of the Royal Statistical Society:Series B (Statistical Methodology) 76:485–493.

Stone, Daniel F. 2011. “Ideological media bias.” Journal of Economic Behavior Organization78:256 – 271.

Stromberg, David. 2015. “Media and Politics.” Annual Review of Economics 7:173–205.

24

Taddy, Matt. 2013. “Multinomial Inverse Regression for Text Analysis.” Journal of theAmerican Statistical Association 108:755–770.

Vehtari, Aki, and Janne Ojanen. 2012. “A survey of Bayesian predictive methods for modelassessment, selection and comparison.” pp. 142–228.

Vehtari, Aki, and Jouko Lampinen. 2002. “Bayesian Model Assessment and ComparisonUsing Cross-Validation Predictive Densities.” Neural Computation 14 (2015/07/19):2439–2468.

Watanabe, Sumio. 2010. “Asymptotic equivalence of Bayes cross validation and widelyapplicable information criterion in singular learning theory.” Journal of Machine LearningResearch 11:3571–3594.

Zaller, John. 1999. A Theory of Media Politics. Unpublished.

25