a systematic analysis of the discrepancies between press … · 2019-09-03 · when freedom house...

A Systematic Analysis of the Discrepancies between Press Freedom as Measured by Reporters

Without Borders and Freedom House

Lee B. Becker

James M. Cox Jr. Center for International Mass Communication Training and Research

Grady College of Journalism and Mass Communication

University of Georgia

U.S.A.

Laura Schneider

Graduate School Media and Communication

Department of Journalism and Communication Science

University of Hamburg

Germany

Tudor Vlad

James M. Cox Jr. Center for International Mass Communication Training and Research

Grady College of Journalism and Mass Communication

University of Georgia

U.S.A.

ABSTRACT

Since 2002, Reporters Without Borders, based in Paris, and Freedom House, based in New York,

have conducted parallel, and ostensibly independent, measures of press freedom around the

world. Across those years, the two nongovernmental organizations have produced measures that

are extremely highly correlated. Perhaps because of the high correlations, no systematic analysis

has been conducted of the discrepancies. This paper does just that, focusing on discrepancies

that are consistent year-to-year as well as those that do not replicate. By using the textual

summaries of discrepant cases, the authors attempt to understand the differences as a way of

illuminating consistencies and discrepancies in the methodologies of the two evaluators.

Paper presented to the international conference Media and the Public Sphere, Lyon, France, July

2-3, 2012.

1

Introduction

Two organizations currently produce quantitative measures of media freedom around the

world based on the work of professional evaluators. The best known and most widely used

measure of press freedom is that of Freedom House (FH). A nongovernmental organization based

in Washington, D.C., Freedom House was founded in 1941 to promote democracy globally. In

1980, Freedom House began conducting its media freedom survey Freedom of the Press: A

Global Survey of Media Independence which in 2011 covered 196 countries and territories

(Freedom House, 2011).

The second organization that rates media freedom globally and ranks countries based on

these evaluations is Reporters Without Borders (RWB). Reporters Without Borders has released

annually since 2002 its media freedom survey Press Freedom Index, which in 2011 covered 179

countries and territories (RWB, 2012a). Based in Paris, RWB defends journalists and media

outlets by condemning attacks on press freedom worldwide, by publishing a variety of annual and

special reports on media freedom, and by appealing to governments and international

organizations on behalf of journalists and media organizations.

One criticism of the FH ranking is that it reflected U.S. perspectives on economic and

political pressures on the media in different countries and on their judicial systems. The creation of

the RWB index has been an alternative tool to the FH ranking.

Both of these indices have been widely used by governments, nongovernmental

organizations and, more recently, by media scholars. They also have been repeatedly challenged

and contested.

In Germany in 2006, for example, the political opposition blamed the ruling coalition for

doing nothing in order to strengthen the “defaulted and endangered” press freedom in the country,

when Reporters Without Borders downgraded Germany by five ranks (Spiegel Online, 2006).

When Freedom House changed the status of the Italian media from “free” to “partly free” in 2009,

Prime Minister Silvio Berlusconi’s spokesman Paolo Bonaiuti said in a plenary session of the

European Parliament: “The left has made them (the press freedom indices, author’s note) become

famous like Pink Floyd. (…) Why do 27 left-wing European MPs accuse Italy over a lack of

2

freedom of information when everybody knows it’s not true?” (adnkronos, 2010). When Malawi

plunged 67 places in the RWB index in 2011, the presidential and the government spokespersons

strongly argued that the report was biased and did not reflect the media reality in the country

(Media Institute of Southern Africa, 2012). Two professors at the Walter Cronkite School of

Journalism and Mass Communication criticized the most recent RWB ranking of the U.S.

(Grobmeier, 2012). They argued that the arrests of journalists who participated in the “Occupy”

movement should not have led to a drop of 27 places in the index. The Freedom House measure

for 2011 showed only a drop of one point in the country’s rating.

Despite the challenges to both measures and the discrepancies between them, relatively

little analysis has been done comparing the two indices (Becker, Vlad and Nusser, 2007; Becker

and Vlad, 2011). This paper focuses on those similarities and differences.

Freedom House and Reporters Without Borders Measurement

Both the Freedom House and the Reporters Without Borders global press freedom indices

are generally based on the principles constituted in Article 19 of the Universal Declaration of

Human Rights approved by the U.N. in 1948, although RWB does not state that explicitly.

Freedom House, however, claims that its study is based on “universal criteria”, taking as a starting

point “the smallest, most universal unit of concern: the individual” (Freedom House, 2011).

Although neither of the two organizations provides a detailed definition of its concept of press

freedom, the methodology sections of their reports indicate what the respective indices attempt to

measure.

Freedom House’s Freedom of the Press index has an institutionalized perspective and

seeks to detect the various ways in which “pressure can be placed upon the flow of information

and the ability of print, broadcast, and internet-based media to operate freely and without fear of

repercussions” (Freedom House, 2011). Thus, it aims to provide an idea of the entire enabling

environment in which the media in each country operate.

Freedom House uses three broad categories in order to measure the status of press

freedom in the almost 200 countries included in the index: the legal environment, the political

environment and the economic environment. Every category comprises between seven and eight

3

methodology questions that are filled out by the evaluators. The legal environment category

“encompasses an examination of both the laws and regulations that could influence media content

and the government’s inclination to use these laws and legal institutions to restrict the media’s

ability to operate” (Freedom House, 2011). Hence, this category includes questions about the

protection of press freedom through the constitution and other basic laws, the independence of the

judiciary, Freedom of Information legislation as well as about the market entry for all kinds of

media. The political environment category evaluates “the degree of political control over the

content of news media.” Issues examined include the control over sources, censorship and

self-censorship, citizens’ access to a wide range of news media and the safety of both local and

foreign journalists. Finally, the last category measures the economic environment for the media,

looking at media ownership and its concentration, limitations to news production and distribution

and control through advertising and subsidies, among others.

Each methodology question contains several sub-questions. Those 109 indicators are

meant to provide guidance and help to evaluate each question correctly. For each country an

analyst prepares the draft rating and country report. The analysts are external scholars as well as

members of the core research team in New York. In the next step, the ratings are reviewed

individually and on a comparative basis during regional and cross-regional meetings with the

analysts and a dozen senior-level advisers. According to Freedom House the information needed

to determine the score for each question is gathered from professional contacts, staff and

consultant travel, international visitors, the findings of human rights and press freedom

organizations, specialists in geographic and geopolitical areas, the reports of governments and

multilateral bodies, and a variety of domestic and international news media. The overall score of a

country is calculated by determining the score for each of the 23 methodology questions and then,

in a last step, adding the scores of the three categories. With 40 possible points, the weighting of

the political environment category is greater in the final index than the scores of the other two,

accounting for a maximum of 30 points each.

The scores resulting from the Freedom House evaluations range from 0 to 100. The

current scale, however, is a replacement for a three-point ordinal measure used from the initiation

of the measures in 1980 until 1992. In those early years, Freedom House differentiated between

media systems judged to be “Free”, “Partly Free”, or “Not Free”. Since the 100-point scale was

created, Freedom House reduces the data by labeling countries with a total score of 0 to 30 as

4

“Free.” Nations that score 31 to 60 are classified as “Partly Free.” And countries with more than 60

points are labeled as “Not Free.” The status of the countries is shown graphically in Freedom

House’s Map of Press Freedom every year.

According to Reporters Without Borders, the Press Freedom Index reflects “the degree of

freedom that journalists, news media and netizens enjoy in each country, and the efforts made by

the authorities to respect and ensure respect for this freedom” (Reporters Without Borders, 2012).

As an organization with the mission of defending journalists RWB’s Press Freedom Index focuses

on attacks on journalists and the media, like physical violence, imprisonment and censorship.

Consequently, the indicators used to compile the index tend to concentrate more on the

journalist’s freedom as an individual, while economic factors play a subordinate role. The French

organization explicitly states that its measurement neither indicates the quality of the media in the

concerned countries nor the existence of human rights violations in general.

In order to measure the status of press freedom in the roughly 180 countries included in

the index, the French organization uses a questionnaire composed of seven categories. The

names of the categories have changed slightly over the years, but they still assess approximately

the same issues. For the 2011 index these categories were: violence and other abusive treatment

of journalists and the state responsibility in these abuses, the state’s role in combating impunity for

those responsible for violence and abuses, censorship and self-censorship, media overview,

including the diversity of news media, media legislation, judicial, business and administrative

pressure as well as Internet and new media (Reporters Without, Borders 2012). Out of the total of

44 questions, 20 deal with physical violence and other abuses against journalists and the state’s

and non-state actors’ role in them. Like Freedom House, Reporters Without Borders allocates

points for each question. Neither of the two organizations explains why a certain question is

weighed with a certain number of points.

Since the first publication of its annual press freedom index in 2002, Reporters Without

Borders has used a 0 to 100 scale for scoring the countries’ media. In 2002 the worst ranked

country, North Korea, had a score of 97.5 points. In 2007, the worst country, Eritrea, had a score of

114.75 points. In 2010, the worst country, again Eritrea, had 105 points. For its 2011 Press

Freedom Index RWB changed its point system and for the first time had more answers assigning

negative points in its questionnaire. As a result, in 2011 the best ranked countries, Finland and

5

Norway, each had a score of -10, while Eritrea at the bottom of the rating scored 142 points.

According to Reporters Without Borders, the change helps to “have a bigger spread in the scores

and increase differentiation between countries” (Reporters Without Borders 2012). Still, the

authors of the index claim that “although the point system has produced a broader distribution of

scores than in 2010, each country’s evolution over the years can still be plotted by comparing its

position in the index rather than its score.”

In order to get the scores for each country, the questionnaire is sent to 18 Reporters

Without Borders’ partner organizations, to its network of 150 correspondents in all five continents,

as well as to journalists, researchers, jurists and human rights activists. The questionnaires are

sent back to the researchers in Paris who, as a final step, convert the responses into points.

An examination of the two indices and their measurement systems demonstrates different

concepts of press freedom. While both focus on the constraints on the media resulting from state

control, Reporters Without Borders gives more attention to threats from actors other than the

state, including armed militias, secret organizations and terrorists (Holtz-Bacha, 2011). In addition,

Freedom House concentrates more on legal limitations, while Reporters Without Borders has a

strong focus on direct attacks on journalists. Freedom House’s questionnaire contains only one

question that explicitly measures the violence against journalists. Reporters Without Borders’

measurement instrument, in contrast, has 11 questions concerning the different kinds of direct

attacks on journalists. Freedom House only allots a maximum of 6 points to the question on

violence against journalists. In the RWB index, up to 95 points are allocated to violence against

journalists. These are important measurement differences between the two indices that can result

in discrepancies of both the scores and the ranks of one country. They are not just minor

differences that could be explained by the evaluation of different experts. Rather, they reflect

somewhat different concepts of press freedom.

The time period reflected in the two indices also differs. Freedom House normally covers

events from the year previous to its publication date. For the 2012 index, for example, that means

that Freedom House took into account events between January 1, 2011, and December 31, 2011.

The Reporters Without Borders index for 2011 reflects the situation between December 1, 2010,

and November 30, 2011.

6

Both indices reverse code, meaning that a low score represents higher levels of press

freedom. The general similarity in the range of possible scores further adds to the suggestion that

the two indices should produce similar outcomes. The finding of a high correlation between the

two scores (Becker, Vlad and Nusser, 2007; Becker & Vlad, 2011), however, is worthy of closer

examination.

Methodology

In order to understand the discrepancies between the Freedom House and the Reporters

Without Borders measures of press freedom, we conducted an analysis of the measures by the

two organizations from 2002, when Reporters Without Borders released its first report, through

2010, the most recent year for which Freedom House measures were available when the analysis

was done.

The analysis focused on the nature of the distributions of scores for the two measures and

on the statistical relationship between the two measures. It next examined discrepant cases,

identified through a comparison of the ranks and the scores.

Findings

Despite the apparent similarities in the two indices, even a cursory examination of the

distributions shows important differences (Table 1). In each of the nine years, the range of scores

assigned to countries by RWB was greater than the range of scores used by Freedom House. The

minimum score is zero or slightly less in the RWB scale. The maximum RWB score ranged from

97.5 in the first year of the report, when only 139 countries were classified, to 115.5 in 2009. For

FH, the range is between 87 and 90 for the nine, with minimum scores of 8 to 10 and maximum

scores of 96 to 99. On both scales, a low score indicates high press freedom, while a high score

indicates low levels of press freedom.

So while both RWB and FH seem to be using roughly the same scale, in fact they are not.

The RWB scale is larger (has a larger range) than the FH scale. Even more striking is the

difference in the mean scores. For RWB, the mean each year hovers around 27, ranging from

26.22 in 2003 to 31.63 in 2007. For Freedom House, the mean is near 46 most years, ranging from

7

44.79 to 47.44. In other words, Freedom House gives higher scores (reflecting lower press

freedom) on average, than does RWB. The standard deviations around these means are

surprisingly similar for the two distributions, and those standard deviations are largely stable

across the nine years for which comparable data are available.

For RWB, the median scores (that is, the score at the middle of the distribution) are lower

than the means, indicating that the scores are skewed. For the FH measures, the medians and the

means for most years are not so different. Table 1 shows the skewness statistic, which measures

the extent to which the distribution deviates from symmetry around the mean. A value of zero

means the distribution is symmetric. (The normal curve has a skewness statistic of 0.) The positive

skewness score for the RWB distributions indicates a greater number of smaller values, in this

case, meaning a preponderance of scores reflecting high levels of press freedom. The Freedom

House measures do not show such a degree of skewness, with the statistic closer to what would

be expected with a normal distribution. In other words, the FH measures are more evenly spread

on the distribution.

Kurtosis is a measure of the peakedness or flatness of a distribution. A kurtosis value near

zero indicates a shape close to normal. A negative value indicates a distribution which is more

peaked than normal, and a positive kurtosis indicates a shape flatter than normal. The RWB

distributions are flatter than normal, that means they are spread widely from the mean. The

Freedom House measures are more peaked than normal, meaning most of the cases are

centered around the mean with relatively few cases in the tails.

The actual distributions are shown in the histograms in Charts 1-9. The clustering of the

cases in the RWB measure near the low end of the scale (high freedom) is most obvious from

these charts. The Freedom House histograms show another feature of those distributions: they

tend towards bimodality. In each year, scores tend to cluster around 30 and around 60. These are

the break points for classification of countries as “Free”, “Partly Free” or “Not Free”. The clustering

at these scores shows the continued impact of these historical distinctions on the use of the data

gathered with the 100-point scale.

Since both the Reporters Without Borders and Freedom House measures are interval

level, the Pearson Product Moment Correlation Coefficient would seem to be the best measure of

8

the relationship between them. That measure is shown in Table 2, and it documents the high

degree of correspondence between the two measures. The coefficient of determination, or r

squared, indicates about 60 percent of the variance in one of the measures is explained by the

other.

But the analysis of the characteristics of the two scales in Table 1 indicated that one of the

key assumptions of Pearson r is violated. Pearson’s r requires that the two variables used in the

analysis be normally distributed. As noted, the RWB measures are badly skewed. The RWB

distributions are flatter than normal, and the Freedom House measures are more peaked than

normal.

Two alternative measures are Kendall’s tau and Spearman’s rho. Neither requires

normally distributed data. The first focuses on cases that are concordant in contrast with cases

that are discordant. Ties are neither. The tau b is a preferred subvariant of Kendall’s tau that

corrects the measure of concordance for tie scores in a more complex fashion. Spearman’s rho

focuses on the ranks, rather than the actual data values themselves.

Table 3 shows that, based on the Kendall tau, the relationship between the RWB and FH

measures of press freedom also is judged to be quite high, though a bit attenuated from what is

shown in Table 2. In Table 4, which shows the Spearman’s rhos, the relationship is a bit larger

than is the case with the Pearson r coefficients shown in Table 2. Of course, the coefficients are

not exactly comparable, but each is designed to show the strength of the relationship on a scale

from -1 to +1.

So, while each of these measures shows high correspondence between press freedom as

measured by Reporters Without Borders and press freedom as measured by Freedom House,

they also indicate that there are discrepancies.

To identify those discrepancies, that is countries that were evaluated differently by RWB

and FH, two procedures were followed. The first focused on the difference of deviation scores and

is consistent with the computation of the Pearson r. The second focused on discrepancies in

ranks, and is consistent with the computation of Spearman’s rho.

9

To identify the discrepant cases based on the means, deviation scores from the mean of all

countries were created for each country for each year. These deviation scores were then

converted to standard deviation units by dividing the deviation score for each country by the

standard deviation of the respective distribution. Finally, the deviation score for a country for FH

were subtracted from the deviation score of a country for RWB for each year. All deviations of 1 or

greater were selected for analysis. In other words, countries which differed in evaluation by RWB

and FH of more than one standard deviation unit were selected.

To identify the discrepant cases based on the ranks, a similar procedure was used. In both

ratings, more than one country often receive the same score and are thus tied in the ranks. Both

RWB and FH handle ties in the same way. All countries tied get the same rank, and that rank is set

as the low value (rather than average) of the range represented by that rank. For example, in

2009, Freedom House assigned four countries (Finland, Iceland, Norway and Sweden) the score

of 10, giving them the rank of 1 in the index. Denmark got a score of 11 and was assigned the rank

5. FH uses a smaller range of scores (Table 1), and it has more ties than RWB. This confounds the

comparison of ranks.

For each year, the rank assigned to a country by RWB was compared with the rank

assigned to that same country by FH. Overall, these discrepancies were quite large, and this

contributes to the view held by many that the RWB and FH ratings are quite different. Countries for

which the discrepancies in the ranks were 50 or larger were selected for analysis.

Table 5 shows 31 countries that Freedom House rated higher (worse) in terms of press

freedom than did RWB by at least one standard deviation unit for at least one year. It also shows

14 countries that RWB rated higher (worse) in terms of press freedom than did FH by one

standard deviation unit at least one year. Two countries were listed worse sometimes and better

sometimes.

Table 6 shows the 75 countries that Freedom House rated with a higher score (less press

freedom) than did Reporters Without Borders across at least one of the nine years. It also shows

the single case (Philippines) that RWB ranked with a higher score (worse) than did Freedom

House.

10

The differences between the numbers in the two tables show the volatility of comparisons

based on the ranks, particularly given the differences in the ways ties are used by RWB and FH.

Since RSF ranked fewer countries in the earlier years, the comparison of ranks also is

problematic. This volatility is reflected in Table 7 as well, which shows that 37 countries had

differences of rank scores of 50 or greater at least one year but had a difference of less than one

standard deviation unit.

Table 8 parallels Table 7 containing eight countries in which the deviation scores are 1 or

greater but in which the rank differences are less than 50.

As a next step, the top 11 countries in Table 5 with differences across at least three years

in which Freedom House assigned a higher score and the six countries in which RWB assigned a

higher score for at least three years were selected out for analysis. Seven of these countries are in

the top 10 in Table 6 as well. Five countries, Bosnia, Bhutan, Chad, Gambia, and Macedonia also

were added to the list. All are in the top 10 in Table 6. Chad and Macedonia appear lower in Table

5. Bosnia is one of the countries in Table 7 that is most discrepant.

These 21 countries with discrepant cases in terms of difference scores (Table 5) or

rankings (Table 6) were analyzed separately by looking at the country reports written to support

their evaluations by RWB and FH. RWB does not publish updated essays each year. Where

possible, essays from the same year were used for this comparison.

Analysis of Country Reports of Discrepant Cases

While Freedom House provides country reports for all countries evaluated each year,

Reporters Without Borders makes available country reports more selectively. Since there is no

country report by RWB available for Burundi, Tajikistan, Bosnia, Macedonia, Bhutan, Liberia and

Chad, only 14 of the 21 countries identified by difference scores or differences in rankings were

analyzed in detail.

The comparison of the RWB and FH country reports of the discrepant cases underscores

the differences in the length and the content of the reports by the two organizations. In general, the

Freedom House reports are longer and thus much more detailed. For example, the RWB country

11

report of Iran for 2010 consists of 701 words, while the FH report of Iran for the same year is 1,608

words long. Furthermore, Freedom House’s country reports have a structure that remains the

same across all countries for all years. The reports deal with the legal environment of a country

first, then describe the political environment and, finally, the economic conditions. These are the

three dimensions used in FH’s questionnaire as well. Each Freedom House essay indicates at its

end the status of access to and freedom of the Internet in the respective country. This facilitates

the analysis of the situation of one country across several years as well as the comparison of

different countries. The essays written by Reporters Without Borders do not have a fixed structure,

and some of them are updated, but not on a regular basis.

The analysis of the discrepant cases underlines the differences detected in the

measurement systems used by RWB and FH, as a result of the differences in the

conceptualization of media freedom. RWB focuses more on the journalists’ individual experiences

and working conditions and on the challenges their media organizations are facing, while FH has a

more institutional approach, examining the legal framework, and the political and economic

environment that allow or restrict the ability of the media to operate.

There are some similarities and some differences that can be found in almost all country

reports across the years. Both the RWB and the Freedom House essays normally contain details

about different kinds of censorship, self-censorship, the situation of foreign journalists and media

inside the country and especially about all kinds of harassment and intimidation of journalists.

More variation occurs regarding legal restrictions, freedom of information legislation, advertising

system and especially the situation of new media. These topics are mentioned regularly in the FH

country reports, while they are rarely addressed in the RWB essays.

The section about direct attacks on journalists is very prominent and detailed in all RWB

country reports, reflecting the organization’s focus on this aspect of press freedom. This can be

seen clearly in the country reports for the Philippines, Nepal and Togo, for example. The essays

on Gambia, the Central African Republic and the Philippines show that Freedom House, in

contrast, gives more importance to the existence or absence of special laws. The FH reports

describe the newest developments of legal bills and amendments. In the more recent RWB

reports, however, special laws also are mentioned. The country reports on Paraguay, Nepal and

Eritrea show that Freedom House focuses on the distribution of advertising as well as on the

12

situation of the new media in the respective countries. In the essay on Eritrea, for example, the

monitoring of email communication is mentioned, and in the report on the United Arab Emirates,

FH deals with the situation of mobile telephony. Furthermore, all Freedom House reports deal both

with the freedom of and with the access to the Internet. Reporters Without Borders, in contrast,

does not regularly take into account the Internet.

The country reports do not wholly explain the discrepancies between the ratings of

Freedom House and those of Reporters Without Borders, but they affirm the differences in terms

of the conceptualization of press freedom and thus the measurement instruments used by RWB

and FH. Neither of the two organizations provides much detail about its conceptualization of press

freedom. It is obvious from the measures used that that the two organizations concentrate on

distinct aspects of press freedom. This different focus is reflected in the measurement system as

well as in the country reports.

Conclusions

On the surface, the Freedom House and Reporters Without Borders measures of press

freedom are quite similar. Both claim to be measuring that general concept. Both use elite

evaluators to obtain their assessments of the media systems. Both ask those evaluators to

complete questionnaires that score the media on a variety of characteristics.

On closer examination, differences in the measures emerge. Freedom House has a more

institutional focus, looking at constraints on the media, mostly from institutional forces. Reporters

Without Borders focuses more heavily on attacks on journalists coming from a variety of sources.

Despite these differences, analysis has shown that the two organizations produce

evaluations that are more similar than different. The data presented here support that conclusion.

But that simple conclusion does mask some important differences. The distributions of

scores across the countries are quite different. In neither case are they normally distributed. In the

case of Reporters Without Borders, the distribution is heavily skewed, with countries concentrated

at the high end of the scale (more press freedom). The Freedom House distribution is consistently

bimodal, reflecting historical classifications of countries that seem to influence how the actual

13

100-point scale is used.

Countries in the real world are not necessarily normally distributed in terms of media

freedom, so existence of flat distributions are even skewed ones is not an indicant of any flaw in

the measurement. The differences between the distributions of the two indices, however, do

suggest differences in the concepts being measured. The comparisons are contaminated by the

differences in the numbers of countries evaluated by the two organizations. Freedom House has a

more extensive list of countries than does Reporters Without Borders, though the difference has

gotten smaller over time.

While the empirical comparisons have focused on similarities, which exist even with the

differences in the underlying distributions, media freedom advocates seem to have focused more

on the differences in ranks. Both Freedom House and Reporters Without Borders score many

countries as tied in terms of ranks. Freedom House, with the larger number of countries and the

more restricted range of scores, has more ties than does Reporters Without Borders. So the use of

ranks is problematic.

To get around this problem, we identified a cluster of countries that differed in terms of

standard deviation units. We then added to the cluster countries that differed greatly in terms of

ranks. We looked at the essays, where they were available, to try to help understand the reasons

why these cases are scored differently by the two organizations.

In the end, the analyses are consistent with the initial examination of the measures used by

the two organizations. While there is overlap in terms of the concepts being measured, there also

are differences. As noted, Freedom Houses has more focus on institutional constraints, while

Reporters Without Borders focuses more on attacks on journalists.

The analyses here have implications for those working in the area of media advocacy as

well as for those who use these indices for empirical analyses both of the antecedents of and

consequences of media freedom. The former should recognize the different emphases of the two

measures and use them accordingly. The latter should understand the same conceptual

distinction. They also should be sensitive to the nature of the distributions and cautious about the

appropriate statistical tools they use as a result.

14

References

Adnkrnonos (2010). Italy: Berlusconi's spokesman responds to media freedom report. Retrieved

on April 10, 2012, from

http://www.adnkronos.com/AKI/English/CultureAndMedia/?id=3.0.3900135626

Becker, L.B., Vlad, T., & Nusser, N. (2007). An evaluation of press

freedom indicators. The International Communication Gazette, 61 (1): 5-28.

Becker, L.B., & Vlad, T (2011). The conceptualization and operationalization of country-level

measures of media freedom. In M. Price & S. Abbott (Eds.), Measures of press freedom and

media contributions to development (pp. 23-46) New York: Peter Lang.

Freedom House (2011): Freedom of the Press 2011. Methodology. Retrieved on April 11, 2012,

from http://www.freedomhouse.org/report/freedom-press-2011/methodology

Grobmeier, D. (2012). US drops 27 places in international press freedom index. Retrieved on April

12, 2012, from

http://www.statepress.com/2012/01/31/us-drops-27-places-in-international-press-freedom-index/

Holtz-Bacha, C. (2011). Freedom of the press: Is a worldwide comparison possible and what is it

good for? In M. E. Price et. Al. (Eds.), Measures of Press Freedom and Media Contributions to

Development. Evaluating the Evaluators (pp. 129-143). New York: Peter Lang.

Media Institute of Southern Africa (2012). Malawi: Nation rated poorest on press freedom index.

Retrieved on April 12, 2012, from http://allafrica.com/stories/201202021297.html

Reporters Without Borders (2012): 2011-2012 Press freedom index. How the index is compiled.

Retrieved on April 11, 2012, from:

http://en.RWB.org/IMG/pdf/how_the_2011-2012_index_was_compiled.pdf

Reporters Without Borders (2012a): Press Freedom Index 2011-2012. Retrieved on May 31,

15

2012, from: http://en.rsf.org/press-freedom-index-2011-2012,1043.html

Spiegel Online (2006). Mehr Pressefreiheit in Bolivien als in Deutschland. Retrieved on April 10,

2012, from: http://www.spiegel.de/politik/deutschland/0,1518,444291,00.html

http://en.rsf.org/press-freedom-index-2011-2012,1043.html

Statistic Std. Error Statistic Std. Error2002 139 97 0.5 97.5 27.42 23.21 23.5 1.39 0.21 1.71 0.41

2003 164 99 0.5 99.5 26.22 23.22 20.25 1.31 0.19 1.60 0.38

2004 164 107 0.5 107.5 29.45 25.30 23.5 1.10 0.19 0.80 0.38

2005 164 108.5 0.5 109 27.13 23.71 20.875 1.24 0.19 1.19 0.38

2006 164 108.5 0.5 109 27.15 23.75 19.085 1.36 0.19 1.46 0.38

2007 166 114 0.75 114.75 31.63 24.34 25.33 1.10 0.19 0.99 0.38

2008 170 96 1.5 97.5 26.36 22.88 19.5 1.26 0.19 1.07 0.37

2009 175 115.5 0 115.5 29.76 25.66 21.5 1.25 0.18 1.21 0.37

2010 177 105 0 105 31.13 24.33 23.75 1.03 0.18 0.49 0.36

2002 192 88 8 96 44.79 25.32 40.5 0.19 0.18 -1.29 0.35

2003 193 90 8 98 45.53 25.08 43 0.19 0.18 -1.23 0.35

2004 191 88 9 97 46.38 24.76 44 0.19 0.18 -1.22 0.35

2005 191 88 9 97 46.70 24.55 45 0.22 0.18 -1.14 0.35

2006 195 88 9 97 46.37 24.48 45 0.25 0.17 -1.13 0.35

2007 195 89 9 98 46.73 24.45 45 0.24 0.17 -1.14 0.35

2008 195 89 9 98 47.08 24.45 47 0.23 0.17 -1.13 0.35

2009 196 89 10 99 47.37 24.36 48 0.21 0.17 -1.14 0.35

2010 196 87 10 97 47.44 24.17 48 0.23 0.17 -1.11 0.35

N of Countries Range Minimum Maximum Mean Median

Std. Deviation

Rep

orte

rs W

ithou

t Bor

ders

Free

dom

Hou

se

Firm YearSkewness Kurtosis

Table 1. Descriptive Statistics

FH02 FH03 FH04 FH05 FH06 FH07 FH08 FH09 FH10Pearson 0.81 0.80 0.81 0.80 0.79 0.79 0.79 0.77 0.78N 137 138 139 139 138 138 138 138 138

Pearson 0.83 0.84 0.84 0.84 0.83 0.83 0.83 0.81 0.81

N 162 163 164 164 163 163 163 163 163

Pearson 0.81 0.81 0.83 0.83 0.82 0.82 0.82 0.80 0.80

N 162 163 164 164 163 163 163 163 163

Pearson 0.81 0.81 0.83 0.84 0.84 0.83 0.82 0.82 0.82

N 161 162 163 163 162 162 163 163 163

Pearson 0.78 0.78 0.81 0.83 0.83 0.83 0.83 0.82 0.82

N 161 162 163 163 162 162 163 163 163

Pearson 0.80 0.81 0.83 0.85 0.86 0.86 0.86 0.85 0.86

N 161 162 163 163 165 165 166 166 166

Pearson 0.76 0.77 0.79 0.81 0.82 0.83 0.83 0.82 0.83

N 165 166 167 167 169 169 170 170 170

Pearson 0.77 0.79 0.81 0.82 0.84 0.84 0.85 0.86 0.86

N 167 168 169 169 171 171 172 172 172

Pearson 0.78 0.79 0.81 0.82 0.84 0.84 0.84 0.85 0.86

N 169 170 171 171 173 173 174 174 174

RSF05

RSF02

RSF03

RSF04

RSF06

RSF07

RSF08

RSF09

RSF10

Table 2. Pearson Product Moment Correlation Coefficients

FH02 FH03 FH04 FH05 FH06 FH07 FH08 FH09 FH10Kendall's 0.73 0.73 0.73 0.72 0.70 0.70 0.70 0.69 0.69N 137 138 139 139 138 138 138 138 138

Kendall's 0.73 0.74 0.74 0.74 0.73 0.73 0.72 0.70 0.70

N 162 163 164 164 163 163 163 163 163

Kendall's 0.69 0.69 0.71 0.71 0.71 0.71 0.70 0.68 0.68

N 162 163 164 164 163 163 163 163 163

Kendall's 0.72 0.71 0.73 0.75 0.74 0.75 0.73 0.72 0.72

N 161 162 163 163 162 162 163 163 163

Kendall's 0.70 0.69 0.72 0.74 0.76 0.76 0.75 0.74 0.73

N 161 162 163 163 162 162 163 163 163

Kendall's 0.67 0.68 0.70 0.72 0.74 0.73 0.73 0.72 0.73

N 161 162 163 163 165 165 166 166 166

Kendall's 0.67 0.67 0.70 0.72 0.73 0.74 0.74 0.74 0.74

N 165 166 167 167 169 169 170 170 170

Kendall's 0.67 0.68 0.71 0.72 0.73 0.74 0.75 0.76 0.76

N 167 168 169 169 171 171 172 172 172

Kendall's 0.65 0.66 0.68 0.69 0.70 0.70 0.71 0.72 0.73

N 169 170 171 171 173 173 174 174 174

RSF10

RSF02

RSF03

RSF04

RSF05

RSF06

RSF07

RSF08

RSF09

Table 3. Kendall's tau b Correlation Coefficients

FH02 FH03 FH04 FH05 FH06 FH07 FH08 FH09 FH10Spearman's 0.90 0.90 0.89 0.89 0.87 0.87 0.87 0.85 0.86N 137 138 139 139 138 138 138 138 138

Spearman's 0.90 0.90 0.90 0.90 0.90 0.90 0.89 0.87 0.87

N 162 163 164 164 163 163 163 163 163

Spearman's 0.87 0.87 0.88 0.89 0.88 0.88 0.87 0.86 0.86

N 162 163 164 164 163 163 163 163 163

Spearman's 0.89 0.88 0.90 0.91 0.91 0.90 0.89 0.89 0.89

N 161 162 163 163 162 162 163 163 163

Spearman's 0.87 0.86 0.89 0.91 0.91 0.91 0.91 0.90 0.89

N 161 162 163 163 162 162 163 163 163

Spearman's 0.84 0.85 0.87 0.88 0.89 0.89 0.89 0.88 0.89

N 161 162 163 163 165 165 166 166 166

Spearman's 0.83 0.84 0.86 0.87 0.88 0.89 0.90 0.89 0.89

N 165 166 167 167 169 169 170 170 170

Spearman's 0.84 0.85 0.87 0.88 0.89 0.90 0.90 0.92 0.92

N 167 168 169 169 171 171 172 172 172

Spearman's 0.83 0.84 0.86 0.86 0.87 0.87 0.88 0.89 0.90

N 169 170 171 171 173 173 174 174 174

RSF06

RSF07

RSF08

RSF09

RSF10

RSF02

RSF03

RSF04

RSF05

Table 4. Spearman's rho Correlation Coefficients

CountryNumber of Years

Discrepant (SD Unit Difference > or = 1)

Years DiscrepantAverage

Discrepancy in SD Units

Togo 8 All except 02 -1.57

Burundi 7 All exc. 06,07 -1.29Tajikistan 7 All exc. 03,05 -1.16

Moldova 5 04,05,06,07,08 -1.09

Paraguay 5 02,03,04,09,10 -1.14

United Arab Emirates 5 06,07,08,09,10 -1.32

Central African Republic 4 02,06,09,10 -1.06

Liberia 4 06,07,08,09 -1.15

Guinea 3 02,04,09 -1.17

Qatar 3 06,07,08 -1.10

Rwanda 3 03,04,05 -1.14

Armenia 2 07,08 -1.07

Bahrain 2 02,08 -1.17

Chad 2 03,10 -1.14

Djibouti 2 09,10 -1.04

Sudan 2 02,04 -1.10

Swaziland 2 02,04 -1.17

Zambia 2 07,08 -1.17

Angola 1 05 -1.13

Cameroon 1 05 -1.03

Cote d’Ivoire 1 02 -1.28

Egypt 1 02 -1.05

Equatorial Guinea 1 04 -1.02

Haiti 1 03 -1.13

IOT/PA 1 03 -1.05

Kenya 1 02 -1.03

Lebanon 1 02 -1.37

Macedonia 1 03 -1.01

Malaysia 1 06 -1.09

Mauritania 1 07 -1.04

Zimbabwe 1 10 -1.04

Eritrea 7 All exc. 02,04 1.22

North Korea 7 All exc. 08,10 1.15

China 5 02,03,04,06,08 1.28

Iran 5 03,05,06,07,09 1.18

Nepal 3 04,05,06 1.33

Philippines 3 05,06,10 1.17

Burma 2 02,03 1.03

Cuba 2 03,04 1.06

Laos 2 02,03 1.38Turkmenistan 2 08,09 1.04

Vietnam 2 03,08 1.20

Israel 1 08 1.76

Pakistan 1 06 1.14

Sri Lanka 1 08 1.32

The Gambia 4, 3x (-) & 1x (+) 02,03,04,10 -0.40

Bhutan 3, 1x (-) & 2x (+) 02,03,08 0.66

Reporters Without Boarders Score Higher (Lower Freedom)

Mixed Discrepancies

Freedom House Score Higher (Lower Freedom)

Table 5. RSF and FH Discrepancies 2002 – 2010: Standard Deviation Units

CountryNumber of Years Discrepant (Rank

Difference > or = 50)Years Discrepant

Average Discrepancy in

Ranks

Togo 9 All 90.44Bosnia 8 All exc. 10 67.25

Burundi 7 All exc. 06,07 73.29Tajikistan 7 All exc. 03,06 61.29

Macedonia 7 All exc. 02,10 59.00Liberia 6 All exc. 03,04,10 65.00

Central African Republic 5 02,06,07, 09,10 64.00United Arab Emirates 5 06,07,08,09,10 75.80

Paraguay 5 02,03,04,09,10 75.60Chad 4 02,03,05,10 58.00

El Salvador 4 02,03,04,05 60.25Palestinian Territory 4 02,03,04,05 66.25

Moldova 4 04,05,06,07 61.75Qatar 4 06,07,08,09 59.75

Rwanda 4 02,03,04,06 64.25Seychelles 4 05,08,09,10 53.50

Venezuela 4 02,03,04,05 65.00Bhutan 3 08,09,10 56.33

Kuwait 3 07,08,09 53.33Lebanon 3 02,08,09 67.67

Zambia 3 07,08,10 63.67Cote d’Ivoire 2 02, 06 72.00

Guinea 2 02,04 78.00Ecuador 2 02,03 65.00

Bahrain 2 02,08 70.00Sudan 2 02,04 65.00

Haiti 2 02,03 67.50Namibia 2 02,10 54.50

Egypt 2 02,03 63.50Swaziland 2 02,04 71.00

Albania 2 03,04 64.00Congo (Kinshasa) 2 02,03 59.00

The Gambia 2 02,04 65.50Armenia 2 04,07 59.00

Argentina 2 02,09 52.00Angola 2 02,05 64.00

Equatorial Guinea 2 02,04 58.00Guatemala 2 04,10 52.00

Cameroon 2 04,05 55.00Gabon 2 02,06 56.00

Malawi 1 09 56.00Guinea-Bissau 1 10 57.00

Tanzania 1 10 55.00Bolivia 1 06 64.00

Kenya 1 02 68.00Georgia 1 07 62.00

Malaysia 1 06 58.00Honduras 1 04 54.00

Panama 1 06 58.00Mauritania 1 07 69.00

Romania 1 07 52.00Dominican Republic 1 04 54.00

Maldives 1 09 51.00Zimbabwe 1 02 65.00

Uzbekistan 1 02 65.00Nigeria 1 02 62.00

Cambodia 1 02 62.00Iraq 1 02 62.00

Indonesia 1 02 61.00Trinidad and Tobago 1 03 52.00

Sri Lanka 1 02 59.00Libya 1 02 59.00

Niger 1 02 58.00Brunei 1 02 57.00

Azerbaijan 1 02 56.00Afghanistan 1 02 56.00

Belarus 1 02 56.00Cuba 1 02 56.00

Kyrgyzstan 1 02 55.00Sierra Leone 1 02 54.00

Burma 1 02 53.00Turkmenistan 1 02 53.00

Vietnam 1 02 52.00Uruguay 1 02 51.00

Philippines 1 10 63.00

Reporters Without Boarders Rank Higher (Lower Freedom)

Freedom House Rank Higher (Lower Freedom)

Table 6. RSF and FH Discrepancies 2002 – 2010: Ranks

Country Years Discrepant 2002-2010

Average Discrepancy in Ranks

Ecuador 02,03 65.00

Venezuela 02,03,04,05 65.00Congo (Kinshasa) 02,03 59.00

Uzbekistan 02 65.00

Bosnia All exc. 10 67.25

Nigeria 02 62.00Cambodia 02 62.00

Iraq 02 62.00

Indonesia 02 61.00Gabon 02,06 56.00

Libya 02 59.00

Niger 02 58.00

Brunei 02 57.00Azerbaijan 02 56.00

Afghanistan 02 56.00

Belarus 02 56.00

Namibia 02,10 54.50Kyrgyzstan 02 55.00

El Salvador 02,03,04,05 60.25

Sierra Leone 02 54.00Uruguay 02 51.00

Argentina 02,09 52.00

Albania 03,04 64.00

Trinidad and Tobago 03 52.00Dominican Republic 04 54.00

Honduras 04 54.00

Guatemala 04,10 52.00

Seychelles 05,08,09,10 53.50Bolivia 06 64.00

Panama 06 58.00

Georgia 07 62.00Romania 07 52.00

Kuwait 07,08,09 53.33

Malawi 09 56.00

Maldives 09 51.00Tanzania 10 55.00

Guinea-Bissau 10 57.00

Table 7. Countries Different by Ranks (> or = 50) but Not by SD Units (< 1)

Country Years Discrepant 2002-2010

Average Discrepancy in SD Units

Djibouti 09,10 -1.04Eritrea All exc. 02,04 1.22North Korea All exc. 08,10 1.15China 02,03,04,06,08 1.28Iran 03,05,06,07,09 1.18Nepal 04,05,06 1.33Loas 02,03 1.38Isreal 08 1.76

Table 8. Countries Different by SD Units (> or = 1) but Not by Ranks (< 50)

Chart 1. Reporters Without Borders and Freedom House Ratings 2002

a systematic analysis of the discrepancies between press … · 2019-09-03 · when freedom house...

Documents