research project sample chapter
TRANSCRIPT
Conversations About News: To What Extent Can Social Media Be Seen As A News Watchdog?
SAMPLE CHAPTER
Chapter 3: Discussion of the Research Design, Methodological Issues and Ethics
Changes from Semester 1
In light of feedback from senior colleagues and others, a number of changes have been
made since the project report was submitted on December 12th 2013. The original intention
was to answer the secondary question of “why do users tweet about news coverage?” by
adopting a mixed methods approach. This would have involved selecting a random sample
of users who had tweeted positively or negatively and contacting them for an interview to
try and ascertain their motivations. It was commented that this approach while
“commendable” was very “ambitious”, and more generally the study could be lacking in
focus. Thus the decision was made to cut the interview process altogether which had
negative but also positive consequences.
It was regrettable that this study was unable to answer the “why” question because it leaves
a gap in the literature that it was had hoped could be examined in more detail. With more
time and resources it is more likely that this study would have been able to answer this
question, but it is true that reformulating the scope of the research and refocusing it, has
set the base level for future research to be conducted. Furthermore, using quantitative data
analysis allowed answers to the question of extent and the covering of some other gaps.
The research was originally limited to studying the BBC due to resource limitations however,
once interviews were cut, those original shortcomings were accounted for by studying
different news outlets take on the same stories. Repeating the analysis across different
news outlets has increased the reliability of the findings and granted us access to a wider
study sample making data collection much easier.
Operationalization
Before discussing the specifics of the research it is important to clarify terms. When this
study talks of “news outlet” it is referring to major UK online news outlets which were
separated into two data sets: TV News and Newspapers. This was done primarily to answer
the question of how the level criticism varies between news outlets but also because, even
though both types of outlet produce regular content online, they could have faced criticisms
for different reasons (such as bad script/print editing).
The outlets chosen for analysis within TV News were the three main terrestrial TV channels
with news programmes; specifically BBC News, ITV News, and Channel 4 News. This was
done because most households who have access to a TV will have access to the terrestrial
channels. Immediately there is a methodological issue to consider here in that these outlets
benefits from a different share of total viewing and as such the results might contain an
inherent bias. Furthermore, the BBC has a 24 news outlet in the form of BBC News 24 which
the others do not and this could make it more prone to immediate criticism as Twitter itself
also operates on a twenty-four hour basis. Indeed none of the other outlets have a
dedicated “breaking news” Twitter account. This is accounted for in the study by
disregarding the @BBCBreaking account for the purposes of the research. This study also
included accounts of the three best-selling newspapers: The Sun, The Daily Mail and The
Daily Mirror in order to answer question 2 of the sociological problem: “How does this vary
between news outlets?”.
When discussing how users hold news outlets to account it is also necessary to define the
term “criticism”. The Press Complaints Commission has had an Editor’s Code of Conduct
published since 1936. It therefore made sense in the course of the design to operationalize
search terms around its latest update ratified in 2012. Criticism of news outlets can be wide
ranging so for the purpose of this study it was decided to narrow the term to the keyword of
“Bias”. Bias is discussed at length by Bob Franklin (2005 pp. 24-25) who says “in everyday
use, bias implies that the ‘real world’ constitutes an objective reality which the media
persistently fail to represent”. This can be done consciously for political reasons or
structurally in the focus on reporting of certain story types or places, for example London
receives a lot of media attention due to the fact Parliament is located there. Crucially
however bias has become the go to criticism for the general public because “the notion of
bias is significant and enjoys affinities with the cognate concepts of objectivity, impartiality,
balance and truth”.
Finally, It is also necessary to define what is meant by the term “news” thought the question
of what stories actually constitute news is one as old as news outlets themselves. Rather
than select specific news stories that all six selected news outlets reported on at the same
time, this study chose instead to look at all news content produced and shared via Twitter in
the first quarter of 2014. This was mainly done because the chosen search tools,
Twitonomy.com and Topsy.com, allowed for easy searching of tweets which mentioned
certain accounts and contained the keyword bias but struggled to produce much data when
also asked to search those terms with a multi-word phrase such as “Leveson Report”. By
searching the term bias more generally this study was able to get a better picture of the
general proportions of twitter criticism of these particular outlets. Moreover certain trends
in tweets which used the term biased were quickly seen, as to what issues certain twitter
users perceive the news outlets as bias.
Quantitative Methods: Measuring Proportions of Mentions of “Bias” Against General
Tweets About News Outlets.
In order to best answer the question of how prolific Twitter users are in their criticism of
news outlets, this study examined secondary data from the Twitter analytic sites
Twitonomy.com and Topsy.com between the period of March 21st 2014 to April 20th 2014. It
is necessary to uses two search tools because they each mainly search for two different data
sets. Twitonomy.com produces extremely accurate data on account output as well the
number of times a tweet from a particular account has been favourite and/or retweeted but
it cannot search for general keyword terms for longer than a 6 day period, nor indeed can it
provide the number of times a keyword has been used in response to a particular account.
This is why data from Topsy.com was included. This provides data on the number of times
an account has been mentioned in a tweet in general, which is necessary for working out
what proportions of those tweets contain the relevant keywords and hashtags. Topsy.com is
case sensitive so it was necessary to search for the terms “bias” and “biased” to get an
accurate reading.
Twitonomy.com was also able to provide data about hashtag and favourite frequency which
was included in this study. The reason for this is because “By including a hashtag in one’s
tweet it becomes included into a larger ‘conversation’ consisting of all tweets with the
hashtag” (Murthy 2011 pp.3). Hashtaging in a certain way therefore serves as a marker of a
particular sentiment by a user because the user is marking their wish to join the
‘conversation’ and have their tweet included in wider searches. Similarly by favouriting a
tweet the user is simultaneously expressing approval at the tweet and storing the tweet so
it can be read a later date. Both of these functions hint at deeper emotional expressions
than might otherwise be present by measuring just the frequency of a keyword.
There are a number of issues with this method of quantitative research such as the issue of
validity because gathering all the tweets about a specific hashtag or using a specific keyword
will make it almost impossible to eliminate “junk” tweets such as those from automated
accounts. These accounts are used primarily for promoting business and they comprise
anywhere between 5%-9% of all Twitter accounts (Elder 2013), which is not an insignificant
number. It is possible that a number of fake twitter accounts will be incorporated in the
statistical findings, so the final calculations will need to be adjusted accordingly. It is also
necessary to account for the disparity of data between Topsy.com and Twitonomy.com,
which occurs because the websites track and process data at different speeds. Whilst
regrettable, the difference between them is not so vast that it invalidates the findings,
rather it must be noted that the proportions are not strictly exact. Finally, it is difficult to
ascertain the context in which tweets have been tweeted: the user might have been hacked,
under the influence of alcohol or simply not well informed on the story for example. In
order to gain a richer understanding as to why users tweet it is necessary to explore more
qualitative methods of research.
Ethics
The issue of how this research project will conducted ethically is one that has needed
careful consideration. In the context of online posts, Bryman (2008 pp.129) has pointed out
“When participants have not given their assent to have their postings used in this way, it
could be argued that the principle of informed consent has been violated”. However it must
be noted that in the case of Twitter, unless the user has changed privacy settings, all tweets
are in the public domain and thus are easily accessible by anyone, anywhere and at any
time. Thus there is little ethical concern for the quantitative phase of the research.