Participatory Journalism and Social News Services01Issues in social news services
Participatory journalism: The act of a citizen or group of citizens playing an
Participatory Journalism?
-Issues in social news services
Participatory journalism: The act of a citizen, or group of citizens, playing an active role in the process of collecting, reporting, analyzing and disseminating news and information.The intent of this participation is to provide independent, reliable, accurate, wide-ranging and relevant information that a democracy requires.
Participatory Journalism in Social News ServicesUsers
AnalyzingCollecting & Reporting Disseminating
1 2 3 …Aggregated News
Social News Services
Ranked News1 2 3
Participatory Journalism and Social News Services (Cont’d)Issues in social news services
01-Issues in social news services
Journalistic values, such as accuracy, credibility, reliability, are considered important p
1997 2009
Media CredibilityPopularity
Slashdot’s Meta-moderation(1999)
Daum’sOpen Editing Algorithm (2008)
Problems in Voting-Based News Ranking System02Fundamental problems: Credibility of news media is determined by news contents, but votes cannot guarantee values of news contents
bl i h l i h f i i l iProblems in each algorithms of representative social news servicesVoting Method
Service Description Limitations
S O GSimpleVoting
Digg Once a submission has earned a critical mass of Diggs, it is promoted to “popular” and “most popular.”“Bury” for spam filtering
Group voting problems (Digg Mafia, Digg Bury Brigade)
Popularity>
Credibility
Reddit Submission score is determined by the number of Upvotes minus the number of downvotes
Group voting problems (Reddit downmodsquad)squad)
NewsCred
Credit Article / Discredit Article Group voting problems“credibility?”
WeightedVoting
Slashdot Moderation-Metamoderation system: metamoderators are randomly chosen in the system and rate moderators
Credibility of metamoderators are not considered
Rating News An rating instrument is given for users to Increased user burdenRating BasedVoting
NewsTrust
An rating instrument is given for users to rate news articles based on news values
Increased user burden
Researches on Journalistic Media Credibility03Elements in Traditional Media Credibility Measures– Trustworthiness, expertise, dynamism (Infante, 1980), p , y ( , )– Trustworthiness, fairness, bias, completeness, respect for privacy, representation of
individual interests, accuracy, concern for community well-being, separation of fact and opinion, concern for public interest, factual foundations of information
bli h d lifi i f ( i d 198 )published, qualifications of reporters (Rimmer and Weaver, 1987) – Believability, fairness, accuracy, depth of information (Johnson and
Kaye, 1998, 2000)
Elements in Online News Credibility Measures– factual/opinionated, unfair/fair, accurate/inaccurate, untrustworthy/trustworthy, balan
ced/unbalanced, biased/unbiased, reliable/unreliable, thorough/not thotough, informative/not informative (Ognianova, 1998)
Ki i (1999) d li dibili b ki h h li i– Kiousis (1999) measured online news credibility by asking whether online news is factual, concerned with making profits, concerned with community well-being, invades people’s privacy with 5-point Likert-type scale
03 Researches on Journalistic Media Credibility (Cont’d)
Related debates concerning news value elements
R l t d k R hRelated works Researcher
We Media S. Bowman and C. Willis, 2003
Blog,ParticipatoryJournalism
We the Media D. Gilmore, 2006
Blogging, Journalism & Credibility: Battleground and Common Ground R MacKinnon, 2005
Online News Credibility: An Examination of the Perceptions of Newspaper Journalists W. P. Cassidy, 2007
Restating News Values: Contemporary Criteria for Selecting the News J. Mcgregor, 2007
Research Questions & Process04Assumption: “There will be a group of people who vote for journalistically valuable news stories”Q1. How to measure journalistic value of social news contents?Q j– What are the attributes that valuable news stories have?– How to measure values of a number of news contents?– Should each news value elements have different weight for online social news contents?Q2 Wh t th t t t t t f j li ti ll l bl ?(V l V t )Q2. Who are voters that tent to vote for journalistically valuable news?(Value Voters)– How to derive value voters?– Do they show consistent tendency in voting behavior?
3. Evaluation1. Measuring
Journalistic Value2. Finding
Alpha Voters
News Set 1with Value Scores
Value Voters
News Set 2voted by Value voters
News Set 2voted by general voters
Value Voters
News Raters 1
N R t 2
News Set 2With Value Scores
voted by general voters News Raters 2
General Voters
Measuring Journalistic Value of Social News Contents05C. Lampe and R. Kelly Garrett’s Methodology
– Compare performances of several rating instruments(normative, descriptive, full, mini review)
– Subjects rated original and degraded news story with 4 rating instruments. – We adopt normative review because it shows the highest level of accuracy in that the
result shows similar news values as those from experts.
Attribute Description
- Representative News Value Elements
Instrument Err2 | Err | sd r
- Accuracy of ratings by instrument and story quality
Attribute Description
Normative
Accuracy
Credibility
Fairness
Instrument Err | Err | sd r
Full .80 .69 .06 .285**
Descriptive 1.39 .94 .07 .234**
Normative .64 .62 .05 .218*Informativeness
Originality
Balance
Clarity
Mini 2.90 1.43 .09 .326**
Descriptive
y
Context
Diversity
Evidence
Objectivity
Transparency
Measuring Journalistic Value of Social News Contents (Cont’d)05369 well-educated people measure values of top 100 weekly best current news stories of Daum View with normative review instrument (Accuracy, Credibility, Fairness, Informativeness, Originality)
To derive importance of each element in online news, our survey asked subjects to rate them with 5-point Likert scale, and we rescaled them so that the news value score ranges from 1 to 5
D i d i h f h l li d f llDerived weights for each elements are listed as follows
Accuracy Credibility Fairness Informativeness Originality
- Weights for each rating items
0.225 0.227 0.198 0.202 0.148
Measuring Journalistic Value of Social News Contents (Cont’d)05Finally, we derive the value scores of top 100 current news of Daum Blogger News
Previous Rank URL Value89 http://v daum net/link/3918133 4 109859
Examples of survey result
89 http://v.daum.net/link/3918133 4.10985993 http://v.daum.net/link/3930134 3.99689876 http://v.daum.net/link/3931410 3.88224445 http://v.daum.net/link/3912458 3.85602922 http://v.daum.net/link/3931027 3.80779147 http://v.daum.net/link/3912861 3.77774667 http://v.daum.net/link/3926142 3.70575529 http://v.daum.net/link/3913246 3.70377787 http://v.daum.net/link/3905833 3.64687533 http://v daum net/link/3948105 3 64247733 http://v.daum.net/link/3948105 3.64247736 http://v.daum.net/link/3915024 3.63390218 http://v.daum.net/link/3902557 3.62594130 http://v.daum.net/link/3902704 3.62299659 http://v.daum.net/link/3922091 3.59138770 http://v.daum.net/link/3900538 3.58417283 http://v.daum.net/link/3913077 3.58085657 http://v.daum.net/link/3911474 3.57849943 http://v.daum.net/link/3928202 3.55226864 http://v daum net/link/3925490 3 52133364 http://v.daum.net/link/3925490 3.52133325 http://v.daum.net/link/3913131 3.519752
Finding Value Voters 06We consider news stories whose value scores are over 3.5 as valuable news and votes for the articles valid.– In 5-point Likert Sclae, score 3 is a neutral point– We assume that news scores from 3.5 to 5 are positive points.
W i l h ’ lid ti t ( b f lid t di id dWe simply measure each users’ valid voting rate (number of valid votes divided by total number of votes)
Voters who made less than 5 votes are excluded from value voter listVoters who made less than 5 votes are excluded from value voter list.– Users with less than 5 votes can have high valid voting rate by chance
We flexibly adjust the lowest limit of valid voting rate from 0.3 to 0.5y j g
Evaluation07232 well-educated people participated in the second survey and evaluate another 100 current news stories of Daum View
We compared the value scores of top 50 news contents with the votes made by alpha voters
The result is as follows– Previous votes does not show any significant correlation with news value scores.
Votes from value voters shows significant correlation with news value scores– Votes from value voters shows significant correlation with news value scores – Selected with the valid voting rate over 0.4, alpha voters’ votes show significant
correlation with news value scores 70
# voters News Value Score
40
50
60# voters News Value Score
PreviousVotes
N/A Pearson Correlation -.063
Sig. (2-tailed) .665
VVR > 273 Pearson Correlation .288*
Vot
es
10
20
300.5 Sig. (2-tailed) .043
VVR > 0.4
620 Pearson Correlation .326*
Sig. (2-tailed) .021
VVR > 914 Pearson Correlation .287*
V
0
2 2.5 3 3.5 4
0.3 Sig. (2-tailed) .043
News Value Score
Works in Progress…08This paper is on the initial stage of deriving value voters and adapting the votes to news articlesWe will conduct researches concerning…
– Diversifying weights on the valuable stories(e g based on news values of articles or excluding malicious votes)(e.g. based on news values of articles or excluding malicious votes)
– Finding correlation between news values and news stories’ attributes.(e.g. correlation between news values of articles and number of trackback or comments)( g )
– Developing a news ranking model based on news values
Calculating
value
CalculatingVoters’
Reputation
Seed Articles Valuable News
valueVoters
Ranking System based on Alpha Voters Votes
Thank YouThank You