Participatory Journalism and Social News Services01Issues in social news services

Participatory journalism: The act of a citizen or group of citizens playing an

Participatory Journalism?

-Issues in social news services

Participatory journalism: The act of a citizen, or group of citizens, playing an active role in the process of collecting, reporting, analyzing and disseminating news and information.The intent of this participation is to provide independent, reliable, accurate, wide-ranging and relevant information that a democracy requires.

Participatory Journalism in Social News ServicesUsers

AnalyzingCollecting & Reporting Disseminating

1 2 3 …Aggregated News

Social News Services

Ranked News1 2 3

Participatory Journalism and Social News Services (Cont’d)Issues in social news services

01-Issues in social news services

Journalistic values, such as accuracy, credibility, reliability, are considered important p

1997 2009

Media CredibilityPopularity

Slashdot’s Meta-moderation(1999)

Daum’sOpen Editing Algorithm (2008)

Problems in Voting-Based News Ranking System02Fundamental problems: Credibility of news media is determined by news contents, but votes cannot guarantee values of news contents

bl i h l i h f i i l iProblems in each algorithms of representative social news servicesVoting Method

Service Description Limitations

S O GSimpleVoting

Digg Once a submission has earned a critical mass of Diggs, it is promoted to “popular” and “most popular.”“Bury” for spam filtering

Group voting problems (Digg Mafia, Digg Bury Brigade)

Popularity>

Credibility

Reddit Submission score is determined by the number of Upvotes minus the number of downvotes

Group voting problems (Reddit downmodsquad)squad)

NewsCred

Credit Article / Discredit Article Group voting problems“credibility?”

WeightedVoting

Slashdot Moderation-Metamoderation system: metamoderators are randomly chosen in the system and rate moderators

Credibility of metamoderators are not considered

Rating News An rating instrument is given for users to Increased user burdenRating BasedVoting

NewsTrust

An rating instrument is given for users to rate news articles based on news values

Increased user burden

Researches on Journalistic Media Credibility03Elements in Traditional Media Credibility Measures– Trustworthiness, expertise, dynamism (Infante, 1980), p , y ( , )– Trustworthiness, fairness, bias, completeness, respect for privacy, representation of

individual interests, accuracy, concern for community well-being, separation of fact and opinion, concern for public interest, factual foundations of information

bli h d lifi i f ( i d 198 )published, qualifications of reporters (Rimmer and Weaver, 1987) – Believability, fairness, accuracy, depth of information (Johnson and

Kaye, 1998, 2000)

Elements in Online News Credibility Measures– factual/opinionated, unfair/fair, accurate/inaccurate, untrustworthy/trustworthy, balan

ced/unbalanced, biased/unbiased, reliable/unreliable, thorough/not thotough, informative/not informative (Ognianova, 1998)

Ki i (1999) d li dibili b ki h h li i– Kiousis (1999) measured online news credibility by asking whether online news is factual, concerned with making profits, concerned with community well-being, invades people’s privacy with 5-point Likert-type scale

03 Researches on Journalistic Media Credibility (Cont’d)

Related debates concerning news value elements

R l t d k R hRelated works Researcher

We Media S. Bowman and C. Willis, 2003

Blog,ParticipatoryJournalism

We the Media D. Gilmore, 2006

Blogging, Journalism & Credibility: Battleground and Common Ground R MacKinnon, 2005

Online News Credibility: An Examination of the Perceptions of Newspaper Journalists W. P. Cassidy, 2007

Restating News Values: Contemporary Criteria for Selecting the News J. Mcgregor, 2007

Research Questions & Process04Assumption: “There will be a group of people who vote for journalistically valuable news stories”Q1. How to measure journalistic value of social news contents?Q j– What are the attributes that valuable news stories have?– How to measure values of a number of news contents?– Should each news value elements have different weight for online social news contents?Q2 Wh t th t t t t t f j li ti ll l bl ?(V l V t )Q2. Who are voters that tent to vote for journalistically valuable news?(Value Voters)– How to derive value voters?– Do they show consistent tendency in voting behavior?

3. Evaluation1. Measuring

Journalistic Value2. Finding

Alpha Voters

News Set 1with Value Scores

Value Voters

News Set 2voted by Value voters

News Set 2voted by general voters

Value Voters

News Raters 1

N R t 2

News Set 2With Value Scores

voted by general voters News Raters 2

General Voters

Measuring Journalistic Value of Social News Contents05C. Lampe and R. Kelly Garrett’s Methodology

– Compare performances of several rating instruments(normative, descriptive, full, mini review)

– Subjects rated original and degraded news story with 4 rating instruments. – We adopt normative review because it shows the highest level of accuracy in that the

result shows similar news values as those from experts.

Attribute Description

- Representative News Value Elements

Instrument Err2 | Err | sd r

- Accuracy of ratings by instrument and story quality

Attribute Description

Normative

Accuracy

Credibility

Fairness

Instrument Err | Err | sd r

Full .80 .69 .06 .285**

Descriptive 1.39 .94 .07 .234**

Normative .64 .62 .05 .218*Informativeness

Originality

Balance

Clarity

Mini 2.90 1.43 .09 .326**

Descriptive

y

Context

Diversity

Evidence

Objectivity

Transparency

Measuring Journalistic Value of Social News Contents (Cont’d)05369 well-educated people measure values of top 100 weekly best current news stories of Daum View with normative review instrument (Accuracy, Credibility, Fairness, Informativeness, Originality)

To derive importance of each element in online news, our survey asked subjects to rate them with 5-point Likert scale, and we rescaled them so that the news value score ranges from 1 to 5

D i d i h f h l li d f llDerived weights for each elements are listed as follows

Accuracy Credibility Fairness Informativeness Originality

- Weights for each rating items

0.225 0.227 0.198 0.202 0.148

Measuring Journalistic Value of Social News Contents (Cont’d)05Finally, we derive the value scores of top 100 current news of Daum Blogger News

Previous Rank URL Value89 http://v daum net/link/3918133 4 109859

Examples of survey result

89 http://v.daum.net/link/3918133 4.10985993 http://v.daum.net/link/3930134 3.99689876 http://v.daum.net/link/3931410 3.88224445 http://v.daum.net/link/3912458 3.85602922 http://v.daum.net/link/3931027 3.80779147 http://v.daum.net/link/3912861 3.77774667 http://v.daum.net/link/3926142 3.70575529 http://v.daum.net/link/3913246 3.70377787 http://v.daum.net/link/3905833 3.64687533 http://v daum net/link/3948105 3 64247733 http://v.daum.net/link/3948105 3.64247736 http://v.daum.net/link/3915024 3.63390218 http://v.daum.net/link/3902557 3.62594130 http://v.daum.net/link/3902704 3.62299659 http://v.daum.net/link/3922091 3.59138770 http://v.daum.net/link/3900538 3.58417283 http://v.daum.net/link/3913077 3.58085657 http://v.daum.net/link/3911474 3.57849943 http://v.daum.net/link/3928202 3.55226864 http://v daum net/link/3925490 3 52133364 http://v.daum.net/link/3925490 3.52133325 http://v.daum.net/link/3913131 3.519752

Finding Value Voters 06We consider news stories whose value scores are over 3.5 as valuable news and votes for the articles valid.– In 5-point Likert Sclae, score 3 is a neutral point– We assume that news scores from 3.5 to 5 are positive points.

W i l h ’ lid ti t ( b f lid t di id dWe simply measure each users’ valid voting rate (number of valid votes divided by total number of votes)

Voters who made less than 5 votes are excluded from value voter listVoters who made less than 5 votes are excluded from value voter list.– Users with less than 5 votes can have high valid voting rate by chance

We flexibly adjust the lowest limit of valid voting rate from 0.3 to 0.5y j g

Evaluation07232 well-educated people participated in the second survey and evaluate another 100 current news stories of Daum View

We compared the value scores of top 50 news contents with the votes made by alpha voters

The result is as follows– Previous votes does not show any significant correlation with news value scores.

Votes from value voters shows significant correlation with news value scores– Votes from value voters shows significant correlation with news value scores – Selected with the valid voting rate over 0.4, alpha voters’ votes show significant

correlation with news value scores 70

# voters News Value Score

40

50

60# voters News Value Score

PreviousVotes

N/A Pearson Correlation -.063

Sig. (2-tailed) .665

VVR > 273 Pearson Correlation .288*

Vot

es

10

20

300.5 Sig. (2-tailed) .043

VVR > 0.4

620 Pearson Correlation .326*

Sig. (2-tailed) .021

VVR > 914 Pearson Correlation .287*

V

0

2 2.5 3 3.5 4

0.3 Sig. (2-tailed) .043

News Value Score

Works in Progress…08This paper is on the initial stage of deriving value voters and adapting the votes to news articlesWe will conduct researches concerning…

– Diversifying weights on the valuable stories(e g based on news values of articles or excluding malicious votes)(e.g. based on news values of articles or excluding malicious votes)

– Finding correlation between news values and news stories’ attributes.(e.g. correlation between news values of articles and number of trackback or comments)( g )

– Developing a news ranking model based on news values

Calculating

value

CalculatingVoters’

Reputation

Seed Articles Valuable News

valueVoters

Ranking System based on Alpha Voters Votes

Thank YouThank You