trusting wikipedia

Trusting WikipediaSu-Laine Yeo Brodsky

December 2014

Today’s Talk

• How does

Wikipedia vet its

own content?

• Questions are

welcome

anytime, or at the

end

Image:Robert Lawton, distributed under a CC BY-SA 2.5 license

The Encyclopedia Anyone

Can Edit

• Hosted by the non-profit

Wikimedia Foundation,

in 250+ languages

• Almost anyone is

encouraged to edit

almost all text and

media on the site

• ~10% of edits are

reverted (rejected)

Image: xkcd.com, distributed under a CC BY-NC 2.5 license

What Happens When You

Edit This Page?

Right Away

Exceptions: Edits You Can’t

Save

• URLs on Wikipedia’s spam blacklist

• Use of certain images, except in particular articles

• Certain types of blatant vandalism

Page History

User Contributions Page

Accounts used only

for vandalism are

easy to spot

Summary: Wiki Mechanics

• All edits are tracked, and

summarized in the article

History page

• All edits can be tracked to an

IP address or username

• Edits can be quickly

reverted by another editor

People and Processes

Roles at Wikipedia

• Editor: Anyone who writes or changes articles, or

uploads images

• Administrator: 1,386 elected administrators have

additional tools:

• Can delete/undelete articles or past versions of

articles

• Can block editors or other administrators

• Can “protect” an article to prevent editing

Reality: For the most part,

everybody has equal privileges

when it comes to controlling article

content

Assumption: Moderators decide

what changes to an article will be

accepted

How Wikipedia Monitors

Edits: A 2-Stage Process

Stage 1: Recent Changes

Page

30 seconds worth

of edits to all

English Wikipedia

articles

Link to “diff”

showing what text

was changed in this

edit

Link to list of editor’s

other edits

Heading of the article

section that was

changed

Editor’s

explanation for

the edit

Red indicates

editor is new

Recent Changes Patrol

• Volunteer patrollers and

robots monitor all

changes to the site

• Quickly revert blatantly

inappropriate changes

Stage 2: Watchlists

Changes to articles

I’ve chosen to

watch, grouped by

date

Watchlists

• Each user has a watchlist of

articles they are interested in

• The Watchlist page shows

recent changes in those

articles

• Primary tool for fact-

checking

Will an Edit Stick?

• Usually decided by the editors who watch the

page

• If no consensus, the editors who watch the page

ask the wider Wikipedia community for input and

mediation

Reality: Wikipedia articles summarize

what reliable published sources say

on the subject…

…as determined by the consensus of

editors who show up

Assumption: Wikipedia articles

summarize what a majority of its

editors on the subject believe

How Editors Scrutinize

Changes

Consider:

• My own knowledge

• Sources cited?

• Who is the editor?

• What does the source say?

Userpage of a Wikipedia

editor

Image: Userpage of Kim Dent-Brown on Wikipedia

• Improve instead of reverting

Dispute Resolution

• Discuss the issue on

the article Talk Page

• Ask the wider

community for input

• Problematic editors

can be blocked by

administrators

What Can Go Wrong?• Patrollers/watchlisters

not paying attention

• Watchlisters lacking the

necessary expertise

• Persistent, highly

motivated agenda-

pushers

Perspective: Core Facts

In an article with a reasonable number of watchers, the core

facts of the article tend to be reliably monitored.

“West Bengal

is a state in

the eastern

region of

India.”

(no source given)

Wikipedians: Question 1Wikipedia currently has 31,000 active, registered

editors. According to surveys of Wikipedia editors,

what percentage are male?

A. 85%

B. 60%

C. 45%

Wikipedians: Question 2Which of the following statements is false?

A. According to editor surveys, more than 10% of

Wikipedia editors are under 18 years old

B. Anyone can track down the IP address that

each edit comes from

C. According to editor surveys, about half of

Wikipedia editors have some post-secondary

education

Systemic Bias• Demographic bias:

• Technical

• Male

• Childless

• First world

• Recentism: Recent

news coverage &

online sources

Summary: People and

Processes

• All editors are equal, in

theory

• Articles should draw only

from reliable published

sources

• 2-stage review process

means that subtle problems

last longer than obvious

ones

Image: Adam Novak, distributed under a CC BY-SA 3.0 license

Wikimania Conference 2013

Summary (cont’d)

• In an article with a

reasonable number of

watchers, the core facts

tend to be relatively

reliable

• Wikipedia has systemic

biases stemming from

community demographics

Trends in Trust

Image: Mariuszjbie, distributed under a CC BY-SA 3.0 license

How Many Watchers?

Article# of

Watchers

Barack Obama 2,592

Angela Merkel 225

Influenza 228

Cephalexin 46

Many articles on living people <5

Four Trends

1. More complaints from the subjects of Wikipedia

articles, even articles with very low readership

2. Higher expectation to cite sources when adding

new content

3. More complete content -> Ratio of good to bad

edits changes

4. More sophisticated PR from organizations

Problematic Edits in Medical

Articles

Source: https://en.wikipedia.org/wiki/Wikipedia:Conflicts_of_interest_%28medicine%29

Looking Out for Bias• Would any personal,

corporate, or ideological

interest benefit from a

certain article slant?

• Does the article Talk page

and/or history show any

concerns with conflict-of-

interest editing or disputes

over neutrality?

Buy!

• Libel

• Hoax

• Advertising

• Pseudoscience

• Propaganda

Don’t bite the

newbies

Reject questionable

edits

Many edits by new

editors are imperfect

and do not cite

sources

A Reversible Trend?

• Community needs to

grow to:

• Maintain quality and

update facts

• Reduce systemic bias

• Requires Wikipedia

community to be skilled

in both skepticism and

openness

Enjoy Wikipedia

• Reliability is variable

• Project reflects huge

volunteer effort

• Future depends on a

strong editor

community

Image: Takeaway, distributed under a CC BY-SA 3.0 license

Thank You!Any questions?

Su-Laine Yeo Brodsky

syeo@interelement.ca

www.interelement.ca

@sulaineyeo