ResearchResearch

Virtual CurrencyCase Study: Prediction Markets

David M. PennockYahoo! Research

ResearchResearch

Virtual Currency

• What actions can be motivated withvirtual currency?

• To what degree of competence?• Case study: Prediction markets• Epilogue: Designing a better virtual

currency

ResearchResearch

ResearchResearch

ResearchResearch

worse... i would not payit would make thingsworse...I would not pay

Not many people wouldbuy it... less answers...

I would die immediately.Nobody will pay...

Keep... the Internet free

ResearchResearch

worse... i would not payit would make thingsworse...I would not pay

Not many people wouldbuy it... less answers...

I would die immediately.Nobody will pay...

Keep... the Internet free Better (if done right)

ResearchResearch

ResearchResearch

Things People Do “For Free”

• Provide information• Write software• Label images• Answer questions

ResearchResearch

Things People Do “For Free”

• Provide information• Write software• Label images• Answer questions

WIRED MAGAZINE: ISSUE 16.03

ResearchResearch

Things People Do “For Free”

• Provide information• Write software• Label images• Answer questions

• Provide predictions

WIRED MAGAZINE: ISSUE 16.03

ResearchResearch

Ask The Crowd• Choose what you’d like to predict:

• Let people bet on iti.e., Buy or sell share of

$1 if $0 ifI am entitled to:

Bird Flu Outbreak US 2008?(Y/N)

Bird FluUS ’08

Bird FluUS ’08

ResearchResearch

http://intrade.com

Screen capture 2008/03/17

ResearchResearch

The Wisdom of CrowdsBacked in dollars• What you can say/learn

% chance that• Hillary wins• GOP wins Texas• YHOO stock > 30• Duke wins tourney• Oil prices fall• Heat index rises• Hurricane hits Florida• Rains at place/time

• Where

• IEM, Intrade.com• Intrade.com• Stock options market• Las Vegas, Betfair• Futures market• Weather derivatives• Insurance company• Weatherbill.com

ResearchResearch

The Wisdom of CrowdsWith Money Without

ResearchResearch

The Widsom of CrowdsBacked in “Points”• HSX.com• Newsfutures.com• InklingMarkets.com• Foresight Exchange• CasualObserver.net• FTPredict.com• Yahoo!/O’Reilly Tech Buzz• ProTrade.com• StorageMarkets.com• TheSimExchange.com• TheWSX.com• Alexadex, Celebdaq, Cenimar, BetBubble, Betocracy, CrowdIQ,

MediaMammon,Owise, PublicGyan, RIMDEX, Smarkets, Trendio,TwoCrowds

• http://www.chrisfmasse.com/3/3/markets/#Play-Money_Prediction_Markets

ResearchResearch

The Wisdom of CrowdsWith Money WithoutIEM: 237 Candidates HSX: 489 Movies

1 2 5 10 20 50 100

estimate

1

2

5

10

20

50

100

actual

ResearchResearch

The Wisdom of CrowdsWith Money Without

ResearchResearch

Real markets vs. market gamesHSX FX, F1P6

probabilisticforecasts

forecast source avg log scoreF1P6 linear scoring -1.84F1P6 F1-style scoring -1.82betting odds -1.86F1P6 flat scoring -2.03F1P6 winner scoring -2.32

ResearchResearch

Does money matter?Play vs real, head to headExperiment• 2003 NFL Season• ProbabilitySports.com

Online football forecastingcompetition

• Contestants assessprobabilities for each game

• Quadratic scoring rule• ~2,000 “experts”, plus:• NewsFutures (play $)• Tradesports (real $)

• Used “last trade” prices

Results:• Play money and real

money performedsimilarly• 6th and 8th respectively

• Markets beat most of the~2,000 contestants• Average of experts

came 39th (caveat)

Electronic Markets, Emile Servan-Schreiber, Justin Wolfers, DavidPennock and Brian Galebach

ResearchResearch

0

25

50

75

100

Tra

deS

po

rts

Pri

ces

0 20 40 60 80 100

NewsFutures Prices

Fitted Value: Linear regression

45 degree line

n=416 over 208 NFL games.Correlation between TradeSports and NewsFutures prices = 0.97

Prices: TradeSports and NewsFutures

Prediction Performance of Markets

Relative to Individual Experts

020406080100120140160180200220240260280300

1 2 3 4 5 6 7 8 9 10 11 12 13 14

Week into the NFL season

Ra

nk NewsFutures

Tradesports

0

10

20

30

40

50

60

70

80

90

100

Ob

serv

ed F

req

uen

cy o

f V

icto

ry

0 10 20 30 40 50 60 70 80 90 100

Trading Price Prior to Game

TradeSports: Correlation=0.96

NewsFutures: Correlation=0.94

Data are grouped so that prices are rounded to the nearest ten percentage points; n=416 teams in 208 games

Market Forecast Winning Probability and Actual Winning Probability

Prediction Accuracy

ResearchResearch

Does money matter?Play vs real, head to head

Probability-

Football Avg

TradeSports

(real-money)

NewsFutures

(play-money)

Difference

TS - NF

Mean Absolute Error

= lose_price

[lower is better]

0.443

(0.012)

0.439

(0.011)

0.436

(0.012)

0.003

(0.016)

Root Mean Squared Error

= ?Average( lose_price2 )

[lower is better]

0.476

(0.025)

0.468

(0.023)

0.467

(0.024)

0.001

(0.033)

Average Quadratic Score

= 100 - 400*( lose_price2 )

[higher is better]

9.323

(4.75)

12.410

(4.37)

12.427

(4.57)

-0.017

(6.32)

Average Logarithmic Score

= Log(win_price)

[higher (less negative) is better]

-0.649

(0.027)

-0.631

(0.024)

-0.631

(0.025)

0.000

(0.035)

Statistically:TS ~ NFNF >> AvgTS > Avg

ResearchResearch

Discussion• Are incentives for virtual currency strong enough?

• Yes (to a degree)• Conjecture: Enough to get what people already know;

not enough to motivate independent research• Reduced incentive for information discovery possibly

balanced by better interpersonal weighting• Statistical validations show HSX, FX, NF are reliable

sources for forecasts• HSX predictions >= expert predictions• Combining sources can help

ResearchResearch

A Problem w/ Virtual CurrencyPrinting Money

Alice1000

Betty1000

Carol1000

ResearchResearch

A Problem w/ Virtual CurrencyPrinting Money

Alice5000

Betty1000

Carol1000

ResearchResearch

YootlesA Social Currency

Alice0

Betty0

Carol0

ResearchResearch

YootlesA Social Currency

I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch

YootlesA Social Currency

credit: 5 credit: 10

I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch

YootlesA Social Currency

credit: 5 credit: 10

I owe you 5 I owe you 5

Alice-5

Betty0

Carol5

ResearchResearch

YootlesA Social Currency

credit: 5 credit: 10

I owe you 5 I owe you 5

Alice3995

Betty0

Carol5

ResearchResearch

YootlesA Social Currency• For tracking gratitude among friends• A yootle says “thanks, I owe you one”

ResearchResearch

New Prediction Game

ResearchResearch

• A. Ghosh, M. Mahdian, D. Reeves, D. Pennock, R. Fugger (2007).Mechanism design on trust networks, Workshop on Internet andNetwork Economics

• E. Servan-Schreiber, J. Wolfers, D.M. Pennock, B. Galebach(2004). Does money matter?, Electronic Markets

• D.M. Pennock, S. Lawrence, C.L. Giles, F.Å. Nielsen (2001). Thereal power of artificial markets. Science, 291(5506), pp. 987-988

• D.M. Pennock, S. Lawrence, F.Å. Nielsen, C. Lee Giles (2001).Extracting collective probabilistic forecasts from web games,Conference on Knowledge Discovery and Data Mining, pp. 174-183

Talk Bibliography