1
Reputation Systems
Andrew A. ChienMay 21, 2004
UCSD CSE225
CSE225 Lecture #15
Administrivia
• Project Presentations, 6/11, 2-4pm, location TBD• Next week:
» 5/26 meet at regular time» 5/28 meet ½ hour early (430pm)
2
CSE225 Lecture #15
Last Time
• Grids are varied in structure and relationship» Internet Desktop Grid, Enterprise Grid» Compute Resource Oriented, Data Resource Oriented
• Virtual Organizations are a Security Policy Overlay» Notion of Virtual Organization is diverse» GSI can be used to build VO’s based on individual identity
– Challenges in administration: local/VO, management, lack of group identity
» Community Authorization Service supports notion of group– Anonymous use, Group management– Lesser Audit and no fine-grained control
CSE225 Lecture #15
Today’s Readings
• Resnick, Paul, Zeckhauser, Richard, Friedman, Eric, and Kuwabara, Ko. Reputation Systems. Communications of the ACM, 43(12), December 2000, pages 45-48.
• Resnick, Paul and Richard Zeckhauser. Trust Among Strangers in Internet Transactions: Empirical Analysis of eBay's Reputation System. The Economics of the Internet and E-Commerce. Michael R. Baye, editor. Volume 11 of Advances in Applied Microeconomics. Amsterdam, Elsevier Science. 2002.
3
Reputation Systems
Courtesy: Paul ResnickUniv. of Michigan
School of [email protected]
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Learning Objectives
Understand– What a reputation system is– Theory about when and why it should work– Open research questions
4
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Outline
What is a reputation system?Theory: when/why they should workEmpirical resultsDesign space
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Definition
A Reputation System…– Collects– Distributes– Aggregates
…information about behavior
5
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Examples
BBBBizrateeBayExpertise sites– Epinions “top reviewers”– Slashdot karma system
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
6
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
7
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
http://cgi2.ebay.com/aw-cgi/eBayISAPI.dll?ViewFeedback&userid=the_sharper_image
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
8
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
9
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Why Reputation Systems
Interacting with strangersSellers (Exchange Partners) Vary– Skill– Effort– Ethics
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Other Trust-Inducing Mechanisms in E-commerce
InsuranceEscrowFraud Prosecution
10
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
How Reputation Systems Should Work
Information– Past interactions inform abilities and
dispositionIncentive– Reciprocity or retaliation: future behavior– “Shadow of the future”
Self-selection– Low quality (ratings) -> less return – High quality -> greater return
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Basic Requirements
Entities are long-lived– Anonymity– Name changes– Name trades
Feedback about interactions is captured and distributed– Non-participation (effort)– Reluctance to record negative– Honesty?
Past feedback guides buyer decisions
11
Interaction Type
ID changes
Anonymous every xaction
Pseudonyms
at will
Identified never
Anonymity Analysis
Interaction type
ID changes
Reputation Sharing
Trust/ cooperation
Anonymous every xaction
Pseudonyms
at will
1L Pseudonyms each arena
Identified never
12
Interaction type
ID changes
Reputation Sharing
Trust/ cooperation
Anonymous every xaction
none none
Pseudonyms
at will + only + only
1L Pseudonyms each arena
Identified never + and – + and 0
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
1L Pseudonyms
Third-party issues pseudonyms– No cost– Not replaceable– Reveal name to third party– Don’t reveal mapping of name to
pseudonym
13
Interaction type
ID changes
Reputation Sharing
Trust/ cooperation
Anonymous every xaction
none none
Pseudonyms
at will + only + only
1L Pseudonyms
each arena
+ and – within arena
+ and 0 within arena
Identified never + and – + and 0
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Empirical Results: eBay
Feedback is providedIt’s almost all positiveReputations are informativeReputation benefits– Effect on probability of sale– Effect on price
14
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Provision of Feedback
Negatives: paid but did not receive; seller cancelled; not as advertised; communication Neutrals: slow shipping, not as advertised, communication
Buyer of Seller Seller of Buyer Frequency Percent Frequency Percent
negative 111 0.3 353 1.0neutral 62 0.2 60 0.2positive 18,569 51.2 21,560 59.5none 17,491 48.3 14,260 39.4Total 36,233 36,233
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Feedback Profiles of Buyers and Sellers
Group N (Sellers)
Percent neutral and negative
(Sellers)
N (Buyers)
Percent neutral and negative
(Buyers) 0-9 positive 4,018 2.83% 13,306 1.99%
10-49 positive 3,932 1.25% 7,366 1.09% 50-199 positive 3,728 0.95% 3,678 0.76%
200-999 positive 1,895 0.79% 738 0.60% 1000+ 122 1.18% 15 0.92%
All 13,695 0.93% 25,103 0.83%
15
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Predicting Problematic Transactions
Logistic Regression
f(0,0) = 1.91% f(100,0) = .18% f(100,3) = .53%
N = 36233Beginning Block Number 0. Initial Log Likelihood Function-2 Log Likelihood 2194.3468-2 Log Likelihood 2075.420
Dependent Variable.. NEGNEUT---------------------- Variables in the Equation -----------------------
Variable B S.E. Wald df Sig R Exp(B)
LNNPOS .7712 .1179 42.7907 1 .0000 .1363 2.1624LNPOS -.5137 .0475 116.8293 1 .0000 -.2288 .5983Constant -3.9399 .1291 931.3828 1 .0000
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Predictive Value
1-specificity (% of unproblematic transactions rejected)
Sensitivity (% of problematic transactions rejected)
Cutoff predicted probability
% of accepted transactions that are problematic
75% 94.2% .20% .11% 50% 81.5% .31% .18% 25% 57.2% .54% .27% 10% 32.4% 1.09% .36% 0% 0% Accept all .48%
Predicting Problem Transactions
1 - Specificity
1.00.75.50.250.00
Sens
itivi
ty
1.00
.75
.50
.25
0.00
16
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Interesting Dynamics
High Courtesy Equilibrium: SymmetrySeller Driven
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Interesting Dynamics
Skunk at the party, Bad applesPaying Initiation dues (buildup; type)– Lower prob of sales; lower prices
Stoning Bad Behavior– Piling on after see others had a problem– Not clear if this happens
17
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Design Space
Rating scalesAggregation of ratingsWho rates?Incentives for ratersIdentification/Anonymity– Exchange partners– Evaluation providers
SCHOOL OF INFORMATION UNIVERSITY OF MICHIGANsi.umich.edu
Summary
Reputation Ssystems inform, incent, selectOpportunity for RS: interactions with strangersDesign space– Scales, aggregation, raters, incentives,
anonymity
18
CSE225 Lecture #15
Discussion
• What do reputation systems have to do with Grids?» Open system for establishing/manage pairwise trust» Doesn’t require a trusted oracle in control» Defaults can tie policy to reputation (effectively)» Can you use reputations predict resource properties?
• What capabilities do RS provide? Can this transaction view be translated to control resource use/transactions?
» Perhaps yes. Can do selection based on reputation» Combine reputation and price =>as basis for allocation
• Can data access be controlled based on reputation?» Scoping and contexts» Not typically, but maybe in some contexts.» Reputation could be one attribute, but there are issues of policy, identity
and other characteristics• Mechanism for generating peer trust, but doesn’t address the whole
security problem
CSE225 Lecture #15
Discussion II
• Do they provide the same capabilities as GSI?» No, GSI is mechanisms to implement policy based on some
predecided trust
• Do these capabilities fit together?» Basis of trust» Mechanisms