2
Some simplifications, again
● Is good reputation a good predictor of cooperation?● Why noise = bad reputation?● Not only prevent meta-level cheating
– To what extent is cheating harmful?– To what extent Online Reputation Systems differ from
reputation in natural societies, and about what?– What is the effect of meta-level cheating in natural
societies? Is it prevented and how?– Are there just two parts (buyer and seller) in the general
game of reputation?
3
Image vs. Reputation
• Both social evaluations concern other agents' (targets) attitudes toward socially desirable behavior, and may be shared by a multitude of agents.
• Image is an evaluative belief– it tells that the target is “good” when it displays behavior b, and that it is
“bad” in the opposite case.– Most software implementation describe image only
• Reputation is instead a shared voice, i.e. a belief about others saying that a given target enjoys or suffers from a shared image.– Reputation is true when it’s actually spread, not when it is accurate
4
Image & Reputation
●image :
global evaluation owned by an agent about a target's property in relation with
an agent's (or others') goal , e.g. a social norm (Miceli & Castelfranchi 2000)
●reputation :
● process and effect of the transmission of (a sub-set of) originally image-
based evalutation(s) (or forged ones)
● meta-belief, i.e. a belief about others' beliefs (either personal:”agent x
says...”, or impersonal “it is said that...”)
● emergent from singol agents' communication (from micro to meso level)
● immergent as an individual property (from meso to micro level)
5
• mental decisions based upon them can be analyzed:– epistemic– pragmatic-strategic– memetic
• Reputation does not bind the speaker to commit himself to the truth value of the evaluation conveyed but only to the existence of rumours about it
• only the acceptance of a meta-belief is required
• And, unlike ordinary deception, reputation implies– no personal commitment (by no means stating that t deserved it)– no responsibility
Image vs. Reputation
6
• Reputation co-evolved with human language and social organization as a multi-purpose social and cognitive artefact (Dunbar, 1996)– incentives cooperation and norm abiding and discourages defection and
free-riding
– allows retaliation on transgressors by cooperating at the level of information exchange
Reputation and uncertainty
7
Necessity of a cognitive approach
• What cannot be represented outside of this approach?– power and responsibility as the main factor for spreading
– correct meta-reasoning
• what will happen when I answer this question sincerely/insincerely?
– Contagion
8
Social Cognitive Phenomenon(Reputation)
Cognitive Object(Image)
Social Process(Gossip)
A Social Cognitive Model of Reputation
ElementsImage• Belief
– Target evaluation– Accepted: the evaluation is believed
true• Agent Roles:
– Evaluator (E)– Target (T)– Beneficiary (B)
• Decisions– Epistemic: (not) to accept– Strategic: how to act w.r. to target.
• Levels involved– Agent cognition
Reputation• Meta-Belief
– How others evaluate the target– Evaluation is not necessarily
accepted! • Agent Roles:
– Gossiper (G)• Decisions
– Memetic: (not) to communicate• No commitment• No responsibility.
Diffusion without condivision!
• Levels involved– Population– Agent
• Cognitive• Objective
10
Sets of Agents
4 non-empty sets of agents are defined by their functional role:
set E = evaluators
set B = beneficiaries (of the social norm)
set T = target
set M = memetic agents, gossipers
image reputation
●Usually
✔ Set E Ç Set B ≠ 0
✔ Set E < set M●.
11
Factors of Gossip
● Whether and Why– Reputation of the source
– Acceptance of evaluation
– Responsibility and accountability for effects
– Benevolence toward the beneficiary as opposed to benevolence toward the target.
• To Whom
• About whom
• How
12
Whether and Why.Responsibility
Gossip decreases with growing responsibility (perception) and accountability:● Increasing perception of effects (Willcox, 2001):
– No “chain-like” structure
– No opacity of reports to targets
– Low “distance” of targets (Barkow, 1992; Boehm, 1999)
• Increasing severity of effects:
• Increasing commitment (firsthand evaluations: G = E) (anthropological evidence)
• Increasing publicity (vs. Anonimity) of reports (Willcox, 2001)
• Small number of gossipers (Latané & Darley, 1970)
13
Whether and Why.Benevolence
● Beneficiary oriented: negativity bias (Skowronsky and Carlston, 1989)– Transmit– Transmit bad even if uncertain
● Target oriented: leniency bias– Transmit if good reputation– Do not transmit if bad.
14
To Whom
• Reciprocity: likelihood to return information
• Discretion about the source: likelihood to protect identity of source and avoid retaliation
• Usefulness: likelihood to provide relevant information
• Persuasibility: likelihood to accept reputation information. – with regard to the source,
– with regard to the information, etc.
• Co-interestedness in sharing information, etc.
15
About Whom
• Salience:– Diversity (Elias, 1985)
• Distance (Barkow, 1992):– Social hierarchy: elite aversion (Bohem, 1999; Karstedt, 1997)
– Cannot retaliate● Bounded perception: long chains, or wide networks.● Limited power
• Credibility of evaluation.
16
How
• Chain-like Vs one-to-many
• Anonymity of gossipers
• Second-hand Vs own evaluation
• Opacity Vs Accessibility of reports by targets
• Choice of recipients Vs public feedback
Operationalise Social Cognitive Factors...
● G E: – Commitment (no secondhand
report) -> responsibility● G B (and E B):
– Beneficiary-oriented benevolence -> negativity bias
● G T (and E T): – Fear of retaliation. ->
responsibility– Target-oriented benevolence ->
leniency bias
● T B:
– No opacity -> responsibility
G E G B G T B T
H Underprovision
ProvisionUnderrat.
Underprovision
OrOverrat.
Underprovision
OrOverrat.
L Provision Underprovision
Provision(Underrating)
Provision(Underrating)
18
Combined Intersections:Two (Extreme) Examples
E G G BE B
G TE T
B T
Students&
Professors L H L L
EBay H H H H
ProvisionUnderrating
Underprovisionoverrating
What is the effect of • Overrating • Underrating?
19
• Positive effect of underrating (Paolucci, 2000).
• Norm-abiders exchanging reputation of violators
Prudence rule• Cooperation is not necessarily a function of good reputation
Gossip > no gossip (unless overrating)
Suggestion from Simulation
Prudence Rule: Negativity Bias in Social Evaluation
• Prudence: common sense or consolidated knowledge?
• In impression formation (Skowronsky and Carlston, 1989), evidence of negativity bias
• Confirming experimental evidence in social evaluation (Ciacci et al., 2002):
● Pairs of stories about one character● x does bad action a &● x does good action b
● and their dual● x does good action a &● x does bad action b
• Results: negative evaluations prevail.
NP
What About Transmission?
● Polarised explanations of same event (lecturer missing) injected to two subgroups of agents:
bad good
G EL
G BH
G TL
B TL
Back To Examples
Courtesy alg. (Dellarocas, 2001)• Pass on good reputation if you can• Otherwise don’t pass
Prudence alg. (Conte and Paolucci, 2002):• Pass on bad reputation, even if uncertain• Don’t pass good one unless certain
ProvisionUnderrating
Underprovisionoverrating
E G G BE B
G BE T
B T
Students&
Professors L H L L
EBay H H H H
23
To sum up
● Different levels of description can be useful in different contexts
● Simulation can contrast the tendency to excessive simplification– GT’s “for simplicity”
● But– Data analysis, targeted simulation experiments, and field
experiments are still in the doing: the theory is waiting for falsification
24
A review of existing system
● We start now with a review of existing online systems, also called Internet User-driven Reputation Applications (IURAs) from Marmo, 2006
● Systems will be analyzed under the light of the theory presented
25
IURAs: contexts of use
Recommender systems: signaling quality about experts / professionists or brands / products
Electronic Auctions: partner selection, quality of exchanges, balancing info. asymmetry
Social Networks: social connection, status, recognizability
Discussion Forums: signaling quality of post, on topic authors, relevance
26
Mechanism taxonomy (1 of 5)
Factual
mere labels assigned top-down from system to target
examples: top reviewer, power seller, premium account
they're not user-driven reputation mechanisms
Amazon:
eBay PayPal Guru
27
Mechanism taxonomy (2 of 5)
●Rating
●evaluation as choice of user out of an options list (vote-like)
● binary versus non binary values
● Numeric (bounded, real) versus verbal values
● global versus multi-attribute parameters
output:
● Algebric sum
● Arithmetic average (weighted versus
plain)
● Percentage of ratings
● Total number of ratings
● List of all ratings
28
Mechanism taxonomy (3 of 5)
Comment
verbal or alphanumeric drawing comment. Used with voting mechanisms,
or with connection lists in social networks. Used to personalize
evaluations.(Friendster)
FLipFLop | 28/9/2006
===THANKS FOR ADD US ====
+YOUR THE MEMBER OF OUR+
++++=PRODUCTION=+++++
=WAIT FOR OUR NEXT EVENT=
+++===CERTIFIED===+++
>>>BIG RESPECT TO YOU
(Yahoo)
29
Mechanism taxonomy (4 of 5)
Connection list
users agree to include each other in their connection list. This may state
“friendship”, “partnership”, general support or acquaintance between
users. Inclusion may as well be one-sided, or even of a negative nature
(list of enemies)Isabel’s Connections (3)
BeateHaertl
Researcher
SamueleMarmo
Ph.d. student at Università di Siena
NoaRiedel
Research Associate at Jack Russell Consulting
(LinkedIn)
30
Mechanism taxonomy (5 of 5)
Messages
- generally available function of messaging between users
- they are within-domain electronic mail
- may be used in many ways, including circulating reputation.
messages have generally a completely private status (hidden
narrowcasting)
31
Mechanism functionality (1 of 3)
●Directionality●
✔ Unilateral : user releases unilateral evaluation of a target.
Expected effect: prudence rule
✔ Bilateral : users release reciprocal evaluation.
Expected effect: courtesy rule
directionality includes both synchronous and asynchronous evaluations
32
Mechanism functionality (2 of 3)
●Access●
✔ Broadcasting : user can not choose the recipient(s) of evaluation, all
evaluations are public domain.
Expected effect: underprovision
✔ Narrowcasting : user addresses evaluation, which is privately
accessed.
Expected effect: provision
33
Mechanism functionality (3 of 3)
●Dynamism✔ Single evaluation : only last evaluation from user on a specific target
is counted.
Expected effect: dynamic change of reputation
✔ Multiple evaluation : users' evaluations on a specific target are
cumulated.
Expected effect: inertial reputation
✔ Time discount of evaluations
Expected effect: dynamic change of reputation
✔ No time discount
Expected effect: inertial reputation
34
a theoretical categorization
●A theoretical categorization of IURAs can be based on
● Access (broad- versus narrowcasting)
● Transmission of personal evaluation (image) versus
personal + meta-evaluation (image and reputation)
There are four possibilities:
(a) centralized image public access, personal evaluation only
(b) distributed reputation similar to natural reputation, private access, personal
+ meta-evaluation
(c) centralized reputation public access, personal + meta-evaluation
(d) distributed image private access, personal evaluation only
35
HypothesisPostulate: roles of agents are exactly defined by implementation
Main hypothesis
E subset of B ; E ≠ T no courtesy (prudence?)
E = T = B courtesy rule
Context hypothesis
(after observation of typical implementations)
Auctions: courtesy rule, centralized image (bidirectional, M = E, broadcasting)
Social Networks: courtesy rule, distributed reputation (bidirectional, M >
E,narrowcasting)
Discussion Forums: centralized image (unilateral/bidirectional, M = E,
broadcasting)
Recommender systems: no courtesy, centralized image (unilateral, M = E,
broadcasting)
36
Method
Use of systems under analysis during year 2005
Compiling a detailed description of the system
Collecting users' evaluation sample
Methods of collecting: alphabetical user profile search, temporal, random
EXTREME SCENARIO
Under a general assumption of inaccurate reputation, we considered a
discretionary threshold of 10% for negative evaluations: we would think of a
courtesy rule situation when negatives are below this value.
We considered a broad concept of negative evaluation, which included
“neutrals”
37
Sample Details
Ebay: 100 profiles, 7379 evaluations, alphabetic
Guru: 2175 profiles (of which 70% empty), random
Amazon Auctions: 50 profiles, 21539 evaluations, alphabetic
Friendster: 100 profiles, alphabetic, random
LinkedIn: 50 profiles, random
Omidyar: 50 profiles, 59625 evaluations, alphabetic, random
Slashdot: 50 moderators, 101 evaluations, temporal
Epinions: 15 profiles, 949 evaluations, random (+ survey from P.
Massa, B. Bhattacharjee with 664824 evaluations)
RateMyTeachers: 50 profiles, 479 evaluations, alphabetic
38
Results
Omidyar-net
Amazon A.
Guru
eBay
0.00% 25.00% 50.00% 75.00% 100.00%
courtesy rule E = T = B
negatives
positives
RateMyTeachers
Epinions
Slashdot
0.00% 25.00% 50.00% 75.00% 100.00%
no courtesy (prudence) E subset of B ; E ≠ T ; T ≠ B
negatives
positives
44
IURAs & natural reputation
●Natural reputation:
● both word of mouth and personal evaluation
● provision, low commitment (mostly narrowcasting)
● negative anchorage (Ciacci 2002)
●IURAs:
● only direct evaluation (image)
● underprovision, high commitment (mostly broadcasting)
● emphasis on positive ratings, risk of system ineffectivess
45
Conclusions on the review
a. Social cognitive theory refinement of reputation concept explains
IURAs dynamics (e.g. the courtesy rule application, or underprovision)
b. Reputation tendencies are consequences of (implementation of) role
overlapping and of commitment factors (anonymity, opacity)
c. Actual IURAs are mainly centralized image ones, which limits
artefact's effectiveness
d. Actual IURAs are generic, based on a naïve concept of reputation
(bidirectional, broadcasting, image only)
46
Appendix: Examined DomainsElectronic Auctions
Qxl, Yahoo Auctions, AaandS, AuctionAddict, BuySellBid, iBidFree, Elance, Universal
Freelancer, Artelino, AuctionandBarter, Auctiondawg, Auction Warehouse, BidMonkey,
Bidalot, Bid4assets, Bidville, Buyselltrades, Cqout, Furbid, Markplaats, Amazon Auctions,
Guru.com, eBay.
Social Networks
RepCheck, Artistnow, Tribe, OpenBC, Backwash, Gorillaexchange, Visiblepath, iKarma,
Isnotwhatyouknow, RitualiUrbani, Ryze, Amicidegliamici, Wowfriends, Asiansocialnetwork,
Netrelate, Hi5, Orkut, Amityzone, Pandora, Friendster, LinkedIn, Studenti.it.
Discussion Forums
Advogato, Kuro5hin, Chiefdelphi, Invisionize, Daniweb, MajesticForum, Isohunt, ArnieAirsoft,
HTML.it, Gratitude.net, Omidyar.net, Slashdot, Cableforum.
Recommender Systems
Amazon, ExpertCentral, AllExperts, MovieLens, Cnet, Bizrate, Barnes&Noble, DontDateHim,
DatingPsycho, Truedater, HotorNot, Ciao.it, Keen, Experts-exchange, Questico.de, Epinions,
RateMyTeachers.