peer ratings in massive online social networks
TRANSCRIPT
![Page 1: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/1.jpg)
Peer Ratings in Massive Online Social Networks
Dmitry Zinoviev
Suffolk University
Boston
Sunbelt-2012
![Page 2: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/2.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 2
To Like or not to Like?
● Like/dislike ratings provide instant peer feedback in massive online social networks (MOSNs)
● Objects subject to ratings: posts; photographs; comments● Questions:
➢ How many stops should rating scales have?
➢ How do MOSN users perceive the ratings?
➢ How do MOSN users react to the ratings?
![Page 3: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/3.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 3
Rating Types
● Single-valued (Facebook ”Like”, Google+ ”+1”); do not allow negative attitudes
● Binary a.k.a. ”Thumbs Up/Down” (Pandora, Yahoo Answers); do not allow fuzzy answers
● Multivalued (Yahoo, Amazon); may be hard to use if there are many stops; these ratings are of particular interest to us
![Page 4: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/4.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 4
“Odnoklassniki.Ru”—Our Testing Range
● Largest Russian-language social network (100 mln users, 31 mln daily visits)
● Most users have more than one personal picture and reasonably credible demographics (age/gender)
● 6-way ratings (“1” through “5+”*) applicable to personal pictures● Anyone can leave comments to any personal picture of to the profile
in general● The system records and displays all profile visits (and visitors)
*The grade of “5+” requires a symbolic payment from the grader
![Page 5: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/5.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 5
A Profile, as Seen by a VisitorMy PageMy Page
Personal PhotographsPersonal Photographs
DemographicsDemographics
FriendsFriends
Profile PictureProfile Picture
![Page 6: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/6.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 6
Picture Rating Interface
PicturePicture
Picture CommentsPicture Comments
DemographicsDemographics
This is a ”5+” pictureThis is a ”5+” picture
Your ratingYour rating
DescriptionDescription
![Page 7: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/7.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 7
Method
● Select a random avatar Q from the three avatars on the right (30/Female, 40/Male, 30/Male)
● Select a random currently logged user and record the user's age A (≥18) and gender G
● Select a random grade Rin between “1” and “5+” and post it
● Record the user's response:➢ did the user visited the test profile? (V)
➢ did the user post a comment? (C)
➢ what is the response grade Rout
?
● Apologize to the user for a low grade. Posted grades and comments can be easily removed without any traces
![Page 8: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/8.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 8
Descriptive Statistics
● 10,799 observations● 3,600 observations per avatar● 1,800 observations per grade● Distribution by gender:
➢ 54% female
➢ 46% male
● Mean age 34
Russia in 2009
![Page 9: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/9.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 9
Empirical Parametrized Mapping
{Rout
, V, C} = f (Rin; Q, A, G )
Free variable
Parameters
![Page 10: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/10.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 10
Response Details with Trend Lines
Responders, by kind and by age:
![Page 11: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/11.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 11
Response Statistics
All RespondersOf them:
Visitors Graders
RateM 47.0% 39.7% 19.7% 4.8%
F 42.4% 31.7% 19.5% 5.0%M -0.1% 0.0% -0.4% -0.1%
F 0.5% 0.6% -0.1% 0.1%
Commenters
Age Trend (%/year)
● Visits are the most common responses, followed by grades, followed by comments
● Male subjects visit the experimenter's profile more often● Older female visitors are more active than younger● Younger male graders are more active than older● Grading/commenting rates are gender-agnostic
![Page 12: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/12.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 12
OutGrade Distribution
● The grades of “5” and “5+” are grouped (“5+” is special because of its associated price)
● Response grades have a sharp bi-modal distribution● Only grades “1” and “5” matter!
![Page 13: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/13.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 13
OutGrades vs InGrades
● Response grades are sharply distributed around “1” and “5” for any stimulus grade (the red lines show best-fit bi-exponential distributions)
● Only two stops on the scale are necessary!
![Page 14: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/14.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 14
RG=R+G
● Response Grade = Reciprocity + Generosity➢ Reciprocity: Social norm of in-kind responses to the behavior of
others (a.k.a. “eye for an eye”)
Responding with the same grade➢ Generosity: Habit of giving freely without expecting anything in
return
Responding with a better of worse grade (positive vs negative generosity)
● How popular are these mechanisms?
![Page 15: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/15.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 15
Reciprocity (1)
● Define reciprocity as the “fraction of reciprocally equal grades out of all grades”:
● Calculate reciprocity for each avatar and for each age-gender group● Calculate linear best-fit estimate● Notation:
➢ Blue lines for male subjects, red lines for female subjects
➢ Solid lines for the 40/M avatar, dashed lines for 30/M, dotted lines for 30/F
Rec=N Ri n=Rout
N
![Page 16: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/16.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 16
Reciprocity (2)
● Older subjects are less reciprocating
● Male subjects are on average less reciprocating
● Younger avatars generate more reciprocity from the subjects of the same gender
![Page 17: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/17.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 17
Generosity (1)
● Define generosity as the average value of the difference between the stimulus grade and the response grade:
● It can be positive and negative● Calculate generosity for each avatar and for each age-gender group● Calculate linear best-fit estimate
Gen=∑i
Rout−Ri nN
![Page 18: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/18.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 18
Generosity (2)
● Older subjects are more generous
● Older avatar generates less generosity
● Subjects are less generous to the avatars of the same gender
● Younger avatars generate less generosity from the subjects of the same gender
![Page 19: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/19.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 19
Benevolence (1)
● Let negative comments have the value of -1, positive comments—the value of 1, and neutral comments—the value of 0. Define benevolence as the average value all comments
● Calculate benevolence for each avatar and for each age-gender group
● Calculate linear best-fit estimate
![Page 20: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/20.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 20
Benevolence (2)
● Older subjects leave better comments
● Female subjects leave better comments
● Younger avatars get worse comments from the subjects of the same gender
![Page 21: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/21.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 21
Grading/Commenting Overview (1)
Generosity G
Reciprocity R
Benevolence B
● Most behaviors are avatar-specific; however, the dependency on age is universal
![Page 22: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/22.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 22
Grading/Commenting Overview (2)
● Generosity and benevolence grow with age● Generosity and benevolence are weaker for same-gender
avatar-subject pairs● Reciprocity slowly declines with age● Reciprocity is stronger for same-gender avatar-subject pairs● All three values can be approximated using quadratic functions
![Page 23: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/23.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 23
Latent Hostility
● Age 18–22: subjects are all negative
● Age 22–36: subjects combine positive generosity and negative benevolence: they post higher grades but leave negative comments
➢ Hypothesis: Higher response grades make subjects feel good, but comments reveal true feelings
● Age 36–80 subjects are all positive
18–22
36–80
22–36
![Page 24: Peer Ratings in Massive Online Social Networks](https://reader033.vdocument.in/reader033/viewer/2022060202/559cdfd61a28ab79288b4597/html5/thumbnails/24.jpg)
SUNBELT-2012 / SUFFOLK UNIVERSITY 24
THANK YOU!