![Page 1: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/1.jpg)
“Supertagger” Behavior in Building Folksonomies
Jared Lorince¹, Sam Zorowitz², Jaimie Murdock¹, and Peter M. Todd¹
¹Indiana University ²John Hopkins University
![Page 2: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/2.jpg)
![Page 3: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/3.jpg)
![Page 4: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/4.jpg)
![Page 5: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/5.jpg)
What’s the difference between prolific and non-‐prolific tagging behavior?
![Page 6: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/6.jpg)
1: How do prolific and non-‐prolific users differ? 2: How do the folksonomies generated by these users differ? 3: What causal factors might be driving these differences?
![Page 7: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/7.jpg)
Dataset
![Page 8: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/8.jpg)
Dataset
![Page 9: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/9.jpg)
Dataset
![Page 10: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/10.jpg)
Problem FormalizaVon
![Page 11: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/11.jpg)
Problem FormalizaVon
![Page 12: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/12.jpg)
Problem FormalizaVon
![Page 13: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/13.jpg)
Problem FormalizaVon
![Page 14: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/14.jpg)
Problem FormalizaVon
5,086 prolific taggers
516,694 non-‐ prolific taggers
![Page 15: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/15.jpg)
QuesVon 1: Are the users different?
![Page 16: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/16.jpg)
QuesVon 1: Are the users different?
Yes, but not in a surprising way.
![Page 17: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/17.jpg)
![Page 18: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/18.jpg)
QuesVon 2: Do the users tag differently?
![Page 19: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/19.jpg)
QuesVon 2: Do the users tag differently?
A: Do they tag with similar vocabularies? B: Do they tag similar content?
![Page 20: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/20.jpg)
Prolific Non-‐Prolific Total users 5,086 516,694
Total AnnotaVons 25,185,083 25,187,812 Total tags 399,552 (78.56) 797,784 (1.54)
Unique tags 231,307 629,539 Shared tags 168,245 Total Items 2,992,046 (588.29) 2,515,070 (4.87)
Unique items 1,962,523 1,485,547 Shared Items 1,029,523
![Page 21: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/21.jpg)
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 22: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/22.jpg)
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
Rank Tag (P) Tag (NP) 1 “rock” “seen live”
![Page 23: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/23.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 24: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/24.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 25: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/25.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 26: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/26.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 27: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/27.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic” 6 “female vocalists” “pop”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 28: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/28.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic” 6 “female vocalists” “pop” 7 “metal” “female vocalists”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 29: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/29.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic” 6 “female vocalists” “pop” 7 “metal” “female vocalists” 8 “jazz” “metal”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 30: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/30.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic” 6 “female vocalists” “pop” 7 “metal” “female vocalists” 8 “jazz” “metal” 9 “experimental” “alternaVve rock”
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 31: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/31.jpg)
Rank Tag (P) Tag (NP) 1 “rock” “seen live” 2 “electronic” “rock” 3 “pop” “indie” 4 “alternaVve” “alternaVve” 5 “indie” “electronic” 6 “female vocalists” “pop” 7 “metal” “female vocalists” 8 “jazz” “metal” 9 “experimental” “alternaVve rock” 10 “classic rock” “classic rock” … … …
N=225
Vocabulary Similarity: Cosine Similarity: 0.872 Rank CorrelaVon: -‐0.219
![Page 32: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/32.jpg)
Item distribuVon Similarity: Cosine Similarity: 0.768 Rank CorrelaVon: 0.216
![Page 33: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/33.jpg)
N=944
![Page 34: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/34.jpg)
Are the differences in items tagged systemaVc?
![Page 35: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/35.jpg)
Are the differences in items tagged systemaVc?
![Page 36: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/36.jpg)
QuesVon 3: What is driving these differences?
![Page 37: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/37.jpg)
QuesVon 3: What is driving these differences?
ExperVse effects? (SPEAR, Yeung, et al.)
MoVvaVonal effects?
(Categorizers and Describers, Körner, et al.)
![Page 38: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/38.jpg)
ExperVse:
SPEAR (SPamming-‐resistant ExperVse Analysis and Ranking) Yeung, et al. Computa6onal Intelligence, 27(3):458–488, 2009.
![Page 39: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/39.jpg)
MoVvaVon:
Categorizers vs. Describers Körner, et al. Proc. Hypertext 2010, pp. 157–166, 2010.
Tags Per Post Orphan RaVo Tag Resource RaVo
![Page 40: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/40.jpg)
Summary:
“Supertaggers” don’t just tag more, they tag differently.
Most notably, they disproporVonately tag content in the long tail of less popular items.
Established measures suggest they demonstrate greater
experVse and descripVve tagging habits.
![Page 41: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/41.jpg)
Coda: What do we make of all this?
Is this division of labor “good” for tagging systems?
IdenVfying “supertaggers” and differenVally interpreVng their tags could improve the quality of tagging systems.
![Page 42: “Supertagger”+Behavior+in+ BuildingFolksonomies · Rank+ +Tag+(P) +Tag+(NP)1 “rock” “seenlive” 2 “electronic”+ “rock” 3 “pop” “indie” 4 “alternave”+](https://reader034.vdocument.in/reader034/viewer/2022042613/5fa60f1014a37570ba4e6a6c/html5/thumbnails/42.jpg)
[email protected] hrps://pages.iu.edu/~jlorince