elizeu santos-neto, flavio figueiredo jussara almeida, miranda mowbray marcos gonçalves, matei...
TRANSCRIPT
Elizeu Santos-Neto, Flavio FigueiredoJussara Almeida, Miranda MowbrayMarcos Gonçalves, Matei Ripeanu
Assessing the Value of Contributions in Tagging Systems
The 2nd IEEE SocialCom/SIN -- August 2010
Decentralized, Collaborative & Non-proprietary [1]
2[1] Y. Benkler. “The Wealth of Networks”. Yale University Press (2006).
Commons-based Peer Production Systems
Offline Online
Car poolingInformation
SharingResource Sharing
Volunteer Firefighters
BitTorrent
SETI@Home
OurGrid.org
Wikis
Tagging
Q&A Portals
Online Peer Production Systems
3
Tagging Systems
BandwidthTimeExpertise CPU
Annotation = Tags + Items
Tags are free-form words
Items can be virtually anything URLs, photos, videos,
citation records, etc…Photos Annotations
Why is it important to study tagging systems?
Increasingly popularMillions of users [2]GBytes of content and annotations daily [2]
User-generated metadata = new opportunities To improve existing systems (e.g., social search)To create new mechanisms (e.g., reputation
systems)
Open problem: how to quantify the value of user contributions in these systems?
4[2] R. Ramakrishnan, A. Tomkins. “Toward a PeopleWeb”. IEEE Computer 40(8): 63-72 (2007)
5
To define a method that quantifies the value of users’ contribution in tagging systems that is
accurate, feasible and robust.
What is the tolerance to
malicious users?
What is the computational complexity?
How close is it to the true value of user contribution?
Long Term Goal
Problem formalization
A solution framework
A method to quantify the value of tagsEntropy-based metricAlgorithms to compute the metric
6
Where Are We? (Contributions to Date)
Solution FrameworkIntroduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
Value of User Contributions inTagging Systems
Contribution = Tags + Items
Value of tags Context: navigation/search Intuition: value ≈ improvement on navigation/search
Value of an itemContext: item usage Intuition: value ≈ usefulness to a user
8
Content Annotations
9
Info Seeker
Relevant Item Set Finder
Contribution Value
Item Usage Monitor
Tag Value Calculator
Info Producer
Item Value Calculator
Tag Value Aggregator
Item Value Aggregator
Information Needs
Past activity
Tags
Items
Items
UsageValues
Values
The Value of User ContributionInformation flow
Value of TagsIntroduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
Intuition: tags are valuable if they narrow the scope of navigation, while retrieving relevant items.
11
Value of Tags
Items in the system
Relevant items to an info seeker
Items retrieved by a set of tags
Value of tags is proportional to this
intersection
Tags published by an info producer
Evaluation CriteriaIntroduct
ion&
Motivation
Solution Framew
ork
Value of Tags
Evaluation
Conclusions & Future Work
Feasibility: is the method efficient?
Accuracy: is the estimation close to the real value?
Robustness: can users boost their contributions maliciously?
13
Evaluation Criteria
14
Preliminary ResultsFeasibility – Part 1
Only 4% of users have more than 100 unique tag
assignments.
15
Preliminary ResultsFeasibility – Part 2
80% of users have NOT produced tags/items in 1
moth or more.
Conclusions & Future Work
Introduction& Motivation
Solution Framework
Value of Tags Evaluation
Conclusions &
Future Work
ConclusionsAssessing the value of user contributions in
social tagging systems is a relevant and challenging problem
This work ……proposes a solution framework…provides preliminary results on feasibility
Current efforts…Evaluating techniques that estimate relevant itemsDesigning algorithms to calculate and aggregate
value17
Future WorkAlgorithms
Exploit user activity similarity Aggregation method that exploits implicit social relations
Evaluation Accuracy – does the estimated value match a ground
truth? Robustness – what about spammers?
Value of items
Build mechanisms that harness the value of contributions Spam detection Social search
18
http://www.ece.ubc.ca/~elizeus
Comments? Questions?
Assessing the Value of Contributions in Tagging Systems