1. Trust Evaluation throughUser Reputation andProvenance AnalysisDavide Ceolin, Paul Groth,Willem Robert van Hage, Archana Nottamkandath, Wan Fokkink

2. Outline Problem Definition Trust estimates as probability distributions Reputation-based trust Provenance-based trust Combining reputation- and provenance-trust Results Future Work 3. Our ProblemTrusting crowdsourced informationSpecifically: video annotationsWe can not tag videos automatically (complex,many layers,...), so we refer to the crowd, butcrowdsourced annotations need to beevaluated. 4. Our Case StudyWaisda, a video tagging platform.http://waisda.nlUsers score if theytag simultaneously.Tags are useful (e.g. to index video) but how totrust tags? 5. Measuring tags trustworthinessHow to measure tags trustworthiness?Lets exploit the game mechanism: #matches trustworthinessHowever, if a tag did not get any match it is notnecessarily wrong. 6. Trust as a Beta Distribution 7. Reputation-based trustA classical approach: Gather all the evidence about a user Estimate the Beta distribution (userreputation) Predict the trustworthiness of the (other)tags inserted by the user (using the Beta) 8. Our hypothesisWe believe we can go beyond the classicalreputation-based trust: Who Who + How 9. (How) Provenance-based trust How-provenance can be used as a stereotype for tags authorship May be less precise than reputation but more easily available 10. Computing provenance-based trustHow-provenance is composed by severalfeatures (timestamp, typingDuration, ...)We use Support Vector Machinesto learn the model and make ourestimates. 11. Combining reputation- andprovenance-trustIs reputation "solid" enough? (E.g. based on 20 obs.) Use Reputation!Use how-provenance-based(More fine-grained) estimate! 12. Results#{tag: Real_Trust(tag)>Threshold = Pred_Trust(tag) > Threshold} Accuracy = #tags 13. DiscussionWho =* HowWho * Who + HowHow * Who + How(At least in our case study)=* statistically not different* statistically different (less) 14. Future Work We plan to apply this approach to other casestudies as well A generic platform for producing trustassessments based on annotatedprovenance graphs is under development We also plan to merge provenance-basedestimates with semantic similarity-basedones in order to provide a tool for a widerange of applications. 15. Thank you!Questions? d.ceolin@vu.nl