Recommender Systems Challenge

ACM RecSys 2012Dublin

September 13 2012

Organizers

● Nikos Manouselis- Agro-Know & ARIADNE Foundation

● Alan Said- PhD student @ TU Berlin -@alansaid

● Domonkos Tikk- CEO @ Gravity R&D -@domonkostikk

● Benjamin KillePhD tudent @ TU Berlin -@bennykille

● Jannis Hermanns- Moviepilot -@jannis

● Katrien Verbert- KULeuven

● Hendrik Drachsler- Open University - The Netherlands

● Kris Jack- Mendeley

The Challenge - 2 tracks

CAMRa● Previously: 2010 & 2011● Finding users to recom-

mend a movie to● moviepilot.com data● live evaluation● camrachallenge.com

● ~60 participants● 1 submitted paper

ScienceRec● First time● Novel algorithms,

visualisations, services for paper recommendation

● Mendeley data (3 datasets)● Several requested data● 4 submitted papers

What went wrong?

● Initial results indicate that RecSys Challenge was not successful ○ measurable result: 5 submissions, 2 accepted papers + 1 accepted

presentation/talk

● Several issues encountered○ “we downloaded the dataset but could not run extensive simulations

because it was difficult to process”○ “we wanted to combine the dataset with live data from the platform

but we didn’t have enough user info”○ “we used different datasets than the ones suggested because they

were easier to access/use”○ too diverse tracks

○ unawareness / difficulty in spreading information about the challenge

What went right?

Why are we all here?

● finding datasets to experiment with (especially from live, industrial systems) instead of working with the old "favorites"

● learning how existing algorithms can be reused (extended, adapted, evolved) instead of coding from scratch

● finding how our algorithm (unique, novel, amazing, the best) can be contributed to the community conceiving designing/deploying a great recommendation service

● make a business case out of our algorithm/service● (become rich/famous/...)

The real challenge

How to make such contests work, being also useful for...

● ...the data publisher [insight into what can/cannot be done with their data]

● ...the research community [insight into new algorithms, approaches, services + contributions to existing frameworks/libraries]

● ...the deployed platform [insight into new services that could work better / be more useful]

● ...everyone [create publicity/awareness]

Our Workshop

● Follows a simple structure similar to how you would participate in a challenge○ Available Data Sets○ Existing Algorithms/Frameworks○ New Investigated Methods○ Prototyped and/or Deployed Services

Program09:00–09:15 Welcome & intro09:15–10:00 Working with Data

● The MovieLens dataset – Michael Ekstrand● Mendeley’s data and perspective on data

challenges – Kris Jack● Processing Rating Datasets for Recommender

Systems’ Research: Preliminary Experience from two Case Studies - Giannis Stoitsis, George Kyrgiazos, Georgios Chinis, Elina Megalou

10:00–10:30: Algorithms & Experiments● Usage-based vs. Citation-based Methods for

Recommending Scholarly Research Articles - André Vellino

● Cross-Database Recommendation Using a Topical Space - Atsuhiro Takasu, Takeshi Sagara, Akiko Aizawa

11:00–12:30: Real Use● From a toolkit of recommendation algorithms

into a real business: the Gravity R&D experience – Domonkos Tikk

● Selecting algorithms from the plista contest to deliver plista’s ads and editorial content on premium publisher’s websites - Torben Brodt

● Mendeley Suggest: engineering a personalised article recommender system - Kris Jack

12:30–14:30: Lunch break

14:30–15:30: Frameworks, Libs & APIs● Hands-on Recommender System Experiments

with MyMediaLite - Zeno Gantner● Using Apache’s Mahout and Contributing to it-

Sebastian Schelter ● Flexible Recommender Experiments with Lenskit

- Michael Ekstrand15:30–17:30: Hands-on work