an architecture for a general purpose multi-algorithm recommender system

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

An Architecture for a General Purpose Multi-Algorithm Recommender System

José Carlos Cortizo, Francisco Manuel Carrero, Borja Monsalve

1

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Index

2

Our starting point

From specific to general

Main challenges

General architecture

API definition

Current state and future work

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Our starting point

Wipley: a social network for videogamers

Integrated our first recommender

Recommended games (CF)

Related games (content)

Gamers to follow (social)

Good performance

3

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

From specific to general

Wipley served us as a test case

We wanted to offer recommendations in other products

FlickrBabel (a multilingual multimedia search engine) -> Babxel

“Sell” recommendations to other web application developers

Use of Software as a Service (SaaS) as a way to decouple the recommender system from the specific applications

4

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

From specific to general

Several commercial services: Strands, DirectedEdge, RichRelevance, etc.

But almost no literature on how to develop an architecture for a commercial general recommender system

Needed to realize about the hardest challenges, and figure it out how to solve them

5

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Main challenges

Interoperability

Accessing sources with different data structures

Not all recommender algorithms need the same information

Configurability and ease of use

Configure the algorithms to be used and the parameters and data to use

Usability may become a problem difficult to solve

6

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Main challenges

Performance

Recommendations must be served in real time

But not everything can be precomputed

Disk usage

As we have to manage several recommenders, the data representation cannot be optimized

Scalability

Growing in users and items implies growing exponentialy the complexity of some kind of recommenders (ie CF based)

7

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

General architecture

We used an API REST to make it easier to integrate the recommender engine within any application

The backend architecture is based on several modules, making it easier to configure the system for each application domain

The REST API allows us to introduce several configuration elements as optional parameters in the requests

8

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

General architecture

9

ClientRequest

Processor

UpdateProfile

Recommendation

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

General architecture

10

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

General architecture

11

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

API definition

12

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

API definition

13

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

API definition

14

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

API definition

15

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

API definition

To make a request for a recommendation

http://webservices.brainsins.com/api/recomm/player/has/videogame/2.xml

http://webservices.brainsins.com/api/recomm/player/has/videogame/2/435.xml

The system returns a XML/JSON file containing the recommendations

16

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Integration

17

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Current state and future work

REST API solves the integration and interoperability problems

And helps with configuration

Need to work on performance and scalability

Studying to re-code the recommenders using map-reduce

Experimental platform

Developing new content based and CF based algorithms

Now we can make exhaustive studies

18

PRSAT 2010: Practical Use of Recommender Systems, Algorithms and Technologies

Questions?

19