recommendations with neo4j

49
GraphAware TM by Michal Bachman Building a high-performance recommendation engine Recommendations with Neo4j

Upload: michal-bachman

Post on 15-Jul-2015

956 views

Category:

Software


0 download

TRANSCRIPT

Page 1: Recommendations with Neo4j

GraphAwareTM

by Michal Bachman

Building a high-performance recommendation engine

Recommendations with Neo4j

Page 2: Recommendations with Neo4j

GraphAwareTM

Quick Intro

Why Graphs?

Business and Technical Challenges

GraphAware Recommendation Engine

About this Talk

Page 3: Recommendations with Neo4j

GraphAwareTM

News you should read

Books you should buy

People you may know

People you should date

People you should market a product to

Recommendation Engines

Page 4: Recommendations with Neo4j

GraphAwareTM

Content-based (features)

Collaborative filtering (user <-> item relationships)

Recommendation Engines

Page 5: Recommendations with Neo4j

GraphAwareTM

Features as well as relationships can be naturally represented as a graph.

Good News

Page 6: Recommendations with Neo4j

GraphAwareTM

Example

IS_OF_GENRE

title: “Love Actually”

Movie

name: “Bob”

User

name: “Comedy”

Genre

RATEDrating: 5

name: “Alice”

Username:

“Romance”

Genre

title: “American Pie”

Movie

IS_OF_GENRE

IS_OF_GENRE

RATEDrating: 5

INTERESTED_IN

rating: 5RATED

Page 7: Recommendations with Neo4j

GraphAwareTM

Easy to understand

Natural to model

Flexible (schema-free)

Fast to query

Graphs (Neo4j)

Page 8: Recommendations with Neo4j

GraphAwareTM

Great for a quick PoC

Great for smaller data sets

Great for relatively simple logic

Cypher

Page 9: Recommendations with Neo4j

GraphAwareTM

Great for a quick PoC

Great for smaller data sets

Great for relatively simple logic

Cypher

Page 10: Recommendations with Neo4j

GraphAwareTM

Requirements of real-world recommendation engines are often much more complex.

The Reality

Page 11: Recommendations with Neo4j

GraphAwareTM

Imagine you’re building the ”people you may know” feature on LinkedIn.

Example

Page 12: Recommendations with Neo4j

GraphAwareTM

After a brainstorming session, your team came up with the following ways of finding people one may know:

Example

Page 13: Recommendations with Neo4j

GraphAwareTM

Common contacts

Facebook friends in common

Email / mobile contacts in common

Each others email / mobile contact

Worked for the same company

Studied at the same school

Share the same interest

Live in the same city

People you may know

Page 14: Recommendations with Neo4j

GraphAwareTM

But that’s just the beginning! Let’s go back and re-visit.

Example

Page 15: Recommendations with Neo4j

GraphAwareTM

More contacts in common = better chance of knowing each other?

Same city / school / company = does size matter? location?

What about emails / numbers that don’t represent a person?

What about people already connected?

And pending…

And rejected…

And repeatedly ignored…

People you may know

Page 16: Recommendations with Neo4j

GraphAwareTM

Finding things to recommend

Serving the most relevant recommendations

Measuring the quality of recommendations

Time to market / cost of development

Business Challenges

Page 17: Recommendations with Neo4j

GraphAwareTM

Performance (real-time!)

Simplicity

Flexibility

Technical Challenges

Page 18: Recommendations with Neo4j

GraphAwareTM

So we came up with an open-source recommendation engine skeleton that will help you solve all the challenges.

We’ve done it a few times

Page 19: Recommendations with Neo4j

GraphAwareTM

plugin to Neo4j (uses GraphAware Framework)

you have to use a JVM-language

opinionated architecture

very fast

very flexible

handles all the plumbing

GraphAware Recommendation Engine

Page 20: Recommendations with Neo4j

GraphAwareTM

One “engine” per recommendation “reason” (core logic)

Engine executes a graph traversal to find items

Engines are assembled into higher-level engines

Design Decisions

Page 21: Recommendations with Neo4j

GraphAwareTM

Example

IS_OF_GENRE

title: “Love Actually”

Movie

name: “Bob”

User

name: “Comedy”

Genre

RATEDrating: 5

name: “Alice”

Username:

“Romance”

Genre

title: “American Pie”

Movie

IS_OF_GENRE

IS_OF_GENRE

RATEDrating: 5

INTERESTED_IN

rating: 5RATED

Page 22: Recommendations with Neo4j

GraphAwareTM

One “engine” per recommendation “reason” (core logic)

Engine executes a graph traversal to find items

Engines are assembled to higher-level engines

Items discovered multiple times are more relevant

Relevance depends on how the item was discovered

Design Decisions

Page 23: Recommendations with Neo4j

GraphAwareTM

Example

IS_OF_GENRE

title: “Love Actually”

Movie

name: “Bob”

User

name: “Comedy”

Genre

RATEDrating: 5

name: “Alice”

Username:

“Romance”

Genre

title: “American Pie”

Movie

IS_OF_GENRE

IS_OF_GENRE

RATEDrating: 5

INTERESTED_IN

rating: 5RATED

Page 24: Recommendations with Neo4j

GraphAwareTM

One “engine” per recommendation “reason” (core logic)

Engine executes a graph traversal to find items

Engines are assembled to higher-level engines

Items discovered multiple times are more relevant

Relevance depends on how the item was discovered

Items that should not be recommended is a “cross-cutting” concern

Design Decisions

Page 25: Recommendations with Neo4j

GraphAwareTM

Example

IS_OF_GENRE

title: “Love Actually”

Movie

name: “Bob”

User

name: “Comedy”

Genre

RATEDrating: 5

name: “Alice”

Username:

“Romance”

Genre

title: “American Pie”

Movie

IS_OF_GENRE

IS_OF_GENRE

RATEDrating: 5

INTERESTED_IN

rating: 5RATED

Page 26: Recommendations with Neo4j

GraphAwareTM

Input -> Engine -> Recommendations

Scores and Score Transformers

Blacklists

Filters

Post-processors

Context (how many, how fast,…?)

Architecture

Page 27: Recommendations with Neo4j

GraphAwareTM

In 5 minutes, we’ll build a simple engine that recommends who you should be friends with.

Let’s Build Something

Page 28: Recommendations with Neo4j

GraphAwareTM

Model

Page 29: Recommendations with Neo4j

GraphAwareTM

(1) Discover

Page 30: Recommendations with Neo4j

GraphAwareTM

(2) Score

Page 31: Recommendations with Neo4j

GraphAwareTM

(2) Score

Page 32: Recommendations with Neo4j

GraphAwareTM

(3) Post-Process

Page 33: Recommendations with Neo4j

GraphAwareTM

(3) Post-Process

Page 34: Recommendations with Neo4j

GraphAwareTM

(4) Filter

Page 35: Recommendations with Neo4j

GraphAwareTM

(5) Assemble

Page 36: Recommendations with Neo4j

GraphAwareTM

(5) Assemble

Page 37: Recommendations with Neo4j

GraphAwareTM

(6) Test

Page 38: Recommendations with Neo4j

GraphAwareTM

Finding things to recommend

Serving the most relevant recommendations

Measuring the quality of recommendations

Time to market / cost of development

Business Challenges

Page 39: Recommendations with Neo4j

GraphAwareTM

Performance (real-time!)

Simplicity

Flexibility

Technical Challenges

Page 40: Recommendations with Neo4j

GraphAwareTM

Getting Started

Page 41: Recommendations with Neo4j

GraphAwareTM

Built-in capability to pre-compute recommendations

Other built-in base-classes

But we need your help!

https://github.com/graphaware/neo4j-reco

There’s more!

Page 42: Recommendations with Neo4j

GraphAwareTM

GraphAware Framework makes it easy to build, test, and deploy generic as well as domain-specific functionality for Neo4j.

GraphAware Framework

Page 43: Recommendations with Neo4j

GraphAwareTM

GraphUnit& RestTest

RelCount WarmUp Schema (wip)Recommendation

Engine

GraphAware Framework

ChangeFeed UUID TimeTree Algorithms NodeRank

Page 44: Recommendations with Neo4j

GraphAwareTM

Open Source (GPL)

Active

Production Ready

Github: github.com/graphaware

Our Web: graphaware.com

Maven Central

GraphAware Framework

Page 45: Recommendations with Neo4j

GraphAwareTM

Try it

Give us feedback

Contribute

Build your own modules

Get in touch for support / consultancy

GraphAware Framework

Page 46: Recommendations with Neo4j

GraphAwareTM

GraphAware Events

19Jan

Recommendation Engines in Berlin

(Meetup)

19 Jan

BEER TONIGHT!

31Jan

Recommendation Engines in Brussels

(FOSDEM)

31Jan

GraphGen in Brussels (FOSDEM)

5Feb

Recommendation Engines Webinar

5Feb

Meetup at GraphAware (build your own

Recommendation Engine)

10Feb

Neo4j Fundamentals in Manchester

17Feb

Neo4j Fundamentals in Edinburgh

Page 47: Recommendations with Neo4j
Page 48: Recommendations with Neo4j

GraphAwareTM

GraphConnect Europe 2015When:

Where:

Tickets:

Call for Papers:

Sponsors:

Thursday, 7th May, 2015 - main Conference Day

Wednesday, 6th May 2015 - Training Day

Etc venues, 155 Bishopsgate, London

(next to Liverpool Street Station)

now available on www.graphconnect.com

199$ early bird plus 100$ for training

499$ full price plus 100$ for training

open now till 29th January

all Neo4j community members, customers or

general graph enthusiasts are invited to submit their talk

open now till 29th January, email:

[email protected]

Page 49: Recommendations with Neo4j

GraphAwareTM

www.graphaware.com @graph_aware @bachmanm

Thank You!