neo4j for ruby and rails

Neo4jGraph Database

for Ruby and Rails

Pablo DelgadoMadrid, Conferencia Rails 2010

Wednesday, November 10, 2010

Presentation Titel | Author | City, 11/05/2010 2

GRAPHS

Extracted from

“The Graph Traversal Programming Pattern”Rodriguez, Marko A.

Graph Systems Architect at AT&T Interactive


Undirected Graph

• VerticesAll Vertices denote the same type of node

• Edges- All edges denote the same type of relationship- All edges denote symmetric relationship


Directed Graph

• VerticesAll Vertices denote the same type of node

• Edges- All edges denote the same type of relationship- All edges denote asymmetric relationship


Single-Relational Graph

Definition• Without a way to differentiate types of edges, all edges have the same meaning, hence single-relational graph

Limitations• Are only able to represent a single kind of vertex (user, city, etc)• Are only able to represent a single kind of edge (follows, citation, friends)


Multi-Relational Graph

Definition• With the labeling of the edges we obtain the possibility to express different relationships (follows, likes, include,etc)

Gain• Edges can now have different meanings and vertex can have different types.• follows: user -> user• likes: user -> webpage• cites: article -> paper

likes PageUser


Increased expressivity with multi-relational graphs

Multi-Relational Graph

likes

likes

likes

likes

likes

likes

follows follows

follows


Property Graph

Definition• A property graph extends a multi-relational graphallowing both, vertices and edges to maintain a key/value property map.• Properties are useful to express non-relational data • This allows adding more meaning to an edge

“John Smith liked http://twitter.com on 2010/10/01”

likes PageUser

name = John Smith url = http://twiiter.com

date = 2010/10/01


http://yumit.com

http://yumit.com

http://twiiter.com

http://twiiter.com

Increased meaning on edges and nodes in a property graphJohn liked http://twittpic.com on 2010/06/06Xurde liked http://yumit.com on 2010/03/03Ivan liked http://uwi.sh on 2010/07/07

Property Graph

http://twittpic.com

John

Paul

2010/06/06

http://facebook.com

http://google.com

http://yumit.com

Xurde

http://uwi.sh/

Ivan

http://panoramio.com

2010/03/03

2007/03/01

2010/7/07

2005/04/05

2005/02/05


http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com

http://twittpic.com


GRAPH DATABASE


Graph with Relational Database

B

A

D

C

outV | inV-----+----- A | B A | C C | D D | A


Graph with JSON Database

B

A

D

C

{ A: { out : [B,C], in : [D] }, B: { in : [A] }, C: { out : [D], in : [A] }, D: { out : [A], in : [C] }}


Graph with XML Database

B

A

D

C

<graphml> <graph> <node id=A /> <node id=B /> <node id=C /> <node id=C /> <edge source=A target=B /> <edge source=A target=C /> <edge source=C target=D /> <edge source=D target=A /> </graph></graphml>


Graph Database

Definition

• Every element (i.e. vertex or edge) has a direct pointer to its adjacent element.

• No O(log2(n)) index lookup required to determine which vertex is adjacent to which other vertex.

• If the graph is connected, the graph as a whole is a single atomic data structure.


Graph Databases and Index-free Adjacency

B

A

D

C

• In a graph database, vertex A has direct references to its adjacent vertices. • Constant time cost to move from A to B and C . It is only dependent upon the number of edges from vertex A (local lookup).

The Graph (explicit)


Non-Graph Databases and Index-Based Adjacency

B

A

D

C

• In a non-graph database, an index lookup is needed to determine what is adjacent to A.• log2(n) time cost to move to B and C. It is dependent upon the total number of vertices and edges in the database (global lookup).

A B C D

[B,C] [D] [A,B][ ]

The Graph (implicit)The Index (explicit)



Neo4j


Neo4j is a Graph Database• Non-relational, transactional (ACID), embedded • Data is stored as a Graph / Network ‣Nodes and relationships with properties ‣“Property Graph” or “edge-labeled multi-digraph”• Schema free, bottom-up data model design

Neo4j is Open Source / Free Software• AGPLv3 • Commercial (“dual license”) license available

What is Neo4j?


• Traversal APIsNeo4j core traversersBlueprint Pipes (TinkerPop Productions)

• SPARQL - “SQL for linked data” - query by graph pattern matchingSELECT ?person WHERE { ?person neo4j:KNOWS ?friend . ?friend neo4j:KNOWS ?foe . ?foe neo4j:name "Larry Ellison"}Find all persons that KNOWS a friend that KNOWS someone named “Larry Ellison”.

• Gremlin - “perl for graphs” - query by traversal./outE[@label='KNOWS']/inV[@age > 30]/@nameNames of all People I know with age > 30.

Neo4j Query Languages


Data Manipulation API

GraphDatabaseService graphDb = getGraphDbInstanceSomehow();

// Create Extractor Cobb Node cobb = graphDb.createNode(); cobb.setProperty( "name", "Cobb" ); cobb.setProperty( "role", "Extractor" ); // Create Ariadne Architect Node ariadne = graphDb.createNode(); ariadne.setProperty( "name", "Ariadne" ); ariadne.setProperty( "role", "Architect" ); // Create relationship representing they know each other cobb.createRelationshipTo( ariadne, RelTypes.KNOWS ); // ... similarly add more relations here


Data Manipulation API

GraphDatabaseService graphDb = getGraphDbInstanceSomehow();Transaction tx = graphDb.beginTx();try { // Create Extractor Cobb Node cobb = graphDb.createNode(); cobb.setProperty( "name", "Cobb" ); cobb.setProperty( "role", "Extractor" ); // Create Ariadne Architect Node ariadne = graphDb.createNode(); ariadne.setProperty( "name", "Ariadne" ); ariadne.setProperty( "role", "Architect" ); // Create relationship representing they know each other cobb.createRelationshipTo( ariadne, RelTypes.KNOWS ); // ... similarly add more relations here tx.success();} finally { tx.finsh();}



Neo4j

with

Ruby and Rails


• neo4j.rb - by Andreas Rongehttps://github.com/andreasronge/neo4jA Graph Database for JRuby. It uses the java lib neo4j as storage and Lucene for quering/indexing

• neo4jr-simple by Matthew Deitershttp://github.com/mdeiters/neo4jr-simpleIts a JRuby wrapper for Neo4j

Neo4j for Ruby and Rails


https://github.com/andreasronge/neo4j

https://github.com/andreasronge/neo4j

http://github.com/mdeiters/neo4jr-simple

http://github.com/mdeiters/neo4jr-simple

Inception

since

= 6

year

s

activity = undisclosed

Knows relationship


Inception

Hire relationship


Inception

In Love relationship


Inception

since

= 6

year

s

activity = undisclosedKnows relationshipHire relationshipIn Love relationship


Inception with Neo4j

Neo4j::Transaction.run do cobb = Node.new :name => "Cobb", :role => "Extractor" ariadne = Node.new :name => "Ariadne", :role => "Architect", location => "Paris" saito = Node.new :name => "Saito", :role => "Employer", :rich => true mal = Node.new :name => "Mal", :role => "Wife", arthur = Node.new :name => "Arthur", :role => "Partner" eames = Node.new :name => "Eames", :role => "Forger", :location => "Mombasa" yusuf = Node.new :name => "Yusuf", :role => "Chemist", :phone => "+254 41 787878" fischer = Node.new :name => "Fisher Jr.", :role => "Target", :age => 29

Relationship.new(:root, Neo4j.ref_node, cobb) Relationship.new(:knows, cobb, arthur)[:since] => "6 years" Relationship.new(:knows, cobb, eames) Relationship.new(:knows, arthur, eames) Relationship.new(:knows, cobb, mal) Relationship.new(:in_love, cobb, mal) Relationship.new(:knows, arthur, mal) Relationship.new(:knows, eames, yusuf)[:activity] => "undisclosed" Relationship.new(:knows, saito, fischer) Relationship.new(:hire, cobb, ariadne) Relationship.new(:hire, cobb, yusuf) Relationship.new(:hire, cobb, eames) Relationship.new(:hire, saito, cobb) Relationship.new(:hire, saito, arthur)end

cobb.outgoing(:knows).depth(:all).filter { self[:name] == "Yusuf" }.first


• ACID Atomicity, Consistency, Isolation, DurabilityOnly write locks, no read locks

Transactions

Neo4j::Transaction.run do # DO STUFFend


Object Oriented Mapping

class Person include Neo4j::NodeMixin property :nameend

Neo4j::Transaction.run do person = Person.new person.name = "pablo"end


Lucene in Neo4j.rb

class Person include Neo4j::NodeMixin property :name index :nameend

Neo4j::Transaction.run do person = Person.new person.name = "pablo"end

Person.find("name: pablo")


Traversals in Neo4j.rb

cobb.outgoing(:knows).depth(:all).filter { self[:name] == "Yusuf" }

pablo.outgoing(:follows).depth(2).filter { self[:age] == 30 }

From all people who cobb knows, and they know is there anyone named Yusuf?

From all people who pablo follows, which ones are 30 years old?


Relations

Neo4j::Transaction.run do pablete = Node.new(:login => "pablete") mauro = Node.new(:login => "malditogeek") tony = Node.new(:login => "antoniogarrote") pablete.outgoing(:friends) << mauro << tony

pablete.outgoing(:friends).each {|friend| puts friend[:login]}end


Relations with NodeMixin

class Person include Neo4j::NodeMixin property :name has_n :friendsend

Neo4j::Transaction.run do pablete = Person.new(:login => "pablete") mauro = Person.new(:login => "malditogeek") tony = Person.new(:login => "antoniogarrote") pablete.friends << mauro << tony

pablete.friends.each {|friend| puts friend[:login]}end


Rails Drop-in Replacement

class User < ActiveRecord::Base attr_accessor :password attr_accessible :name, :email, :password,:password_confirmation after_save :encrypt_password validates :name, :presence => true, :length => { :maximum => 50} validates :password, :presence => true, :confirmation => true, :length => {:withing => 6..40} has_one :profile private def encrypt_password self.salt = make_salt if new_record? self.encrypted_password = encrypt(password) endend


Rails Drop-in Replacement

class User < Neo4j::Model attr_accessor :password attr_accessible :name, :email, :password,:password_confirmation after_save :encrypt_password validates :name, :presence => true, :length => { :maximum => 50} validates :password, :presence => true, :confirmation => true, :length => {:withing => 6..40} property :name, :email, :salt, :encrypted_password index :email has_one :profile private def encrypt_password self.salt = make_salt if new_record? self.encrypted_password = encrypt(password) endend



DEMO TIME


Gephi - Makes Graph Handy

Github Example:5 Users:-pablete-martincik-ppeszko-antoniogarrote-malditogeek

Users in REDRepositories in BLUE

http:://github.com/pablete/conferenciarails2010



Special thanks to:

Andreas Rongehttp://twitter.com/ronge

Peter Neubauerhttp://twitter.com/peterneubauer

Marko A. Rodriguezhttp://twitter.com/twarko


http://twitter.com/ronge

http://twitter.com/ronge

http://twitter.com/peterneubauer







Thanks!

Pablo [email protected]

@pablete


mailto:[email protected]

mailto:[email protected]

neo4j for ruby and rails

Technology