Download - Titan: Big Graph Data with Cassandra
![Page 1: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/1.jpg)
AURELIUS THINKAURELIUS.COM
TITAN BIG GRAPH DATA WITH CASSANDRA
Matthias Broecheler, CTO August VIII, MMXII
#TITANDB #GRAPHDB #CASSANDRA12
![Page 2: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/2.jpg)
Abstract
Titan is an open source distributed graph database build on top of Cassandra that can power real-time applications with thousands of concurrent users over graphs with billions of edges. Graphs are a versatile data model for capturing and analyzing rich relational structures. Graphs are an increasingly popular way to represent data in a wide range of domains such as social networking, recommendation engines, advertisement optimization, knowledge representation, health care, education, and security.
This presentation discusses Titan's data model, query language, and novel techniques in edge compression, data layout, and vertex-centric indices which facilitate the representation and processing of Big Graph Data across a Cassandra cluster. We demonstrate Titan's performance on a large scale benchmark evaluation using Twitter data.
![Page 3: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/3.jpg)
Titan Graph Database
supports real time local traversals (OLTP)
is highly scalable in the number of concurrent users
in the size of the graph
is open source under the Apache2 license
builds on top of Apache Cassandra for distribution and replication
![Page 4: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/4.jpg)
AURELIUS THINKAURELIUS.COM
I The Graph Data Model
![Page 5: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/5.jpg)
Hercules: demigod Alcmene: human Jupiter: god Saturn: titan Pluto: god Neptune: god Cerberus: monster
Entities
![Page 6: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/6.jpg)
Table
Name Type
Hercules demigod
Alcmene human
Jupiter god
Saturn titan
Pluto god
Neptune god
Cerberus monster
![Page 7: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/7.jpg)
Documents
Name:
Alcmene Type:
human
Name:
Hercules Type:
demigod
Name:
Jupiter Type:
god
Name:
Saturn Type:
titan
Name:
Neptune Type:
god
Name:
Pluto Type:
god
Name:
Cerberus Type:
monster
![Page 8: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/8.jpg)
Key->Value
Hercules type:demigod
Alcmene type:human
Jupiter type:god
Saturn type:titan
Pluto type:god
Neptune type:god
Cerberus type:monster
![Page 9: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/9.jpg)
Graph
name: Jupiter type: god
name: Pluto type: god
name: Neptune type: god
name: Hercules type: demigod
name: Cerberus type: monster
name: Alcmene type: god
name: Saturn type: titan
Vertex Property
![Page 10: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/10.jpg)
Graph
name: Jupiter type: god
name: Pluto type: god
name: Neptune type: god
name: Hercules type: demigod
name: Cerberus type: monster
name: Alcmene type: god
name: Saturn type: titan
father father
mother brother
brother battled
pet
time:12
Edge
Edge Property
Edge Type
![Page 11: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/11.jpg)
I Graph = Agile Data Model
![Page 12: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/12.jpg)
AURELIUS THINKAURELIUS.COM
II Graph Use Cases
![Page 13: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/13.jpg)
Recommendations
![Page 14: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/14.jpg)
name: Hercules
Recommendation?
![Page 15: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/15.jpg)
name: Hercules name: “Muscle building for beginners” type: book
bought
![Page 16: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/16.jpg)
name: Newton
name: Hercules name: “Muscle building for beginners” type: book
bought
bought
![Page 17: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/17.jpg)
name: Newton
name: Hercules
name: “How to deal with Father issues” type: book
name: “Muscle building for beginners” type: book
bought
bought
bought
![Page 18: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/18.jpg)
name: Newton
name: Hercules
name: “How to deal with Father issues” type: book
name: “Muscle building for beginners” type: book
bought
bought
bought
Traversal
recommend
![Page 19: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/19.jpg)
name: Newton
name: Hercules
name: “How to deal with Father issues” type: book
name: “Muscle building for beginners” type: book
bought
bought
bought
name: “Dancing with the Stars” type: DVD
name: “Friends forever bracelet” type: Accessory
viewed
in-Cart
![Page 20: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/20.jpg)
name: Newton
name: Hercules
name: “How to deal with Father issues” type: book
name: “Muscle building for beginners” type: book
bought
friends
bought
bought
name: “Dancing with the Stars” type: DVD
name: “Friends forever bracelet” type: Accessory
viewed
in-Cart
![Page 21: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/21.jpg)
name: Newton
name: Hercules
name: “How to deal with Father issues” type: book
name: “Muscle building for beginners” type: book
bought
friends
time:24 bought
bought time:22
time:20
name: “Dancing with the Stars” type: DVD
name: “Friends forever bracelet” type: Accessory
viewed
in-Cart
![Page 22: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/22.jpg)
Recommendations
Path Finding
![Page 23: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/23.jpg)
Path Finding
X
X
name: Jupiter type: god
name: Pluto type: god
name: Neptune type: god
name: Hercules type: demigod
name: Cerberus type: monster
name: Alcmene type: god
name: Saturn type: titan
father father
mother brother
brother battled
pet
time:12
![Page 24: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/24.jpg)
Path Finding
X
X
name: Jupiter type: god
name: Pluto type: god
name: Neptune type: god
name: Hercules type: demigod
name: Cerberus type: monster
name: Alcmene type: god
name: Saturn type: titan
father father
mother brother
brother battled
pet
time:12
![Page 25: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/25.jpg)
![Page 26: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/26.jpg)
Credibility?
cnn.com
<html> … </html>!
yahoo.com
<html> … </html>!
geocities.com/johnlittlesite
<html> … </html>!
![Page 27: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/27.jpg)
url: yahoo.com html: <html>…!
url: geocities.com/johnlittlesite html: <html>…!
url: cnn.com html: <html>…!
Link Graph
![Page 28: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/28.jpg)
url: yahoo.com html: <html>…!
url: geocities.com/johnlittlesite html: <html>…!
url: cnn.com html: <html>…!
elections
funny cat foreign policy
Link Graph
![Page 29: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/29.jpg)
II Graph = Milk Your Connections
![Page 30: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/30.jpg)
AURELIUS THINKAURELIUS.COM
III The Titan Graph Database
![Page 31: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/31.jpg)
numerous concurrent users
real-time traversals (OLTP)
high availability
dynamic scalability
built on Apache Cassandra
Titan Features
![Page 32: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/32.jpg)
Titan Ecosystem
Native Blueprints Implementation
Gremlin Query Language
Rexster Server any Titan graph can be exposed
as a REST endpoint Generic
Graph API
Dataflow Processing
TraversalLanguage
Object-GraphMapper
GraphAlgorithms
GraphServer
![Page 33: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/33.jpg)
Titan Internals
I. Data Management
II. Edge Compression
III. Vertex-Centric Indices
![Page 34: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/34.jpg)
AURELIUS THINKAURELIUS.COM
IV Rebuilding Twitter with Titan
![Page 35: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/35.jpg)
User Tweet
text: string time: long!
tweets
follows
name: string!
time: long!
time: long!
![Page 36: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/36.jpg)
User Tweet
text: string time: long!
tweets
follows
name: string!
time: long!
time: long!
stream
time: long!
![Page 37: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/37.jpg)
Titan Storage Model
Adjacency list in one column family
Row key = vertex id Each property and edge
in one column Denormalized, i.e. stored twice
Direction and label/key as column prefix Use slice predicate for quick retrieval
5
5
![Page 38: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/38.jpg)
Connecting Titan
titan$ bin/gremlin.sh! \,,,/! (o o)!-----oOOo-(_)-oOOo-----!gremlin> conf = new BaseConfiguration();!==>org.apache.commons.configuration.BaseConfiguration@763861e6!gremlin> conf.setProperty("storage.backend","cassandra");!gremlin> conf.setProperty("storage.hostname","77.77.77.77");!gremlin> g = TitanFactory.open(conf); ==>titangraph[cassandra:77.77.77.77]!gremlin>!
![Page 39: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/39.jpg)
Defining Property Keys
gremlin> g.makeType().name(“time”).!! ! dataType(Long.class).!! ! functional().!! ! makePropertyKey();!
gremlin> g.makeType().name(“text”).dataType(String.class).!! ! functional().makePropertyKey();!
gremlin> g.makeType().name(“name”).dataType(String.class).!! ! indexed().!! ! unique().!! ! functional().makePropertyKey();!
![Page 40: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/40.jpg)
Defining Property Keys
gremlin> g.makeType().name(“time”).!! ! dataType(Long.class).!! ! functional().!! ! makePropertyKey();!
gremlin> g.makeType().name(“text”).dataType(String.class).!! ! functional().makePropertyKey();!
gremlin> g.makeType().name(“name”).dataType(String.class).!! ! indexed().!! ! unique().!! ! functional().makePropertyKey();!
Each type has a unique name
The allowed data type
If a key is functional, each vertex can have at most one property for this key
![Page 41: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/41.jpg)
Defining Property Keys
gremlin> g.makeType().name(“time”).!! ! dataType(Long.class).!! ! functional().!! ! makePropertyKey();!
gremlin> g.makeType().name(“text”).dataType(String.class).!! ! functional().makePropertyKey();!
gremlin> g.makeType().name(“name”).dataType(String.class).!! ! indexed().!! ! unique().!! ! functional().makePropertyKey();!
Creates and maintains an index over property values
Ensures that each property value is uniquely associated with only one vertex by acquiring a lock.
![Page 42: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/42.jpg)
Titan Indexing
Vertices can be retrieved by property key + value
Titan maintains index in a separate column family as graph is updated
Only need to define a property key as .index()
5
9
name : Hercules
name : Jupiter
![Page 43: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/43.jpg)
Titan Locking Locking ensures consistency
when it is needed Titan uses time stamped
quorum reads and writes on separate CFs for locking
Uses Property uniqueness: .unique() Functional edges: .functional() Global ID management
5
9
name : Hercules
name : Hercules
name : Jupiter
name : Pluto
father
father
x
![Page 44: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/44.jpg)
Defining Edge Labels
gremlin> g.makeType().name(“follows”).!! ! primaryKey(time).!! ! makeEdgeLabel();!
gremlin> g.makeType().name(“tweets”).!! ! primaryKey(time).makeEdgeLabel();!
gremlin> g.makeType().name(“stream).!! ! primaryKey(time).!! ! unidirected().!! ! makeEdgeLabel();!
![Page 45: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/45.jpg)
Defining Edge Labels
gremlin> g.makeType().name(“follows”).!! ! primaryKey(time).!! ! makeEdgeLabel();!
gremlin> g.makeType().name(“tweets”).!! ! primaryKey(time).makeEdgeLabel();!
gremlin> g.makeType().name(“stream).!! ! primaryKey(time).!! ! unidirected().!! ! makeEdgeLabel();!
Sort/index key for edges of this label
![Page 46: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/46.jpg)
Defining Edge Labels
gremlin> g.makeType().name(“follows”).!! ! primaryKey(time).!! ! makeEdgeLabel();!
gremlin> g.makeType().name(“tweets”).!! ! primaryKey(time).makeEdgeLabel();!
gremlin> g.makeType().name(“stream).!! ! primaryKey(time).!! ! unidirected().!! ! makeEdgeLabel();!
Store edges of this label only in outgoing direction
![Page 47: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/47.jpg)
Vertex-Centric Indices
Sort and index edges per vertex by primary key Primary key can be composite
Enables efficient focused traversals Only retrieve edges that matter
Uses slice predicate for quick, index-driven retrieval
![Page 48: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/48.jpg)
v
time: 123
follows follows follows
follows
tweets tweets tweets
tweets
time: 334 time: 624
time: 1112
v.query()!
![Page 49: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/49.jpg)
v
time: 123
follows follows
tweets tweets tweets
tweets
time: 334 time: 624
time: 1112
v.query()!.direction(OUT)!
![Page 50: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/50.jpg)
v
time: 123
tweets tweets tweets
tweets
time: 334 time: 624
time: 1112
v.query()!.direction(OUT)!.labels(“tweets”)!
![Page 51: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/51.jpg)
v tweets
time: 1112
v.query()!.direction(OUT)!.labels(“tweets”)!.has(“time”,T.gt,1000)!
![Page 52: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/52.jpg)
gremlin> hercules = g.addVertex(['name':'Hercules']);!
gremlin> pluto = g.addVertex(['name':'Pluto']);!
name: Pluto name: Hercules Create Accounts
![Page 53: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/53.jpg)
gremlin> hercules = g.addVertex(['name':'Hercules']);!
gremlin> pluto = g.addVertex(['name':'Pluto']);!
gremlin> g.addEdge(hercules,pluto,"follows",['time':2]);!
name: Pluto
follows
name: Hercules
time:2
Add Followship
![Page 54: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/54.jpg)
gremlin> hercules = g.addVertex(['name':'Hercules']);!
gremlin> pluto = g.addVertex(['name':'Pluto']);!
gremlin> g.addEdge(hercules,pluto,"follows",['time':2]);!
gremlin> tweet = g.addVertex(['text':'A tweet!','time':4])!
gremlin> g.addEdge(pluto,tweet,"tweets",['time':4]) !
text: A tweet! time: 4!
name: Pluto
follows
tweets
name: Hercules
time:2
time:4
Publish Tweet
![Page 55: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/55.jpg)
gremlin> hercules = g.addVertex(['name':'Hercules']);!
gremlin> pluto = g.addVertex(['name':'Pluto']);!
gremlin> g.addEdge(hercules,pluto,"follows",['time':2]);!
gremlin> tweet = g.addVertex(['text':'A tweet!','time':4])!
gremlin> g.addEdge(pluto,tweet,"tweets",['time':4]) !
gremlin> pluto.in("follows").each{g.addEdge(it,tweet,"stream",['time':4])} !
text: A tweet! time: 4!
name: Pluto
follows
stream tweets
name: Hercules
time:2
time:4 time:4
Update Streams
![Page 56: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/56.jpg)
gremlin> hercules = g.addVertex(['name':'Hercules']);!
gremlin> pluto = g.addVertex(['name':'Pluto']);!
gremlin> g.addEdge(hercules,pluto,"follows",['time':2]);!
gremlin> tweet = g.addVertex(['text':'A tweet!','time':4])!
gremlin> g.addEdge(pluto,tweet,"tweets",['time':4]) !
gremlin> pluto.in("follows").each{g.addEdge(it,tweet,"stream",['time':4])} !
gremlin> hercules.outE('stream')[0..9].inV.map!
text: A tweet! time: 4!
name: Pluto
follows
stream tweets
name: Hercules
time:2
time:4 time:4
Read Stream
Sorted by time because its ‘stream’s primary key
![Page 57: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/57.jpg)
follows = g.V('name',’Hercules’).out('follows').toList()!
follows20 = follows[(0..19).collect{random.nextInt(follows.size)}]!
m = [:]!
follows20.each !
{ it.outE('follows’[0..29].inV.except(follows).groupCount(m).iterate() }!
m.sort{a,b -> b.value <=> a.value}[0..4]!
name: Neptune
name: Pluto
follows
follows follows
name: Hercules
time:2
time:9
Followship Recommendation
![Page 58: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/58.jpg)
AURELIUS THINKAURELIUS.COM
IV Titan Performance Evaluation on Twitter-like Benchmark
![Page 59: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/59.jpg)
Twitter Benchmark 1.47 billion followship edges
and 41.7 million users Loaded into Titan using BatchGraph Twitter in 2009, crawled by Kwak et. al
4 Transaction Types Create Account (1%) Publish tweet (15%) Read stream (76%) Recommendation (8%)
Follow recommended user (30%) Kwak, H., Lee, C., Park, H., Moon, S., “What is Twitter, a Social Network or a News Media?,” World Wide Web Conference, 2010.
![Page 60: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/60.jpg)
Benchmark Setup
6 cc1.4xl Cassandra nodes in one placement group Cassandra 1.10
40 m1.small worker machines repeatedly running transactions simulating servers handling user
requests
EC2 cost: $11/hour
![Page 61: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/61.jpg)
Benchmark Results
Transaction Type Number of tx Mean tx time Std of tx time
Create account 379,019 115.15 ms 5.88 ms
Publish tweet 7,580,995 18.45 ms 6.34 ms
Read stream 37,936,184 6.29 ms 1.62 ms
Recommendation 3,793,863 67.65 ms 13.89 ms
Total 49,690,061
Runtime 2.3 hours 5,900 tx/sec
![Page 62: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/62.jpg)
Peak Load Results
Transaction Type Number of tx Mean tx time Std of tx time
Create account 374,860 172.74 ms 10.52 ms
Publish tweet 7,517,667 70.07 ms 19.43 ms
Read stream 37,618,648 24.40 ms 3.18 ms
Recommendation 3,758,266 229.83 ms 29.08 ms
Total 49,269,441
Runtime 1.3 hours 10,200 tx/sec
![Page 63: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/63.jpg)
Benchmark Conclusion
Titan can handle 10s of thousands of concurrent users with short response 5mes even for complex traversals on a simulated social networking applica5on based on real-‐world network data with billions of edges and millions of users in a standard EC2 deployment.
For more informa5on on the benchmark: hDp://thinkaurelius.com/2012/08/06/5tan-‐provides-‐real-‐5me-‐big-‐graph-‐data/
![Page 64: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/64.jpg)
Future Titan
Titan+Cassandra embedding sending Gremlin queries into
the cluster
Graph partitioning together with ByteOrderedPartitioner data locality = better performance
Let us know what you need!
![Page 65: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/65.jpg)
Titan goes OLAP
Stores a massive-scale property graph allowing real-time traversals and updates
Batch processing of large graphs with Hadoop
Runs global graph algorithms on large, compressed,
in-memory graphs
Map/Reduce Load & Compress
Analysis results back into Titan
![Page 66: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/66.jpg)
III Graph = Scalable + Practical
![Page 67: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/67.jpg)
TITAN THINKAURELIUS.GITHUB.COM/TITAN
![Page 68: Titan: Big Graph Data with Cassandra](https://reader033.vdocument.in/reader033/viewer/2022042714/554bd6f7b4c905706a8b5167/html5/thumbnails/68.jpg)
AURELIUS THINKAURELIUS.COM