Download - FluxGraph: a time-machine for your graphs
![Page 1: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/1.jpg)
FluxGraph: A time-machine for your graphs
Davy SuveeMichel Van Speybroeck
Janssen Pharmaceutica
![Page 2: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/2.jpg)
about me
➡ working as an it lead / software architect @ janssen pharmaceutica• dealing with big scientific data sets
• hands-on expertise in big data and NoSQL technologies
who am i ...
Davy Suvee@DSUVEE
➡ founder of datablend• provide big data and NoSQL consultancy
• share practical knowledge and big data use cases via blog
![Page 3: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/3.jpg)
![Page 4: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/4.jpg)
graphs and time ...
➡ graphs are continuously changing ...
![Page 5: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/5.jpg)
graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ... ★ neo-versioning by david montag 1
★ representing time dependent graphs in neo4j by the isi foundation 2
★ modeling a multilevel index in neo4j by peter neubauer 3
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
![Page 6: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/6.jpg)
graphs and time ...
➡ graphs are continuously changing ...
➡ graphs and time ... ★ neo-versioning by david montag 1
★ representing time dependent graphs in neo4j by the isi foundation 2
★ modeling a multilevel index in neo4j by peter neubauer 3
copy and relink semantics
1. http://github.com/dmontag/neo4j-versioning 2. http://github.com/ccattuto/neo4j-dynagraph/wiki 3. http://blog.neo4j.org/2012/02/modeling-multilevel-index-in-neoj4.html
๏ graph size
๏ object identity
๏ mixing data-model and time-model
![Page 7: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/7.jpg)
FluxGraph ...
➡ towards a time-aware graph ...
![Page 8: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/8.jpg)
FluxGraph ...
➡ implement a blueprints-compatible graph on top of Datomic
➡ towards a time-aware graph ...
![Page 9: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/9.jpg)
FluxGraph ...
➡ implement a blueprints-compatible graph on top of Datomic
➡ make FluxGraph fully time-aware ★ travel your graph through time★ time-scoped iteration of vertices and edges★ temporal graph comparison
➡ towards a time-aware graph ...
![Page 10: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/10.jpg)
travel through time
FluxGraph fg = new FluxGraph();
![Page 11: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/11.jpg)
travel through time
FluxGraph fg = new FluxGraph();
Vertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
Davy
![Page 12: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/12.jpg)
travel through time
FluxGraph fg = new FluxGraph();
Vertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
Davy
Peter
Vertex peter = ...
![Page 13: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/13.jpg)
travel through time
FluxGraph fg = new FluxGraph();
Vertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
Michael
Davy
Peter
Vertex peter = ...Vertex michael = ...
![Page 14: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/14.jpg)
travel through time
FluxGraph fg = new FluxGraph();
Vertex davy = fg.addVertex();davy.setProperty(“name”,”Davy”);
Michael
Davy
Peter
Vertex peter = ...Vertex michael = ...
Edge e1 = fg.addEdge(davy, peter,“knows”);
knows
![Page 15: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/15.jpg)
travel through time
Date checkpoint = new Date();
Michael
Davy
Peter
knows
![Page 16: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/16.jpg)
travel through time
Date checkpoint = new Date();
davy.setProperty(“name”,”David”);
Michael
Peter
knows
David
![Page 17: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/17.jpg)
travel through time
Date checkpoint = new Date();
davy.setProperty(“name”,”David”);
Michael
Peter
Edge e2 = fg.addEdge(davy, michael,“knows”);
knows
David
knows
![Page 18: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/18.jpg)
travel through time
Davy
Peter
Michael
knows
time
![Page 19: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/19.jpg)
travel through time
Davy
Peter
Michael
knows
checkpoint
time
![Page 20: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/20.jpg)
travel through time
Michael
Davy
Peter
DavidDavy
Peter
knows
knows
Michael
knows
checkpoint
time
![Page 21: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/21.jpg)
travel through time
Michael
Davy
Peter
DavidDavy
Peter
knows
knows
Michael
knows
checkpoint
currenttime
![Page 22: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/22.jpg)
travel through time
Michael
Davy
Peter
DavidDavy
Peter
knows
knows
Michael
knows
checkpoint
currenttime
by default
![Page 23: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/23.jpg)
travel through time
Michael
Davy
Peter
DavidDavy
Peter
knows
knows
Michael
knows
checkpoint
currenttime
fg.setCheckpointTime(checkpoint);
![Page 24: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/24.jpg)
time-scoped iteration
t1
Davy
![Page 25: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/25.jpg)
t2
time-scoped iteration
change
Davy’
t1
Davy
![Page 26: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/26.jpg)
t3t2
time-scoped iteration
change change
Davy’ Davy’’
t1
Davy
![Page 27: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/27.jpg)
tcurrrentt3t2
time-scoped iteration
change change change
Davy’’’Davy’ Davy’’
t1
Davy
![Page 28: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/28.jpg)
tcurrrentt3t2
time-scoped iteration
change change change
Davy’’’Davy’ Davy’’
t1
Davy
➡ how to find the version of the vertex you are interested in?
![Page 29: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/29.jpg)
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
![Page 30: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/30.jpg)
next next next
previouspreviousprevious
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
![Page 31: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/31.jpg)
next next next
previouspreviousprevious
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
Vertex previousDavy = davy.getPreviousVersion();
![Page 32: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/32.jpg)
next next next
previouspreviousprevious
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
![Page 33: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/33.jpg)
next next next
previouspreviousprevious
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);
![Page 34: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/34.jpg)
next next next
previouspreviousprevious
tcurrrentt3t2
time-scoped iteration
Davy’’’Davy’ Davy’’
t1
Davy
Vertex previousDavy = davy.getPreviousVersion();Iterable<Vertex> allDavy = davy.getNextVersions();
Iterable<Vertex> selDavy = davy.getPreviousVersions(filter);Interval valid = davy.getTimerInterval();
![Page 35: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/35.jpg)
time-scoped iteration
➡ When does an element change?
![Page 36: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/36.jpg)
time-scoped iteration
➡ vertex:★ setting or removing a property ★ add or remove it from an edge★ being removed
➡ When does an element change?
![Page 37: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/37.jpg)
time-scoped iteration
➡ vertex:★ setting or removing a property ★ add or remove it from an edge★ being removed
➡ When does an element change?
➡ edge:★ setting or removing a property ★ being removed
![Page 38: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/38.jpg)
time-scoped iteration
➡ vertex:★ setting or removing a property ★ add or remove it from an edge★ being removed
➡ When does an element change?
➡ edge:★ setting or removing a property ★ being removed
➡ ... and each element is time-scoped!
![Page 39: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/39.jpg)
MichaelMichael
Davy
Peter
David Davy
Peter
temporal graph comparison
knows
knows
knows
current checkpoint
what changed?
![Page 40: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/40.jpg)
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
![Page 41: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/41.jpg)
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
![Page 42: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/42.jpg)
temporal graph comparison
➡ difference (A , B) = union (A , B) - B
➡ ... as a (immutable) graph!
difference ( , ) =
David
knows
![Page 43: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/43.jpg)
FluxGraph ...
http://github.com/datablend/fluxgraph
➡ available on github
![Page 44: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/44.jpg)
t3t2t1
use case: longitudinal patient data
patient patient
smoking
patient
smoking
t4
patient
cancer
t5
patient
cancer
death
![Page 45: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/45.jpg)
use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
![Page 46: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/46.jpg)
use case: longitudinal patient data
➡ historical data for 15.000 patients over a period of 10 years (2001- 2010)
➡ example analysis: ★ if a male patient is no longer smoking in 2005★ what are the chances of getting lung cancer in 2010, comparing
patients that smoked before 2005
patients that never smoked
![Page 47: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/47.jpg)
use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
![Page 48: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/48.jpg)
use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
![Page 49: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/49.jpg)
use case: longitudinal patient data
➡ get all male non-smokers in 2005
fg.setCheckpointTime(new DateTime(2005,12,31).toDate());
Iterator<Vertex> males = fg.getVertices("gender", "male").iterator()
while (males.hasNext()) { Vertex p2005 = males.next(); boolean smoking2005 = p2005.getEdges(OUT,"smokingStatus").iterator().hasNext();}
![Page 50: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/50.jpg)
use case: longitudinal patient data
boolean smokingBefore2005 = ((FluxVertex)p2005).getPreviousVersions(new TimeAwareFilter() { public TimeAwareElement filter(TimeAwareVertex element) { return element.getEdges(OUT, "smokingStatus").iterator().hasNext() ? element : null; }
}).iterator().hasNext();
➡ which patients were smoking before 2005?
![Page 51: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/51.jpg)
use case: longitudinal patient data
Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
➡ which patients have cancer in 2010
working set of smokers
![Page 52: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/52.jpg)
use case: longitudinal patient data
Graph g = fg.difference(smokerws, time2010.toDate(), time2005.toDate());
➡ which patients have cancer in 2010
working set of smokers
➡ extract the patients that have an edge to the cancer node
![Page 53: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/53.jpg)
![Page 54: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/54.jpg)
gephi plugin for fluxgraph2010
![Page 55: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/55.jpg)
gephi plugin for fluxgraph2001
![Page 56: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/56.jpg)
gephi plugin for blueprints!
http://github.com/datablend/gephi-blueprints-plugin
➡ available on github
➡ Support for neo4j, orientdb, dex, rexter, ...
1. Kudos to Timmy Storms (@timmystorms)
1
![Page 57: FluxGraph: a time-machine for your graphs](https://reader033.vdocument.in/reader033/viewer/2022051610/548208adb479590f0d8b467c/html5/thumbnails/57.jpg)
Questions?