graphs for enterprise architects
TRANSCRIPT
![Page 1: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/1.jpg)
Graphs for enterprise architects
Jesus Barrasa @barrasadvMark Needham @markhneedham
![Page 2: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/2.jpg)
Presentation Outline
‣ Context: what are graph DBs & Neo4j ?‣ Graph Dbs features matching EA’s demands‣ Cypher‣ Architectural patterns‣ Integration, adoption, partners and the
Neo4j ecosystem‣ Q&A
![Page 3: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/3.jpg)
Case Studies
![Page 4: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/4.jpg)
Introducing Neo4j
‣ Most popular graph database...
...
‣ ...and in top 20 DBMS platforms!
![Page 5: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/5.jpg)
2000 2003 2007 2009 2011 2013 2014 20152012
GraphConnect, first conference for graph DBs
First Global 2000
Customer
Introduced first and only
declarative query language for
property graph
Published O’Reilly book
on Graph Databases
$11M Series A from Fidelity,
Sunstoneand Conor
$11M Series B from Fidelity,
Sunstoneand Conor
CommercialLeadership
First native
graph DB in 24/7
production
Invented property
graph model
Contributed first graph DB to open
source
$2.5M SeedRound from
Sunstone and Conor
Funding
Extended graph data model to labeled
property graph
150+ customers
50K+ monthlydownloads
500+ graph DB eventsworldwide
$20M Series C led by
Creandum, with Dawn and
existing investors
TechnicalLeadership
Neo4j: The Graph Database Leader
![Page 6: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/6.jpg)
Graph?
![Page 7: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/7.jpg)
Graph!
![Page 8: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/8.jpg)
Demands on EA
Time to market
FlexibilityOperational Efficiency
![Page 9: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/9.jpg)
Flexibility
‣ Data is the model ‣ Optional schema ‣ Multi-dimensional queries
![Page 10: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/10.jpg)
Flexibility
‣ Data is the model ‣ Optional schema ‣ Multi-dimensional queries
![Page 11: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/11.jpg)
Whiteboard model = physical model
![Page 12: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/12.jpg)
Whiteboard model = physical model
![Page 13: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/13.jpg)
Whiteboard model = physical model
![Page 14: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/14.jpg)
Flexibility
‣ Data is the model ‣ Optional schema ‣ Multi-dimensional queries
![Page 15: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/15.jpg)
Optional Schema
CREATE CONSTRAINT ON (book:Book)
ASSERT book.isbn IS UNIQUE
CREATE CONSTRAINT ON (book:Book)
ASSERT exists(book.isbn)
![Page 16: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/16.jpg)
Flexibility
‣ Data is the model ‣ Optional schema ‣ Multi-dimensional queries
![Page 17: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/17.jpg)
Operational efficiency
‣ Minutes to millisecond performance• Savings on hardware/software costs
‣ Horizontal + Vertical scaling‣ Mature DB
• In production for >10y, over 200 enterprise
subscription customers
‣ Professional support, services & training
![Page 18: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/18.jpg)
Time to Market
‣ Agile/incremental delivery‣ Reduced Business-IT gap‣ Lower modeling impedance
![Page 19: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/19.jpg)
Time to Market
‣ Agile/incremental delivery‣ Reduced Business-IT gap‣ Lower modeling impedance
![Page 20: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/20.jpg)
Agile/incremental delivery
‣ Graph model is naturally adaptive• Build the model for the questions that we need to
answer• Evolve your model as new requirements arise
‣ No need to design the whole thing up front
![Page 21: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/21.jpg)
Time to Market
‣ Agile/incremental delivery‣ Reduced Business-IT gap‣ Lower modeling impedance
![Page 22: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/22.jpg)
Reduced Business-IT gap
‣ Intuitiveness of the graph model‣ Code easier to read, better communication
between teams‣ Faster ramp-up for new project members
![Page 23: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/23.jpg)
Time to Market
‣ Agile/incremental delivery
‣ Reduced Business-IT gap
‣ Lower modeling impedance
![Page 24: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/24.jpg)
Relational vs Graph models
Relational Model Graph Model
KNOWS
KNOWS
KNOWS
ANDREAS
TOBIAS
MICA
DELIA
Person FriendPerson-Friend
ANDREASDELIA
TOBIAS
MICA
![Page 25: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/25.jpg)
Cypher: The Graph Query Language
LOVES
Dan Ann
![Page 26: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/26.jpg)
Cypher: The Graph Query Language
MATCH (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} )
LOVES
Dan Ann
NODE NODERELATIONSHIP
![Page 27: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/27.jpg)
Cypher: The Graph Query Language
MATCH (:Person { name:“Dan”} ) -[:LOVES]-> (:Person { name:“Ann”} )
LOVES
Dan Ann
LABEL PROPERTY
NODE NODE
LABEL PROPERTY
RELATIONSHIP
![Page 28: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/28.jpg)
Cypher: The Graph Query Language
Find all direct reports and how many people they manage, up to 3 levels down
![Page 29: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/29.jpg)
Cypher: The Graph Query Language
Find all direct reports and how many people they manage, up to 3 levels down
SQL Query
![Page 30: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/30.jpg)
Cypher: The Graph Query Language
MATCH (boss)-[:MANAGES*0..3]->(sub), (sub)-[:MANAGES*1..3]->(report)WHERE boss.name = “John Doe”RETURN sub.name AS Subordinate, count(report) AS Total
Find all direct reports and how many people they manage, up to 3 levels down
Cypher Query
SQL Query
![Page 31: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/31.jpg)
Cypher: The Graph Query Language
‣ A query language designed for connectedness‣ Impact on projects
• Less time writing / optimising /debugging queries• Improved code maintainability
‣ Impact on teams• Improved communication• Faster ramp-up for new project members
![Page 32: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/32.jpg)
Patterns in Neo4j architectures
‣ Primary Store• Reads + Write
‣ Secondary Store• Mostly read, scheduled batch updates
‣ ODS / MDS• Mostly read, scheduled batch updates + integration
logic
![Page 33: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/33.jpg)
Primary Store case: Telenor
‣ Identity and access management problem. Need to compute resource authorization in real time
Was taking up to 20 min for large customers -> precalculation & cache -> stale data
![Page 34: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/34.jpg)
Original architecture
500 requests / sec
Middleware services
Backend Backend Backend
Channel Channel Channel42 channels
35 systems
![Page 35: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/35.jpg)
New architecture
500 requests / sec
Middleware services
Backend Backend Backend
Channel Channel Channel42 channels
35 systems
![Page 36: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/36.jpg)
Did it work?
‣ Response times reduced to seconds and milliseconds
‣ Code maintainability improved. Access rules in Cypher as graph patterns.
"The Neo4j graph database gives us drastically improved performance and a simple language to query our connected data" – Sebastian Verheughe, Architect & Developer
![Page 37: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/37.jpg)
Who uses Neo4j like this?
‣ UBS using it to control access to systems• Also migrated from RDBMS based solution• The Value
‣ Compliance‣ Faster onboarding‣ Real-time provisioning‣ Real-time deprovisioning
‣ AYI by Snap Interactive• Needed third degree connections in social network
![Page 38: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/38.jpg)
Secondary Store case : Shutl/eBay
Wanted to come up with quotes for delivery slots.
It took 2 seconds to come up with a quote for a single delivery slot and they wanted to extend the service to show 20 potential delivery slots
![Page 39: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/39.jpg)
Original architecture
Requests quote
Rails monolith Relational
![Page 40: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/40.jpg)
Solution
Broke application up into services e.g. quoting, booking, feedback.
Key goal was to improve the speed of quoting and neo4j backed this service
![Page 41: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/41.jpg)
New architecture
Requests quote
Quoting service Relational
Booking service
Feedback service
Nightly job to populate
ETL
![Page 42: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/42.jpg)
Did it work?
‣ Quoting for 20 time windows down from 82,000 ms to 80ms
‣ Code complexity much reduced
![Page 43: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/43.jpg)
Did it work?
‣ Quoting for 20 time windows down from 82,000 ms to 80ms
‣ Code complexity much reduced
“Our Neo4j solution is literally thousands of times faster than the prior MySQL solution, with queries that require 10-100 times less code. At the same time, Neo4j allowed us to add functionality that was previously not possible.”
Volker Pacher, Member of Technical Staff, eBay
![Page 44: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/44.jpg)
Who uses Neo4j like this?
‣ World’s largest courier service• They noticed that they would not have been able to
deliver packages by Christmas 2013 !!• Hierarchical routing system (RDBMS backed)• Replaced it with Neo4j using lateral routing • Value:
‣ Minimized time‣ Maximized usage of road network
![Page 45: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/45.jpg)
ODS / MDS case: Gov. Agency
‣ The graph integrates data from multiple sources• ODS: Additional operations on the integrated data
• MDS: Provide reference data to other operational
systems
‣ European government agencies
![Page 46: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/46.jpg)
Traditional ODS/MDS architecture
RelationalODS
Data Source 1
Data Source 2
Semi / unstructured data source
ETL
Operational BI
RelationalMDS
EDW
Operational Systems
![Page 47: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/47.jpg)
Graph based ODS/MDS architecture
ETL
Real time connected data analysis and
exploration
EDW
Operational Systems
Data Source 1
Data Source 2
Semi / unstructured data source
![Page 48: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/48.jpg)
Who uses Neo4j like this?
‣ Schleich: Toy manufacturer. Germany
• PDM holding a MPD from: component manufacturers,
suppliers, different countries regulations• Rich and complex data used across entire value chain.• Previous arch: RDBMS backed unfit for Schleich’s
requirements for flexibility, perf and ease of operation‣ Adidas
• Metadata Store• Used to drive content personalization on their web site
![Page 49: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/49.jpg)
Neo4j: Integration surfaces
‣ Cypher integration• JDBC• Transactional REST endpoint
‣ Java integration• Extensions / Procedures
‣ Import Tool‣ Batch Inserter
Tran
sact
iona
l
Non
Transactional
![Page 50: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/50.jpg)
Adopting Neo4j
‣ Start with PoC‣ Bootcamp‣ First project
• an obviously graphy problem
‣ Multiple projects
![Page 51: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/51.jpg)
Community vs Enterprise
![Page 52: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/52.jpg)
Community vs Enterprise
![Page 53: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/53.jpg)
The Neo4j Ecosystem
Graph ComputeMazerunner Spark Integrationhttps://github.com/neo4j-contrib/neo4j-mazerunner
Datastore IntegrationCassandra https://github.com/neo4j-contrib/neo4j-cassandra-connector
MongoDBhttps://github.com/neo4j-contrib/neo4j_doc_manager
Elastichttps://github.com/neo4j-contrib/neo4j-elasticsearch
Cloud HostingAWShttp://neo4j.com/developer/guide-cloud-deployment/
Graphene DBhttp://www.graphenedb.com/
Graph Storyhttp://graphstory.com/
Neo4j on Dockerhttps://hub.docker.com/r/neo4j/neo4j/
Graph Visualisation
Popoto.JShttp://www.popotojs.com/
Linkurious.jshttps://github.com/Linkurious/linkurious.js
Keylineshttp://cambridge-intelligence.com/keylines/neo4j
App Frameworks
Structrhttps://structr.org/
Graph Awarehttp://graphaware.com/products/
DB MigrationsLiquigraphhttp://fbiville.github.io/liquigraph
Business IntelligenceQlikView http://irregular-bi.tumblr.com/tagged/neo4j
DB IntrospectionSylva DBhttp://sylvadb.com/
NeoProfilerhttps://github.com/moxious/neoprofiler
![Page 54: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/54.jpg)
‣ Training courses• Graph fundamentals• Graph modeling • SI training
‣ Meetups• introductory/modeling/use case specific
‣ Consulting
Getting the team up to speed
![Page 55: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/55.jpg)
Partners
![Page 56: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/56.jpg)
Lots of ways to learn Neo4j
![Page 57: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/57.jpg)
Adoption by selected verticalsFinancialServices Communications Health &
Life SciencesHR &
RecruitingMedia &
PublishingSocialWeb
Industry & Logistics
Entertainment Consumer Retail Information ServicesBusiness Services
![Page 58: Graphs for Enterprise Architects](https://reader030.vdocument.in/reader030/viewer/2022020410/588460f51a28abbd308b4bb9/html5/thumbnails/58.jpg)
How customers use Neo4jNetwork &
Data Center Master DataManagementSocial Recom–
mendationsIdentity &
AccessSearch &Discovery GEO