the dark side of nosql
DESCRIPTION
NoSQL databases like MongoDB are making it easier than ever for developers to quickly build complex, agile applications that scale well. Unfortunately, the rise of NoSQL has dark side: data stored in NoSQL databases is invisible to traditional business analytics and intelligence apps. As a result, operations often spends a massive amount of time and effort developing complex pipelines to transform NoSQL data into a format compatible with traditional RDBMS. In this talk, John A. De Goes introduces SlamData, a new open source project designed to build a NoSQL BI application by extending the operations and data model of relational algebra. The result is backward compatible with SQL, but allows freeform analytics on semi-structured, heterogeneous, deeply nested data. John discusses the design of SlamData and highlights some of the issues involved in bringing the project to life.TRANSCRIPT
The Dark
Side of NoSQLJohn A. De Goes - @jdegoes
CTO SlamData Inc.
May 20, 2014 -- campDevOps
What
is N
oSQ
L? NoSQL = non-relational, operational,
(generally) distributed
databases.
Battle
of th
e
Para
dig
ms
SQL
• Flat
• Schema-ful
• Uniform
• SQL QL
• Atomic
• Ops Power
• Old/Boring
NoSQL
• Nested
• Schema-less
• Non-Uniform
• Ad Hoc QL
• Varies
• Dev Power
• New/
Exciting!
The R
ise o
f N
oSQ
L
Google Trends“MongoDB vs. Oracle DB”
So what’s the #1
problem with NoSQL?It’s not scaling, migrations, training, etc.
A S
ad, C
om
mon
Sto
ry
1.Dev chooses NoSQL DB
2.App generates data3.PHB wants answers4.Dev / Ops morph into
report monkeys, or5.Dev / Ops build ETL
equivalent of Death Star
What’s the solution?[Drumroll…]
“BI fo
r N
oSQ
L”
• 100% SQL
• 100% NoSQL
• 100% Open Source
• Doors open June 23!!!
“So simple my cat could use it!”*
*Simulated quote from hypothetical user
HTML5 GUI
SlamSQL
SlamEngineMRA
Ria
k
MongoD
B
Etc
.
SlamData
Snazzy BI interface for NoSQL
SlamData
Snazzy BI interface for NoSQL
• Run SlamSQL on different backends and visualize
results
• Create static or interactive reports
• Publish reports online or send them by email
• Very simple now but will evolve into NoSQL BI
SlamSQL
SQL for NoSQL
SELECT C
OUNT(*)
AS cnt,
v
iews[*].
searchTe
rms[*] A
S term,
v
iews[*].
date AS
date
FROM pro
ducts
GROUP BY
views[*
].date,
views[*
].search
Terms[*]
SlamSQL
SQL for NoSQL
• Nothing new to learn
• Backward compatible with ANSI SQL
• New operators to work with heterogeneous, nested
data
• Substantially more composable & expressive than
SQL
• Fully-inferred structural typing
• Intuitive JS syntax for accessing nested data
MRA
Multi-Dimensional Relational Algebra
1.Extends relational algebra
2.Extends relational data model
3.Strictly more powerful than relational
1. Unlike most NoSQL QLs designed for operations!
4.Natively designed for schema-less data
SlamEngine
The Brainz of SlamData
SlamSQL Parser
Structural Type-Inferencer
Logical Plan Compiler
Ria
k Pl
anner
MongoD
B
Planner
Etc
.
SlamEngine
The Brainz of SlamData
1.Written in 100% Scala
2.Clean, modular, robust architecture
3.Geared at supporting many back-ends
4.MongoDB first supported backend
5.Early in development (doesn’t fully support MRA)
6.Rapid progress with growing team
7.Easy admin: install, point & launch!
Quest
ions?
Q: Why is all the text
slanted?
A: Because you should
be paying attention to
the speaker and not trying to read the slides!
Thank you!*
*Special thanks to Laura Crow & the awesome JumpCloud team for
making this talk possible on shorter-than-short notice!
MRA
More
deta
il th
an y
ou e
ver
wante
d t
o k
now
• [*] – array flatten / hidden deconstructing expansion
• {*} – object flatten / hidden deconstructing
expansion• [<] – array dimensional expansion
• {<} – object dimensional expansion
• [e…] – array dimensional aggregation
• {e…} – object dimensional aggregation
• INDICES A – Array index replicating expansion
• FIELDS A – Field name replicating expansion
• A GROUP BY B – dimensional contractionPlus all the standard and extended relational operators!