yes, sql! - qcon · yes, sql! uri cohen . ® copyright 2010 gigaspaces ltd. all rights reserved...
TRANSCRIPT
![Page 1: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/1.jpg)
Yes, SQL!
Uri Cohen
![Page 2: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/2.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
> SELECT * FROM qcon2010.speakers WHERE
name=„Uri Cohen‟+-----------------------------------------------------+
| Name | Company | Role | Twitter |
+-----------------------------------------------------+
| Uri Cohen | GigaSpaces | Product Manager | @uri1803 |
+-----------------------------------------------------+
2
> db.speakers.find({name:”Uri Cohen”}){
“name”:”Uri Cohen”,
“company”: {
name:”GigaSpaces”,
products:[“XAP”, “IMDG”]
domain: “In memory data grids”
}
“role”:”product manager”,
“twitter”:”@uri1803”
}
![Page 3: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/3.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Agenda
3
• SQL
– What it is and isn’t good for
• NoSQL
– Motivation & Main Concepts of Modern Distributed Data Stores
– Common interaction models
• Key/Value, Column, Document
• NOT consistency and distribution algorithms
• One Data Store, Multiple APIs
– Brief intro to GigaSpaces
– Key/Value challenges
– SQL challenges: Add-hoc querying, Relationships (JPA)
![Page 4: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/4.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A FEW (MORE) WORDS ABOUT SQL
4
![Page 5: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/5.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
(Usually) Centralized Transactional, consistent
Hard to Scale
5
![Page 6: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/6.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Static, normalized
data schema
• Don’t duplicate, use FKs
6
![Page 7: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/7.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Add hoc query support
Model first, query later
7
![Page 8: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/8.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL
Standard
Well known
Rich ecosystem
8
![Page 9: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/9.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
(BRIEF) NOSQL RECAP
9
![Page 10: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/10.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive Attempt to Define It)
A loosely coupled
collection of
non-relational data
stores
10
![Page 11: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/11.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive Attempt to Define It)
(Mostly)
d i s t r i b u t e d
11
![Page 12: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/12.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive Attempt to Define It)
scalable (Up & Out)
12
![Page 13: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/13.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
NoSql (or a Naive Attempt to Define It)
Not (always) ACID
• BASE anyone?
13
![Page 14: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/14.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Why Now?
Timing is everything…• Exponential Increase in data & throughput
• Non or semi structured data that changes
frequently
14
![Page 15: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/15.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Universe of Data Models
15
Key / Value Column
{
“name”:”uri”,
“ssn”:”213445”,
“hobbies”:[”…”,“…”],
“…”: {
“…”:”…”
“…”:”…”
}
}
{
{ ... }
}
{
{ ... }
}
Document
![Page 16: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/16.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
• Have the key? Get the value
– That’s about it when it comes to querying
– Map/Reduce (sometimes)
– Good for
• cache aside (e.g. Hibernate 2nd level cache)
• Simple, id based interactions (e.g. user profiles)
• In most cases, values are Opaque
16
K1 V1
K2 V2
K3 V3
K4 V1
![Page 17: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/17.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
Scaling out is relatively easy
(just hash the keys)
• Some will do that automatically for you
• Fixed vs. consistent hashing
17
![Page 18: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/18.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Key/Value
• Implementations:
– Memcached, Redis, Riak
– In memory data grids (mostly Java-based) started this way
• GigaSpaces, Oracle Coherence, WebSphere XS,
JBoss Infinispan, etc.
18
![Page 19: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/19.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
19
![Page 20: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/20.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
• Mostly derived from Google’s BigTable /
Amazon Dynamo papers
• One giant table of rows and columns
– Column == pair (name and a value, sometimes timestamp)
– Each row can have a different number of
columns
– Table is sparse:
(#rows) × (#columns) ≥ (#values)
20
![Page 21: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/21.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Column Based
• Query on row key
– Or column value (aka secondary index)
• Good for a constantly changing,
(albeit flat) domain model
21
![Page 22: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/22.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Document
Think JSON
(or BSON, or XML)
22
_id:1
_id:2
_id:3
{
“name”:”Lady Gaga”,
“ssn”:”213445”,
“hobbies”:[”Dressing up”,“Singing”],
“albums”:
[{“name”:”The fame”
“release_year”:”2008”},
{“name”:”Born this way”
“release_year”:”2011”}]
}
{
{ ... }
}
{
{ ... }
}
![Page 23: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/23.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Document
• Model is not flat, data store is aware of it
– Arrays, nested documents
• Better support for ad hoc queries
– MongoDB excels at this
• Very intuitive model
• Flexible schema
![Page 24: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/24.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
What if you
didn’t have to choose?
24
JPA
JDBC
![Page 25: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/25.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
In Memory Data Grid
• With optional write behind to
a secondary storage
25
![Page 26: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/26.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Tuple based
• Aware of nested tuples (and soon collections)
– Document like
• Rich querying and map/reduce
semantics
26
![Page 27: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/27.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Transparent partitioning & HA
• Fixed hashing based on a chosen
property
27
JAVA Virtual MachineJAVA Virtual Machine JAVA Virtual MachineJAVA Virtual Machine
Replication
Primary 1Backup 1
Replication
Backup 2Primary 2
![Page 28: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/28.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
A Brief Intro to GigaSpaces
Transactional (Like, ACID)
• Local (single partition)
• Distributed (multiple partitions)
28
![Page 29: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/29.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Use the Right API for the Job
29
• Even for the same data…
– POJO & JPA for Java apps with complex domain model
– Document for a more dynamic view
– Memcached for simple, language neutral
data access
– JDBC for:
• Interaction with legacy apps
• Flexible ad-hoc querying (e.g. projections)
![Page 30: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/30.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Memcached (the Daemon is in the Details)
MemcachedClient
![Page 31: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/31.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
MemcachedClient
Memcached (the Daemon is in the Details)
![Page 32: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/32.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL/JDBC – Query Them All
32
Query may involve Map/Reduce
• Reduce phase includes merging and sorting
![Page 33: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/33.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
SQL/JDBC – Things to Consider
• Unique and FK constraints are not practically
enforceable
• Sorting and aggregation may be expensive
• Distributed transactions are evil
– Stay local…
33
![Page 34: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/34.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA
It’s all about relationships…
34
![Page 35: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/35.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA Relationships
To embed or not to embed, that is the question….
Easy to partition and scale
Easy to query:
user.accounts[*].type = „checking‟
× Owned relationships only
![Page 36: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/36.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
JPA Relationships
To embed or not to embed, that is the question….
Any type of relationship
× Partitioning is hard
× Querying involves joining
![Page 37: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/37.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
Summary
• One API doesn’t fit all
– Use the right API for the job
• Know the tradeoffs
– Always ask what you’re giving up, not just what you’re
gaining
37
![Page 38: Yes, SQL! - QCon · Yes, SQL! Uri Cohen . ® Copyright 2010 Gigaspaces Ltd. All Rights Reserved > SELECT * FROM qcon2010.speakers WHERE](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b0cdc6e6dc46c7c1b1f4c/html5/thumbnails/38.jpg)
® Copyright 2010 Gigaspaces Ltd. All Rights Reserved
THANK YOU!
@uri1803http://blog.gigaspaces.com
38