nosql in the cloud
TRANSCRIPT
19. und 20. Oktober 2010 in Nürnberg www.prioconference.de
NOSQL Datenspeicherung in der Cloud
Sergey Shishkin [email protected] http://shishkin.org @sshishkin
Setting Expectations
• Motivators for cloud solutions
• Distributed database issues
• NOSQL solutions
• Code examples
• It’s not a deep dive session
Typical Motivators
• Low cost start-up
• Grow fast
• Peak loads
• Security
• Maintenance costs
Commodity Infrastructure
• Infrastructure-, Platform as a Service
• Raise the level of abstraction – Computational power
instead of CPU, GHz and L2 Cache
– Persistent storage instead of SATA, RAID and SCSI
– Communication channels instead of Ethernet, DHCP and TCP
• Everything will be in the cloud
Who can benefit from Cloud?
• Web 2.0 start-ups
• Social media
Source: http://www.appdata.com/leaderboard/apps
Casual Online Games
• Huge opportunity for start-ups
• Low cost, low risk
• Grow fast
60 Million Active Monthly Users
• Don’t know how FarmVille actually works
• But their scalability requirements are huge
Classical Web-farm Scale
Data persistence is a bottleneck
• ACID Transactions
• Locks are expensive
• RDBMS power of choice
– Optimized for massive writes
– Or for ad-hoc structured queries
– What a surprise!
• How does RDBMS scale?
Vertical Scale
• Costs grow exponentially
Horizontal Scale
• Master-Slave Replication
– Still bad for massive writes
• Master-Master Replication
– Resolving conflicts is hard in a normalized model
• Sharding/Partitioning
– Give up unique constraints, foreign keys, joins
What was the point of all that again?
Pretty much nothing left over of RDBMS.
Consistency All clients always have the same view of the data
Availability Each client can always read and write
Partition Tolerance The system works well despite physical network partitions
Brewer’s CAP-Theorem
Meet CAP Theorem
• You can choose only two!
Visual Guide to NOSQL
SimpleDB AzureTS CouchDB RDBMS
Consistency All clients always have the same view of the data
Availability Each client can always read and write
Partition Tolerance The system works well despite physical network partitions MongoDB
Key-Value Store Collection: BlogPosts
key: 1 ew0KICAgIHRpdGxlOiCTQ291Y2hEQpQsDQogICAgY29udGVudDogk4WUDQp9DQo=
key: 2 ew0KICAgIHRpdGxlOiCTTW9uZ29EQpQsDQogICAgY29udGVudDogk4WUDQp9DQo=
Document Store Collection: BlogPosts
key: 1 { title: “CouchDB”, content: “…” links: [“…”, “…”] }
key: 2 { title: “MongoDB”, content: “…” }
Domain: BlogPosts
Item: 1
Column: Title
Value: “SimpleDB”
Column: Content
Value: “…”
Colulmn: Links
Value: amazonwebservices.com
Value: amazon.com
Item: 2
...
Column Store
Data Storage Ecosystem
• NOSQL = Not Only SQL
• RDBMS still has its strengths
– Unique constraints
– But they don’t scale horizontally
• You don’t need a hammer when you have the right tools ;)
Düsseldorf 2010
25. November 2010, Ratingen http://cloudcamp.org/dusseldorf
Beratung Projekte Schulungen
Application Lifecycle Design Entwicklung
Architektur SOA Cloud Computing
BalckeBalcke-Dürr-Allee 9, 40882 Ratingen www.mt-ag.com [email protected]
Links
• Code Examples – MongoDB
• http://github.com/shishkin/MyBlog/tree/mongodb/MyBlog.Web/Data
– CouchDB
• http://github.com/shishkin/MyBlog/tree/couchdb/MyBlog.Web/Data
– SimpleDB
• http://github.com/shishkin/MyBlog/tree/simpledb/MyBlog.Web/Data
– Azure Table Service
• http://github.com/shishkin/MyBlog/tree/azure/MyBlog.Web/Data
• Full Visual Guide to NOSQL – http://blog.nahurst.com/visual-guide-to-nosql-systems
Images
• http://www.flickr.com/photos/brianauer/2599299352/ (swimmingpool)
• http://www.flickr.com/photos/orangeacid/227642583/ (cloud)
• http://www.flickr.com/photos/runningclouds/3220810175/ (city in clouds)
• http://isparade.jp (twitter parade)
• http://www.flickr.com/photos/seeminglee/4090890825/ (farmville)
• http://www.flickr.com/photos/icatus/2992269179/ (bottleneck)
• http://www.flickr.com/photos/theplanetdotcom/4879421740/ (data center)
• http://www.flickr.com/photos/redbullfanclub/3788029453/ (f1)
• http://www.flickr.com/photos/stevendepolo/4536694260/ (broken glass)
• http://www.flickr.com/photos/starstreak007/3232853321/ (toy car)
• http://browsertoolkit.com/fault-tolerance.png
• http://www.flickr.com/photos/ebarney/3348965007/ (tools)