neotys conference
TRANSCRIPT
Technical Evangelist, MongoDB@tgrall
Tugdual “Tug” Grall
@Neotys
SQL vs NoSQL Performance A MongoDB perspective
{ “about” : “me” }Tugdual “Tug” Grall • MongoDB
• Technical Evangelist
• Couchbase
• Technical Evangelist
• eXo
• CTO
• Oracle
• Developer/Product Manager
• Mainly Java/SOA
• Developer in consulting firms
• Web
• @tgrall
• http://blog.grallandco.com
• tgrall
• NantesJUG co-founder
• Pet Project :
• http://www.resultri.com
Why MongoDB ?
And makes things hard to change
Name Age Phone Email
New Column
New Table
New Table
New Column
It runs on expensive hardware
“Clients can also opt to run zEC12 without a raised datacenter floor -- a first for high-end IBM mainframes.” !!
IBM Press Release 28 Aug, 2012!
This was a problem for Google
250
,000
+ M
BP’s
== 4
.1 m
iles2010 Search Index Size: !
100,000,000 GB
New data added per day!100,000+ GB
Databases they could use!0
What is MongoDB?
MongoDB Vision
To provide the best database for how we build and run apps today
Build
• New and complex data
• Flexible
• New languages
• Faster development
Run
• Big Data scalability
• Real-time
• Commodity hardware
• Cloud
Documents are easier
Relational MongoDB{ first_name: "Paul", surname: "Miller", city: "London", location: [45.123,47.232], cars: [ { model: "Bentley", year: 1973, value: 100000, … }, { model: "Rolls Royce", year: 1965, value: 330000, … } ] }
Full Featured
Queries• Find Paul’s cars • Find everybody in London with a car built
between 1970 and 1980
Geospatial • Find all of the car owners within 5km of Trafalgar Sq.
Text Search • Find all the cars described as having leather seats
Aggregation • Calculate the average value of Paul’s car collection
Map Reduce• What is the ownership pattern of colors
by geography over time? (is purple trending up in China?)
{ first_name: ‘Paul’, surname: ‘Miller’, city: ‘London’, location: { ! type: “Point”, !coordinates : ! ! [-0.128, 51.507] ! },! cars: [ { model: ‘Bentley’, year: 1973, value: 100000, … }, { model: ‘Rolls Royce’, year: 1965, value: 330000, … } } }
Easier and Cheaper to Scale
Vs.
Pric
e
Scale
Pric
e
Scale
7,000,000+ MongoDB Downloads
150,000+ Online Education Registrants
25,000+ MongoDB User Group Members
25,000+ MongoDB Management Service (MMS) Users
20,000+ MongoDB Days Attendees
MongoDB Use Cases
16
Big Data Product & Asset Catalogs
Security & Fraud
Internet of Things
Database-as-a-Service
Mobile Apps
Customer Data Management
Data Hub
Social & Collaboration
Content Management
Intelligence Agencies
Top Investment and Retail Banks
Top US Retailer
Top Global Shipping Company
Top Industrial Equipment Manufacturer
Top Media Company
Top Investment and Retail Banks
MongoDB Deployments
Horizontal Scalability: Sharding
Shard 1 Shard 1 Shard nShard 1 …
Horizontally Scalable
Performance
MongoDB is a Distributed Database
Insert Update Delete Query Aggregations
Analyze Query Performance
• Use query explain() cursor method
• Use Database Profiler to find slow queriesdb.inventory.find( { type: 'food' } ).explain()!!{! "cursor" : "BtreeCursor type_1",! "isMultiKey" : false,! "n" : 5,! "nscannedObjects" : 5,! "nscanned" : 5,! "nscannedObjectsAllPlans" : 5,! "nscannedAllPlans" : 5,! …! "scanAndOrder" : false,! "millis" : 0,! "indexBounds" : { "type" : [! [ "food",! "food" ]! ] },! "server" : "mongodbo0.example.net:27017" }
Monitor: What to look?
Locks Avoid long running operations that could slow down the database
Page Faults Check that the working set stays in RAM, reduce I/O operations
Nbr of Connections Reduce RAM consumption
Replication Lag Time to copy the data to secondaries
OpLog Size Size of the replication queue
Chunk Distribution Data should be balanced on all the nodes
Many tools / Many Metrics
• MMS : MongoDB Management Services
• CLI and REST API
• Neotys
More..
You must prepare your deployment
Node 1SecondaryConfigServer
Node 1SecondaryConfigServer
Node 1SecondaryConfigServer
Shard Shard Shard
Mongos
App Server
Mongos
App Server
Mongos
App Server
A good shard key
Example : email storage
Cardinality!Write Scaling!
Query Isolation!
Reliability!Index!
Locality!
_id! Doc level! One shard!Scatter/gather!
All users affected!
Good!
hash(_id)! Hash level! All Shards!Scatter/gather!
All users affected!
Poor!
user! Many docs! All Shards! Targeted Some users affected Good!
user, time! Doc level! All Shards! Targeted! Some users affected Good!
Conclusion
Conclusion
• MongoDB is a general purpose database
• Document Design
• Create Index and check the explain plans
• MongoDB is a distributed database
• Choose the shard key wisely
• MongoDB use replication
• Check OpLog and Replication Lag
• Monitor all these with tools
Questions?
@tgrall
@neotys
Thank You