![Page 1: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/1.jpg)
Optimising for performance, scale, analytics
![Page 2: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/2.jpg)
~3333 write ops/s 0.07 - 0.05 ms response
![Page 3: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/3.jpg)
David Mytton
Woop Japan!
![Page 4: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/4.jpg)
![Page 5: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/5.jpg)
MongoDB at Server Density
![Page 6: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/6.jpg)
•27 nodes
MongoDB at Server Density
![Page 7: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/7.jpg)
• June 2009 - +3yrs
•27 nodes
MongoDB at Server Density
![Page 8: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/8.jpg)
•MySQL -> MongoDB
•27 nodes
MongoDB at Server Density
• June 2009 - +3yrs
![Page 9: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/9.jpg)
•MySQL -> MongoDB
•17TB data per month
•27 nodes
MongoDB at Server Density
• June 2009 - +3yrs
![Page 10: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/10.jpg)
MongoDB at Server Density
Queues
Primary data store
Time series
![Page 11: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/11.jpg)
Why?
![Page 12: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/12.jpg)
Why?
• Replication
![Page 13: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/13.jpg)
Why?
• Replication
• Official drivers
![Page 14: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/14.jpg)
Why?
• Replication
• Official drivers
• Easy deployment
![Page 15: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/15.jpg)
Why?
• Replication
• Official drivers
• Easy deployment
• Fast out of the box (sort of)
![Page 16: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/16.jpg)
Fast out of the box?
Photo: dannychoo.com
![Page 17: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/17.jpg)
~3333 write ops/s 0.07 - 0.05 ms response
![Page 18: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/18.jpg)
Fast out of the box?
Photo: dannychoo.com
• Softlayer cloud (1 core, 8GB)
![Page 19: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/19.jpg)
Fast out of the box?
Photo: dannychoo.com
• Local instance storage
• Softlayer cloud (1 core, 8GB)
![Page 20: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/20.jpg)
Fast out of the box?
Photo: dannychoo.com
• Local instance storage
• Ubuntu 10/12.04 LTS
• Softlayer cloud (1 core, 8GB)
![Page 21: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/21.jpg)
![Page 22: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/22.jpg)
Fast out of the box?
Photo: dannychoo.com
• Journaling
![Page 23: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/23.jpg)
Fast out of the box?
Photo: dannychoo.com
• Journaling
• Replication
![Page 24: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/24.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
![Page 25: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/25.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
![Page 26: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/26.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
![Page 27: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/27.jpg)
mongos> db.metrics_20120508_15_1m.stats(){ "sharded" : true, "flags" : 1, "ns" : "metrics.metrics_20120508_15_1m", "count" : 2752934, "numExtents" : 46,
"size" : 746837640, "storageSize" : 823717888,
"totalIndexSize" : 517581680, "indexSizes" : { "_id_" : 130358144, "a_1_i_1" : 155711920, "a_1_i_1_m_1_t_1" : 231511616 }, "avgObjSize" : 271.2878841265355, "nindexes" : 3, "nchunks" : 61,
![Page 28: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/28.jpg)
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
![Page 29: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/29.jpg)
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
![Page 30: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/30.jpg)
"size" : 746837640, "totalIndexSize" : 517581680
Indexes = 493MB Data = 712MB
Total = 1205MB
![Page 31: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/31.jpg)
http://www.flickr.com/photos/comedynose/4388430444/
Where should it go?
What? Should it be in memory?
Indexes Always
Data If you can
![Page 32: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/32.jpg)
How you’ll know
1) Slow queries
Thu Oct 14 17:01:11 [conn7410] update sd.apiLog query: { c: "android/setDeviceToken", a: 1466, u: "blah", ua: "Server Density Android" } 51926ms
www.flickr.com/photos/tonivc/2283676770/
![Page 33: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/33.jpg)
How you’ll know
2) Timeouts
cursor timed out (20000 ms)
![Page 34: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/34.jpg)
How you’ll know
3) Disk i/o spikes
www.flickr.com/photos/daddo83/3406962115/
![Page 35: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/35.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
![Page 36: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/36.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• Fast disks (optional)
![Page 37: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/37.jpg)
![Page 38: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/38.jpg)
![Page 39: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/39.jpg)
Growing documents is bad
![Page 40: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/40.jpg)
Updates in place
db.my_collection.update( { _id : ... }, { $inc : { y : 2 } } )
![Page 41: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/41.jpg)
BSON
![Page 42: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/42.jpg)
BSON
![Page 43: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/43.jpg)
Growing documents is bad
> db.coll.stats(){ "ns" : "...", ..., "paddingFactor" : 1, ..., "ok" : 1}
![Page 44: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/44.jpg)
• Global DB lock
Scaling writes
![Page 45: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/45.jpg)
• Concurrency
Scaling writes
• Global DB lock
![Page 46: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/46.jpg)
http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
![Page 47: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/47.jpg)
http://blog.pythonisito.com/2011/12/mongodbs-write-lock.html
![Page 49: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/49.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• Fast disks (optional)
• Sharding (optional)
![Page 50: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/50.jpg)
• Sharding
• Concurrency
Scaling writes
• Global DB lock
![Page 51: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/51.jpg)
Scaling writes
![Page 52: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/52.jpg)
![Page 53: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/53.jpg)
• Collection location
Scaling writes
![Page 54: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/54.jpg)
• Collection location
Scaling writes
• Pre-split / moveChunk
![Page 55: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/55.jpg)
• Collection location
Scaling writes
• Hashing (v2.4)
• Pre-split / moveChunk
![Page 56: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/56.jpg)
Failover
•Replica sets
![Page 57: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/57.jpg)
Failover
•Replica sets
•Master/slave
![Page 58: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/58.jpg)
Failover
•Replica sets
•Min 3 nodes
•Master/slave
![Page 59: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/59.jpg)
Failover
•Replica sets
•Min 3 nodes
•Master/slave
•Automatic failover
![Page 60: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/60.jpg)
rs.status()
www.ex-astris-scientia.org/inconsistencies/ent_vs_tng.htm (yes it’s a replicator from Star Trek)
{! "_id" : 1,! "name" : "rs3b:27018",! "health" : 1,! "state" : 2,! "stateStr" : "SECONDARY",! "uptime" : 1886098,! "optime" : {! ! "t" : 1291252178000,! ! "i" : 13! },! "optimeDate" : ISODate("2010-12-02T01:09:38Z"), "lastHeartbeat" : ISODate("2010-12-02T01:09:38Z")},
![Page 61: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/61.jpg)
1) myState
rs.status()
Value Meaning0 Starting up (phase 1)1 Primary2 Secondary3 Recovering4 Fatal error5 Starting up (phase 2)6 Unknown state7 Arbiter8 Down
![Page 62: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/62.jpg)
2) Optime
rs.status()
www.flickr.com/photos/robbie73/4244846566/
"optimeDate" : ISODate("2010-12-02T01:09:38Z")
![Page 63: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/63.jpg)
3) Heartbeat
rs.status()
www.flickr.com/photos/drawblindfaith/3400981091/
"lastHeartbeat" : ISODate("2010-12-02T01:09:38Z")
![Page 64: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/64.jpg)
Scaling reads
![Page 65: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/65.jpg)
Scaling reads
•Replica slaves
![Page 66: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/66.jpg)
Scaling reads
•Replica slaves
•Consistency
![Page 67: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/67.jpg)
•Replica slaves
Scaling reads
•Consistency
•w flag / tags
![Page 68: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/68.jpg)
WriteConcern
Changed Nov 27 2012
![Page 69: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/69.jpg)
• Safe by default
WriteConcern
>>> from pymongo import MongoClient>>> connection = MongoClient()
![Page 70: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/70.jpg)
• Safe by default
WriteConcern
>>> from pymongo import MongoClient>>> connection = MongoClient(w=int/str)
Value Meaning
0 Unsafe
1 Primary
2 Primary + x1 secondary
3 Primary + x2 secondaries
![Page 71: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/71.jpg)
Tags
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "veryImportant"})
![Page 72: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/72.jpg)
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "veryImportant"})
(A or B) + (C or D) + E
Tags
![Page 73: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/73.jpg)
{ _id : "someSet", members : [ {_id : 0, host : "A", tags : {"dc": "ny"}}, {_id : 1, host : "B", tags : {"dc": "ny"}}, {_id : 2, host : "C", tags : {"dc": "sf"}}, {_id : 3, host : "D", tags : {"dc": "sf"}}, {_id : 4, host : "E", tags : {"dc": "cloud"}} ] settings : { getLastErrorModes : { veryImportant : {"dc" : 3}, sortOfImportant : {"dc" : 2} } }}> db.foo.insert({x:1})> db.runCommand({getLastError : 1, w : "sortOfImportant"})
(A + C) or (D + E) ...
Tags
![Page 74: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/74.jpg)
• J
WriteConcern
• Safe by default
![Page 75: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/75.jpg)
• J
• fsync
WriteConcern
• Safe by default
![Page 76: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/76.jpg)
Bottlenecks
•EC2
![Page 77: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/77.jpg)
Bottlenecks
•EC2
•Local storage
![Page 78: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/78.jpg)
Bottlenecks
•EC2
•EBS: RAID10 4-8 volumes
•Local storage
![Page 79: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/79.jpg)
Bottlenecks
•EC2
•EBS: RAID10 4-8 volumes
•Local storage
•i/o: rand but not sequential
![Page 80: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/80.jpg)
http://www.slideshare.net/jrosoff/mongodb-on-ec2-and-ebs
![Page 81: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/81.jpg)
http://bit.ly/ec2mongodb
![Page 82: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/82.jpg)
Bottlenecks
•CPU
•Index building
![Page 83: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/83.jpg)
Tips: rand()
•_id
![Page 84: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/84.jpg)
Tips: rand()
•_id
•Field names
![Page 85: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/85.jpg)
Tips: rand()
•_id
•Field names
•Covered indexes
![Page 86: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/86.jpg)
Tips: rand()
•_id
•Field names
•Covered indexes
•Collections / databases
![Page 87: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/87.jpg)
mongostat
![Page 88: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/88.jpg)
Locks/Queues
mongostat
![Page 89: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/89.jpg)
Diagnostics
mongostat
![Page 90: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/90.jpg)
Current operations
www.flickr.com/photos/jeffhester/2784666811/
db.currentOp();{! ! ! "opid" : "shard1:299939199",! ! ! "active" : true,! ! ! "lockType" : "write",! ! ! "waitingForLock" : false,! ! ! "secs_running" : 15419,! ! ! "op" : "remove",! ! ! "ns" : "sd.metrics",! ! ! "query" : {! ! ! ! "accId" : 1391,! ! ! ! "tA" : {! ! ! ! ! "$lte" : ISODate("2010-11-24T19:53:00Z")! ! ! ! }! ! ! },! ! ! "client" : "10.121.12.228:44426",! ! ! "desc" : "conn"! ! },
![Page 91: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/91.jpg)
Monitoring tools
Run yourself
Ganglia
![Page 92: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/92.jpg)
Monitoring tools
Server Density
![Page 93: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/93.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
![Page 94: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/94.jpg)
Picture is unrelated! Mmm, ice cream.
Fast out of the box?
• Fast network
• Working set in RAM
• bit.ly/benchrun
![Page 95: MongoDB: Optimising for Performance, Scale & Analytics](https://reader034.vdocument.in/reader034/viewer/2022052622/558e7ec71a28aba50b8b45b9/html5/thumbnails/95.jpg)
Woop Japan!
www.serverdensity.com/mdb