Download - Sharding with MongoDB -- MongoDC 2012
![Page 1: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/1.jpg)
Sharding with MongoDB
Tyler Brock@TylerBrock
![Page 2: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/2.jpg)
Philosophy
Concepts
Architecture
Mechanics
![Page 3: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/3.jpg)
Philosophy
![Page 4: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/4.jpg)
Philosophy
![Page 5: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/5.jpg)
“Ruby is designed to make
programmers happy.”
Philosophy
![Page 6: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/6.jpg)
Philosophy
MongoDB is a database for developers.
![Page 7: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/7.jpg)
Build
Philosophy
![Page 8: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/8.jpg)
BuildScale
Philosophy
![Page 9: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/9.jpg)
How does MongoDB scale?
Philosophy
![Page 10: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/10.jpg)
> db.runCommand({enablesharding: "<dbname>" })
> db.runCommand({ shardcollection: "<namespace>", key: <shardkeypatternobject> })
Philosophy
![Page 11: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/11.jpg)
Concepts
![Page 12: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/12.jpg)
datastore
app
Read/Write
Simple Web Application
![Page 13: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/13.jpg)
What happens when your working set exceeds memory?
![Page 14: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/14.jpg)
What happens if your write load is enormous?
![Page 15: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/15.jpg)
datastore
app
Vertical Scaling
![Page 16: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/16.jpg)
app
Vertical Scaling
datastore
![Page 17: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/17.jpg)
app
Vertical Scaling
datastore
appapp
68 GB RamRaid10 EBS
![Page 18: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/18.jpg)
datastore
app
Vertical Scaling
appapp
512 GB RamRaid10 SSD
![Page 19: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/19.jpg)
Horizontal Scaling
![Page 20: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/20.jpg)
app
datastoredatastoredatastore
60gb
Horizontal Scaling
![Page 21: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/21.jpg)
app
datastoredatastore datastore
20gb 20gb 20gb
Horizontal Scaling
![Page 22: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/22.jpg)
Routing Logic
app
datastoredatastore datastore
20gb 20gb 20gb
metadata
Horizontal Scaling
![Page 23: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/23.jpg)
Routing Logic
app
datastoredatastore datastore
20gb 20gb
metadata
60gb
Horizontal Scaling
![Page 24: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/24.jpg)
app
Routing Logic
Balancer
datastoredatastore datastore
20gb 20gb
metadata
60gb
Horizontal Scaling
![Page 25: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/25.jpg)
app
Routing Logic
Balancer
datastoredatastore datastore
metadata
30gb 30gb 30gb
Horizontal Scaling
![Page 26: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/26.jpg)
Architecture
![Page 27: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/27.jpg)
Really is just a mongod (or replica set)Where your data lives
mongod
Shard
![Page 28: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/28.jpg)
Mongod started with --configsvr optionMust have 3 (or 1 in development)Data is commited using 2 phase commit
config
Config Server
![Page 29: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/29.jpg)
mongos
Acts just like shard router / proxyOne or as many as you wantLight weight -- can run on App serversCaches meta-data from config servers
mongos
![Page 30: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/30.jpg)
Routing Logic
Balancingmetadata
datastore datastoredatastore
![Page 31: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/31.jpg)
metadata
datastore
mongos
datastoredatastore
![Page 32: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/32.jpg)
metadata
datastore
mongos
datastoredatastore
app
![Page 33: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/33.jpg)
datastore
mongos
config
datastoredatastore
app
![Page 34: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/34.jpg)
datastore
mongos
config
datastoredatastore
config
config
app
![Page 35: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/35.jpg)
mongos
config
mongod mongodmongod
config
config
app
![Page 36: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/36.jpg)
mongos
config
mongod mongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
config
config
app
![Page 37: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/37.jpg)
mongos
config
mongod mongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
config
config
app
![Page 38: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/38.jpg)
Configuration
![Page 39: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/39.jpg)
mongodmongodmongod
Bring up mongods or Replica Sets
mongod mongodmongod
mongod mongodmongod
RS RS RS
mongod --shardsvrmongod --replSet --shardsvr
![Page 40: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/40.jpg)
config
mongodmongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
Bring up Config Servers
config
config
mongod --configsvr
![Page 41: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/41.jpg)
config
mongodmongodmongod
mongod mongodmongod
mongod mongodmongod
RS RS RS
Bring up Mongos
config
config
mongos
mongos --configdb <list of configdb uris>
![Page 42: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/42.jpg)
> use admin> db.runCommand({"addShard": <shard uri>})
Connect to Mongos+ Add Shards
+Enable Sharding
> db.runCommand( { enablesharding : "<dbname>" } );
> db.runCommand( { shardcollection : "<namespace>", key : <key> });
+Shard a Collection
![Page 43: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/43.jpg)
Mechanics
![Page 44: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/44.jpg)
How does MongoDB balance my data?
![Page 45: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/45.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
Keys
test.users
![Page 46: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/46.jpg)
> db.runCommand({
})
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
shardcollection: “test.users”,
Keys
key: { email: 1 }
test.users
![Page 47: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/47.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
shardcollection: “test.users”,
Keys
key: { email: 1 }
test.users
![Page 48: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/48.jpg)
{ name: “Joe”, email: “[email protected]”,},{ name: “Bob”, email: “[email protected]”,},{ name: “Tyler”, email: “[email protected]”,}
Keys
key: { email: 1 }
test.users
![Page 49: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/49.jpg)
Chunks
-∞ +∞
![Page 50: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/50.jpg)
Chunks
-∞ +∞
![Page 51: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/51.jpg)
Chunks
-∞ +∞
Split!
![Page 52: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/52.jpg)
Chunks
-∞ +∞
Split!This is a chunk
This is a chunk
![Page 53: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/53.jpg)
Chunks
-∞ +∞
![Page 54: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/54.jpg)
Chunks
-∞ +∞
![Page 55: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/55.jpg)
Chunks
-∞ +∞
Split!
![Page 56: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/56.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 57: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/57.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 58: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/58.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 59: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/59.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 60: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/60.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Split this big chunk into 2
chunks
![Page 61: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/61.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 62: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/62.jpg)
Splitting
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
These chunks have split
![Page 63: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/63.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 64: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/64.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move a chunk to
Shard2
![Page 65: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/65.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 66: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/66.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move another chunk
to Shard3
![Page 67: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/67.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 68: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/68.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
Shard1, move another chunk
to Shard4
![Page 69: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/69.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 70: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/70.jpg)
Balancing
config
config
config
mongos
Shard 1 Shard 2 Shard 3 Shard 4
![Page 71: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/71.jpg)
How does MongoDB route my queries?
![Page 72: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/72.jpg)
Routed Request
mongos
shard shard shard
![Page 73: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/73.jpg)
Routed Request1
mongos
shard shard shard
1. Query arrives at Mongos
![Page 74: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/74.jpg)
Routed Request1
2
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
![Page 75: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/75.jpg)
Routed Request1
2
3
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
3. Shard returns results of query
![Page 76: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/76.jpg)
Routed Request1
2
3
4
mongos
shard shard shard
1. Query arrives at Mongos
2. Mongos routes query to a single shard
3. Shard returns results of query
4. Results returned to client
![Page 77: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/77.jpg)
Scatter Gather Request
shard shard shard
mongos
![Page 78: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/78.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
shard shard shard
mongos
![Page 79: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/79.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
2 22
shard shard shard
mongos2. Mongos broadcasts queryto all shards
![Page 80: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/80.jpg)
Scatter Gather Request1
1. Query arrives at Mongos
2 22
3 33
shard shard shard
mongos2. Mongos broadcasts queryto all shards
3. Each shard returns resultsfor query
![Page 81: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/81.jpg)
Scatter Gather Request1
41. Query arrives at Mongos
2 22
3 33
shard shard shard
mongos2. Mongos broadcasts queryto all shards
3. Each shard returns resultsfor query
4. Results combined andreturned to client
![Page 82: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/82.jpg)
mongos
Distributed Merge Sort Req.
shard shard shard
![Page 83: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/83.jpg)
mongos
Distributed Merge Sort Req.1
shard shard shard
1. Query arrives at Mongos
![Page 84: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/84.jpg)
mongos
Distributed Merge Sort Req.1
22 2
shard shard shard
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
![Page 85: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/85.jpg)
mongos
Distributed Merge Sort Req.1
22 2
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
![Page 86: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/86.jpg)
mongos
Distributed Merge Sort Req.1
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
![Page 87: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/87.jpg)
mongos
Distributed Merge Sort Req.1
5
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
5. Mongos merges sorted results
![Page 88: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/88.jpg)
mongos
Distributed Merge Sort Req.1
6
5
22 2
4 44
shard shard shard3 3 3
1. Query arrives at Mongos
2. Mongos broadcasts query to all shards
3. Each shard locally sorts results
4. Results returned to mongos
5. Mongos merges sorted results
6. Combined results returned to client
![Page 89: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/89.jpg)
Queries
By Shard Key Routed db.users.find({email: “[email protected]”})
Sorted by shard key
Routed in order db.users.find().sort({email:-1})
Find by non shard key
Scatter Gather db.users.find({state:”NY”})
Sorted by non shard key
Distributed merge sort
db.users.find().sort({state:1})
![Page 90: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/90.jpg)
Writes
Inserts Requires shard key db.users.insert({ name: “Bob”, email: “[email protected]”})
Removes Routed db.users.delete({ email: “[email protected]”})
Removes
Scattered db.users.delete({name: “Bob”})
Updates Routed db.users.update( {email: “[email protected]”}, {$set: { state: “NY”}})
Updates
Scattered db.users.update( {state: “CA”}, {$set:{ state: “NY”}} )
![Page 91: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/91.jpg)
How do I choose my shard key?
![Page 92: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/92.jpg)
Choose a field that is common to your queries.
Rule of Thumb
![Page 93: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/93.jpg)
Cardinality
Chunks should be able to split.
![Page 94: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/94.jpg)
Bad {node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Chunks should be able to split
![Page 95: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/95.jpg)
Better {node:1, time:1}
Bad {node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Chunks should be able to split
![Page 96: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/96.jpg)
Write Scaling
Writes should be distributed.
![Page 97: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/97.jpg)
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Bad { time : 1 }
Writes should be distributed
![Page 98: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/98.jpg)
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken"}
Bad { time : 1 }
Better {node:1, application:1, time:1}
Writes should be distributed
![Page 99: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/99.jpg)
Query Isolation & Data Locality
Queries should be routed to one shard.
![Page 100: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/100.jpg)
Bad {msg: 1, node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken”}
Queries should be routed to one shard
![Page 101: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/101.jpg)
Better {node: 1, time: 1}
Bad {msg: 1, node: 1}
{ node: "ny153.example.com", application: "apache", time: "2011-01-02T21:21:56Z", level: "ERROR", msg: "something is broken”}
Queries should be routed to one shard
![Page 102: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/102.jpg)
> db.runCommand({enablesharding: "<dbname>" })
> db.runCommand({ shardcollection: "<namespace>", key: <shardkeypatternobject> })
![Page 103: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/103.jpg)
Thanks!
![Page 104: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/104.jpg)
Extra Slides
![Page 105: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/105.jpg)
Config Servers
![Page 106: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/106.jpg)
Config Servers
mongod
![Page 107: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/107.jpg)
Config Servers
mongod
mongod
mongod
![Page 108: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/108.jpg)
mongoDB Scaling - Single Node
write
read
node_a1
![Page 109: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/109.jpg)
Read scaling - add Replicas
write
read
node_b1
node_a1
![Page 110: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/110.jpg)
Read scaling - add Replicas
write
read
node_c1
node_b1
node_a1
![Page 111: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/111.jpg)
Write scaling - Sharding
shard1
write
read
node_c1
node_b1
node_a1
![Page 112: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/112.jpg)
Write scaling - add shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
![Page 113: Sharding with MongoDB -- MongoDC 2012](https://reader036.vdocument.in/reader036/viewer/2022062312/554f731cb4c905bb178b5335/html5/thumbnails/113.jpg)
Write scaling - add shards
write
read
shard1
node_c1
node_b1
node_a1
shard2
node_c2
node_b2
node_a2
shard3
node_c3
node_b3
node_a3