mongodb days uk: scaling mongodb with docker and cgroups
TRANSCRIPT
Scaling MongoDB with Docker and cgroups
Marco Bonezzi Technical Services Engineer, MongoDB
@marcobonezzi
2
Agenda
• MongoDBandWiredTiger
• WhyshouldyouuseDockerwithMongoDB?
• ScalingMongoDBwithDocker
• Results
PS
S
config
SS
PS
S
SHARD2
SHARD1
mongos
MongoDB and WiredTiger
5
MongoDB3.0andWiredTiger
MongoDBStorageEngineAPI+WiredTigerstorageengineMainfeatures• Document-levelconcurrencycontrol• Durability• Compression• EncrypHonatrest(3.2)
6
WiredTigerincreasedscalability
• MulH-corescaling
• Lock-freealgorithms(hazard-pointers)
• Noin-placeupdate(skiplists)
• Eliminateblockingwithconcurrencycontrol
7
WiredTigerincreasedscalability
• Higherthroughputonwriteandupdates(skiplists)
8
Bestprac>cesforWiredTiger
• XFSfilesystem
• #concurrentac+veopera+ons• Bestthroughputwhen<=#CPU
• MemoryandWiredTigercache
• storage.wiredTiger.engineConfig.cacheSizeGB
9
MongoDBMemoryandWiredTigercache
cacheSizeGB=max(1GB,0.5*memory)
WiredTigercache: Usedfordataandindexesonly
10
MongoDBMemoryandWiredTigercache
WiredTigercache: Usedtostorethecurrentworkingset
cacheSizeGB=max(1GB,0.5*memory)
11
MongoDBMemoryandWiredTigercache
MongoDBMemory:mongodprocess:connecHons,aggregaHons,mapReduce,etc
cacheSizeGB=max(1GB,0.5*memory)
Totalmemory=mem(mongod)+cacheSizeGB
12
ContainerizingMongoDB
• MongoDBdockerfile(orcreateyourown)hPps://github.com/sisteming/mongodocker/[mongodb.docker]
• Genericmongod.confconfiguraHonfile• Automateyourdeploymentforscaling:
• #mongods• Portsandnetworkinterfaces• SingleormulHplehosts• CPUormemorylimits
13
Summary
• MongoDBwithWiredTiger:• Higherthroughputwithincreasedscalability• AddiHonalfeatures:compression,encrypHon• WiredTigercachememory
• MongoDBonDocker:• Genericimage• Automatetoscale
Why should you use Docker for MongoDB?
15
DockerAccordingtodocker.com:
HowcanthisbeusefulformyMongoDBdeployments?
(That’swhyyouareheretoday)
16
WhyshouldyouuseDockerforMongoDB?
Build• anddefineyour
containerbasedonOS,mongodbreleaseorconfiguraHon
• Buildonce,deployeverywhere
17
WhyshouldyouuseDockerforMongoDB?
Ship• DeploythesameimageinDev,Pre-prod(or
evenProd!)withthesamesetup• MongoDBstandalone,ReplicaSetsorcluster
withmulHpleshardsfromthesameimage
18
WhyshouldyouuseDockerforMongoDB?
Run• YourmongoDBprocessesisolatedfromthe
restofprocessesonthesystem• Isolatetheresourcesforco-locatedmongodb
processes
19
VS
20
Resourcemanagement
CgroupsImplementaHononDocker
21
Resourcemanagement
CgroupsImplementaHononDocker
--cpu-shares--cpuset-cpus
--memory
--blkio-weight
--net
22
Summary
UsingDockerforMongoDBisusefulformanyreasons:
• Buildonce,deployeverywhere• Environmentportability• MongodprocessisolaHon• Fasterdeploymentsofcomplexclusters• Resourcemanagementwithcgroups
Scaling MongoDB with Docker
config
SS
PS
S
PS
S
SHARD2
SHARD1
mongos
24
UsecasesofMongoDBandDockerMicrosharding
• MulHpleco-locatedmongodinstances
• Eachmongod=shard• AggregaHons
Mul>pleinstances• Controlco-locatedmongod
processesonsameserver• Differentinstances/uses
25
Memorylimit
MEM=($TOTAL_MEMORY-2048)/$NUM_MONGODWiredTigercache=$MEM/2
26
CPUlimit
NUMBER=from0toMAX_CPU_CORES(i.e.with32cores->--cpuset=0,16)
27
Docker+cgroups:Memorymapping
WiredTigercachesize:definedforeachshard50%MongoDBmemory–50%WiredTigercache
28
MongoDBonDocker+cgroups:CPUmapping
Eachshardmappedto1(ormore)core(using--cpuset-cpus)
core0 core1 core2 core3
core4 core5 core6 core7
core8 core9 core10 core11
core12 core13 core14 core15
29
ResourceusagewithDocker• Understandingresourceusage:• ContainerstatsavailablethroughDockerremoteAPI:
GET/containers/(id)/statsecho-e"GET/containers/$id/statsHTTP/1.0\r\n"|
nc-U/var/run/docker.sock
30
Summary
• UsesofscalingMongoDBwithDocker
• LimiHngCPUonyourcontainer
• DefiningmemoryandWiredTigercachelimits
• ResourceusagewithDocker
Results
32
TargetandscenariosScalabilitybymicroshardingMongoDBwith
WiredTigerusingdocker(andcgroups)
• Maintestscenarios:– ShardedclusterwithP/S/SReplicaSet
• From2upto16shards– Eachscenario:
• numberofthreads:8,16,32• Write/ReadraHo:95/5• memoryandWiredTigercache
33
Workload
POCDriver (https://github.com/johnlpage/POCDriver) • Easy to use
– java -jar POCDriver.jar -d 610 -c mongoDB://node1:37019 -o test.log -i 95 -t 10 -k 5 –e
• Automatic collection sharding with even distribution
34
Workloadresultssummary
ScenarioAMEM=2GB
WiredTigercache=1GB
ScenarioBMEM=32GB/$NUM_SHARDS
WiredTigercache=$MEM/2(GB)
35
TotalInserts
36
Inserts/second
37
Workloadresultssummary
3xc3x4largeAWSinstances:16cores32GBram
SSDvolumes
0
10
20
30
40
50
0 2 4 6 8 10 12 14 16 18
Millions
TotalInsertsvs#shards
TotalInserts
0
10
20
30
40
50
0 2 4 6 8 10 12 14 16 18
Millions
TotalInsertsvs#shards
TotalInserts
ScenarioAMEM=2GB
WiredTigercache=1GB
ScenarioBMEM=32GB/$NUM_SHARDS
WiredTigercache=$MEM/2(GB)
Summary
39
Keys for scaling MongoDB and WiredTiger with Docker
• Startfromagenericimage,automatetoscale
• Setlimitstoallyourcontainers
• Define–WiredTigerCacheSizeGBoneachmongod
• EsHmateanddefineresourcedistribuHonforyourarchitecture
• Deployandtestwithyourownworkload
#MDBDaysmongodb.com
Getyourtechnicalques+onsansweredBenjaminBri3enlounge(3rdfloor),10:00-17:00Byappointmentonly–registerinperson