mongodb on public clouds : architecture and options · title: 10 questions to ask (and answer) when...
TRANSCRIPT
![Page 1: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/1.jpg)
MongoDB on Public Clouds :
Architecture and Options
Dharshan RangegowdaFounder, ScaleGrid.io
@dharshanrg
Copyright © ScaleGrid.io
![Page 2: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/2.jpg)
Cluster Configuration
• Standalone
• Dev & test
• Replica set
• Read scaling
• Typically 2+1 or 3 nodes
• Shards
• Read and write scaling
Copyright © ScaleGrid.io
![Page 3: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/3.jpg)
High Availability
• Standalone vs. Replica set
• No SLA for standalone
instances• Availability zones (AZ)
• Multiple regions
• Availability set
• Multiple regions
Copyright © ScaleGrid.io
![Page 4: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/4.jpg)
Disaster Readiness
• Do you need to handle a
“datacenter down” scenario?
• Entire AWS/Azure region down
• Automatic or manual?
• Distribute replicas across
datacenters
• Security concerns
• Latency concerns
Copyright © ScaleGrid.io
![Page 5: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/5.jpg)
Backups
• Multiple ways to backup
• Mongodump & Mongorestore
• Volume/LVM snapshots
• MongoDB MMS
• EBS snapshots
• S3 upload
• Mongodump & restore
• Storage blob snapshots
Copyright © ScaleGrid.io
![Page 6: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/6.jpg)
Performance
• Choose the right
instance type & size
(R3, I2)
• Enable EBS
optimization
• EBS (magnetic or SSD)
• Provisioned IOPS
• Local SSD (not a
permanent store)
Copyright © ScaleGrid.io
![Page 7: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/7.jpg)
Performance
• Choose “Standard” over “Basic” instance type
• “D_v2 series” usually a good fit – memory optimized
• Disk• Standard disks (upto 500 IOPS)
• RAID for better performance!
• Local SSD (not a permanent store)
• Premium storage
Copyright © ScaleGrid.io
![Page 8: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/8.jpg)
Network
• Classic shared LAN
• SDN (Software defined network)
• Azure App Service Environment (ASE)
Copyright © ScaleGrid.io
![Page 9: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/9.jpg)
Security
• Firewall
• Who has access to the DB?
• AWS
• Security groups
• Azure
• VNET network security groups
Copyright © ScaleGrid.io
![Page 10: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/10.jpg)
Security
• SSL
• Use SSL to encrypt on wire data
• Important for geo distribution
• Disk encryption
• Encryption “at rest”
• Encryption for backups
• AWS
• Native support for encrypted EBS volumes
Copyright © ScaleGrid.io
![Page 11: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/11.jpg)
Scale Up
• Scale up (vertical)
• Larger instance sizes
• More CPU
• More memory
• Larger disks
• on AWS
• EBS snapshots
• on Azure
• Resize underlying blob
Copyright © ScaleGrid.io
![Page 12: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/12.jpg)
Scale Out
• Scale out (horizontal)
• Replica sets
• Add replicas for read scaling
• Doesn’t help with write scaling
• Can have a max of 50 nodes
• Shards
• Add shards for write scaling
• Shard rebalancing takes time
Copyright © ScaleGrid.io
![Page 13: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/13.jpg)
Maintenance Operations
• OS patching
• Periodic & on demand OS patching
• Compact
• Deletes & updates fragment the database
• MongoDB does not auto compact
• MongoDB version upgrade
• Minor versions & Major versions
• Test before your upgrade
• Upgrade server by server
Copyright © ScaleGrid.io
![Page 14: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/14.jpg)
Monitoring
• 24 x 7 monitoring
• OS metrics
• CPU
• Disk IOPS
• Mongo metrics
• Lock %, Read/Write tickets
• Connections
• Disk queues
• Replication delay
Copyright © ScaleGrid.io
![Page 15: MongoDB on Public Clouds : Architecture and Options · Title: 10 questions to ask (and answer) when hosting MongoDB on public clouds Author: DHARSHAN RANGEGOWDA Created Date: 5/2/2017](https://reader030.vdocument.in/reader030/viewer/2022041222/5e0c24da566cde02753d9d1c/html5/thumbnails/15.jpg)
Query Analysis
• Regularly monitor query performance
• Problem queries
• Frequent queries
• Slowest queries
• Queries scanning most documents
• Queries returning large number of results
• Queries causing high lock %
Copyright © ScaleGrid.io