mongo db parse-v17
DESCRIPTION
Parse powers over 250k mobile apps on MongoDB. Since we can have no prior knowledge of the schemas or usage patterns or scaling requirements for each app, we rely on advanced operational techniques and best practices for managing MongoDB and maintaining peak performance. In this talk we will talk about the unique challenges of this use case, such as dealing with cotenancy issues and how to automatically generate good indexes for each app. I will talk about some of the interesting operational techniques involved in managing hundreds of databases and millions of collections on MongoDB, as well as running large and growing database ecosystems on AWS. I will also talk about some of our favorite black belt tips for tuning and scaling MongoDB. And finally we will talk about how to recover from performance-impacting incidents gracefully, without making everything worse for yourself.TRANSCRIPT
Charity Majors @mipsytipsy
</>
Mobile Backend as a Service
Data Storage
Push Notifications
Analytics
Server-Side Code
300K
250K
200K
150K
100K
50K
02011 2012 2013 2014 2015
Apps Built On Parse
270,000+
250%developers
500%api requests
a love^Wless hateful storyMongoDB + Ops:
?MongoDB + Ops:
?
Reliability
Flexibility
Automation
1
2
3
Reliability
Flexibility
Automation
1
2
3
re · li · abil · i · ty (noun) \ri-ˌlī-ə-ˈbi-lə-tē\1. Through resiliency 2. Magic of the replica set
Replica Set
Pets?
No Pets!
Arbiters
Extra Votes
Spanning AZs
Spanning DCs
Non-voting
Hidden
Client Logic
More!
Planning for failure.
Reliability
Flexibility
Automation
1
2
3
When you change a schema, everythi ng breaks !
DATA MODEL FLEXIBILITY
DATA MODELWORKLOAD FLEXIBILITY
270kapps hosted on Parse
270kapps hosted on Parse
270kdifferent workloads
=
270kapps hosted on Parse
270kdifferent workloads
=Every app must be individually performant!
DEPLOYMENT
BACKUPS
RECOVERY
DISASTER RECOVERY
DRIVERS
PARTIAL FAILURES
PERFORMANCE TUNING
INDEXING PLAN
DESIGN FAILOVER PLAN
OPERATIONAL EXPERTISE
ONEre-usable solution
Reliability
Flexibility
Automation
1
2
3
Replica Set
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
VPC
+
PLEASE. SHARE YOUR TOOLS.
MongoDB Chef Cookbook github.com/ParsePlatform/Ops/tree/master/chef/cookbooks/mongodb
AWS Chef Cookbook (ec2 raid recipe) github.com/opscode-cookbooks/aws
Continuous Compaction blog.parse.com/2013/03/26/always-be-compacting/
Warmup Scripts blog.parse.com/2013/03/07/techniques-for-warming-up-mongodb/
Mongo Proxy github.com/xxxxxx
Flashback github.com/xxxxxx
NEW
NEW
Mongo Proxy url url url url -- naitik shah (& blog post url)
NEW
Flashback url url url url -- kai liu (& blog post url)
NEW
a love story[*]. The End
* mutual loathing, bitterness, mistrust. Love the one you’re with.
MongoDB + Ops:
“Which database do I hate the most?”
“Which database do I hate the most?”
“The one I’m using right now”
Charity Majors @mipsytipsy