mongo db at community engine
DESCRIPTION
This presentation was delivered by community engine Lead Platform Engineer Mathieu Kempe at the recent Mongo DB conference in Sydney.TRANSCRIPT
![Page 1: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/1.jpg)
MongoDB at
Community Engine
![Page 3: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/3.jpg)
Agenda
Brain dump on our experience with MongoDB• Why NoSQL• Why we chose MongoDB• Moving away from an hybrid solution• MongoDB and Amazon• SOLR and MongoDB• Ease of development• Zero downtime database deployment
![Page 4: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/4.jpg)
About Community Engine• Social network based on locality and
small business• Launching in April• Built on:
• ASP.NET MVC 3• Amazon Web services• MongoDB• SOLR• Mahout• …
![Page 5: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/5.jpg)
NoSQL?
• How to store the big amounts of data required in social networking applications
• Data complexity, NoSQL handle hierarchical and graph data structures better
• Change management is always difficult with RDBMS
• Scaling
![Page 6: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/6.jpg)
Why we chose MongoDB?
• Reviewed different products RavenDB, CouchDB...• Selected MongoDB because we had the best
experience– Very easy to install and get started– Great developer experience– Replication very easy to setup– Good documentation– Much of the convenience of SQL, Dynamic
Queries, Indexing
![Page 7: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/7.jpg)
![Page 8: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/8.jpg)
Why Hybrid?
• Team had a lot of experience with SQL Server and Entity Framework
• Reporting• Transaction
![Page 9: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/9.jpg)
![Page 10: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/10.jpg)
No more SQL Server• Simplify our infrastructure• Easier to Backup• Better performance, not slowed down by SQL
Server, too many queries joined in the application
• Development speed• Lower cost
![Page 11: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/11.jpg)
Transaction?
Transactions we could go around that using the atomic document updates and a good schema design
MongoDB supports atomic operations on single document.
When transactions across documents are neededTwo phase commits
![Page 12: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/12.jpg)
Hosting with Amazon Web Service
• Elasticity and scalability• Configure MongoDB using Amazon EC2 instance
bundled into an AMI.• 64 bits EC2 instance• Raid10 + EBS volumes• Multi-datacenter 3-node replica set in different
availability zone• Use secondaries for zero downtime backup• We are not yet using sharded replica sets
![Page 13: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/13.jpg)
![Page 14: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/14.jpg)
What about durability?
Use journalingUse replica sets
![Page 15: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/15.jpg)
Critical writes
Verify that replication is working at write time
mongodb://host1,host2,host3/?safe=true;w=2;
•safe=true : Use safemode•w=2: wmode, connect to a replica set waiting for replication to succeed on the majority of nodes
![Page 16: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/16.jpg)
![Page 17: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/17.jpg)
![Page 18: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/18.jpg)
![Page 19: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/19.jpg)
![Page 20: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/20.jpg)
![Page 21: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/21.jpg)
Why we kept SOLR?
• Right tool for the right job• Proven technology• SOLR best solution for Full Text Indexing• Faceted search, Spelling suggestions… • Team already skilled with SOLR• SOLR scales well
![Page 22: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/22.jpg)
MongoDB/SOLR How we do it
![Page 23: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/23.jpg)
MongoDB/SOLR How we do it
![Page 24: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/24.jpg)
MongoDB/SOLR How we do it
![Page 25: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/25.jpg)
MongoDB/SOLR How we do it
![Page 26: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/26.jpg)
MongoDB/SOLR How we do it
![Page 27: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/27.jpg)
Ease of development
![Page 28: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/28.jpg)
Hierarchical data in SQL Server
![Page 29: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/29.jpg)
Single table
![Page 30: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/30.jpg)
Mongo Database Schema
Using Type discriminator
{"_id" : ObjectId("4f504e7acd3e1c190ce04198"),"_t" : "PhotoSpark","Photo" : "MyMotorcycle.png ""DateCreated" : ISODate("2012-02-24T09:23:12.246Z")
}{
"_id" : ObjectId("4f504e7ccd3e1c190ce04199"),"_t" : "PostSpark","Body" : "Hello World“" DateCreated" : ISODate("2012-02-28T10:44:12.858Z")
}
![Page 31: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/31.jpg)
![Page 32: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/32.jpg)
![Page 33: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/33.jpg)
![Page 34: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/34.jpg)
Views
![Page 35: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/35.jpg)
Deployment of database with zero downtime
• We release every week• We aim at zero downtime• Our domain model change often
![Page 36: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/36.jpg)
Deployment of database with zero downtime
Make sure that our code can handle both "versions" of the data structure
When saving we updates to the new structure
![Page 37: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/37.jpg)
Deployment of database with zero downtime
• Use a migration script
![Page 38: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/38.jpg)
Expansion Script Deploy new version Compression Script
t
Deployment of database with zero downtime
![Page 39: Mongo DB at Community Engine](https://reader033.vdocument.in/reader033/viewer/2022061110/5451ce5aaf7959ca018b8672/html5/thumbnails/39.jpg)
Questions?