migrating to mongodb: best practices
DESCRIPTION
Are you in the process of evaluating or migrating to MongoDB? We will cover key aspects of migrating to MongoDB from a RDBMS, including Schema design, Indexing strategies, Data migration approaches as your implementation reaches various SDLC stages, Achieving operational agility through MongoDB Management Services (MMS).TRANSCRIPT
![Page 1: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/1.jpg)
Migrating to MongoDBBest Practices
Muthu ChinnasamySenior Solutions Architect
![Page 2: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/2.jpg)
Agenda
• Project Team
• Schema Design
• Application Integration
• Data Migration Options
• Ops considerations
![Page 3: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/3.jpg)
Why MongoDB
• Rich documents
• Promote business agility
• Achieve higher scalability
• Lower budget strain compared to RDBMS
![Page 4: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/4.jpg)
RDBMS to MongoDB – Success stories
![Page 5: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/5.jpg)
Project Team
![Page 6: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/6.jpg)
Organizing for Success - Stakeholders
• Key to success: Involve all key stakeholders for the applicationo Line of businesso Developerso Data Architectso DBAso Systems Administratorso Security
![Page 7: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/7.jpg)
Organizing for Success – Project Charter
• Develop project chartero Define business and technical objectiveso Define timeliness and responsibilitieso Monitor progress and address any issues
![Page 8: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/8.jpg)
Organizing for Success – Help needed?
• Partner services and resources available from MongoDBo Community supporto Build skills and proficiency through web based trainingo Support and consulting services
![Page 9: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/9.jpg)
Schema Design
![Page 10: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/10.jpg)
Definitions
RDBMS MongoDB
Table Collection
Row Document
Column Field
Index Index
JOIN Embedded Document or Reference
![Page 11: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/11.jpg)
Document Model Benefits
RDBMS MongoDB
{
_id : ObjectId("4c4ba5e5e8aabf3"),
employee_name: "Dunham, Justin",
department : "Marketing",
title : "Product Manager, Web",
report_up: "Neray, Graham",
pay_band: “C",
benefits : [
{ type : "Health",
plan : "PPO Plus" },
{ type : "Dental",
plan : "Standard" }
]
}
![Page 12: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/12.jpg)
Schema Design – Blogging Platform
![Page 13: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/13.jpg)
Schema Design - Indexing
• Compound Indexes• Unique indexes• Array Indexes• Text Indexes• Geospatial indexes• Sparse Indexes
![Page 14: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/14.jpg)
Schema Design – For more details
• Afternoon session: Data modeling deep dive
• Google: "6 rules of thumb for MongoDB schema design"
• Google: "MongoDB compound index optimization"
![Page 15: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/15.jpg)
Application Integration
![Page 16: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/16.jpg)
Morphia
MEAN Stack
Java Python PerlRuby
MongoDB API is implemented as methods
Drivers & Ecosystem
![Page 17: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/17.jpg)
Developer Efficiency
Object-Relational impedance?
Alter 2TB table to modify a column?
MongoDB APIs are classes and packages
Dynamic schema
Rigid schema
Modify code to use MongoDB APIs
RDBMS
MongoDB
![Page 18: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/18.jpg)
Data Migration
![Page 19: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/19.jpg)
Data Migration
Source DatabaseSource
Database
ApplicationApplication
![Page 20: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/20.jpg)
Data Migration – Can you have downtime?
Source DatabaseSource
DatabaseMasterMaster ExportingExporting
ImportingImporting MasterMaster
AvailableAvailable DegradedDegraded DownDown AvailableAvailableApplication View
Time
T1
T2
T3
![Page 21: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/21.jpg)
Data Migration – mongoimport$ mongoimport --db test --collection customers < customers.json connected to: 127.0.0.12014-11-26T08:36:47.509-0800 imported 1000 objects$ mongoMongoDB shell version: 2.6.5connecting to: test> db.customers.findOne(){
"_id" : 363862536,"first_name" : "Landon”,"last_name" : "Moore","created_date" : ISODate("2010-03-02T22:48:35Z"),"is_active" : true,"phone" : [
{ "type" : "Work”, "number" : "683-560-1311” },{ "type" : "Other”, "number" : "437-849-4219” }
],"address" : {
"street_number" : 14,"street" : "Granite","street_type" : "Way","city" : "New Jersey","zip_code" : 96881
},"company" : ”Example"
}
![Page 22: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/22.jpg)
Data Migration – ETL tools
Source DatabaseSource Database ETLETL
![Page 23: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/23.jpg)
Data Migration – Hadoop
Source DatabaseSource Database
jobjobjobjob
jobjobjobjob
![Page 24: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/24.jpg)
App Driven Migration
Source DatabaseSource Database
ApplicationApplication
![Page 25: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/25.jpg)
Data Migration - Options
Source DatabaseSource Database
SnapshotSnapshot
Continuous Sync
Batch Migration
Batch Migration
ApplicationApplication
Application Managed
![Page 26: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/26.jpg)
Case Study
![Page 27: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/27.jpg)
Case Study
Uses MongoDB to safeguard over 6 billion images served to millions of customers
Problem Why MongoDB Results
• 6B images, 20TB of data
• Brittle code base on top of Oracle database – hard to scale, add features
• High SW and HW costs
• JSON-based data model
• Agile, high performance, scalable
• Alignment with Shutterfly’s services-based architecture
• 80% cost reduction
• 900% performance improvement
• Faster time-to-market
• Dev. cycles in weeks vs. tens of months
![Page 28: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/28.jpg)
• Meta data stored in XML Blobs• App responsible for content of blob
Shutterfly – Original Data store
OracleOracle
Photo ID XML Blob
1 <xml><meta-data>…</xml>
2 <xml><meta-data>…</xml>
3 <xml><meta-data>…</xml>
![Page 29: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/29.jpg)
Schema Migration – Initial
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
![Page 30: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/30.jpg)
Schema Migration – Phase 1
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
![Page 31: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/31.jpg)
Schema Migration – Phase 2
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
<?xml version="1.0" encoding="utf16"?><votes> <voteItem user="00000000" vote="1" /> <voteItem user="11111111" vote="1" /> <voteItem user="22222222" vote="1" /></votes>
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "data" : "<?xml version=\"1.0\" encoding=\"utf16\"?> <votes> <voteItem user=\"00000000\" vote=\"1\" /> <voteItem user=\"11111111\" vote=\"1\" /> <voteItem user=\"22222222\" vote=\"1\" /> </votes>" }
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "votes" : { 000000000:1, 111111111:1, 222222222:1 }}
{ _id : "site/the3colbys/3326/_votes", "V" : 0, "cD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "wD" : "Thu Sep 23 2010 20:38:54 GMT-0700 (PDT)", "md5" : "71199d82ee730f271feface722a74d30", "votes" : { 000000000:1, 111111111:1, 222222222:1 }}
![Page 32: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/32.jpg)
Data Migration – Application driven
1. Request for photo
2. Try to read from MongoDB
3. If cache miss, read from Oracle
4. Translate document & write to MongoDB
5. Return to clientSource DatabaseSource
Database
ApplicationApplication
11
2233
44
55
![Page 33: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/33.jpg)
Ops Considerations
![Page 34: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/34.jpg)
Replica Sets - No downtime maintenances
Replica set provides ops agility & HA
•Database upgrades
•Hardware swaps/maintenance
•Maintenance operations
•Automatic failover
![Page 35: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/35.jpg)
MongoDB Management Service (MMS)
Provision
Upgrade
Scale
Continuous Backup
Point-in-Time Recovery
Alerts
1. Automation 2. Backups 3. Monitoring
Cloud Managed MongoDB
![Page 36: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/36.jpg)
36
Defense in Depth Security Architecture
Authentication•Database •LDAP •Kerberos•x.509 Certificates
Authorization•Built-in Roles•User-Defined Roles•Field-Level Redaction
Auditing•Admin operations•Queries
Encryption•Network : SSL•Disk: Partner solutions
![Page 37: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/37.jpg)
Help available from MongoDBMongoDB Enterprise AdvancedThe best way to run MongoDB in your data center
MongoDB Management Service (MMS)The easiest way to run MongoDB in the cloud
Production SupportIn production and under control
Development SupportLet’s get you running
ConsultingWe solve problems
TrainingGet your teams up to speed.
![Page 38: Migrating to MongoDB: Best Practices](https://reader034.vdocument.in/reader034/viewer/2022050922/55980e021a28ab5f7c8b45af/html5/thumbnails/38.jpg)