mongodb for coder training (coding serbia 2013)
DESCRIPTION
Slides of my MongoDB Training given at Coding Serbia Conference on 18.10.2013 Agenda: 1. Introduction to NoSQL & MongoDB 2. Data manipulation: Learn how to CRUD with MongoDB 3. Indexing: Speed up your queries with MongoDB 4. MapReduce: Data aggregation with MongoDB 5. Aggregation Framework: Data aggregation done the MongoDB way 6. Replication: High Availability with MongoDB 7. Sharding: Scaling with MongoDBTRANSCRIPT
![Page 1: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/1.jpg)
Uwe Seileruweseiler
Training: MongoDB for Coder
![Page 2: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/2.jpg)
About me
Big Data Nerd
TravelpiratePhotography Enthusiast
Hadoop Trainer MongoDB Author
![Page 3: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/3.jpg)
About usis a bunch of…
Big Data Nerds Agile Ninjas Continuous Delivery Gurus
Enterprise Java Specialists Performance Geeks
Join us!
![Page 4: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/4.jpg)
Agenda I
1. Introduction to NoSQL & MongoDB
2. Data manipulation: Learn how to CRUD with MongoDB
3. Indexing: Speed up your queries with MongoDB
4. MapReduce: Data aggregation with MongoDB
![Page 5: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/5.jpg)
Agenda
5. Aggregation Framework: Data aggregation done the MongoDB way
6. Replication: High Availability with MongoDB
7. Sharding: Scaling with MongoDB
![Page 6: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/6.jpg)
Ingredients
• Slides
• Live Coding
• Discussion
• Labs on your own computer
![Page 7: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/7.jpg)
And please…
If you have questions, please share them with us!
![Page 8: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/8.jpg)
And now start your downloads…
Lab files: http://bit.ly/1aT8RXY
![Page 9: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/9.jpg)
Buzzword Bingo
![Page 10: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/10.jpg)
NoSQL
![Page 11: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/11.jpg)
Classification of NoSQL
Key-Value StoresK V
K V
K V
K V
K V
11 1 1
1 11 11
11
Column Stores
Graph Databases Document Stores
_id_id_id
![Page 12: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/12.jpg)
Big Data
![Page 13: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/13.jpg)
My favorite definition
![Page 14: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/14.jpg)
The classic definition
• The 3 V’s of Big Data
•VarietyVolume Velocity
![Page 15: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/15.jpg)
«Big Data» != Hadoop
![Page 16: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/16.jpg)
Horizontal Scaling
![Page 17: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/17.jpg)
Vertical Scaling
RAMCPU
Storage
![Page 18: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/18.jpg)
RAMCPU
Storage
Vertical Scaling
![Page 19: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/19.jpg)
RAMCPU
Storage
Vertical Scaling
![Page 20: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/20.jpg)
Horizontal Scaling
RAMCPU
Storage
![Page 21: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/21.jpg)
Horizontal Scaling
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
![Page 22: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/22.jpg)
RAMCPU
Storage
Horizontal Scaling
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
RAMCPU
Storage
![Page 23: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/23.jpg)
The problemwith
distributeddata
![Page 24: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/24.jpg)
Partition
Tolerancefailure of
single nodes doesn‘t effect
the overall system
Partition
Tolerancefailure of
single nodes doesn‘t effect
the overall system
The CAP Theorem
Consistency
all nodes see the same data
at the same time
Consistency
all nodes see the same data
at the same time
Availability
a guarantee that every
request receives a response
Availability
a guarantee that every
request receives a response
![Page 25: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/25.jpg)
Consistency
all nodes see the same data
at the same time
Consistency
all nodes see the same data
at the same time
Availability
a guarantee that every
request receives a response
Availability
a guarantee that every
request receives a response
Overview of NoSQL systems
Partition Tolerance
failure of single nodes doesn‘t effect
the overall system
Partition Tolerance
failure of single nodes doesn‘t effect
the overall system
![Page 26: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/26.jpg)
The problem with
consistency
![Page 27: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/27.jpg)
ACID
vs.
BASE
![Page 28: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/28.jpg)
ACID vs. BASE
Atomicity
Consistency
Isolation
Durability1983
RDBMS
![Page 29: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/29.jpg)
ACID is a goodconcept but it is not
a written law!
ACID vs. BASE
![Page 30: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/30.jpg)
BasicallyAvailable
Soft State
Eventually consistent2008
NoSQL
ACID vs. BASE
![Page 31: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/31.jpg)
ACID
- Strong consistency- Isolation & Transactions- Two-Phase-Commit- Complex Development- More reliable
BASE
- Eventual consistency- Highly Available- "Fire-and-forget"- Eases development- Faster
ACID vs. BASE
![Page 32: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/32.jpg)
Overview of MongoDB
![Page 33: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/33.jpg)
MongoDB is a…
• document
• open source
• highly performant
• flexible
• scalable
• highly available
• feature-rich…database
![Page 34: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/34.jpg)
Document Database
• Not PDF, Word, etc. … JSON!
![Page 35: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/35.jpg)
Open Source Database
• MongoDB is a open source project
• Available on GitHub– https://github.com/mongodb/mongo
• Uses the AGPL Lizenz
• Started and sponsored by MongoDB Inc. (prior: 10gen)
• Commercial version and support available
• Join the crowd!– https://jira.mongodb.org
![Page 36: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/36.jpg)
Datalocality
Performance
In-Memory Caching
In-Place Updates
![Page 37: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/37.jpg)
Flexible Schema
RDBMS MongoDB
{
_id :
ObjectId("4c4ba5e5e8aabf3"),
employee_name: "Dunham, Justin",
department : "Marketing",
title : "Product Manager, Web",
report_up: "Neray, Graham",
pay_band: “C",
benefits : [
{ type : "Health",
plan : "PPO Plus" },
{ type : "Dental",
plan : "Standard" }
]
}
![Page 38: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/38.jpg)
Scalability
Auto Sharding
• Increase capacity as you go
• Commodity and cloud architectures
• Improved operational simplicity and cost visibility
![Page 39: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/39.jpg)
High Availability
• Automated replication and failover
• Multi-data center support
• Improved operational simplicity (e.g., HW swaps)
• Data durability and consistency
![Page 40: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/40.jpg)
MongoDB Architecture
![Page 41: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/41.jpg)
Rich Query Language
![Page 42: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/42.jpg)
Aggregation Framework
![Page 43: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/43.jpg)
Map/Reduce
MongoDB DataGroup(k)
Sort(k)
Finalize(k, v)
Map()
emit(k,v)
Reduce(k, values)
Shard 1
Shard 2
Shard n
…
![Page 44: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/44.jpg)
Geo Information
![Page 45: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/45.jpg)
Driver & Shell
Shell to interact with the database
Drivers are available for almost all popular programming languages and frameworks
> db.collection.insert({product:“MongoDB”, type:“Document Database”})> > db.collection.findOne(){
“_id” : ObjectId(“5106c1c2fc629bfe52792e86”),“product” : “MongoDB”“type” : “Document Database”
}
Java
Python
Perl
Ruby
Haskell
JavaScript
![Page 46: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/46.jpg)
Indeed.com Trends
Top Job Trends
1.HTML 5
2.MongoDB
3.iOS
4.Android
5.Mobile Apps
6.Puppet
7.Hadoop
8.jQuery
9.PaaS
10.Social Media
NoSQL TrendsLinkedIn Job Skills
MongoDB
Competitor 1
Competitor 2
Competitor 3
Competitor 4
Competitor 5
All Others
Google Search
MongoDB
Competitor 1
Competitor 2
Competitor 3
Competitor 4
Jaspersoft Big Data Index
Direct Real-Time Downloads
MongoDB
Competitor 1
Competitor 2
Competitor 3
![Page 47: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/47.jpg)
Data manipulation
![Page 48: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/48.jpg)
RDBMS MongoDBTable / View ➜ CollectionRow ➜ DocumentIndex ➜ IndexJoin ➜ Embedded documentForeign Key ➜ Referenced documentPartition ➜ Shard
Terminology
![Page 49: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/49.jpg)
Example: Simple blog model
![Page 50: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/50.jpg)
MongoDB Collections
• User
• Article
• Tag
• Category
![Page 51: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/51.jpg)
Schema design for the blog
![Page 52: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/52.jpg)
Let’s have a look…
![Page 53: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/53.jpg)
// Show all databases> show dbsdigg 0.078125GBenron 1.49951171875GB
// Switch to a database> use blog
// Show all databases again> show dbsdigg 0.078125GBenron 1.49951171875GB
Create a database
![Page 54: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/54.jpg)
// Show all collections> show collections
// Insert a user> db.user.insert(
{ name : “Sheldon“, mail : “[email protected]“ }
)
Create a collection I
No feedback about the result of the insert, use:
db.runCommand( { getLastError: 1} )
![Page 55: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/55.jpg)
// Show all collections> show collectionssystem.indexesuser
// Show all databases> show dbsblog 0.0625GBdigg 0.078125GBenron 1.49951171875GB
Create a collection II
Databases and collections areautomatically created duringthe first insert operation!
![Page 56: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/56.jpg)
// Show the first document> db.user.findOne(){
"_id" : ObjectId("516684a32f391f3c2fcb80ed"),"name" : "Sheldon","mail" : "[email protected]"
}
// Show all documents of a collection> db.user.find(){
"_id" : ObjectId("516684a32f391f3c2fcb80ed"),"name" : "Sheldon","mail" : "[email protected]"
}
Read from a collection
![Page 57: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/57.jpg)
// Find a specific document> db.user.find( { name : ”Penny” } ){
"_id" : ObjectId("5166a9dc2f391f3c2fcb80f1"),"name" : "Penny","mail" : "[email protected]"
}
// Show only certain fields of the document> db.user.find( { name : ”Penny” },
{_id: 0, mail : 1} )
{ "mail" : "[email protected]" }
Find documents
![Page 58: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/58.jpg)
_id
• _id is the primary key in MongoDB
• _id is created automatically
• If not specified differently, it‘s type isObjectId
• _id can be specified by the user during theinsert of documents, but needs to beunique (and can not be edited afterwards)
![Page 59: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/59.jpg)
ObjectId
• A ObjectId is a special 12 Byte value
• It‘s uniqueness in the whole cluster is guaranteed as following:
ObjectId("50804d0bd94ccab2da652599")|-------------||---------||-----||----------|
ts mac pid inc
![Page 60: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/60.jpg)
// Use a cursor with find()> var myCursor = db.user.find( )
// Get the next document> var myDocument =
myCursor.hasNext() ? myCursor.next() : null;
> if (myDocument) { printjson(myDocument.mail); }
// Show all other documents> myCursor.forEach(printjson);
Cursor
By default the shell displays20 documents
![Page 61: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/61.jpg)
// Find documents using OR> db.user.find(
{$or : [ { name : “Sheldon“ }, { mail : [email protected] }
] } )
// Find documents using AND> db.user.find(
{$and : [ { name : “Sheldon“ }, { mail : [email protected] }
] } )
Logical operators
![Page 62: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/62.jpg)
// Sort documents> db.user.find().sort( { name : 1 } ) // Aufsteigend> db.user.find().sort( { name : -1 } ) // Absteigend
// Limit the number of documents> db.user.find().limit(3)
// Skip documents> db.user.find().skip(2)
// Combination of both methods> db.user.find().skip(2).limit(3)
Manipulating results
![Page 63: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/63.jpg)
// Updating only the mail address (How not to do…)> db.user.update( { name : “Sheldon“ },
{ mail : “[email protected]“ } )
// Result of the update operationdb.user.findOne(){
"_id" : ObjectId("516684a32f391f3c2fcb80ed"),"mail" : "[email protected]"
}
Updating documents I
Be careful when updatingdocuments!
![Page 64: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/64.jpg)
// Deleting a document> db.user.remove(
{ mail : “[email protected]“ } )
// Deleting all documents in a collection> db.user.remove()
// Use a condition to delete documents> db.user.remove(
{ mail : /.*mother.com$/ } )
// Delete only the first document using a condition> db.user.remove( { mail : /.*.com$/ }, true )
Deleting documents
![Page 65: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/65.jpg)
// Updating only the mail address (This time for real)> db.user.update( { name : “Sheldon“ },
{ $set : { mail : “[email protected]“
} } )
// Show the result of the update operationdb.user.find(name : “Sheldon“){
"_id" : ObjectId("5166ba122f391f3c2fcb80f5"),"mail" : "[email protected]","name" : "Sheldon"
}
Updating documents II
![Page 66: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/66.jpg)
// Adding a array> db.user.update( {name : “Sheldon“ },
{ $set : {enemies : [ { name : “Wil Wheaton“ },
{ name : “Barry Kripke“ } ]
} } )
// Adding a value to the array> db.user.update( { name : “Sheldon“},
{ $push : {enemies : { name : “Leslie Winkle“}
} } )
Adding to arrays
![Page 67: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/67.jpg)
// Deleting a value from an array> db.user.update( { name : “Sheldon“ },
{$pull : {enemies : {name : “Barry Kripke“ }
} } )
// Deleting of a complete array> db.user.update( {name : “Sheldon“},
{$unset : {enemies : 1}})
Deleting from arrays
![Page 68: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/68.jpg)
// Adding a subdocument to an existing document> db.user.update( { name : “Sheldon“}, {
$set : { mother :{ name : “Mary Cooper“, residence : “Galveston, Texas“, religion : “Evangelical Christian“ }}})
{"_id" : ObjectId("5166cf162f391f3c2fcb80f7"),"mail" : "[email protected]","mother" : {
"name" : "Mary Cooper","residence" : "Galveston, Texas","religion" : "Evangelical Christian"
},"name" : "Sheldon"
}
Adding a subdocument
![Page 69: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/69.jpg)
// Finding out the name of the mother> db.user.find( { name : “Sheldon“},
{“mother.name“ : 1 } )
{"_id" : ObjectId("5166cf162f391f3c2fcb80f7"),"mother" : {
"name" : "Mary Cooper"}
}
Querying subdocuments
Compound field names need tobe in “…“!
![Page 70: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/70.jpg)
For fields:$inc$rename$set$unset
Bitwise operation:$bit
Isolation:$isolated
Overview of all update operators
For arrays:$addToSet$pop$pullAll$pull$pushAll$push$each (Modifier)$slice (Modifier)$sort (Modifier)
![Page 71: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/71.jpg)
Createhttp://docs.mongodb.org/manual/core/create/
Readhttp://docs.mongodb.org/manual/core/read/
Updatehttp://docs.mongodb.org/manual/core/update/
Deletehttp://docs.mongodb.org/manual/core/delete/
Dokumentation
![Page 72: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/72.jpg)
Lab time!
Lab Nr. 02
Time box:20 min
![Page 73: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/73.jpg)
Indexing
![Page 74: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/74.jpg)
What is an index?
![Page 75: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/75.jpg)
Chained lists
1 2 3 4 5 6 7
![Page 76: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/76.jpg)
Find Nr. 7 in the chained list!
1 2 3 4 5 6 7
![Page 77: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/77.jpg)
Find Nr. 7 in a tree!
1
2
3
4
7
6
5
![Page 78: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/78.jpg)
Indices in MongoDB are B-Trees
![Page 79: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/79.jpg)
Find, Insert and Delete Operations:
O(log(n))
![Page 80: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/80.jpg)
Missing or non-optimal indices are the single-
most avoidable performance issue
![Page 81: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/81.jpg)
// Create a non-existing index for a field> db.recipes.createIndex({ main_ingredient: 1 })
// Make sure there is an index on the field> db.recipes.ensureIndex({ main_ingredient: 1 })
* 1 for ascending, -1 for descending
How do I create an index?
![Page 82: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/82.jpg)
// Multiple fields (Compound Key Indexes)> db.recipes.ensureIndex({
main_ingredient: 1,calories: -1
})
// Arrays with values (Multikey Indexes){
name: 'Chicken Noodle Soup’,ingredients : ['chicken', 'noodles']
}
> db.recipes.ensureIndex({ ingredients: 1 })
What can be indexed?
![Page 83: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/83.jpg)
// Subdocuments{
name : 'Apple Pie', contributor: {
name: 'Joe American',id: 'joea123'
}}
db.recipes.ensureIndex({ 'contributor.id': 1 })
db.recipes.ensureIndex({ 'contributor': 1 })
What can be indexed?
![Page 84: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/84.jpg)
// List all indices of a collection
> db.recipes.getIndexes()
> db.recipes.getIndexKeys()
// Drop an index
> db.recipes.dropIndex({ ingredients: 1 })
// Drop and recreate all indices of a collection
db.recipes.reIndex()
How to maintain indices?
![Page 85: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/85.jpg)
More options
• Unique Index– Allows only unique values in the indexed field(s)
• Sparse Index– For fields that are not available in all documents
• Geospatial Index– For modelling 2D and 3D geospatial indices
• TTL Collections – Are automatically deleted after x seconds
![Page 86: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/86.jpg)
// Make sure the name of a recipe is unique
> db.recipes.ensureIndex( { name: 1 }, { unique: true } )
// Force an index on a collection with non-unique values// Duplicates will be deleted more or less randomly!
> db.recipes.ensureIndex(
{ name: 1 },
{ unique: true, dropDups: true }
)
* dropDups should be used only with caution!
Unique Index
![Page 87: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/87.jpg)
// Only documents with the field calories will be indexed
> db.recipes.ensureIndex(
{ calories: -1 },
{ sparse: true }
)
// Combination with unique index is possible
> db.recipes.ensureIndex(
{ name: 1 , calories: -1 },
{ unique: true, sparse: true }
)
* Missing fields will be saved as null in the index!
Sparse Index
![Page 88: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/88.jpg)
// Add longitude and altitude
{
name: ‚codecentric Frankfurt’,
loc: [ 50.11678, 8.67206]
}
// Index the 2D coordinates
> db.locations.ensureIndex( { loc : '2d' } )
// Find locations near codecentric Frankfurt
> db.locations.find({
loc: { $near: [ 50.1, 8.7 ] }
})
Geospatial Index
![Page 89: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/89.jpg)
// Documents need a field of type BSON UTC
{ ' submitted_date ' : ISODate('2012-10-12T05:24:07.211Z'), … }
// Documents will be deleted automatically by a daemon process// after 'expireAfterSeconds'
> db.recipes.ensureIndex(
{ submitted_date: 1 },
{ expireAfterSeconds: 3600 }
)
TTL Collections
![Page 90: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/90.jpg)
Limitations of indices
• Collections can‘t have more than 64 indices
• Index keys are not allowed to be larger than 1024 Byte
• The name of an index (including name space) must beless than 128 character
• Queries can only make use of one index– Exception: Queries using $or
• Indices are tried to be kept in-memory
• Indices slow down the writing of data
![Page 91: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/91.jpg)
Optimizing indices
![Page 92: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/92.jpg)
Best practice
1. Identify slow queries
2. Find out more about the slow queriesusing explain()
3. Create appropriate indices on the fieldsbeing queried
4. Optimize the query taking theavailable indices into account
![Page 93: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/93.jpg)
> db.setProfilingLevel( n , slowms=100ms )
n=0: Profiler off
n=1: Log all operations slower than slowms
n=2: Log all operations
> db.system.profile.find()
* The collection profile is a capped collection with a limited number of entries
1. Identify slow queries
![Page 94: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/94.jpg)
> db.recipes.find( { calories:
{ $lt : 40 } }
).explain( )
{
"cursor" : "BasicCursor" ,
"n" : 42,
"nscannedObjects” : 53641
"nscanned" : 53641,
...
"millis" : 252,
...
}
2. Usage of explain()
![Page 95: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/95.jpg)
2. Metrics of the execution plan I
• Cursor– The type of the cursor: BasicCursor means no idex
has been used
• n – The number of matched documents
• nscannedObjects– The number of scanned documents
• nscanned– The number of scanned entries (Index entries or
documents)
![Page 96: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/96.jpg)
2. Metrics of the execution plan II
• millis– Execution time of the query
• Complete reference can be found here– http://docs.mongodb.org/manual/reference/explain
Optimize for
������������������������
������������ℎ�����������= 1
![Page 97: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/97.jpg)
3. Create appropriate indiceson the fields being queried
![Page 98: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/98.jpg)
// Using the following index…
> db.collection.ensureIndex({ a:1, b:1 , c:1, d:1 })
// … these queries and sorts can make use of the index
> db.collection.find( ).sort({ a:1 })
> db.collection.find( ).sort({ a:1, b:1 })
> db.collection.find({ a:4 }).sort({ a:1, b:1 })
> db.collection.find({ b:5 }).sort({ a:1, b:1 })
4. Optimize queries taking theavailable indices into account
![Page 99: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/99.jpg)
// Using the following index…
> db.collection.ensureIndex({ a:1, b:1, c:1, d:1 })
// … the these queries can not make use of it
> db.collection.find( ).sort({ b: 1 })
> db.collection.find({ b: 5 }).sort({ b: 1 })
4. Optimize queries taking theavailable indices into account
![Page 100: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/100.jpg)
// Using the following index…
> db.recipes.ensureIndex({ main_ingredient: 1, name: 1 })
// … this query can be complete satisfied using the index!
> db.recipes.find(
{ main_ingredient: 'chicken’ },
{ _id: 0, name: 1 }
)
// The metric indexOnly using explain() verifies this:
> db.recipes.find(
{ main_ingredient: 'chicken' },
{ _id: 0, name: 1 }
).explain()
{
"indexOnly": true,
}
4. Optimize queries taking theavailable indices into account
![Page 101: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/101.jpg)
// Tell MongoDB explicitly which index to use
> db.recipes.find({
calories: { $lt: 1000 } }
).hint({ _id: 1 })
// Switch the usage of idices completely off (e.g. for performance// measurements)
> db.recipes.find(
{ calories: { $lt: 1000 } }
).hint({ $natural: 1 })
Use specific indices
![Page 102: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/102.jpg)
Caveats using indices
![Page 103: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/103.jpg)
// MongoDB can only use one index per query!
> db.collection.ensureIndex({ a: 1 })
> db.collection.ensureIndex({ b: 1 })
// For this query only one of those two indices can be used
> db.collection.find({ a: 3, b: 4 })
Using multiple indices
![Page 104: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/104.jpg)
// Compound indices are often very efficient!
> db.collection.ensureIndex({ a: 1, b: 1, c: 1 })
// But only if the query is a prefix of the index...
// This query can make use of the index
db.collection.find({ c: 2 })
// …but this query can
db.collection.find({ a: 3, b: 5 })
Compound indices
![Page 105: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/105.jpg)
// The following field has only few distinct values
> db.collection.distinct('status’)
[ 'new', 'processed' ]
// A index on this field is not the best idea…
> db.collection.ensureIndex({ status: 1 })
> db.collection.find({ status: 'new' })
// Better use a adequate compound index with other fields
> db.collection.ensureIndex({ status: 1, created_at: -1 })
> db.collection.find(
{ status: 'new' }
).sort({ created_at: -1 })
Indices with low selectivity
![Page 106: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/106.jpg)
> db.users.ensureIndex({ username: 1 })
// Left-bound regular expressions can make usage of this index
> db.users.find({ username: /^joe smith/ })
// But not queries with regular expressions in general…
> db.users.find({username: /smith/ })
// Also not case-insensitive queries…
> db.users.find({ username: /^Joe/i })
Regular expressions & Indices
![Page 107: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/107.jpg)
// Negations can not make use of indices
> db.things.ensureIndex({ x: 1 })
// e.g. queries using not equal
> db.things.find({ x: { $ne: 3 } })
// …or queries with not in
> db.things.find({ x: { $nin: [2, 3, 4 ] } })
// …or queries with the $not operator
> db.people.find({ name: { $not: 'John Doe' } })
Negations & Indices
![Page 108: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/108.jpg)
Lab time!
Lab Nr. 03
Time box:20 min
![Page 109: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/109.jpg)
Map/Reduce
![Page 110: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/110.jpg)
What is Map/Reduce?
• Programming model coming fromfunctional languages
• Framework for– parallel processing– of big volume data– using distributed systems
• Made popular by Google– Has been invented to calculate the inverted search
index for web sites to keywords (Page Rank)– http://research.google.com/archive/mapreduce.html
![Page 111: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/111.jpg)
Basics
• Not something special about MongoDB– Hadoop– Disco– Amazon Elastic MapReduce– …
• Based on key-value-pairs
• Prior to version 2.4 and the introduction of the V8 JavaScript engine only one threadper shard
![Page 112: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/112.jpg)
The „Hello world“ of Map/Reduce: Word Count
![Page 113: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/113.jpg)
Word Count: Problem
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{ MongoDB
uses MapReduce
}
{There is a
map phase}
{There is a
reduce phase
}
a: 2is: 2
map: 1
mapreduce: 1mongodb: 1
phase: 2
reduce: 1there: 2uses: 1
Problem:How often doesone word appearin all documents?
![Page 114: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/114.jpg)
Word Count: Mapping
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{MongoDB
uses MapReduce
}
{There is a
map phase}
{There is a
reduce phase
}
(doc1, “…“)
(doc2, “…“)
(doc3, “…“)
(mongodb, 1)(uses, 1)(mapreduce, 1)
(there, 1)(is, 1)(a, 1)(map, 1)(phase, 1)
(there, 1)(is, 1)(a, 1)(reduce, 1)(phase, 1)
![Page 115: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/115.jpg)
Word Count: Group/Sort
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{MongoDB
uses MapReduce
}
{There is a
map phase}
{There is a
reduce phase
}
(doc1, “…“)
(doc2, “…“)
(doc3, “…“)
a-l
m-q
r-z
(phase, 1)
(map, 1)
(reduce, 1)
(there, 1)
![Page 116: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/116.jpg)
Word Count: Reduce
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{MongoDB
uses MapReduce
}
{There is a
map phase}
{There is a
reduce phase
}
(doc1, “…“)
(doc2, “…“)
(doc3, “…“)
(a, [1, 1])(is, [1, 1])(map, [1])
(mapreduce, [1])(mongodb, [1])(phase, [1, 1])
(reduce, [1])(there, [1, 1])
(uses, [1])
![Page 117: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/117.jpg)
Word Count: Result
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{MongoDB
uses MapReduce
}
{There is a
map phase}
{There is a
reduce phase
}
(doc1, “…“)
(doc2, “…“)
(doc3, “…“)
(a, [1, 1])(is, [1, 1])(map, [1])
(mapreduce, [1])(mongodb, [1])(phase, [1, 1])
(reduce, [1])(there, [1, 1])
(uses, [1])
a: 2is: 2
map: 1
mapreduce: 1mongodb: 1
phase: 2
reduce: 1there: 2uses: 1
![Page 118: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/118.jpg)
Word Count: In a nutshell
INPUT MAPPER GROUP/SORT REDUCER OUTPUT
{MongoDB
uses MapReduce
}
(doc1, “…“)
(a, [1, 1])(is, [1, 1])(map, [1])
a: 2is: 2
map: 1
map()Transforms one key-value-pair in 0–N key-value-pairs
reduce()Reduces 0-N key-value-pairs into onekey-value-pair
![Page 119: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/119.jpg)
Map/Reduce: Overview
MongoDB Data group(k)
sort(k)
finalize(k, v)
map()
emit(k,v)
reduce(k, values)
Shard 1
Shard 2
Shard n
…
Iterates all documents
• Input = Output• Can run multiple
times
![Page 120: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/120.jpg)
// Example: Twitter database with tweets> db.tweets.findOne(){
"_id" : ObjectId("4fb9fb91d066d657de8d6f38"),"text" : "RT @RevRunWisdom: The bravest thing that men do is
love women #love","created_at" : "Thu Sep 02 18:11:24 +0000 2010",
…
"user" : {"friends_count" : 0,"profile_sidebar_fill_color" : "252429","screen_name" : "RevRunWisdom","name" : "Rev Run",
},…
Word Count: Tweets
![Page 121: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/121.jpg)
// Map function with simple data cleansingmap = function() {
this.text.split(' ').forEach(function(word) {
// Remove whitespaceword = word.replace(/\s/g, "");
// Remove all non-word-charactersword = word.replace(/\W/gm,"");
// Finally emit the cleaned up wordif(word != "") {
emit(word, 1)}
});};
Word Count: map()
![Page 122: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/122.jpg)
// Reduce functionreduce = function(key, values) {
return values.length;};
Word Count: reduc()
![Page 123: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/123.jpg)
// Show the results using the console> db.tweets.mapReduce(map, reduce, { out : { inline : 1 } } );
// Save the results to a collection> db.tweets.mapReduce(map, reduce, { out : "tweets_word_count"} );
{"result" : "tweets_word_count","timeMillis" : 19026,"counts" : {
"input" : 53641,"emit" : 559217,"reduce" : 102057,"output" : 131003
},"ok" : 1,
}
Word Count: Call
![Page 124: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/124.jpg)
// Top-10 of most common words in tweets> db.tweets_word_count.find().sort({"value" : -1}).limit(10)
{ "_id" : "Miley", "value" : 31 }{ "_id" : "mil", "value" : 31 }{ "_id" : "andthenihitmydougie", "value" : 30 }{ "_id" : "programa", "value" : 30 }{ "_id" : "Live", "value" : 29 }{ "_id" : "Super", "value" : 29 }{ "_id" : "cabelo", "value" : 29 }{ "_id" : "listen", "value" : 29 }{ "_id" : "Call", "value" : 28 }{ "_id" : "DA", "value" : 28 }
Word Count: Result
![Page 125: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/125.jpg)
Recommendation
![Page 126: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/126.jpg)
Typical use cases
• Counting, Aggregating & Suming up– Analyzing log entries & Generating log reports– Generating an inversed index– Substitute existing ETL processes
• Counting unique values– Counting the number of unique visitors of a website
• Filtering, Parsing & Validation– Filtering of user data– Consolidation of user-generated data
• Sorting– Data analysis using complex sorting
![Page 127: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/127.jpg)
Summary
• The Map/Reduce framework is veryversatile & powerful
• Is implemented in JavaScript– Necessity to write own map()- und reduce() functions in JavaScript– Difficult to debug– Performance is highly influenced by the JavaScript engine
• Can be used for complex data analytics
• Lots of overhead for simple aggregation tasks– Suming up of data– Average of data– Grouping of data
![Page 128: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/128.jpg)
Map/Reduce should be used asultima ratio!
![Page 129: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/129.jpg)
Lab time!
Lab Nr. 04
Time box:20 min
![Page 130: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/130.jpg)
Aggregation Framework
![Page 131: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/131.jpg)
Why?
SELECT customer_id, SUM(price)FROM orders WHERE active=trueGROUP BY customer_id
![Page 132: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/132.jpg)
That‘s why!
SELECT customer_id, SUM(price)FROM orders WHERE active=trueGROUP BY customer_id
Calculationof fields
Calculationof fields
Groupingof data
Groupingof data
![Page 133: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/133.jpg)
The Aggregation Framework
• Has been introduced to allow 90% of real-world aggregation use cases without usingthe„big hammer“ Map/Reduce
• Framework of methods & operators– Declarative
– No own JavaScript code needed
– Fixed set of methods and operators (but constantly underdevelopment by MongoDB Inc.)
• Implemented in C++– Limitations on JavaScript Engine are avoided
– Better performance
![Page 134: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/134.jpg)
The Aggregation Pipeline
{document}
Pipeline Operator
Pipeline Operator
Pipeline Operator
Result{
sum: 337avg: 24,53min: 2max : 99
}
![Page 135: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/135.jpg)
The Aggregation Pipeline
• Processes a stream of documents– Input is a complete collection– Output is a document containing the results
• Succession of pipeline operators– Each tier filters or transforms the documents– Input documents of a tier are the output documents
of the previous tier
![Page 136: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/136.jpg)
db.tweets.aggregate(
{ $pipeline_operator_1 },
{ $pipeline_operator_2 },
{ $pipeline_operator_3 },
{ $pipeline_operator_4 },
...
);
Call
![Page 137: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/137.jpg)
// Old friends*
$match
$sort
$limit
$skip
Pipeline Operators
// New friends
$project
$group
$unwind
* from the query functionality
![Page 138: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/138.jpg)
// Example: Twitter database with tweets> db.tweets.findOne(){
"_id" : ObjectId("4fb9fb91d066d657de8d6f38"),"text" : "RT @RevRunWisdom: The bravest thing that men do is
love women #love","created_at" : "Thu Sep 02 18:11:24 +0000 2010",
…
"user" : {"friends_count" : 0,"profile_sidebar_fill_color" : "252429","screen_name" : "RevRunWisdom","name" : "Rev Run",
},…
Example: Tweets
![Page 139: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/139.jpg)
// Show all german users
> db.tweets.aggregate(
{ $match : {"user.lang" : "de"}},
);
// Show all users with 0 to 10 followers
> db.tweets.aggregate(
{ $match : {"user.followers_count" : { $gte : 0, $lt : 10 } } }
);
$match
> Filters documents> Equivalent to .find()
![Page 140: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/140.jpg)
// Sorting using one field
> db.tweets.aggregate(
{ $sort : {"user.friends_count" : -1} },
);
// Sorting using multiple fields
> db.tweets.aggregate(
{ $sort : {"user.lang" : 1, "user.time_zone" : 1, "user.friends_count" : -1} },
);
$sort
> Sorts documents> Equivalent to .sort()
![Page 141: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/141.jpg)
// Limit the number of resulting documents to 3
> db.tweets.aggregate(
{ $sort : {"user.friends_count" : -1} },
{ $limit : 3 }
);
$limit
> Limits resulting documents> Equivalent to .limit()
![Page 142: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/142.jpg)
// Get the No.4-Twitterer according to number of friends
> db.tweets.aggregate(
{ $sort : {"user.friends_count" : -1} },
{ $skip : 3 },
{ $limit : 1 }
);
$skip
> Skips documents> Equivalent to .skip()
![Page 143: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/143.jpg)
// Limit the result document to only one field
> db.tweets.aggregate(
{ $project : {text : 1} },
);
// Remove _id
> db.tweets.aggregate(
{ $project : {_id: 0, text : 1} },
);
$project I
> Limits the fields in resulting documents
![Page 144: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/144.jpg)
// Rename a field
> db.tweets.aggregate(
{ $project : {_id: 0, content_of_tweet : "$text"} },
);
// Add a calculated field
> db.tweets.aggregate(
{ $project : {_id: 0, content_of_tweet : "$text", number_of_friends : {$add: ["$user.friends_count", 10]} } },
);
$project II
![Page 145: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/145.jpg)
// Add a subdocument
> db.tweets.aggregate(
{ $project : {_id: 0,
content_of_tweet : "$text",
user : {
name : "$user.name",
number_of_friends : {$add: ["$user.friends_count", 10]} }
} } );
$project III
![Page 146: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/146.jpg)
// Grouping using a single field
> db.tweets.aggregate(
{ $group : {
_id : "$user.lang",
anzahl_tweets : {$sum : 1} }
}
);
$group I
> Groups documents> Equivalent to GROUP BY in SQL
![Page 147: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/147.jpg)
// Grouping using multiple fields
> db.tweets.aggregate(
{ $group : {
_id : { background_image:
"$user.profile_use_background_image",
language: "$user.lang" },
number_of_tweets: {$max : 1} }
}
);
$group II
![Page 148: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/148.jpg)
// Grouping with multiple calculated fields
> db.tweets.aggregate(
{ $group : {
_id : "$user.lang",
number_of_tweets : {$sum : 1},
average_of_followers : {$avg : "$user.followers_count"},
minimum_of_followers : {$min : "$user.followers_count"},
maximum_of_followers : {$max : "$user.followers_count"} }
}
);
$group III
![Page 149: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/149.jpg)
$min
$max
$avg
$sum
Group Aggregation Functions
$addToSet
$first
$last
$push
![Page 150: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/150.jpg)
// Unwind an array
> db.tweets.aggregate(
{ $project : {_id: 0, content_of_tweet : "$text", mentioned_users : "$entities.user_mentions.name" } },
{ $skip : 18 },
{ $limit : 1 },
{ $unwind : "$mentioned_users" }
);
$unwind I
> Unwinds arrays andcreates one document per value in the array
![Page 151: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/151.jpg)
// Resulting document without $unwind
{
„content_of_tweet" : "RT @Philanthropy: How shouldnonprofit groups measure their social-media efforts? A new podcast from @afine http://ht.ly/2yFlS",
„mentioned_users" : [
"Philanthropy",
"Allison Fine"
]
}
$unwind II
![Page 152: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/152.jpg)
// Resulting documents with $unwind
{
" content_of_tweet " : "RT @Philanthropy: How shouldnonprofit groups measure their social-media efforts? A new podcast from @afine http://ht.ly/2yFlS",
" mentioned_users " : "Philanthropy"
},
{
" content_of_tweet " : "RT @Philanthropy: How shouldnonprofit groups measure their social-media efforts? A new podcast from @afine http://ht.ly/2yFlS",
" mentioned_users " : "Allison Fine"
}
$unwind III
![Page 153: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/153.jpg)
Best Practices
![Page 154: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/154.jpg)
Place $match at the beginning of the pipeline to reduce the number of documents as soon as possible!
Best Practice #1
![Page 155: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/155.jpg)
Use $project to remove not needed fields in the documents as soon as possible!
Best Practice #2
![Page 156: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/156.jpg)
When being placed at the beginning of the pipeline theseoperators can make use of indices:
$match$sort$limit$skip
The above operators can equally use indices when placedbefore these operators:
$project$unwind$group
Best Practice #3
![Page 157: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/157.jpg)
Mapping of MongoDB to SQL
![Page 158: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/158.jpg)
MappingSQL MongoDB Aggregation
WHERE $match
GROUP BY $group
HAVING $match
SELECT $project
ORDER BY $sort
LIMIT $limit
SUM() $sum
COUNT() $sum
joinNo equivalent operator($unwind has somehow equivalentfunctionality for embedded fields)
![Page 159: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/159.jpg)
Example: Online shopping
{ cust_id: “sheldon1", ord_date:
ISODate("2013-04-018T19:38:11.102Z"), status: ‘purchased', price: 105,69, items:
[ { sku: “nobel_price_replica", qty: 3, price: 29,90 },
{ sku: “wheaton_voodoo_doll", qty: 1, price: 15,99 } ]
}
![Page 160: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/160.jpg)
Count all orders
SQL MongoDB Aggregation
SELECT COUNT(*) AScount FROM orders
db.orders.aggregate( [ { $group: { _id: null,
count: { $sum: 1 } } } ] )
![Page 161: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/161.jpg)
Average order price per customer
SQL MongoDB Aggregation
SELECT cust_id, SUM(price) AS total FROM orders GROUP BY cust_id ORDERBY total
db.orders.aggregate( [ { $group: { _id: "$cust_id", total: { $sum: "$price" } } },
{ $sort: { total: 1 } } ] )
![Page 162: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/162.jpg)
Sum up all orders over 250$
SQL MongoDB Aggregation
SELECT cust_id, SUM(price) astotal
FROM orders WHERE status = ‘purchased'GROUP BY cust_idHAVING total > 250
db.orders.aggregate( [ { $match: { status: 'A' } }, { $group: { _id: "$cust_id", total: { $sum: "$price" } } },
{ $match: { total: { $gt: 250} } } ] )
![Page 163: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/163.jpg)
http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/
More examples
![Page 164: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/164.jpg)
Lab time!
Lab Nr. 05
Time box:20 min
![Page 165: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/165.jpg)
Replication: High Availability with MongoDB
![Page 166: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/166.jpg)
Why do we need replication?
• Hardware is unreliable and is doomed to fail!
• Do you want to be the person being called at night to do a manual failover?
• How about network latency?
• Different use cases for your data– “Regular” processing– Data for analysis– Data for backup
![Page 167: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/167.jpg)
Life cycle of a replica set
![Page 168: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/168.jpg)
Replica set – Create
![Page 169: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/169.jpg)
Replica set – Initializing
![Page 170: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/170.jpg)
Replica set – Node down
![Page 171: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/171.jpg)
Replica set – Failover
![Page 172: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/172.jpg)
Replica set – Recovery
![Page 173: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/173.jpg)
Replica set – Back to normal
![Page 174: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/174.jpg)
Roles & Configuration
![Page 175: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/175.jpg)
Replica sets - Roles
![Page 176: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/176.jpg)
> conf = {
_id : "mySet",
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration I
![Page 177: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/177.jpg)
> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration II
Primary data center
![Page 178: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/178.jpg)
> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration III
Secondary data center(Default priority = 1)
![Page 179: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/179.jpg)
> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration IV
Analytical data e.g. for Hadoop, Storm, BI, …
![Page 180: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/180.jpg)
> conf = {
_id : "mySet”,
members : [
{_id : 0, host : "A”, priority : 3},
{_id : 1, host : "B", priority : 2},
{_id : 2, host : "C”},
{_id : 3, host : "D", hidden : true},
{_id : 4, host : "E", hidden : true, slaveDelay : 3600}
]
}
> rs.initiate(conf)
Configuration V
Back-up node
![Page 181: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/181.jpg)
Data consistency
![Page 182: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/182.jpg)
Strong consistency
![Page 183: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/183.jpg)
Eventual consistency
![Page 184: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/184.jpg)
Write Concern
• Different levels of data consistency
• Acknowledged by– Network– MongoDB– Journal– Secondaries– Tagging
![Page 185: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/185.jpg)
Acknowledged by network„Fire and forget“
![Page 186: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/186.jpg)
Acknowledged by MongoDBWait for Error
![Page 187: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/187.jpg)
Acknowledged by JournalWait for Journal Sync
![Page 188: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/188.jpg)
Acknowledged by SecondariesWait for Replication
![Page 189: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/189.jpg)
Tagging while writing data
• Available since 2.0
• Allows for fine granular control
• Each node can have multiple tags– tags: {dc: "ny"}– tags: {dc: "ny", subnet: „192.168", rack: „row3rk7"}
• Allows for creating Write Concern Rules (per replica set)
• Tags can be adapted without code changes and restarts
![Page 190: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/190.jpg)
{
_id : "mySet",
members : [
{_id : 0, host : "A", tags : {"dc": "ny"}},
{_id : 1, host : "B", tags : {"dc": "ny"}},
{_id : 2, host : "C", tags : {"dc": "sf"}},
{_id : 3, host : "D", tags : {"dc": "sf"}},
{_id : 4, host : "E", tags : {"dc": "cloud"}}],
settings : {
getLastErrorModes : {
allDCs : {"dc" : 3},
someDCs : {"dc" : 2}} }
}
> db.blogs.insert({...})
> db.runCommand({getLastError : 1, w : "someDCs"})
Tagging - Example
![Page 191: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/191.jpg)
Acknowledged by TaggingWait for Replication (Tagging)
![Page 192: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/192.jpg)
// Wait for network acknowledgement
> db.runCommand( { getLastError: 1, w: 0 } )
// Wait for error (Default)
> db.runCommand( { getLastError: 1, w: 1 } )
// Wait for journal sync
> db.runCommand( { getLastError: 1, w: 1, j: "true" } )
// Wait for replication
> db.runCommand( { getLastError: 1, w: “majority" } )
> db.runCommand( { getLastError: 1, w: 3 } ) // # of secondaries
Configure the Write Concern
![Page 193: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/193.jpg)
Read Concerns
• Only primary (primary)
• Primary preferred (primaryPreferred)
• Only secondaries (secondary)
• Secondaries preferred (secondaryPreferred)
• Nearest node (Nearest)
General: If more than one node is available, the nearest node will be chosen (All modes except Primary)
![Page 194: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/194.jpg)
Only primary(primary)
Read
![Page 195: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/195.jpg)
Primary preferred(primaryPreferred)
Read
Read
![Page 196: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/196.jpg)
Only secondaries(secondary)
Read
Read
![Page 197: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/197.jpg)
Secondaries preferred(secondaryPreferred)
Read
Read
Read
![Page 198: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/198.jpg)
Nearest node(nearest)
Read
Read
Read
![Page 199: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/199.jpg)
Tagging while reading data
• Allows for a more fine granular control where data will be read from – e.g. { "disk": "ssd", "use": "reporting" }
• Can be combined with other read modes– Except for mode „Only primary“
![Page 200: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/200.jpg)
// Only primary
> cursor.setReadPref( “primary" )
// Primary preferred
> cursor.setReadPref( “primaryPreferred" )
…
// Only secondaries with tagging
> cursor.setReadPref( “secondary“, [ rack : 2 ] )
Configure the Read Concern
Read Concern must be configured before using the cursor to read data!
![Page 201: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/201.jpg)
MongoDB Operation
![Page 202: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/202.jpg)
Maintenance & Upgrades
• Zero downtime
• Rolling upgrades and maintenance– Start with all secondaries– Step down the current primary– Primary as last one– Restore previous primary (if needed)
• Commands: – rs.stepDown(<secs>) – db.version() – db.serverBuildInfo()
![Page 203: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/203.jpg)
Replica set – 1 data center
• One – Data center– Switch– Power Supply
• Possible errors:
– Failure of 2 nodes– Power Supply– Network– Data Center
• Automatic recovery
![Page 204: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/204.jpg)
Replica set – 2 data center
• Additional node for data recovery
• No writing to both data center since only one node in data center No. 2
![Page 205: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/205.jpg)
Replica set – 3 data center
• Can recover from a complete data center failure
• Allows for usage of w= { dc : 2 } to guarantee writing to 2 data centers (via tagging)
![Page 206: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/206.jpg)
Commands
• Administration of the nodes– rs.conf() – rs.initiate(<conf>) & rs.reconfig(<conf>) – rs.add(host:<port>) & rs.addArb(host:<port>) – rs.status() – rs.stepDown(<secs>)
• Reconfiguration if a minority of the nodes is not available – rs.reconfig( cfg, { force: true} )
![Page 207: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/207.jpg)
Best Practices
![Page 208: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/208.jpg)
Best Practices
• Uneven number of nodes
• Adapt the write concern to your use case
• Read from primary except for– Geographical distribution– Data analytics
• Use logical names and not IP addresses for configuration
• Monitor the lags of the secondaries (e.g. MMS)
![Page 209: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/209.jpg)
Lab time!
Lab Nr. 06
Time box:20 min
![Page 210: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/210.jpg)
Sharding: Scaling with MongoDB
![Page 211: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/211.jpg)
Visual representation of vertical scalingVisual representation of vertical scaling
1970 - 2000: Vertical Scaling„Scale up“
![Page 212: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/212.jpg)
Visual representation of horizontal scalingVisual representation of horizontal scaling
Since 2000: Horizontal Scaling „Scale out“
![Page 213: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/213.jpg)
When to use Sharding?
![Page 214: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/214.jpg)
Not enough disk space
![Page 215: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/215.jpg)
The working set doesn‘t fit into the memory
![Page 216: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/216.jpg)
The needs for read-/write throughput are higher than the I/O capabilities
![Page 217: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/217.jpg)
Sharding MongoDB
![Page 218: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/218.jpg)
Partitioning of data
• The user needs to define a shard key
• The shard key defines the distribution of data across the shards
![Page 219: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/219.jpg)
Partitioning of data into chunks
• Initially all data is in one chunk
• Maximum chunk size: 64 MB
• MongoDB divides and distributes chunks automatically once the maximum size is met
![Page 220: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/220.jpg)
One chunk contains data of a certain value range
![Page 221: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/221.jpg)
Chunks & Shards
• A shard is one node in the cluster
• A shard can be one single mongod or a replica set
![Page 222: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/222.jpg)
Metadata Management
• Config Server– Stores the value ranges of the chunks and their
location – Number of config servers is 1 or 3 (Production: 3)– Two Phase Commit
![Page 223: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/223.jpg)
Balancing & Routing Service
• mongos balances the data in the cluster
• mongos distributes data to new nodes
• mongos routes queries to the correct shard or collects results if data isspread on multiple shards
• No local data
![Page 224: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/224.jpg)
Automatic Balancing
Balancing will be automatically done once the number of chunks between shards hits a certain threshold
![Page 225: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/225.jpg)
Splitting of a chunk
• Once a chunk hits the maximum size it will be split
• Splitting is only a logical operation, no data needs to be moved
• If the splitting of a chunk results in a misbalance of data, automatic rebalancing will be started
![Page 226: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/226.jpg)
Sharding Infrastructure
![Page 227: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/227.jpg)
MongoDB Auto Sharding
• Minimal effort– Usage of the same interfaces for mongod and
mongos
• Easy configuration– Enable sharding for a database
• sh.enableSharding("<database>")
– Shard a collection in a database
• sh.shardCollection("<database>.<collection>", shard-key-pattern)
![Page 228: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/228.jpg)
Configuration example
![Page 229: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/229.jpg)
Example of a very simple cluster
• Never use this in production!– Only one config server (No fault tolerance)– Shard is no replica set (No high availability)– Only one mongos and one shard (No performance
improvement)
![Page 230: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/230.jpg)
// Start the config server (Default port 27019)
> mongod --configsvr
Start the config server
![Page 231: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/231.jpg)
// Start the mongos router (Default port 27017)
> mongos --configdb <hostname>:27019
// When using 3 config servers
> mongos --configdb <host1>:<port1>,<host2>:<port2>,<host3>:<port3>
Start the mongos routing service
![Page 232: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/232.jpg)
// Start a shard with one mongod (Default port 27018)
> mongod --shardsvr
// Shard is not yet added to the cluster!
Start the shard
![Page 233: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/233.jpg)
// Connect to mongos and add the shard
> mongo
> sh.addShard(‘<host>:27018’)
// When adding a replica set, you only need to add one of the nodes!
Add the shard
![Page 234: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/234.jpg)
// Check if the shard has been added
> db.runCommand({ listShards:1 })
{ "shards" : [ { "_id”: "shard0000”, "host”: ”<hostname>:27018” } ],
"ok" : 1}
Check configuration
![Page 235: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/235.jpg)
// Enable the sharding for a database
> sh.enableSharding(“<dbname>”)
// Shard a collection using a shard key
> sh.shardCollection(“<dbname>.user”, { “name” : 1 } )
// Use a compound shard key
> sh.shardCollection(“<dbname>.cars”,{“year”:1, ”uniqueid”:1})
Configure sharding
![Page 236: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/236.jpg)
Shard Key
![Page 237: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/237.jpg)
Shard Key
• The shard key can not be changed
• The values of a shard key can not be changed
• The shard key needs to be indexed
• The uniqueness of the field _id is only guaranteed within a shard
• The size of a shard key is limited to 512 bytes
![Page 238: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/238.jpg)
Considerations for the shard key
• Cardinality of data– The value range needs to be rather large. For example sharding
on the field loglevel with the 3 values error, warning, infodoesn‘t make sense.
• Distribution of data– Always strive for equal distribution of data throughout all
shards!
• Patterns during reading and writing– For example for log data using the timestamp as a shard key
can be useful if chronological very close data needs to be read or written together.
![Page 239: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/239.jpg)
Choices for the shard key
• Single field– If the value range is big enough and data is distributed almost
equally
• Compound fields– Use this if a single field is not enough in respect to value range
and equal distribution
• Hash based– In general a random shard key is a good choice for equal
distribution of data– For performance the shard key should be part of the queries – Only available since 2.4
• sh.shardCollection( “user.name", { a: "hashed" } )
![Page 240: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/240.jpg)
{
_id: 346,
username: “sheldinator”,
password: “238b8be8bd133b86d1e2ba191a94f549”,
first_name: “Sheldon”
last_name: “Cooper”
created_on: “Mon Apr 15 15:30:32 +0000 2013“
modified_on: “Thu Apr 18 08:11:23 +0000 2013“
}
Example: User
Which shard key would you choose and why?
![Page 241: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/241.jpg)
{
log_type: “error” // Possible values “error, “warn”, “info“
application: “JBoss v. 4.2.3”
message: “Fatal error. Application will quit.”
created_on: “Mon Apr 15 15:38:05 +0000 2013“
}
Example: Log data
Which shard key would you choose and why?
![Page 242: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/242.jpg)
Routing of queries
![Page 243: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/243.jpg)
Possible types of queries
• Exact queries– Data is exactly on one shard
• Distributed query– Data is distributed on different shards
• Distributed query with sorting– Data is distributed on different shards and needs to
be sorted
![Page 244: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/244.jpg)
Exact queries
![Page 245: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/245.jpg)
1. mongos receives the query from the client
![Page 246: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/246.jpg)
2. Query is routed to the shard with the data
![Page 247: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/247.jpg)
3. Shard returns the data
![Page 248: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/248.jpg)
4. mongos returns the data to the client
![Page 249: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/249.jpg)
Distributed queries
![Page 250: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/250.jpg)
1. mongos receives the query from the client
![Page 251: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/251.jpg)
2. mongos routes the query to all shards
![Page 252: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/252.jpg)
3. Shards return the data
![Page 253: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/253.jpg)
4. mongos returns the data to the client
![Page 254: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/254.jpg)
Distributed queries with sorting
![Page 255: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/255.jpg)
1. mongos receives the query from the client
![Page 256: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/256.jpg)
2. mongos routes the query to all shards
![Page 257: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/257.jpg)
3. Execute the query and local sorting
![Page 258: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/258.jpg)
4. Shards return sorted data
![Page 259: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/259.jpg)
5. mongos sorts the data globally
![Page 260: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/260.jpg)
6. mongos returns the sorted data to the client
![Page 261: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/261.jpg)
Lab time!
Lab Nr. 07
Time box:20 min
![Page 262: MongoDB for Coder Training (Coding Serbia 2013)](https://reader038.vdocument.in/reader038/viewer/2022103000/554f6e90b4c905bb178b5020/html5/thumbnails/262.jpg)
Still want moar?
https://education.mongodb.com