Download - MongoDB With Style
![Page 2: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/2.jpg)
Styl
e
query
&d
esig
n
scale
![Page 3: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/3.jpg)
mongo console
$ ~/Work/opt/mongodb-‐1.6.5/bin/mongod \ -‐-‐dbpath=~/Work/src/nosqlday/db/mongodb.01 \ -‐-‐logpath=~/Work/src/nosqlday/log/mongodb.01 \ -‐-‐fork -‐-‐port 30001
$ ~/Work/opt/mongodb-‐1.6.5/bin/mongo localhost:30001MongoDB shell version: 1.6.5connecting to: localhost:30001/test
> use nosqldayswitched to db nosqlday
> db.getCollectionNames()[ "system.indexes", "users" ]
> db.users.find({ "name": "Gabriele" }){ "_id" : ObjectId("4d8706767bb037a8a8f98db2"), "name" : "Gabriele", "surname" : "Lana", "job" : "softwarecraftsman" }
> exitbye
![Page 4: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/4.jpg)
ruby driver
require "mongo"
db = Mongo::Connection.new("localhost", 30001).db("nosqlday")
puts "Collections:"db.collections.each do |collection| puts "\t#{collection.name}"end
puts "Gabriele:"db["users"].find(:name => "Gabriele").each do |user| puts "\t#{user["_id"]}"end
db.connection.close
![Page 5: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/5.jpg)
require "mongo"
db = Mongo::Connection.new("localhost", 30001).db("nosqlday")
puts "Collections:"db.collections.each do |collection| puts "\t#{collection.name}"end
puts "Gabriele:"db["users"].find(:name => "Gabriele").each do |user| puts "\t#{user["_id"]}"end
db.connection.close
ruby driver
$ ruby src/connect.rb Collections: users system.indexesGabriele: 4d8706767bb037a8a8f98db2
![Page 6: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/6.jpg)
Styl
e
query
&d
esig
n
scale
![Page 7: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/7.jpg)
Styl
eStyl
e
know yourdriver
![Page 8: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/8.jpg)
document object mapper
smart driver
mongo
![Page 9: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/9.jpg)
puts "Gabriele:"db["users"].find(:name => "Gabriele").each do |user| puts "\t#{user["_id"]}"end
puts "Gabriele:"db["users"].select{|user| user["name"] == "Gabriele"}.each do |user| puts "\t#{user["_id"]}"end
smart driver
mongo
![Page 10: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/10.jpg)
puts "Gabriele:"db["users"].find(:name => "Gabriele").each do |user| puts "\t#{user["_id"]}"end
puts "Gabriele:"db["users"].select{|user| user["name"] == "Gabriele"}.each do |user| puts "\t#{user["_id"]}"end
smart driver
mongo
$ ruby src/find_vs_select.rb Gabriele: 4d8706767bb037a8a8f98db2Gabriele: 4d8706767bb037a8a8f98db2
![Page 11: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/11.jpg)
puts "Gabriele:"db["users"].find(:name => "Gabriele").each do |user| puts "\t#{user["_id"]}"end
puts "Gabriele:"db["users"].select{|user| user["name"] == "Gabriele"}.each do |user| puts "\t#{user["_id"]}"end
smart driver
mongo
![Page 12: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/12.jpg)
Styl
eStyl
e incremental design
based onapplicationbehavior
![Page 13: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/13.jpg)
the best design is the one where needed data can be easily extracted
the way you need to query your data should influence your design
![Page 14: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/14.jpg)
Styl
eStyl
e incremental design
based onapplicationmonitoring
![Page 15: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/15.jpg)
monitoring and adapting is better than doing it right the first time
...actually the first timeis the worst time :-)
![Page 16: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/16.jpg)
monitoring & adapting
> db.setProfilingLevel(1, 5) { "was" : 1, "slowms" : 100, "ok" : 1 }
// after product usage find problematic queries
> db.system.profile.find().sort({millis:-‐1}) { "ts": "Mon Mar 21 2011 14:30:56 GMT+0100 (CET)", "info": " query pomodorist.pomodori reslen:202 nscanned:26950 query: { $query: { task_id: ObjectId('4d6f1d3931f2386e9c089796') }} nreturned:1 ", "millis":17}
![Page 17: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/17.jpg)
monitoring & adapting
> db.pomodori.find({ $query: { task_id: ObjectId('4d6f1d3931f2386e9c089796') }, $explain: true}) { "cursor": "BasicCursor", "nscanned": 26950, "nscannedObjects": 26950, "n": 1, "millis": 17, "indexBounds": { }, "allPlans": [ { "cursor" : "BasicCursor", "indexBounds" : { } } ]}
![Page 18: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/18.jpg)
monitoring & adapting
> db.pomodori.ensureIndex({"task_id": 1}) > db.pomodori.find({ $query: { task_id: ObjectId('4d6f1d3931f2386e9c089796') }, $explain: true})
{ "cursor": "BtreeCursor task_id_1", "nscanned": 1, "nscannedObjects": 1, "n": 1, "millis": 0, "indexBounds": { "task_id": [ [ ObjectId("4d6f1d3931f2386e9c089796"), ObjectId("4d6f1d3931f2386e9c089796") ] ]}, "allPlans": [...]}
![Page 19: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/19.jpg)
Styl
e
query
&d
esig
n
scale
![Page 20: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/20.jpg)
use $inoperatorfor batch
query
query
&d
esig
n
![Page 21: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/21.jpg)
retrieve all objects with $in
users = [{:name => "Gabriele", :surname => "Lana", :job => "softwarecraftsman"},{:name => "Federico", :surname => "Galassi", :job => "softwarecraftsman"},{:name => "Giordano", :surname => "Scalzo", :job => "softwarecraftsman"}]
ids = users.map{|user| db["users"].insert(user)}
puts ids.map{|id| db["users"].find_one(:_id => id)}
![Page 22: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/22.jpg)
retrieve all objects with $in
users = [{:name => "Gabriele", :surname => "Lana", :job => "softwarecraftsman"},{:name => "Federico", :surname => "Galassi", :job => "softwarecraftsman"},{:name => "Giordano", :surname => "Scalzo", :job => "softwarecraftsman"}]
ids = users.map{|user| db["users"].insert(user)}
puts ids.map{|id| db["users"].find_one(:_id => id)}
$ ruby src/find_by_all_ids.rb {"_id"=>BSON::ObjectId('4d87605731f23824a0000001'), ...}{"_id"=>BSON::ObjectId('4d87605731f23824a0000002'), ...}{"_id"=>BSON::ObjectId('4d87605731f23824a0000003'), ...}
![Page 23: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/23.jpg)
retrieve all objects with $in
users = [{:name => "Gabriele", :surname => "Lana", :job => "softwarecraftsman"},{:name => "Federico", :surname => "Galassi", :job => "softwarecraftsman"},{:name => "Giordano", :surname => "Scalzo", :job => "softwarecraftsman"}]
ids = users.map{|user| db["users"].insert(user)}
puts ids.map{|id| db["users"].find_one(:_id => id)}
![Page 24: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/24.jpg)
retrieve all objects with $in
users = [{:name => "Gabriele", :surname => "Lana", :job => "softwarecraftsman"},{:name => "Federico", :surname => "Galassi", :job => "softwarecraftsman"},{:name => "Giordano", :surname => "Scalzo", :job => "softwarecraftsman"}]
ids = users.map{|user| db["users"].insert(user)}ids = db["users"].insert(users)
puts ids.map{|id| db["users"].find_one(:_id => id)}puts db["users"].find(:_id => {:$in => ids}).all
![Page 25: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/25.jpg)
retrieve all objects with $in
users = [{:name => "Gabriele", :surname => "Lana", :job => "softwarecraftsman"},{:name => "Federico", :surname => "Galassi", :job => "softwarecraftsman"},{:name => "Giordano", :surname => "Scalzo", :job => "softwarecraftsman"}]
ids = users.map{|user| db["users"].insert(user)}ids = db["users"].insert(users)
puts ids.map{|id| db["users"].find_one(:_id => id)}puts db["users"].find(:_id => {:$in => ids}).all
$ ruby src/find_by_all_ids.rb {"_id"=>BSON::ObjectId('4d87605731f23824a0000001'), ...}{"_id"=>BSON::ObjectId('4d87605731f23824a0000002'), ...}{"_id"=>BSON::ObjectId('4d87605731f23824a0000003'), ...}
![Page 26: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/26.jpg)
use conventions to
build smartobject
identifiers
query
&d
esig
n
![Page 27: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/27.jpg)
conventions are fun to play with
> db.user_scores.find({}, {"_id": 1})
{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106" }{ "_id" : "4d873ce631f238241d00000d-‐week-‐200944" }{ "_id" : "4d873ce631f238241d00000d-‐month-‐200911" }{ "_id" : "4d873ce631f238241d00000d-‐year-‐2009" }{ "_id" : "4d873ce631f238241d00000d-‐user" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐advertising" }{ "_id" : "4d873ce631f238241d00000d-‐week-‐200944-‐advertising" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐art" }{ "_id" : "4d873ce631f238241d00000d-‐week-‐200944-‐art" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐artist" }{ "_id" : "4d873ce631f238241d00000d-‐week-‐200944-‐artist" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐information" }
![Page 28: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/28.jpg)
conventions are fun to play with
> db.user_scores.findOne( {"_id": "4d873ce631f238241d00000d-‐day-‐20091106"} )
{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106", "pomodori" : 15, "pomodori_squashed" : 3, "breaks" : 7, "tasks_created" : 8, "tasks_done" : 6, "estimation_accuracy" : 0, "seconds_of_focused_time" : 22500, "seconds_of_wasted_time" : 1999, "seconds_of_breaks" : 8820}
![Page 29: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/29.jpg)
conventions are fun to play with(user scores in day per tag)
> db.user_scores.find( {"_id": /^4d873ce631f238241d00000d-‐day-‐20091106-‐/}, {"_id": 1} ) { "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐advertising" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐art" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐artist" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐blogging" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐culture" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐html" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐illustration" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐information" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐inspiration" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐marketing" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐movies" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐resources" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐technology" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐tool" }{ "_id" : "4d873ce631f238241d00000d-‐day-‐20091106-‐tutorials" }
![Page 30: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/30.jpg)
conventions are fun to play with(list of tags per day)
> db.user_scores.find( {"_id": /^4d873ce631f238241d00000d-‐day-‐20091106-‐/}, {"_id": 1}
).map(function(document) { return document._id.replace( "4d873ce631f238241d00000d-‐day-‐20091106-‐", "" ) })
[ "advertising", "art", "artist", "blogging", "culture", "html", "illustration", "information", ...]
![Page 31: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/31.jpg)
conventions are fun to play with(anchored regexp uses indexes)
> db.user_scores.find( {"_id": /^4d873ce631f238241d00000d-‐day-‐20091106-‐/}, {"_id": 1} ).explain() { "cursor" : "BtreeCursor _id_ multi", "nscanned" : 15, "nscannedObjects" : 15, "n" : 15, "millis" : 0, "indexBounds" : { "_id" : [ [ "4d873ce631f238241d00000d-‐day-‐20091106-‐", "4d873ce631f238241d00000d-‐day-‐20091106." ], [ /^4d873ce631f238241d00000d-‐day-‐20091106-‐/, /^4d873ce631f238241d00000d-‐day-‐20091106-‐/ ] ]
![Page 32: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/32.jpg)
conventions are fun to play with(anchored regexp uses indexes)
> db.user_scores.find( {"_id": /4d873ce631f238241d00000d-‐day-‐20091106-‐/}, {"_id": 1} ).explain()
{ "cursor" : "BtreeCursor _id_ multi", "nscanned" : 109349, "nscannedObjects" : 15, "n" : 15, "millis" : 217, "indexBounds" : { "_id" : [ ... ] }}
![Page 33: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/33.jpg)
use “group”method todo small
computationswithoutfetchingrelated
documentsquery
&d
esig
n
![Page 34: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/34.jpg)
group to compute data in mongo(inject client side)
days = [ 20091110, 20091111, 20091112 ]scores_id = %r{^4d87d00931f2380c7700000d-day-(#{days.join("|")})$}
scores = db["user_scores"].find(:_id => scores_id)
pomodori = scores.inject(0) do |pomodori, scores| pomodori + scores["pomodori"]end
puts "Pomodori in days #{days.join(",")}: #{pomodori}"
![Page 35: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/35.jpg)
group to compute data in mongo(inject client side)
days = [ 20091110, 20091111, 20091112 ]scores_id = %r{^4d87d00931f2380c7700000d-day-(#{days.join("|")})$}
scores = db["user_scores"].find(:_id => scores_id)
pomodori = scores.inject(0) do |pomodori, scores| pomodori + scores["pomodori"]end
puts "Pomodori in days #{days.join(",")}: #{pomodori}"
$ ruby src/inject_for_reduce.rb Pomodori in days 20091110,20091111,20091112: 36
![Page 36: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/36.jpg)
group to compute data in mongo(group server side)
days = [ 20091110, 20091111, 20091112 ]scores_id = %r{^4d87d00931f2380c7700000d-day-(#{days.join("|")})$}
result = db["user_scores"].group( :cond => { :_id => scores_id }, :initial => { :pomodori => 0 }, :reduce => <<-EOF function(document, result) { result.pomodori += document.pomodori } EOF)
puts "Pomodori in days #{days.join(",")}: #{result.first["pomodori"]}"
![Page 37: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/37.jpg)
group to compute data in mongo(group server side)
days = [ 20091110, 20091111, 20091112 ]scores_id = %r{^4d87d00931f2380c7700000d-day-(#{days.join("|")})$}
result = db["user_scores"].group( :cond => { :_id => scores_id }, :initial => { :pomodori => 0 }, :reduce => <<-EOF function(document, result) { result.pomodori += document.pomodori } EOF)
puts "Pomodori in days #{days.join(",")}: #{result.first["pomodori"]}"
$ ruby src/group_for_reduce.rb Pomodori in days 20091110,20091111,20091112: 36
![Page 38: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/38.jpg)
group to compute data in mongo(ex. sum pomodori by tag “ruby”)
result = db["user_scores"].group( :cond => { :_id => /^4d87d00931f2380c7700000d-day-\d{8}-ruby$/ }, :initial => { :pomodori => 0, :days => 0 }, :reduce => <<-EOF function(document, result) { result.days += 1 result.pomodori += document.pomodori } EOF).first
puts "In #{result["days"]} days, #{result["pomodori"]} done for ruby"
![Page 39: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/39.jpg)
group to compute data in mongo(ex. sum pomodori by tag “ruby”)
result = db["user_scores"].group( :cond => { :_id => /^4d87d00931f2380c7700000d-day-\d{8}-ruby$/ }, :initial => { :pomodori => 0, :days => 0 }, :reduce => <<-EOF function(document, result) { result.days += 1 result.pomodori += document.pomodori } EOF).first
puts "In #{result["days"]} days, #{result["pomodori"]} pomodori"
$ ruby src/group_for_ruby_tag.rb In 43 days, 45 pomodori
![Page 40: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/40.jpg)
group to compute data in mongo(ex. sum pomodori by tag “ruby”)
> db.user_scores.find({ "_id": /^4d87d00931f2380c7700000d-‐day-‐\d{8}-‐ruby$/ }).explain()
{ "cursor" : "BtreeCursor _id_ multi", "nscanned" : 43, "nscannedObjects" : 43, "n" : 43, "millis" : 3, "indexBounds" : { "_id" : [...] }}
![Page 41: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/41.jpg)
create indexes on arrays to create local
reverse indexes in documents
query
&d
esig
n
![Page 42: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/42.jpg)
reverse index in place(an array could be indexed)
> db.tasks.find({ "tags": { $in: [ "nosqlday" ] } }) { "_id" : ObjectId("4d7de446175ca8243d000004"), "tags" : [ "nosqlday" ], "description" : "#nosqlday keynote", "is_recurrent" : false, "estimated" : 0, "worked_in" : [ "Mon Mar 14 2011 00:00:00 GMT+0100 (CET)", "Tue Mar 15 2011 00:00:00 GMT+0100 (CET)" ], "done_at" : "Tue Mar 15 2011 13:05:03 GMT+0100 (CET)", "todo_at" : null, "created_at" : "Mon Mar 14 2011 10:47:50 GMT+0100 (CET)", "updated_at" : "Tue Mar 15 2011 13:05:03 GMT+0100 (CET)", "keywords": [ "nosqldai", "keynot" ], "user_id": ObjectId("4d53996c137ce423ff000001"), "annotations" : [ ]}
![Page 43: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/43.jpg)
reverse index in place(an array could be indexed)
> db.tasks.getIndexes()[ { "name" : "_id_", "ns" : "app435386.tasks", "key" : { "_id" : 1 } }, { "name" : "tags_1", "ns" : "app435386.tasks", "key" : { "tags" : 1 }, "unique" : false }, ...]
![Page 44: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/44.jpg)
reverse index in place(container for deduced data, array)
db["orders"].insert({ :placed_at => [ now.strftime("%Y"), # year: "2011" now.strftime("%Y%m"), # month: "201103" now.strftime("%Yw%U"), # week: "2011w11" now.strftime("%Y%m%d") # day: "20110316" ], :user_id => user, :items => items_in_order.map{|item| item[:id]}, :total => items_in_order.inject(0){|total,item| total += item[:price]}})
# ...
db["orders"].ensure_index([["placed_at", Mongo::DESCENDING]])
![Page 45: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/45.jpg)
> db.orders.findOne()
{ "_id" : ObjectId("4d88bf1f31f23812de0003fd"), "placed_at" : [ "2011", "201103", "2011w11", "20110316" ], "user_id" : ObjectId("4d88bf1f31f23812de0003e9"), "items" : [ ObjectId("4d88bf1f31f23812de0003da"), ObjectId("4d88bf1f31f23812de000047"), ObjectId("4d88bf1f31f23812de000078"), ObjectId("4d88bf1f31f23812de000068"), ObjectId("4d88bf1f31f23812de000288") ], "total" : 3502}
reverse index in place(container for deduced data, array)
![Page 46: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/46.jpg)
> db.orders.find({ "placed_at": "20110310" }).count()77
> db.orders.find({ "placed_at": "20110310" }).explain(){ "cursor" : "BtreeCursor placed_at_-‐1", "nscanned" : 77, "nscannedObjects" : 77, "n" : 77, "millis" : 0, "indexBounds" : { "placed_at" : [ [ "20110310", "20110310" ] ] }}
reverse index in place(container for deduced data, array)
![Page 47: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/47.jpg)
reverse index in place(container for deduced data, hash)
db["orders"].insert({ :placed_at => [ { :year => now.strftime("%Y") }, { :month => now.strftime("%Y%m") }, { :week => now.strftime("%Y%U") }, { :day => now.strftime("%Y%m%d") } ], :user_id => user, :items => items_in_order.map{|item| item[:id]}, :total => items_in_order.inject(0){|total,item| total += item[:price]} })
# ...
db["orders"].ensure_index([["placed_at", Mongo::DESCENDING]])
![Page 48: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/48.jpg)
> db.orders.findOne() { "_id" : ObjectId("4d88c31531f23812fe0003ea"), "placed_at" : [ { "year" : "2009" }, { "month" : "200911" }, { "week" : "200945" }, { "day" : "20091109" } ], "user_id" : ObjectId("4d88c31531f23812fe0003e9"), "items" : [ ObjectId("4d88c31531f23812fe00013f"), ObjectId("4d88c31531f23812fe000176"), ObjectId("4d88c31531f23812fe0003e2"), ObjectId("4d88c31531f23812fe0003d1"), ObjectId("4d88c31531f23812fe0001c1"), ObjectId("4d88c31531f23812fe000118"), ObjectId("4d88c31531f23812fe00031d") ], "total" : 10149}
reverse index in place(container for deduced data, hash)
![Page 49: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/49.jpg)
> db.orders.find({ "placed_at.week": "201101" }).count() 331
> db.orders.find({ "placed_at.week": "201101" }).explain() { "cursor" : "BasicCursor", "nscanned" : 22374, "nscannedObjects" : 22374, "n" : 331, "millis" : 23, "indexBounds" : { }}
reverse index in place(container for deduced data, hash)
![Page 50: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/50.jpg)
> db.orders.find({ "placed_at": { "week": "201101" }}).count() 331
> db.orders.find({ "placed_at": { "week": "201101" }}).explain(){ "cursor" : "BtreeCursor placed_at_-‐1", "nscanned" : 331, "nscannedObjects" : 331, "n" : 331, "millis" : 0, "indexBounds" : { "placed_at" : [ [ { "week" : "2011w01" }, { "week" : "2011w01" } ] ] }}
reverse index in place(container for deduced data, hash)
![Page 51: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/51.jpg)
use dates butbe aware of
some pitfalls
query
&d
esig
n
![Page 52: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/52.jpg)
db["orders"].insert({ :placed_at => now, :user_id => user, :items => items_in_order.map{|item| item[:id]}, :total => items_in_order.inject(0){|total,item| total += item[:price]} })
# ...
db["orders"].ensure_index([["placed_at", Mongo::DESCENDING]])
plain dates are good too
![Page 53: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/53.jpg)
> db.orders.findOne() { "_id" : ObjectId("4d88d1f931f23813a10003ea"), "placed_at" : "Mon Nov 09 2009 08:00:00 GMT+0100 (CET)", "user_id" : ObjectId("4d88d1f931f23813a10003e9"), "items" : [ ObjectId("4d88d1f931f23813a100016d"), ObjectId("4d88d1f931f23813a1000346"), ObjectId("4d88d1f931f23813a10001e7"), ObjectId("4d88d1f931f23813a10000db"), ObjectId("4d88d1f931f23813a1000091"), ObjectId("4d88d1f931f23813a10001c1"), ObjectId("4d88d1f931f23813a10001d3"), ObjectId("4d88d1f931f23813a100031b"), ObjectId("4d88d1f931f23813a1000130") ], "total" : 5871}
plain dates are good too
![Page 54: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/54.jpg)
> db.orders.find({ "placed_at": { $gte: new Date(2011,2,10), $lt: new Date(2011,2,11) } }).explain()
{ "cursor" : "BtreeCursor placed_at_-‐1", "nscanned" : 53, "nscannedObjects" : 53, "n" : 53, "millis" : 0, "indexBounds" : { "placed_at" : [ [ "Fri Mar 11 2011 00:00:00 GMT+0100 (CET)", "Thu Mar 10 2011 00:00:00 GMT+0100 (CET)" ] ] }
plain dates are good too
![Page 55: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/55.jpg)
# find all mondays of the yearnow = Time.now.beginning_of_year
now += 1.day until now.monday?mondays = [ now ]mondays << now += 7.days while now.year == Time.now.year
# find all orders placed on mondaysquery = { :$or => mondays.map do |day| { :placed_at => { :$gte => day.beginning_of_day, :$lte => day.end_of_day } } end}
puts query
plain dates are good too, but...(total sold on this year’s mondays)
![Page 56: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/56.jpg)
# find all mondays of the yearnow = Time.now.beginning_of_year
now += 1.day until now.monday?mondays = [ now ]mondays << now += 7.days while now.year == Time.now.year
# find all orders placed on mondaysquery = { :$or => mondays.map do |day| { :placed_at => { :$gte => day.beginning_of_day, :$lte => day.end_of_day } } end}
puts query
$ ruby src/orders_on_mondays.rb
{:$or=>[ {:placed_at=>{ :$gte=>2011-‐01-‐03 00:00:00 +0100, :$lte=>2011-‐01-‐03 23:59:59 +0100 }}, {:placed_at=>{ :$gte=>2011-‐01-‐10 00:00:00 +0100, :$lte=>2011-‐01-‐10 23:59:59 +0100 }}, {:placed_at=>{ :$gte=>2011-‐01-‐17 00:00:00 +0100, :$lte=>2011-‐01-‐17 23:59:59 +0100 }}, ...]}
plain dates are good too, but...(total sold on this year’s mondays)
![Page 57: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/57.jpg)
db["orders"].find({ :$or => mondays.map do |day| { :placed_at => { :$gte => day.beginning_of_day, :$lte => day.end_of_day } } end})
plain dates are good too, but...(it works but it’s too slooow)
![Page 58: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/58.jpg)
> db.orders.find({ $or: [ "placed_at":{ $gte: new Date(2011,2,3), $lt: new Date(2011,2,4) }, "placed_at":{ $gte: new Date(2011,2,10), $lt: new Date(2011,2,11) } ] }).explain()
{ "clauses" : [{ "cursor" : "BtreeCursor placed_at_-‐1", "indexBounds" : { "placed_at" : [[ "Tue Mar 3 2011 00:00:00 GMT+0100 (CET)", "Wed Mar 4 2011 00:00:00 GMT+0100 (CET)" ]]} }, { "cursor" : "BtreeCursor placed_at_-‐1", "indexBounds" : { "placed_at" : [[ "Tue Mar 10 2011 00:00:00 GMT+0100 (CET)", "Wed Mar 11 2011 00:00:00 GMT+0100 (CET)"
plain dates are good too, but...(why it’s too slow)
![Page 59: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/59.jpg)
> db.orders.findOne()
{ "_id" : ObjectId("4d88bf1f31f23812de0003fd"), "placed_at" : [ "2011", "201103", "2011w11", "20110316" ], "user_id" : ObjectId("4d88bf1f31f23812de0003e9"), "items" : [ ObjectId("4d88bf1f31f23812de0003da"), ObjectId("4d88bf1f31f23812de000047"), ObjectId("4d88bf1f31f23812de000078"), ObjectId("4d88bf1f31f23812de000068"), ObjectId("4d88bf1f31f23812de000288") ], "total" : 3502}
with destructured dates(total sold on mondays this year)
![Page 60: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/60.jpg)
now = Time.now.beginning_of_year
now += 1.day until now.monday?mondays = [ now ]mondays << now += 7.days while now.year == Time.now.year
orders = db["orders"].find({ :placed_at => { :$in => mondays.map {|day| day.strftime("%Y%m%d")} }})
puts orders.explain
with destructured dates(total sold on mondays this year)
![Page 61: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/61.jpg)
with destructured dates(total sold on mondays this year)
now = Time.now.beginning_of_year
now += 1.day until now.monday?mondays = [ now ]mondays << now += 7.days while now.year == Time.now.year
orders = db["orders"].find({ :placed_at => { :$in => mondays.map {|day| day.strftime("%Y%m%d")} }})
puts orders.explain
$ ruby src/orders_on_mondays.rb
{ "cursor"=>"BtreeCursor placed_at_-‐1 multi", "nscanned"=>744, "nscannedObjects"=>744, "n"=>744, "millis"=>1, "indexBounds"=>{ "placed_at"=>[ ["20120102", "20120102"], ["20111226", "20111226"], ["20111219", "20111219"], ["20111212", "20111212"], ["20111205", "20111205"], ["20111128", "20111128"], ["20111121", "20111121"], ... ] }}
![Page 62: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/62.jpg)
full query power with
$whereoperator
query
&d
esig
n
![Page 63: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/63.jpg)
pomodori(find who is ticking)
> db.pomodori.findOne(){ "_id" : ObjectId("4d8916ed31f2381480000021"), "duration" : 1500, "interruptions" : 0, "after_break_of" : 0, "started_at" : "Mon Mar 14 2011 08:05:00 GMT+0100 (CET)", "squashed_at" : "Mon Mar 14 2011 08:07:31 GMT+0100 (CET)", "in_day" : { "position" : 1, "is_last" : false }, "task_id" : ObjectId("4d8916ec31f2381480000014"), "user_id" : ObjectId("4d8916ec31f2381480000010"), "annotations" : [ ]}
![Page 64: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/64.jpg)
now = Time.now.yesterday.beginning_of_day + 10.hourstimestamp_of_now = now.to_i
ticking = db["pomodori"].find( :$where => <<-EOF var startedAt = this.started_at.getTime()/1000 return ((startedAt + this.duration) > #{timestamp_of_now}) && (startedAt < #{timestamp_of_now}) EOF)
puts ticking.map{|pomodoro| pomodoro["_id"]}
pomodori(find who is ticking)
![Page 65: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/65.jpg)
pomodori(find who is ticking)
now = Time.now.yesterday.beginning_of_day + 10.hourstimestamp_of_now = now.to_i
ticking = db["pomodori"].find( :$where => <<-EOF var startedAt = this.started_at.getTime()/1000 return ((startedAt + this.duration) > #{timestamp_of_now}) && (startedAt < #{timestamp_of_now}) EOF)
puts ticking.map{|pomodoro| pomodoro["_id"]}
$ ruby src/find_who_is_ticking.rb 4d8916ef31f238148000011d4d8916f231f23814800002714d8916f931f23814800004dd4d8916f931f23814800004e0
![Page 66: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/66.jpg)
now = Time.now.yesterday.beginning_of_day + 10.hourstimestamp_of_now = now.to_iuser_id = BSON::ObjectId.from_string("4d8916ec31f2381480000010")
ticking = db["pomodori"].find( :user_id => user_id, :$where => <<-EOF var startedAt = this.started_at.getTime()/1000 return ((startedAt + this.duration) > #{timestamp_of_now}) && (startedAt < #{timestamp_of_now}) EOF)
puts ticking.map{|pomodoro| pomodoro["_id"]}
pomodori(find who is ticking for an user)
![Page 67: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/67.jpg)
now = Time.now.yesterday.beginning_of_day + 10.hourstimestamp_of_now = now.to_iuser_id = BSON::ObjectId.from_string("4d8916ec31f2381480000010")
ticking = db["pomodori"].find( :user_id => user_id, :$where => <<-EOF var startedAt = this.started_at.getTime()/1000 return ((startedAt + this.duration) > #{timestamp_of_now}) && (startedAt < #{timestamp_of_now}) EOF)
puts ticking.map{|pomodoro| pomodoro["_id"]}
pomodori(find who is ticking for an user)
$ ruby src/find_who_is_ticking_for_an_user.rb 4d8916ef31f238148000011d
![Page 68: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/68.jpg)
related_to_maps = db["pomodori"].find( :$where => <<-EOF db.tasks.findOne({ "_id": this.task_id }).tags.indexOf("maps") >= 0 EOF)
puts related_to_maps.map{|pomodoro| pomodoro["_id"]}
pomodori(related to tasks tagged with “maps”)
![Page 69: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/69.jpg)
related_to_maps = db["pomodori"].find( :$where => <<-EOF db.tasks.findOne({ "_id": this.task_id }).tags.indexOf("maps") >= 0 EOF)
puts related_to_maps.map{|pomodoro| pomodoro["_id"]}
pomodori(related to tasks tagged with “maps”)
$ ruby src/related_to_maps.rb 4d8916fa31f23814800005794d8916fa31f238148000057b4d8916fa31f238148000057d4d8916fa31f2381480000580
![Page 70: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/70.jpg)
related_to_maps = db["pomodori"].find( :$where => <<-EOF db.tasks.findOne({ "_id": this.task_id }).tags.indexOf("maps") >= 0 EOF)
puts related_to_maps.explain
pomodori(don’t be carried away :-))
$ ruby src/related_to_maps.rb { "cursor"=>"BasicCursor", "nscanned"=>461, "nscannedObjects"=>461, "n"=>4, "millis"=>52, "indexBounds"=>{}, "allPlans"=>[...]}
![Page 71: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/71.jpg)
related_to_maps = db["pomodori"].find(:task_id => { :$in => db["tasks"].find( {:tags => "maps"}, :fields => {:_id => 1} ).map{|task| task["_id"]}}) puts related_to_maps.map{|pomodoro| pomodoro["_id"]}
pomodori(related to... a better solution)
$ ruby src/related_to_maps.rb 4d8916fa31f23814800005794d8916fa31f238148000057b4d8916fa31f238148000057d4d8916fa31f2381480000580
![Page 72: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/72.jpg)
related_to_maps = db["pomodori"].find(:task_id => { :$in => db["tasks"].find( {:tags => "maps"}, :fields => {:_id => 1} ).map{|task| task["_id"]}}) puts related_to_maps.map{|pomodoro| pomodoro["_id"]}
$ ruby src/related_to_maps.rb { "cursor"=>"BtreeCursor tags_1", "nscanned"=>3, "nscannedObjects"=>3, "n"=>3, "millis"=>0, ...}
{ "cursor"=>"BtreeCursor task_id_1 multi", "nscanned"=>4, "nscannedObjects"=>4, "n"=>4, "millis"=>0, ...}
pomodori(related to... a better solution)
![Page 73: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/73.jpg)
real time analytics with increments
query
&d
esig
n
![Page 74: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/74.jpg)
result = db["visits"].update( { :_id => Digest::MD5.hexdigest(url) }, { :$inc => { :hits => 1 } }, :upsert => true, :safe => true)
puts "Update: #{result.inspect}"
puts db["visits"].find_one(:_id => Digest::MD5.hexdigest(url))
keep track of url’s visits(upsert with custom id)
![Page 75: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/75.jpg)
keep track of url’s visits(upsert with custom id)
result = db["visits"].update( { :_id => Digest::MD5.hexdigest(url) }, { :$inc => { :hits => 1 } }, :upsert => true, :safe => true)
puts "Update: #{result.inspect}"
puts db["visits"].find_one(:_id => Digest::MD5.hexdigest(url))
$ ruby src/realtime_analytics.rb Update: { "err"=>nil, "updatedExisting"=>false, "n"=>1, "ok"=>1.0}{"_id"=>"2d86a774beffe90e715a8028c7bd177b", "hits"=>1}
$ ruby src/realtime_analytics.rb Update: { "err"=>nil, "updatedExisting"=>true, "n"=>1, "ok"=>1.0}{"_id"=>"2d86a774beffe90e715a8028c7bd177b", "hits"=>2}
![Page 76: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/76.jpg)
url_digest = Digest::MD5.hexdigest(url)ids = [ [ url_digest, Time.now.strftime("%Y%m%d") ].join("-"), [ url_digest, Time.now.strftime("%Y%m") ].join("-"), [ url_digest, Time.now.strftime("%Y") ].join("-"), [ url_digest, user_id ].join("-")]puts "Expect to upsert: \n#{ids}"
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)puts result.inspectputs db["visits"].all
url’s visits aggregated by time(upsert with multiple documents)
![Page 77: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/77.jpg)
url_digest = Digest::MD5.hexdigest(url)ids = [ [ url_digest, Time.now.strftime("%Y%m%d") ].join("-"), [ url_digest, Time.now.strftime("%Y%m") ].join("-"), [ url_digest, Time.now.strftime("%Y") ].join("-"), [ url_digest, user_id ].join("-")]puts "Expect to upsert: \n#{ids}"
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)puts result.inspectputs db["visits"].all
$ ruby src/realtime_analytics_with_aggregation.rb Expect to upsert:[ "2d86a774beffe90e715a8028c7bd177b-‐20110323", "2d86a774beffe90e715a8028c7bd177b-‐201103", "2d86a774beffe90e715a8028c7bd177b-‐2011", "2d86a774beffe90e715a8028c7bd177b-‐4d899fab31f238165c000001"]
{ "err"=>nil, "updatedExisting"=>false, "upserted"=>BSON::ObjectId('4d899fabe23bd37e768ae76d'), "n"=>1, "ok"=>1.0}
{"_id"=>BSON::ObjectId('4d899fabe23bd37e768ae76d'), "hits"=>1}
url’s visits aggregated by time(upsert with multiple documents)
![Page 78: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/78.jpg)
url_digest = Digest::MD5.hexdigest(url)ids = [ [ url_digest, Time.now.strftime("%Y%m%d") ].join("-"), [ url_digest, Time.now.strftime("%Y%m") ].join("-"), [ url_digest, Time.now.strftime("%Y") ].join("-"), [ url_digest, user_id ].join("-")]puts "Expect to upsert: \n#{ids}"
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)puts result.inspectputs db["visits"].all
$ ruby src/realtime_analytics_with_aggregation.rb Expect to upsert:[ "2d86a774beffe90e715a8028c7bd177b-‐20110323", "2d86a774beffe90e715a8028c7bd177b-‐201103", "2d86a774beffe90e715a8028c7bd177b-‐2011", "2d86a774beffe90e715a8028c7bd177b-‐4d899fab31f238165c000001"]
{ "err"=>nil, "updatedExisting"=>false, "upserted"=>BSON::ObjectId('4d899fabe23bd37e768ae76e'), "n"=>1, "ok"=>1.0}
{"_id"=>BSON::ObjectId('4d899fabe23bd37e768ae76d'), "hits"=>1}{"_id"=>BSON::ObjectId('4d899fabe23bd37e768ae76e'), "hits"=>1}
url’s visits aggregated by time(upsert with multiple documents)
![Page 79: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/79.jpg)
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)
if result["n"] != ids.size updated_ids = db["visits"].find( { :_id => { :$in => ids } }, :fields => { :_id => true } ).map{|document| document["_id"]}
db["visits"].insert((ids - updated_ids).map do |id| { :_id => id, :hits => 1 } end)
db["visits"].remove(:_id => result["upserted"]) if result["upserted"]end
url’s visits aggregated by time(look before you leap)
![Page 80: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/80.jpg)
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)
if result["n"] != ids.size updated_ids = db["visits"].find( { :_id => { :$in => ids } }, :fields => { :_id => true } ).map{|document| document["_id"]}
db["visits"].insert((ids - updated_ids).map do |id| { :_id => id, :hits => 1 } end)
db["visits"].remove(:_id => result["upserted"]) if result["upserted"]end
$ ruby src/realtime_analytics_with_aggregation.rb{ "err"=>nil, "updatedExisting"=>false, "upserted"=>BSON::ObjectId('4d89a5ebe23bd37e768ae76f'), "n"=>1, "ok"=>1.0
}
{"_id"=>"<url_digest>-‐20110323", "hits"=>1}{"_id"=>"<url_digest>-‐201103", "hits"=>1}{"_id"=>"<url_digest>-‐2011", "hits"=>1}{"_id"=>"<url_digest>-‐4d89a43b31f238167a000001", "hits"=>1}
url’s visits aggregated by time(look before you leap)
![Page 81: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/81.jpg)
result = db["visits"].update( { :_id => { :$in => ids } }, { :$inc => { :hits => 1 } }, :multi => true, :upsert => true, :safe => true)
if result["n"] != ids.size updated_ids = db["visits"].find( { :_id => { :$in => ids } }, :fields => { :_id => true } ).map{|document| document["_id"]}
db["visits"].insert((ids - updated_ids).map do |id| { :_id => id, :hits => 1 } end)
db["visits"].remove(:_id => result["upserted"]) if result["upserted"]end
$ ruby src/realtime_analytics_with_aggregation.rb{ "err"=>nil, "updatedExisting"=>true, "n"=>3, "ok"=>1.0}
{"_id"=>"<url_digest>-‐20110323", "hits"=>2}{"_id"=>"<url_digest>-‐201103", "hits"=>2}{"_id"=>"<url_digest>-‐2011", "hits"=>2}{"_id"=>"<url_digest>-‐4d89a43b31f238167a000001", "hits"=>1}{"_id"=>"<url_digest>-‐4d89a44231f238167e000001", "hits"=>1}
url’s visits aggregated by time(look before you leap)
![Page 82: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/82.jpg)
incrementalmap/reduce
query
&d
esig
n
![Page 83: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/83.jpg)
map/reduce hits per day(we have raw events)
> db.visit_events.findOne(){ "_id" : ObjectId("4d89fc6531f2381d2c00000b"), "url" : "8aa8b68e0b849f70df6dbb3031c6182b", "user_id" : ObjectId("4d89fc6531f2381d2c000005"), "at" : "Thu Jan 13 2011 08:00:06 GMT+0100 (CET)"}
![Page 84: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/84.jpg)
def generate_events(visits, db, now) visits.times do |time| now += BETWEEN_VISITS.sample.seconds db["visit_events"].insert( :url => Digest::MD5.hexdigest(URLS.sample), :user_id => USERS.sample[:id], :at => now ) endend
generate_events(10_000, db, now)
map/reduce hits per day(generate data WITH something like)
![Page 85: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/85.jpg)
MAP = <<-EOF function() { emit([ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 }) }EOF
REDUCE = <<-EOF function(key, values) { var hits = 0 for(var index in values) hits += values[index]["hits"] return { "hits": hits } }EOF
result = db["visit_events"].map_reduce( MAP, REDUCE, :out => "visits", :raw => true, :verbose => true)
puts result.inspect
map/reduce hits per day(simple map/reduce)
![Page 86: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/86.jpg)
MAP = <<-EOF function() { emit([ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 }) }EOF
REDUCE = <<-EOF function(key, values) { var hits = 0 for(var index in values) hits += values[index]["hits"] return { "hits": hits } }EOF
result = db["visit_events"].map_reduce( MAP, REDUCE, :out => "visits", :raw => true, :verbose => true)
puts result.inspect
map/reduce hits per day(date.prototype.format don’t exists)
![Page 87: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/87.jpg)
MAP = <<-EOF function() { Date.prototype.format = function(format) { ... } emit([ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 }) }EOF
REDUCE = <<-EOF function(key, values) { var hits = 0 for(var index in values) hits += values[index]["hits"] return { "hits": hits } }EOF
map/reduce hits per day(implement format in place)
![Page 88: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/88.jpg)
MAP = <<-EOF function() { if (!Date.prototype.format) { Date.prototype.format = function(format) { ... } } emit([ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 }) }EOF
REDUCE = <<-EOF function(key, values) { var hits = 0 for(var index in values) hits += values[index]["hits"] return { "hits": hits } }EOF
map/reduce hits per day(implement format only if needed)
![Page 89: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/89.jpg)
db[Mongo::DB::SYSTEM_JS_COLLECTION].save( :_id => "formatDate", :value => BSON::Code.new( <<-EOF function(date, format) { if (!Date.prototype.format) { Date.prototype.format = function(format) { ... } } return date.format(format) } EOF ))
MAP = <<-EOF function() { emit([ this.url, formatDate(this.at, "Ymd") ].join("-"), {"hits":1}) }EOF
map/reduce hits per day(implement format once and for all)
![Page 90: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/90.jpg)
db[Mongo::DB::SYSTEM_JS_COLLECTION].save( :_id => "load", :value => BSON::Code.new( <<-EOF function(module) { if ((module === "date") && !Date.prototype.format) { Date.prototype.format = function(format) { ... } } return true } EOF ))
MAP = <<-EOF function() { load("date") && emit( [ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 } ) }EOF
map/reduce hits per day(implement format once and for all)
![Page 91: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/91.jpg)
MAP = <<-EOF function() { emit([ this.url, this.at.format("Ymd") ].join("-"), { "hits": 1 }) }EOF
REDUCE = <<-EOF function(key, values) { var hits = 0 for(var index in values) hits += values[index]["hits"] return { "hits": hits } }EOF
result = db["visit_events"].map_reduce( MAP, REDUCE, :out => "visits", :raw => true, :verbose => true)
puts result.inspect
map/reduce hits per day(ok, but could be taking too long)
$ ruby src/incremental_mr.rb { "result"=>"visits", "timeMillis"=>4197, "timing"=> { "mapTime"=>3932, "emitLoop"=>4170, "total"=>4197 }, "counts"=> { "input"=>10000, "emit"=>10000, "output"=>200 }, "ok"=>1.0}
![Page 92: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/92.jpg)
> db.visits.find()
{ "_id" : "019640ff7952425b1b8695605459d223-‐20110316", "value" : { "hits" : 47 }}
{ "_id" : "019640ff7952425b1b8695605459d223-‐20110317", "value" : { "hits" : 49 }}
{ "_id" : "019640ff7952425b1b8695605459d223-‐20110318", "value" : { "hits" : 59 } }
{ "_id" : "019640ff7952425b1b8695605459d223-‐20110319", "value" : { "hits" : 37 } }
map/reduce hits per day(ok, every time we need to start over)
![Page 93: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/93.jpg)
map/reduce hits per day(incremental with savepoints)
temporarycollection
visit-elementscollection
visitcollection
map/reduceon last changed
documentsupsert
![Page 94: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/94.jpg)
map/reduce hits per day(incremental with savepoints)
db.create_collection("visit_events", :capped => true, :max => 50_000, :size => 5_000_000)
temporarycollection
visit-elementscollection
map/reduceon last changed
documents
![Page 95: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/95.jpg)
FINALIZE = <<-EOF function(key, value) { db.visits.update( { "_id": key }, { $inc: { "hits": value.hits } }, true ) }EOF
map/reduce hits per day(incremental with savepoints)
temporarycollection
visitcollection
upsert
![Page 96: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/96.jpg)
generate_events(number_of_events, db, now)
from = from_last_updated(db)to = to_last_inserted(db)
result = db["visit_events"].map_reduce( MAP, REDUCE, :finalize => FINALIZE, :query => { :_id => { :$gt => from, :$lte => to } }, :raw => true, :verbose => true)
db["visits"].save(:_id => "savepoint", :at => to)
map/reduce hits per day(incremental with savepoints)
![Page 97: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/97.jpg)
generate_events(number_of_events, db, now)
from = from_last_updated(db)to = to_last_inserted(db)
result = db["visit_events"].map_reduce( MAP, REDUCE, :finalize => FINALIZE, :query => { :_id => { :$gt => from, :$lte => to } }, :raw => true, :verbose => true)
db["visits"].save(:_id => "savepoint", :at => to)
map/reduce hits per day(incremental with savepoints)
$ ruby src/incremental_mr.rb -‐e 10000 { "result"=>"tmp.mr.mapreduce_1300892393_60", "timeMillis"=>4333, "timing"=>{...}, "counts"=>{ "input"=>10000, "emit"=>10000, "output"=>196 }, "ok"=>1.0}
{ "_id"=>"05241f07d0e3ab6a227e67b33ea0b509-‐20110113", "hits"=>26}
![Page 98: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/98.jpg)
generate_events(number_of_events, db, now)
from = from_last_updated(db)to = to_last_inserted(db)
result = db["visit_events"].map_reduce( MAP, REDUCE, :finalize => FINALIZE, :query => { :_id => { :$gt => from, :$lte => to } }, :raw => true, :verbose => true)
db["visits"].save(:_id => "savepoint", :at => to)
map/reduce hits per day(incremental with savepoints)
$ ruby src/incremental_mr.rb -‐e 4999 { "result"=>"tmp.mr.mapreduce_1300892399_61", "timeMillis"=>2159, "timing"=>{...}, "counts"=>{ "input"=>4999, "emit"=>4999, "output"=>146 }, "ok"=>1.0}
{ "_id"=>"05241f07d0e3ab6a227e67b33ea0b509-‐20110113", "hits"=>64}
![Page 99: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/99.jpg)
def savepoint(db) db["visits"].find_one(:_id => "savepoint") or { "at" => BSON::ObjectId.from_time(10.years.ago) }end
def from_last_updated(db) savepoint["at"]end
def to_last_inserted(db) db["visit_events"].find.sort([:_id, Mongo::DESCENDING]).first["_id"]end
map/reduce hits per day(incremental with savepoints)
![Page 100: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/100.jpg)
externalmap/reduce
query
&d
esig
n
![Page 101: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/101.jpg)
master slave
replicate data
use an external mongod processto execute map/reduce jobs
![Page 102: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/102.jpg)
master slave
map/reduceon last
replicateddata
use an external mongod processto execute map/reduce jobs
![Page 103: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/103.jpg)
master slave
push back results
use an external mongod processto execute map/reduce jobs
![Page 104: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/104.jpg)
look at the shell sourceis more powerful than you think
![Page 105: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/105.jpg)
documents
embeddedor
linked?
query
&d
esig
n
![Page 106: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/106.jpg)
life cycle:when root document
is deleted, he can stand for himself?
if yesembedded
if nolinked
![Page 107: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/107.jpg)
if yesembedded
if nolinked
are always fetched together?
![Page 108: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/108.jpg)
if yesembedded
if nolinked
his attributes are used to find the root
document?
![Page 109: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/109.jpg)
if yesembedded
if nolinked
he’s small?
![Page 110: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/110.jpg)
if yesembedded
if nolinked
he’s unique or there are less then
hundreds?
![Page 111: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/111.jpg)
Styl
e
query
&d
esig
n
scale
![Page 112: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/112.jpg)
distributedreads with
replicasets
scale
![Page 113: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/113.jpg)
master
slave
slave
read/write
read
read
replicate
replicate
+ Durability+ fault tolerance
![Page 114: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/114.jpg)
(seems stupid but...)
pumpyour
hardwarescale
![Page 115: MongoDB With Style](https://reader034.vdocument.in/reader034/viewer/2022051314/54b7903c4a7959db528b4a1a/html5/thumbnails/115.jpg)
(seems stupid but...)
call 10gensure they can
help :-)scale