Download - Debugging Ruby (Aman Gupta)
![Page 1: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/1.jpg)
Debugging Rubywith MongoDB
Aman Gupta@tmm1
![Page 2: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/2.jpg)
debugging ruby?
• i use ruby
![Page 3: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/3.jpg)
debugging ruby?
• i use ruby
• my ruby processes use a lot of ram
![Page 4: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/4.jpg)
debugging ruby?
• i use ruby
• my ruby processes use a lot of ram
• i want to fix this
![Page 5: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/5.jpg)
let’s build a debugger
• step 1: collect data
• list of all ruby objects in memory
![Page 6: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/6.jpg)
let’s build a debugger
• step 1: collect data
• list of all ruby objects in memory
• step 2: analyze data
• group by type
• group by file/line
![Page 7: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/7.jpg)
• simple patch to ruby VM (300 lines of C)
• http://gist.github.com/73674
• simple text based output format
0x154750 @ -e:1 is OBJECT of type: T0x15476c @ -e:1 is HASH which has data0x154788 @ -e:1 is ARRAY of len: 00x1547c0 @ -e:1 is STRING (SHARED) len: 2 and val: hi0x1547dc @ -e:1 is STRING len: 1 and val: T0x154814 @ -e:1 is CLASS named: T inherits from Object0x154a98 @ -e:1 is STRING len: 2 and val: hi0x154b40 @ -e:1 is OBJECT of type: Range
version 1: collect data
![Page 8: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/8.jpg)
version 1: analyze data$ wc -l /tmp/ruby.heap
1571529 /tmp/ruby.heap
![Page 9: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/9.jpg)
version 1: analyze data
$ cat /tmp/ruby.heap | awk '{ print $3 }' | sort | uniq -c | sort -g | tail -1
236840 memcached/memcached.rb:316
$ wc -l /tmp/ruby.heap
1571529 /tmp/ruby.heap
![Page 10: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/10.jpg)
version 1: analyze data
$ cat /tmp/ruby.heap | awk '{ print $3 }' | sort | uniq -c | sort -g | tail -1
236840 memcached/memcached.rb:316
$ grep "memcached.rb:316" /tmp/ruby.heap | awk '{ print $5 }' | sort | uniq -c | sort -g | tail -5
10948 ARRAY 20355 OBJECT 30744 DATA 64952 HASH 123290 STRING
$ wc -l /tmp/ruby.heap
1571529 /tmp/ruby.heap
![Page 11: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/11.jpg)
version 1
![Page 12: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/12.jpg)
version 1
• it works!
![Page 13: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/13.jpg)
version 1
• it works!
• but...
![Page 14: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/14.jpg)
version 1
• it works!
• but...
• must patch and rebuild ruby binary
![Page 15: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/15.jpg)
version 1
• it works!
• but...
• must patch and rebuild ruby binary
• no information about references between objects
![Page 16: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/16.jpg)
version 1
• it works!
• but...
• must patch and rebuild ruby binary
• no information about references between objects
• limited analysis via shell scripting
![Page 17: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/17.jpg)
version 2 goals
![Page 18: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/18.jpg)
• better data format
version 2 goals
![Page 19: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/19.jpg)
• better data format
• simple: one line of text per object
version 2 goals
![Page 20: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/20.jpg)
• better data format
• simple: one line of text per object
• expressive: include all details about object contents and references
version 2 goals
![Page 21: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/21.jpg)
• better data format
• simple: one line of text per object
• expressive: include all details about object contents and references
• easy to use: easy to generate from C code & easy to consume from various scripting languages
version 2 goals
![Page 22: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/22.jpg)
JSON!
![Page 23: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/23.jpg)
version 2 is memprof
![Page 24: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/24.jpg)
version 2 is memprof• no patches to ruby necessary
• gem install memprof
• require ‘memprof’
• Memprof.dump_all(“/tmp/app.json”)
![Page 25: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/25.jpg)
version 2 is memprof• no patches to ruby necessary
• gem install memprof
• require ‘memprof’
• Memprof.dump_all(“/tmp/app.json”)
• C extension for MRI ruby VMhttp://github.com/ice799/memprof
• uses libyajl to dump out all ruby objects as json
![Page 26: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/26.jpg)
stringsMemprof.dump{ “hello” + “world”}
![Page 27: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/27.jpg)
{ "_id": "0x19c610",
"file": "-e", "line": 1,
"type": "string", "class": "0x1ba7f0", "class_name": "String",
"length": 10, "data": "helloworld"}
memory address of object
stringsMemprof.dump{ “hello” + “world”}
![Page 28: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/28.jpg)
{ "_id": "0x19c610",
"file": "-e", "line": 1,
"type": "string", "class": "0x1ba7f0", "class_name": "String",
"length": 10, "data": "helloworld"}
memory address of object
file and line where string was created
stringsMemprof.dump{ “hello” + “world”}
![Page 29: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/29.jpg)
{ "_id": "0x19c610",
"file": "-e", "line": 1,
"type": "string", "class": "0x1ba7f0", "class_name": "String",
"length": 10, "data": "helloworld"}
memory address of object
file and line where string was created
address of the class object “String”
stringsMemprof.dump{ “hello” + “world”}
![Page 30: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/30.jpg)
{ "_id": "0x19c610",
"file": "-e", "line": 1,
"type": "string", "class": "0x1ba7f0", "class_name": "String",
"length": 10, "data": "helloworld"}
memory address of object
file and line where string was created
length and contentsof this string instance
address of the class object “String”
stringsMemprof.dump{ “hello” + “world”}
![Page 31: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/31.jpg)
arraysMemprof.dump{ [ 1, :b, 2.2, “d” ]}
![Page 32: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/32.jpg)
{ "_id": "0x19c5c0",
"class": "0x1b0d18", "class_name": "Array",
"length": 4, "data": [ 1, ":b",
"0x19c750", "0x19c598" ]}
arraysMemprof.dump{ [ 1, :b, 2.2, “d” ]}
![Page 33: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/33.jpg)
{ "_id": "0x19c5c0",
"class": "0x1b0d18", "class_name": "Array",
"length": 4, "data": [ 1, ":b",
"0x19c750", "0x19c598" ]}
integers and symbols are stored in the array itself
arraysMemprof.dump{ [ 1, :b, 2.2, “d” ]}
![Page 34: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/34.jpg)
floats and strings are separate ruby objects
{ "_id": "0x19c5c0",
"class": "0x1b0d18", "class_name": "Array",
"length": 4, "data": [ 1, ":b",
"0x19c750", "0x19c598" ]}
integers and symbols are stored in the array itself
arraysMemprof.dump{ [ 1, :b, 2.2, “d” ]}
![Page 35: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/35.jpg)
hashesMemprof.dump{ { :a => 1, “b” => 2.2 }}
![Page 36: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/36.jpg)
hashes{ "_id": "0x19c598",
"type": "hash", "class": "0x1af170", "class_name": "Hash",
"default": null,
"length": 2, "data": [ [ ":a", 1 ], [ "0xc728", "0xc750" ] ]}
Memprof.dump{ { :a => 1, “b” => 2.2 }}
![Page 37: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/37.jpg)
hashes{ "_id": "0x19c598",
"type": "hash", "class": "0x1af170", "class_name": "Hash",
"default": null,
"length": 2, "data": [ [ ":a", 1 ], [ "0xc728", "0xc750" ] ]}
hash entries as key/value pairs
Memprof.dump{ { :a => 1, “b” => 2.2 }}
![Page 38: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/38.jpg)
hashes{ "_id": "0x19c598",
"type": "hash", "class": "0x1af170", "class_name": "Hash",
"default": null,
"length": 2, "data": [ [ ":a", 1 ], [ "0xc728", "0xc750" ] ]}
hash entries as key/value pairs
no default proc
Memprof.dump{ { :a => 1, “b” => 2.2 }}
![Page 39: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/39.jpg)
classesMemprof.dump{ class Hello @@var=1 Const=2 def world() end end}
![Page 40: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/40.jpg)
classesMemprof.dump{ class Hello @@var=1 Const=2 def world() end end}
{ "_id": "0x19c408",
"type": "class", "name": "Hello", "super": "0x1bfa48", "super_name": "Object",
"ivars": { "@@var": 1, "Const": 2 }, "methods": { "world": "0x19c318" }}
![Page 41: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/41.jpg)
classesMemprof.dump{ class Hello @@var=1 Const=2 def world() end end}
{ "_id": "0x19c408",
"type": "class", "name": "Hello", "super": "0x1bfa48", "super_name": "Object",
"ivars": { "@@var": 1, "Const": 2 }, "methods": { "world": "0x19c318" }}
superclass object reference
![Page 42: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/42.jpg)
classesMemprof.dump{ class Hello @@var=1 Const=2 def world() end end}
{ "_id": "0x19c408",
"type": "class", "name": "Hello", "super": "0x1bfa48", "super_name": "Object",
"ivars": { "@@var": 1, "Const": 2 }, "methods": { "world": "0x19c318" }}
class variables and constants are stored in the instance variable table
superclass object reference
![Page 43: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/43.jpg)
classesMemprof.dump{ class Hello @@var=1 Const=2 def world() end end}
{ "_id": "0x19c408",
"type": "class", "name": "Hello", "super": "0x1bfa48", "super_name": "Object",
"ivars": { "@@var": 1, "Const": 2 }, "methods": { "world": "0x19c318" }}
class variables and constants are stored in the instance variable table
superclass object reference
references to method objects
![Page 44: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/44.jpg)
version 2: analyze data
![Page 45: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/45.jpg)
version 2: memprof.coma web-based heap visualizer and leak analyzer
![Page 46: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/46.jpg)
built on...
![Page 47: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/47.jpg)
built on...
$ mongoimport -d memprof -c rails --file /tmp/app.json$ mongo memprof
![Page 48: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/48.jpg)
built on...
$ mongoimport -d memprof -c rails --file /tmp/app.json$ mongo memprof
let’s run some queries.
![Page 49: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/49.jpg)
how many objects?> db.rails.count()809816
• ruby scripts create a lot of objects
• usually not a problem, but...
• MRI has a naïve stop-the-world mark/sweep GC
• fewer objects = faster GC = better performance
![Page 50: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/50.jpg)
what types of objects?> db.rails.distinct(‘type’)
[‘array’, ‘bignum’, ‘class’, ‘float’, ‘hash’, ‘module’, ‘node’, ‘object’, ‘regexp’, ‘string’, ...]
![Page 51: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/51.jpg)
mongodb: distinct
![Page 52: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/52.jpg)
mongodb: distinct• distinct(‘type’)
list of types of objects
![Page 53: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/53.jpg)
mongodb: distinct• distinct(‘type’)
list of types of objects
• distinct(‘file’)list of source files
![Page 54: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/54.jpg)
mongodb: distinct• distinct(‘type’)
list of types of objects
• distinct(‘file’)list of source files
• distinct(‘class_name’)list of instance class names
![Page 55: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/55.jpg)
mongodb: distinct• distinct(‘type’)
list of types of objects
• distinct(‘file’)list of source files
• distinct(‘class_name’)list of instance class names
• optionally filter first
• distinct(‘name’, {type:“class”})names of all defined classes
![Page 56: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/56.jpg)
improve performancewith indexes
> db.rails.ensureIndex({‘type’:1})
![Page 57: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/57.jpg)
improve performancewith indexes
> db.rails.ensureIndex({‘type’:1})
> db.rails.ensureIndex( {‘file’:1}, {background:true})
![Page 58: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/58.jpg)
mongodb: ensureIndex
• add an index on a field (if it doesn’t exist yet)
• improve performance of queries against common fields: type, class_name, super, file
![Page 59: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/59.jpg)
mongodb: ensureIndex
• add an index on a field (if it doesn’t exist yet)
• improve performance of queries against common fields: type, class_name, super, file
• can index embedded field names
• ensureIndex(‘methods.add’)
• find({‘methods.add’:{$exists:true}})find classes that define the method add
![Page 60: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/60.jpg)
> db.rails.group({ initial: {count:0}, key: {type:true}, cond: {}, reduce: function(obj, out) { out.count++ }}).sort(function(a,b){ return a.count - b.count})
how many objs per type?
group on type
![Page 61: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/61.jpg)
> db.rails.group({ initial: {count:0}, key: {type:true}, cond: {}, reduce: function(obj, out) { out.count++ }}).sort(function(a,b){ return a.count - b.count})
how many objs per type?
group on type
increment countfor each obj
![Page 62: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/62.jpg)
> db.rails.group({ initial: {count:0}, key: {type:true}, cond: {}, reduce: function(obj, out) { out.count++ }}).sort(function(a,b){ return a.count - b.count})
how many objs per type?
group on type
increment countfor each obj
sort results
![Page 63: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/63.jpg)
[ ..., {type: ‘array’, count: 7621}, {type: ‘string’, count: 69139}, {type: ‘node’, count: 365285}]
how many objs per type?
![Page 64: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/64.jpg)
[ ..., {type: ‘array’, count: 7621}, {type: ‘string’, count: 69139}, {type: ‘node’, count: 365285}]
lots of nodes
how many objs per type?
![Page 65: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/65.jpg)
[ ..., {type: ‘array’, count: 7621}, {type: ‘string’, count: 69139}, {type: ‘node’, count: 365285}]
• nodes represent ruby code
• stored like any other ruby object
• makes ruby completely dynamic
lots of nodes
how many objs per type?
![Page 66: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/66.jpg)
mongodb: group
![Page 67: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/67.jpg)
mongodb: group
• cond: query to filter objects before grouping
![Page 68: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/68.jpg)
mongodb: group
• cond: query to filter objects before grouping
• key: field(s) to group on
![Page 69: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/69.jpg)
mongodb: group
• cond: query to filter objects before grouping
• key: field(s) to group on
• initial: initial values for each group’s results
![Page 70: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/70.jpg)
mongodb: group
• cond: query to filter objects before grouping
• key: field(s) to group on
• initial: initial values for each group’s results
• reduce: aggregation function
![Page 71: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/71.jpg)
mongodb: group
![Page 72: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/72.jpg)
mongodb: group• by type or class
• key: {type:1}• key: {class_name:1}
![Page 73: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/73.jpg)
mongodb: group• by type or class
• key: {type:1}• key: {class_name:1}
• by file & line• key: {file:1, line:1}
![Page 74: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/74.jpg)
mongodb: group• by type or class
• key: {type:1}• key: {class_name:1}
• by file & line• key: {file:1, line:1}
• by type in a specific file• cond: {file: “app.rb”},
key: {file:1, line:1}
![Page 75: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/75.jpg)
mongodb: group• by type or class
• key: {type:1}• key: {class_name:1}
• by file & line• key: {file:1, line:1}
• by type in a specific file• cond: {file: “app.rb”},
key: {file:1, line:1}
• by length of strings in a specific file• cond: {file:“app.rb”,type:‘string’},
key: {length:1}
![Page 76: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/76.jpg)
what subclasses String?> db.rails.find( {super_name:"String"}, {name:1})
{name: "ActiveSupport::SafeBuffer"}{name: "ActiveSupport::StringInquirer"}{name: "SQLite3::Blob"}{name: "ActiveModel::Name"}{name: "Arel::Attribute::Expressions"}{name: "ActiveSupport::JSON::Variable"}
![Page 77: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/77.jpg)
what subclasses String?> db.rails.find( {super_name:"String"}, {name:1})
{name: "ActiveSupport::SafeBuffer"}{name: "ActiveSupport::StringInquirer"}{name: "SQLite3::Blob"}{name: "ActiveModel::Name"}{name: "Arel::Attribute::Expressions"}{name: "ActiveSupport::JSON::Variable"}
select only name field
![Page 78: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/78.jpg)
mongodb: find
![Page 79: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/79.jpg)
mongodb: find
• find({type:‘string’})all strings
![Page 80: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/80.jpg)
mongodb: find
• find({type:‘string’})all strings
• find({type:{$ne:‘string’}})everything except strings
![Page 81: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/81.jpg)
mongodb: find
• find({type:‘string’})all strings
• find({type:{$ne:‘string’}})everything except strings
• find({type:‘string’}, {data:1})only select string’s data field
![Page 82: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/82.jpg)
the largest objects?> db.rails.find( {type: {$in:['string','array','hash']} }, {type:1,length:1}).sort({length:-1}).limit(3) {type: "string", length: 2308}{type: "string", length: 1454}{type: "string", length: 1238}
![Page 83: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/83.jpg)
mongodb: sort, limit/skip
![Page 84: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/84.jpg)
mongodb: sort, limit/skip
• sort({length:-1,file:1})sort by length desc, file asc
![Page 85: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/85.jpg)
mongodb: sort, limit/skip
• sort({length:-1,file:1})sort by length desc, file asc
• limit(10)first 10 results
![Page 86: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/86.jpg)
mongodb: sort, limit/skip
• sort({length:-1,file:1})sort by length desc, file asc
• limit(10)first 10 results
• skip(10).limit(10)second 10 results
![Page 87: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/87.jpg)
when were objs created?• useful to look at objects over time
• each obj has a timestamp of when it was created
![Page 88: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/88.jpg)
when were objs created?• useful to look at objects over time
• each obj has a timestamp of when it was created
• find minimum time, call it start_time
![Page 89: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/89.jpg)
when were objs created?• useful to look at objects over time
• each obj has a timestamp of when it was created
• find minimum time, call it start_time
• create buckets for every minute of execution sincestart
![Page 90: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/90.jpg)
when were objs created?• useful to look at objects over time
• each obj has a timestamp of when it was created
• find minimum time, call it start_time
• create buckets for every minute of execution sincestart
• place objects into buckets
![Page 91: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/91.jpg)
when were objs created?> db.rails.mapReduce(function(){ var secs = this.time - start_time; var mins_since_start = secs % 60; emit(mins_since_start, 1); }, function(key, vals){ for(var i=0,sum=0; i<vals.length; sum += vals[i++]); return sum; }, { scope: { start_time: db.rails.find().sort({time:1}).limit(1)[0].time } }){result:"tmp.mr_1272615772_3"}
start_time = min(time)
![Page 92: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/92.jpg)
mongodb: mapReduce• arguments
•map: function that emits one or more key/value pairs given each object this
• reduce: function to return aggregate result, given key and list of values
• scope: global variables to set for funcs
![Page 93: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/93.jpg)
mongodb: mapReduce• arguments
•map: function that emits one or more key/value pairs given each object this
• reduce: function to return aggregate result, given key and list of values
• scope: global variables to set for funcs
• results
• stored in a temporary collection(tmp.mr_1272615772_3)
![Page 94: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/94.jpg)
when were objs created?> db.tmp.mr_1272615772_3.count()12
script was running for 12 minutes
![Page 95: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/95.jpg)
when were objs created?> db.tmp.mr_1272615772_3.count()12
script was running for 12 minutes
> db.tmp.mr_1272615772_3.find().sort({value:-1}).limit(1){_id: 8, value: 41231}
41k objects created 8 minutes after start
![Page 96: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/96.jpg)
references to this object?ary = [“a”,”b”,”c”]
ary references “a”“b” referenced by ary
• ruby makes it easy to “leak” references
• an object will stay around until all references to it are gone
• more objects = longer GC = bad performance
• must find references to fix leaks
![Page 97: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/97.jpg)
references to this object?• db.rails_refs.insert({
_id:"0xary", refs:["0xa","0xb","0xc"]})create references lookup table
![Page 98: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/98.jpg)
references to this object?• db.rails_refs.insert({
_id:"0xary", refs:["0xa","0xb","0xc"]})create references lookup table
• db.rails_refs.ensureIndex({refs:1})add ‘multikey’ index to refs array
![Page 99: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/99.jpg)
references to this object?• db.rails_refs.insert({
_id:"0xary", refs:["0xa","0xb","0xc"]})create references lookup table
• db.rails_refs.ensureIndex({refs:1})add ‘multikey’ index to refs array
• db.rails_refs.find({refs:“0xa”})efficiently lookup all objs holding a ref to 0xa
![Page 100: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/100.jpg)
mongodb: multikeys
• indexes on array values create a ‘multikey’ index
• classic example: nested array of tags
• find({tags: “ruby”})find objs where obj.tags includes “ruby”
![Page 101: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/101.jpg)
version 2: memprof.coma web-based heap visualizer and leak analyzer
![Page 102: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/102.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 103: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/103.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 104: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/104.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 105: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/105.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 106: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/106.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 107: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/107.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 108: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/108.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 109: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/109.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 110: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/110.jpg)
memprof.coma web-based heap visualizer and leak analyzer
![Page 111: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/111.jpg)
plugging a leak in rails3• in dev mode, rails3 is leaking 10mb per request
![Page 112: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/112.jpg)
plugging a leak in rails3• in dev mode, rails3 is leaking 10mb per request
# in environment.rbrequire `gem which memprof/signal`.strip
let’s use memprof to find it!
![Page 113: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/113.jpg)
plugging a leak in rails3
send the app some requests so it leaks
$ ab -c 1 -n 30 http://localhost:3000/
![Page 114: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/114.jpg)
plugging a leak in rails3
tell memprof to dump out the entire heap to json
$ memprof --pid <pid> --name <dump name> --key <api key>
send the app some requests so it leaks
$ ab -c 1 -n 30 http://localhost:3000/
![Page 115: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/115.jpg)
plugging a leak in rails3
tell memprof to dump out the entire heap to json
$ memprof --pid <pid> --name <dump name> --key <api key>
send the app some requests so it leaks
$ ab -c 1 -n 30 http://localhost:3000/
![Page 116: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/116.jpg)
![Page 117: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/117.jpg)
2519 classes
![Page 118: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/118.jpg)
2519 classes
30 copies of TestController
![Page 119: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/119.jpg)
2519 classes
30 copies of TestController
![Page 120: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/120.jpg)
2519 classes
30 copies of TestController
mongo query for all TestController classes
![Page 121: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/121.jpg)
2519 classes
30 copies of TestController
mongo query for all TestController classes
details for one copy of TestController
![Page 122: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/122.jpg)
![Page 123: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/123.jpg)
find references to object
![Page 124: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/124.jpg)
find references to object
![Page 125: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/125.jpg)
find references to object
holding references to all controllers
![Page 126: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/126.jpg)
find references to object
holding references to all controllers
“leak” is on line 178
![Page 127: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/127.jpg)
• In development mode, Rails reloads all your application code on every request
![Page 128: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/128.jpg)
• In development mode, Rails reloads all your application code on every request
• ActionView::Partials::PartialRenderer is caching partials used by each controller as an optimization
![Page 129: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/129.jpg)
• In development mode, Rails reloads all your application code on every request
• ActionView::Partials::PartialRenderer is caching partials used by each controller as an optimization
• But.. it ends up holding a reference to every single reloaded version of those controllers
![Page 130: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/130.jpg)
• In development mode, Rails reloads all your application code on every request
• ActionView::Partials::PartialRenderer is caching partials used by each controller as an optimization
• But.. it ends up holding a reference to every single reloaded version of those controllers
![Page 131: Debugging Ruby (Aman Gupta)](https://reader031.vdocument.in/reader031/viewer/2022020101/5403a71b8d7f72de4c8b485b/html5/thumbnails/131.jpg)
Questions?
Aman Gupta@tmm1