rabbitmq operations - github...• rabbitmq’s disk monitor isn’t supported on all platforms. ram...
TRANSCRIPT
![Page 1: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/1.jpg)
RabbitMQ Operations
![Page 2: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/2.jpg)
About me
![Page 3: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/3.jpg)
About me• RabbitMQ staff engineer at Pivotal
![Page 4: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/4.jpg)
About me• RabbitMQ staff engineer at Pivotal
• @michaelklishin just about everywhere
![Page 5: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/5.jpg)
![Page 6: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/6.jpg)
About this talk
![Page 7: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/7.jpg)
About this talk• Brain dump from years of answering questions
![Page 8: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/8.jpg)
About this talk• Brain dump from years of answering questions
• Focusses on the most recent release (3.5.6)
![Page 9: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/9.jpg)
![Page 10: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/10.jpg)
Provisioning
![Page 11: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/11.jpg)
Provisioning• Be aware of mirrors: GitHub, Bintray, …
![Page 12: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/12.jpg)
Provisioning• Be aware of mirrors: GitHub, Bintray, …
• Looking into community-hosted mirrors
![Page 13: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/13.jpg)
Provisioning• Be aware of mirrors: GitHub, Bintray, …
• Looking into community-hosted mirrors
• Use packages + Chef/Puppet/…
![Page 14: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/14.jpg)
![Page 15: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/15.jpg)
OS resources
![Page 16: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/16.jpg)
OS resources• Modern Linux defaults are absolutely inadequate
for servers
![Page 17: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/17.jpg)
ulimit -n default: 1024
![Page 18: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/18.jpg)
![Page 19: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/19.jpg)
Set ulimit -n and fs.file-max to 500K and forget about it
![Page 20: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/20.jpg)
TCP keepalive timeout: from 11 minutes to over 2
hours by default
![Page 21: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/21.jpg)
net.ipv4.tcp_keepalive_time = 6 net.ipv4.tcp_keepalive_intvl = 3 net.ipv4.tcp_keepalive_probes = 3
![Page 22: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/22.jpg)
enable client heartbeats, e.g. with an interval of 6-12 seconds
![Page 23: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/23.jpg)
OS resources• Modern Linux defaults are absolutely inadequate
for servers
• Tuning for throughput vs. high number of concurrent connections
![Page 24: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/24.jpg)
Throughput: larger TCP buffers
![Page 25: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/25.jpg)
net.core.rmem_max = 16777216 net.core.wmem_max = 16777216
![Page 26: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/26.jpg)
rabbit.hipe_compile = true (only on Erlang 17.x or 18.x)
![Page 27: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/27.jpg)
Concurrent connections: smaller TCP buffers, low tcp_fin_timeout,
tcp_tw_reuse = 1, …
![Page 28: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/28.jpg)
rabbit.tcp_listen_options.sndbuf rabbit.tcp_listen_options.recbuf rabbit.tcp_listen_options.backlog
![Page 29: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/29.jpg)
Reduce per connection RAM use by 10x
rabbit.tcp_listen_options.sndbuf = 16384 rabbit.tcp_listen_options.recbuf = 16384
![Page 30: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/30.jpg)
![Page 31: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/31.jpg)
Reduce per connection RAM use by 10x
Throughput drops by a comparable amount
![Page 32: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/32.jpg)
net.ipv4.tcp_fin_timeout = 5
![Page 33: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/33.jpg)
net.ipv4.tcp_tw_reuse = 1
![Page 34: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/34.jpg)
Careful with tcp_tw_reuse behind NAT*
* http://vincent.bernat.im/en/blog/2014-tcp-time-wait-state-linux.html
![Page 35: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/35.jpg)
net.core.somaxconn = 4096
![Page 37: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/37.jpg)
![Page 38: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/38.jpg)
Disk space
![Page 39: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/39.jpg)
Disk space• Pay attention to what partition /var/lib ends up on
![Page 40: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/40.jpg)
Disk space• Pay attention to what partition /var/lib ends up on
• Transient messages can be paged to disk
![Page 41: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/41.jpg)
Disk space• Pay attention to what partition /var/lib ends up on
• Transient messages can be paged to disk
• RabbitMQ’s disk monitor isn’t supported on all platforms
![Page 42: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/42.jpg)
![Page 43: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/43.jpg)
RAM usage
![Page 44: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/44.jpg)
RAM usage• rabbit.vm_memory_high_watermark
![Page 45: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/45.jpg)
RAM usage• rabbit.vm_memory_high_watermark
• rabbit.vm_memory_high_watermark_paging_ratio
![Page 46: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/46.jpg)
rabbitmqctl status rabbitmqctl report
![Page 47: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/47.jpg)
RAM usage• rabbit.vm_memory_high_watermark
• rabbit.vm_memory_high_watermark_paging_ratio
• Significant paging efficiency improvements in 3.5.5-3.5.6
![Page 48: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/48.jpg)
RAM usage• rabbit.vm_memory_high_watermark
• rabbit.vm_memory_high_watermark_paging_ratio
• Significant paging efficiency improvements in 3.5.5-3.5.6
• Disable rabbit.fhc_read_buffering (3.5.6+)
![Page 49: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/49.jpg)
rabbitmqctl eval ‘file_handle_cache:clear_read_cache().’
![Page 50: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/50.jpg)
recon
![Page 51: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/51.jpg)
Ability to set VM RAM watermark as absolute value is coming in 3.6
![Page 52: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/52.jpg)
![Page 53: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/53.jpg)
Stats collector falls behind
![Page 54: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/54.jpg)
Stats collector falls behind• Management DB stats collector can get
overwhelmed
![Page 55: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/55.jpg)
Stats collector falls behind• Management DB stats collector can get
overwhelmed
• Key symptom: disproportionally higher RAM use on the node that hosts management DB
![Page 56: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/56.jpg)
rabbitmqctl eval 'P = whereis(rabbit_mgmt_db), erlang:process_info(P).'
![Page 57: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/57.jpg)
[{registered_name,rabbit_mgmt_db}, {current_function,{erlang,hibernate,3}}, {initial_call,{proc_lib,init_p,5}}, {status,waiting}, {message_queue_len,0}, {messages,[]}, {links,[<5477.358.0>]}, {dictionary,[{'$ancestors',[<5477.358.0>,rabbit_mgmt_sup,rabbit_mgmt_sup_sup, <5477.338.0>]}, {'$initial_call',{gen,init_it,7}}]}, {trap_exit,false}, {error_handler,error_handler}, {priority,high}, {group_leader,<5477.337.0>}, {total_heap_size,167}, {heap_size,167}, {stack_size,0}, {reductions,318}, {garbage_collection,[{min_bin_vheap_size,46422}, {min_heap_size,233}, {fullsweep_after,65535}, {minor_gcs,0}]}, {suspending,[]}]
![Page 58: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/58.jpg)
rabbit.collect_statistics_interval = 30000
![Page 59: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/59.jpg)
rabbitmq_management.rates_mode = none
![Page 60: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/60.jpg)
rabbitmqctl eval 'P = whereis(rabbit_mgmt_db), erlang:exit(P, please_crash).'
![Page 61: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/61.jpg)
Parallel stats collector is coming in 3.7
![Page 62: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/62.jpg)
![Page 63: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/63.jpg)
Cluster formation
![Page 64: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/64.jpg)
Cluster formation• Node restart order dependency
![Page 65: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/65.jpg)
Cluster formation• Node restart order dependency
• github.com/rabbitmq/rabbitmq-clusterer
![Page 66: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/66.jpg)
Cluster formation• Node restart order dependency
• github.com/rabbitmq/rabbitmq-clusterer
• github.com/aweber/rabbitmq-autocluster
![Page 67: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/67.jpg)
![Page 68: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/68.jpg)
Backups
![Page 69: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/69.jpg)
How do I back up?• cp $RABBITMQ_MNESIA_DIR + tar
![Page 70: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/70.jpg)
How do I back up?• cp $RABBITMQ_MNESIA_DIR + tar
• Replicate everything off-site with exchange federation + set message TTL via a policy
![Page 71: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/71.jpg)
![Page 72: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/72.jpg)
Hostname changes
![Page 73: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/73.jpg)
rabbitmqctl rename_cluster_node [old name] [new name]
![Page 74: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/74.jpg)
![Page 75: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/75.jpg)
Network partition handling
![Page 76: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/76.jpg)
Network partition handling• When in doubt, use “autoheal”
![Page 77: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/77.jpg)
Network partition handling• When in doubt, use “autoheal”
• “Merge” is coming but has very real downsides, too
![Page 78: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/78.jpg)
![Page 79: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/79.jpg)
Misc
![Page 80: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/80.jpg)
Misc• Don’t use default vhost and/or credentials
![Page 81: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/81.jpg)
Misc• Don’t use default vhost and/or credentials
• Don’t use 32-bit Erlang
![Page 82: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/82.jpg)
Misc• Don’t use default vhost and/or credentials
• Don’t use 32-bit Erlang
• Use reasonably up-to-date releases
![Page 83: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/83.jpg)
Misc• Don’t use default vhost and/or credentials
• Don’t use 32-bit Erlang
• Use reasonably up-to-date releases
• Participate in rabbitmq-users
![Page 84: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/84.jpg)
Misc• OCF resource template from Fuel (by Mirantis)
![Page 85: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/85.jpg)
Misc• OCF resource template from Fuel (by Mirantis)
• Use TLS
![Page 86: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/86.jpg)
![Page 87: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/87.jpg)
Coming in 3.6
![Page 88: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/88.jpg)
Coming in 3.6• In process file buffering disabled by default
![Page 89: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/89.jpg)
Coming in 3.6• In process file buffering disabled by default
• Queue master to node distribution strategies
![Page 90: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/90.jpg)
Coming in 3.6• In process file buffering disabled by default
• Queue master to node distribution strategies
• SHA-256 (or 512) for password hashing
![Page 91: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/91.jpg)
Coming in 3.6• In process file buffering disabled by default
• Queue master to node distribution strategies
• SHA-256 (or 512) for password hashing
• More responsive management UI with pagination
![Page 92: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/92.jpg)
Coming in 3.6• In process file buffering disabled by default
• Queue master to node distribution strategies
• SHA-256 (or 512) for password hashing
• More responsive management UI with pagination
• Streaming rabbitmqctl
![Page 93: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/93.jpg)
Coming past 3.6
![Page 94: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/94.jpg)
Coming past 3.6• Pluggable cluster formation (à la ElasticSearch)
![Page 95: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/95.jpg)
Coming past 3.6• Pluggable cluster formation (à la ElasticSearch)
• On disk data recovery tools
![Page 96: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/96.jpg)
Coming past 3.6• Pluggable cluster formation (à la ElasticSearch)
• On disk data recovery tools
• Better CLI tools
![Page 97: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/97.jpg)
Coming past 3.6• Pluggable cluster formation (à la ElasticSearch)
• On disk data recovery tools
• Better CLI tools
• Easier off-site replication
![Page 98: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/98.jpg)
Coming past 3.6• Pluggable cluster formation (à la ElasticSearch)
• On disk data recovery tools
• Better CLI tools
• Easier off-site replication
• “Merge” partition handling strategy (no earlier than 3.8)
![Page 99: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/99.jpg)
![Page 100: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/100.jpg)
Thank you
![Page 101: RabbitMQ Operations - GitHub...• RabbitMQ’s disk monitor isn’t supported on all platforms. RAM usage. RAM usage ... RabbitMQ Operations Created Date: 10/27/2015 5:08:59 AM](https://reader036.vdocument.in/reader036/viewer/2022062414/5ec9d92f233be5791b22743d/html5/thumbnails/101.jpg)
Thank you• @michaelklishin
• github.com/michaelklishin
• rabbitmq-users
• Our team is hiring!