![Page 1: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/1.jpg)
#monitoringlovewith Sensu
OSMC 2014, Nuremberg, Germany
![Page 3: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/3.jpg)
Why Sensu?• Written in clean Ruby
• Scalable architecture
• Plugins in any language
• Can use Nagios checks
• Collects both checks and metrics
• Great community
![Page 4: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/4.jpg)
Jochen Lillichhttp://freistil.it
@geewiz
![Page 5: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/5.jpg)
Sensu Core
![Page 6: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/6.jpg)
Sensu Enterprise
![Page 7: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/7.jpg)
Installation• Omnibus packaging
• Configuration in JSON files
• Sensu cookbook for Chef
• Puppet module
![Page 8: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/8.jpg)
![Page 9: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/9.jpg)
![Page 10: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/10.jpg)
• Connects all Sensu components
• Asynchronous communication
![Page 11: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/11.jpg)
Sensu Server• Orchestrates check execution
• Processes check results
• Triggers event handlers
![Page 12: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/12.jpg)
Sensu Client• Registers automatically with the Server
• Sends keepalive information
• Receives check execution requests
• Schedules checks locally
• Executes checks
• Publishes check results
• Publishes external events
![Page 13: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/13.jpg)
API• get event data
• get agent data
• trigger check execution
• resolve events
• silence checks
![Page 14: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/14.jpg)
Dashboard• Uchiwa • sensu-admin
![Page 15: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/15.jpg)
![Page 16: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/16.jpg)
Scheduling• Standard checks (server)
• Standalone checks (client)
• Manual checks (API)
![Page 17: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/17.jpg)
{ "checks": { "disk_free": { "type": "status", "subscribers": [ "all" ], "handlers": [ "default" ], "command": "/usr/lib/nagios/plugins/check_disk -w :::disk_warn::: -c :::disk_crit::: -A -x /dev/shm -X nfs -i /boot", "interval": 60 } }}
![Page 18: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/18.jpg)
Checks in Chefsensu_check 'mysql_server' do command "/usr/lib/nagios/plugins/check_mysql " + "-u 'monitoring' " + "-p '#{node['mysql']['server_mon_password']}'" handlers ['default'] standalone true interval 30end
![Page 19: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/19.jpg)
Metrics check{ "checks": { "load_metrics": { "type": "metric", "command": "load-metrics.rb", "subscribers": [ "production" ], "interval": 10 } }}
![Page 20: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/20.jpg)
Metrics output$ ruby load-metrics.rbsrv3.local.load_avg.one 0.89 1365270842srv3.local.load_avg.five 1.01 1365270842srv3.local.load_avg.fifteen 1.06 1365270842$ echo $?0
![Page 21: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/21.jpg)
External eventsecho '{ "name": "my_check", "output": "some output", "status": 0 }' > /dev/tcp/localhost/3030
Useful: https://github.com/solarkennedy/sensu-shell-helper
![Page 22: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/22.jpg)
![Page 23: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/23.jpg)
Handler types• Pipe • TCP • UDP • Transport
• Sets
![Page 24: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/24.jpg)
Common event handlers• Email• PagerDuty• Graphite• IRC• Slack
![Page 25: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/25.jpg)
![Page 26: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/26.jpg)
Example handler code#!/usr/bin/env ruby
require 'rubygems'require 'json'
# Read event dataevent = JSON.parse(STDIN.read, :symbolize_names => true)# Write the event data to a filefile_name = "/tmp/sensu_#{event[:client][:name]}_" + "#{event[:check][:name]}"File.open(file_name, 'w') do |file| file.write(JSON.pretty_generate(event))end
![Page 27: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/27.jpg)
Example handler configuration{ "handlers": { "file": { "type": "pipe", "command": "/etc/sensu/handlers/file.rb" } }}
![Page 28: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/28.jpg)
Sensu CLIhttps://github.com/agent462/sensu-cli
• sensu-cli resolve srv3 apache_http
• sensu-cli client delete srv3
• sensu-cli silence srv3 --reason "Shut up already"
--expire 3600
![Page 29: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/29.jpg)
#chatopshttps://github.com/sensu/sensu-hubot
• sensu events summarize
• sensu events filter severity critical
• sensu events filter subscription webservers
![Page 30: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/30.jpg)
Monitoring your monitoring• Check RabbitMQ ready queue!
![Page 31: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/31.jpg)
Scaling Sensu
![Page 32: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/32.jpg)
Scaling a single site• Sensu Server • Sensu API • RabbitMQ • Redis
![Page 33: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/33.jpg)
Multi-DC operation
![Page 34: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/34.jpg)
![Page 35: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/35.jpg)
![Page 36: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/36.jpg)
![Page 38: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/38.jpg)
References
![Page 39: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/39.jpg)
Community pluginshttps://github.com/sensu/sensu-community-plugins
• Over 600 plugins
• 80 contributors
![Page 40: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/40.jpg)
Support• #sensu on FreeNode IRC
• sensu-users mailing list
• Commercial support from HeavyWater
![Page 42: OSMC 2014: MonitoringLove with Sensu | Jochen Lillich](https://reader034.vdocument.in/reader034/viewer/2022052601/559452411a28abd94f8b4680/html5/thumbnails/42.jpg)
Credits• Samuel Beckett Bridge by Miguel Mendez https://flic.kr/p/
dyn2FU