Grid Toolkits

Metrics that talk on Cloud using Ganglia

How do we monitor performance on Megam Cloud

We'll Cover

Our Experience Using Ganglia

How does it work in our Platform

Chef cookbooks for metering and setup.

Dashboard integration

What is Ganglia

Scalable distributed monitoring system for high-performance computing systems.

Sends information about your cloud instance.

Can be used as a live cloud monitor.

Can be extended using python plugins

What have we accomplished ?

Oh Yeah -

Dash #2(AngularJs Client)Metrics API

gmon

gmetad

gmetad

gmetad

gmon

gmon

gmon

Would you like to setup one ?

Yes you can

(or) http://www.megam.co

Few facts on gmetad/gmond

gmetad can run standalone or along with gmond.

gmetad can be configured to collect metrics of gmond servers of same cluster or different cluster.

gmetad stores data at

/var/lib/ganglia/rrds/CLUSTER_NAME/GMOND_SERVER_NAME

GMOND_SERVER_NAME can be changed in gmond.conf

gmetad is the metrics collector

gmond is the metrics sender

Ok. Got it.

What is our setup

#1 gmetad : monitor1.megam.co#2 gmetad : montior2.megam.co

Several gmonds(Cloud Apps) pumping data to gmetad

What are Cloud Apps

Any app

for lang := range ProgLanguages {JavaScalaGo . meteor}

&& DB, Queue

Do you need Graphite

No

Why ?

Needs rrds formatted metric files

It copies rrds files from gmetad

Twice storage

Is this the only soln ? Eager to hear feedback.

Let us setup gmetad 3.3.8-1

Ubuntu(raring) : package is gmetad

Ubuntu(saucy) has 3.6.0

sudo apt-get install gmetad

sudo apt-get install ganglia-webfrontend (*optional)

We used Opscode cookbook to setup => Link

Configure gmetad

The above says megcluster collects metrics from .megam.co and .megam.coWhich is like monitoring a Java App in .megam.co

(or) Your favorite App in .megam.co

nano /etc/ganglia/gmetad.confdata_source megcluster .megam.co:8649 .megam.co:8649

gmetad - start/stop.

Start :sudo gmetad

Stop : good old killps -ef | grep gmetadsudo kill -9

Cool gmetad - monitor1.megam.co is running

gmond

Install ganglia-monitor-python package will be installed in a server which is to be monitored.

Package has methods to collect basic metrics(cpu...) using python scripts in /usr/lib/ganglia/

Extended by enabling additional python scripts at /usr/lib/ganglia/python_modules.

For an exhaustive list : https://github.com/ganglia/gmond_python_modules

Let us setup gmond 3.3.8-1

Ubuntu(raring) : package is ganglia-monitor-python

Ubuntu(saucy) has 3.6.0

sudo apt-get install ganglia-monitor-python

We used Opscode cookbook to setup => Link

Configure gmond

The above says megcluster collects metrics from .megam.co and .megam.coWhich is like monitoring a Java App in .megam.co

(or) Your favorite App in .megam.co

nano /etc/ganglia/gmond.conf

globals { override_hostname = .megam.co override_ip = 127.0.0.1}udp_send_channel { host = monitor1.megam.co port = 8649 ttl = 1 }cluster { name = "megcluster" owner = "unspecified" }

What did we configure ?

In the globals we say our monitoring Java App's host name .megam.co

We provide the UDP channel of the gmetad (monitor1.megam.co)

We need to specify the gmetad cluster (megcluster)

cluster attribute groups all gmond to a gmetd in our case megcluster.

gmond - start/stop.

Start :sudo gmond

Stop : good old killps -ef | grep gmondsudo kill -9

Cool gmond - .megam.co is running

&

pumping to monitor1.megam.co

We customized chef - cookbooks

How do we use the chef - cookbooks

Tweak cookbook for gmetad

https://github.com/indykish/chef-repo/tree/master/cookbooks/megam_ganglia

Attributes

default[:ganglia][:cluster_name] = "megcluster"default[:ganglia][:unicast] = truedefault[:ganglia][:hostname] = monitoring1.megam.co

Chef Run : gmetad

Run chef :

runlist 'recipe[megam_ganglia::gmetad]'

Tweak recipes for gmond

For Any AppDefault : installs and configures ganglia-monitor-python. It collects the basic meterings like cpu_usage, memory_usage etc.

Nginx : collects nginx status details.

Rabbit : collects rabbbitmq metrics.

Redis : collects redis metrics.

Riak : collects riak metrics.

Chef Run : gmond

To monitor an appinclude_recipe megam_ganglia

Nginx frontended apps include_recipe megam_ganglia::nginx

Rabbitmq apps include_recipe megam_ganglia::rabbit

Riak apps include_recipe megam_ganglia::riak

Redis apps include_recipe megam_ganglia::redis

Configure gmond

nano /etc/ganglia/gmond.conf

globals { daemonize = yes setuid = yes user = nobody debug_level = 0 max_udp_msg_len = 1472 mute = no deaf = no host_dmax = 86400 /*secs */ cleanup_threshold = 300 /*secs */ gexec = no send_metadata_interval = 30 override_hostname = ".megam.co" override_ip = 127.0.0.1} cluster {

name = "megcluster"

owner = "unspecified"

latlong = "unspecified"

url = "unspecified"

}

/* The host section describes attributes of the host, like the location */

host {

location = "unspecified"

}

/* Feel free to specify as many udp_send_channels as you like. Gmond

used to only support having a single channel */

udp_send_channel {

host = monitoring1.megam.co

port = 8649

ttl = 1

}

Verifying gmond data

Open up your browser http://gmond1.megam.co:8649

It will list the metrics of the gmond instance.

Not recommended for prod.

Sample gmond data (for redis server)

Checking gmeta data

Data files in the below dir increasesGet metrics data at /var/lib/ganglia/rrds/megamcluster/gmond1.megam.co

Sample gmetad data (for thomas.work.local)

Dash Integration in rails

Built on

We'll cover it detail in a separate slideshare.If you are hungry Code is the design :)

For questions on this area:rajthilak@megam.co.in

References

Ganglia Wiki

megam chef-repo

Our Organization(Megam Systems)

Beta Launch of Megam Cloud (Polygot PaaS)

Our PaaS design => Link

Register http://www.megam.co for an invite

Twitter : @indykish

Screencast illustrating the Cloud API Servers working live

Thank you

for watching

2012-2013 Megam Systems

2012-2013 Megam Systems