cloud watch - subhankar sengupta

Cloud WatchFrom Automation Perspective

Subhankar Sengupta

My Intro

● Sysadmin since 2002, started my career in a typical startup, a web-hosting company and daily firefight

● Worked in JP Morgan, Sabre Inc etc.

● Started JBUG –Bangalore (Jboss User Group) in 2010

● AWS user since since 2011 and the romance continues

Start with basics

What is CloudWatch1) Repository of metrics

● For monitoring AWS services cloudwatch could be a starting point● AWS service such as Auto Scaling, rely on CloudWatch to execute● Data points has a timestamp● Raw data in and statistics out (We will revisit this)

2) Alarm● Trigger Autoscaling Policies● SNS Notification

Functional Diagram

Architecture (Image Courtesy Amazon)

Some vocabulary

● Namespace- Container for metrics

● Metrics- Basically Data Points like CPUUtilization, DiskReadBytes,

DiskWriteBytes, NetworkIn, NetworkOut Metrics

○ We could use https://www.npmjs.com/package/cloudwatch-agent to push custom metrics

● Statistics- Metric data aggregated over period of time (min,max,sum,avg

etc)

● Units- Measurement units (example seconds, bytes, percentage etc)

● Periods- Length of time, default is 60 sec

● Alarms

I guess picture is more powerful

Raw Data In and Statistics Out

Alarm

Example:

● Average CPU utilization is more than 70% for 7 minutes

● Here CPU utilization is the metric

● Average is statistics

● State is 70%

● Specific Time is 7 minutes

Cloudwatch can do AWS billing alarm (Say 20 USD for a personal account)

Cloudwatch Automation Options

● Mainly used for custom metrics● Push the metric to CloudWatch and it will store and graph● Could be done using

○ AWS APIs (Boto Python or Fog Ruby)○ RESTful API○ Tools (Ansible does it)○ AWS CLI (Easiest)

●

AWS CLI example

● app_data.json[ { "MetricName": "Tasks", "Value": 100, "Unit": "Count" }]● aws cloudwatch put-metric-data --namespace "SomeName" --metric-data

file://app_data.json● Any of the above approach could help to embed monitoring into App

During BootStrap

● Add it when you spin it● Use CloudFormation

Pros

● Natively available from Amazon● Single Console● API provides great potential for customization● Ingest API allows for custom metric integration● Allows for SNS notification and automated actions● Has pager duty integration

Limitations

● Low retention (two weeks)● Can’t play with dashboards● 1 Minute interval is good● May be some more I am not aware of :(

What about APM?

● Please don’t compare with Newrelic or Appdynamic● Get some stats● Method Execution Time● Method Execution Count● Exceptions● Error Count● Gather using AspectJ● Publish to Cloudwatch and Visualize● One of the largest travel tech uses this method but minus the cloudwatch

Add more meat in Cloudwatch- Libarto

● http://blog.librato.com/posts/make-cloudwatch-awsome

● Store data for one year● Better UI

Q and A

Drop an email:

subhankar.aws@gmail.com