monitoring vmware vfabric with hyperic and spring insight
DESCRIPTION
Webinar presented by Steve Millidge, C2B2 Director, on the 19th of November 2012. This webinar demonstrates how you can significantly improve your support of production tcServer environments using Hyperic and Spring Insight. In the webinar C2B2's experienced support engineer will show you how to gather extensive internal metrics from your production tcServer instances, how to configure alerts to warn you of potential problems in your production tcServer cluster and how to diagnose and triage poorly performing transactions with Spring Insight. To watch high quality version of this webinar please visit https://www4.gotomeeting.com/register/941934207TRANSCRIPT
© C2B2 Consulting Limited 2012All Rights Reserved
Monitoring VMware vFabric with Hyperic and Spring Insight
Steve Millidge
© C2B2 Consulting Limited 2012All Rights Reserved
vFabric tcServer
© C2B2 Consulting Limited 2012All Rights Reserved
Why vFabric tcServer?
• Pre-configured Templates• Many instances from a single binary install• High Performance JDBC connection pool• Improved Diagnostics• JMX management enabled by default• Unix & Windows Service Scripts• Server Management Tools• Latest security patches as binaries• Integration with vFabric components
© C2B2 Consulting Limited 2012All Rights Reserved
Features over Tomcat• Templates• Single Binary Install• High Performance JDBC Connection Pool• Clustering• Diagnostics• Elastic Memory for Java• Management
© C2B2 Consulting Limited 2012All Rights Reserved
Diagnostics• Templates• Single Binary Install• High Performance JDBC Connection Pool• Clustering• Diagnostics• Elastic Memory for Java• Management
© C2B2 Consulting Limited 2012All Rights Reserved
Diagnostics Template
./tcruntime-instance.sh create <instance> -t diagnostics
© C2B2 Consulting Limited 2012All Rights Reserved
SlowQueryReportJmx
• Reports on SLOW and FAILED Queries• Subscribe for JMX Notifications
© C2B2 Consulting Limited 2012All Rights Reserved
ThreadDiagnosticsValve
$CATALINA_BASE/logs/catalina.YYYY-MM-DD.log
© C2B2 Consulting Limited 2012All Rights Reserved
+ Extended Logging
© C2B2 Consulting Limited 2012All Rights Reserved
JMX Notifications
• Reports on “slow”, “stuck” and “failed” requests• Subscribe for JMX notifications or view log files
© C2B2 Consulting Limited 2012All Rights Reserved
EM4J• Templates• Single Binary Install• High Performance JDBC Connection Pool• Clustering• Diagnostics• Elastic Memory for Java• Management
© C2B2 Consulting Limited 2012All Rights Reserved
Traditional Best Practice for Virtualised JVM’s
• Reserve 100% of the required VM memory• Tune JVM Heap to occupy minimum• Avoid GC Thrashing “Both ballooning and swapping should be
prevented for Java application”
• No benefit from Virtualisation memory sharing• All resources must be provisioned upfront
© C2B2 Consulting Limited 2012All Rights Reserved
ESXi 5.0 Memory Reclamation
• Transparent Page Sharing• Memory Compression• VMware Tools Balloon OR EM4J Balloon• Swap
© C2B2 Consulting Limited 2012All Rights Reserved
EM4J Balloon Driver
Guest OS
JVM Heap JVM Heap
Guest OS
JVM Heap JVM Heap
Hypervisor
...
© C2B2 Consulting Limited 2012All Rights Reserved
Elastic Memory for Java EM4J
• Requires ESXi 5.0• Redhat / Centos v5 (x32 or x64)• Hotspot JDK 1.6• Heap > 512Mb• Recommend no more than 4 JVMs per VM• JVMs should account for the majority of the
VM memory usage
© C2B2 Consulting Limited 2012All Rights Reserved
Switch EM4J On
• Enable EM4J in the VM– sched.mem.pshare.guestHintsSyncEnable=true– sched.mem.maxmemctl=-1 (max balloon size)
• Create EM4J instance– ./tcruntime-instance.sh create instanceName -t elastic-
memory
© C2B2 Consulting Limited 2012All Rights Reserved
Troubleshooting your App
• How many database calls does that page generate?• What was the SQL executed?• How long did the query take?• Why does this page take a long time to load?• Which part was slow?• What happened then?!
© C2B2 Consulting Limited 2012All Rights Reserved
Spring Insight to the Rescue
What does it provide?• Information on all database queries • Information on all web request and response information• Lists component calls and parameters
Where can I use it?• Agile Development• QA• Load Testing• Production Monitoring
© C2B2 Consulting Limited 2012All Rights Reserved
Under the Covers
How does it work?• Agent instruments application code using AspectJ• Reports to “dashboard” (insight.war) component
responsible for rendering statistics
Increased Memory Requirements• Increased memory –Xmx• Increased permgen –XX:MaxPermSize
© C2B2 Consulting Limited 2012All Rights Reserved
Applications Summary
© C2B2 Consulting Limited 2012All Rights Reserved
Application View
© C2B2 Consulting Limited 2012All Rights Reserved
End Point View
© C2B2 Consulting Limited 2012All Rights Reserved
Response Time Histogram
© C2B2 Consulting Limited 2012All Rights Reserved
Request & Response Summary
© C2B2 Consulting Limited 2012All Rights Reserved
JDBC Query
© C2B2 Consulting Limited 2012All Rights Reserved
Recent Activity
© C2B2 Consulting Limited 2012All Rights Reserved
vFabric Hyperic Integration
More later!
© C2B2 Consulting Limited 2012All Rights Reserved
vFabric Hyperic
© C2B2 Consulting Limited 2012All Rights Reserved
Overview
• Monitoring & Management Solution• Out-of-box monitors over 75 technologies• Automatic discovery of resources• Metric Collection• Event Tracking• Resource Control• Alerting & Notifications• Access to Live Information
© C2B2 Consulting Limited 2012All Rights Reserved
Architecture
© C2B2 Consulting Limited 2012All Rights Reserved
Resource Hierarchy
• Platform – Win2008, linux etc
• Platform Services– CPU– File System Mount– Network Interfaces
• Server– tcServer instance– RabbitMQ Server
• Service– tcServer Session Manager– RabbitMQ Queue
© C2B2 Consulting Limited 2012All Rights Reserved
Hyperic User Interface
© C2B2 Consulting Limited 2012All Rights Reserved
Resource View
© C2B2 Consulting Limited 2012All Rights Reserved
Resource Alerts
• Metric Alerting• Event Alerting
– Log file tracking– Config file tracking
© C2B2 Consulting Limited 2012All Rights Reserved
tcServer Application Management
© C2B2 Consulting Limited 2012All Rights Reserved
vSphere Monitoring
• Server side HQU Extension• Remote agent accesses vSphere vCenter
© C2B2 Consulting Limited 2012All Rights Reserved
vCenter Resource Hierarchy
Agent installed on VM• Platform
– Win2008, linux etc
• Platform Services– CPU– File System Mount– Network Interfaces
• Server– tcServer instance– RabbitMQ Server
• Service– tcServer Session Manager– RabbitMQ Queue
Agent monitoring vCenter• vCenter – Software• ESX Host – Platform• VM – PlatformFeatures• Auto disovery of ESX
Hosts and VM’s• Auto detection of new
ESX Hosts and VM’s• Auto detection of
architecture changes e.g. vMotion
© C2B2 Consulting Limited 2012All Rights Reserved
ESX Host & VM Metics
© C2B2 Consulting Limited 2012All Rights Reserved
Quest