![Page 1: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/1.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.
Justin Bruns , Performance Test Lead, Fireman's Fund Insurance
Robert May , Enterprise Software Tools Administrator
Fireman's Fund Insurance
![Page 2: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/2.jpg)
© Copyright 2012 Hewlett-Packard Development Company, L.P.
The information contained herein is subject to change without notice.
HP Tools for Software Performance Testing and Analysis
Justin Bruns, Performance Test Lead - Fireman’s Fund Insurance Company
June 2012
![Page 3: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/3.jpg)
3
Agenda
• Fireman’s Fund Overview
• Problem Statement
• Solution (Sitescope, Performance Center, Diagnostics)
• Performance Analysis Methodology
• Test Execution and Monitoring, Analysis File Review, Exploratory Testing
• Additional Tools for Analysis
• Q & A
![Page 4: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/4.jpg)
4
Fireman’s Fund Overview • Company profile: Fireman's Fund Insurance Company is a premier property and
casualty insurer, providing personal and commercial insurance products nationwide. The company is differentiated from its competition through its product and service excellence, and exceptional claims handling.
• Based in: Novato, California Employees: 3,900
• Company history: Founded in San Francisco in 1863, Fireman's Fund was named for its founding social mission: contributing a portion of its profits to widows and orphans of firefighters. Today the company continues its philanthropic mission, supporting firefighters for safer communities through its Fireman's Fund Heritage SM program.
• Parent company: Allianz SE of Munich, Germany is one of the world's largest providers of insurance and other financial services with more than $1 trillion in assets. It is rated A+ by A.M. Best.
![Page 5: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/5.jpg)
5
Problem Statement • In addition to performance test response times, customers want answers. • Performance bottlenecks sometimes difficult to isolate and correct • Challenges Encountered
– Multiple causes of performance slowdowns – Insufficient communication between performance testing and development teams – Disparate monitoring tools and techniques
![Page 6: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/6.jpg)
6
Solution • Establish integrated and comprehensive monitors
– HP Sitescope for Load Testing 11.10 – HP ALM – Performance Center 11.00 – HP Diagnostics Server 9.12 – HP LoadRunner Analysis Tool 11.03 – HP Virtual User Generator 11.03
• Gather and analyze results
– Performance Analysis Methodology – Inspect information in real-time or after completing tests
• Make recommendations and necessary changes – revalidate results
![Page 7: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/7.jpg)
Application Architecture
![Page 8: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/8.jpg)
8
Sitescope Monitors
![Page 9: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/9.jpg)
9
Sitescope Monitors • Wintel performance counters (CPU, memory, disk, network) • AIX performance counters (CPU, memory, disk, network)
– AIX process level details (CPU, memory size, threads)
• WebSphere MQ counters – Queue details (current depth, open input count, open output count) – Channel details (status, bytes sent/received)
• IIS counters – High level (bytes/min, hits/min) – Web service counters (current connections, method requests/sec) – .NET CLR (SQL client connection Data, garbage collection statistics)
• SQL server counters – Average buffer chain length, % physical I/O used by process – CPU, memory, disk
• Custom solution templates – JBoss, Oracle, Linux, WebLogic
![Page 10: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/10.jpg)
10
Sitescope - Performance Center • Link sitescope monitors in Performance Center • Performance Center test resources section
– Add a Sitescope monitor profile
• Enter the appropriate Sitescope information – Account details not the same as login details
![Page 11: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/11.jpg)
11
Sitescope - Performance Center
• Enter Server and Port Details • Click on Get Counters, Monitors Should Appear • Select the Appropriate Sitescope Monitors • Repeat this Process for all Desired Sitescope Monitors
![Page 12: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/12.jpg)
12
Sitescope - Performance Center
• Add the monitors in Performance Center • Performance test designer window • Select desired monitor(s) from profile tree
![Page 13: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/13.jpg)
13
Performance Center Monitors
• Monitors may be added directly in Performance Center
![Page 14: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/14.jpg)
14
Performance Center Monitors
• Multiple monitor types – Windows Resources, UNIX Resources, SNMP, IIS, SQL Server, Oracle
![Page 15: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/15.jpg)
15
Diagnostics Monitors
• Types of Monitors – .NET Probe – J2EE Probe – SQL Server Collector – Oracle Collector – WebSphere MQ Collector
• Steps Involved – Install Probe or Collector Software – Configure Probe or Collector – Instrument Probe
![Page 16: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/16.jpg)
16
Diagnostics Monitors • .NET Probe
– Standard View (CPU, VM Heap Utilization, Latency, Throughput)
![Page 17: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/17.jpg)
17
Diagnostics Monitors • .NET Probe
– View Server Request Details
![Page 18: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/18.jpg)
18
Diagnostics Monitors • .NET Probe
– View Server Request Details
![Page 19: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/19.jpg)
19
Diagnostics Monitors • WebSphere MQ collector
– Queue information (depth, messages enqueued/dequeued per second)
![Page 20: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/20.jpg)
20
Diagnostics Monitors • SQL Server and Oracle collectors
– View SQL statements, latency, throughput
![Page 21: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/21.jpg)
21
Diagnostics - Performance Center
• In Performance Center’s test designer view – Diagnostics tab • Diagnostics can be enabled for desired % of Vusers • Configure J2EE/.NET application diagnostics
![Page 22: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/22.jpg)
22
Diagnostics - Performance Center
• Select the Desired Probes or Collectors
![Page 23: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/23.jpg)
23
Analysis Methodology
• Start at the web layer • Look deeply into front end application • Examine the database layer • Know the integration layer • Determine the type of problem
![Page 24: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/24.jpg)
24
Test Execution and Monitoring - Analysis File Review -
• Execute the monitored load test • Upon completion, collate and analyze the results • Using the LoadRunner analysis tool, we can
– View transaction response time results – View other Performance Center measurements (hits per second, throughput, etc.) – View sitescope monitoring results – View J2EE/.NET diagnostic information – Overlay graphs and correlate information – Identify trends and hotspots
![Page 25: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/25.jpg)
25
Test Execution and Monitoring - Analysis File Review -
• View transaction response times to identify potential problems
![Page 26: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/26.jpg)
26
Test Execution and Monitoring - Analysis File Review -
• Show J2EE/.NET chain of calls
![Page 27: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/27.jpg)
27
Test Execution and Monitoring - Analysis File Review -
• Overlay graphs to correlate information
![Page 28: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/28.jpg)
28
Test Execution and Monitoring - Analysis File Review -
• Overlay graphs to correlate information
![Page 29: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/29.jpg)
29
Test Execution and Monitoring - Exploratory Testing -
• Profile single user transactions
![Page 30: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/30.jpg)
30
Additional Tools for Analysis
• HttpWatch / Fiddler / Firebug – Inspect browser HTTP traffic
• Database Monitoring Reports – SQL Server Profiler – Oracle AWR Report
• JVM Inspection – Garbage collection logs – Thread dumps – Heap dump analyzer
![Page 31: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/31.jpg)
31
Q & A
![Page 32: Justin Bruns , Performance Test Lead, Fireman's Fund ... · • Problem Statement • Solution (Sitescope, Performance Center, Diagnostics) ... • JVM Inspection – Garbage collection](https://reader035.vdocument.in/reader035/viewer/2022062219/5f020b8e7e708231d4024db9/html5/thumbnails/32.jpg)
32
Contact Details – Justin Bruns, Performance Test Lead Fireman’s Fund Insurance Company
• Work email – [email protected]
• Home email – [email protected]