benchmarking otm and java - is your platform limiting performance
TRANSCRIPT
Benchmarking OTM and Java:Is Your Platform Limiting
Performance?
OTM SIG User Conference ‘09
AgendaSetting ExpectationsOTM – Hardware and PlatformsTools – System and MonitoringBenchmarksHands-on Benchmarking (Real-time)Interpreting and Applying to OTMQ & A
This Presentation WILL:Teach you to benchmark various OTM platformsShow you how to interpret the benchmark resultsShow real-world platform results
This Presentation will NOT:Teach you to load-test an OTM installationShow you how to predict OTM SCA (scalability / clustering) performanceUtterly fail (I hope!)
Hardware / PlatformsCPU and Hardware Platform Matter! A LOT!CPU Speed – Not a Good Indicator of Performance
Other factors (cores, memory bandwidth, on-chip cache) necessitate benchmarking
OTM Requires both high multi-threading AND high single-thread performance
Lots of cores and high per-core performance
Performance of Current PlatformsLinux / x86-64Windows / x86-64 (note: memory limitations)SolarisHP-UX / PA-RISC
Note: HP-UX / Itanium currently unknown
AIX / POWER
Operating System / Stats – Helpful but LimitedReview system performance under production load for the previous 2 weeksUtilize System Tools to Monitor
sar / kSartop / prstat / topas / etc
Utilize Tools to TrendNagios / Munin / etc
Benchmarking – Which Benchmarks?
VolanoMarkDaCapoSoap StoneHammerora
Why?Java basedReplicate OTM behaviorEasily repeatable
Why Not Load-Test OTM?
Complicated to setupTime-consuming to runRequires OTM installed AND configuredNot Necessary to determine platform capacity
Benchmarking - VolanoMarkRunning VolanoMark (Hands-On Exercise)Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/6-benchmarking-part-1-java-jvm-performance-volanomark.html
Benchmarking - VolanoMarkVolanoMark
Java-based benchmark that simulates high transactional and multi-threaded loadReflects the performance of the following OTM activities
Web UI, Agents, Integration, General Workflow, General OTM Activities (not including optimization and planning based)
Higher numbers are better
Benchmarking - DaCapoRunning DaCapo (Hands-On Exercise)Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/12-benchmarking-part-3-java-jvm-performance-dacapo.html
Benchmarking - DaCapoDaCapo
Java-based benchmark that simulates highly computational, algorithmic, single-threaded processingReflects the performance of the following OTM activities
Optimization and Planning / Bulk Planning
Lower numbers are better
Benchmarking – Soap StoneRunning Soap Stone (Hands-On Exercise)Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/13-benchmarking-part-4-real-world-network-performance-soap-stone.html
Benchmarking – Soap StoneSoap Stone
Java-based benchmark that tests data throughput between servers and replicates application protocols, such as HTTP, RMI and RAW.Reflects the throughput and protocols utilized between the various OTM Tiers
Browser / Web: HTTPWeb / App: RMIApp / DB: RAW
Higher numbers are better
Benchmarking – HammeroraRunning Hammerora (Hands-On Exercise)Full instructions at:
http://www.otmfaq.com/forums/blogs/chrisplough/11-benchmarking-part-2-oracle-db-performance-hammerora.html
Benchmarking – Hammerora
Just Kidding!!!There’s no way we’d have enough time
Benchmarking – HammeroraHammerora
Benchmark based on the TPC-C and TPC-H benchmarks.Reflects the performance and scalability of the DB TierLower numbers are better
Interpreting the ResultsThese benchmarks have linear trends (twice as fast is twice as fast)VolanoMark
Higher is betterApplies to Web and App tiers
DaCapoLower is betterApplies to App tier
Soap StoneHigher is better40Mbit or faster Web App400Mbit or faster App DB
HammeroraHigher is betterApplies to all tiers
Online ResourcesPerformance
kSarhttp://ksar.atomique.net/
Nagioshttp://www.nagios.org/
Muninhttp://munin.projects.linpro.no/
BenchmarkingFull Replication Details
http://www.otmfaq.com/forums/blogs/chrisplough/
VolanoMarkhttp://www.volano.com/benchmarks.html
DaCapohttp://dacapobench.org/
Soap Stonehttp://soap-stone.sourceforge.net/
Hammerorahttp://hammerora.sourceforge.net/
Q & A and Discussion
Questions?
