Measuring Performance: the basicsby Stack Evolution
2 © Stack Evolution Ltd
Why?I just finished building my IaaS cloud.
But…
Is it any good?
3 © Stack Evolution Ltd
Lots of metrics and measuresSystem monitoringSystem metric graphingLog file analysis
These are focused on the infrastructure, not the tenant workload
4 © Stack Evolution Ltd
IaaS is a Shared PlatformTunable memory/processor/disk overcommitDynamic workloadsBatch jobsTime of day/month/yearMany variables
5 © Stack Evolution Ltd
Measuring the tenant workloadsTenant only cares about workload
performanceHow does the disk perform?Is the processor faster or slower?What is performance of a specific Database?How do HPC jobs perform?
6 © Stack Evolution Ltd
BenchmarksMany standard benchmark tools available:IperfUnixbenchFioSysbench_OLTPAerospikeRedisScimark2HPCC
7 © Stack Evolution Ltd
Standardise Suite of BenchmarksSelect flavours to directly compare memoryVolumes of standard size (20GB)Network: Internal and ExternalSpecific applications
DatabaseLoggingStreamingHPC
8 © Stack Evolution Ltd
Run on multiple platformsCollect data from
Public CloudsPrivate cloud deploymentsStaging/Test Clouds
9 © Stack Evolution Ltd
CompareNow compare the data to evaluate how good your cloud really is.
Demo
11 © Stack Evolution Ltd
Variability of performance for similar flavours across cloud providers, notice the latency and boot time graphs higher is worse, others higher is better
12 © Stack Evolution Ltd
Overall Index score for combined benchmarks, notice the variability in providers is as much as 7 times as much performance for the same flavour size
13 © Stack Evolution Ltd
Mimicking real world workloads: Database and Logging and measuring disk performance using a 20GB Volume of a standard type
14 © Stack Evolution Ltd
A High Performance Computing Benchmark, then weighted by flavour hourly cost and finally weighted by Volume monthly cost (for 20GB Volume).
Notice how much the graphs change once you add in pricing information to the performance data.
15 © Stack Evolution Ltd
Data provided by:
Thanks
Contact [email protected] if you want to find out more.