london openstack meetup: measuring performance: the basics

Measuring Performance: the basicsby Stack Evolution

2 © Stack Evolution Ltd

Why?I just finished building my IaaS cloud.

But…

Is it any good?

3 © Stack Evolution Ltd

Lots of metrics and measuresSystem monitoringSystem metric graphingLog file analysis

These are focused on the infrastructure, not the tenant workload

4 © Stack Evolution Ltd

IaaS is a Shared PlatformTunable memory/processor/disk overcommitDynamic workloadsBatch jobsTime of day/month/yearMany variables

5 © Stack Evolution Ltd

Measuring the tenant workloadsTenant only cares about workload

performanceHow does the disk perform?Is the processor faster or slower?What is performance of a specific Database?How do HPC jobs perform?

6 © Stack Evolution Ltd

BenchmarksMany standard benchmark tools available:IperfUnixbenchFioSysbench_OLTPAerospikeRedisScimark2HPCC

7 © Stack Evolution Ltd

Standardise Suite of BenchmarksSelect flavours to directly compare memoryVolumes of standard size (20GB)Network: Internal and ExternalSpecific applications

DatabaseLoggingStreamingHPC

8 © Stack Evolution Ltd

Run on multiple platformsCollect data from

Public CloudsPrivate cloud deploymentsStaging/Test Clouds

9 © Stack Evolution Ltd

CompareNow compare the data to evaluate how good your cloud really is.

Demo

11 © Stack Evolution Ltd

Variability of performance for similar flavours across cloud providers, notice the latency and boot time graphs higher is worse, others higher is better

12 © Stack Evolution Ltd

Overall Index score for combined benchmarks, notice the variability in providers is as much as 7 times as much performance for the same flavour size

13 © Stack Evolution Ltd

Mimicking real world workloads: Database and Logging and measuring disk performance using a 20GB Volume of a standard type

14 © Stack Evolution Ltd

A High Performance Computing Benchmark, then weighted by flavour hourly cost and finally weighted by Volume monthly cost (for 20GB Volume).

Notice how much the graphs change once you add in pricing information to the performance data.

15 © Stack Evolution Ltd

Data provided by:

Thanks

Contact darryl@stackevolution.com if you want to find out more.