Testing adaptive workload management

Harumi KunoHP Labs

Stefan Krompass (TUM), Kevin Wilkinson, Umeshwar Dayal, Goetz Graefe, Janet Wiener

Compile-timeQuery

Optimization

Run-timeQuery

Execution

QueryPlan

Performance

Actualconditions

(work + resources)

Expectedconditions

(work + resources)

2

Controlling resources for a complex (dynamic mixed) workload is hard

Traditional solution is to isolate components, by partitioning work across hardware or time multiplexing.

3

Grand Challenge (1):Managing dynamic mixed workloads

• Ignore it • Avoid problem through isolating systems or

using time multiplexing• Provide rich tools to be used with manual

workload management

• Adaptive workload management

4

Dynamic mixed workloads are difficult to manage because resource contention

… changes resource requirements

Disk and memory usage needed to execute a sort changes with the amount of available memory

Goetz Graefe, Harumi Kuno, Janet Wiener. Visualizing the Robustness of Query Execution. Proc. Conference on Innovative Data Systems Research (CIDR). January 4-7, 2009.

5

Dynamic mixed workloads are difficult to manage because resource contention

… changes performance

Throughput/MPL for a single homogenous workload varies with different cache hit rates

6

Janet L. Wiener, Harumi Kuno, Goetz Graefe. Benchmarking Query Execution Robustness. First TPC Technology Conference on Performance Evaluation and Benchmarking (http://www.tpc.org/tpctc2009), held in conjunction with VLDB 2009.

Dynamic mixed workloads are difficult to manage because resource contention

… and is difficult to predict

Throughput/MPL for OLTP queries changes as various report queries run

Report 1

Report 3Report 2

7Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

Can static workload management policies handle unreliable cost estimates?

none

Admission control threshold

0

20

40

60

80

100

120

140

160

Weig

hted

mak

espa

n (i

n th

ousa

nds o

f sim

ulat

or ti

me u

nits)

3A3C

0A0C

3A3C

3A2C3A0C

3A0C

0A0C0A0C0A0C

0A0C

1.0m

absolute 12000

penalty absolute 5000none absolute 5000,

progress <30%relative 1.2x

under-informed admission control and scheduling decisions

8Stefan Krompass, Harumi Kuno, Janet L.Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. A Testbed for Managing Dynamic Mixed Workloads. Demonstration at VLDB 2009.

Can static workload management policies handleunobserved resource contention?

monitored resource not the source of contention

9Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

Can static workload management policies handle system overload?

0

20

40

60

80

100

120

140

160

Weig

hted

mak

espa

n(in

thou

sand

s of s

imul

ator

tim

e uni

ts)

2A2C 2A0C2A0C

2A0C2A0C 2A0C

2A0C

2A0C

2A0C

2A2C

MPL 4 MPL 10 (overload)

penalty

relative 1.2xabsolute 12000none

absolute 5000absolute 5000,progress <30%

No. Challenge: Static policies have a hard time correcting overload situations.

10Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

Adaptive Workload Management Hypothesis

We can build a system that uses feedback to add a policy control loop.

11‹#›Stefan Krompass, Harumi Kuno, Janet Wiener, Kevin Wilkinson, Umeshwar Dayal, Alfons Kemper. Managing Long-Running Queries. Proc. EDBT 2009.

Testing Adaptive Workload Management

• Functionality of management tools• Configuration for a particular anticipated workload

• How it handles the unexpected.

12

Grand Challenge (2)

Underlying challenges• Characterize workloads and service classes: queries,

mega-queries (graphs of queries considered as a single unit), loads, continuous inserts, etc. Plus usual features: estimated arrival rates, execution times, resource usage, etc., together with SLOs.

• Map the components of the workload to service classes.

• Develop recommendations for policies for the query control loop and policy control loop.

• Consider elasticity (dynamic scale out of resources) and outage avoidance.

• Evaluate responses to the unexpected. 13‹#›

You’d like to make decisions are made based on expected performance, but….

expected unexpected

unex

pect

ed

Resource Requirements (e.g., degree of skew)

Reso

urce

Ava

ilabi

lity

expe

cted

Performance predictable

Performance harder to predict

Performance harder to predict

Runtime performance really, really hard to predict

• BI queries -> skew + complex queries

• Skew + complex queries -> unexpected amount of work.

• More work -> more resource usage.

• Multiple queries with unexpected resource usage -> unexpected resource availability.

14‹#›