1 scheduling for today’s computer systems: scheduling for today’s computer systems: bridging...

1

SCHEDULING FOR TODAY’S SCHEDULING FOR TODAY’S COMPUTER SYSTEMS:COMPUTER SYSTEMS:BRIDGING THEORY AND PRACTICE

Adam Wierman

Mor Harchol-BalterJohn Lafferty

Bruce MaggsAlan Scheller-Wolf

Ward Whitt

Thesis Committee

Carnegie Mellon UniversityCarnegie Mellon UniversityComputer Science Department

2

““SCHEDULING SUCCESS STORIES” ARE SCHEDULING SUCCESS STORIES” ARE EVERYWHEREEVERYWHERE

Biersack, Rai, Urvoy-Keller, Harchol-Balter, Schroeder, Agrawal, Ganger, Petrou, Misra, Feng, Hu, Zhang, Mangharam, Sadowsky, Rawat, Dinda, McWherter, Ailamaki, & others

WebServers

users

Routers

Internet

Disks

CPUs

LocksDatabases

…also p2p, wireless, operating systems…


3

web server,edge router,

etc.

Goal Minimize user response times

THE ESSENCE OF A “SCHEDULING THE ESSENCE OF A “SCHEDULING SUCCESS STORY”SUCCESS STORY”

ProcessorSharing

(PS)

bottleneck resource

Use a different scheduling policy

Use a different scheduling policy


4

load0 0.25 0.5 0.75

mea

n r

esp

on

se t

ime

PS

SRPT?

Sched-

ulingSched

- u

ling FCFS

Assumption: M/GI/1 Queue

SRPT WINS BIGSRPT WINS BIG

Can we trustthis comparison?

Can we trustthis comparison?

?

WHAT POLICY WHAT POLICY SHOULD WE USE?SHOULD WE USE? SRPTSRPT

5

mean response time

SRPT

M / GI / 1

Can’t implementpure SRPT

What aboutmultiserver systems?Real users are

interactive

What aboutfairness tolarge jobs?

HOW DO REAL SYSTEMS HOW DO REAL SYSTEMS DIFFER?DIFFER?

What aboutQoS?

What aboutuser impatience?

What abouttime-varying

arrivals?

What aboutpower

management?

6

mean response time

SRPT

M / GI / 1



interactive

What aboutfairness tolarge jobs?What about

QoS?


What abouttime-varying

arrivals?

What aboutpower

management?

Idealized policiesThe idealized policies studied in theory cannot be used in practice

Limited metricsMany performance metrics that are important in practice are not studied in theory

Simplistic modelsTraditional models include many unrealistic assumptions

3 TYPES OF GAPS BETWEEN 3 TYPES OF GAPS BETWEEN THEORY AND PRACTICETHEORY AND PRACTICE


7

7

THE GOAL OF THE THESIS: THE GOAL OF THE THESIS: BRIDGE THE GAPS BETWEEN BRIDGE THE GAPS BETWEEN

THEORY AND PRACTICETHEORY AND PRACTICE

Moving beyondidealized policies

1Moving beyond

mean response time

2Moving beyond

the M/GI/1

3

60% of the talk 30% of the talk 10% of the talk

8

SRPT

Policies are hybridsof SRPT and PS

How can westudy all these

variations at once?

How can westudy all these

variations at once?

?

Policies use only 2 levels

In practice...

Policies use estimates of

job sizes

In practice...

Time-varying workloads time-varying policies

In practice...

Designers adjust SRPT due to overheads

In practice...

IDEALIZED IDEALIZED POLICIESPOLICIES

9

THE IDEATHE IDEA Study scheduling

classifications instead of idealized policies

Study schedulingclassifications instead

of idealized policies

SRPTSRPT

SMART

SMART formalizes the heuristic“give priority to small jobs”

10

SRPT



In practice...


job sizes

In practice...


In practice...


In practice...SMART

SMARTεHow do we definethe SMART class?

How do we definethe SMART class?

?



11

11

THE SMART THE SMART CLASSCLASS

1. Bias Property2. Consistency Property3. Transitivity Property

SMAll Response Times

coherencyproperties

12

TWO TWO NOTIONS OF NOTIONS OF “SMALL” “SMALL” JOBSJOBS

small original size small remaining size

13

a b

[Sigmetrics 2005a]

BIAS BIAS PROPERTYPROPERTY

If OriginalSize(a) < RemainingSize(b)then a has priority over b

DON’

T

FORG

ET

14

original size00

remainingsize



[Sigmetrics 2005a]

DON’

T

FORG

ET

15

original size00

remainingsize

lowerpriority

?higher priority



[Sigmetrics 2005a]

DON’

T

FORG

ET


16

EXAMPLESEXAMPLES PSJF

original size

remainingsize

?

RS

Many others!

SRPT

If OrigSize(a) < RemSize(b)then a has priority over b

Bias Propertyallows time varying

policies

Bias Propertyallows time varying

policies

!

17

SRPT



In practice...


job sizes

In practice...


In practice...


In practice...SMART

SMARTεHow close to

SRPT are SMARTpolicies?

How close to SRPT are SMART

policies?

?


18

Bound T(x)SMART

ANALYSIS ANALYSIS SETTING:SETTING:

M/GI/1 preempt-resume queue

APPROACH:APPROACH: E[T]SMART

19

Theorem: Under the M/GI/1, for all SMART policies P,

CONDITIONAL RESPONSE TIME CONDITIONAL RESPONSE TIME UNDER SMART POLICIESUNDER SMART POLICIES

( ) ( ) ( ) ( ) ( )PSJF SRPT SMART SRPT PSJFst stW x R x T x W x R x

Waiting time

Residence time

Waiting time

Residence time

Response timefor a job of size x

[Sigmetrics 2005a]

20

PSJF

SRPT

remainingsize

original size

SMART

?

Picture “proof”: Waiting time

Theorem: Under the M/GI/1, for all SMART policies P,

CONDITIONAL RESPONSE TIME CONDITIONAL RESPONSE TIME UNDER SMART POLICIESUNDER SMART POLICIES

( ) ( ) ( ) ( ) ( )PSJF SRPT SMART SRPT PSJFst stW x R x T x W x R x

21

SMART SMART POLICIES ARE POLICIES ARE

“2-“2-COMPETITIVE”COMPETITIVE”

Theorem: In the M/GI/1, [ ] [ ] 2 [ ]SRPT SMART SRPTE T E T E T

mean response time

[Sigmetrics 2005a]


22


“2-“2-COMPETITIVE”COMPETITIVE”

load, ρ 10

mea

n r

esp

on

se

tim

e

PS

SMART


These bounds are tight


!

SRPT

23

SRPT



In practice...


job sizes

In practice...


In practice...


In practice...SMART

SMARTεAll SMART

policies are withina factor of 2

All SMARTpolicies are within

a factor of 2

!



24

If OrigSize(a) = x andε(x) < RemSize(b)then a has priority over b

SMARTSMARTεεSMARTSMARTIf OrigSize(a) < RemSize(b)

then a has priority over b

remainingsize

original size

? original size

?

ε(x)


25

ε(x) = x + error

How can you characterizejob size estimates?

If OrigSize(a) = x andε(x) < RemSize(b)then a has priority over b

SMARTSMARTεε

original size

?

ε(x)ε(x) can also be defined

to include 2-level policies

26

SMARTSMARTεε POLICIES ARE POLICIES ARE “CONSTANT COMPETITIVE”“CONSTANT COMPETITIVE”

Theorem: In an M/GI/1 under SMARTε policy P

21

[ ] [ ] 2 [ ]1

SRPT P SRPTE T E T E T

( ) (1 ) x xφ bounds the SIZESIZE of largerjobs that get higher priorityδ bounds the LOADLOAD of larger jobs that get higher priority

( ( )) ( )

( )

x x

x orig. size

?

rem.size

ε(x)

x


27

load 10

mea

n r

esp

on

se t

ime

SMART ε

PS

SRPT

real sizes web server traceestimates within 50%

WHAT DOES THIS WHAT DOES THIS TRANSLATETRANSLATE

TO IN PRACTICE? TO IN PRACTICE?

SMARTε allows

adversarial job size errors

SMARTε allows

adversarial job size errors

!

28

SRPT



In practice...


job sizes

In practice...


In practice...


In practice...SMART

SMARTε


29

Analyzing the SMART class beyond E[T]

Introducing & analyzingother classifications

MUCH MORE WORK ON MUCH MORE WORK ON CLASSIFICATIONSCLASSIFICATIONS

[Sigmetrics 2005a][Sigmetrics 2006]

[Perf. Eval. Review 2006][Operations Research 2007]

[Freidman and Hurley, 2004][Rai, Urvoy-Keller, Vernon, Biersack 2005]

[Nunez-Queija, Kherani 2006][Misra, Rubenstein, Feng 2007]

[Kherani 2007]

[Sigmetrics 2003][Sigmetrics 2005b]

[Perf. Eval. Review 2006]

Collaborations with Zwart, Nuyens, Shakkottai, Yang, Harchol-Balter,Osogami, and others


30

30




1Moving beyond

mean response time

2Moving beyond

the M/GI/1

3


31

Designers careabout power usage

In practice...

Designers careabout QoS – Pr(T>x)

In practice...

Designers care aboutweighted response times

In practice...

MEAN MEAN RESPONSE RESPONSE

TIMETIME

Designers care about fairness

In practice...

Designers careabout buffer overflow

probabilities

32

Designers careabout buffer overflow

probabilities

Designers careabout power usage

In practice...

Designers careabout QoS – Pr(T>x)

In practice...

Designers care aboutweighted response times

In practice...

Designers care about fairness

In practice...

MEAN MEAN RESPONSE RESPONSE

TIMETIME

[Perf Eval 2002][Sigmetrics 2003]

[Sigmetrics 2005a][PER 2007]

[Sigmetrics 2005b][Sigmetrics 2006]

[OR 2007]


33

33

ARE POLICIES THAT ARE POLICIES THAT PRIORITIZE PRIORITIZE

SMALL JOBS UNFAIR SMALL JOBS UNFAIR TO LARGE JOBS?TO LARGE JOBS?

34

WHAT DOES WHAT DOES “FAIRNESS” “FAIRNESS”

MEAN?MEAN?...it depends entirely on the application

OUR SETTING:OUR SETTING: Are the response times of large jobs “unfairly” long?

How can weformalize this?

How can weformalize this?

?


35

[Sigmetrics 2003: Best student paper award]

Definition: In an M/GI/1 queue,a policy P is fair if, for all x: [ ( )] 1

1

PE T x

x

WHY IS THIS WHY IS THIS FAIR?FAIR?

Aristotle’s notion of fairnessLike cases should be treated alike,different cases should be treated

differently, and different cases should be treated differently

in proportion to their differences.

[ ( )]

PE T x

x

DON’

T

FORG

ET


36


Rawls’ Theory of Social Justice All social goods should be distributed equally, unless

unequal distribution is to the advantage of the least favored

[ ( )] 1

1

PSE T x

x


1

PE T x

x


DON’

T

FORG

ET


37


Min-Max fairness (Pareto Efficiency)

All jobs deserve an equal shareof the resources

... but if some jobs can use morewithout hurting others, that’s okay

[ ( )] 1min max

1

P

P x

E T x

x


1

PE T x

x


DON’

T

FORG

ET

38

HOW UNFAIR ARE SMART HOW UNFAIR ARE SMART POLICIES?POLICIES?

x

E[T

(x)]

/ x

1/(1-ρ)

SRPT

Theorem: For all service distributions, SRPT is fair if ρ≤0.5.Theorem: For all power law (α) service distributions withα < 1.5, all SMART policies are fair.

1/(1-ρ)

SMART

39

x

E[T

(x)]

/ x

1/(1-ρ)

Theorem: For all service distributions with finite variance, all SMART policies are unfair for high enough load.

the largest jobs aretreated the same asunder PS

small degreeof unfairness

BUT SMART POLICIES CAN BE BUT SMART POLICIES CAN BE UNFAIRUNFAIR

SMART

<4% of the job sizes

What about otherclassifications?

What about otherclassifications?

?

40

FAIRNESS AND CLASSIFICATIONSFAIRNESS AND CLASSIFICATIONS

Always Fair

Sometimes Fair

Always Unfair

PS

[Sigmetrics 2003 Best student paper award]

SMART

PSJF

LRPT

PLJF

FOOLISH

SRPT

41

Always Fair

Sometimes Fair

Always Unfair

Remaining size based

LRPT

Preemptivesize basedPSJF PLJF

SJFLJF

Non-preemptive size based

Non-preemptivenon-size basedLCFS

FCFS

Age basedLAS

PLCFS

PS

SMART FOOLISH

SRPT

FAIRNESS AND CLASSIFICATIONSFAIRNESS AND CLASSIFICATIONS

FSP

[Sigmetrics 2003 Best student paper award]

SYMMETRIC

PROTECTIVE

42

MUCH MORE WORK ON FAIRNESSMUCH MORE WORK ON FAIRNESS

[Williamson, Gong 2003, 2004][Brown 2006]

and others.

analyzing policies& classifications

extending definitionto higher moments

defining othertypes of fairness

Many other papers by:Henderson, Friedman,Biersack, Rai, Ayesta,Aalto, Nunez-Queija,Misra, Feng, Vernon,Williamson, Brown, Bansal, and others

[Raz, Avi-Itzhak 2004][Levy, Raz, Avi-Itzhak 04]

[Sandmann 2005]

[Perf. Eval 2002][Sigmetrics 2003]

[Sigmetrics 2005b][Under submission]

[Under submission]


43

43




1Moving beyond

mean response time

2Moving beyond

the M/GI/1

3


44

Multiserver designsare prevalent

In practice...

Job sizes may becorrelated

Arrivals arebursty

In practice...

Users are interactive

In practice...

The arrival process is time-varying

In practice...M/GI/1M/GI/1

45

Job sizes may becorrelated

Arrivals arebursty

In practice...

Real users are interactive

In practice...

The arrival process is time-varying

In practice...

Multiserver designsare prevalent

In practice...

M/GI/1M/GI/1

[PER 2004][QUESTA 2005][Perf Eval 2006]

[NSDI 2006]


46

REAL USERS ARE INTERACTIVEREAL USERS ARE INTERACTIVE

How does this difference affect

scheduling?

How does this difference affect

scheduling?

?

47

Open System

Send Receive

[NSDI 2006]

Closed System

0 .25 .5 .75 1load

me

an

re

spo

nse

tim

e 300

200

100

me

an

re

spo

nse

tim

e 300

200

100

load0 .25 .5 .75 1

SRPT

PSFCFS

# users=75

PS

SRPT

FCFS

48

REAL USERS ARE NOT OPEN OR REAL USERS ARE NOT OPEN OR CLOSEDCLOSED

[NSDI 2006]

me

an

re

spo

nse

tim

e

mean number of requests per session

300

200

100

00 5 10 15 20

OPEN CLOSEDPS

SRPT

load = 0.7

Where do realworkloads fall?


?

49

[NSDI 2006]

me

an

re

spo

nse

tim

e

mean number of requests per session

300

200

100

00 5 10 15 20

OPEN CLOSED

slashdotted siteCMU web server

Kasparov vs. Deep Blueonline

shopping

world cup siteonline gaming site



?


50

USER BEHAVIOR IMPACTS SYSTEM USER BEHAVIOR IMPACTS SYSTEM DESIGNDESIGN

When evaluatingnew designs, choosea workload generator

carefully

When evaluatingnew designs, choosea workload generator

carefully

!


51

51




1Moving beyond

mean response time

2Moving beyond

the M/GI/1

3


52




1Moving beyond

mean response time

2Moving beyond

the M/GI/1

3

Scheduling classificationsFairness

&QoS – Pr(T>x)

Interactive users&

Multiserver systems

53

Adam WiermanCarnegie Mellon University

[email protected]

The thesis is available at: http://www.cs.cmu.edu/~acw/thesis

55

Non-preemptive

SMART

LCFS

SMART*Remaining size basedSRPT

RS

SMARTЄ

Preemptive size basedPSJF

FOOLISH

FOOLISH*

LRPT

PLJF

ROSBlind

Age based

PLCFS

SYMMETRIC

PROTECTIVEPS

FSP

SJF

LJF

Non-preemptivesize based

FCFS FB

56

THE BIAS THE BIAS PROPERTYPROPERTY

ISN’T ISN’T ENOUGH ENOUGH

orig. size

remainingsize

?

CONSISTENCY

TRANSITIVITY

[Sigmetrics 2005a]

at most 1 hashigher priority

+If a is served ahead ofb then a will always have priority over b

If an arriving job b preempts c, then until b leaves, every arriving job a with original size smaller than b has priority over c.

57

Theorem:

Proof sketch: If there are two, one was the first to have priority over the tagged job.

x1

2a2b

By Consistency 2a can’t receive service

2b has lower priority than x (Bias).If 2b is run, then 1 has lower priority than 2b (Consistency). So, 1 has lower priority than x (Transitivity).

at most 1 has higher priority

?

58

WEB WORKLOADWEB WORKLOADGENERATORSGENERATORS

SurgeSPECWeb

TPC-WSclientRUBiS

WebBenchWebjamma

DO YOU USE AN OPEN OR CLOSED DO YOU USE AN OPEN OR CLOSED MODEL?MODEL?

Open System

Closed Systemhttperf

59

Theorem: In an M/GI/1 with an unbounded,continuous service distribution having finite E[X2],under any non-idling policy we have

and further1

min max [ ( )] /1

PP x E T x x

[Wierman and Harchol-Balter 2003]

11 lim [ ( )] /

1P

xE T x x

Is dividing by “x” the right

scaling?

Is dividing by “x” the right

scaling?

? Is 1/(1-ρ)really a

min-max criteria

Is 1/(1-ρ)really a

min-max criteria

?

60

First Come First ServedFirst Come First Served

x

Under a Paretowith ρ=0.8, this is >80%of the jobs

The unfairness can beunbounded

PS

FCFSE[T

(x)]

/ x

61

SJF LJF

SRPT LRPT

ROS

LCFS

FCFS

FB

PS

PLCFS

PSJF PLJF

Always Fair

Always Unfair

Sometimes Fair

FAIRNESS VS. FAIRNESS VS. EFFICIENCYEFFICIENCY

more circlesbetter meanresponse time

Is there a fair policy with near

optimal performance?

Is there a fair policy with near

optimal performance?

?

62

Fair Sojourn Protocol (FSP)Fair Sojourn Protocol (FSP)“Do SRPT on the PS remaining times”

FSP

00.5

11.5

22.5

33.5

0 1 2 3 4 5

Time

Rem

aini

ng S

ize

PS

00.5

11.5

22.5

33.5

0 1 2 3 4 5

Re

ma

inin

g S

ize

FSP did the same thing

as SRPT

FSP did the same thing

as SRPT

!

63

BEYOND BEYOND EXPECTATION: EXPECTATION:

Higher Moments

Raw moments E[T(x)i]

Central moments Var[T(x)], etc

Cumulant moments

XX

[Wierman and Harchol-Balter 2005]

E[T(x)] ?

64

CUMULANCUMULANTSTS

Cumulants are a descriptive statistic, similar to the moments.

They can be found as a function of the moments:

or from the log of the moment generating function:

0

[ ] log ( )i

i Xi sX M s

s

1

1

1[ ] [ ] [ ] [ ]

1

ii i j

i jj

iX E X X E X

j

1

2

[ ] [ ]

[ ] [ ]

X E X

X Var X

Do theselook familiar?

Do theselook familiar?

?

65

WHYWHYCUMULANCUMULAN

TS?TS?

Cumulants have many nice properties:

1[ ] [ ] 1 i i iX c X c

[ ] [ ] [ ] i i iX Y X Y

[ ] [ ] ii icX c X

additivity:

homogeneity:

1st cumulant is shift-equivariant &the rest are shift-invariant

66

Why is thisthe right

generalization?

Why is thisthe right

generalization?

?

MIN-MAX FAIRNESSMIN-MAX FAIRNESSDefinition: Consider an M/GI/1 queue.A policy P is min-max fair if, for all i:

Wierman and Harchol-Balter 2005

i1[ ( )] 1 + E[B ] for all xP

i iT x x

Lots of open questions here

Lots of open questions here

!

67

TEMPORAL FAIRNESSTEMPORAL FAIRNESS

Definition:The politeness experienced by a job of size x under policy P, Pol(x)P,is the fraction of the response time during which the seniority of the jobis respected.

It is unfair to violate the seniority of a job

[Wierman 2004]

68

Min-max Fairness

Po

lite

nes

sless fair more fair

less

pol

ite

mor

e po

lite

PS

PLCFS

FCFS

SRPT

FSP

FCFS

PS

PLCFS

SRPT

FSP

LRPTLRPT

TheoremIn an M/GI/1 any Always Fair policy has

lim [ ( )] 1x

E Pol x

69

Min-max Fairness

Po

lite

nes

sless fair more fair

less

pol

ite

mor

e po

lite

FCFS

PS

PLCFSLRPT

more circlesbetter meanresponse time

FSP

SRPT

70

MANY OTHER INTERESTING MANY OTHER INTERESTING FAIRNESS METRICSFAIRNESS METRICS

Discrimination(i) ( ) 1/ ( )departure

i

arrival

s t N t dt

percent of service given to job ifair service percentage

DiscFreq = ni + c∙mi

ni = number of jobs that arrived later and completed earlier than job i

mi = number of larger jobs (at the arrival of job i) that complete earlier than job i

[Levy, Raz, Avi-Itzhak 04]

[Sandmann 2005]

71


2-2-COMPETITIVECOMPETITIVE




!Consider the M/D/1

SRPT does FCFS (only in M/D/1). So as ρ1

As ρ1, E[T]PLCFS 2 E[T]SRPT

2[ ] [ ][ ] [ ]

2(1 ) 2(1 )

FCFS E X E XE T E X

PLCFS is in SMART (only in M/D/1)[ ]

[ ]1

PLCFS E XE T

72

ONLINE MULTI-ONLINE MULTI-OBJECTIVE OBJECTIVE

SCHEDULING SCHEDULING USING SMARTUSING SMART

1. Use a parameterized policy setthat is (nearly) dense in SMART,e.g. iRj + S

2. Search (i,j) space for policy thatoptimizes secondary objectives,e.g. fairness and predictability

Partial ordering allows time varying

policies


policies

!

73

A

B

CD


policies


policies

!

time

rem

aini

ng s

ize

AB CD

*PSJF

74

A

B

CD

*PSJF

*SRPT

time

rem

aini

ng s

ize

*PSJF

AB

D ?


policies


policies

!

75

TAIL BEHAVIOR OF SMARTTAIL BEHAVIOR OF SMART

Pr(T>y) is difficult to study directlyso it is typically it is studied asymptotically

Large buffer

Pr( ) as y T y

Many sourcesNμ

NB

λ1λ2

λN

SMART policiesare asymptoticallyequivalent in both


!

76

LARGE BUFFER SCALINGLARGE BUFFER SCALINGPr( ) as y T y

[ ] sXE e

0

( (1 ))lim lim 1

( )x

F x

F x

X is of intermediate regular variation if

X is light-tailed if for some s>0

For this talk, assume no mass inthe upper bound.

LIGHT-TAILED JOB SIZES

HEAVY-TAILED JOB SIZES

77

SMART POLICIES ARE SMART POLICIES ARE ASYMPTOTICALLY EQUIVALENTASYMPTOTICALLY EQUIVALENT

[Nuyens, Wierman, Zwart 2005]

Theorem: Under the GI/GI/1, for all SMART policies:

• when the service distribution is light-tailed with no mass in the endpoint

• when the service distribution is of intermediate regular variation

log Pr( ) ~ log Pr( ) as y T y B y

Pr( ) ~ Pr( (1 ) ) as y T y X y

service distribution

busy period length

Pr( ) ~ Pr( ) as y T y B y

78

[Nunez-Queija, Boxma, Zwart, Borst, Nuyens, and many others]

Pr(T>y) ~ busy period

SMART

worseworse


LIGHT-TAILED JOB SIZES

HEAVY-TAILED JOB SIZES

Log Pr(T>y) ~ busy period

SMARTLCFSSJF

Log Pr(T>y) ~ workload

Pr(T>y) ~ workload

FCFS...

FCFSLCFSSJF...

79

TAIL BEHAVIOR OF SMARTTAIL BEHAVIOR OF SMART

Pr(T>y) is difficult to study directlyso it is typically it is studied asymptotically

Large buffer

Pr( ) as y T y

Many sourcesNμ

NB

λ1λ2

λN



!

80

MANY MANY SOURCES SOURCES SCALINGSCALING

( , )Pr( ( ) ) ~ N I x yT x y e decay rate

The same under all SMART policies

μ

B

λ Nμ

NB

λ1

λ2

λN

[Yang, Wierman, Shakkottai, Harchol-Balter, 2006]

81

T(x) RESULT, plot for E[T(x)]T(x) RESULT, plot for E[T(x)]

Theorem: For all ε, x, y > 0

where PRIO is a 2 class priority queueing policy.

( , ) ( , ) SMART PRIOI x y I x y


[Yang, Wierman, Shakkottai, Harchol-Balter, 2006]

original size

?

remainingsize

Empty!

Picture “proof”:

82

T(x) RESULT, plot for E[T(x)]T(x) RESULT, plot for E[T(x)]

Theorem: Under the M/GI/1, for all SMARTε:

• when the service distribution is unbounded and light-tailed with no mass in the endpoint

• when the service distribution is of intermediate regular variation and

log Pr( ) ~ log Pr( ) as T x B x x

Nuyens, Wierman, under preparation

busy period length

( ) (1 ) x x

TAIL BEHAVIOR OF SMARTTAIL BEHAVIOR OF SMARTεε POLICIESPOLICIES

Pr( ) ~ Pr( ) as y T y B y

83

low variability high variability

mea

n r

esp

on

se

tim

e

1500

1000

500

Open

Closed (MPL=10)Closed (MPL=100)

Closed (MPL=1000)Web

Workloads

HOW QUICKLY DOES HOW QUICKLY DOES CLOSED CLOSED OPEN? OPEN?

84

CHOOSING A SYSTEM MODELCHOOSING A SYSTEM MODEL

1. A site being “Slashdotted” 2. Online gaming site3. Science Institute - USGS4. Online dept. store5. Financial service provider6. Kasparov vs Deep Blue7. CMU web server8. World cup site

Web workloads

Open or closed?

Use a partly-open model...

85

FITTING A PARTLY-OPEN MODELFITTING A PARTLY-OPEN MODEL

12 ip1 GET a.gif HTTP/1.020 ip2 GET b.htm HTTP/1.025 ip1 GET c.jpg HTTP/1.027 ip1 GET d.txt HTTP/1.028 ip3 GET a.htm HTTP/1.035 ip4 GET d.gif HTTP/1.045 ip2 GET e.htm HTTP/1.0::

Trace

service demands

file sizes from trace

PARTLY-OPEN PARTLY-OPEN

86

FITTING A PARTLY-OPEN MODELFITTING A PARTLY-OPEN MODEL

12 ip1 GET a.gif HTTP/1.020 ip2 GET b.htm HTTP/1.025 ip1 GET c.jpg HTTP/1.027 ip1 GET d.txt HTTP/1.028 ip3 GET a.htm HTTP/1.035 ip4 GET d.gif HTTP/1.045 ip2 GET e.htm HTTP/1.0::

Trace

PARTLY-OPEN PARTLY-OPEN

Fitting the interarrival times

• Distinguish userse.g. use ip address in a web trace

• Identify user session boundaries Use periods of inactivity of length > timeout

Can’t use trace directlybecause dependencies

between completions andfollow-up requests would

be lost!

87

CHOOSING A TIMEOUT VALUECHOOSING A TIMEOUT VALUE

Nu

mb

er

of

ses

sio

ns

2e5

1e5

00 30min

Timeout length

financial

world cup

dept store

88

HOW TO HOW TO CHOOSECHOOSE

A SYSTEM A SYSTEM MODELMODEL

Gathera

trace

How many simult. users are

there?

Fit a partlyopen modelto the trace

OPEN ≈ CLOSED

>>1000

else

What is theexpected num.

of visits?

OPEN CLOSED???

<5 5-10 >10

Me

an

nu

m. o

f vi

sit

s

15

10

5

00 30min

Timeout length

world cup

dept store

financial

89

MULTISERVER QUEUESMULTISERVER QUEUES• Preemptive-Resume Priority• Homogeneous hosts

jobs L H HL

H

H

90

HOW MANY SERVERS ARE BEST?HOW MANY SERVERS ARE BEST?

1 best

2 best

3 best

4 best

1 best

23

4 best

1 fast (rate 1) vs. k slow (rate 1/k)

91

mean response time

SRPT

M / GI / 1

What aboutQoS?



interactive

What aboutfairness tolarge jobs?

What abouttime-varyingworkloads?


What aboutpower usage?

GAPS BETWEEN THEORY AND PRACTICEGAPS BETWEEN THEORY AND PRACTICE

1 scheduling for today’s computer systems: scheduling for today’s computer systems: bridging...

Documents

srpt policies

srpt slide

practice slide

idealized policies slide

ps policies

mean response time srpt

hybrids of srpt

pure srpt