computer science 1 adaptive overload control for busy internet servers matt welsh and david culler...
TRANSCRIPT
![Page 1: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/1.jpg)
1Computer Science
Adaptive Overload Control for Busy Internet Servers
Matt Welsh and David Culler
USITS 2003
Presented by: Bhuvan Urgaonkar
![Page 2: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/2.jpg)
2Computer Science
Internet Services Today Massive concurrency demands
Yahoo: 1.2 billion+ pageviews/day AOL web caches: 10 billion hits/day
Load spikes are inevitable Peak load is orders of magnitude greater than average Traffic on September 11, 2001 overloaded many news
sites Load spikes occur exactly when the service is most
valuable!• In this regime, overprovisioning is infeasible
Increasingly dynamic Days of the “static” web are over Majority of services based on dynamic content
• E-commerce, stock trading, driving directions etc.
![Page 3: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/3.jpg)
3Computer Science
Problem Statement Supporting massive concurrency is hard
Threads/processes don’t scale very well
Static resource containment is inflexible How to set a priori resource limits for widely varying
loads? Load management demands a feedback loop
Replication alone does not solve the load management problem Individual nodes may still face huge variations in
demand
![Page 4: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/4.jpg)
4Computer Science
Proposal: The Staged Event-Driven Architecture
SEDA: A new architecture for Internet services A general-purpose framework for high concurrency and
load conditioning Decomposes applications into stages separated by queues
Enable load conditioning Event queues allow inspection of request streams Can perform prioritization or filtering during heavy load Apply control for graceful degradation
• Perform load shedding or degrade service under overload
![Page 5: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/5.jpg)
5Computer Science
Staged Event-Driven Architecture
Decompose service into stages separated by queues Each stage performs a subset of request processing Stages internally event-driven, typically nonblocking Queues introduce execution boundary for isolation
Each stage contains a thread pool to drive stage execution Dynamic control grows/shrinks thread pools with demand
![Page 6: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/6.jpg)
6Computer Science
Per-stage admission control
Admission control done at each stage Failure to enqueue a request => backpressure on
preceding stages Application has flexibility to respond as appropriate Less conservative than single AC
![Page 7: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/7.jpg)
7Computer Science
Response time controller
90th percentile response time over some interval passed to the controller
AIMD heuristic used to determine token bucket rate Exact scheduling mechanisms unspecified Future work: Automatic tuning of parameters
![Page 8: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/8.jpg)
8Computer Science
Overload management
Class based differentiation Segragate request processing for each class into its
own set of stages Or, have a common set of stages but make the
admission controller aware of the classes
Service degradation SEDA signals occurrence of overload to applications If application wants it may degrade service
![Page 9: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/9.jpg)
9Computer Science
Arashi: A SEDA-based email service
A web-based email service
o Managing folders, deleting/refiling mails, search etc
Client workload emulates several simultaneous users, user behavior derived from traces of the UCB CS IMAP server
![Page 10: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/10.jpg)
10Computer Science
Controller operation
![Page 11: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/11.jpg)
11Computer Science
Overload control with increased user load
![Page 12: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/12.jpg)
12Computer Science
Increased user load (contd)
![Page 13: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/13.jpg)
13Computer Science
Overload control under a massive load spike
![Page 14: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/14.jpg)
14Computer Science
Per-stage AC Vs Single AC
![Page 15: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/15.jpg)
15Computer Science
Advantages of SEDA
Exposure of the request stream Request level performance made available to
application
Focused, application-specific admission control Fine-grained admission control at each stage Application can provide own admission control policy
Modularity and performance isolation Inter-stage communication via event passing enables
code modularity
![Page 16: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/16.jpg)
16Computer Science
Shortcomings
Biggest shortcoming: Heuristic based May work for some applications, fail for others
Not completely self-managed Response time targets supplied by administrator Controller parameters set manually
Limited to apps based on the SEDA approach
Evaluation of overheads missing
Exact scheduling mechanisms missing
![Page 17: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/17.jpg)
17Computer Science
Some thoughts/directions…
Formal ways to reason about the goodness of resource management policies Also, the distinction between transient and
drastic/persistent overloads
Policy issues: Revenue maximization and predictable application performance Designing Service Level Agreements Mechanisms to implement them
Application modeling and workload prediction
![Page 18: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/18.jpg)
18Computer Science
Overload control: a big picture
Unavoidable overload
Avoidable overload
Underload
Detection of overloads Formal and rigorous ways of defining the goodness of “self-managing” techniques UO and AO involve different actions (e.g. admission control versus reallocation). Are they fundamentally different?
![Page 19: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/19.jpg)
19Computer Science
Knowing where you are!
Distinguish avoidable overloads from unavoidable overloads Need accurate application models, workload predictors
Challenges: multi-tiered applications, multiple resources, dynamically changing appl behavior
Simple models based on networks of queues? How good would they prove?
Resource allocations Workload(predicted)
Performance Goal
MODEL
![Page 20: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/20.jpg)
20Computer Science
Workload prediction: a simple example
A static application model Find cpu and nw usage distributions by offline
profiling Use the 99th percentiles as cpu, nw requirements
When the application runs “for real” We don’t get to see what the tail would have been So … resort to some prediction techniques E.g., a web server:
• record # requests N• record # requests serviced M• extrapolate to predict the cpu, nw requirements
of N requests
![Page 21: Computer Science 1 Adaptive Overload Control for Busy Internet Servers Matt Welsh and David Culler USITS 2003 Presented by: Bhuvan Urgaonkar](https://reader036.vdocument.in/reader036/viewer/2022062720/56649f135503460f94c27481/html5/thumbnails/21.jpg)
21Computer Science
Service-level agreements We may want…
Workload Response time w1 r1 w2 r2
… wN rN
Is this possible to achieve? Maybe not. How about:
Response time Revenue/requestr1 $$1
…rN $$N