seda – staged event-driven architecture seda: an architecture for well-conditioned scalable...
TRANSCRIPT
![Page 1: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/1.jpg)
SEDA – Staged Event-Driven Architecture
SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer
Presented by Rahul Agarwal
![Page 2: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/2.jpg)
2
Overview
Motivation Key Points Thread vs. Event concurrency SEDA Experimental Evaluation My Evaluation Questions to Consider
![Page 3: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/3.jpg)
3
Authors Main Author
Matt Welsh’s PhD thesis at UC Berkeley Now Assistant Professor in CS Dept at
Harvard Currently working on wireless sensor
networks Research interests in OS, Networks and
Large Scale Distributed Systems Culler & Brewer - Advisors
![Page 4: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/4.jpg)
4
Motivation
High demand for web content – for example concurrent millions of users for Yahoo
Load Spikes – “Slashdot Effect” Over provisioning support not feasible
financially and events unannounced Web services getting more
complex, requiring maximum server performance
![Page 5: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/5.jpg)
5
Motivation – Well-conditioned Service
A well-conditioned service behave like a simple pipeline
As offered load increase throughput increases proportionally
When saturated it does not degrade throughput – graceful degradation
![Page 6: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/6.jpg)
6
Pipelining
How can we improve throughput in a pipeline? Potential problems?
![Page 7: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/7.jpg)
7
Key Points Application as a network of event-
driven stages connected by queues Dynamic Resource Controllers Automatic load tuning
Thread pool sizing Event Batching Adaptive load shedding
Support massive concurrency and prevent resources from being overcommitted
![Page 8: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/8.jpg)
8
Thread-Based vs. Event-Based Concurrency Thread
Thread-per-request OS switches and
overlaps computation and I/O
Synchronization required Performance
Degradation Eg: RPC, RMI, DCOM
Overcome by more control to OS
Eg: SPIN, Exokernel, Nemesis
Overcome by reuse of threads
Eg: Apache, IIS, Netscape… everyone!
Event One thread per CPU
(controller) Process events
generated by apps and kernel
Each task implemented as a FSM with transitions triggered by events
Pipelining! Eg: Flash, Zeus,
JAWS Harder to modularize
and engineer
![Page 9: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/9.jpg)
9
Thread-Based vs. Event-Based Concurrency (Contd.)
Thread-Based
![Page 10: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/10.jpg)
10
Thread-Based vs. Event-Based Concurrency (Contd.)
Event-Based
![Page 11: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/11.jpg)
11
SEDA
![Page 12: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/12.jpg)
12
SEDA - Stage Each stage has its
own thread pool Controller adjusts
resources Controller may set
“admission control policy”
Threshold, rate-control, load shedding
Adjust thread pool size Adjust number of
events processed
![Page 13: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/13.jpg)
13
Overload Management Resource Containment
Static method Usually set by admin
Admission Control Parameter based Static or dynamically adjusted
Control-theoretic approach Service Degradation
Deliver lower fidelity Service
![Page 14: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/14.jpg)
14
SEDA Admission Control Policy
Using 90th percentile of set response time as performance metric
Adaptive admission control Uses user classes (IPs,
HTTP header info) Uses token bucket traffic
shaper Possible to use drop-tail,
FIFO, RED or other algorithms
![Page 15: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/15.jpg)
15
Event-driven programming Benefits
Applications can map cleanly into modules
Each stage self-contained Typically little/no data sharing
Challenges Determining stages Stages can block Managing continuations between stages Tracking events
![Page 16: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/16.jpg)
16
Software Contribution
Sandstorm: SEDA Framework NBIO: Non-blocking I/O
implementation Haboob: Implementation of a web-
server aTLS: library for SSL and TLS support
All Java implementations, NBIO uses JNI Last updated July 2002
![Page 17: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/17.jpg)
17
Experimental Evaluation Haboob web-
server Static and
dynamic file load – SpecWEB99 benchmark
1 to 1024 clients Total files size
3.31Gb Memory Cache of
200Mb
Throughput vs. #Clients
![Page 18: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/18.jpg)
18
Evaluation (Contd.) Jain’s Fairness
Index Equality of services
to all clients Suppose server can
support 100 requests
Totally fair if it processes 10 requests of each
Unfair if say it processes 20 requests each for 5 users
Throughput and Fairness vs. #Clients
![Page 19: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/19.jpg)
19
Evaluation (Contd.)Cumulative response time
distribution for 1024 clients
![Page 20: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/20.jpg)
20
Evaluation (Contd.) Gnutella packet router
Non-traditional internet service, routing P2P packets
Ping: Discover other nodes Query: Search for files being served
![Page 21: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/21.jpg)
21
Summary
Notion of stages Explicit event queues Dynamic resource controllers + Improved performance + Well-conditioned services + Scalability
![Page 22: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/22.jpg)
22
My Evaluation Increased performance at higher
loads which was the motivation Marginal increase in throughput but
significantly more fair for higher loads
Throughput does not degrade when resources exhausted
Response times of other servers better at lower loads
![Page 23: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/23.jpg)
23
My Evaluation
In context of duality of OS structures (Lauer & Needham) SEDA is message-oriented Message and Procedure oriented can
be directly mapped, however independent of the application under consideration can performance indeed be similar?
We will see how Capriccio does this – but no simple mapping!
![Page 24: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/24.jpg)
24
Questions to consider SEDA is so good but the whole world
(Apache, IIS, BEA, IBM, Netscape…) still uses thread-based servers?
In real world scenarios how often are there load spikes, should goal be to increase average case performance instead?
Is throughput or fairness a better metric? Being faster “despite” being in Java a
bias or poor choice of language?
![Page 25: SEDA – Staged Event-Driven Architecture SEDA: An Architecture for well-conditioned scalable internet services. Matt Welsh, David Culler & Eric Brewer Presented](https://reader030.vdocument.in/reader030/viewer/2022032600/56649db55503460f94aa690d/html5/thumbnails/25.jpg)
25
References Welsh, M., Culler, D., Brewer, E. (2001). SEDA: An architecture for
well-conditioned, scalable internet services. Proceeding of 18th SOSP, Banff, Canada, October 2001
SEDA Project http://sourceforge.net/projects/seda Welsh, M. (2002). An architecture for well-conditioned, scalable
internet services. PhD Thesis, University of California, Berkeley Welsh, M. (2001) SEDA: An architecture for well-conditioned,
scalable internet services. Presentation at 18th SOSP, Lake Louise, Canada, October 24, 2001
Welsh, M., Culler, D., Adaptive overload control for busy internet servers. Unpublished.
Pipelining: http://cse.stanford.edu/class/sophomore-college/projects-00/risc/pipelining/
Lauer, H. C. & Needham, R. M. (1978). On the duality of operating systems structures. In proceedings 2nd International Symposium on Operating Systems, IRIA, October 1978, reprinted in Operating Systems Review, 13, 2 April 1979, pp 3-19.
Behren, R. et al. (2003). Capricco: Scalable Threads for Internet Services. Proc SOSP, New York, October 2003