Threads versus EventsCSE451Andrew Whitaker

This ClassThreads vs. events is an ongoing debateSo, neat-and-tidy answers arent necessarily availableOur goals:Understand the debateAnalyze argumentsForm our own opinion

A Brief History of a Debate1995: Why Threads are a Bad Idea (for most purposes)John Ousterhout, (UC Berkeley, Sun Labs)2001: SEDA: An Architecture for Well-Conditioned, Scalable Internet ServicesStaged, Event-driven ArchitectureM. Welsh, D. Culler, and Eric Brewer (UC Berkeley)2003: Why Events are a Bad Idea (for high-concurrency servers)R. van Behren, J. Condit, Eric Brewer (UC Berkeley)

BackgroundProblem: How do scale up servers to handle many simultaneous requests

Server workloads have a lot of I/O, a little CPU

Send HTTP request over networkRead and parse HTTP requestRead file from diskInvoke write syscallSend response over networktimeHTTP processing

Quick ExampleQ: Suppose it takes 10 seconds for a web server to transfer a file to the client. Of this time, 10 milliseconds is dedicated to CPU processing. How many simultaneous requests do we need to keep the CPU fully utilized?

A: 1000

Concurrency Strategy #1: Thread-per-RequestRun each web request in its own threadAssuming kernel threads, blocking I/O operations only stall one thread

Review: Web Server Worker ThreadRead from socket to extract request URI (e.g., index.html)Read desired file into a bufferWrite file over the socketClose the socket

while (true) {}

Concurrency Strategy #2: Event-driven ExecutionUse a single thread for all requestsUse non-blocking I/OReplace blocking I/O with calls that return immediatelyProgram is notified about interesting I/O events

This is philosophically similar to hardware interruptsTell me when something interesting happens

Event Loop Pseudocodewhile (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock);}In a pure event system, this is the only thread

Details: UNIX Select System Call

Select takes three arrays of file descriptorsReads, writes, and exceptional eventsNote that file descriptors may represent an open file or a socketSelect returns the subset of each array that is ready for reading, writing, or has an exceptional conditionOr, blocks if no FDs are activeint select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

Event Handler Pseudocodeprivate Hashtable perSocketState;

// this code *never* blockspublic void handleRead(Socket sock) { // lookup state about this socket SocketState state = perSocketState.get(sock);

state.bytesRead += sock.nonBlockingRead(state.buffer,state.bytesRead);

// if weve read the entire request, send a response if (state.bytesRead >= REQUEST_LENGTH) { sendResponse(sock); } else {} // do nothing; wait for another event}

Can You Think of a System that Works This Way?GUI frameworks (like Swing)Single event-handling threadUser programs install event listeners to receive callbacksEventLoopEvent Handlers

Why Events?A potentially simpler programming modelNo concurrency, locks, conditional variables, etc.Less memory footprintA single stack (at most, a stack per CPU)More scalable?

Cut to Ousterhout

Critiquing OusterhoutEvents dont handle true CPU concurrencyWe can fix this with an event-loop per processorBut, then we have multiple threadsProgramming model becomes complex if event handlers do not run to completionThis requires stack ripping to maintain state

State Transition Diagram for the Web ServerIn a threaded system, state transitions are maintained by the stackIn an event system, we must manually maintain the state of each requestStack rippingReadingrequestReadingFileWritingFile

Andrews Opinion: Code UnderstandingEvent systems are difficult to read / understandWhich events happen in which order?while (true) { // Ask the OS for Sockets with active I/O; // this blocks if no socket is active Socket sock = getActiveSocket(); if (sock.isReadable()) handleReadEvent(sock); if (sock.isWriteable()) handleWriteEvent(sock);}

Threads vs Events in PracticeBoth are used in production systemsApache is multi-threaded / multi-processLighttpd is event-driven

Which is better? Why?

The Devil is in the (Implementation) DetailsMany systems did not support kernel threadsMaking events more attractiveSupport for non-blocking I/O is unevene.g., UNIX/Linux do not support non-blocking file I/OThe UNIX select command has poor scalability (Banga et al, 1998)But, more recent alternatives to select are much more scalable (Linux epoll)Thread packages often do not scale (Welsh, 2001)But, optimizations can vastly improve thread scalabilityVan Behren 2003, Linux 2.6 and NPTL

So, What Can We Say For Sure?Event-based systems are harder to programRequires stack rippingHarder to track control flowPossible exception: run-to-completion event handlers (e.g., Java Swing GUIs)Event systems use fewer resourcesAt most a stack per processorEvent systems require less synchronizationBut, they still require some on multi-processor systems (which is every system!)Open question: is this is any easier to program?

Software Engineering IssuesUsually, choice of concurrency model is made for historical reasonsUnfortunately, changing concurrency models is very difficultUsethreadsUseeventsBeginproject

Use this slide to explain why a web server must utilize multiple threadsCut to Ousterhout