concurrent programming - carnegie mellon school of ... · carnegie mellon bryant and o’hallaron,...
TRANSCRIPT
![Page 1: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/1.jpg)
Carnegie Mellon
1 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Programming 15-‐213: Introduc�on to Computer Systems 23rd Lecture, Nov. 15, 2016
Instructor: Randy Bryant
![Page 2: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/2.jpg)
Carnegie Mellon
2 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Programming is Hard!
¢ The human mind tends to be sequen�al
¢ The no�on of �me is o�en misleading
¢ Thinking about all possible sequences of events in a computer system is at least error prone and frequently impossible
![Page 3: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/3.jpg)
Carnegie Mellon
3 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Data Race
![Page 4: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/4.jpg)
Carnegie Mellon
4 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Deadlock
![Page 5: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/5.jpg)
Carnegie Mellon
5 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Deadlock ¢ Example from signal handlers. ¢ Why don’t we use prin� in handlers?
void catch_child(int signo) { printf("Child exited!\n"); // this call may reenter prin�/puts! BAD! DEADLOCK! while (waitpid(-1, NULL, WNOHANG) > 0) continue; // reap all children } int main(int argc, char** argv) { ... for (i = 0; i < 1000000; i++) { if (fork() == 0) { // in child, exit immediately exit(0); } // in parent sprintf(buf, "Child #%d started\n", i); printf("%s", buf); } return 0; }
IcurrInext
Acquire lock
Request lock
Receive signal
![Page 6: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/6.jpg)
Carnegie Mellon
6 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Livelock
![Page 7: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/7.jpg)
Carnegie Mellon
7 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Livelock
![Page 8: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/8.jpg)
Carnegie Mellon
8 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Programming is Hard!
¢ Classical problem classes of concurrent programs: § Races: outcome depends on arbitrary scheduling decisions elsewhere in the system § Example: who gets the last seat on the airplane?
§ Deadlock: improper resource alloca�on prevents forward progress § Example: traffic gridlock
§ Livelock / Starva�on / Fairness: external events and/or system scheduling decisions can prevent sub-‐task progress § Example: people always jump in front of you in line
¢ Many aspects of concurrent programming are beyond the scope of our course.. § but, not all J § We’ll cover some of these aspects in the next few lectures.
![Page 9: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/9.jpg)
Carnegie Mellon
9 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Programming is Hard!
It may be hard, but …
it can be useful and some�mes necessary!
![Page 10: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/10.jpg)
Carnegie Mellon
10 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Client / Server Session
Reminder: Itera�ve Echo Server Client Server socket socket
bind
listen
rio_readlineb
rio_writen rio_readlineb
rio_writen
Connec�on request
rio_readlineb
close
close EOF
Await connec�on request from next client
open_listenfd
open_clientfd
accept connect
![Page 11: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/11.jpg)
Carnegie Mellon
11 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Itera�ve Servers
¢ Itera�ve servers process one request at a �me
Client 1 Server connect
accept
write read
call read
close close
write ret read read
![Page 12: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/12.jpg)
Carnegie Mellon
12 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Itera�ve Servers
¢ Itera�ve servers process one request at a �me
Client 1 Server Client 2 connect
accept connect
write read
call read
close
accept
write
read
close Wait for server to finish with Client 1
call read
write
ret read
write ret read read
![Page 13: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/13.jpg)
Carnegie Mellon
13 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Where Does Second Client Block?
¢ Second client a�empts to connect to itera�ve server
¢ Call to connect returns § Even though connec�on not yet accepted
§ Server side TCP manager queues request
§ Feature known as “TCP listen backlog”
¢ Call to rio_writen returns § Server side TCP manager buffers input data
¢ Call to rio_readlineb blocks § Server hasn’t wri�en anything for it to read yet.
Client socket
rio_readlineb
rio_writen
Connec�on request
open_clientfd
connect
![Page 14: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/14.jpg)
Carnegie Mellon
14 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Fundamental Flaw of Itera�ve Servers
¢ Solu�on: use concurrent servers instead § Concurrent servers use mul�ple concurrent flows to serve mul�ple clients at the same �me
User goes out to lunch
Client 1 blocks wai�ng for user to type in data
Client 2 blocks wai�ng to read from server
Server blocks wai�ng for data from Client 1
Client 1 Server Client 2 connect
accept connect
write call read
call read write
call read write ret read
call read
![Page 15: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/15.jpg)
Carnegie Mellon
15 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Approaches for Wri�ng Concurrent Servers Allow server to handle mul�ple clients concurrently
1. Process-‐based § Kernel automa�cally interleaves mul�ple logical flows § Each flow has its own private address space
2. Event-‐based § Programmer manually interleaves mul�ple logical flows § All flows share the same address space § Uses technique called I/O mul�plexing.
3. Thread-‐based § Kernel automa�cally interleaves mul�ple logical flows § Each flow shares the same address space § Hybrid of of process-‐based and event-‐based.
![Page 16: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/16.jpg)
Carnegie Mellon
16 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Approach #1: Process-‐based Servers ¢ Spawn separate process for each client
client 1 server
call connect call accept
call read
ret accept
call fgets fork child 1
User goes out to lunch
Client 1 blocks
wai�ng for user to type
in data
call accept
Child blocks wai�ng for data from Client 1
![Page 17: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/17.jpg)
Carnegie Mellon
17 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Approach #1: Process-‐based Servers ¢ Spawn separate process for each client
client 1 server client 2
call connect call accept
call read
ret accept call connect
call fgets fork child 1
User goes out to lunch
Client 1 blocks
wai�ng for user to type
in data
call accept ret accept
call fgets
write fork
call read
child 2
write
call read
ret read close
close
...
Child blocks wai�ng for data from Client 1
![Page 18: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/18.jpg)
Carnegie Mellon
18 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); echo(connfd); Close(connfd); } exit(0); }
Itera�ve Echo Server
echoserverp.c
§ Accept a connec�on request § Handle echo requests un�l client terminates
![Page 19: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/19.jpg)
Carnegie Mellon
19 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); echo(connfd); /* Child services client */ Close(connfd); /* child closes connection with client */ exit(0); } }
Making a Concurrent Echo Server
echoserverp.c
![Page 20: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/20.jpg)
Carnegie Mellon
20 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } }
Making a Concurrent Echo Server
echoserverp.c
![Page 21: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/21.jpg)
Carnegie Mellon
21 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } }
Making a Concurrent Echo Server
echoserverp.c Why?
![Page 22: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/22.jpg)
Carnegie Mellon
22 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } }
Making a Concurrent Echo Server
echoserverp.c
![Page 23: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/23.jpg)
Carnegie Mellon
23 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
int main(int argc, char **argv) { int listenfd, connfd; socklen_t clientlen; struct sockaddr_storage clientaddr; Signal(SIGCHLD, sigchld_handler); listenfd = Open_listenfd(argv[1]); while (1) { clientlen = sizeof(struct sockaddr_storage); connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); if (Fork() == 0) { Close(listenfd); /* Child closes its listening socket */ echo(connfd); /* Child services client */ Close(connfd); /* Child closes connection with client */ exit(0); /* Child exits */ } Close(connfd); /* Parent closes connected socket (important!) */ } }
Process-‐Based Concurrent Echo Server
echoserverp.c
![Page 24: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/24.jpg)
Carnegie Mellon
24 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Process-‐Based Concurrent Echo Server (cont)
void sigchld_handler(int sig) { while (waitpid(-1, 0, WNOHANG) > 0) ; return; }
§ Reap all zombie children
echoserverp.c
![Page 25: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/25.jpg)
Carnegie Mellon
25 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Server: accept Illustrated listenfd(3)
Client 1. Server blocks in accept, wai�ng for connec�on request on listening descriptor listenfd
clientfd
Server
listenfd(3)
Client
clientfd
Server 2. Client makes connec�on request by calling connect
Connec�on request
listenfd(3)
Client
clientfd
Server 3. Server returns connfd from accept. Forks child to handle client. Connec�on is now established between clientfd and connfd
Server Child
connfd(4)
![Page 26: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/26.jpg)
Carnegie Mellon
26 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Client 2 data
Process-‐based Server Execu�on Model
§ Each client handled by independent child process § No shared state between them § Both parent & child have copies of listenfd and connfd
§ Parent must close connfd § Child should close listenfd
Client 1 server process
Client 2 server process
Listening server process
Connec�on requests
Client 1 data
![Page 27: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/27.jpg)
Carnegie Mellon
27 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Issues with Process-‐based Servers
¢ Listening server process must reap zombie children § to avoid fatal memory leak
¢ Parent process must close its copy of connfd § Kernel keeps reference count for each socket/open file § A�er fork, refcnt(connfd) = 2 § Connec�on will not be closed un�l refcnt(connfd) = 0
![Page 28: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/28.jpg)
Carnegie Mellon
28 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Pros and Cons of Process-‐based Servers
¢ + Handle mul�ple connec�ons concurrently ¢ + Clean sharing model
§ descriptors (no) § file tables (yes) § global variables (no)
¢ + Simple and straigh�orward ¢ –– Addi�onal overhead for process control ¢ –– Nontrivial to share data between processes
§ (This example too simple to demonstrate)
![Page 29: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/29.jpg)
Carnegie Mellon
29 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Approach #2: Event-‐based Servers
¢ Server maintains set of ac�ve connec�ons § Array of connfd’s
¢ Repeat: § Determine which descriptors (connfd’s or listenfd) have pending inputs
§ e.g., using select func�on § arrival of pending input is an event
§ If listenfd has input, then accept connec�on § and add new connfd to array
§ Service all connfd’s with pending inputs
¢ Details for select-‐based server in book
![Page 30: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/30.jpg)
Carnegie Mellon
30 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
I/O Mul�plexed Event Processing
10
connfd’s
7 4 -‐1 -‐1 12 5 -‐1 -‐1 -‐1
0 1 2 3 4 5 6 7 8 9
Ac�ve
Inac�ve
Ac�ve
Never Used
listenfd = 3 Ac�ve Descriptors
10
connfd’s
7 4 -‐1 -‐1 12 5 -‐1 -‐1 -‐1
listenfd = 3 Pending Inputs
Read and service
Anything happened?
![Page 31: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/31.jpg)
Carnegie Mellon
31 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Pros and Cons of Event-‐based Servers
¢ + One logical control flow and address space. ¢ + Can single-‐step with a debugger. ¢ + No process or thread control overhead.
§ Design of choice for high-‐performance Web servers and search engines. e.g., Node.js, nginx, Tornado
¢ –– Significantly more complex to code than process-‐ or thread-‐based designs.
¢ –– Hard to provide fine-‐grained concurrency § E.g., how to deal with par�al HTTP request headers
¢ –– Cannot take advantage of mul�-‐core § Single thread of control
![Page 32: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/32.jpg)
Carnegie Mellon
32 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Approach #3: Thread-‐based Servers
¢ Very similar to approach #1 (process-‐based) § …but using threads instead of processes
![Page 33: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/33.jpg)
Carnegie Mellon
33 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Tradi�onal View of a Process
¢ Process = process context + code, data, and stack
Program context: Data registers Condi�on codes Stack pointer (SP) Program counter (PC)
Code, data, and stack
Stack SP
Shared libraries
Run-‐�me heap
0
Read/write data Read-‐only code/data PC
brk
Process context
Kernel context: VM structures Descriptor table brk pointer
![Page 34: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/34.jpg)
Carnegie Mellon
34 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Alternate View of a Process
¢ Process = thread + code, data, and kernel context
Shared libraries
Run-‐�me heap
0
Read/write data Thread context: Data registers Condi�on codes Stack pointer (SP) Program counter (PC)
Code, data, and kernel context
Read-‐only code/data
Stack SP
PC
brk
Thread (main thread)
Kernel context: VM structures Descriptor table brk pointer
![Page 35: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/35.jpg)
Carnegie Mellon
35 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
A Process With Mul�ple Threads ¢ Mul�ple threads can be associated with a process
§ Each thread has its own logical control flow § Each thread shares the same code, data, and kernel context § Each thread has its own stack for local variables
§ but not protected from other threads § Each thread has its own thread id (TID)
Thread 1 context: Data registers Condi�on codes SP1 PC1
stack 1
Thread 1 (main thread)
shared libraries
run-‐�me heap
0
read/write data
Shared code and data
read-‐only code/data
Kernel context: VM structures Descriptor table brk pointer
Thread 2 context: Data registers Condi�on codes SP2 PC2
stack 2
Thread 2 (peer thread)
![Page 36: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/36.jpg)
Carnegie Mellon
36 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Logical View of Threads
¢ Threads associated with process form a pool of peers § Unlike processes which form a tree hierarchy
P0
P1
sh sh sh
foo
bar
T1
Process hierarchy Threads associated with process foo
T2 T4
T5 T3
shared code, data and kernel context
![Page 37: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/37.jpg)
Carnegie Mellon
37 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Threads
¢ Two threads are concurrent if their flows overlap in �me
¢ Otherwise, they are sequen�al
¢ Examples: § Concurrent: A & B, A&C § Sequen�al: B & C
Time
Thread A Thread B Thread C
![Page 38: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/38.jpg)
Carnegie Mellon
38 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Concurrent Thread Execu�on
¢ Single Core Processor § Simulate parallelism by �me slicing
¢ Mul�-‐Core Processor § Can have true parallelism
Time
Thread A Thread B Thread C Thread A Thread B Thread C
Run 3 threads on 2 cores
![Page 39: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/39.jpg)
Carnegie Mellon
39 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Threads vs. Processes ¢ How threads and processes are similar
§ Each has its own logical control flow § Each can run concurrently with others (possibly on different cores) § Each is context switched
¢ How threads and processes are different § Threads share all code and data (except local stacks)
§ Processes (typically) do not § Threads are somewhat less expensive than processes
§ Process control (crea�ng and reaping) twice as expensive as thread control
§ Linux numbers: – ~20K cycles to create and reap a process – ~10K cycles (or less) to create and reap a thread
![Page 40: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/40.jpg)
Carnegie Mellon
40 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Posix Threads (Pthreads) Interface ¢ Pthreads: Standard interface for ~60 func�ons that manipulate threads from C programs § Crea�ng and reaping threads
§ pthread_create()
§ pthread_join()
§ Determining your thread ID § pthread_self()
§ Termina�ng threads § pthread_cancel()
§ pthread_exit() § exit() [terminates all threads] § return [terminates current thread]
§ Synchronizing access to shared variables § pthread_mutex_init
§ pthread_mutex_[un]lock
![Page 41: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/41.jpg)
Carnegie Mellon
41 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
void *thread(void *vargp) /* thread routine */ { printf("Hello, world!\n"); return NULL; }
The Pthreads "hello, world" Program /* * hello.c - Pthreads "hello, world" program */ #include "csapp.h" void *thread(void *vargp); int main(int argc, char** argv) { pthread_t tid; Pthread_create(&tid, NULL, thread, NULL); Pthread_join(tid, NULL); return 0; }
Thread a�ributes (usually NULL)
Thread arguments (void *p)
Return value (void **p)
hello.c
Thread ID
Thread rou�ne
hello.c
![Page 42: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/42.jpg)
Carnegie Mellon
42 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Execu�on of Threaded “hello, world” Main thread
Peer thread
return NULL; Main thread waits for peer thread to terminate
exit() Terminates
main thread and any peer threads
call Pthread_create()
call Pthread_join()
Pthread_join() returns
printf()
Peer thread terminates
Pthread_create() returns
![Page 43: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/43.jpg)
Carnegie Mellon
43 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Thread-‐Based Concurrent Echo Server int main(int argc, char **argv) { int listenfd, *connfdp; socklen_t clientlen; struct sockaddr_storage clientaddr; pthread_t tid; listenfd = Open_listenfd(argv[1]); while (1) {
clientlen=sizeof(struct sockaddr_storage); connfdp = Malloc(sizeof(int)); *connfdp = Accept(listenfd, (SA *) &clientaddr, &clientlen); Pthread_create(&tid, NULL, thread, connfdp);
} return 0; }
echoservert.c
§ Spawn new thread for each client § Pass it copy of connec�on file descriptor § Note use of Malloc()! [but not Free()]
![Page 44: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/44.jpg)
Carnegie Mellon
44 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Thread-‐Based Concurrent Server (cont)
/* Thread routine */ void *thread(void *vargp) { int connfd = *((int *)vargp); Pthread_detach(pthread_self()); Free(vargp); echo(connfd); Close(connfd); return NULL; }
§ Run thread in “detached” mode. § Runs independently of other threads § Reaped automa�cally (by kernel) when it terminates
§ Free storage allocated to hold connfd. § Close connfd (important!)
echoservert.c
![Page 45: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/45.jpg)
Carnegie Mellon
45 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Thread-‐based Server Execu�on Model
§ Each client handled by individual peer thread § Threads share all process state except TID § Each thread has a separate stack for local variables
Client 1 server peer thread
Client 2 server peer thread
Listening server
main thread
Connec�on requests
Client 1 data Client 2 data
![Page 46: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/46.jpg)
Carnegie Mellon
46 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Issues With Thread-‐Based Servers
¢ Must run “detached” to avoid memory leak § At any point in �me, a thread is either joinable or detached § Joinable thread can be reaped and killed by other threads
§ must be reaped (with pthread_join) to free memory resources § Detached thread cannot be reaped or killed by other threads
§ resources are automa�cally reaped on termina�on § Default state is joinable
§ use pthread_detach(pthread_self()) to make detached
¢ Must be careful to avoid unintended sharing § For example, passing pointer to main thread’s stack
§ Pthread_create(&tid, NULL, thread, (void *)&connfd);
¢ All func�ons called by a thread must be thread-‐safe § (next lecture)
![Page 47: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/47.jpg)
Carnegie Mellon
47 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Poten�al Form of Unintended Sharing
main thread
peer1
while (1) { int connfd = Accept(listenfd, (SA *) &clientaddr, &clientlen); Pthread_create(&tid, NULL, thread, &connfd);
}
connfd
Main thread stack
vargp
Peer1 stack
connfd = connfd1
connfd = *vargp
peer2
connfd = connfd2
connfd = *vargp
Race!
Why would both copies of vargp point to same loca�on?
Peer2 stack
vargp
![Page 48: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/48.jpg)
Carnegie Mellon
48 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Could this race occur?
int i; for (i = 0; i < 100; i++) { Pthread_create(&tid, NULL, thread, &i); }
¢ Race Test § If no race, then each thread would get different value of i § Set of saved values would consist of one copy each of 0 through 99
Main
void *thread(void *vargp) { int i = *((int *)vargp); Pthread_detach(pthread_self()); save_value(i); return NULL; }
Thread
![Page 49: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/49.jpg)
Carnegie Mellon
49 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Experimental Results
¢ The race can really happen!
No Race
Mul�core server
0
1
2
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98
0
2
4
6
8
10
12
14
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98
Single core laptop
0 1 2 3
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 54 56 58 60 62 64 66 68 70 72 74 76 78 80 82 84 86 88 90 92 94 96 98
![Page 50: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/50.jpg)
Carnegie Mellon
50 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Pros and Cons of Thread-‐Based Designs
¢ + Easy to share data structures between threads § e.g., logging informa�on, file cache
¢ + Threads are more efficient than processes
¢ –– Uninten�onal sharing can introduce subtle and hard-‐to-‐reproduce errors! § The ease with which data can be shared is both the greatest strength and the greatest weakness of threads
§ Hard to know which data shared & which private § Hard to detect by tes�ng
§ Probability of bad race outcome very low § But nonzero!
§ Future lectures
![Page 51: Concurrent Programming - Carnegie Mellon School of ... · Carnegie Mellon Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec ve, Third Edi on 1 Concurrent Programming](https://reader034.vdocument.in/reader034/viewer/2022052320/5ca93d9788c9931f068c05c7/html5/thumbnails/51.jpg)
Carnegie Mellon
51 Bryant and O’Hallaron, Computer Systems: A Programmer’s Perspec�ve, Third Edi�on
Summary: Approaches to Concurrency
¢ Process-‐based § Hard to share resources: Easy to avoid unintended sharing § High overhead in adding/removing clients
¢ Event-‐based § Tedious and low level § Total control over scheduling § Very low overhead § Cannot create as fine grained a level of concurrency § Does not make use of mul�-‐core
¢ Thread-‐based § Easy to share resources: Perhaps too easy § Medium overhead § Not much control over scheduling policies § Difficult to debug
§ Event orderings not repeatable