module 2
DESCRIPTION
Module 2. Programming with Processes. Processes. Process -- a program in execution May share code segments Typically do not share data, stack, heap OS data structures (file handles, coroutine, context block) kept in struct named User One User struct per process. Login. - PowerPoint PPT PresentationTRANSCRIPT
Module 2Module 2
Programming with ProcessesProgramming with Processes
ProcessesProcesses
•Process -- a program in execution–May share code segments–Typically do not share data, stack, heap
•OS data structures (file handles, coroutine, context block) kept in struct named User•One User struct per process
•Process -- a program in execution–May share code segments–Typically do not share data, stack, heap
•OS data structures (file handles, coroutine, context block) kept in struct named User•One User struct per process
LoginLogin• Authenticates via name/password• Name converted to number(s)
– User Id– Group Id : used for sharing
Macintosh-2:~ shawne$ ls -ltotal 48drwx------+ 89 shawne staff 3026 Mar 1 14:03 Desktopdrwx------+ 7 shawne staff 238 Feb 13 15:50 Documentsdrwx------+ 5 shawne staff 170 Feb 19 11:07 Downloads
• Authenticates via name/password• Name converted to number(s)
– User Id– Group Id : used for sharing
Macintosh-2:~ shawne$ ls -ltotal 48drwx------+ 89 shawne staff 3026 Mar 1 14:03 Desktopdrwx------+ 7 shawne staff 238 Feb 13 15:50 Documentsdrwx------+ 5 shawne staff 170 Feb 19 11:07 Downloads
User id group id
Process Creation and Destruction
Process Creation and Destruction
int execle(const char *filepath,const char *arg, ..., (char *)0, char *const envp[] );/* replaces caller's memory image with "program" */
void exit(int status);/* terminates execution; returns status to parent */
pid_t wait(int *stat_loc);pid_t waitpid(pid_t pid, int *stat_loc, int options);/* blocks caller until a child terminates.
pid_t fork(void);/* duplicates the memory image of the caller */
int execle(const char *filepath,const char *arg, ..., (char *)0, char *const envp[] );/* replaces caller's memory image with "program" */
void exit(int status);/* terminates execution; returns status to parent */
pid_t wait(int *stat_loc);pid_t waitpid(pid_t pid, int *stat_loc, int options);/* blocks caller until a child terminates.
pid_t fork(void);/* duplicates the memory image of the caller */
#include <stdio.h>#include <stdlib.h>#include <unistd.h>
int main() { int pid, out; char *envp[]={"ME=BOB",0}; pid=fork(); printf("I am %d\n",pid); if (pid!=0) { wait(&out); printf("child returned %d\n",out); execle("/bin/echo","dog","cat",0,envp); } exit(55);}
#include <stdio.h>#include <stdlib.h>#include <unistd.h>
int main() { int pid, out; char *envp[]={"ME=BOB",0}; pid=fork(); printf("I am %d\n",pid); if (pid!=0) { wait(&out); printf("child returned %d\n",out); execle("/bin/echo","dog","cat",0,envp); } exit(55);}
C program usesOS APIs to write
commands
Shell programs
Window programs
Browser
Browser programs
C program usesOS APIs to write
commands
Shell programs
Window programs
Browser
Browser programs
UNIX ShellUNIX Shell
Boot-up to CommandBoot-up to Command
Remote Executionssh user@hostname command
Remote Executionssh user@hostname command
• Automatic login is desirable– The scheme is based on public-key
cryptography, using cryptosystems where encryption and decryption are done using separate keys, and it is infeasible to derive the decryption key from the encryption key.
– The idea is that each user creates a public/private key pair for authentication purposes. The server knows the public key, and only the user knows the private key.
– Private key encrypts login info; public key decrypts
• Automatic login is desirable– The scheme is based on public-key
cryptography, using cryptosystems where encryption and decryption are done using separate keys, and it is infeasible to derive the decryption key from the encryption key.
– The idea is that each user creates a public/private key pair for authentication purposes. The server knows the public key, and only the user knows the private key.
– Private key encrypts login info; public key decrypts
Data ParallelismData Parallelism• Why: speed-up
• How (greatly simplified): – split work into independent similar pieces– execute pieces in parallel– collect results
• Simple example– Count number of identifiers in a set of files
• Why: speed-up
• How (greatly simplified): – split work into independent similar pieces– execute pieces in parallel– collect results
• Simple example– Count number of identifiers in a set of files
Practical Parallel and Concurrent Programming DRAFT: comments to [email protected]
106/16/2010
Count ids in several files./test foo.txt moo.txt boo.txt
Count ids in several files./test foo.txt moo.txt boo.txt
#include <unistd.h>#include <stdio.h>#include <sys/wait.h>
int main(int argc, char *argv[]) { int count=0, i;for (i=1; i<argc; i++) { if (fork()) continue;
execl("./cnt","./cnt",argv[i],0);} //forfor (i=1; i<argc; i++) { int cnt; wait(&cnt); count+=WEXITSTATUS(cnt);}printf("count = %d¥n", count);exit(0); }
#include <unistd.h>#include <stdio.h>#include <sys/wait.h>
int main(int argc, char *argv[]) { int count=0, i;for (i=1; i<argc; i++) { if (fork()) continue;
execl("./cnt","./cnt",argv[i],0);} //forfor (i=1; i<argc; i++) { int cnt; wait(&cnt); count+=WEXITSTATUS(cnt);}printf("count = %d¥n", count);exit(0); }
Shell procedure to count ids
Shell procedure to count ids
declare -i x=0for i in $* ; do x=x+`./cnt $i` ; echo $x ;DoneCommand Line./foo.sh /Users/bobcook/Desktop/*.txtOutput31826053141
declare -i x=0for i in $* ; do x=x+`./cnt $i` ; echo $x ;DoneCommand Line./foo.sh /Users/bobcook/Desktop/*.txtOutput31826053141
Amdahl’s hypothesisAmdahl’s hypothesis• Assume that sorting takes 70% of the
execution time of a sequential program• You replace the sorting algorithm with
one that scales perfectly on multi-core hardware
• How many cores do you need to get a 4x speed-up of the program?
• Assume that sorting takes 70% of the execution time of a sequential program
• You replace the sorting algorithm with one that scales perfectly on multi-core hardware
• How many cores do you need to get a 4x speed-up of the program?
6/16/2010 Practical Parallel and Concurrent Programming DRAFT: comments to [email protected]
14
6/16/2010 Practical Parallel and Concurrent Programming
DRAFT: comments to [email protected] 15
Amdahl’s Formula
6/16/2010 Practical Parallel and Concurrent Programming
DRAFT: comments to [email protected] 16
Speedup achieved with perfect scalingSpeedup achieved
with perfect scaling
Amdahl’s law limit, just 1.11x
Amdahl’s law limit, just 1.11x
f = 10%
6/16/2010 Practical Parallel and Concurrent Programming
DRAFT: comments to [email protected] 17
Desired 4x speedup
Desired 4x speedup
Speedup achieved (perfect scaling on 70%)
Speedup achieved (perfect scaling on 70%)
Limit as c→∞ = 1/(1-f) = 3.33Limit as c→∞ = 1/(1-f) = 3.33
f = 70%
6/16/2010 Practical Parallel and Concurrent Programming
DRAFT: comments to [email protected] 18
f = 98%
LessonLesson
• Speedup is limited by sequential code
• Even a small percentage of sequential code can greatly limit potential speedup
• Speedup is limited by sequential code
• Even a small percentage of sequential code can greatly limit potential speedup
6/16/2010 Practical Parallel and Concurrent Programming DRAFT: comments to [email protected]
19
Reasons for IPC
Information sharing
Program speedup
Fault tolerance
Resource limitations per node
Heterogeneity
Modularity
Convenience
Privilege separation -- different levels of protection
Inter-Process Communication (IPC)
Inter-Process Communication (IPC)
PipesMessage queuesShared memoryShared filesSignalsSockets (TCP/IP protocols)Message passing (MPI, discussed later)Semaphores (discussed later)Remote variable access (discussed later)Remote procedure call (discussed later)Remote process invocation (ssh commands)
Common IPC APIsCommon IPC APIs
#include <unistd.h>#include <stdlib.h>#include <stdio.h>#include <string.h>
int main() { //pipe input to sort command FILE *write; char *str[]={"hi", "by", "so", "mo"}; int i; write = popen("sort", "w"); if (write !=NULL) { for (i=0; i<4; i++) fprintf(write, "%s\n", str[i]); pclose(write); exit(EXIT_SUCCESS); } exit(EXIT_FAILURE);}OUTPUT by hi mo so
Pipe Command Example(invoke commands from C
programs)
Pipe Command Example(invoke commands from C
programs)
#include <unistd.h>
int pipe(int fildes[2] /* out parameter*/);fildes[0] - read end of pipefildes[1] - write end of pipe
Be aware that write() can write less than you request and read() can read less. File handles are inherited by fork()/exec() processes.
Pipe System Call APIPipe System Call API
FIFO File Object System APIFIFO File Object System API#include <sys/types.h>#include <sys/stat.h>
int mkfifo(const char *path, mode_t mode);
Like a file, a FIFO can be opened for reading or writing or both intermittently.
Message Queue APIMessage Queue API
Message Queue -record oriented
Message Queue -record oriented
#include <fcntl.h> /* For O_* constants */#include <sys/stat.h> /* For mode constants */#include <mqueue.h>struct mq_attr { long mq_flags; /* Flags: 0|O_NONBLOCK */ long mq_maxmsg; /* Max. # of msgs on Q*/ long mq_msgsize; /* Max. msg size (bytes) */ long mq_curmsgs; /* # msgs currently in Q*/};mqd_t mq_open(const char name[], int oflag); //to open onlymqd_t mq_open(const char name[], int oflag, mode_t access_mode, struct mq_attr *attr);int mq_close(mqd_t mqdes);int mq_unlink(const char name[]);
int mq_send(mqd_t mqdes, const char msg_ptr[], size_t msg_len, unsigned msg_priority);ssize_t mq_receive(mqd_t mqdes, char msg_ptr[], size_t msg_len, unsigned *msg_priority);
Processing Multiple Files in ParallelOne thread per Chunk
Shared Memory Among Multiple Processes
//returns shmid for new segment of size bytesint shmget(key_t key, size_t size, int mode_t);
//control ops such as stat, unlink, lock, unlockint shmctl(int shmid, int cmd, struct shmid_ds *buf);
//attach segment to process at shmaddrvoid *shmat(int shmid, const void *shmaddr, int shmflag);
//detach segment at shmaddr from this processint shmdt(const void *shmaddr);
Shared Memory API
Memory-Mapped Files forIncreased Efficiency
#include <sys/mman.h>
void *mmap(void *start_addr, size_t length, int protection, int flags, int filedescriptor, off_t offset);
int munmap(void *start_addr, size_t length);
Memory-Mapped File API