processes 1: fork & exec cs 360 proc1. page 2 proc1 cs 360, wsu vancouver agenda what are...

Processes 1: Fork & Exec

CS 360

proc1

proc1 CS 360, WSU Vancouver

Agenda

What are processes?

Creating a process

Miscellaneous details

Tiny shell example

Wrap-up

This week we begin learning about Unix processes - independently running programs


What Are Processes?


Layered Architecture Kernel:

essential basic services only one kernel general

Libraries: useful additional services

many libraries specific

Programs:executable files

numerous & specific

Shell:command line environment

few & powerful

Users:humans

users

shell

programs

unixlibraries

unixkernel


The Two Unix Building Blocks

Files- storage areas on disk

storedata

Processes- running programs in memory

manipulatedata


What Is a Process? An instance of an executing program

A fundamental computer science concept and powerful coding technique

program bits

a file a process

• independent computations• interact via messages


% factor 19384779834734444128792384733311159 &% echo hellohello% ps PID TTY S TIME CMD 23122 ttyp1 I 0:00.17 -ksh (ksh) 24849 ttyp1 I + 0:00.04 vi pipe2.c 23793 ttyp2 S 0:00.05 factor% kill 23793

Examples of Processes 1 Start a "background"

process:

Notes: "pid" = "process id" (an integer) a process can be in one of several states:

– running actually has machine– sleeping ready to run, but waiting for turn– idle sleeping for more than 20 seconds– waiting waiting for an event (such as disk I/O)– zombie ended, but parent has not yet received notification– ...


Pipe together several processes:

% tr " " "\n" | sort | uniq

print unique words

tr " " "\n"stdin sort uniq stdout

process 2process 1 process 3

buffer buffer

new stdout 1 new stdin 2 new stdout 2 new stdin 3

Examples of Processes 2

simple,powerful

Notes: each process has it's own independent existence each process runs asynchronously they can communicate only in limited ways (pipes, signals, ...) they will wait when stdin pipe is not yet ready, or stdout pipe is full


Examples of Processes 3 Graphics server:

program 1

program 2

program 3graphicsserver

workstationscreen

Web server:

serverdisk

webserver

machine 1

machine 2

one machine

many machines


Key Process Routines

Creating: how to create a process fork, exec, wait

Communicating: how to read/write between processes pipe, dup, sockets

Controlling: how to cause and handle exceptions signal, alarm

Scripting: how to use the shell from a program system, popen

key routines

We won't cover: administration process groups, sessions, terminals, job control advanced comm IPC flavors accounting priorities, resources, accounting error handling setjmp, longjmp

today'stopic

everythingelse next week


Creating Processes


Process = Image + ContextProcess1

image

context

Processn

image

context

...

• key addressescurrent instruction counter (IC)stack pointer (SP)frame pointer (FP)heap bottom, stack bottom, etc.

• register values• scheduling priority

• process id's (me & parent) • user & group id's• file descriptors

0: ...1: ...2: ...

• current working directory

aka "thread"data heap stackavail

aka "address space"

000 ... ... fff

shell stuff

"text" "bss"

typical layout

kernel loads un-initialized data

code

environmentvariables


How Do We Create a Process?

As we have seen, a process is a very complicated thing

We might expect a complicatedway to create a process

Is there a simple way to avoidthis complexity?

Page 14 proc1 Roger Ray, CptS 360, WSU Vancouver


image

context

Processn

image

context

...




0: ...1: ...2: ...



aka "address space"

000 ... ... fff

shell stuff

"text" "bss"

typical layout


code



Page 14 proc1 Roger Ray, CptS 360, WSU Vancouver


image

context

Processn

image

context

...




0: ...1: ...2: ...



aka "address space"

000 ... ... fff

shell stuff

"text" "bss"

typical layout


code


To Create a Process, Clone an Existing Process

Then the new process modifies itself gradually using the usual kernel

routines (open, ...).

a general principle: create complex objects simply by copying an existing objectPage 14 proc1 Roger Ray, CptS 360, WSU Vancouver


image

context

Processn

image

context

...




0: ...1: ...2: ...



aka "address space"

000 ... ... fff

shell stuff

"text" "bss"

typical layout


code


1. "parent"

3. "child"

2. "clone"


"Fork": Clones a Process

step 1:

step 2:

step 3:

step 4:

step 5:

single process

parent process child processtwo independentprocesses

two independentprocesses



step 1:

step 2:

parent process child process

step 3:

step 4:

step 5:



step 1: execute "fork ()"

step 2:

step 3:

step 4:

step 5:





step 2: complete parent stateis cloned ... ... and child is created

step 3:

step 4:

step 5:

clone image &clone context

(stdin, stdout, ...)






fork returns pid of child fork returns 0step 3:

step 4:

step 5:

two asynchronousprocesses







if statement chooses a "parent path"

if statement chooses a "child path"step 4:

step 5:

two asynchronousprocesses









compute something ... compute something else ...step 5:









compute something ... compute something else ...step 5:

cloned image &cloned context


completelyindependent



#include <unistd.h>#include <stdio.h>

int main (int argc, char *argv[]) {

char *who; int i;

if (fork ()) { who = "parent"; } else { who = "child"; }

for (i = 0; i < 6; i++) {

printf ("*fork1: %s\n", who); }

exit (0);}

The Parent and Child are Duplicates

fork1.c % fork1*fork1: parent*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent *fork1: child*fork1: child

program code program behavior

make sure you understand this

fork returns twice!


Again: The Parent and Child are Duplicates

parent process program behaviorint main (int argc, char *argv[]) {

char *who;int i;if (fork ()) {

who = "parent";} else {

who = "child";}for (i = 0; i < 6; i++) {

printf ("*fork1: %s\n", who);}exit (0);

}

child process

int main (int argc, char *argv[]) {char *who;int i;if (fork ()) {


who = "child";}for (i = 0; i < 6; i++) {


}



who = "child";}for (i = 0; i < 6; i++) {


}

process is cloned;fork returns new pid in original, 0 in clone



who = "child";}for (i = 0; i < 6; i++) {


}



who = "child";}for (i = 0; i < 6; i++) {


}

% fork1*fork1: parent*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent*fork1: child*fork1: parent *fork1: child*fork1: child


Execution Is Asynchronous

#include <unistd.h>#include <stdio.h>


int i; /* loop counter */char *who; /* name of process */int n; /* seconds to sleep */

if (fork ()) {who = "parent";n = 2;

} else {who = "child";n = 1;

}

for (i = 1; i <= 10; ++i) {fprintf (stdout,

"*%2d. %7s: my pid = %6d, ppid = %6d\n",i, who, getpid (), getppid ());

fflush (stdout);sleep (n);

} exit(0);

}

fork2.c % fork2* 1. parent: my pid = 11597, ppid = 7125* 1. child: my pid = 10843, ppid = 11597* 2. child: my pid = 10843, ppid = 11597* 2. parent: my pid = 11597, ppid = 7125* 3. child: my pid = 10843, ppid = 11597* 4. child: my pid = 10843, ppid = 11597* 3. parent: my pid = 11597, ppid = 7125* 5. child: my pid = 10843, ppid = 11597* 6. child: my pid = 10843, ppid = 11597* 4. parent: my pid = 11597, ppid = 7125* 7. child: my pid = 10843, ppid = 11597* 8. child: my pid = 10843, ppid = 11597* 5. parent: my pid = 11597, ppid = 7125* 9. child: my pid = 10843, ppid = 11597*10. child: my pid = 10843, ppid = 11597* 6. parent: my pid = 11597, ppid = 7125* 7. parent: my pid = 11597, ppid = 7125* 8. parent: my pid = 11597, ppid = 7125* 9. parent: my pid = 11597, ppid = 7125*10. parent: my pid = 11597, ppid = 7125

program code program behavior

unpredictable interleaving

sleep n seconds


"Exec": Loads a Program






compute somethingstep 5:

step 6:new steps



"Exec": Loads a Program






compute something overlay with new imagefrom an executable filestep 5:

cloned image &cloned context


new image(but stdin etc.not changed)

wait for child to completestep 6: exit with status code



You Can Exec a Program

#include <unistd.h>#include <stdio.h>#include <math.h>


char *who; int status;

if (fork ()) {

who = "parent"; printf ("pi=%f\n", 4*atan(1));

wait (&status); exit (0);

} else { who = "child"; execlp ("/usr/bin/date", "date", (char *)0); }

}

exec1.c

% exec1pi=3.141593Fri Feb 19 17:11:16 PST 1999

program code

program behavior

exec doesn't return!

wait for child to exit

overlays with new imagepreserves most of context (e.g.: open file descriptors)

asynchronous execution


Again: You Can Exec a Programprogram behavior

parent processif (fork ()) {

who = "parent";printf ("pi=%f\n", 4*atan(1));wait (&status);exit (0);

} else {who = "child";execlp ("/usr/bin/date", "date", (char *)0);

}

% exec1pi=3.141593Fri Feb 19 17:11:16 PST 1999

if (fork ()) {



}

/usr/bin/date:argc: 1argv[0]: "date"argv[1]: NULL

exec

exit

samestdout

poof!

overlay!

clone!

poof!

child process

if (fork ()) {



}

if (fork ()) {



}

fork


Exec Issues Why does the "date" output appear on the terminal? Is that output guaranteed to follow the parent output? What would happen to the parent if the child did this ...?

close (1); What if the child closed 0 and then did an open? What if the child did another exec? What if the parent process exited before the child? What is the output from this program?

...printf ("a");if (fork ()) {

printf ("b");} else {printf ("c");

}exit (0);...

% foofoo.c


You Can Also Exec a Scriptprogram code

#include <unistd.h>#include <stdio.h>#include <math.h>


char *who; int status;

if (fork ()) { who = "parent"; printf pi=%f\n", 4*atan(1));

wait (&status); exit (0);

} else { who = "child";

execlp ("/bin/my-script", "my-script", "a", "b", (char *)0); }}

exec2.c

#! /bin/kshecho "*my-script: argv[1]=$1 argv[2]=$2"

/bin/my-script

% exec2pi= 3.141593*my-script: argv[1]=a argv[2]=b

program behavior

pathname of script

which shell to use


"Fork" and "Exec" Synopsis

Exec = start new program:

$PATH is searched to resolve the filename– can resolve to a program or a script (shell used in that case)

image of current process is replaced; can only return if that fails most of context preserved (stdin/stdout/stderr, pid, uid, gid, cwd, getenv, ...) but: stdio buffers not flushed (use fflush or fclose before the exec)

execlp (filename, argv0, argv1, ..., (char *)0);execvp (filename, argv);

final execvp argvmust be NULL

Fork = create new process:

returns twice: with 0 in child; with a new pid in parent both child & parent continue with duplicated image & context

– however, parent of child == parent (not the parent's parent) child completes via "exit(status)"; parent can "wait" on child completion processes can also "signal" each other (more next week)

if ((child-pid = fork ()) != 0) {parent-code

} else {child-code

}


Some Miscellaneous Details


Environment Variables Shell makes "environment variables" available to your program:

% export FOO="something important"% doitFOO is something important

...char *foo;foo = getenv ("FOO"); if (foo == NULL) foo = "undefined";printf ("FOO is %s\n", foo);...

doit.c

NULL if "FOO" not exported

Notes: by convention environment variables are all upper case the shell always sets several useful variables: PATH, HOME, LOGNAME, ... you can change a value via "putenv" change is only "downwards" -- can't change parent's environment it's possible to scan the entire environment via an array, but that is unusual


A Real Example: Tiny Shell

the TSH icon

See the process routinesin action and understandmore about how a shell works


Our Tiny Shell is a Subset of the Full Shell Tsh is a real shell:

% tsh? echo "hello, world!"hello, world? ls -l-rwxrwxrwa 1 0 0 2689 Feb 18 12:44 main.c-rwxrwxrwa 1 0 0 47 Feb 17 22:59 makefile.txt? cp main.c backup? cat < foo > bar? exit%

However, command syntax is greatly simplified: single line commands from stdin (no line continuation or semi-colons) arguments are words or strings (with either " or ' quotes) I/O redirection only for stdin and stdout; no pipes no substitutions ($, ~, *, ...) simple program invocation only (no scripts, no control structures, no arithmetic) foreground operation only (no ampersands)


Tsh Overall Logicvarious includes, defines, externs

tsh.h

main (...) {repeat until eof or exit command:

get command line into a string area (see get.c)parse command into argc, argv and redirect [0..1](see parse.c)exec command using argc, argv and redirect (see exec.c below)note: exec command returns 0 if the command was an exit, else 1

}

main.c

int exec_command (int argc, char *argv[], char *redirect[]) {handle the built-in commands (exit & cd) speciallyotherwise, call exec_program to do the work (see prog.c)return 0 if did an "exit", else return 1

exec.c

int exec_program (int argc, char *argv[], char *redirect[]) {fflush (stdout)forkif I am the child:

redirect stdin & stdout as needed using redirect [0..1](see redirect.c)make argv[argc] = 0execvp (argv[0], argv)if execvp returns, complain to stderr and terminate myself

else I am the parent:wait for child process to terminatereturn the exit status (which exec-command currently ignores)

prog.c

list of all files:• tsh.h• main.c, get.c, parse.c, misc.c• exec.c, prog.c, redirect.c


/* standard includes ... */

#include <stdio.h>#include <stdlib.h>#include <assert.h>#include <string.h>

/* defines and variables ... */

#define VERSION "v7" /* version number */#define MAX_LENGTH 1024 /* max length of a command line */#define MAX_ARGC 50 /* max number of args */#define PROMPT "? " /* command prompt */

/* command routines ... */

extern int get_command (char *cmd, int max_length);extern int parse_command (char *argv[], int max_argc, char *redirect[], char *cmd); extern int exec_command (int argc, char *argv[], char *redirect[]); extern int exec_program (int argc, char *argv[], char *redirect[]);extern void redirect_stdin (char *pathname); extern void redirect_stdout (char *pathname);

/* misc routines (see misc.c for details) ... */

extern void initialize (int argc, char *argv[]);extern int complain (char *message, void *value); /* fprintf to stderr; returns 0 */extern int verify (char *argv[], int argc, int min, int max); /* min <= argc <= max */

/* debugging ... */

extern int debug; /* initialized in misc.c */#define trace if (debug) printf

The Header Filetsh.h

main routines

helper routines


#include "tsh.h"int debug = 0;

/**********************************************************************main:

Usage: tsh [-d]**********************************************************************/

int main (int main_argc, char *main_argv[]) {

/* declare variables */char cmd[MAX_LENGTH]; /* command line area */char *argv[MAX_ARGC+1]; /* args found in cmd */int argc; /* number of args found */char *redirect[2]; /* filenames for stdin/stdout or null */int more; /* when 0, command loop below ends */

/* process command line and get ready */initialize (main_argc, main_argv);

/* get, parse, and execute commands until eof or "exit" command */more = 1;while (more && get_command (cmd, MAX_LENGTH)) {

argc = parse_command (argv, MAX_ARGC, redirect, cmd);more = exec_command (argc, argv, redirect);

}

/* exit */exit (0);

}

main.c

The Main Routineenvironment

main loop


#include "tsh.h"

/**********************************************************************get_command:

Prompt and read command.Complain and ignore lines that are longer than max_length.Return 0 if eof.

**********************************************************************/

int get_command (char *cmd, int max_length) {

/* declare variables */int eof, c;

/* prompt user for a command */fputs (PROMPT, stdout);

/* read one line into cmd area */eof = (fgets (cmd, max_length, stdin) == NULL);

/* complain if line was truncated; and if so, ignore the entire command */if (!eof && (cmd[strlen(cmd)-1] != '\n')) {

complain ("line longer than %d\n", (void *) max_length);while (c = fgetc (stdin), c != '\n' && c != EOF) {}; /* ignore rest of line */cmd[0] = '\0'; /* make command empty */

}

/* return with true if not eof */return !eof;

}

Get_Commandget.c

prompt and then read command line


Parse_Command

Source omitted: breaks cmd into argv and returns argc understands single and double quotes redirect[0] is a pathname if there is a '<', else null ditto redirect[1] for '>'

#include "tsh.h"#include <ctype.h>/*****************************************************************************parse_command :

Parse cmd into argv and redirect. Return argc (0 if errors or empty cmd).*****************************************************************************/int parse_command (char *argv[], int max_argc, char *redirect[], char *cmd) {

...}

parse.c

argc: 4argv[0]: "ls"argv[1]: "-l"argv[2]: "foo.c"argv[3]: "bar.c"

cmd: "ls -l foo.c bar.c > /tmp/listing\n"

redirect[0]: 0redirect[1]: "/tmp/listing"

note the 0

Example:


#include "tsh.h"

/**********************************************************************exec_command :

Execute a single command per argv[0..argc-1] and redirect[0..1].Command is built-in or a program.Return 1 (0 if command was "exit").

**********************************************************************/

int exec_command (int argc, char *argv[], char *redirect[]) {

/* declare variables */int result, status;

/* execute the command and set result */if (argc <= 0) {

result = 1; /* empty command; keep going */} else if (strcmp (argv[0], "exit") == 0) {

result = 0; /* exit command; stop the tsh */} else if (strcmp (argv[0], "cd") == 0) {

if (verify (argv, argc, 1, 2)) {if (argc == 1) argv[1] = getenv ("HOME");if (chdir (argv[1]) < 0) complain ("can't cd to %s", argv[1]);

}result = 1; /* keep going */

} else {status = exec_program (argc, argv, redirect);result = 1; /* keep going */

}

/* return result */return result;

}

Exec_Commandexec.c

exec a program

built-in: empty, "exit" and "cd"

commands


#include "tsh.h"#include <unistd.h>

/**********************************************************************exec_program:

Execute a program per argv[0..argc-1] and redirect[0..1].Return exit status.

**********************************************************************/int exec_program (int argc, char *argv[], char *redirect[]) {

/* declare variables */pid_t child; /* child process pid or 0 */int status; /* child process exit status */

/* fork into child and parent processes */fflush (stdout);if ((child = fork ()) != 0) {

/* this is the parent: wait for child to end */wait (&status);return status;

} else {

/* this is the child: overlay with another program */if (redirect[0] != NULL) redirect_stdin (redirect[0]);if (redirect[1] != NULL) redirect_stdout (redirect[1]);argv[argc] = NULL;execvp (argv[0], argv);complain ("can't execvp %s", argv[0]);exit (1);

}}

Exec_Programprog.c

fork, changestdin/stdout as needed,then exec the program

flush and fork


#include "tsh.h"#include <unistd.h>#include <fcntl.h>/**********************************************************************redirect routines:

Close and then reopen file descriptor 0 or 1.Exit if errors.

**********************************************************************/

void redirect_stdin (char *pathname) {int fd;

close (0);fd = open (pathname, O_RDONLY);

if (fd < 0) {complain ("can't open %s as stdin", pathname);exit (1);

}}

void redirect_stdout (char *pathname) {int fd;close (1);fd = open (pathname, O_WRONLY|O_TRUNC|O_CREAT, 0666);if (fd < 0) {

complain ("can't open %s as stdout", pathname);exit (1);

}}

Exec_Redirectredirect.c

we exploit the "lowest avail" file descriptor logic of "open"

why do we exit when thereis an error?


Discussion The full source is posted in /encs_share/cs/class/cs360/src/tsh

Worth examination see how a medium-sized program is structured see how debugging routines are used in source if adventurous, read "parse.c" invoke tsh with debugging switch (-d) to see detailed operation

Areas for improvement: built-in commands semicolons pipes

which common commandsneed "built-in" handling?


Summary The concept of "process" is fundamental to both

computer science and engineering A process is

an instance of an executing program It is implemented as

an image (address space) + a context (thread) Unix creates processes by cloning

achieves simplicity by copying a complex object, instead of creating piecemeal The key process creation routines are:

fork -- both image & context duplicated; returns twice

exec -- image replaced, most of context preserved; doesn't return

wait With ingenuity, you can do many things with fork/exec and wait

e.g.: open file descriptors

processes 1: fork & exec cs 360 proc1. page 2 proc1 cs 360, wsu vancouver agenda what are...

Documents