pipes - cs.usfca.edu
TRANSCRIPT
![Page 1: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/1.jpg)
PipesCS 326: Operating Systems
Lecture 9
![Page 2: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/2.jpg)
âȘ Code reviews and grading will start Friday in labâȘ Donât worry, you wonât be bored during lab time! (Iâll
post new lab assignments that will help with P2)
Code Review
CS 326: Operating Systems 2
![Page 3: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/3.jpg)
âȘ Pipes: The Basics
âȘ I/O StreamsâȘ Piping Data
Todayâs Agenda
CS 326: Operating Systems 3
![Page 4: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/4.jpg)
âȘ Pipes: The Basics
âȘ I/O StreamsâȘ Piping Data
Todayâs Agenda
CS 326: Operating Systems 4
![Page 5: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/5.jpg)
âȘ Pipes are a common way for programs tocommunicate on Unix systems
âȘ Most useful for sharing unstructured data (such a text)between processes
âȘ They work like how they sound: if you want to senddata to another process, send it through the pipe
Pipes [1/2]
CS 326: Operating Systems 5
![Page 6: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/6.jpg)
âȘ Pipes are one of the fundamental forms of Unix IPC
âȘ With pipes, we can âglueâ several utilities together:âȘ grep neato file.txt | sort
âȘ This will search for âneatoâ in ïżœle.txt and print eachmatch
âȘ Next, these matches get sent over to the âsortâutility
âȘ This is achieved via I/O redirectionâȘ We can redirect the standard output stream away
from our terminal and into another program instead!
Pipes [2/2]
CS 326: Operating Systems 6
![Page 7: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/7.jpg)
âȘ As weâve seen, pipes are used frequently in the shell
âȘ We can mix and match diïżœerent utilities, and they allwork well together
âȘ Awesome!
âȘ Some genius must have designed all these programsto work this way, right?
âȘ Well, no. They all just read from stdin and then writeto stdout (and stderr )
âȘ No coordination required between developers
In the Shell
CS 326: Operating Systems 7
![Page 8: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/8.jpg)
âȘ When you enter âlsâ in your shell, youâre running aprogram
âȘ This functionality is NOT built into your shell. Bashsimply ïżœnds and runs the âlsâ program. Thatâs it!
âȘ So the most basic shell is:âȘ fork()âȘ exec()
âȘ Wait for the child process to ïżœnish
To be clearâŠ
CS 326: Operating Systems 8
![Page 9: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/9.jpg)
âȘ There are some shell âcommandsâ that actually arenâtprograms, called built-ins
âȘ For example, history : printing previous commandsâȘ The shell knows what commands youâve entered, so
having an external program do this would beâŠsomewhat questionable
âȘ And, obviously, exitâȘ If you create a new process, itâs not like it can quit for
the parent process, right?âȘ Well, I guess there IS a way to do thatâŠ
Built-Ins
CS 326: Operating Systems 9
![Page 10: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/10.jpg)
âȘ Can you think of why cd needs to be a built-incommand?
âȘ Itâs similar to the problem with exit
âȘ We can change the current directory of ourselves, butnot our parent
âȘ This is true for general Unix-based systems, but if wethink back to our discussion on clone() âŠ
âȘ âŠLinux allows us to actually share the CWD with ourparent (man pages of clone talk about this cd case)
Another Built-In: cd
CS 326: Operating Systems 10
![Page 11: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/11.jpg)
âȘ I have posted a video from Bell Labs on the schedulethat discusses several design aspects of Unix
âȘ Itâs actually pretty interesting⊠well, at least as far asOS content goes!
âȘ Discussion on pipes starts right around the 5 minutemark
Going to the Source
CS 326: Operating Systems 11
![Page 12: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/12.jpg)
Another Take (Julia Evans)
CS 326: Operating Systems 12
![Page 13: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/13.jpg)
âȘ Pipes: The Basics
âȘ I/O StreamsâȘ Piping Data
Todayâs Agenda
CS 326: Operating Systems 13
![Page 14: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/14.jpg)
âȘ Each program gets allocated three streams by default:âȘ stdout (standard output)âȘ stderr (standard error)
âȘ stdin (standard input)
âȘ As with most things in Unix, these are represented asïżœles (via ïżœle descriptors)
âȘ These streams have diïżœerent functionsâŠ
Input/Output Streams
CS 326: Operating Systems 14
![Page 15: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/15.jpg)
âȘ When you call printf , you are writing to stdout
âȘ This stream is designed for general program output;for example, if you type ls then the list of ïżœles shoulddisplay on stdout
âȘ You can pipe stdout to other programs:âȘ ls -l / | grep 'bin'
âȘ âŠor redirect to a ïżœle:âȘ ls -l / > list_of_files.txt
stdout
CS 326: Operating Systems 15
![Page 16: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/16.jpg)
âȘ The standard error stream is used for diagnosticinformation
âȘ This way, program output can still be passed to otherprograms/ïżœles but weâll still see diagnostics printed tothe terminal
âȘ Helps us know when something went wrong
âȘ Unlike stdout , stderr is not buïżœered
stderr
CS 326: Operating Systems 16
![Page 17: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/17.jpg)
âȘ The ïżœnal stream, stdin, one way to provide programinput (via scanf , for example)
âȘ This can be entered by the user, or we can pipe inputdirectly into a program:
âȘ ls -l / | grep 'bin'
âȘ ls -l / â writes ïżœle list to stdoutâȘ grep 'bin' â reads ïżœle list from stdin
stdin
CS 326: Operating Systems 17
![Page 18: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/18.jpg)
âȘ We can control where printed output goes with fprintf
âȘ File printf
âȘ stderr is actually a ïżœle â in fact, on Unix systems mostdevices are represented as ïżœles
âȘ Try ls /dev to view the devices on your machine(includes macOS)
âȘ So if we want to write data to a ïżœle, just pass it in:âȘ fprintf(file, "My name is: %s", "Bob");
fprintf
CS 326: Operating Systems 18
![Page 19: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/19.jpg)
âȘ Pipes: The Basics
âȘ I/O StreamsâȘ Piping Data
Todayâs Agenda
CS 326: Operating Systems 19
![Page 20: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/20.jpg)
âȘ Now back to pipes: we can create them with the pipe()function
âȘ Returns a set of ïżœle descriptors: the input andoutput sides of the pipe
âȘ Pipes arenât very useful if they arenât connected toanything, though
âȘ We can do this by fork() ing another process
The pipe function
CS 326: Operating Systems 20
![Page 21: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/21.jpg)
âȘ After calling fork() , both processes have a copy ofthe pipe ïżœle descriptors
âȘ Pipes only operate in one direction, though, so weneed to close the appropriate ends of the pipe
âȘ You can think of a forked() pipe as one with fourends: two input and output ends each
âȘ We eliminate the ends we donât need to control thedirection of data ïżœow
âȘ Amazing ASCII art drawing: >---<
Piping to Another Process
CS 326: Operating Systems 21
![Page 22: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/22.jpg)
âȘ To control data ïżœow through the pipe, we close theends we wonât use
âȘ For example:âȘ Child process closes FD 0 and reads from FD 1
âȘ Parent process closes FD 1 and writes to FD 0
Controlling Flow
CS 326: Operating Systems 22
![Page 23: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/23.jpg)
âȘ You may be wondering: what good are pipes when wehave to start all the cooperating processes?
âȘ Thereâs actually another option: FIFOs, aka namedpipes
âȘ Create with the mkfifo command, then open as youwould a regular ïżœle descriptor
Async Process Creation
CS 326: Operating Systems 23
![Page 24: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/24.jpg)
âȘ dup2 allows us to redirect streamsâȘ int dup2(int fildes, int fildes2);
âȘ Letâs say we want to make our standard output streamgo through the pipe we just created
âȘ Weâll do:âȘ dup2(fd[0], STDOUT_FILENO);
âȘ This also deallocates (closes) the second fd (ïżœldes2)âȘ You wonât see text printing directly to your terminal
Redirecting Streams: dup2
CS 326: Operating Systems 24
![Page 25: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/25.jpg)
âȘ We can also redirect to a ïżœle instead:âȘ int output = open("output.txt",
O_CREAT | O_WRONLY | O_TRUNC, 0666);
âȘ dup2(output, STDOUT_FILENO);
âȘ Cool, right? Ok, letâs try this outâŠ
Redirecting to a File
CS 326: Operating Systems 25
![Page 26: Pipes - cs.usfca.edu](https://reader031.vdocument.in/reader031/viewer/2022012920/61c8990cdc10713fc52e7231/html5/thumbnails/26.jpg)
âȘ pipe.c
âȘ io-redir.câȘ Shell commands
Demos
CS 326: Operating Systems 26