chapter 7 unix and linux. 2 outline overview processes in unix memory management in unix i/o in unix...
Post on 02-Jan-2016
291 Views
Preview:
TRANSCRIPT
2
Outline
• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
3
Overview• Why some programmers like UNIX better than
Windows?– Although GUI may be easy for beginners, they provide
little flexibility and no insight into how the system works!– UNIX has GUI, too!
• History of UNIX– UNICS in Bell Labs– PDP-11 UNIX– Portable UNIX, portable C compiler– Berkeley UNIX (BSD)– UNIX standards: POSIX from IEEE– MINIX and Linux
4
UNIX Goals (Philosophy)
• Designed by programmers, for programmers
• Simple, elegant and consistent
• Power and flexibility– A small number of basic elements– One program should do just one thing– An infinite combination to suit the applications
• No useless redundancy– Simple interface
5
Interfaces to UNIX
Hardware(CPU, memory, disks, terminals, etc)
UNIX operating system(process management, memory management,
the file system, I/O, etc)
Standard library(e.g. open, close, read, write, fork, etc)
Standards utility programs(shell, editors, compilers, etc)
Users
User interface
Library interface
System call interface
User mode
Kernel mode
6
The UNIX Shell: A Command Line Interface• Wait for a command line• Extract the first word from the command line, use
it as program name, run the program• Pass arguments• Flags: argument controlling the operation or
specify an optional value• Wild cards and magic characters• Standard input/output• Filter and pipe symbol
– grep ter *.t | sort | head –20 | tail –5 > foo
• Background execution and shell scripts
7
UNIX Utility Programs
• File and directory manipulation commands: cp, mv
• Filters: grep, head, sort, tail
• Program development tools: cc
• Text processing: vi, edit
• System administration: ps
• Miscellaneous: time
8
Structure of UNIX KernelSystem calls Interrupts and traps
Terminal handling SocketsFile
namingMapping
Page faults Signal
handling
Process creation
and termination
Raw tty
Cooked tyy
Network protocols
File systems
Virtual memory
Line disciplines
RoutingBuffer cache
Page cache Process scheduling
Character devicesNetwork device drivers
Disk device drivers Process dispatching
Hardware
9
Outline
• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
10
Basic Concepts
• UNIX is a multiprogramming system– User processes– Daemons
• Create new processes by system call fork()– Parent process and child process– PID, parent process gets child PID from fork()
• Pipes between two processes– Shell pipelines are implemented using pipes
• Signals: soft interrupt
11
Process Management System Calls in UNIX pid=fork() Create a child process identical to the parent
pid=waitpid(pid, &statloc, opts) Wait for a child to terminate
s=execve(name, argv, envp) Replace a process’ core image
exit(status) Terminate process execution and return status
s=sigaction(sig, &act, &oldact) Define action to take on signals
s=sigreturn(&context) Return from a signal
s=sigprocmask(how, &set, &old) Examine or change the signal mask
s=sigpending(set) Get the set of blocked signals
s=sigsuspend(sigmask) Replace the signal mask and suspend the process
s=kil(pid, sig) Send a signal to a process
residual=alarm(seconds) Sent the alarm clock
s=pause() Suspend the caller until the next signal
12
Thread Management
• Not every UNIX has threads package– Threads package becomes popular
• Threads can be implemented in either user space or kernel– POSIX does not specify
13
Thread Management System CallsThread Call Description
pthread_create Create a new thread in the caller’s address space
pthread_exit Terminate the calling thread
pthread_join Wait for a thread to terminate
pthread_mutex_init Create a new mutex
pthread_mutex_destroy Destroy a mutex
pthread_mutex_lock Lock a mutex
pthread_mutex_unlock Unlock a mutex
pthread_cond_init Create a condition variable
pthread_cond_destroy Destroy a condition variable
pthread_cond_wait Wait on a condition variable
pthread_cond_signal Release one thread waiting on a condition variable
14
Implementation of Processes• Two key data structures for processes• Process table: always in main memory
– Scheduling parameters, memory image, signals, miscellaneous
– A process has one entry in the table
• User structure: in memory only when the process is in memory– Machine registers, system call state, file
descriptor table, accounting info, kernel stack– A block per process, adjacent to the stack
segment
15
Executing “ls” Typed to the Shell
sh sh sh
PID=501 PID=748 PID=748
Fork code Exec code
1. Fork call
2. New sh created
3. Exec call
4. Sh overlaid with ls
New process Same process
1. Allocate child’s process table entry2. Fill child’s entry from parent3. Allocate child’s stack and user area4. Fill child’s user area from parent5. Allocate PID for child6. Set up child to share parent’s text7. Copy page tables for data and stack8. Set up sharing of open files9. Copy parent’s registers to child
1. Find the executable program2. Verify the execute permission3. Read and verify the header4. Copy arguments, environ to kernel5. Free the old address space6. Allocate new address space7. Copy arguments, environ to stack8. Reset signals9. Initialize registers
16
Threads in UNIX
• Threads in user space: a user space library
• Threads in kernel: may lead to many problems– How to maintain the correct traditional UNIX
semantics?– File I/O– Signal handling– All solutions to these problems cause
something to break somewhere
17
Threads in Linux
• System call clone– pid=clone(func, stack_ptr, sharing_flgs, arg)
• Sharing_flgs determine whether the thread is in current process or in a new process– In the same process: changes visible to other
threads
• The new thread executes func
• The new thread has its own stack
18
Scheduling in UNIX
• A two level algorithm– Low-level algorithm: pick the process to run– High-level algorithm: move processes between
memory and disk
• Low-level algorithm uses multiple queues– Priority=CPU_usage+nice+base
19
Multilevel Queue Structure
Highest …
Process waiting in
kernel mode
-4 Waiting for disk I/O
-3 Waiting for disk buffer
-2 Waiting for terminal input
-1 Waiting for terminal output
0 Waiting for child to exist
0 User priority 0 Process waiting in
user mode1 User priority 1
2 User priority 2
3 User priority 3
Lowest …
20
Scheduling in Linux
• Scheduling is based on threads
• Three classes of threads– Real-item FIFO, non-preemptive, highest
priority– Real-time round robin, preemptive, high priority– Timesharing
• Scheduling based on priority and quantum
21
Booting UNIX
• Read & run first sector of the boot disk– MBR has a small program (up to 512 bytes)– Load program boot
• Boot reads the root directory of boot device• Boot reads OS kernel, boot ends• Kernel starts, set parameters & environment• Initialization, build kernel data structure• System auto-configuration, detect devices• Load device drivers• Start process 0
22
Process 0
• Continue initialization
• Program the real-time clock
• Mount the root file system
• Create init (process 1) and page daemon (process 2)
23
Process 1
• Check flags of single-user/multi-user mode
• Single user mode– Fork off a process running shell– Wait for that process to exit
• Multi-user mode– Fork off a process
• Run system initialization shell script /etc/rc
– Read /etc/ttys, fork off processes for terminals• Run gtty
24
Sequence of Processes in Booting
Process 0
Page
daemo
n
Process 2ini
t
Process 1
gett
ylogin sh
cp
Login:
Terminal 0
Password:
Terminal 1 Terminal 2
% cp f1 f2
25
Loading Drivers
• Dynamically loading– One binary code can be used everywhere– Drivers are loaded dynamically, even through a
network
• Non-dynamically loading– Only the system administrator can make kernel
binary code– No one can insert any component into the
kernel
26
Outline
• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
27
Three Segments in UNIX Processes• Text segment: program’s executable code
– Read-only, size fixed after process is created
• Data segment: program’s variables, strings, and other data– Initialized data and un-initialized data (BSS)– Un-initialized global variables are in BSS, save
space– Size can change
• Stack segment
29
Memory Management System Calls
• POSIX does not specify
• Common system calls
System call description
s=brk(addr) Change data segment size
a=mmap(addr, len, prot, flags, fd, offset) Map a file in
s=unmap(addr, len) Unmap a file
30
Swapping
• High level scheduling: swapper
• What cause swapping?– Fork() needs memory for a child process– Brk() needs to expand a data segment– Stack needs to be expanded
• Which process will be swapped out?– The ones blocked/waiting for I/O– The ones with high (priority + residence time)
• The ones consuming much CPU/staying long in mem
31
Swapping Back Processes
• Check processes on disk every few seconds– Find processes are ready– Select the one staying on disk longest
• If have enough free memory (easy swap), bring that process in
• Otherwise, swap out some processes in main memory
• No process is swapped out until it stays in memory for 2 seconds
32
Paging in UNIX• Do not use the working set model• Done by kernel & page daemon (process 2)• Never page out kernel and core map
– Core map records info about contents of the page frames
33
Page Replacement Algorithm
• Run every 250 msec by page daemon
• If insufficient free page frames, page daemon transfers pages to disk
• A modified version of the clock algorithm
34
Memory Management in Linux
• A three-level paging scheme– Directory, middle, page + offset
• Buddy algorithm
64
32
32
32
16
16
32
16
8
8
32
16
8
8
32
8
8
8
8
32
8448
8
32
8448
8
32
844
16
35
Outline
• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
36
Input/Output in UNIX
• Integrate devices into file system as special files– Each I/O device is assigned a path name
• Block special files and character special files
37
Networking and Sockets• Socket: interface to network• Types of networking supported by sockets
– Reliable connection-oriented byte stream– Reliable connection-oriented packet stream– Unreliable packet transmission
Sending process
Socket
Receiving process
User space
kernel space
38
I/O System Calls in UNIX
• POSIX specifies function calls for terminal
• Function call ioctl is used in many UNIXFunction call Description
s=cfsetospeed(&termios, speed) Set the output speed
s=cfsetispeed(&termios, speed) Set the input speed
s=cfgetospeed(&termios, speed) Get the output speed
s=cfgetispeed(&termios, speed) Get the input speed
s=tcsetattr(fd, opt, &termios) Set the attributes
s=tcgetattr(fd, &termios) Get the attributes
39
Outline
• Overview
• Processes in UNIX
• Memory management in UNIX
• I/O in UNIX
• UNIX file system
40
UNIX File• A sequence of bytes
– No distinction between ASCII, binary, or any other kinds of files
• Directories are stored as files
• Important directories in UNIX systemsDirectory Contents
bin Binary (executable) programs
dev Special files for I/O devices
etc Miscellaneous system files
lib Libraries
usr User directories
41
Mounting and Locking
• Only one file system (tree)– One disk is mounted in another disk’s file tree
• UNIX allows user to lock files– From single byte to an entire file– Shared locks and exclusive locks– A process specifies whether it wants to block if
the lock cannot be placed– Shared locked regions may overlap
42
System Calls About FilesSystem call Description
fd=create(name, mode) One way to create a new file
fd=open(file, how, …)Open a file for reading, writing or both
s=close(fd) Close an open file
n=read(fd, buffer, nbytes) Read data from a file into a buffer
n=write(fd, buffer, nbytes) Write data from a buffer into a file
position=lseek(fd, offset, whence) Move the file pointer
s=stat(name, &buf) Get a file’s status information
s=fstat(fd, &buf) Get a file’s status information
s=pipe(&fd[0]) Create a pipe
s=fcntl(fd, cmd, …) File locking and other operations
43
System Calls About DirectoriesSystem call Description
s=mkdir(path, mode) Create a new directory
s=rmdir(path) Remove a directory
s=link(oldpath, newpath) Create a link to an existing file
s=unlink(path) Unlink a file
s=chdir(path) Change the working directory
dir=opendir(path) Open a directory for reading
s=closedir(dir) Close a directory
dirent=readdir(dir) Read one directory entry
Rewinddir(dir)Rewind a directory so it can be reread
44
Disk Layout of Classical UNIX• Boot block: contain code to boot computer
• Superblock: contain critical info about the layout of the file system– # of i-nodes, # of disk blocks, the start of the list
of free disk blocks, …
• I-nodes
• Data blocksBoot block Super blk I-nodes Data blocks
45
Reading A File• System call n=read(fd, buffer, nbytes)• Start with file descriptor fd, find i-node
– Put pointer to the i-node in file descriptor?• Only one current position for one file, recorded in i-
node• Two processes open one file with different current
position
– Put current position info in file descriptor?• One file can have multiple current positions• P1 and P2 open f at the same time, P2 want to write
after P1 finishes• P2 cannot get the correct current position
47
Berkeley Fast File System
• Allow file names up to 255 characters
• Divide the disk up into cylinder groups– Each with its own superblock, i-nodes and data
blocks– Whenever possible, blocks are allocated in the
cylinder group containing the i-node
• Two block sizes– Large files use large block size
top related