Operating SystemsIntroduction
Stephan Sigg
Distributed and Ubiquitous SystemsTechnische Universitat Braunschweig
November 9, 2010
Stephan Sigg Operating Systems 1/66
Overview and Structure
Introduction to operating systemsHistoryArchitectures
ProcessesProcesses, threads and interprocess communicationProcess scheduling and deadlocks
Memory managementPagingSegmentation
Filesystems
Security and Protection
Distributed systems
Cryptography
Stephan Sigg Operating Systems 2/66
OrganisationOrganisation
Participate !
Participation during lecture and exercises to improvecomprehension
Opportunities:
Interactive lecture and exercise: Participate!Solve the exercises in advance
(you will understand the problems better and detect thingsthat you did not understand)
Present your solutionPresent a short Wrap-Up (ca. 5 min.) of the last lectureParticipate in the summary of the current lecture (at the endof the lecture)
Stephan Sigg Operating Systems 3/66
OutlineIntroduction
1 Definition of operating systems
2 A condensed history of operating systems
3 Operating system basicsThe computer systemOperating system structureUser mode and kernel modeArchitectures of operating systems
4 Summary
Stephan Sigg Operating Systems 4/66
What is an operating system?Definition of operating systems
What is an operating system?
Stephan Sigg Operating Systems 5/66
What is an operating system?Definition of operating systems
Operating system
”Ein Computer ist, wenn er genau betrachtet wird, nur eineAnsammlung von Plastik und Metall, das zur Leitung von Strombenotigt wird. Dieser ’Industriemull’ kann somit nichtausschliesslich das sein, was wir unter einem modernen Computerverstehen, etwas, das dem Computer ’Leben’ einhaucht und ihn zudem Werkzeug unseres Jahrhunderts macht. Es ist dasBetriebssystem, das die Kontrolle uber das Plastik und Metall(Hardware) ubernimmt und anderen Softwareprogrammen (Excel,Word, . . . ) eine standardisierte Arbeitsplattform (Windows,Unix, OS/2) schafft.”a
aEwert et al., Literatur zu ’Freehand 10’
Stephan Sigg Operating Systems 6/66
What is an operating system?Definition of operating systems
Definition
Programmbundel, das die Bedienung eines Computers ermoglicht.a
aUniversalworterbuch Rechtschreibung
Definition
Summe derjenigen Programme, die als residenter Teil einerEDV-Anlage fur den Betrieb der Anlage und fur die Ausfuhrungder Anwenderprogramme erforderlich ista
aLexikon der Informatik
Stephan Sigg Operating Systems 7/66
What is an operating system?Definition of operating systems
Definition
Die Programme eines digitalen Rechensystems, die zusammen mitden Eigenschaften der Rechenanlage die Grundlage der moglichenBetriebsarten des digitalen Rechensystems bilden und insbsonderedie Abwicklung von Programmen steuern und uberwachen.a
aDIN 44300
Definition
Ein Programm das als Vermittler zwischen Rechnerbenutzer undRechnerhardware fungiert. Der Sinn des Betriebssystems ist eineUmgebung bereitzustellen, in der Benutzer bequem und effizientProgramme ausfuhren konnen.a
aA. Silberschatz et al. Operating system concepts, Wiley, 2004.
Stephan Sigg Operating Systems 8/66
What is an operating system?Definition of operating systems
Definition
Eine Softwareschicht, die alle Teile des Systems verwaltet und demBenutzer eine Schnittstelle oder virtuelle Maschine anbietet, dieleichter zu verstehen und zu programmieren ista
aA. Tanenbaum, Modern operating systems, 3rd edition, Prentice Hall,2008.
Stephan Sigg Operating Systems 9/66
What is an operating system?Definition of operating systems
Functions of an operating system
An operating system basically performs a set of unrelatedfunctions:
Extending the machine
Managing resources
Providing an environment where multiple programs canexecute
Stephan Sigg Operating Systems 10/66
What is an operating system?The operating system as an extended machine
Issue: The architecture of most computers at the machinelevel is primitive and hard to program
Solution: The operating system provides an abstraction layerto ease interaction with the hardware
Stephan Sigg Operating Systems 11/66
What is an operating system?The operating system as an extended machine
Example
The floppy disk
Basic commands are read and writeRequire 13 parameters packed into 9 bytesAddress of the disk block, number of sectors per track,recording mode on physical medium, ...Controller chip returns 23 status end error fields (7 bytes)
Programmer must constantly be aware if motor is on or offIf off: switch motor on (long startup delay)Switch off frequently to prevent the disk from wearing out
.Stephan Sigg Operating Systems 12/66
What is an operating system?The operating system as an extended machine
The operating system manages all pieces of a complex system
Provides an orderly and controlled allocation of the processor,memories, and I/O devices among the various programs
Stephan Sigg Operating Systems 13/66
What is an operating system?The operating system as an extended machine
Example
Imagine several programs accessing the same physical printersimultaneously.
The output might easily consist of several lines from the firstprint job, then several lines from the next and so on
The operating system has to schedule access to the hardwareamong programs
.
Stephan Sigg Operating Systems 14/66
OutlineIntroduction
1 Definition of operating systems
2 A condensed history of operating systems
3 Operating system basicsThe computer systemOperating system structureUser mode and kernel modeArchitectures of operating systems
4 Summary
Stephan Sigg Operating Systems 15/66
A condensed history of operating systemsEarly punchcard systems
Punchcards/Lochkarten
Since 1725 Punchcards have been utilised for the stearing ofweaving machines
1890 Herman Hollerith utilises punchcards for a census ofpopulation (Volkszahlung)
until 1970s Utilised as storage media
.Stephan Sigg Operating Systems 16/66
A condensed history of operating systemsFirst universal electronic calculators
ENIAC, 1946US armyCalculation of ballistictables
Size: 10m x 17m x 2.7m
Weight: 27 tons
Performance: 174 kW (> 17000tubes)
Price: 468.000 $
Comp.Power: 500 Additions persecond
Stephan Sigg Operating Systems 17/66
A condensed history of operating systemsFirst universal electronic calculators
Stephan Sigg Operating Systems 18/66
A condensed history of operating systemsSince 1945: Serial operation
Programming
Machine codeInput via punchcardsOutput via printerErrors indicated by control lights
Minimum CPU utilisation
Much time spent for slow I/Odevices (punchcard reader, printer)
Scheduling of CPU resources onpaper calendars
Waste of resources due to too widetime windows or abort through error
First system software: reusableprogram libraries
Device drivers, Debugger
Stephan Sigg Operating Systems 19/66
A condensed history of operating systemsSince 1955: Simple batch systems
Reduce the frequency of manualoperation
First operating systems: residentmonitors
Interpretation of Job controlinstructionsLoad and execution of programsDevice triggering
$FTN
$JOB
$LOAD
$RUN
$END
Fortran code
Control cards (new)
Input data
Stephan Sigg Operating Systems 20/66
A condensed history of operating systemsSince 1955: Simple batch systems
The monitor stayed in memory whilecontinuously executing applicationprograms
Problems due to application programerrors
Program does not terminateProgram writes to memory area ofmonitorProgram accesses the card readerand interpretes control input as data
Device driver
sequential
job control
Control
interpreter
language
Application
programarea
Memory
Mo
nit
or
Stephan Sigg Operating Systems 21/66
A condensed history of operating systemsSince 1955: Simple batch systems
Solutions
Timer provides interruptsTraps for erroneous programs
Fence register for primitivememory protection
Supervisor mode (PriviligierterArbeitsmodus) of CPU
Deactivate fence registerIn/Out
Application
programarea
Memory
Interrupt
mechanism
Device driver
sequential
job control
Control
interpreter
language
UnusedM
on
ito
r
02FF
0300
0000
1FFF
0300
registerFence
Timer
Stephan Sigg Operating Systems 22/66
A condensed history of operating systemsThe input-/Output- bottleneck
Problem: CPU works faster than card reader
Processing resources wasted by active waiting
Solution 1: Off-line processing
With the help of tape drivesParallel input-/output via several satellite computers
1
1A. Tanenbaum, Modern operating systems, 3rd edition, Prentice Hall, 2008.
Stephan Sigg Operating Systems 23/66
A condensed history of operating systemsThe input-/Output- bottleneck
Problem: CPU works faster than card reader
Processing resources wasted by active waiting
Solution 2: SpoolingWith the help of disc drives
Random access (Wahlfreier Zugriff) and Direct MemoryAccess (DMA)Calculations and In-/Output then overlappingRules for CPU allocation
Card reader
Disk
CPU
Printer
Waiting jobs Result data
.
Stephan Sigg Operating Systems 24/66
A condensed history of operating systemsSince 1965: Multiple programs simultaneously
Despite spooling a single program utilises the CPU notefficiently
CPU bursts and I/O bursts in which the CPU is idle alternate
With multitasking, the CPU processes several jobs in parallel
CPU
CPU CPU
CPU CPU
CPUCPU
CPUI/O
I/O
I/O I/O
I/O
I/O
Multiprocessing
Program A
Program B
Program A and B
CPU CPUI/O I/O
Single processing
Program A
.
Stephan Sigg Operating Systems 25/66
A condensed history of operating systemsSince 1965: Multiple programs simultaneously
The operating system is gaining complexity
Parallel I/O activities (Nebenlaufigkeit)
Administration of the memory for various programs
Internal administration of executing processes
Scheduling of CPU resources
Multi-user operation (security and accounting)
Stephan Sigg Operating Systems 26/66
A condensed history of operating systemsSince 1965: Multiple programs simultaneously
Memory management
Allocate free memory dynamically
Memory protection
Simple fence register not sufficientto isolate single programsSolution: Simple MemoryManagement Unit (MMU)
Process management
Every process has its own contextExchange context at process change
0000
FFFF
systemOperating
Program B
Program A
Unused
Memory CPU and MMU
Lo
Hi
Context BContext A
Lo
SP
IP
Hi
SP
IP
Lo
Hi
4000
5000
4000
430A
4EEO
5000
F000
62E2
E340
(text, data, bss, stack)
(text, data, bss, stack)
EFFF
3FFF
4000
4FFF
5000
− −
− −
Bounds register
pointerinstruction
stackpointer
5000
Stephan Sigg Operating Systems 27/66
A condensed history of operating systemsDialogue driven
New In/Output devices enableinteractive software
Keyboard, Monitor, later mouse
Time-sharing operation
Improved response-time forinteractive usersTiming interrupts suppress longrunning processes
Interactive software development dueto system programs
Editor, shell, compiler, debugger
Disks and file systems allow access ofprograms and data at any time
First Apple Macintosh(January 24, 1984)
Stephan Sigg Operating Systems 28/66
A condensed history of operating systemsToday’s operating systems
Microsoft Windows
Evolved from MS-DOS
Proprietary system
Windows Vista:Approximately 50 Mio LOC
Hardware
Kern
el m
od
eHardware Abstraction Layer (HAL)
Object Manager
Executive Services
Executi
ve
User
mod
e
Integral subsystems Environment subsystems
I/OManager
SecurityReferenceMonitor
IPCManager
VirtualMemoryManager(VMM)
ProcessManager
PnPManager
PowerManager
WindowManager
GDI
Work-stationservice
Serverservice
SecurityWin32 POSIX OS/2
Win32Application
POSIXApplication
OS/2Application
Kernel mode drivers Microkernel
Hardware
Stephan Sigg Operating Systems 29/66
A condensed history of operating systemsToday’s operating systems
Minix
Developed by Andrew S.Tanenbaum for teachingpurposes
Less than 4000 LOC
Device driver as programs inuser space
Hardware
Kernel mode
User
mod
e
Device processesDisk TTY Net
Kernel
Hardware
Print Other
Clock task System task
Server processesFile PM Reinc ... Other
User processesShell Make User ... Other
Stephan Sigg Operating Systems 30/66
A condensed history of operating systemsToday’s operating systems
Berkeley Software Distribution (BSD)
Developed at UC Berkeley in 1977
Basis for many further operatingsystems
FreeBSDOpenBSDEarly SunOSMac OS X
Stephan Sigg Operating Systems 31/66
A condensed history of operating systemsToday’s operating systems
Linux
System composed ofapplications, Services,Kernel, Hardware controllers
Kernel presents virtualinterface to user processes
Pure OS-core ca. 1 Mio LOC
Therefore suited forembedded systems
Hardware
Kernel
User level
CPU
Hardware controlersSystem memory terminal
Process Process Process Process process
NICdisk
signals system calls
scheduler
file systemnetwork
driversinterrupts
device
memorytraps/faults
virtual memory
network
protocolsblock
drivers
chardevicedrivers
Stephan Sigg Operating Systems 32/66
A condensed history of operating systemsToday’s operating systems
Mac OS X
Distributed since 2001 byApple for Macintoshcomputers
Successor of NeXTStep andMac OS 9
Based on Darvin (which isitself based on BSD)
Hardware
Cor
e O
S (D
arw
in)
Gui (Aqua)
HardwareM
ach
Ke
rne
l ('
xn
u')
I/O Kit Drivers
BS
D
File SystemsNetworking
POSIXNKE
System utilities
Java
JRE
JVM
AP
I
Application services
Core servicesCore foundation Core services non-GUI-api
Carbon
CoreCarbon
Quartz OpenGL PrintCore
QuickTimeBSD Classic Cocoa
Stephan Sigg Operating Systems 33/66
A condensed history of operating systemsToday’s operating systems
QNX
Developed by Gordon Bell and DanDodge in 1982 for Intel 8088
Real-time operating system
Tiny OS-core holds only CPUscheduler, inter processcommunication, interrupt handlingand timer
Modular system
Mainly embedded systems(Automobile, Router)
Proprietary system
Stephan Sigg Operating Systems 34/66
OutlineIntroduction
1 Definition of operating systems
2 A condensed history of operating systems
3 Operating system basicsThe computer systemOperating system structureUser mode and kernel modeArchitectures of operating systems
4 Summary
Stephan Sigg Operating Systems 35/66
Operating system basicsIntroduction
To understand the operation of operating systems, somebackground in
I/OComputer ArchitectureStorage conceptsNetworking
is required
Stephan Sigg Operating Systems 36/66
Operating system basicsThe computer system
Computer system operation
A general purpose computer system consists of
One or more CPUsA number of device controllersShared memoryconnected through a common bus
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 37/66
Operating system basicsThe computer system
Computer system operation
CPU and device controllers can execute concurrently andcompete for the memory
A memory controller synchronises the memory access
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 38/66
Operating system basicsThe computer system
Computer system operation
When started, a computer requires an initial program to run
bootstrap programSimple ’firmware’Typically stored in read-only memory (ROM) or Electricallyerasable programmable read-only memory (EEPROM)Initialises all aspects of the system, e.g.
CPU registersDevice controllersMemory contents
Loads and starts operating system
Loads into memory the operating system kernel
Starts the first process of the system
Waits for events to occur
Stephan Sigg Operating Systems 39/66
Operating system basicsThe computer system
Storage structure
The CPU can load instructions only from memory
Any executable programs have to be loaded into memory
Memory provides an array of words
Each word has its own address
Interaction through sequence of load or store instructions
Load moves a word from memory to register within the CPUStore moves the content of a register to main memory
Load
Store
Load executableprograms into
memory
Stephan Sigg Operating Systems 40/66
Operating system basicsThe computer system
Storage structure
A typical von Neumann-execution cycle:1 Fetch an instruction from memory2 Store that instruction into the instruction register3 Decode instruction (probably fetch operands from memory)4 Execution5 Result may be written back to memory
1: Fetch
5: Write back
2: Store to IR
3: D
ecod
e
3: execute
Stephan Sigg Operating Systems 41/66
Operating system basicsThe computer system
22Hennessey and Patterson, Computer Architecture: A quantitative
approach, Morgan Kaufmann, 2002Stephan Sigg Operating Systems 42/66
Operating system basicsThe computer system
33Hennessey and Patterson, Computer Architecture: A quantitative
approach, Morgan Kaufmann, 2002Stephan Sigg Operating Systems 43/66
Operating system basicsThe computer system
Storage structure
Since access to main memory is fast, we naturally want allprograms and data to reside there
This is typically not possible, since1 Main memory is usually too small2 Main memory is a volatile storage device that loses its content
after power is turned off
Therefore: Secondary storage is required as extension
?Stephan Sigg Operating Systems 44/66
Operating system basicsThe computer system
Semi
cond
uctor
mem
ory Registers
Cache
Main memory
Electronic disk
Magnetic disk
Optical disk
Magnetic tapes
fast
slowch
eap
expensiv
e
cost
access
expensiv
e
Stephan Sigg Operating Systems 45/66
Operating system basicsThe computer system
I/O structure
Large portion of OS-code is dedicated to managing I/O
Typically, multiple device controllers connected to CPU over acommon bus
Each device controller is in charge of a specific type of device
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 46/66
Operating system basicsThe computer system
I/O structure
A device controller maintains
Local buffer storageSome special purpose register
Moves data between devices and its local buffer storage
OS has device driver for each device controller
Driver presents a uniform inferface to the operating system
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 47/66
Operating system basicsThe computer system
I/O structure
To start an I/O operation
Driver loads appropriate registers within device controllerController examines their contents to obtain action to takeController starts data transfer from device to its local bufferWhen transfer is complete, controller informs the driverDevice driver then returns control to the operating system
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 48/66
Operating system basicsThe computer system
I/O structure
This form of interrupt-driven operation is not feasible forgreater amounts of data
Solution: Direct Memory Access (DMA)
Device controller transfers entire block of data directly to orfrom its own buffer storage to memoryCPU available during this time
CPUdisk
controllerUSB
controllergraphicsadapter
memory
Stephan Sigg Operating Systems 49/66
Operating system basicsOperating system structure
Memory management
OS provides environment within whichprograms are executed
Execution quasi parallel
OS keeps several programs inmemory simultaneouslySchedules these for their CPU timeWhen programs are idle (e.g. due toI/O operation), they are temporaryremoved from memoryOther programs can then beexecuted instead
MemoryOperating
system
Program 1
Program 2
Program 3
Program 4
Stephan Sigg Operating Systems 50/66
Operating system basicsOperating system structure
Operating system operations
In order to ensure proper execution of the operating systemwe distinguish between
Kernel modeUser mode
For each operation a bit is added to the hardware of thecomputer that indicates user mode or kernel mode
Potentially harmful operations executed only in kernel mode
Usermode
Kernelmode
User process Callsystem call
Executesystem call
Return fromsystem call
mod
e b
it = 0
mode b
it =
1Stephan Sigg Operating Systems 51/66
User mode and kernel modeCPU-modi
Separation of operating system process and application processes
Protection of the operating system from software failures
Trend: Outplacement of parts of the operating system (e.g.driver)
Switch between user mode and kernel mode
CHM (Change Mode) Operation (only privileged processes)
Events: Interrupts or Traps
Usermode
Kernelmode
User process Callsystem call
Executesystem call
Return fromsystem call
mod
e b
it = 0
mode b
it =
1Stephan Sigg Operating Systems 52/66
User mode and kernel modeCPU-modi
User mode
Processor processes instructionsof application programs
Instruction set of processorrestricted to subset(unprivileged instruction set)
Hardware access restricted(hardware protection)
Direct access to executingprograms forbidden
Usermode User process Call
system callReturn fromsystem call
mode bit =
0 mod
e bi
t =
1Stephan Sigg Operating Systems 53/66
User mode and kernel modeCPU-modi
Kernel mode
Processor processes instructions ofthe operating system
Complete instruction set available(privileged instruction set)
Direct, unrestricted access tohardware
Manipulation of internal data fromrunning processes possible
Errors compromise system stability
Kernelmode Execute
system call
mode bit =
0
mod
e bi
t =
1Stephan Sigg Operating Systems 54/66
Entry points to an operating systemTraps
System calls, supervisor calls, software traps
Synchronous to running application (e.g.count=read(file,buffer,size))
Parameter forwarded via register, stack or parameter block
Hardware traps
Synchronous to running applications
OS forwards errors to application responsible for it
Stephan Sigg Operating Systems 55/66
Entry points to an operating systemInterrupts
Hardware interrupts
Asynchronous with running application
E.g. Receiving of bit-sequence on modem
Software interrupts
Asynchronous to running application
Often created internally to shift sophisticated calculation tolower priority processes
Asynchronous interrupts can interrupt system calls and otheroperating system processes
Stephan Sigg Operating Systems 56/66
Entry points to an operating systemInterrupts and I/O
Program flow typically not reproducible
Interrupts depend on environmental conditions
Processes should not be implemented under timingassumptions
I/O-devices can work in parallel with the CPU
Goal: Keep all devices and CPU busy most of the time
Typical strategy: Higher priority for I/O-intensive tasks
Stephan Sigg Operating Systems 57/66
Architectures of operating systemsOperating system architectures
We distinguish between
Monolithic architecturesLayered architectures
Hardware
Hardware
Stephan Sigg Operating Systems 58/66
Architectures of operating systemsLayered architectures
Layers to control complexity
Increased flexibility
Often overhead to control interoperation among layers
Hardware
Syst
em
prog
ram
sH
ardw
are
Physical devices
Microarchitecture
Machine language
Operating system
Compilers Editors Command interpreter
prog
ram
sAp
plica
tion
Banking system
AirlineReservation
Web browser
Stephan Sigg Operating Systems 59/66
Architectures of operating systemsLayered architectures
OS consists of several layers
Higher layers provide new functionality
Modular structure improves error detection and qualitymanagement
Strict satisfaction of layers increases overhead
Hardware
Syst
em
prog
ram
sHa
rdwa
re
Physical devices
Microarchitecture
Machine language
Operating system
Compilers Editors Command interpreter
progra
msAp
plicati
on
Banking system
AirlineReservation
Web browser
Stephan Sigg Operating Systems 60/66
Architectures of operating systemsLayered architectures
Layers to control complexity
Increased flexibility
Often overhead to control interoperation among layers
Hardware
Service procedures
Main procedure
Utility procedures
Stephan Sigg Operating Systems 61/66
Architectures of operating systemsLayered architectures
OS as set of procedures and functions without structure
Every procedure can call arbitrary other procedures
Functions that realise subfunction of OS are grouped
Hardware
Service procedures
Main procedure
Utility procedures
Stephan Sigg Operating Systems 62/66
OutlineSummary
1 Definition of operating systems
2 A condensed history of operating systems
3 Operating system basicsThe computer systemOperating system structureUser mode and kernel modeArchitectures of operating systems
4 Summary
Stephan Sigg Operating Systems 63/66
Introduction to operating systemsQuestions, discussion, remarks
Questions?
Stephan Sigg Operating Systems 64/66
OrganisationOrganisation
Participate !
Participation during lecture and exercises to improvecomprehension
Opportunities:
Interactive lecture and exercise: Participate!Solve the exercises in advance
(you will understand the problems better and detect thingsthat you did not understand)
Present your solutionPresent a short Wrap-Up (ca. 5 min.) of the last lectureParticipate in the summary of the current lecture (at the endof the lecture)
Stephan Sigg Operating Systems 65/66
LiteratureRecommended literature
A. Tanenbaum, Moderne Betriebssysteme, 2nd edition,Prentice Hall, 2009.
A. Tanenbaum, Modern operating systems, 3rd edition,Prentice Hall, 2008.
A. Silberschatz et al. Operating system concepts, Wiley, 2004.
W. Stallings, Operating systems, 6th edition, Prentice Hall,2008.
Stephan Sigg Operating Systems 66/66