operating systems part – m13 reentrance coroutines file systems...

Operating Systems Part – M13 Reentrance Coroutines File Systems I Florina Ciorba 10 November 2015

Projects Info

•  Odroids vs Raspberry Pi: https://www.youtube.com/watch?v=vSgka1awkXs

•  iRobot’s Roomba: http://www.irobot.com

Operating Systems Concepts 2 Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15

Today’s Overview

•  Concurrent programming •  Reentrance (in Unix libraries)

•  Concurrency abstractions •  Coroutines, COBEGIN/COEND, Future, fork/join •  Coordination via tuplespace

•  File systems •  Four layer model


Reentrance

A wide spread mistake of concurrent programming: “non-reentrant code” Definition A procedure is reentrant if it returns the same result, independent of the number of (simultaneous) calls •  Non-self-modifying code •  Global variables as most common sources of errors (non-reentrance) •  Java has no global variables (though class variables)


Non-Reentrance: Example

int t; // this is a global variable

void swap(int *x, int *y){ t = *x; *x = *y;

// hardware interrupt might invoke isr() here! *y = t;}

void isr(){ int x = 1, y = 2; swap(&x, &y);}


Non-Reentrance with Thread Safety: Example

#include <threads.h>thread_local int t; // this is a thread-local variable

void swap(int *x, int *y){ t = *x; *x = *y;

// hardware interrupt might invoke isr() here! *y = t;}



Reentrance without Thread Safety: Example

int t; // this is a global variable

void swap(int *x, int *y){ int s; s = t; // save global variable t = *x; *x = *y; // hardware interrupt might invoke isr() here! *y = t; t = s; // restore global variable}



Reentrance (continued)

Non-reentrant code can be found in surprisingly many places •  Library code is often not “multi thread-safe” (Introduction of LWP in SunOS required substantial rewriting) •  Result values

errno in C A call of a system function one one thread may overwrite the value previously set by a call on a different thread

•  Statistical result buffer

asctime() in C Return value points to a statically allocated string which might be overwritten by subsequent calls to any of the date and time functions

Lesson: Avoid global variables! Or protect access to such code/data via mutexes


Coroutines = “Cooperative Routines”

Previously seen: pseudo parallelism Subroutines •  Implicit context switch

Example: via timer interrupt (preemptive scheduling) Processes do not need to care to give up CPU

→ Synchronous control flow •  For semaphores / monitors

•  Context switching remains implicit •  Can also be left out, if not necessary

Coroutines •  Responsible for explicit context switching (on high level language)

Processes do need to care to give up CPU → Asynchronous control flow

•  Such that one can implement semaphores / monitors Operating Systems Concepts 9 Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15

Coroutines (Continued)

•  Coroutine that implements a “generator” (using fictitious C) int coroutine Generate123() { yield 1; /* Execution begins here upon first call to Generate123 */ yield 2; /* Execution continues here upon 1st "resume Generate123" */ yield 3; /* Execution continues here upon 2nd "resume Generate123" */ }

main() {printf("%d\n", Generate123()); /* prints "1" */printf("%d\n", resume Generate123()); /* prints "2" */printf("%d\n", resume Generate123()); /* prints "3" */

}




subroutine subroutines calling subroutines subroutine calling coroutine calling (yielding to) subroutine


Modula-2 •  1977-1985 ETH by Niklaus Wirth •  As revision to Pascal and Modula •  Intended for the personal workstation Lilith Module •  Encapsulates a set of related subprograms and data structures •  Restricts their visibility from other portions of the program •  Strict scope control

•  Two parts •  Definition: uses qualified object export •  Implementation: contains working code internal to the module



Coroutine primitives in Modula-2 •  NEWPROCESS(mem, proc)

Parameters: – Memory area (for stack) and – Procedure name

•  TRANSFER(old, new) Saved control state in “old”, yields to “new”


Example: Modula-2 Coroutines

MODULE corout; FROM System IMPORT ADDRESS, NEWPROCESS, TRANSFER, PROCESS; FROM Storage IMPORT ALLOCATE; FROM InOut IMPORT WriteLn, WriteString;

CONST WsSize = 2048; VAR guten, tag: ADDRESS; coroutGuten, coroutTag, coroutMain: PROCESS; count: CARDINAL; PROCEDURE procGuten; BEGIN

LOOP WriteString("Guten");TRANSFER(coroutGuten, coroutTag); END;

END procGuten; Operating Systems Concepts 14 Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15

Example: Modula-2 Coroutines (Continued)

PROCEDURE procTag; BEGIN

WHILE count > 0 DO WriteString(“Tag"); WriteLn; TRANSFER(coroutTag, coroutGuten); DEC(count);

END; TRANSFER(coroutTag, coroutMain); END procGuten;

BEGIN // main count := 5; ALLOCATE(guten, WsSize); ALLOCATE(tag, WsSize); NEWPROCESS(procGuten, guten, WsSize, coroutGuten); NEWPROCESS(procTag, tag, WsSize, coroutTag); TRANSFER(coroutMain, coroutGuten); WriteString("--ende."); WriteLn; END corout.


Fork and Join (Conway 1963, Dennis and Van Horn)

•  fork label Creates a second thread, which begins execution from address label

•  quit Terminates current thread

•  join var, label Performs the following in an un-interruptible manner var = var – 1; if (var == 0) goto label;

Unix has (roughly) fork / wait / exit


Fork and Join Example

int cnt1, cnt2;

L1: cnt1 = 2; fork L4; cnt2 = 2; fork L2; t1 = a + b; join cnt2, L3; quit; L2: t2 = c + d; join cnt2, L3; quit; L3: t4 = t1 * t2; join cnt1, L5; quit; L4: t3 = e / f; join cnt1, L5; quit; L5: t5 = t4 - t3;


Problems with Fork and Join

•  Code not very easy to read

•  Mixture of •  Thread parallelism •  Synchronization

•  Therefore, today there is a separation •  Thread (new, wait, exit) •  Synchronization (mutex, etc.)


Concurrency at Programming Level: COBEGIN | PARBEGIN and COEND | PAREND

Dijkstra’s 1968: PARBEGIN and PAREND •  COBEGIN S1 | S2 … | Sn COEND; •  Implicit synchronization

•  Arbitrary begin order •  Blocking end

Example BEGIN a = 1; COBEGIN b = a + a; | c = a + 1; COEND; d = b + c;END


a = 1;

d = b + c

b =

a +

a;

c =

a +

1;

Concurrency at Programming Level: COBEGIN (Cont.)

Arithmetic expression: (a+b) * (c+d) – (e/f) BEGIN COBEGIN COBEGIN t1 = a + b; | t2 = c + d; COEND; t4 = t1 * t2; | t3 = e / f; COEND; t5 = t4 – t3;END


a + b

e / f

*

c + d

−

Usefulness of COBEGIN and COEND

•  Dijkstra’s method for parallel | concurrent activity was too early for its time → Theoretical usefulness

•  In practice •  Setting up and management of processes was extensive •  Processes were doing little work during their lifetimes

•  Hardware developments

•  Approach is more practical

+ Communication, mutual exclusion, and synchronization → Languages adopt the approach (Concurrent Pascal, Pascal-FC, Pascal-Plus, Modula, Edison, Concurrent Euclid, occam and Ada)


“Future” Data Type

•  Proposed in 1977 by Baker and Hewitt •  Provides calculations with a “future data type” Same arithmetic expression: (a+b) * (c+d) – (e/f) BEGIN future t1 = a + b; future t2 = c + d; future t3 = e / f; t4 = t1 * t2; t5 = t4 – t3;END Implicit control of parallelism Languages: E, Joule, Oz, AmbientTalk, Alice ML, C++11 (std::future)


Implicit Concurrency, Example Linda

Linda is a “coordination language” developed in the 1980s by Gelernter and Carriero •  Processes are connected to a common “associative” memory

(virtual shared memory)

•  Process can write (out), read (rd), remove (in) “data tuples” (ordered collections of primitive data items) eval allows the calculation (and writing) of active tuples (that turn into passive tuples after evaluation)

•  Processes coordinate their calculations by putting (out) tuples into the storage

space and wait (in) until a result is readable

Synchronization is the side effect of the write/read operations


Process Coordination in Linda


Semaphores Linda-Style

Initialization (e.g., for a capacity of 2) out(`s’); out(`s’); wait() operation in(`s’); signal() operation out(`s’);


Producer-Consumer Linda-Style

•  Common name for a single “pipe” is known (pipeName) •  The count variable “count” enforces the proper order

Remark: Producer is only blocked when the entire Tuplespace is full


/* Producer */ /* Consumer */

count = 0 count = 0

while (true) { while (true) {

message = produce(); in(pipeName, count, &message)

out(pipeName, count, message) consume(message);

count = count + 1; count = count + 1;

} }

Rendez-Vous in Ada

•  Task concept: task ⟷ thread

•  Environment of a task controls its lifespan •  Only when all tasks have finished can the environment end •  Environment can be: package, procedure, task, etc.

•  Communication between tasks: “entries” •  Only possibility to exchange data

task type Encapsulated_Buffer_Task_Type is entry Insert (An_Item : in Item); entry Remove (An_Item : out Item); end Encapsulated_Buffer_Task_Type;


Rendez-Vous in Ada (Continued)

Operating Systems Concepts 28

Flow of a “rendez-vous” •  Caller wants to perform

entry, potentially must wait

•  Called task must perform accept

•  Invoking task will be blocked during rendez-vous

Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15

Rendez-Vous in Ada (Continued)

Evaluation •  Not shown here: many versions of accept

•  select, with timeouts, and guards

•  Clear regulation of memory access (in and out declaration)

•  Simple message passing as special case

•  Example of an “abstraction inversion”? (i.e., the abstraction is indeed powerful, but simple tasks are not complex or expensive)

•  Semaphores with rendez-vous are expensive to perform


END Concurrency → BEGIN File Systems

BEGIN Concurrency •  Concurrent programming

•  Reentrance (in Unix libraries)

•  Concurrency abstractions •  Coroutines, fork/join, COBEGIN/COEND, Future •  Coordination via tuplespace

END Concurrency


High Level Abstractions and Objectives

•  Processes are an abstraction for processors

•  Virtual memory is an abstraction for memory

•  File systems are an abstraction for disk (disk blocks)

•  Objectives for file systems •  Function •  Interfaces •  Access methods, file sharing, file locking, directory structures •  Protection


File Systems

File = logical address space of (persistent) data storage Different internal file structures •  Bit/Byte sequence

•  “Records” sequence (table entries, lines, data structures of fixed / variable length)

•  Complex data structures (records with substructure, e.g., tree)

OS can only support byte sequence → User space programs must implement the internal structure themselves

Operating Systems Concepts 32 Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15 11

File Structure Alternatives Byte sequence

Read or write a number of bytes Unstructured or linear Unix, Windows

Record sequence Fixed or variable length Read or write a number of records Not used: punch card days

Tree Records with keys Read, insert, delete a record

(typically using B-tree, sorted on key) Used in mainframes for commercial

data processing

…

… … …

File Types

•  ASCII (numeric, alphabetic, alphanumeric) •  Text: sequence of characters (lines and pages) •  Source: sequence of subroutines and functions (declarations and

executable statements) •  Binary files

•  Record •  Tree •  Object: sequence of bytes (in blocks) understandable by the linker •  Executable (Unix): series of code sections loaded by the loader in memory

for execution •  Header: magic number, size, entry point, flags •  Text •  Data •  Relocation bits •  Symbol table

•  Devices •  Everything else in the system


Common File Types 428 Chapter 10

"''' ·;· .. •·· ... · ... ·.·••·. :. ':·:•c •·· executable exe, com, bin machine-

or none language program object obj, o compiled, machine

language, not linked source code c, cc, java, pas, source code in various

asm, a languages batch bat, sh commands to the command

interpreter text txt, doc textual data, documents wo rdprocessor wp,tex, rtf, various wordcprocessor

doc formats library lib, a, so, dll libraries o.troutines for

.programmers print or view ps, pdf, jpg ASCII or binary file in a

format for printing or viewing

archive arc, zip, .tar 1·· related files grouped into .one file,sometimes com-pressed, for archiving or storage

multimedia mpeg, mov, rm, binary file containing mp3, avi audio or A/V information

Figure 10.2 Common file types.

end with a .doc extension. These extensions are not required, so a user may specify a file without the extension (to save typing), and the application will look for a file with the given name and the extension it expects. Because these extensions are not supported by the operating system, they can be considered as "hints" to the applications that operate on them.

Another example of the utility of file types comes from the TOPS-20 operating system. If the user tries to execute an object program whose source file has been modified (or edited) since the object file was produced, the source file will be recompiled automatically. This function ensures that the user always runs an up-to-date object file. Otherwise, the user could waste a significant amount of time executing the old object file. For this function to be possible, the operating system must be able to discriminate the source file from the object file, to check the time that each file was created or last modified, and to determine the language of the source program (in order to use the correct compiler).

Consider, too, the Mac OS X operating system. In this system, each file has a type, such as TEXT (for text file) or APPL (for application). Each file also has a creator attribute containing the name of the program that created it. This attribute is set by the operating system during the create() call, so its use is enforced and supported by the system. For instance, a file produced by a word processor has the word processor's name as its creator. When the user opens that file, by double-clicking the mouse on the icon representing the file,


Four Layers of File Systems

1  Logical file system File attributes, as well as linear memory area for data

2  Memory block level Mapping of logical blocks of data to hard disk blocks

3  Hard disk level Mapping of hard disk block to hard disk parameters (cylinders, sectors)

4  Hard disk access (I/O controller) Mapping of read/write instructions to bus signals


Logical File Systems: File Attributes

-  File name (only information in human readable format) -  Identifier (unique tag, identifies file within the system = non human readable

name) -  Type – discussed next -  Location (pointer to a device and to the location of the file on that device) -  Size (in bytes, words, or blocks) -  Owner (user, group) -  Protection (access rights permissions) -  Different times (file creation, last modification, last use)


Logical File Systems: File Attributes (Continued)

•  File attributes are stored either •  In directory, or •  Deferred to the file itself (as part of the data)

Do you know how these are handled by Unix, Windows or Apple’s HFS?


File Type Attribute

File type is required to enable starting the correct application •  Minimum one file type structure: executable •  Apple’s HFS:

•  “resource fork” of the file explicitly contains the type CODE (executable program), PICT (image), DITL (dialog box)

•  “data fork” contains program code or data

•  Windows: extension (*.exe, *.com, *.jpg)

•  Unix: each file is a sequence of 8-bit bytes Applications use in part extensions, otherwise “magic bytes” in the first few bytes of the file •  #!/bin/bash•  %!PS-Adobe-2.0 •  %PDF-1.4(see `man 1 file’ and `man 5 magic’)


Directory and Files

Operating Systems Concepts 39

F2F1 F4F3Fn

Directory

Files

Directory must also be stored on the hard disk. At least one directory (eventually even more). In Unix, the directory is a file (vi .).

Computer Architecture and Operating Systems, Florina Ciorba, 10.11.15

Single-Level Directories


436 Chapter 10

names may indicate a relationship between files, we may want to be able to find all files whose names match a particular pattern.

Create a file. New files need to be created and added to the directory.

Delete a file. When a file is no longer needed, we want to be able to remove it from the directory.

List a directory. We need to be able to list the files in a directory and the contents of the directory entry for each file in the list.

Rename a file. Because the name of a file represents its contents to its users, we must be able to change the name when the contents or use of the file changes. Renaming a file may also allow its position within the directory structure to be changed.

Traverse the file system. We may wish to access every directory and every file within a directory structure. For reliability, it is a good idea to save the contents and structure of the entire file system at regular intervals. Often, we do this by copyin.g all files to magn.etic tape. This technique provides a backup copy in case of system failure. In addition, if a file is no longer in use, the file can be copied to tape and the disk space of that file released for reuse by another file.

In. the following sections, we describe the most common schemes for defining the logical structure of a directory.

10.3.3 Single-level Directory

The simplest directory structure is the single-level directory. All files are contained in the same directory, which is easy to support and understand (Figure 10.8).

A single-level directory has significant limitations, however, when the number of files increases or when the system has more than one user. Since all files are in the same directory, they must have unique names. If two users call their data file test, then the unique-name rule is violated. For example, in one programming class, 23 students called the program for their second assignment prog2; another 11 called it assign2. Although file names are generally selected to reflect the content of the file, they are often limited in length, complicating the task of making file names unique. The MS-DOS operating system allows only 11-character file names; UNIX, in contrast, allows 255 characters.

Even a single user on a single-level directory may find it difficult to remember the names of all the files as the number of files increases. It is not

directory

files

Figure 10.8 Single-level directory.

Two-Level Directories


10.3 437

uncommon for a user to have hundreds of files on one computer system and an equal number of additional files on another system. Keeping track of so many files is a daunting task.

10.3.4 Two-Level Directory

As we have seen, a single-level directory often leads to confusion of file names among different users. The standard solution is to create a separate directory for each user.

In the two-level directory structure, each user has his own The UFDs have similar structures, but each lists only the files

of a single user. W11en a user job starts or a user logs in, the system's is searched. The MFD is indexed by user name or account

number, and each entry points to the UFD for that user (Figure 10.9). When a user refers to a particular file, only his own UFD is searched. Thus,

different users may have files with the same name, as long as all the file names within each UFD are unique. To create a file for a user, the operating system searches only that user's UFD to ascertain whether another file of that name exists. To delete a file, the operating system confines its search to the local UFD; thus, it cannot accidentally delete another user's file that has the same name.

The user directories themselves must be created and deleted as necessary. A special system program is run with the appropriate user name and account information. The program creates a new UFD and adds an entry for it to the MFD. The execution of this program might be restricted to system administrators. The allocation of disk space for user directories can be handled with the teduciques discussed in Chapter 11 for files themselves.

Although the two-level directory structure solves the name-collision prob-lem, it still has disadvantages. This structure effectively isolates one user from another. Isolation is an advantage when the users are completely independent but is a disadvantage when the users want to cooperate on some task and to access one another's files. Some systems simply do not allow local user files to be accessed by other users.

If access is to be pennitted, one user must have the ability to name a file in another user's directory. To name a particular file "Lmiquely in a two-level directory, we must give both the user name and the file name. A two-level directory can be thought of as a tree, or an inverted tree, of height 2. The root of the tree is the MFD. Its direct descendants are the UFDs. The descendants of

user file directory

Figure i 0.9 Two-level directory structure.

Tree-Based Directories

root spell bin programs

p e mailstat mail dist find count hex reorder

prog

list

copy prt exp reorder list find hex count

obj spell all last first


What Have We Learned Today?

•  Concurrent programming •  Reentrance (in Unix libraries)

•  Concurrency abstractions •  Coroutines, COBEGIN/COEND, Future, fork/join •  Coordination via tuplespace

•  File systems •  Four layer mode


operating systems part – m13 reentrance coroutines file systems...

Documents