operating systems recitation summary what you should know for the exam

Operating Systems

Recitation summaryWhat you should know for the exam

You should know• System/API calls and their parameters discussed at the class or

used in hw assignments– (e.g. CreateFile creates/opens file depending on a parameter)

• Some details on Windows system internals we discussed– (e.g. object reference counting, memory management, file system)

• Windows Networking Tools– (e.g. use ipconfig to find out my ip address)

• Purpose and capabilities of networking protocols we have discussed– (e.g. Dns protocol is used to resolve symbolic name into numeric ip)

System Calls

• No need to memorize exact details• But ,you are expected to know– Approximate name (exact for important function)– What it does (behavior)– Effect of most important parameters– Usage Pattern

Subjects1. Win32 Types and Objects, Unicode Strings2. Files3. Processes4. Threads5. Process/Thread Synchronization6. Virtual Memory 7. Memory Mapped Files8. Dynamic Link Libraries9. Networking and Windows Net Utilities10. Sockets Programming11. Windows Security12. Structured Exception Handling

1. Win32 types, objects, Unicode

• Reference Counting and CloseHandle• Handle Permissions and Security• Signaled State of an object• LPTSTR, TCHAR, DWORD• L”aa”, _T(“aa”)• GetLastError()

6

HandleCount = 1ReferenceCount = 1

Event Object

1.1 Handles, Pointers, and Objects

Handle Table

Process A

Handle Table

Process B

System Space

handles

index

• Handle to a kernel object is an index into the process handle table, and hence is invalid in any other process

• Handle table entry contains the system-space address (8xxxxxxx or above) of the data structure; this address is the same regardless of process context

• Although handle table is per-process, it is actually in system address space (hence protected)

7


Event Object

1.1 Handles, Pointers, and Reference Count

Handle Table

Process A

Handle Table

Process B

System Spacehandles

index HandleCount = 2ReferenceCount = 0

DuplicateHandle



Thread(in a wait statefor the event)

Thread(in a wait statefor the event)

Note: there is actually another data structure, a “wait block”, “between” the thread and the object it’s waiting for

2.Files

• CreateFile• ReadFile• WriteFile• SetFilePointer• FindFirstFile• FindNextFile• CloseHandle/FindClose• GetFileSize

2.1 CreateFile

• HANDLE WINAPI CreateFile( __in LPCTSTR lpFileName, __in DWORD dwDesiredAccess, __in DWORD dwShareMode, __in_opt LPSECURITY_ATTRIBUTES lpSecurityAttributes, __in DWORD dwCreationDisposition, __in DWORD dwFlagsAndAttributes, __in_opt HANDLE hTemplateFile );

2.1 CreateFileArgument Sample Values

pFileName _T(“c:\\a.txt”)

dwDesiredAccess GENERIC_READ | GENERIC_WRITE

wShareMode FILE_SHARE_READ|FILE_SHARE_WRITE

pSecurityAttributes NULL

wCreationDisposition CREATE_ALWAYS, OPEN_EXISTING

dwFlagsAndAttributes FILE_ATTRIBUTE_READONLY, FILE_ATTRIBUTE_HIDDENFILE_FLAG_SEQUENTIAL_SCANFILE_FLAG_RANDOM_ACCESS

2.2 SetFilePointer

• DWORD WINAPI SetFilePointer( __in HANDLE hFile, __in LONG lDistanceToMove, __inout_opt PLONG lpDistanceToMoveHigh, __in DWORD dwMoveMethod );

2.2 SetFilePointerArgument Sample Value

dwMoveMethod FILE_BEGIN, FILE_CURRENT, FILE_END

lDistanceToMove 1000

hFile handle

3.Processes

• BOOL WINAPI CreateProcess( __in_opt LPCTSTR lpApplicationName, __inout_opt LPTSTR lpCommandLine, __in_opt LPSECURITY_ATTRIBUTES lpProcessAttributes, __in_opt LPSECURITY_ATTRIBUTES lpThreadAttributes, __in BOOL bInheritHandles, __in DWORD dwCreationFlags, __in_opt LPVOID lpEnvironment, __in_opt LPCTSTR lpCurrentDirectory, __in LPSTARTUPINFO lpStartupInfo, __out LPPROCESS_INFORMATION lpProcessInformation );

3.Processes

• CreateProcess• GetProcessExitCode• TerminateProcess• WaitForSingleObject

3.1 CreateProcessArgument Value

lpApplicationName _T(“Program.exe”)

pCommandLine _T(“program.exe 1”)

pProcessAttributes NULL

lpThreadAttributes NULL

dwCreationFlags CREATE_NO_WINDOW

lpProcessInformation &pi

4. Threads

• CreateThread• TerminateThread• WaitForSingleObject

4.Threads

• HANDLE WINAPI CreateThread( __in_opt LPSECURITY_ATTRIBUTES lpThreadAttributes, __in SIZE_T dwStackSize, __in LPTHREAD_START_ROUTINE lpStartAddress, __in_opt LPVOID lpParameter, __in DWORD dwCreationFlags, __out_opt LPDWORD lpThreadId );

18

Single and Multithreaded Processes

code data files

registers stack

Thread

single-threaded

code data files

registers

stack

Thread

multi-threaded

stack

registers

stack

registers

Thread Thread

19

Thread States• Five-state diagram for thread scheduling:

– init: The thread is being created– ready: The thread is waiting to be assigned to a CPU– running: The thread’s instructions are being executed– waiting: The thread is waiting for some event to occur– terminated: The thread has finished execution

init

ready

waiting

running

terminated

schedulerdispatch

waiting forI/O or event

I/O or eventcompletion

interrupt quantum expired

admitted exit

4.1 CreateThreadArgument Value

pThreadAttributes NULL

dwStackSize 0

lpStartAddress pfThreadFunc

lpParameter (LPVOID)a

21

Process Control Block (PCB)

Program Counter

Parent PID

…

Handle Table

Process ID (PID)

Registers

Next Process Block

Image File Name

PCB

List of ThreadControl Blocks

List of open files

…

• This is an abstract view• Windows implementation of PCB is split

in multiple data structures

Next TCB

…

Thread Control Block (TCB)

22

CPU Switch from Thread to ThreadThread T1

executing

executing

ready orwaiting

Save state into TCB2

Reload state from TCB1

Save state into TCB1

Reload state from TCB2

Interrupt or system call Thread T2

executingInterrupt or system call

ready orwaiting

ready orwaiting

5.Synchronization

• CreateEvent/SetEvent• CreateMutex/ReleaseMutex• CreateSemaphore/ ReleaseSemaphore• InitializeCriticalSection/DeleteCriticalSection• WaitForSingleObject/WautForMultipleObjects• GetLastError()

5.1 Events

• HANDLE WINAPI CreateEvent( __in_opt LPSECURITY_ATTRIBUTES lpEventAttributes, __in BOOL bManualReset, __in BOOL bInitialState, __in_opt LPCTSTR lpName );

• BOOL WINAPI SetEvent( __in HANDLE hEvent );

5.2 Mutex

• HANDLE WINAPI CreateMutex( __in_opt LPSECURITY_ATTRIBUTES lpMutexAttributes,__in BOOL bInitialOwner, __in_opt LPCTSTR lpName );

• BOOL WINAPI ReleaseMutex( __in HANDLE hMutex );

If a thread already owns a mutex =>WaitForSingleObject does not block

5.3 Semaphores

• HANDLE CreateSemaphore( LPSECURITY_ATTRIBUTES lpSemaphoreAttributes, LONG lInitialCount, LONG lMaximumCount, LPCTSTR lpName );

• BOOL ReleaseSemaphore( HANDLE hSemaphore, LONG lReleaseCount, LPLONG lpPreviousCount );

5.4 Critical Section• void WINAPI InitializeCriticalSection( __out

LPCRITICAL_SECTION lpCriticalSection );

• void WINAPI EnterCriticalSection( __inout LPCRITICAL_SECTION lpCriticalSection );

• void WINAPI LeaveCriticalSection( __inout LPCRITICAL_SECTION lpCriticalSection );

• void WINAPI DeleteCriticalSection( __inout LPCRITICAL_SECTION lpCriticalSection );

6. Virtual Memory

• VirualAlloc/VirtualFree• VirtualLock/VirtualUnlock• SetProcessWorkingSetSize

6.1 Allocation

• LPVOID WINAPI VirtualAlloc( __in_opt LPVOID lpAddress, __in SIZE_T dwSize, __in DWORD flAllocationType, __in DWORD flProtect );

• BOOL WINAPI VirtualFree( __in LPVOID lpAddress, __in SIZE_T dwSize, __in DWORD dwFreeType );

30

x86 Virtual Address Translation

Page tableselector

Page tableselector

Page tableentry selector

Page tableentry selector Byte within pageByte within page

index

Page Directory(1024 entries)

index

Page Tables(up to 512 per process,

plus up to 512 system-wide)

physicalpage number(“page framenumber” or

“PFN”)

Physical Pages(up to 2^20)

1

CR3CR3physicaladdress

2

3

4

5

6

7

8

9

10

11

12

PFN 0

31 0

31

Byte within pageByte within pageVirtual page numberVirtual page number

Virtual Address Translation

• The hardware converts each valid virtual address to a physical address

Physical page numberPhysical page number Byte within pageByte within page

PageDirectory

PageDirectory

PageTables

PageTables

virtual address

physical address

TranslationLookaside

Buffer

TranslationLookaside

Buffer

a cache of recently-used page table entries

Address translation (hardware)Address translation (hardware)

if pagenot valid...

page fault(exception,handled bysoftware)

6.1 AllocationArgument Value

wSize 4096

lAllocationType MEM_COMMIT|MEM_RESERVE

flProtect PAGE_EXECUTE |PAGE_READWRITE

6.2 Lock Memory

• BOOL WINAPI VirtualLock( __in LPVOID lpAddress, __in SIZE_T dwSize );

• BOOL WINAPI VirtualUnlock( __in LPVOID lpAddress, __in SIZE_T dwSize );

7.Memory mapped Files

• CreateFileMapping• MapViewOfFile/UnmapViewOfFile

35

Shared and Private Pages

00000000

7FFFFFFF

C0000000

C1000000

80000000

FFFFFFFF

• For shared pages, multiple processes’ PTEs point to same physical pages

Process A Process B

PhysicalMemory

7.1 CreateFileMapping

• HANDLE WINAPI CreateFileMapping( __in HANDLE hFile, __in_opt LPSECURITY_ATTRIBUTES lpAttributes, __in DWORD flProtect, __in DWORD dwMaximumSizeHigh, __in DWORD dwMaximumSizeLow, __in_opt LPCTSTR lpName );

7.1 CreateFileMappingArgument

hFile INVALID_HANDLE_VALUE,

lpAttributes NULL

flProtect PAGE_READWRITE

lpName _T(“MyMapping.1”)

7.2 MapViewOfFile

• LPVOID WINAPI MapViewOfFile( __in HANDLE hFileMappingObject, __in DWORD dwDesiredAccess, __in DWORD dwFileOffsetHigh, __in DWORD dwFileOffsetLow, __in SIZE_T dwNumberOfBytesToMap );

• BOOL WINAPI UnmapViewOfFile( __in LPCVOID lpBaseAddress );

8.Dynamic Link Libraries

• LoadLibrary/FreeLibrary• GetProcAddress• __declspec(dllexport), __declspec(dllimport)• DllMain

40

Address Binding

• Addresses in source programs are symbolic

• Compiler binds symbolic to relocatable addresses

• Loader binds relocatable addresses to absolute addresses

Binding can be done at any step:• i.e., compiler may generate

absolute code (as for MS-DOS .COM programs)

Sourceprogram

Compiler orassembler

Object module

other object

modules

Linkageeditor

Loadmodule

loaderr

Systemlibraries

In-memorybinary

memoryimage

dynamicallyloadedsystemlibraries

loadtime

Compiletime

executiontime(run time)

8.1 Run-Time

• HMODULE WINAPI LoadLibrary( __in LPCTSTR lpFileName );

• FARPROC WINAPI GetProcAddress( __in HMODULE hModule, __in LPCSTR lpProcName );

9.Net Utilities

• Ipconfig• Ping• Tracert• Route• Arp• Nslookup• Netstat

9. Opening Browser1. Plug Network Cable->Broadcast DHCP to config

2. Type address->use dns server to translate

3. Have destination IP->use routing table to find next hop

4. Have IP of next hop->use arp table/protocol to translate destination IP into MAC address

5. Connect on TCP port 80 and send HTTP GET request

6. Wait for ack, resend if needed

7. Obtain HTML content , disconnect and show to a user

10.Sockets Programming

• Socket• Bind• Listen• Accept• Connect• Send/Recv• Closesocket

10. Socket FunctionsFunction

bind function associates a local address with a socket.

listen places a socket in a state in which it is listening for an incoming connection

accept permits an incoming connection attempt on a socket

connect establishes a connection to a specified socket

send sends data on a connected socket

recv receives data from a connected socket

11.Security

• OpenProcessToken• LookupAccountSid• GetTokenInformation• ACL/ACE format/purpose

11. OpenProcessTokenBOOL WINAPI OpenProcessToken(

__in HANDLE ProcessHandle, __in DWORD DesiredAccess, __out PHANDLE TokenHandle );

BOOL WINAPI GetTokenInformation( __in HANDLE TokenHandle, __in TOKEN_INFORMATION_CLASS TokenInformationClass, __out_opt LPVOID TokenInformation, __in DWORD TokenInformationLength, __out PDWORD ReturnLength );

11. ACL/ACE

12.Structured Exception Handling

• __try• __except• GetExceptionCode()

12.1 Try-except

__try { //guarded code } __except ( expression ) {

//exception handler code }

12.2 Filter ExpressionValue Effect

EXCEPTION_CONTINUE_EXECUTION Exception is dismissed. Continue execution at the point where the exception occurred

EXCEPTION_CONTINUE_SEARCH Exception is not recognized. Continue to search up the stack for a handler

EXCEPTION_EXECUTE_HANDLER Exception is recognized. Transfer control to the exception handler by executing the __except compound statement, then continue execution after the __except block.

In additional to system calls you have to understand

• Threads vs. Processes• Context switch/Scheduling and related bugs• Choice of synchronization mechanisms• Compile-time vs. Run-time binding of DLL• Physical vs. Virtual Addresses• Virtual Memory Performance Issues• Build IPC using MMF and sockets• Build/explain ACL structure• Build/explain exception handling

operating systems recitation summary what you should know for the exam

Documents