multithreaded programming with the win32 api andrew tucker andrew tucker debugger development lead...

Multithreaded Multithreaded Programming With the Programming With the

Win32 APIWin32 API

Andrew TuckerAndrew Tucker

Debugger Development LeadDebugger Development Lead

March 13, 1998March 13, 1998

What We Will CoverWhat We Will Cover

• Intro to Multithreaded ConceptsIntro to Multithreaded Concepts

•Starting and Stopping ThreadsStarting and Stopping Threads

•SynchronizationSynchronization

•Debugging and Testing IssuesDebugging and Testing Issues

•Interprocess CommunicationInterprocess Communication

•Advanced Topics and Additional Advanced Topics and Additional ResourcesResources

CaveatsCaveats

• Multithreaded feature sets differ Multithreaded feature sets differ between NT, Win95 and CE and between NT, Win95 and CE and versions of the same OSversions of the same OS

Intro to Multithreaded Intro to Multithreaded ConceptsConcepts

What is a thread? “path of execution in a What is a thread? “path of execution in a process”process”

• owned by a single processowned by a single process• all processes have main thread, some all processes have main thread, some have morehave more• has full access to process address spacehas full access to process address space

Process1

Process2ProcessNMain

T1 T2T3

MainMain T1

Operating System

Intro to Multithreaded Intro to Multithreaded ConceptsConcepts

Scheduling - cooperative vs preemptiveScheduling - cooperative vs preemptive• Preemptive - allow a thread to execute for a Preemptive - allow a thread to execute for a

specified amount of time and then specified amount of time and then automatically performs a “context switch” to automatically performs a “context switch” to change to a new thread (e.G. NT, win95, WCE) change to a new thread (e.G. NT, win95, WCE)

• Cooperative - performs context switch only Cooperative - performs context switch only when the user specifies (“manually when the user specifies (“manually scheduled”)scheduled”)

• Win16 is neither: multitasking, but not Win16 is neither: multitasking, but not multithreadmultithread

Starting and Stopping Starting and Stopping ThreadsThreads

CreateThread APICreateThread APIHANDLE CreateThread( LPSECURITY_ATTRIBUTES lpsa, // pointer to thread security attributes

DWORD dwStackSize, // initial thread stack size, in bytes

LPTHREAD_START_ROUTINE lpStartAddress, // pointer to thread function

LPVOID lpParameter, // argument for new thread

DWORD dwCreationFlags, // creation flags

LPDWORD lpThreadId // pointer to returned thread identifier

);

_beginthreadex CRT function_beginthreadex CRT functionunsigned long _beginthreadex( void *security,

unsigned stack_size,

unsigned ( __stdcall *start_address )( void * ),

void *arglist,

unsigned initflag,

unsigned *thrdaddr );

So, what’s the difference?So, what’s the difference?


• Difference is the initialization of the CRT libraryDifference is the initialization of the CRT library• Linking with multithreaded CRT is not enoughLinking with multithreaded CRT is not enough

DWORD ThreadFunc(PVOID pv){ char *psz = strtok((char*)pv, “;”);

while ( psz ) { …. process data …. psz = strtok(NULL. “;”); }}

int main()

{ // BUG - use _beginthreadex to ensure thread safe CRT HANDLE hthrd1 = CreateThread( … ThreadFunc … ); HANDLE hthrd2 = CreateThread( … ThreadFunc … );}

•_beginthreadex _beginthreadex creates a creates a structure to structure to ensure global ensure global and static CRT and static CRT variables are variables are thread-specificthread-specific


• Thread functions have the Thread functions have the following prototype:following prototype:

DWORD WINAPI ThreadFunc(PVOID pv);

• It is very useful to use pv as a It is very useful to use pv as a pointer to a user-defined structure pointer to a user-defined structure to pass extra datato pass extra data


• A return will automatically call the A return will automatically call the respective _endthreadex or EndThread respective _endthreadex or EndThread APIAPI

• A return does not close the handle A return does not close the handle from the creation routine (user must from the creation routine (user must call CloseHandle to avoid resource call CloseHandle to avoid resource leak)leak)

• Threads should be self-terminating Threads should be self-terminating (avoid the TerminateThread API)(avoid the TerminateThread API)


Reasons to avoid the TerminateThread API:Reasons to avoid the TerminateThread API:• If the target thread owns a critical section, it If the target thread owns a critical section, it

will not be releasedwill not be released• If the target thread is executing certain If the target thread is executing certain

kernel calls, the kernel state for the thread’s kernel calls, the kernel state for the thread’s process could be inconsistentprocess could be inconsistent

• If the target thread is manipulating the global If the target thread is manipulating the global state of a shared DLL, the state of the DLL state of a shared DLL, the state of the DLL could be destroyed, affecting other users of could be destroyed, affecting other users of the DLLthe DLL


• Using a C++ member as a thread function (fixing the ‘this’ problem):Using a C++ member as a thread function (fixing the ‘this’ problem):

class ThreadedClass{ public: ThreadedClass(); BOOL Start(); void Stop(); private: HANDLE m_hThread; BOOL m_bRunning; static UINT WINAPI StaticThreadFunc(LPVOID lpv); DWORD MemberThreadFunc();};UINT WINAPI ThreadedClass::StaticThreadFunc(

LPVOID lpv){ ThreadedClass *pThis = (ThreadedClass *)lpv; return pThis->MemberThreadFunc();}DWORD ThreadedClass::MemberThreadFunc(){ while ( m_bRunning ) { … do processing... }}

BOOL ThreadedClass::Start(DWORD dwStart){ UINT nTID; m_hThread = (HANDLE)_beginthreadex(NULL, 0,

StaticThreadFunc, this, 0, &nTID) return TRUE;}void ThreadedClass::Stop(){ m_bRunning = FALSE; // wait for thread to finish DWORD dwExitCode; GetExitCodeThread(m_hThread, &dwExitCode); while ( dwExitCode == STILL_ACTIVE ) { GetExitCodeThread(m_hThread, &dwExitCode); } m_hThread = 0;}int main(){ ThreadedClass tc1, tc2; tc1.Start(5); tc2.Start(5000); Sleep(3000); tc1.Stop(); tc2.Stop(); return 0;}


• SuspendThread and ResumeThread SuspendThread and ResumeThread allow you to pause and restart any allow you to pause and restart any threadthread

• Suspension state is a count not a Suspension state is a count not a boolean - calls should be balancedboolean - calls should be balanced

• Example: hitting a bp in a debugger Example: hitting a bp in a debugger causes all current threads to be causes all current threads to be suspended and resumed on step or gosuspended and resumed on step or go


• GetCurrentThread and GetCurrentThread and GetCurrentThreadId are useful for GetCurrentThreadId are useful for identifying current threadidentifying current thread

• GetExitCodeThread is useful for GetExitCodeThread is useful for determining if a thread is still alivedetermining if a thread is still alive

• GetThreadTimes is useful for GetThreadTimes is useful for performance analysis and performance analysis and measurementmeasurement

SynchronizationSynchronization

• Used to coordinate the activities of Used to coordinate the activities of concurrently running threadsconcurrently running threads

• Always avoid coordinating with a Always avoid coordinating with a poll loop when possible for poll loop when possible for efficiency reasonsefficiency reasons


• Interlocked functionsInterlocked functions• Critical SectionsCritical Sections• Wait functionsWait functions• MutexesMutexes• SemaphoresSemaphores• EventsEvents• Waitable TimersWaitable Timers


• Interlocked functions:Interlocked functions:PVOID InterlockedCompareExchange(PVOID *destination, PVOID Exchange, PVOID Comperand)

if ( *Destination == Comperand )

*Destination = Exchange;

LONG InterlockedExchange(LPLONG Target, LONG Value )

*Target = Value;

LONG InterlockedExchangeAdd(LPLONG Addend, LONG Increment)

*Addend += Increment;

LONG InterlockedDecrement(LPLONG Addend)

*Addend -= 1;

LONG InterlockedIncrement(LPLONG Addend)

*Addend += 1;

• All operations are guaranteed to be All operations are guaranteed to be “atomic” - the entire routine will “atomic” - the entire routine will execute w/o a context switchexecute w/o a context switch


Why must simple operations like Why must simple operations like incrementing an integer be “atomic”?incrementing an integer be “atomic”?

Multiple CPU instructions are required for Multiple CPU instructions are required for the actual implementation. If we the actual implementation. If we retrieved a variable and were then retrieved a variable and were then preempted by a thread that changed preempted by a thread that changed that variable, we would be using the that variable, we would be using the wrong value.wrong value.


• A critical section is a tool for A critical section is a tool for guaranteeing that only one thread guaranteeing that only one thread is executing a section of code at is executing a section of code at any timeany time

void InitializeCriticalSection(LPCRITICAL_SECTION lpCritSec)

void DeleteCriticalSection(LPCRITICAL_SECTION lpCritSec)

void EnterCriticalSection(LPCRITICAL_SECTION lpCritSec)

void LeaveCriticalSection(LPCRITICAL_SECTION lpCritSec)

BOOL TryEnterCriticalSection(LPCRITICAL_SECTION lpCritSec)


• EnterCriticalSection will not block on EnterCriticalSection will not block on nested calls as long as the calls are in the nested calls as long as the calls are in the same thread. Calls to same thread. Calls to LeaveCriticalSection must still be LeaveCriticalSection must still be balancedbalanced


• Critical section example: counting Critical section example: counting source lines in multiple filessource lines in multiple files

DWORD g_dwTotalLineCount = 0;

DWORD CountLinesThread(PVOID pv){

PSTR pszFileName = (PSTR)pv;

DWORD dwCount;

dwCount = CountSourceLines(pszFileName);

EnterCriticalSection(&cs);

g_dwTotalLineCount += dwCount;

LeaveCriticalSection(&cs);

return 0;

}

void UpdateSourceLineCount(){

FileNameList fnl; HANDLE *pHandleList; CRITICAL_SECTION cs;

GetFileNameList(&fnl); InitializeCriticalSection(&cs);

pHandleList = malloc(sizeof(HANDLE)*fnl.Size());

for ( int i = 0; i < FileNameList.Size(); i++) pHandleList[i] = _beginthreadex(…CountLinesThread, fnl[i]…);

//we’ll cover this shortly… WaitForMultipleObjects(fnl.Size(), pHandleList, TRUE, INFINITE);

DeleteCriticalSection(&cs);

…process g_dwTotalLineCount...

}


• Wait functions - allow you to pause Wait functions - allow you to pause until one or more objects become until one or more objects become signaledsignaled

• At all times, an object is in one of two At all times, an object is in one of two states: signaled or nonsignaledstates: signaled or nonsignaled

• Picture signaled as a flag being raised Picture signaled as a flag being raised and nonsignaled as a flag being and nonsignaled as a flag being lowered - the wait functions are lowered - the wait functions are watching for a flag to be raisedwatching for a flag to be raised


Types of objects that can be “waited on“:Types of objects that can be “waited on“:•ProcessesProcesses•ThreadsThreads•Console InputConsole Input•File Change File Change NotificationsNotifications•Mutexes*Mutexes*•Semaphores*Semaphores*• Events*Events*•Waitable Timers*Waitable Timers*


• Processes and threads are non-Processes and threads are non-signaled at creation and become signaled at creation and become signaled when they terminatesignaled when they terminate


DWORD WaitForSingleObject(HANDLE hHandle, DWORD dwMilliseconds)

• Returns WAIT_OBJECT_0 if hHandle has become signaled or Returns WAIT_OBJECT_0 if hHandle has become signaled or WAIT_TIMEOUT if dwMilliseconds elapsed and the object is still WAIT_TIMEOUT if dwMilliseconds elapsed and the object is still non-signalednon-signaled

DWORD WaitForMultipleObjects(DWORD nCount, HANDLE *pHandles, BOOL bWaitAll, DWORD dwMilliseconds)

• If bWaitAll is FALSE and one of the object handles was signaled, If bWaitAll is FALSE and one of the object handles was signaled, the return value minus WAIT_OBJECT_0 is the array index of the return value minus WAIT_OBJECT_0 is the array index of that handle. If bWaitAll is TRUE and all of the objects become that handle. If bWaitAll is TRUE and all of the objects become signaled the return value minus WAIT_OBJECT_0 is a valid index signaled the return value minus WAIT_OBJECT_0 is a valid index into the handle array. If dwMilliseconds elapsed and no object into the handle array. If dwMilliseconds elapsed and no object was signaled, WAIT_TIMEOUT is returned. nCount can be no was signaled, WAIT_TIMEOUT is returned. nCount can be no more than MAXIMUM_WAIT_OBJECTS (currently defined as 64)more than MAXIMUM_WAIT_OBJECTS (currently defined as 64)

•INFINITE can be used as a INFINITE can be used as a timeout valuetimeout value


• Mutexes provide mutually exclusive Mutexes provide mutually exclusive access to an object (hence the name)access to an object (hence the name)

HANDLE CreateMutex(LPSECURITY)ATTRIBUTES lpsa,

BOOL bInitialOwner, LPCTSTR lpName)

• Ownership is equivalent to the Ownership is equivalent to the nonsignaled state - if bInitialOwner is TRUE nonsignaled state - if bInitialOwner is TRUE the creation state of the mutex is the creation state of the mutex is nonsignalednonsignaled

• lpName is optionallpName is optional• ReleaseMutex is used to end ownershipReleaseMutex is used to end ownership


What’s the difference between a What’s the difference between a critical section and a mutex?critical section and a mutex?

A mutex is a OS kernel object, and A mutex is a OS kernel object, and can thus be used across process can thus be used across process boundaries. A critical section is boundaries. A critical section is limited to the process in which it limited to the process in which it was createdwas created


Two methods to get a handle to a Two methods to get a handle to a named mutex created by another named mutex created by another process:process:

• OpenMutex - returns handle to an OpenMutex - returns handle to an existing mutexexisting mutex

• CreateMutex - creates or returns CreateMutex - creates or returns handle to an existing mutex. handle to an existing mutex. GetLastError will return GetLastError will return ERROR_ALREADY_EXISTS for the latter ERROR_ALREADY_EXISTS for the latter case case


• Comparing mutex and critical Comparing mutex and critical section performancesection performance

Performance of Mutex vs Critical Section

0

200

400

600

800

1000

1200

1000

7000

13000

19000

25000

31000

37000

43000

49000

55000

61000

67000

73000

79000

85000

91000

97000

Iterations

Tim

e (i

n m

illi

seco

nd

s)

CS

Mutex


• A mutex will not block on nested calls A mutex will not block on nested calls as long as they are in the same as long as they are in the same thread. ReleaseMutex calls must still thread. ReleaseMutex calls must still be balancedbe balanced

• Examples of when to use a mutex:Examples of when to use a mutex:• Error logging system that can be used Error logging system that can be used

from any processfrom any process• Detecting multiple instances of an Detecting multiple instances of an

applicationapplication


• Semaphores allow access to a resource to Semaphores allow access to a resource to be limited to a fixed numberbe limited to a fixed number

HANDLE CreateSemaphore(LPSECURITY_ATTRIBUTE lpsa, LONG cSemInitial, LONG cSemMax, LPCTSTR lpName)

• Semaphores are in the signaled state Semaphores are in the signaled state when their available count is greater than when their available count is greater than zerozero

• ReleaseSemaphore is used to decrement ReleaseSemaphore is used to decrement usageusage

• Conceptually, a mutex is a binary Conceptually, a mutex is a binary semaphoresemaphore


• Named semaphores can be used Named semaphores can be used across process boundaries with across process boundaries with OpenSemaphore and OpenSemaphore and CreateSemaphoreCreateSemaphore

• Can be used to solve the classic Can be used to solve the classic “single writer / multiple readers” “single writer / multiple readers” problemproblem


• Example: limiting number of Example: limiting number of entries in a queueentries in a queue

const int QUEUE_SIZE = 5;

HANDLE g_hSem = NULL;long g_iCurSize = 0;

UINT WINAPI PrintJob(PVOID pv)

{ WaitForSingleObject(g_hSem, INFINITE);

InterlockedIncrement(&g_iCurSize);

printf("%08lX - entered queue: size = %d\n", GetCurrentThreadId(), g_iCurSize );

Sleep(500); // print job....

InterlockedDecrement(&g_iCurSize);

long lPrev; ReleaseSemaphore(g_hSem, 1, &lPrev);

return 0;}

int main(){

const int MAX_THREADS = 64; HANDLE hThreads[MAX_THREADS];

g_hSem = CreateSemaphore(NULL, QUEUE_SIZE, QUEUE_SIZE, NULL );

UINT dwTID; for ( int i = 0; i < MAX_THREADS; i++ ) hThreads[i] = (HANDLE)_beginthreadex(NULL, 0, PrintJob, NULL, 0, &dwTID);

WaitForMultipleObjects(MAX_THREADS, hThreads, TRUE, INFINITE);

return 0;}


• Events provide notification when Events provide notification when some condition has been metsome condition has been met

HANDLE CreateEvent(LPSECURITY_ATTRIBUTES lpsa,

BOOL bManualReset, BOOL bInitialState,

LPCTSTR lpName)

• If bInitialState is TRUE, object is If bInitialState is TRUE, object is created in the signaled statecreated in the signaled state

• bManualReset specifies the type of bManualReset specifies the type of event requestedevent requested


Two kinds of event objects:Two kinds of event objects:• Auto reset - when signaled it is Auto reset - when signaled it is

automatically changed to a automatically changed to a nonsignaled state after a single nonsignaled state after a single waiting thread has been releasedwaiting thread has been released

• Manual reset - when signaled it Manual reset - when signaled it remains in the signaled state until it is remains in the signaled state until it is manually changed to the nonsignaled manually changed to the nonsignaled state state


• Named event objects can be used across Named event objects can be used across process boundaries with OpenEvent and process boundaries with OpenEvent and CreateEventCreateEvent

• SetEvent sets the object state to signaledSetEvent sets the object state to signaled• ResetEvent sets the object state to ResetEvent sets the object state to

nonsignalednonsignaled• PulseEvent conceptually calls PulseEvent conceptually calls

SetEvent/ResetEvent sequentially, but ...SetEvent/ResetEvent sequentially, but ...


PulseEvent vs SetEventPulseEvent vs SetEvent

Auto Reset Manual ResetSetEvent

Exactly one thread isreleased. If none arecurrently waiting on theevent, the first thread to waiton it in the future will bereleased immediately.

All currentlywaiting threads arereleased. The eventremains signaleduntil reset by somethread.

PulseEventExactly one thread isreleased, but only if a threadis currently waiting on theevent.

All currentlywaiting threads arereleased, and theevent is then reset.


• Example: displaying OutputDebugString Example: displaying OutputDebugString text without a debuggertext without a debugger

int main(){ HANDLE hAckEvent, hReadyEvent; PSTR pszBuffer;

hAckEvent = CreateEvent(NULL, FALSE,

FALSE,

“DBWIN_BUFFER_READY”);

if (GetLastError() == ERROR_ALREADY_EXISTS)

{ // handle multiple instance case }

hReadyEvent = CreateEvent(NULL, FALSE, FALSE, “DBWIN_DATA_READY”);

pszBuffer = /* get pointer to data in memory mapped file */;

SetEvent(hAckEvent);

while ( TRUE )

{

int ret = WaitForSingleObject(hReadyEvent, INFINITE);

if ( ret != WAIT_OBJECT_0) { // handle error } else { printf(pszBuffer); SetEvent(hAckEvent); } }}


• Waitable timers are kernel objects Waitable timers are kernel objects that provide a signal at a specified that provide a signal at a specified time intervaltime interval

HANDLE CreateWaitableTimer(LPSECURITY_ATTRIBUTES lpsa, BOOL bManualReset, LPCTSTR lpName)

• Manual/auto reset behavior is Manual/auto reset behavior is identical to eventsidentical to events

• Time interval is specified with Time interval is specified with SetWaitableTimerSetWaitableTimer


BOOL SetWaitableTimer(HANDLE hTimer, LARGE_INTEGER *pDueTime, LONG lPeriod, PTIMERACPROUTINE pfnCompletion, PVOID pArg, BOOL fResume)

• pDueTime specifies when the timer pDueTime specifies when the timer should go off for the first time (positive should go off for the first time (positive is absolute, negative is relative)is absolute, negative is relative)

• lPeriod specifies how frequently to go lPeriod specifies how frequently to go off after the initial timeoff after the initial time

• fResume controls whether the system is fResume controls whether the system is awakened when timer is signaledawakened when timer is signaled


• Consecutive calls to Consecutive calls to SetWaitableTimer overwrite each SetWaitableTimer overwrite each otherother

• CancelWaitabletimer stops the CancelWaitabletimer stops the timer so that it will not go off again timer so that it will not go off again (unless SetWaitableTimer is called)(unless SetWaitableTimer is called)


• Example: firing an event every N secondsExample: firing an event every N secondsconst int MAX_TIMES = 3;const int N = 10;

DWORD WINAPI ThreadFunc(PVOID pv){ HANDLE hTimer = (HANDLE)pv; int iCount = 0; DWORD dwErr = 0;

while (TRUE) { if ( WaitForSingleObject(hTimer, INFINITE) == WAIT_OBJECT_0 ) { … handle timer event...

if ( ++iCount >= MAX_TIMES ) break; }

return 0;

}

int main(){ HANDLE hTimer = CreateWaitableTimer(NULL, FALSE, NULL); LARGE_INTEGER li;

const int nNanosecondsPerSecond = 10000000; __int64 qwTimeFromNow = N * nNanosecondsPerSecond; qwTimeFromNow = -qwTimeFromNow;

li.LowPart = (DWORD)(qwTimeFromNow & 0xFFFFFFFF); li.HighPart = (DWORD)(qwTimeFromNow >> 32);

SetWaitableTimer(hTimer, &li, N * 1000, NULL, NULL, FALSE);

DWORD dwTID; HANDLE hThread = CreateThread(NULL, 0, ThreadFunc, hTimer, 0, &dwTID);

WaitForSingleObject(hThread, INFINITE);

return 0;}

Debugging and Testing Debugging and Testing IssuesIssues

• Very difficult , if not impossible, to Very difficult , if not impossible, to reproduce and test every possible reproduce and test every possible deadlock and race conditiondeadlock and race condition

• Stepping through code will not Stepping through code will not necessarily helpnecessarily help

• OutputDebugString can be very OutputDebugString can be very helpfulhelpful

• Don’t underestimate the value of Don’t underestimate the value of peer review and code inspectionpeer review and code inspection

• Hint: after every wait or release Hint: after every wait or release ask yourself “what if a context ask yourself “what if a context switch occurred here”switch occurred here”

Debugging and Testing Debugging and Testing IssuesIssues

Interprocess Interprocess CommunicationCommunication

• Tools to provide the ability to pass Tools to provide the ability to pass data between processes or machinesdata between processes or machines

• Types of IPC:Types of IPC:

• ClipboardClipboard• DDEDDE• OLEOLE• Memory Mapped Memory Mapped

FilesFiles• MailslotsMailslots

• PipesPipes• RPCRPC• SocketsSockets• WM_COPYDATAWM_COPYDATA

Advanced TopicsAdvanced Topics

• Thread local storageThread local storage• UI vs worker threadsUI vs worker threads• CreateRemoteThreadCreateRemoteThread• Scheduling and Scheduling and

Get/SetThreadPriority)Get/SetThreadPriority)• Fibers (NT only)Fibers (NT only)• Asynchronous procedure callsAsynchronous procedure calls

ResourcesResources

• Advanced Windows by Jeff RichterAdvanced Windows by Jeff Richter• Win32 System Programming by Win32 System Programming by

Johnson HartJohnson Hart• Win32 Multithreaded Programming by Win32 Multithreaded Programming by

Aaron Cohen and Mike WoodringAaron Cohen and Mike Woodring• Windows NT Programming in Practice Windows NT Programming in Practice

by editors of WDJ (including Paula T)by editors of WDJ (including Paula T)

multithreaded programming with the win32 api andrew tucker andrew tucker debugger development lead...

Documents