C++ Network C++ Network ProgrammingProgramming

Mastering Complexity with Mastering Complexity with ACE & PatternsACE & Patterns

Dr. Douglas C. Schmidtd.schmidt@vanderbilt.edu

www.cs.wustl.edu/~schmidt/tutorials-ace.html

Professor of EECS Vanderbilt University Nashville, Tennessee

2

Introduction (1/3)

ACE Adaptive Communication Environment

The key contribution: Creation of uniform models which capture a broad spectrum of services Easier to do on a specific domain Easier to do when dealing with small building blocks

• For instance: The scoped-locking constructs

The motivation: Development of a concurrent server program (socket-based)

3

Comments (2/3)

Huge scope General overview Specific inspection of certain parts Our goal: A brief introduction (Slides per minute – High rate)

API Specifications (Doxygen) http://www.dre.vanderbilt.edu/Doxygen/Current/html/a

ce/hierarchy.html

4

Introduction (3/3)

ACE is somewhat outdated does not use exceptions

C++ oriented Some of the ideas seem trivial to Java programmers Some of the ideas cannot be supported on Java Thru ACE we can understand the design rationale Useful for C++ developers Useful for any developer of concurrent/distributed

programs

Example: Java’s thread class is a realization of ACE’s Active Objects pattern (see next slide)

5

Active Object: intent

Decouples method execution from method invocation and simplifies synchronized

access to shared resources by concurrent threads

Decouples method execution from method invocation and simplifies synchronized

access to shared resources by concurrent threads

6

Terminology

Framework

Toolkit

Patterns

7

Motivation: Challenges of Networked ApplicationsObservation• Building robust, efficient, & extensible concurrent & networked applications is hard• e.g., we must address many complex topics that are less problematic for non-concurrent, stand-alone applications

Accidental Complexities• Low-level APIs•Poor debugging tools•Algorithmic decomposition•Continuous re-invention/discovery of core concepts & components

Inherent Complexities• Latency•Reliability•Load balancing•Causal ordering•Scheduling & synchronization•Deadlock

Complexities in networked applications

8

Event Handling & IPCService Access & Control

Concurrency Synchronization

Key Capabilities Provided by ACE

9

The Layered Architecture of ACEFeatures•Open-source•200,000+ lines of C++

•40+ person-years of effort

•Ported to many OS platforms

www.cs.wustl.edu/~schmidt/ACE.html

10

The Pattern Language for ACEPattern Benefits• Preserve crucial design information used by applications & middleware frameworks & components

• Facilitate reuse of proven software designs & architectures

• Guide design choices for application developers

11

The Frameworks in ACE

ACE Framework Inversion of Control

Reactor & Proactor Calls back to application-supplied event handlers to perform processing when events occur synchronously & asynchronously

Service Configurator Calls back to application-supplied service objects to initialize, suspend, resume, & finalize them

Task Calls back to an application-supplied hook method to perform processing in one or more threads of control

Acceptor-Connector Calls back to service handlers to initialize them after they are connected

Streams Calls back to initialize & finalize tasks when they are pushed & popped from a stream

12

Example: Applying ACE in Real-time Avionics

Key System Characteristics• Deterministic & statistical deadlines

• ~20 Hz• Low latency & jitter

• ~250 usecs• Periodic & aperiodic processing• Complex dependencies• Continuous platform upgrades

• Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF• www.cs.wustl.edu/~schmidt/TAO-boeing.html

• Also used on SOFIA project by Raytheon• sofia.arc.nasa.gov

• First use of RT CORBA in mission computing• Drove Real-time CORBA standardization

• Test flown at China Lake NAWS by Boeing OSAT II ‘98, funded by OS-JTF• www.cs.wustl.edu/~schmidt/TAO-boeing.html

• Also used on SOFIA project by Raytheon• sofia.arc.nasa.gov

• First use of RT CORBA in mission computing• Drove Real-time CORBA standardization

Key Results

Goals• Apply COTS & open systems to mission-critical real-time avionics

13

Limitations with the Socket APIs (1/2)

Poorly structured, non-uniform, & non-portable

• API is linear rather than hierarchical

• i.e., the API is not structured according to the different phases of connection lifecycle management and the roles played by the participants

• No consistency among the names

• Non-portable & error-prone

• Function names: read() & write() used for any I/O handle on Unix but Windows needs ReadFile() & WriteFile()

• Function semantics: different behavior of same function on different OS e.g., accept () can take NULL client address parameter on Unix/Windows, but will crash on some operating systems, such as VxWorks

• Socket handle representations: different platforms represent sockets differently e.g., Unix uses unsigned integers whereas Windows uses pointers

• Header files: Different platforms use different names for header files for the socket API

14

Limitations with the Socket APIs (2/2)Lack of type safety• I/O handles are not amenable to strong type checking at compile time

• e.g., no type distinction between a socket used for passive listening & a socket used for data transfer

Steep learning curve due to complex semantics• Multiple protocol families & address families

• Options for infrequently used features such as broadcasting, async I/O, non blocking I/O, urgent data delivery

• Communication optimizations such as scatter-read & gather-write

• Different communication and connection roles, such as active & passive connection establishment, & data transfer

Too many low-level details• Forgetting to use the network byte order before data transfer

• Possibility of missing a function, such as listen()

• Possibility of mismatch between protocol & address families

• Forgetting to initialize underlying C structures e.g., sockaddr

• Using a wrong socket for a given role

15

1 #include <sys/types.h>

2 #include <sys/socket.h>

3

4 const int PORT_NUM = 10000;

5

6 int echo_server ()

7 {

8 struct sockaddr_in addr;

9 int addr_len;

10 char buf[BUFSIZ];

11 int n_handle;

12 // Create the local endpoint.

Example of Socket API Limitations (1/3)

Forgot to initialize to sizeof (sockaddr_in)

Use of non-portable handle type

Possible differences in header file names

16

13 int s_handle = socket (PF_UNIX, SOCK_DGRAM, 0);

14 if (s_handle == -1) return -1;

15

16 // Set up address information where server listens.

17 addr.sin_family = AF_INET;

18 addr.sin_port = PORT_NUM;

19 addr.sin_addr.addr = INADDR_ANY;

20

21 if (bind (s_handle, (struct sockaddr *) &addr,

22 sizeof addr) == -1)

23 return -1;

24

Use of non-portable return value

Unused structure members not zeroed out

Protocol and address family mismatch

Wrong byte order

Missed call to listen()

Example of Socket API Limitations (2/3)

17

25 // Create a new communication endpoint.

26 if (n_handle = accept (s_handle, (struct sockaddr *) &addr,

27 &addr_len) != -1) {

28 int n;

29 while ((n = read (s_handle, buf, sizeof buf)) > 0)

30 write (n_handle, buf, n);

31

32 close (n_handle);

33 }

34 return 0;

35 }

SOCK_DGRAM handle illegal here

Reading from wrong handle

No guarantee that “n” bytes will be written

Example of Socket API Limitations (3/3)

18

The Wrapper Facade Pattern (1/2)

Problem•The diversity of hardware & operating systems makes it hard to build portable & robust networked application software

•Programming directly to low-level OS APIs is tedious, error-prone, & non-portable

Context •Networked applications must manage a variety of OS services, including processes, threads, socket connections, virtual memory, & files

•OS platforms provide low-level APIs written in C to access these services

Win2K Linux LynxOS

Solaris VxWorks

Applications

19

The Wrapper Facade Pattern (2/2)

This pattern encapsulates data & functions provided by existing non-OO APIs within more concise, robust, portable, maintainable, & cohesive OO class interfaces

: Application

method()

: WrapperFacade

: APIFunctionA

functionA()

: APIFunctionB

functionB()

Applicationcalls methods

callsAPI FunctionA()

callsAPI FunctionB()

calls API FunctionC()

void methodN(){functionA();

}

void method1(){functionA();

}functionB();

Wrapper Facade

data

method1()…methodN()

Solution•Apply the Wrapper Facade design pattern to avoid accessing low-level operating system APIs directly

Solution•Apply the Wrapper Facade design pattern to avoid accessing low-level operating system APIs directly

20

ACE Socket Wrapper Facade Classes

These classes are designed in accordance

with the Wrapper Facade design pattern

ACE defines a set of C++ classes that address the limitations with the Socket API•Enhance type-safety•Ensure portability•Simplify common use cases

•Building blocks for higher-level abstractions

21

Roles in the ACE Socket Wrapper Facade

•The active connection role (ACE_SOCK_Connector) is played by a peer application that initiates a connection to a remote peer

•The passive connection role (ACE_SOCK_Acceptor) is played by a peer application that accepts a connection from a remote peer &

•The communication role (ACE_SOCK_Stream) is played by both peer applications to exchange data after they are connected

22

Connector/Acceptor: intent

Decouples active/passive connection establishment from the service performed

once the connection is established

Decouples active/passive connection establishment from the service performed

once the connection is established

23

The ACE_SOCK_Connector ClassMotivation

•There is a confusing asymmetry in the Socket API between (1) connection roles & (2) socket modes

•e.g., an application may accidentally call recv() or send() on a data-mode socket handle before it's connected

•This problem can't be detected until run time since C socket handles are weakly-typed

int buggy_echo_client (u_short port_num, const char *s) { int handle = socket (PF_UNIX, SOCK_DGRAM, 0); write (handle, s, strlen (s) + 1);

sockaddr_in s_addr; memset (&s_addr, 0, sizeof s_addr); s_addr.sin_family = AF_INET; s_addr.sin_port = htons (port_num); connect (handle, (sockaddr *) &s_addr, sizeof s_addr);}

Operations called in wrong order

24

The ACE_SOCK_Connector Class

Class Capabilities

•ACE_SOCK_Connector is factory that establishes a new endpoint of communication actively & provides capabilities to

•Initiate a connection with a peer acceptor & then to initialize an ACE_SOCK_Stream object after the connection is established

•Initiate connections in either a blocking, nonblocking, or timed manner

25

int main (int argc, char *argv[]) { const char *server_hostname = argv[1];

ACE_SOCK_Connector connector; ACE_SOCK_Stream peer; ACE_INET_Addr peer_addr; if (peer_addr.set (80, server_hostname) == -1) return 1; else if (connector.connect (peer, peer_addr) == -1) return 1;

Using the ACE_SOCK_Connector

• Block until connection established or connection request failure

• Instantiate the connector, data transfer, & address objects

•This example shows how the ACE_SOCK_Connector can be used to connect a client application to a Web server

26

The ACE_SOCK_Acceptor Class (1/2)

Motivation

•The C functions in the Socket API are weakly typed, which makes it easy to apply them incorrectly in ways that can’t be detected until run-time

•The ACE_SOCK_Acceptor class ensures type errors are detected at compile-time

int buggy_echo_server (u_short port_num) { sockaddr_in s_addr; int acceptor = socket (PF_UNIX, SOCK_DGRAM, 0); s_addr.sin_family = AF_INET; s_addr.sin_port = port_num; s_addr.sin_addr.s_addr = INADDR_ANY; bind (acceptor, (sockaddr *) &s_addr, sizeof s_addr); int handle = accept (acceptor, 0, 0); for (;;) { char buf[BUFSIZ]; ssize_t n = read (acceptor, buf, sizeof buf); if (n <= 0) break; write (handle, buf, n); }}

Reading from wrong handle

27

Class Capabilities

•This class is a factory that establishes a new endpoint of communication passively & provides the following capabilities:

The ACE_SOCK_Acceptor Class (2/2)

•It accepts a connection from a peer connector & then initializes an ACE_SOCK_Stream object after the connection is established

•Connections can be accepted in either a blocking, nonblocking, or timed manner

•C++ traits are used to support generic programming techniques that enable the wholesale replacement of functionality via C++ parameterized types

28

Using the ACE_SOCK_Acceptor

extern char *get_url_pathname (ACE_SOCK_Stream *);int main (){ ACE_INET_Addr server_addr; ACE_SOCK_Acceptor acceptor; ACE_SOCK_Stream peer;

if (server_addr.set (80) == -1) return 1; if (acceptor.open (server_addr) == -1) return 1;

for (;;) { if (acceptor.accept (peer) == -1) return 1; peer.disable (ACE_NONBLOCK); // Ensure blocking <send_n>.

ACE_Auto_Array_Ptr<char *> pathname (get_url_pathname (peer)); ACE_Mem_Map mapped_file (pathname.get ());

if (peer.send_n (mapped_file.addr (), mapped_file.size ()) == -1) return 1; peer.close (); }

return acceptor.close () == -1 ? 1 : 0;}

• Instantiate the acceptor, data transfer, & address objects

• Initialize a passive mode endpoint to listen for connections on port 80

• Accept a new connection

• Send the requested data

• Close the connection to the sender

• Stop receiving any connections

•This example shows how an ACE_SOCK_Acceptor & ACE_SOCK_Stream can be used to accept connections & send/receive data to/from a web client

29

Reactor: intent

Decouples event demultiplexing and event handler dispatching from application

services performed in response to events

Decouples event demultiplexing and event handler dispatching from application

services performed in response to events

30

*

Reactor

Reactor

handleEvents()registerHandler()removeHandler()

IEventHandler

handleEvent()getHandle()

ConcreteHandler

handleEvent()getHandle()Iterator handles = select();

while(handles.hasNext()) { Handle h = handles.next(); IEventHandler eh = table[h]; eh.handleEvent();}

31

Proactor: intent

Demultiplexes and dispatches service requests that are triggered by the completion

of asynchronous events.

Demultiplexes and dispatches service requests that are triggered by the completion

of asynchronous events.

32

<<create>>

*

Proactor

Proactorread()write()

IEventHandler

handleEvent()

ConcreteHandler

handleEvent()

AsyncIODevice

read()write()

CompletionDispatcher

completed()<<implements>>

33

Half-Sync/Half-Async: intent

Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without

degrading execution efficiency

Decouples synchronous I/O from asynchronous I/O in a system to simplify concurrent programming effort without

degrading execution efficiency

34

The Half-Sync/Half-Async Pattern

This pattern yields two primary benefits:

1.Threads can be mapped to separate CPUs to scale up server performance via multi-processing

2.Each thread blocks independently, which prevents a flow-controlled connection from degrading the QoS that other clients receive

SyncServiceLayer

AsyncService Layer

QueueingLayer

<<read/write>><<read/write>>

<<read/write>>

<<dequeue/enqueue>> <<interrupt>>

Sync Service 1 Sync Service 2 Sync Service 3

ExternalEvent Source

Queue

Async Service

35

Half-Sync/Half-Async Pattern Dynamics

•This pattern defines two service processing layers—one async & one sync—along with a queueing layer that allows services to exchange messages between the two layers

: External EventSource

: Async Service : Queue

notification

read()

enqueue()

message

: Sync Service

work()

message

read()

message

work()

notification

•The pattern allows sync services (such as processing log records from different clients) to run concurrently, relative both to each other & to async/reactive services (such as event demultiplexing)

36

Drawbacks with Half-Sync/Half-Async

Problems:•Overhead when crossing inter-layer boundary•Lack of support for Async. operations in the sync. Layer•Solution: Leader/Followers pattern

<<get>><<get>>

<<get>>

<<put>>

Worker Thread 1

Worker Thread 3

Event source

Request Queue

acceptorhandlers

Worker Thread 2

37

The ACE_TSS Class (2/2)

Class Capabilities•This class implements the Thread-Specific Storage pattern, which encapsulates & enhances the native OS Thread-Specific Storage (TSS) APIs to provide the following capabilities:

•It supports data that are ``physically'' thread specific, that is, private to a thread, but allows them to be accessed as though they were ``logically'' global to a program

•It uses the C++ delegation operator: operator->()

•It encapsulates the management of the keys associated with TSS objects

•For platforms that lack adequate TSS support natively (such as VxWorks) ACE_TSS emulates TSS efficiently

38

The Thread-Specific Storage Pattern• The Thread-Specific Storage pattern allows multiple threads to use one ‘logically global’ access point to retrieve an object that is local to a thread, without incurring locking overhead on each object access

key 1

key n

thread m

Thread-SpecificObjectaccesses

manages

[k,t]

thread 1

39

Using ACE_TSS (1/3)

template <class TYPE> TYPE *ACE_TSS<TYPE>::operator-> () { if (once_ == 0) { // Ensure that we're serialized. ACE_GUARD_RETURN(ACE_Thread_Mutex, guard, keylock_, 0);

if (once_ == 0) { ACE_OS::thr_keycreate(&key_); once_ = 1; } }

• In this implementation, each thread gets its own request count that resides in thread-specific storage to alleviate race conditions on the request count without requiring a mutex

• This example illustrates how to implement & apply ACE_TSS to our thread-per-connection logging server

We used the double-checked locking optimization pattern here

40

Using ACE_TSS (2/3)

TYPE *ts_obj = 0;

// Initialize <ts_obj> from thread-specific storage. ACE_OS::thr_getspecific (key_, (void **) &ts_obj);

// Check if this method's been called in this thread. if (ts_obj == 0) { // Allocate memory off the heap and store it in a pointer. ts_obj = new TYPE;

// Store the dynamically allocated pointer in TSS. ACE_OS::thr_setspecific (key_, ts_obj); } return ts_obj;}

41

Using ACE_TSS (3/3)class Request_Count {public: Request_Count (): count_ (0) {} void increment () { ++count_; } int value () const { return count_; }

private: int count_;};

static ACE_TSS<Request_Count> request_count;

virtual int handle_data (ACE_SOCK_Stream *) { while (logging_handler_.log_record () != -1) // Keep track of number of requests. request_count->increment ();

ACE_DEBUG ((LM_DEBUG, "request_count = %d\n", request_count->value ())); }

This call increments variable in thread-specific storage

42

Reminder: Using Window’s critical-section

CRITICAL_SECTION cs; // Global variable

void main() { if(!InitializeCriticalSection(&cs)) return; // Create threads...

DeleteCriticalSection(&cs)}

DWORD WINAPI ThreadProc(LPVOID lpParameter){ EnterCriticalSection(&cs);

// Access the shared resource.

LeaveCriticalSection(&cs);}

•Now, let’s think about a CriticalSection class…

•How should its interface look like?

43

The ACE Synchronization Wrapper Facades

•Different operating systems provide different synchronization mechanisms with different semantics using different APIs•Some of these APIs conform to international standards, such as Pthreads•Other APIs conform to de facto standards, such as Win32

•Below we describe the following ACE classes that networked applications can use to synchronize threads and/or processes portably

44

The ACE_Lock* Pseudo-Class

•The ACE mutex, readers/writer, semaphore, & file lock mechanisms all support the ACE_LOCK* interface shown below

•ACE_LOCK* is a “pseudo-class,” i.e., it's not a real C++ class in ACE

•We use it to illustrate the uniformity of the signatures supported by many of the ACE synchronization classes

•e.g., ACE_Thread_Mutex, ACE_Process_Mutex, & ACE_Thread_Semaphore

45

The ACE_Guard ClassesMotivation

•When acquiring and releasing locks explicitly, it can be hard to ensure that all paths through the code release the lock, especially when C++ exceptions are thrown

•ACE provides the ACE_Guard class & its associated subclasses to help assure that locks are acquired & released properly

Class Capabilities•These classes implement the Scoped Locking idiom, which leverages the semantics of C++ class constructors & destructors to ensure a lock is acquired & released automatically upon entry to and exit from a block of C++ code, respectively

46

The Scoped Locking IdiomMotivation•Code that shouldn’t execute concurrently must be protected by some type of lock that is acquired/released when control enters/leaves a critical section

•If programmers must acquire & release locks explicitly, it is hard to ensure that the locks are released in all paths through the code

•e.g., in C++ control can leave a scope due to a return, break, continue, or goto statement, as well as from an unhandled exception being propagated out of the scopevoid method () { lock_.acquire (); // The implementation may return prematurely… lock_.release (); } •The Scoped Locking idiom defines a guard class whose constructor

automatically acquires a lock when control enters a scope & whose destructor automatically releases the lock when control leaves the scope void method () { ACE_Guard <ACE_Thread_Mutex> guard (lock_); // The lock is released when the method returns}

47

Implementing Scoped Locking in ACEtemplate <class LOCK> class ACE_Guard {public: // Store a pointer to the lock and acquire the lock. ACE_Guard (LOCK &lock) : lock_ (&lock) { lock_->acquire (); }

// Release the lock when the guard goes out of scope, ~ACE_Guard () { lock_->release (); }

// Other methods omitted…

private: // Pointer to the lock we’re managing. LOCK *lock_;};

Generic ACE_Guard Wrapper Facade

•ACE_Write_Guard & ACE_Read_Guard acquire write locks & read locks, respectively

•Instances of the ACE_Guard<T> classes can be allocated on the run-time stack to acquire & release locks in method or block scopes that define critical sections

48

ACE Condition Variable Classes (1/2)Motivation

•Condition variables allow threads to coordinate & schedule their processing efficiently

•Condition variables are more appropriate than mutexes or semaphores when complex condition expressions or scheduling behaviors are needed

•e.g., condition variables are often used to implement synchronized message queues that provide “producer/consumer” communication to pass messages between threads

usesuses 2

Request Queue

put()get()

ACE_Thread_Mutex

acquire()release()

Producer Thread

ACE_Thread_Condition

wait()signal()broadcast()

ConsumerThread

<<put>> <<get>>

49

Class Capabilities

•The ACE_Condition_Thread_Mutex uses the Wrapper Façade pattern to guide its encapsulation of process-scoped condition variable semantics

•The ACE_Null_Condition is a zero-cost class whose interface conforms to the ACE_Condition_Thread_Mutex

ACE Condition Variable Classes (2/2)