patterns and performance of real-time high …schmidt/pdf/tao4.pdf · patterns and performance of...
TRANSCRIPT
Patterns and Performance of Real-timeObject Request Brokers
Douglas C. SchmidtAssociate Professor Elec. & Comp. Eng. [email protected] University of California, Irvinewww.ece.uci.edu/�schmidt/ (949) 824-1901
SponsorsNSF, DARPA, ATD, BBN, Boeing, Cisco, Comverse, GDIS, Experian, Global MT,
Hughes, Kodak, Krones, Lockheed, Lucent, Microsoft, Mitre, Motorola, NASA, Nokia,Nortel, OCI, Oresis, OTI, Raytheon, SAIC, Siemens SCR, Siemens MED, Siemens
ZT, Sprint, Telcordia, USENIX
High-performance, Real-time ORBs Douglas C. Schmidt
Motivation: the QoS-enabled Software Crisis
www.arl.wustl.edu/arl/� Symptoms
– Communication hardware getssmaller, faster, cheaper
– Communication software gets larger,slower, more expensive
� Culprits
– Inherent and accidental complexity
� Solution Approach
– Standards-based COTS Hardware &Software
UC Irvine 1
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: the COTS Hardware & Software Crisis
MISSIONMISSION--CRITICALCRITICAL
REALREAL--TIMETIMESYSTEMSSYSTEMS
COTSCOTSHARDWAREHARDWARE & &
SOFTWARESOFTWARE
� Context
– Adopting COTS hardware &software is increasingly essential forreal-time mission-critical systems
� Problems
– Inherent and accidental complexity– Integration woes
� Solution Approach
– Standards-based adaptive COTSmiddleware
UC Irvine 2
High-performance, Real-time ORBs Douglas C. Schmidt
Context: Levels of Abstraction inInternetworking and Middleware
HTTPFTP
FDDIATMETHERNET
IP
FIBRE CHANNEL
UDP TCP
TELNET
DNS
LYNXOSLINUXWIN NT
CORBA
SOLARIS
CORBA SERVICES
CORBA
APPLICATIONS
VXWORKS
MIDDLEWARE ARCHINTERNETWORKING ARCH
RTP
UC Irvine 3
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Lack of QoS-enabled Middleware
INFRASTRUCTUREMIDDLEWARE
DISTRIBUTIONMIDDLEWARE
COMMONMIDDLEWARE
SERVICES
APPLICATIONS
Cons
ConsConsEVENTEVENT
CHANNELCHANNEL
OPERATINGOPERATINGSYSTEMS SYSTEMS &&PROTOCOLSPROTOCOLS
HARDWAREHARDWARE DEVICESDEVICES
� Many applications require QoSguarantees
– e.g., avionics, telecom, WWW,medical, high-energy physics
� Building these applicationsmanually is hard and inefficient
� Existing middleware doesn’tsupport QoS effectively
– e.g., CORBA, DCOM, DCE, Java
� Solutions must be integratedhorizontally & vertically
UC Irvine 4
High-performance, Real-time ORBs Douglas C. Schmidt
Candidate Solution: CORBA
INTERFACE
REPOSITORY
IMPLEMENTATION
REPOSITORY
IDLCOMPILER
DII ORBINTERFACE
ORBORB CORECORE GIOPGIOP//IIOPIIOP//ESIOPSESIOPS
IDLIDLSTUBSSTUBS
operation()operation()in args
out args + return value
CLIENTOBJECT(SERVANT)
OBJ
REF
STANDARD INTERFACE STANDARD LANGUAGE MAPPING
ORB-SPECIFIC INTERFACE STANDARD PROTOCOL
INTERFACE
REPOSITORY
IMPLEMENTATION
REPOSITORY
IDLCOMPILER
IDLSKELETON
DSI
OBJECT
ADAPTER
www.cs.wustl.edu/�schmidt/corba.html
Goals of CORBA
� Simplify distributionby automating
– Object location &activation
– Parametermarshaling
– Demultiplexing– Error handling
� Provide foundationfor higher-levelservices
UC Irvine 5
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Cav
eat:
Req
uire
men
ts/L
imita
tions
ofC
OR
BA
for
QoS
-ena
bled
Sys
tem
s
NE
TW
OR
KN
ET
WO
RK
OP
ER
AT
ION
SO
PE
RA
TIO
NS
CE
NT
ER
CE
NT
ER
HSM
HSM
AR
CH
IVE
AR
CH
IVE
SE
RV
ER
SE
RV
ER
AG
EN
TA
GE
NT
INT
ER
AC
TIV
EIN
TE
RA
CT
IVE
AU
DIO
AU
DIO
//VID
EO
VID
EO
AG
EN
TA
GE
NT
AR
CH
ITE
CT
UR
EA
RC
HIT
EC
TU
RE
SP
CS
PC
HA
RD
WA
RE
HA
RD
WA
RE
EM
BE
DD
ED
EM
BE
DD
ED
TA
OT
AO
MIB
MIB
AG
EN
TA
GE
NT
ww
w.c
s.w
ustl.
edu/
�
schm
idt/R
T-O
RB
.ps.
gz
Req
uire
men
ts
�
Loca
tion
tran
spar
ency
�
Per
form
ance
tran
spar
ency
�
Pre
dict
abili
tytr
ansp
aren
cy
�
Rel
iabi
lity
tran
pare
ncy
Lim
itatio
ns
�
Lack
ofQ
oSsp
ecifi
catio
ns
�
Lack
ofQ
oSen
forc
emen
t
�
Lack
ofre
al-t
ime
prog
ram
min
gfe
atur
es
�
Lack
ofpe
rform
ance
optim
izat
ions
UC
Irvi
neHigh-performance, Real-time ORBs Douglas C. Schmidt
Overview of the Real-time CORBA Specification
OS KERNEL
OS I/O SUBSYSTEM
NETWORK ADAPTERS
STANDARD
SYNCHRONIZERS
END-TO-END PRIORITY
PROPAGATION
ORB CORE
OBJECT ADAPTER
CLIENT
GIOP
PROTOCOL
PROPERTIES
THREAD
POOLSEXPLICIT
BINDING
NETWORK
OS KERNEL
OS I/O SUBSYSTEM
NETWORK ADAPTERS
operation()
out args + return value
in args
OBJECT
REF
OBJECT
(SERVANT)
STUBSSKELETON
Features
1. Portable priorities
2. End-to-end prioritypropagation
3. Protocol properties
4. Thread pools andbuffering
5. Explicit binding
6. Standard synchronizers
www.cs.wustl.edu/�schmidt/oorc.ps.gz
UC Irvine 7
High-perform
ance,R
eal-time
OR
Bs
Douglas
C.S
chmidt
Portable
Priorities
OR
B E
ND
SY
ST
EM
A
32
76
7
0 RTCORBA::Priority
0
255
OR
B E
ND
SY
ST
EM
B
0
31
Features
�
Designed
tosupport
heterogeneousreal-tim
eplatform
s
�
CO
RB
Apriorities
rangefrom
0
!
32767
�
Users
canm
apC
OR
BA
prioritiesto
nativeO
Spriorities
�
No
silverbullet,butrather
an“enabling
technique”
UC
Irvine8
High-performance, Real-time ORBs Do
End-to-End Priority Propagation
GLOBAL CORBA PRIORITY = 100
LYNXOS
PRIORITY
= 100
WINNT
PRIORITY
= 5
SOLARIS
PRIORITY
= 135
Current::priority(100)
to_native(100) => 100
Current::priority(100)
to_native(100) => 5
Current::priority(100)
to_native(100) => 135
SERVICECONTEXT
= 100ORBENDSYSTEM
A
ORBENDSYSTEM
B
ORBENDSYSTEM
C
(B) CLIENT
PROPAGATED
MODEL
SERVICECONTEXT
= 100
(A) SERVER
DECLARED
MODEL
ORBENDSYSTEM
A(3) CLIENT'S PRIORITY
IS NOT PROPAGATED
BY INVOCATION
(1) SERVER
PRIORITY
IS PRE-SET
ORBENDSYSTEM
B
(2) PRIORITY IS
EXPORTED IN IOR
Features
� Client priorities can propagate end-to-end
� Servers can also declare priority
UC Irvine
High-perform
ance,R
eal-time
OR
Bs
Douglas
C.S
chmidt
Configurable
ProtocolP
roperties
inte
rface
Pro
toco
lPro
pe
rties
{};
type
de
fstru
ct{
IOP
::Pro
fileId
pro
toco
l_typ
e;
Pro
toco
lPro
pe
rties
orb
_p
roto
col_
pro
pe
rties;
Pro
toco
lPro
pe
rties
tran
spo
rt_p
roto
col_
pro
pertie
s;}
Pro
toco
l;typ
ed
ef
seq
ue
nce
<P
roto
col>
Pro
toco
lList;
inte
rface
TC
PP
roto
colP
rop
ertie
s:
Pro
toco
lPro
pe
rties
{a
ttribu
telo
ng
sen
d_
bu
ffer_
size;
attrib
ute
lon
gre
cv_b
uffe
r_size
;a
ttribu
teb
oo
lea
nke
ep
_a
live;
attrib
ute
bo
ole
an
do
nt_
rou
te;
attrib
ute
bo
ole
an
no
_d
ela
y;};
Features
�
Selectand
configurecom
munication
protocols
–e.g.,T
CP
socketoptions
�
Supports
OR
Bprotocol
andtransportprotocol
configuration
�
Ordering
inP
roto
colL
istindicates
preferences
UC
Irvine10
High-performance, Real-time ORBs Do
Protocol Selection and Configuration
NETWORKOS KERNEL
ATMVME RTP
OS KERNEL
ORBCORE
CLIENTOBJECT(SERVANT)
2: PASS
OBJECT
REFERENCE
OBJECT ADAPTER
ATMVME TCP
4: INVOKE OPERATION 1: CREATE OBJECT
REFERENCE3: SELECT
PROTOCOL
� Protocol policies control protocol selection andconfiguration
– Order of protocols indicates protocolpreference
� Both server-side and client-side policiessupported
– Some policies control protocol selection,others control protocol configuration
– Some policies are exported to client in objectreference
UC Irvine
High-performance, Real-time ORBs Do
Thread Pools
SERVER ORB COREI/OTHREADS
Root POA
Thread Pool A
PRIORITY
10PRIORITY
35PRIORITY
50PRIORITY
20
Thread Pool B
DEFAULT
PRIORITY
DefaultThread Pool
S3DEFAULT
S1DEFAULT
S2DEFAULT
POA A
S410
S550
S650
S735 POA B
S8 S9 S10
POA C
S1120
S1225
S1315
� Pre-allocate threads and thread attributes
– Stacksize– Static threads and maximum threads– Default priority
UC Irvine
High-performance, Real-time ORBs Do
Buffering Requests
SERVER ORB COREI/OTHREADS
Thead Pool A
PRIORITY 10 PRIORITY 35 PRIORITY 20
Thead Pool B
Requests are bufferedwhen all threads arebusyBuffering can bespecified in terms of:
� Number of bytes
� Number of requests
When buffers are full:
� A transient exception isthrown to client
� Request is dropped byserver
� Request can bereissued later by client
UC Irvine
High-performance, Real-time ORBs Do
Explicit Binding
_validate_connection (out CORBA::PolicyListinconsistent_policies);
CLIENTORB CORE
P1-5 P10-20 P21-100
SERVERORB CORE
PRIORITY-BANDEDPRIVATE CONNECTIONS
P1-5 P10-20 P21-100
Features
� Enables pre-establishment of connections
– Priority-banded connections– Private connections– Protocol policies
UC Irvine
High-performance, Real-time ORBs Do
Standard Synchronizers
CLIENT
Mutexlock()
unlock()
try_lock()
ORB CORE
OBJECT
ADAPTER
OBJECT(SERVANT)
mutex3
mutex2
mutex4
mutex1
Features
� A portable Mutex API
– e.g., lock , unlock , try lock
� Necessary to ensure consistency between ORBand application synchronizers
– e.g., priority inheritance and priority ceilingprotocols
� Locality constrained
UC Irvine
High-performance, Real-time ORBs Douglas C. Schmidt
Additional Information on Real-time CORBA
� Real-time CORBA 1.0 specification
– www.cs.wustl.edu/�schmidt/RT-ORB-std-new.pdf.gz
� Many papers at my Web site
– www.cs.wustl.edu/�schmidt/corba-research-realtime.html
� Upcoming OMG Real-time and Embedded CORBA Workshop
– www.omg.org/meetings/realtime/
� Real-time ORBs
– HighComm ! www.highcomm.com– ORB Express ! www.ois.com– TAO ! www.theaceorb.com
UC Irvine 16
High-performance, Real-time ORBs Douglas C. Schmidt
Our Approach: The ACE ORB (TAO)
NETWORKNETWORK
ORBORB RUN RUN--TIMETIME
SCHEDULERSCHEDULER
operation()operation()
IDLIDLSTUBSSTUBS
IDLIDLSKELETONSKELETON
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK INTERFACENETWORK INTERFACE
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
OBJECTOBJECT((SERVANTSERVANT))
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK INTERFACENETWORK INTERFACE
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
ACEACE COMPONENTSCOMPONENTS
OBJOBJ
REFREF
REALREAL--TIMETIME ORBORB CORECOREIOPIOP
PLUGGABLEPLUGGABLEORBORB & & XPORTXPORTPROTOCOLSPROTOCOLS
IOPIOPPLUGGABLEPLUGGABLE
ORBORB & & XPORTXPORTPROTOCOLSPROTOCOLS
REALREAL--TIMETIME
OBJECTOBJECT
ADAPTERADAPTER
www.cs.wustl.edu/�schmidt/TAO.html
TAO Overview !
� An open-source,standards-based,real-time,high-performanceCORBA ORB
� Runs onPOSIX/UNIX,Win32, & RTOSplatforms– e.g., VxWorks,
Chorus, LynxOS
� Leverages ACE
UC Irvine 17
High-performance, Real-time ORBs Douglas C. Schmidt
The ADAPTIVE Communication Environment (ACE)
PROCESSES//THREADSTHREADS
DYNAMICDYNAMIC
LINKINGLINKING
MEMORYMEMORY
MAPPINGMAPPING
SELECTSELECT//IO COMPIO COMP
SYSTEMSYSTEM
VV IPCIPCSTREAMSTREAM
PIPESPIPES
NAMEDNAMED
PIPESPIPES
CAPISS
SOCKETSSOCKETS//TLITLI
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUAL MEMORYVIRTUAL MEMORY
SUBSYSTEMSUBSYSTEM
GENERAL POSIX AND WIN32 SERVICES
PROCESSPROCESS//THREADTHREAD
SUBSYSTEMSUBSYSTEM
FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR
SELF-CONTAINED
DISTRIBUTED
SERVICE
COMPONENTS
NAMENAME
SERVERSERVER
TOKENTOKEN
SERVERSERVER
LOGGINGLOGGING
SERVERSERVER
GATEWAYGATEWAY
SERVERSERVER
SOCKSOCK__SAPSAP//TLITLI__SAPSAP
FIFOFIFO
SAPSAP
LOGLOG
MSGMSG
SERVICESERVICE
HANDLERHANDLER
TIMETIME
SERVERSERVER
C++WRAPPER
FACADES
SPIPESPIPE
SAPSAP
CORBACORBA
HANDLERHANDLER
SYSVSYSVWRAPPERSWRAPPERS
SHAREDSHARED
MALLOCMALLOC
THE ACE ORBTHE ACE ORB
((TAOTAO))
JAWS ADAPTIVEJAWS ADAPTIVE
WEB SERVERWEB SERVER
MIDDLEWARE
APPLICATIONS
REACTORREACTOR//PROACTORPROACTOR
PROCESSPROCESS//THREADTHREAD
MANAGERSMANAGERS
STREAMSSTREAMS
SERVICESERVICE
CONFIGCONFIG--URATORURATOR
SYNCHSYNCH
WRAPPERSWRAPPERS
MEMMEM
MAPMAP
OS ADAPTATION LAYER
www.cs.wustl.edu/�schmidt/ACE.html
ACE Overview !
� A concurrentOO networkingframework
� Available inC++ and Java
� Ported toPOSIX, Win32,and RTOSs
Related work !
� x-Kernel
� SysVSTREAMS
UC Irvine 18
High-performance, Real-time ORBs Douglas C. Schmidt
ACE and TAO Statistics� Over 50 person-years of effort
– ACE > 200,000 LOC– TAO > 200,000 LOC– TAO IDL compiler > 130,000
LOC– TAO CORBA Object Services >
150,000 LOC
� Ported to UNIX, Win32, MVS, andRTOS platforms
� Large user community
– �schmidt/ACE-users.html
� Currently used by dozens ofcompanies
– Bellcore, BBN, Boeing,Ericsson, Hughes, Kodak,Lockheed, Lucent,Motorola, Nokia, Nortel,Raytheon, SAIC, Siemens,etc.
� Supported commercially
– ACE ! www.riverace.com– TAO !
www.theaceorb.com
UC Irvine 19
High-performance, Real-time ORBs Douglas C. Schmidt
Applying TAO to Avionics Mission Computing
REPLICATION
SERVICE
OBJECT REQUEST BROKER
1: SENSORS
GENERATE
DATA
FLIRGPS IFF
3:PUSH (EVENTS)
2: SENSOR PROXIES DEMARSHAL DATA
& PASS TO EVENT CHANNEL
3:PUSH (EVENTS)
EVENT
CHANNEL
HUD NavAir
FrameWTS
4: PULL(DATA)
www.cs.wustl.edu/�schmidt/JSAC-98.ps.gz
Domain Challenges
� Deterministic & statisticalreal-time deadlines
� Periodic & aperiodic processing
� COTS and open systems
� Reusable components
� Support platform upgrades
www.cs.wustl.edu/�schmidt/TAO-boeing.html
UC Irvine 20
High-performance, Real-time ORBs Douglas C. Schmidt
Applying TAO to Distributed Interactive Simulations
www.cs.wustl.edu/�schmidt/Words99.ps.gz
Domain Challenges
� High scalability and groupcommunication
� High throughput and lowlatency
� “Interactive” real-time
� Multi-platform
hlasdc.dmso.mil/RTISUP/hla soft/hla soft.htm
UC Irvine 21
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
App
lyin
gTA
Oto
Em
bedd
edN
etw
ork
Ele
men
tMan
agem
enta
ndC
ontro
lM
AN
AG
EM
EN
T
CLI
EN
T
RIO
TA
ON
ET
WO
RK
O
PE
RA
TIO
NS
C
EN
TE
R
VS
I
MA
ST
ER
RIO
TA
OS
IGN
AL
ING
P
RO
CE
SS
OR
VS
I
MA
ST
ER
RIO
TA
OS
IGN
AL
ING
P
RO
CE
SS
OR
VS
I
SLA
VE
CO
NT
RO
L
PR
OC
ES
SO
R
TA
OR
IO
WU
GS
WU
GS
WU
GS
EN
DS
YS
TE
M A
UN
I
CL
IEN
T
TA
O
CO
NN
EC
T
RE
QU
ES
T
EN
DS
YS
TE
M A
UN
I
CL
IEN
TTA
O
AT
MS
WIT
CH
AT
MS
WIT
CH
AT
MS
WIT
CH
VS
I
SLA
VE
CO
NT
RO
L
PR
OC
ES
SO
R
TA
OR
IO
VS
I
SLA
VE
CO
NT
RO
L
PR
OC
ES
SO
R
TA
OR
IO
AD
D C
ON
NE
CT
ION GE
T S
TA
TS
EV
EN
T
PO
RT
DO
WN
AD
D C
ON
NE
CT
ION
GE
T S
TA
TS
AD
D C
ON
NE
CT
ION
CO
NN
EC
T
RE
QU
ES
T
INT
ER
-OR
B P
RO
TO
CO
L
Dom
ain
Cha
lleng
es
�
Hig
h-sp
eed
(20
Gbp
s)AT
Msw
itche
sw
/em
bedd
edco
ntro
llers
�
Low
-late
ncy
and
stat
istic
alre
al-t
ime
dead
lines
�
CO
TS
infr
astr
uctu
re,s
tand
ards
-bas
edop
ensy
stem
s,an
dsm
allf
ootp
rint
UC
Irvi
neHigh-performance, Real-time ORBs Douglas C. Schmidt
Optimization Challenges for QoS-enabled ORBs
ORBORBCORECORE
PLUGGABLEPLUGGABLEPROTOCOLSPROTOCOLS
PLUGGABLEPLUGGABLEPROTOCOLSPROTOCOLS
OS KERNELOS KERNEL
OS IOS I//O SUBSYSTEMO SUBSYSTEM
NETWORK INTERFACESNETWORK INTERFACES
OS KERNELOS KERNEL
OS IOS I//O SUBSYSTEMO SUBSYSTEM
NETWORK INTERFACESNETWORK INTERFACES
NETWORKNETWORK
ORBORBINTERFACEINTERFACE
operation()operation()
IDLIDLSTUBSSTUBS
OBJECTOBJECT
ADAPTERADAPTER
IDLIDLSKELETONSKELETON
in argsin args
out args + return valueout args + return value
11
22
33
44
55
66
77
1)1) CLIENT MARSHALING CLIENT MARSHALING
2)2) CLIENT PROTOCOL CLIENT PROTOCOL
3)3) NETWORK LATENCY NETWORK LATENCY
4)4) SERVER PROTOCOL SERVER PROTOCOL
5) THREAD DISPATCHING
6) REQUEST DEMUXING
7) OPERATION DEMUXING
8) SERVANT DEMARSHALING
OBJECTOBJECT((SERVANTSERVANT))
88
CLIENTCLIENT
IOPIOP IOPIOP
Key Challenges
� Alleviate priority inversionand non-determinism
� Reduce demultiplexinglatency/jitter
� Ensure protocol flexibility
� Specify QoS requirements
� Schedule operations
� Eliminate (de)marshalingoverhead
� Minimize footprint
UC Irvine 23
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Optimizing Complex Software
DIAGNOSTIC STATIONSDIAGNOSTIC STATIONS
ATMATMMANMAN
ATM
LAN
ATM
LAN
MODALITIES
(CT, MR, CR) CENTRAL
BLOB STORE
CLUSTER
BLOB
STORE
DX
BLOB
STORE
www.cs.wustl.edu/�schmidt/JSAC-99.ps.gz
Common Problems !
� Optimizing complex softwareis hard
� Small “mistakes” can be costly
Solution Approach (Iterative) !
� Pinpoint overhead viawhite-box metrics– e.g., Quantify and
VMEtro
� Apply patterns and frameworkcomponents
� Revalidate via white-box andblack-box metrics
UC Irvine 24
High-performance, Real-time ORBs Douglas C. Schmidt
Solution 1: Patterns and Framework Components
ACCEPTORCONNECTOR
ABSTRACT
FACTORY
SERVANTCLIENT
OS KERNELOS KERNEL
ACTIVE
OBJECT
THREAD-SPECIFIC
STORAGE
SERVICE
CONFIGURATOR
STRATEGY
REACTORREACTOR
WRAPPER FACADESWRAPPER FACADES
www.cs.wustl.edu/�schmidt/ORB-patterns.ps.gz
Definitions
� Pattern
– A solution to a problem ina context
� Framework
– A “semi-complete”application built withcomponents
� Components
– Self-contained, “pluggable”ADTs
UC Irvine 25
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Sol
utio
n2:
OR
BO
ptim
izat
ion
Prin
cipl
eP
atte
rns
Defi
nitio
n
�
Opt
imiz
atio
npr
inci
ple
patte
rns
docu
men
tru
les
for
avoi
ding
com
mon
desi
gnan
dim
plem
enta
tion
prob
lem
sth
atca
nde
grad
eth
eef
ficie
ncy,
scal
abili
ty,a
ndpr
edic
tabi
lity
ofco
mpl
exsy
stem
s
Opt
imiz
atio
nP
rinci
ple
Pat
tern
sU
sed
inTA
O
#O
ptim
izat
ion
Prin
cipl
eP
atte
rn1
Opt
imiz
efo
rth
eco
mm
onca
se2
Rem
ove
grat
uito
usw
aste
3R
epla
cein
effic
ient
gene
ral-p
urpo
sefu
nctio
nsw
ithef
ficie
ntsp
ecia
l-pur
pose
ones
4S
hift
com
puta
tion
intim
e,e.
g.,p
reco
mpu
te5
Sto
rere
dund
ants
tate
tosp
eed-
upex
pens
ive
oper
atio
ns6
Pas
shi
nts
betw
een
laye
rsan
dco
mpo
nent
s7
Don
’tbe
tied
tore
fere
nce
impl
emen
tatio
ns/m
odel
s8
Use
effic
ient
/pre
dict
able
data
stru
ctur
es
UC
Irvi
neHigh-performance, Real-time ORBs Douglas C. Schmidt
ORB Latency and Priority Inversion Experiments
������
C 1C 0
Requests
C n
�� ����������������������������
Client
...
...
2ATM Switch
Ultra 2 Ultra 2
I/O SUBSYSTEM
Server
Object Adapter
ORB Core
Servants RU
NT
IME
SCH
ED
UL
ER
www.cs.wustl.edu/�schmidt/RT-perf.ps.gz
Method
� Vary ORBs, hold OSconstant
� Solaris real-time threads
� High priority client C0
connects to servant S0
with matching priorities
� Clients C1 : : : Cn havesame lower priority
� Clients C1 : : : Cn
connect to servant S1
� Clients invoke two-wayCORBA calls that cube anumber on the servantand returns result
UC Irvine 27
High-performance, Real-time ORBs Douglas C. Schmidt
ORB Latency and Priority Inversion Results
0
4
8
12
16
20
24
28
32
36
40
44
48
52
1 5 10 15 20 25 30 35 40 45 50
Number of Low Priority Clients
Late
ncy p
er
Tw
o-w
ay R
eq
uest
in M
illiseco
nd
s
CORBAplus High Priority CORBAplus Low Priority
MT-ORBIX High Priority MT-ORBIX Low Priority
miniCOOL High Priority miniCOOL Low Priority
TAO High Priority TAO Low Priority
Synopsis of Results
� TAO’s latency is lowest forlarge # of clients
� TAO avoids priority inversion
– i.e., high priority clientalways has lowest latency
� Primary overhead stemsfrom concurrency andconnection architecture
– e.g., synchronization andcontext switching
UC Irvine 28
High-performance, Real-time ORBs Douglas C. Schmidt
ORB Jitter Results
1 5 10 15 20 25 30 35 40 45 50
TAO High Priority
TAO Low Priority
miniCOOL High Priority
miniCOOL Low Priority
MT-ORBIX High Priority
MT-ORBIX Low Priority
CORBAplus High Priority
CORBAplus Low Priority
0
50
100
150
200
250
300
Jitt
er in
mill
isec
on
ds
Number of Low Priority Clients
Definition
� Jitter ! standarddeviation fromaverage latency
Synopsis ofResults
� TAO’s jitter islowest and mostconsistent
� CORBAplus’ jitteris highest andmost variable
UC Irvine 29
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Improper ORB Concurrency Models
OBJECTOBJECTADAPTERADAPTER
SERVANTSERVANT DEMUXERDEMUXER
SERVANTSERVANTSKELETONSSKELETONS
U
ORBORB CORECORE
FILTERFILTER
FILTERFILTER
FILTERFILTER
SERVANTSERVANTSKELETONSSKELETONS
U
2: enqueue(data)2: enqueue(data)
3: dequeue,3: dequeue,filterfilter
request,request,&enqueue&enqueue
4: dispatch4: dispatchupcall()upcall()
II//OO SUBSYSTEMSUBSYSTEM
THREADTHREAD
FILTERFILTER
1: recv()1: recv()
CONNECTION THREADSCONNECTION THREADS
Common Problems
� High context switching andsynchronization overhead
� Thread-level andpacket-level priorityinversions
� Lack of application controlover concurrency model
www.cs.wustl.edu/�schmidt/CACM-arch.ps.gz
UC Irvine 30
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: ORB Shared Connection Models
SERVERSERVERORBORB CORECORE
II//OO SUBSYSTEMSUBSYSTEM
II//OO SUBSYSTEMSUBSYSTEM
COMMUNICATIONCOMMUNICATION LINKLINK
II//OO SUBSYSTEMSUBSYSTEM
SERVANTSSERVANTS
ONEONE TCPTCP
CONNECTIONCONNECTION
ORBORB CORECORE
SEMAPHORESSEMAPHORES
2: select()2: select()4: release()
3: read()B
OR
RO
WE
D T
HR
EA
D
BO
RR
OW
ED
TH
RE
AD
S
APPLICATION
5: dispatch_return()
1: invoke_twoway()
www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz
Common Problems
� Request-levelpriority inversions
– Sharing multiplepriorities on asingle connection
� Complex connectionmultiplexing
� Synchronizationoverhead
UC Irvine 31
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: High Locking Overhead
0
94
199
175
0
231216
599
0
100
200
300
400
500
600
700
TAO miniCOOL CORBAplus MT ORBIX
ORBs Tested
Use
r L
evel
Lo
ck O
per
atio
ns
per
Req
ues
t
client server
Common Problems
� Locking overhead affectslatency and jitter significantly
� Memory managementcommonly involves locking
www.cs.wustl.edu/�schmidt/RTAS-98.ps.gz
UC Irvine 32
High-performance, Real-time ORBs Douglas C. Schmidt
Solution: TAO’s ORB Endsystem Architecture
RR
UU
NN
TT
II
MM
EE
S S
CC
HH
EE
DD
UU
LL
EE
RR
HIGH-SPEED NETWORKINTERFACES(e.g., APIC, VME)
ZZ
EE
RR
OO
CC
OO
PP
YY
BB
UU
FF
FF
EE
RR
SSRTRT I/O I/OSUBSYSTEMSUBSYSTEM
RTRT OBJECTOBJECTADAPTERADAPTER
OO(1) (1) REQUESTREQUEST DEMUXERDEMUXER
CLIENTSCLIENTS
STUBSSTUBS
SERVANTSSERVANTS
SKELETONSSKELETONS
U
RTRT ORBORB CORECORE
REACTORREACTOR
((PP11))
REACTORREACTOR
((PP22))
REACTORREACTOR
((PP33))
REACTORREACTOR
((PP44))
SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER
PLUGGABLEPLUGGABLE PROTOCOLSPROTOCOLS
Solution Approach !
� Integrate scheduler into ORBendsystem
� Co-schedule threads
� Leader/followers thread pool
Principle Patterns !
� Pass hints, precompute,optimize common case,remove gratuitous waste,store state, don’t be tied toreference implementations &models
UC Irvine 33
High-performance, Real-time ORBs Douglas C. Schmidt
Thread Pool Comparison Results
SERVANTSSERVANTS
ORBORB CORECORE
1: select()1: select()
II//OO SUBSYSTEMSUBSYSTEM
5: dispatch upcall()5: dispatch upcall()
2: read()2: read()
3: enqueue()3: enqueue()
4: dequeue()4: dequeue()
II//OOTHREADTHREAD
WORKER THREADSWORKER THREADS
REQUESTREQUEST
QUEUEQUEUE
Worker Thread Pool
SERVANTSSERVANTS
ORBORB CORECORE
SEMAPHORESEMAPHORE
LEADERLEADER FOLLOWERSFOLLOWERS
1: select()1: select()3: release()3: release()
II//OO SUBSYSTEMSUBSYSTEM
4: dispatch upcall()4: dispatch upcall()
2: read()2: read()
Leader/FollowerThread Pool
1 2 3 45
6
0
25
5075
100
0
500
1000
1500
2000
2500
3000
Pe
rfo
rma
nc
e Im
pro
ve
me
nt
Threads
Lo
ad
UC Irvine 34
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Reducing Demultiplexing Latency
OP
ER
AT
ION1
OP
ER
AT
ION2
OP
ER
AT
IONK
2: DEMUX TO
I/O HANDLE
...
...
...POA1
OS I/O SUBSYSTEM
NETWORK ADAPTERS
SERVANT 1
5: DEMUX TO
SKELETON
6: DISPATCH
OPERATION
1: DEMUX THRU
PROTOCOL STACK
4: DEMUX TO
SERVANT
ORB COREORB CORE
ROOT POA3: DEMUX TO
OBJECT
ADAPTER
POA2 POAN
SERVANT N
...SKEL 1 SKEL 2 SKEL N
SERVANT 2
OS
KERNEL
LAYER
ORB
LAYER
SERVANT
LAYER
Design Challenges
� Minimize demuxing layers
� Provide O(1) operationdemuxing through all layers
� Avoid priority inversions
� Remain CORBA-compliant
www.cs.wustl.edu/�schmidt/POA.ps.gz
UC Irvine 35
High-perform
ance,R
eal-time
OR
Bs
Douglas
C.S
chmidt
Solution:
TAO
’sR
equestDem
ultiplexingO
ptimizations
.........
.........
IDL
IDL
SK
EL
S
KE
L 22
OB
JEC
T A
DA
PT
ER S
ER
VA
NT
S
ER
VA
NT
500
500
(A)
(A) L
AY
ER
ED
DE
MU
XIN
G L
AY
ER
ED
DE
MU
XIN
G,,
LIN
EA
R S
EA
RC
HL
INE
AR
SE
AR
CH
SE
RV
AN
T
SE
RV
AN
T 11
SE
RV
AN
T
SE
RV
AN
T 22
IDL
IDL
SK
EL
S
KE
L 11
OPERATIONOPERATION100100
OPERATIONOPERATION22.........
OPERATIONOPERATION11
IDL
IDL
SK
EL
NS
KE
L N
............
.........
SERVANTSERVANT500::500::OPERATIONOPERATION100100
SERVANTSERVANT500::500::OPERATIONOPERATION11
SERVANTSERVANT1::1::OPERATIONOPERATION100100
SERVANTSERVANT1::1::OPERATIONOPERATION22
SERVANTSERVANT1::1::OPERATIONOPERATION11
OB
JEC
T A
DA
PT
ER
.........
.........
IDL
IDL
SK
EL
S
KE
L 22
OB
JEC
T A
DA
PT
ER
(C) L
AY
ER
ED
DE
MU
XIN
G,
PE
RF
EC
T H
AS
HIN
G
SE
RV
AN
T
SE
RV
AN
T 11
SE
RV
AN
T
SE
RV
AN
T 22
OPERATIONOPERATION100100
OPERATIONOPERATION22.........
OPERATIONOPERATION11
IDL
IDL
SK
EL
NS
KE
L N
search(o
bject k
ey)
hash
(operatio
n)
IDL
IDL
SK
EL
S
KE
L 11
(D) D
E-L
AY
ER
ED
AC
TIV
E D
EM
UX
ING
hash
(object k
ey)
search(o
peratio
n)
index
(object k
ey/o
peratio
n)
(B) L
AY
ER
ED
DE
MU
XIN
G,
DY
NA
MIC
HA
SH
ING
SE
RV
AN
T
SE
RV
AN
T 5
00
500
Dem
uxing
�
ww
w.cs.w
ustl.edu/�
schmidt/
fieeetc-97,C
OO
TS
-99
g.ps.gz
Perfecthashing
�
ww
w.cs.w
ustl.edu/�
schmidt/
gperf.ps.gz
UC
Irvine36
High-performance, Real-time ORBs Do
POA Demultiplexing Results
15
1015
2025
012345
Lat
ency
(us)
POA Depth
TransientPersistent
Synopsis of Results
� Active demux isefficient & predictablefor both transient andpersistent objectreferences.
Principle Patterns
� Precompute, passhints, usespecial-purpose &predictable datastructures, ignore refmodels
UC Irvine
High-performance, Real-time ORBs Do
Servant Demultiplexing Results
1 100 200 300 400 500
0
50
100
150
200
Lat
ency
(u
s)
No. of Objects
Active Demux
Dynamic Demux
Linear Demux
Synopsis of Results
� Linear demux iscostly
� Active demux ismost efficient &predictable
Principle Patterns
� Precompute, passhints, usespecial-purpose &predictable datastructures
UC IrvineH
igh-performance,
Real-tim
eO
RB
sD
ouglasC
.Schm
idt
Operation
Dem
ultiplexingR
esults
110
2030
4050
0 5
10
15
20
25
Latency (us)
No
. of M
etho
ds
Perfect H
ashing
Binary S
earch
Dynam
ic Hashing
Linear Search
Synopsis
ofResults
!
�
PerfectH
ashing–
Highly
predictable–
Low-latency
�
Others
strategiesslow
er
Principle
Patterns
!
�
Precom
pute,usepredictable
datastructures,rem
ovegratuitous
waste
UC
Irvine39
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
TAO
Req
uest
Dem
ultip
lexi
ngS
umm
ary
...
...
...
...
...
...
PO
AP
OA
11
SE
RV
AN
T
SE
RV
AN
T 11 O
RB
C
OR
EO
RB
C
OR
E
RO
OT
P
OA
AC
TIV
E
DE
MU
XIN
G
PO
A2
PO
AN
SE
RV
AN
T N
...
SK
EL
1S
KE
L 2
SK
EL
N
SE
RV
AN
T 2
AC
TIV
E
DE
MU
XIN
G
PE
RF
EC
TH
AS
HIN
G
Dem
ultip
lexi
ngS
tage
Abs
olut
eT
ime
(
�
s)1.
Req
uest
pars
ing
22.
PO
Ade
mux
23.
Ser
vant
dem
ux3
4.O
pera
tion
dem
ux2
5.P
aram
eter
dem
arsh
alin
gop
erat
ion
depe
nden
t6.
Use
rup
call
serv
antd
epen
dent
7.R
esul
tsm
arsh
alin
gop
erat
ion
depe
nden
t
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Real-time ORB/OS Performance Experiments
[P ]0
[P ]1
SCH
ED
UL
ER
0
RU
NT
IME
[P ]
I/O SUBSYSTEM
Server
0 nS1
nC
������������������Pentium II
S S0C 1C
...
...
Object AdapterServants
ORB Core
Client
...
[P]
Requests
Priority
...
[P ][P ]1
[P ]
n
n
www.cs.wustl.edu/�schmidt/RT-OS.ps.gz
Method
� Vary OS, hold ORBsconstant
� Single-processor IntelPentium II 450 Mhz, 256Mbytes of RAM
� Client and servant run onthe same machine
� Client Ci connects toservant Si with priority Pi
– i ranges from 1 : : : 50
� Clients invoke two-wayCORBA calls that cube anumber on the servant andreturns result
UC Irvine 41
High-performance, Real-time ORBs Douglas C. Schmidt
Real-time ORB/OS Performance Results'&
$%
0
200
400
600
800
1000
1200
1400
1600
1800
2000
0 1 2 5 10 15 20 25 30 35 40 45 50
Low Priority Clients
Tw
o-w
ay R
equ
est
Lat
ency
, use
c
Linux
LynxOS
NT
Solaris
VxWorks
'&
$%
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
0 1 2 5 10 15 20 25 30 35 40 45 50
Low Priority Clients
Tw
o-w
ay R
equ
est
Lat
ency
, use
c
Linux
LynxOS
NT
Solaris
VxWorks
High-priority Client Latency Low-priority Clients Latency
UC Irvine 42
High-performance, Real-time ORBs Douglas C. Schmidt
Real-time ORB/OS Jitter Results'&
$%
0
2
10
20
30
40
50
0
500
1000
1500
2000
2500
3000
3500
Tw
o-w
ay J
itte
r, u
sec
Low Priority Clients
LynxOSNTVxWorksLinuxSolaris
'&
$%
0 2
10
20
30
40
50
0
5000
10000
15000
20000
25000
30000
35000
40000
Tw
o-w
ay J
itte
r, u
sec
Low Priority Clients
LynxOSLinuxSolarisNTVxWorks
High-priority Client Jitter Low-priority Clients Jitter
UC Irvine 43
High-perform
ance,R
eal-time
OR
Bs
Douglas
C.S
chmidt
Problem
:H
ard-codedO
RB
Messaging
andTransportP
rotocolsB
OA
RD
B
OA
RD
11
VM
EV
ME
15
53
15
53
BO
AR
D
BO
AR
D 22
�
GIO
P/IIO
Pare
notsufficient,e.g.:
–G
IOP
message
footprintmay
betoo
large–
TC
Placks
necessaryQ
oS–
Legacycom
mitm
entsto
existingprotocols
�
Many
OR
Bs
donotsupport
“pluggableprotocols”
–T
hism
akesO
RB
sinflexible
andinefficient
UC
Irvine44
High-performance, Real-time ORBs Do
One Solution: Hacking GIOP
� GIOP requests include fields that aren’t neededin homogeneous embedded applications
– e.g., GIOP magic #, GIOP version, byte order,request principal, etc.
� These fields can be omitted without any changesto the standard CORBA programming model
� TAO’s -ORBgioplite option save 15 bytesper-request, yielding these calls-per-second:
Marshaling-enabled Marshaling-disabledmin max avg min max avg
GIOP 2,878 2,937 2,906 2,912 2,976 2,949GIOPlite 2,883 2,978 2,943 2,911 3,003 2,967
� The result is a measurable improvement inthroughput/latency
– However, it’s so small (2%) that hacking GIOPis of minimal gain except for low-bandwidthlinks
UC Irvine
High-performance, Real-time ORBs Do
Better Solution: TAO’sPluggable Protocols Framework
CLIENT
STUBS SKELETONS
TCP
MULTICAST
IOP
VMEUDP
ORB MESSAGING COMPONENT
ORB TRANSPORT ADAPTER COMPONENT
ESIOP
REAL -TIME
IOPEMBEDDED
IOP
RELIABLE,BYTE-STREAM
ATMTCP
MEMORY
MANAGEMENT
CONCURRENCY
MODEL
OTHER
ORBCORE
SERVICES
COMMUNICATION INFRASTRUCTUREHIGH SPEED NETWORK INTERFACE
REAL -TIME I /O SUBSYSTEM
ORB MESSAGE
FACTORY
ORB TRANSPORT
ADAPTER FACTORY
OBJECT ADAPTER
GIOP GIOPLITE
ADAPTIVE Communication Environment (ACE)
OBJECT (SERVANT)operation (args)IN ARGS
OUT ARGS & RETURN VALUE
POLICY
CONTROL
Features
� Pluggable ORBmessaging andtransport protocols
� Highly efficient andpredictable behavior
Principle Patterns
� Replacegeneral-purposefunctions(protocols) withspecial-purposeones
UC IrvineH
igh-performance,
Real-tim
eO
RB
sD
ouglasC
.Schm
idt
CO
RB
AP
rotocolInteroperabilityA
rchitecture
OR
B M
ES
SA
GIN
G
CO
MP
ON
EN
T
OR
B T
RA
NS
PO
RT
AD
AP
TE
R C
OM
PO
NE
NT
TR
AN
SP
OR
T L
AY
ER
NE
TW
OR
K L
AY
ER
GIO
P
IIOP
TC
P
IP
VM
E
DR
IVE
R
AA
L5
AT
M
GIO
PL
ITE
VM
E-IO
P
ES
IOP
AT
M-IO
PR
ELIA
BLE
SE
QU
EN
CE
D
PR
OT
OC
OL C
ON
FIG
UR
AT
ION
S
ST
AN
DA
RD
CO
RB
A P
RO
GR
AM
MIN
G A
PI F
eatures
!
�
Presentation
layer–
e.g.,CD
R
�
Message
formats
–e.g.,G
IOP,
DC
E
�
Transportassum
ptions–
e.g.,TC
P,ATM
�
Object
addressing–
e.g.,IIOP
IOR
ww
w.cs.w
ustl.edu/�
schmidt/pluggable
protocols.ps.gz
UC
Irvine47
High-performance, Real-time ORBs Douglas C. Schmidt
Embedded System Benchmark Configuration
MARSHALMARSHAL
PARAMSPARAMS
OBJECT OBJECT ((SERVANTSERVANT))
OBJECT ADAPTEROBJECT ADAPTER
ACTIVEACTIVE
OBJECTOBJECT
MAPMAP
IDLIDLSKELETONSKELETON
VME BUSVME BUS
OS KERNELOS KERNEL
VMEVME
DRIVERDRIVER
OS KERNELOS KERNEL
VMEVME
DRIVERDRIVER
IDLIDLSTUBSSTUBS
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
CLIENTCLIENT
INC
OM
ING
INC
OM
ING
CDRCDR
DATA COPYDATA COPY
DATA COPYDATA COPY
DMADMA COPY COPY
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
CDRCDR
DATA COPYDATA COPY
DEMARSHALDEMARSHAL
PARAMSPARAMS
DATA COPYDATA COPY
OU
TG
OIN
GO
UT
GO
ING
ORBORBCORECORE
obj->op (params)
VxWorks running on 200 Mhz PowerPC over 320 Mbps VME & 10Mbps Ethernet
UC Irvine 48 Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Eth
erne
t&V
ME
Two-
way
Late
ncy
Res
ults
010
0020
0030
0040
0050
0060
0070
0080
0090
00
void
shor
t
octe
t
long
stru
ct
shor
t seq
<oc
tet>
long
seq
<oc
tet>
shor
t seq
<lo
ng>
long
seq
<lo
ng>
Data Type
Lat
ency
(u
sec)
VM
E/G
IOP
lite
avg.
VM
E/G
IOP
avg
.
Eth
erne
t/GIO
Plit
e av
g.
Syn
opsi
sof
Res
ults
�
VM
Epr
otoc
olis
muc
hfa
ster
than
Eth
erne
t
�
No
appl
icat
ion
chan
ges
are
requ
ired
tosu
ppor
tV
ME
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Pinpointing ORB Overhead with VMEtro Timeprobes
ORB TRANSPORTORB TRANSPORT
RECVRECV
OBJECT (SERVANT)
ORBORBCORECORE
OBJECT ADAPTEROBJECT ADAPTER
IDLIDLSKELETONSKELETON
VME BUSVME BUS
OS KERNELOS KERNEL
VME DRIVERVME DRIVER
OS KERNELOS KERNEL
VME DRIVERVME DRIVER
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
ACTIVEACTIVE
OBJECTOBJECT
MAPMAP
GET OBJECTGET OBJECT
REFREF
IDLIDLSTUBSSTUBS
MARSHALMARSHAL
PARMATERSPARMATERS
ORB MESSAGINGORB MESSAGING
SENDSEND
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
ORB TRANSPORTORB TRANSPORT
SENDSEND
II//O SENDO SEND
II//O RECVO RECV77
88
ORB MESSAGINGORB MESSAGING
RECVRECV99
CLIENTCLIENT
DEMARSHALDEMARSHAL
PARAMETERSPARAMETERS
7799
II//O RECVO RECV
ORB TRANSPORTORB TRANSPORT
RECVRECV
ORB MESSAGINGORB MESSAGING
RECVRECV
PARSE OBJECTPARSE OBJECT
KEY KEY
OBJECTOBJECT
DEMUXDEMUX
DEMARSHALDEMARSHAL
PARAMETERSPARAMETERS
USER UPCALLUSER UPCALL
ORB MESSAGINGORB MESSAGING
SENDSEND
ORB TRANSPORTORB TRANSPORT
SENDSEND
II//O SENDO SEND
MARSHALMARSHAL
PARAMETERSPARAMETERS
OPERATIONOPERATION
DEMUXDEMUX
131311
22
44
661111
1010
1212
33
88INITIALIZATIONINITIALIZATION
33
55
44
11
22
66
1010
OUTGOINGOUTGOING INCOMINGINCOMING
INC
OM
ING
INC
OM
ING
OU
TG
OIN
GO
UT
GO
ING
POAPOA
DEMUXDEMUX55
� Timeprobes use VMEtro monitor,which measures end-to-end time
� Timeprobe overhead isminimal, i.e., 1 �sec
UC Irvine 50 Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
OR
B&
VM
EO
ne-w
ayO
verh
ead
Res
ults
200
200
198
200
203
205
212
226
257
309
51
49
50
50
49
51
50
51
52
56
24
23
23
22
23
23
22
23
23
23
68
66
66
66
66
66
67
66
69
72
0%10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
48
1632
6412
825
651
210
2420
48S
eque
nce
leng
th (
byte
s)
usecs
VM
E +
OS
ove
rhea
dO
RB
clie
nt p
re-w
rite
OR
B c
lient
pos
t-w
rite
OR
B s
erve
r ov
erhe
ad
Syn
opsi
sof
Res
ults
�
OR
Bov
erhe
adis
rela
tivel
yco
nsta
ntan
dlo
w
–e.
g.,�
110
�
secs
per
end-
to-e
ndop
erat
ion
�
Bot
tlene
ckis
VM
Edr
iver
and
OS
,not
OR
B
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Workstation Benchmark Configuration
MARSHALPARAMS
OBJECT (SERVANT)
OBJECT ADAPTER
ACTIVE
OBJECT
MAP
IDLSKELETON
NETWORKLINK
REAL-TIME I /OINTERFACE
DRIVER
IDLSTUBS
ORB MESSAGING
ORB TRANSPORT
CLIENT
BUFFER
MANAGEMENT
SEND MESSAGE
ORB MESSAGING
ORB TRANSPORT
DEMARSHALPARAMS
DISPATCH
MESSAGE
ORBCORE
obj->op (params)
DEMUX
REAL-TIME I /OINTERFACE
DRIVER
Debian Linux running on 400 Mhz workstation over Local IPC
UC Irvine 52 Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Bla
ckbo
xTw
o-w
ayLa
tenc
yR
esul
ts
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
void
shor
toc
tet
long
struc
t
small
seq<
octe
t>
large
seq<
octe
t>
small
seq<
long>
large
seq<
long>
small
seq<
struc
t>
large
seq<
struc
t>
Dat
a T
ype
Calls per Second
0.00
%
10.0
0%
20.0
0%
30.0
0%
40.0
0%
50.0
0%
60.0
0%
Performance Improvement (%)
IIOP
IIOP
/GIO
Plit
eU
IOP
UIO
P/G
IOP
lite
Per
form
ance
Incr
ease
Syn
opsi
sof
Res
ults
�
Loca
lIP
Cm
ore
effic
ient
than
TC
P/IP
over
loop
back
�
No
appl
icat
ion
chan
ges
are
requ
ired
tosu
ppor
tm
ultip
lepr
otoc
ols
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Client Whitebox Latency ResultsDirection Client Activities Absolute Time ( �s)Outgoing 1. Initialization 6.30
2. Get object reference 15.63. Parameter marshal 0.74 (param. dependent)4. ORB messaging send 7.785. ORB transport send 1.026. I/O 8.70 (op. dependent)7. ORB transport recv 50.78. ORB messaging recv 9.259. Parameter demarshal op. dependent
Xeon platform is quad-CPU 400 Mhz with 1 Gigabytes RAM
UC Irvine 54
High-performance, Real-time ORBs Douglas C. Schmidt
Server Whitebox Latency Results on Xeon/NTDirection Server Activities Absolute Time ( �s)Incoming 1. I/O 7.0 (op. dependent)
2. ORB transport recv 24.83. ORB messaging recv 4.54. Parsing object key 4.65. POA demux 1.396. Servant demux 4.67. Operation demux 4.528. User upcall 3.84 (op. dependent)
Outgoing 9. ORB messaging send 4.5610. ORB transport send 93.6
UC Irvine 55
High-performance, Real-time ORBs Do
ORB & Transport Overhead Results
112
111
52 51
49 48
49 47
24 23
27 27
31
27
30
27
0
50
100
150
200
250
IIOP IIOP w/GIOPlite UIOP UIOP w/GIOPlite
Transport Protocol
To
tal T
ime
(use
cs)
OS and I/O ORB Transport Messaging
Synopsis of Results
� ORB overhead is relatively constant and low
– e.g., �49 �secs per two-way operation
� Bottleneck is OS and I/O operation
UC Irvine
High-performance, Real-time ORBs Do
Data Copies in the Pluggable Protocols
MARSHALMARSHAL
PARAMSPARAMS
OBJECT OBJECT ((SERVANTSERVANT))
OBJECT ADAPTEROBJECT ADAPTER
ACTIVEACTIVE
OBJECTOBJECT
MAPMAP
IDLIDLSKELETONSKELETON
VME BUSVME BUS
OS KERNELOS KERNEL
VMEVME
DRIVERDRIVER
OS KERNELOS KERNEL
VMEVME
DRIVERDRIVER
IDLIDLSTUBSSTUBS
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
CLIENTCLIENT
INC
OM
ING
INC
OM
ING
CDRCDR
DATA COPYDATA COPY
DATA COPYDATA COPY
DMADMA COPY COPY
ORB MESSAGINGORB MESSAGING
ORB TRANSPORTORB TRANSPORT
CDRCDR
DATA COPYDATA COPY
DEMARSHALDEMARSHAL
PARAMSPARAMS
DATA COPYDATA COPY
OU
TG
OIN
GO
UT
GO
ING
ORBORBCORECORE
obj->op (params)
� Marshal parameters, data copy to CDR stream
� VME send, data copy from CDR stream to VMEbuffers
� DMA, data copy over VME Bus
� VME read, data copy to CDR stream
� Demarshal parameters, data copy to methodparameters
UC Irvine
High-performance, Real-time ORBs Do
One-Way Delayed Buffering Strategy
MARSHALPARAMS
OBJECT (SERVANT)
OBJECT ADAPTER
ACTIVE
OBJECT
MAP
IDLSKELETON
VME BUS
OS KERNEL
VMEDRIVER
OS KERNEL
VMEDRIVER
IDLSTUBS
ORB MESSAGING
ORB TRANSPORT
CLIENT
BUFFERED
DATA COPY
DMA COPY
ORB MESSAGING
ORB TRANSPORT
DEMARSHALPARAMS
DATA COPY
ORBCORE
obj->op (params)
DEMUX
� Copy params to new buffer
� Requests buffered in the Transport Adaptor
� Flush at byte count or timeout
� Send as one ORB message but multiplerequests
� Server demultiplexes individual requests
UC Irvine
High-performance, Real-time ORBs Do
Shared Buffer Strategy
MARSHALPARAMS
OBJECT (SERVANT)
OBJECT ADAPTER
ACTIVE
OBJECT
MAP
IDLSKELETON
VME BUS
OS KERNEL
VMEDRIVER
OS KERNEL
VMEDRIVER
IDLSTUBS
ORB MESSAGING
ORB TRANSPORT
CLIENT
BUFFERED
DATA COPY
DMA COPY
ORB MESSAGING
ORB TRANSPORT
DEMARSHALPARAMS
DATA COPY
ORBCORE
obj->op (params)
DEMUX
� Request free buffer
� Add to Send queue
� Return to Free pool
� Request free buffer
� Add to Revc queue
� Return to Free pool
UC Irvine
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Overly Large Memory Footprint
INTERFACE
REPOSITORY
IMPLEMENTATION
REPOSITORY
IDLCOMPILER
DII ORBINTERFACE
ORBORB CORECORE GIOPGIOP//IIOPIIOP//ESIOPSESIOPS
IDLIDLSTUBSSTUBS
operation()operation()in args
out args + return value
CLIENTOBJECT(SERVANT)
OBJ
REF
STANDARD INTERFACE STANDARD LANGUAGE MAPPING
ORB-SPECIFIC INTERFACE STANDARD PROTOCOL
INTERFACE
REPOSITORY
IMPLEMENTATION
REPOSITORY
IDLCOMPILER
IDLSKELETON
DSI
OBJECT
ADAPTER
www.cs.wustl.edu/�schmidt/COOTS-99.ps.gz
� Problem
– ORB footprint is toobig for someembedded apps
� Unnecessary Features
– DSI, DII, & DynamicAny
– Interface Repository– Advanced POA
features– CORBA/COM
interworking
UC Irvine 60
High-performance, Real-time ORBs Douglas C. Schmidt
Solution: Minimum CORBA
Component CORBA Minimum PercentageCORBA Reduction
POA 282k 207k 26.5ORB Core 347k 330k 4.8Dynamic Any 131k 0 100CDR Interpreter 69k 69k 0IDL Compiler 10k 11k 0Pluggable Protocols 15k 15k 0Default Resources 8k 8k 0
Total 862k 640k 25.8
Applying Minimum CORBA subsetting to TAO reduces memoryfootprint by �25% (on SPARC with EGCS) and increases ORB
determinism
UC Irvine 61
High-performance, Real-time ORBs Douglas C. Schmidt
Problem: Providing QoS to CORBA Operations
RTRTOperationOperation
PERIODIC SCHEDULINGPERIODIC SCHEDULING//DISPATCHINGDISPATCHING
TT++PP
P:P: PERIOD PERIOD
C:C: REQUEST COMPUTATION TIME REQUEST COMPUTATION TIME
PP
CC CC
TIMETIME
TT TT++22PP
struct RT_Info {struct RT_Info { Time worstcase_exec_time_; Time worstcase_exec_time_; Period period_; Period period_; Criticality criticality_; Criticality criticality_; Importance importance_; Importance importance_;};};
� Design Challenges
– Specifying/enforcing QoSrequirements
– Focus on Operations upon Objects
� Rather than on communication channels orthreads/synchronization
– Support static and dynamic scheduling
� Solution Approach
– Servants publish resource (e.g., CPU)requirements and (periodic) deadlines
– Most clients are also servants
UC Irvine 62
High-performance, Real-time ORBs Douglas C. Schmidt
Solution: TAO’s Real-time Static Scheduling Service
RT_INFO
REPOSITORY
OFF-LINE
SCHEDULER
DEPENDS UPON =EXECUTES AFTER
4: ASSESS
SCHEDULABILITY
5: ASSIGN OS THREAD
PRIORITIES AND
DISPATCH QUEUE
ORDERING
SUBPRIORITIES
6: SUPPLY
PRIORITIES
TO ORB
3: POPULATE
RT_INFO
REPOSITORY
RUN-TIME
SCHEDULER
Priority/SubpriorityTable Per
Mode
MODE 0
MODE 1
MODE 2
MODE 3
CURRENT
MODE
SELECTOR
struct RT_Info { Time worstcase_exec_time_; Period period_; Criticality criticality_; Importance importance_;};
1: CONSTRUCT CALL
CHAINS OF RT_OPERATIONS
2: IDENTIFY THREADS
RT
Operation
RT
Operation
RT
Operation
I/O SUBSYSTEM
ORB CORE
OBJECT ADAPTER
www.cs.wustl.edu/�schmidt/TAO.ps.gz
UC Irvine 63
High-performance, Real-time ORBs Douglas C. Schmidt
TAO’s RT Event Service Architecture
0 1 2 3 4
1: push (event)
Supplier ProxiesSupplier
2: push (event)
3: push (event, consumer)
4: push (event, consumer)
5: enqueue (event, consumer)
6: dequeue (event, consumer)
7: push (event)
8: push (event)
Consumer
Consumer Proxies
Dispatcher
Priority Queues
Run-Time Scheduler
Event Correlation
Subscription & Filtering
Consumer
ConsumerEvent Channel
Dispatching Module
Features !
� Integrated with RTScheduler
� Stream-basedarchitecture– Enhance pluggability
� Source and type-basedfiltering
� Event correlations– Conjunctions (A+B+C)– Disjunctions (AjBjC)
www.cs.wustl.edu/�schmidt/JSAC-98.ps.gz
UC Irvine 64
High-performance, Real-time ORBs Douglas C. Schmidt
Visualizing Periods for Avionics Operations
UC Irvine 65
High-performance, Real-time ORBs Douglas C. Schmidt
Example: Applying TAO to Real-time Avionics
I/O Facade
SensorProxy
SensorProxy
SensorProxy
SensorProxy
I/O Facade I/O Facade
2: Demarshaled data
High LevelAbstraction
Low LevelAbstraction
1: I/O via interrupts
AircraftSensors
� Synopsis
– Typical Interactions
� I/O arrives
� Proxies demarshaldata
� Facades process data– Advantages:
� Efficient control flow
� Clean layeredarchitecture
– Disadvantages:
� Coupled layers� Inflexible scheduling
UC Irvine 66
High-performance, Real-time ORBs Douglas C. Schmidt
Forces/Domain Characteristics� I/O driven
– Periodic processing requirements
� Complex dependencies
– e.g., I/O Facades depend on multiple sensor proxies
� Real-time constraints
– Deterministic and statistical deadlines– Static scheduling (e.g., rate monotonic)
� Single-Processor (VxWorks)
– Single address space– No distribution requirements (yet)
UC Irvine 67
High-performance, Real-time ORBs Douglas C. Schmidt
Candidate Solution: COS Event Service
push()push()
CONSUMERCONSUMER
CONSUMERCONSUMER
EVENTEVENT
CHANNELCHANNEL
CONSUMERCONSUMER
SUPPLIERSUPPLIER
SUPPLIERSUPPLIER
push()push()push()push()
push()push()
push()push()
www.cs.wustl.edu/�schmidt/report-doc.html
� Features
– Decoupledconsumers andsuppliers
– Transparent groupcommunication
– Asynchronouscommunication
– Abstraction fordistribution
– Abstraction forconcurrency
UC Irvine 68
High-performance, Real-time ORBs Douglas C. Schmidt
Applying the COS Event Service to Real-time Avionics
� TAO is currently used at Boeing for avionics mission computing
– Initial flight dates are mid-summer 1998
� Extensive benchmarks demonstrate it is possible to meet stringentperformance goals with real-time CORBA
– e.g., for Boeing, target latency for CORBA oneway operations is150 �secs for 100 Mhz PowerPC running over MVME 177 boards
� Technology transfer to commercial vendors via OMG RT SIG andDARPA Quorom program & OCI
UC Irvine 69
High-performance, Real-time ORBs Douglas C. Schmidt
Overview of Avionics Mission ComputingConsumers
I/O Facade
SensorProxy
SensorProxy
SensorProxy
SensorProxy
1: I/O via interrupts
I/O Facade I/O Facade
EventChannel
2: push (demarshaled data)
AircraftSensors
3: push (demarshaled data)
Suppliers
� Typical Interactions
– I/O arrives– Proxies demarshal
data– Proxies push to
channel– EC pushes to facades– Facades process data
� Advantages:
– Anonymousconsumers/suppliers
– Group communication– Asynchronous pushes
UC Irvine 70
High-performance, Real-time ORBs Douglas C. Schmidt
Issues Not Addressed by COS Event Service� No support for complex event dependencies
– Consumer-specified event filtering– Event correlations (e.g., waiting for events A and B before
pushing)
� No support for real-time scheduling policies
– Priority-based dispatching (e.g., which consumer is dispatchedfirst)
– Priority-based preemption policies and mechanisms– Interval timeouts for periodic processing– Deadline timeouts for “failed” event dependencies
UC Irvine 71
High-performance, Real-time ORBs Douglas C. Schmidt
TAO’s Event Service Architecture
Subscription& Filtering
EventCorrelation
DispatchingModule
EVENTCHANNEL
Consumer
ConsumerConsumer
Supplier SupplierSupplier
push (event)
push (event)
ConsumerProxies
SupplierProxies
PriorityTimers
EventFlow
� Features
– Stream-based architecture
� Enhance pluggability– Subscription/filtering
� Source and type-basedfiltering
– Event correlations
� Conjunctions (A+B+C)
� Disjunctions (AjBjC)
UC Irvine 72
High-performance, Real-time ORBs Douglas C. Schmidt
Collaborations in the RT Event Channel
Subscription& Filtering
SupplierProxies
PriorityTimers
EventCorrelation
DispatchingModule
ConsumerProxies
EVENTCHANNEL
Object Ref
Subscription Info Publish Types
CorrelationSpecs
RT_Info
Object Ref
Timeout Registration
Consumer
Supplier
CONNECT_PUSH
SUPPLIER
CONNECT_PUSH
CONSUMER Source IDEvent TypeTimestamp
Data
EVENT
STRUCTURE
www.cs.wustl.edu/�schmidt/events tutorial.html
� Well-defined eventstructure
– CORBA Anys areinefficient
� Augmented COSinterfaces:
– Extra QoS structureto connect suppliersand consumers
UC Irvine 73
High-performance, Real-time ORBs Douglas C. Schmidt
Real-Time Event Dispatching with TAO’s Event Service
0 1 2 3 4
1: push (event)
Supplier ProxiesSupplier
2: push (event)
3: push (event, consumer)
4: push (event, consumer)
5: enqueue (event, consumer)
6: dequeue (event, consumer)
7: push (event)
8: push (event)
Consumer
Consumer Proxies
Dispatcher
Priority Queues
Run-Time Scheduler
Event Correlation
Subscription & Filtering
Consumer
ConsumerEvent Channel
Dispatching Module
� Features
– Run-time scheduler
� Determines event priority– 2-level priority queues
� Preemption groups
� Priority queues– Dispatcher
� Encapsulatesconcurrency policy
UC Irvine 74
High-performance, Real-time ORBs Douglas C. Schmidt
Real-time Event Channel Dispatching Experiments
0 1 2 3 4
(B) RTU Dispatching
Dispatcher
Run-TimeScheduler
3:3: DEQUEUE DEQUEUE
REQUEST REQUEST
4:4: DISPATCH DISPATCH
REQUEST REQUEST
1:1: INCOMING INCOMING
REQUEST REQUEST
2:2: ENQUEUE ENQUEUE
REQUESTREQUEST
0 1 2 3 4
(C) Threaded Dispatching
Run-TimeScheduler
1:1: INCOMING INCOMING
REQUEST REQUEST
2:2: ENQUEUE ENQUEUE
REQUESTREQUEST
3:3: DEQUEUE DEQUEUE
REQUEST REQUEST
4:4: DISPATCH DISPATCH
REQUEST REQUEST
Dispatcher
(A) FIFO Dispatching
Dispatcher
4:4: DISPATCH DISPATCH
REQUEST REQUEST
1:1: INCOMING INCOMING
REQUEST REQUEST
2:2: ENQUEUE ENQUEUE
REQUESTREQUEST
3:3: DEQUEUE DEQUEUE
REQUEST REQUEST
www.cs.wustl.edu/�schmidt/oopsla.ps.gz
UC Irvine 75
High-performance, Real-time ORBs Douglas C. Schmidt
Multi-Threaded Dispatching
UC Irvine 76
High-performance, Real-time ORBs Douglas C. Schmidt
Single-Threaded Dispatching
UC Irvine 77
High-performance, Real-time ORBs Douglas C. Schmidt
Dimensions of ORB Extensibility
1 POSIX,POSIX, WWININ32,32, RTOSRTOSSS,, MVSMVS
3 DLLDLLSS
2
strategy2
strategy1
strategy3 profile2
profile1
profile3
profile4
1. Extensible to retargeting on new platforms
2. Extensible via custom implementation strategies
3. Extensible via dynamic configuration of custom strategies
UC Irvine 78
High-performance, Real-time ORBs Douglas C. Schmidt
Applying Patterns to Develop Extensible ORBs
ACCEPTORCONNECTOR
ABSTRACT
FACTORY
SERVANTCLIENT
OS KERNELOS KERNEL
ACTIVE
OBJECT
THREAD-SPECIFIC
STORAGE
SERVICE
CONFIGURATOR
STRATEGY
REACTORREACTOR
WRAPPER FACADESWRAPPER FACADES
www.cs.wustl.edu/�schmidt/ORB-patterns.ps.gz
� Factories produce Strategies
� Strategies implementinterchangeable policies
� Concurrency strategies useReactor and Active Object
� Acceptor-Connectordecouple transport fromGIOP operations
� Service Configurator permitsdynamic configuration
UC Irvine 79
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Portability and Typesafety Challenges
� Problem
– Building an ORB using low-level system APIs is hard
� Forces
– Low-level APIs are tedious to program– Low-level APIs are error-prone– Low-level APIs are non-portable
� Solution
– Apply the Wrapper Facade pattern to encapsulate low-level OSprogramming details
UC Irvine 80
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing Portability and Typesafety with the WrapperFacade Pattern
1: method_k()
2: function_k()
clientclient
FunctionsFunctions
function_1()...function_n()
WrapperWrapperFacadeFacade
method_1()...method_m()
www.cs.wustl.edu/�schmidt/wrapper-facade.ps.gz
� Intent
– Encapsulates low-levelsystem calls within type-safe,modular, and portable classinterfaces
� Forces Resolved
– Avoid tedious, error-prone,and non-portable system APIs
– Create cohesive abstractions– Don’t compromise
performance
UC Irvine 81
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Wrapper Facade Pattern in TAO
'&
$%
POSIXPOSIXPthreadsPthreads
BSDBSDSocketsSockets
WinSockWinSock
SocketSocketMacrosMacros Other OSOther OS
System CallsSystem Calls
pthread_create()pthread_create()
pthread_mutex_*pthread_mutex_*
socket(), bind(),socket(), bind(),
recv(), send()recv(), send()
gettimeofday(),gettimeofday(),
select(), etc.select(), etc.
SunSoft IIOP's ORB CoreSunSoft IIOP's ORB Core
GENERALGENERAL
POSIXPOSIX &&WWININ3232
SERVICESSERVICES
'&
$%
TAO's ORB CoreTAO's ORB Core
spawn()spawn()
acquire()acquire()
open(),open(),close(),close(),
recv(), send()recv(), send()
dlopen()dlopen()
dlsym()dlsym()
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUALVIRTUAL
MEMORY MEMORY
SUBSYSTEMSUBSYSTEM
GENERALGENERAL
POSIXPOSIX && WWININ3232
SERVICESSERVICES
PROCESSPROCESS//THREADTHREAD
SUBSYSTEMSUBSYSTEM
THREADTHREAD
WRAPPERSWRAPPERS
DYNAMIC
LINKING
SELECT/IO COMP
SOCKETS/TLI
ACEWRAPPER
FACADES
handle_events()
SunSoft IIOP TAO
� TAO’s wrapper facades are based on the ACE framework
� The Wrapper Facade pattern substantially increased portability andreduced the amount of ad hoc code
UC Irvine 82
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Demuxing/Dispatching Challenges� Problem
– ORBs must process many different types of events simultaneously
� Forces
– Multi-threading may not be available– Multi-threading may be inefficient– Multi-threading may be inconvenient– Tightly coupling general event processing with ORB-specific logic
is inflexible
� Solution
– Use the Reactor pattern to decouple generic event processingfrom ORB-specific processing
UC Irvine 83
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing Demuxing with the Reactor Pattern
Initiation DispatcherInitiation Dispatcher
handle_events()register_handler(h)remove_handler(h)
select (handles);foreach h in handles loop table[h].handle_event(type)end loop
Event HandlerEvent Handler
handle_event(type)get_handle()
handlers
HandleHandle ownsuses
notifies
ConcreteConcreteEventEvent
HandlerHandler
Synchronous EventSynchronous EventDemultiplexerDemultiplexer
select()
1 N
www.cs.wustl.edu/�schmidt/Reactor.ps.gz
� Intent
– Decouplessynchronous eventdemuxing/dispatchingfrom event handling
� Forces Resolved
– Demuxing eventsefficiently within onethread
– Extending applicationswithout changingdemux infrastructure
UC Irvine 84
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Reactor Pattern in TAO'&
$%
ORB COREORB CORE
1: RUN EVENT LOOP
OBJECTOBJECT
ADAPTERADAPTER
APPLICATIONAPPLICATION
2: GET REQUEST
3: BLOCK FOR CONNECTION 6: INCOMING
MESSAGE
5: DISPATCH
SERVANTSERVANT
7: UPCALL
GIOPGIOP
EngineEngine4: select()
serverserver
endpointsendpoints'
&
$%
ORB COREORB CORE
1: RUN EVENT LOOP
OBJECTOBJECT
ADAPTERADAPTER
APPLICATIONAPPLICATION
3: handle_input()
4: DISPATCH
SERVANTSERVANT
5: UPCALL
ACTIVE OBJECT MAPACTIVE OBJECT MAP
2: select()
ReactorReactor
ConnectionConnectionHandlerHandler
ConnectionConnectionHandlerHandler
ConnectionConnectionHandlerHandler
SunSoft IIOP TAO
� The Reactor pattern and ACE Reactor are widely used
UC Irvine 85
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Endpoint Initialization Challenges
� Problem
– The communication protocol used between ORBs is oftenorthogonal to its connection establishment and service handlerinitialization protocols
� Forces
– Low-level connection APIs are error-prone and non-portable– Separating initialization from processing increases software reuse
� Solution
– Use the Acceptor-Connector pattern to decouple passive/activeconnection establishment and GIOP connection handlerinitialization from the subsequent ORB interoperability protocol(e.g., IIOP)
UC Irvine 86
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing Endpoint Initialization with theAcceptor-Connector Pattern'
&
$%
CREATE CREATE &&
ACTIVATEACTIVATE
ConnectionConnectionHandlerHandler
peer_stream_
open()
EventEventDispatcherDispatcher
NO
TIFIE
S
NO
TIFIE
S
Strategy StrategyAcceptorAcceptor
peer_acceptor_
handle_input()
Connection HandlerConnection Handler
'&
$%
HANDLE ASYNC
CONNECTION COMPLETION
CREATE &
ACTIVATE
ConnectionHandler
peer_stream_
open()
EventDispatcher
StrategyConnector
connect(sh, addr)
complete()
Connection Handler
Acceptor Structure Connector Structure
� Intent
– Decouple connection establishment and service handlerinitialization from subsequent service processing
UC Irvine 87
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Acceptor-Connector Pattern in TAO
'&
$%
ORB COREORB CORE1: lookup()
2: connect() 4: accept()
5: read()/write()
5: read()/write()clientclient
endpointendpoint
clientclientendpointendpoint
clientclientendpointendpoint
serverserverendpointendpoint
listenerlistenerendpointendpoint
serverserverendpointendpoint
3: select()3: select()
SERVERCLIENT
'&
$%
GIOPGIOP
HandlerHandler
Strategy StrategyConnectorConnector
Cached Cached
ConnectConnect
StrategyStrategy
ConnectionConnectionHandlerHandler
ReactorReactor
ORBORB CORECORE
SERVERSERVERCLIENTCLIENT
5:5: REQUEST REQUEST//RESPONSERESPONSE
2: connect()2: connect() 3: accept()3: accept()
0..N
6: DISPATCH()
4: CREATE & ACTIVATE
1: operation()
ConnectionHandler
ConnectionHandler
ConnectionHandler
Concurrency
Strategy
StrategyAcceptor
SunSoft IIOP TAO
� Forces Resolved
– (1) Improve portability and reuse and (2) avoid common mistakes
UC Irvine 88
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Concurrency Challenges
� Problem
– Multi-threaded ORBs are needed since Reactive ORBs are ofteninefficient, non-scalable, and non-robust
� Forces
– Multi-threading can be very hard to program– No single multi-threading model is always optimal
� Solution
– Use the Active Object pattern to allow multiple concurrent serveroperations using an OO programming style
UC Irvine 89
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing ORB Concurrency withthe Active Object Pattern
ProxyProxy
Future m1()Future m2()Future m3()
HIDDENHIDDENFROMFROM
CLIENTSCLIENTS
VISIBLEVISIBLETOTO
CLIENTSCLIENTS
2: enqueue(M1)
ActivationActivationQueueQueue
enqueue()dequeue()
ServantServant
n1
loop { m = act_queue.dequeue() if (m.guard()) m.call()}
SchedulerSchedulerdispatch()enqueue()
m1()m2()m3() 4: m1()
11 11
1: enqueue(new M1)
3: dispatch()
MethodMethodRequestRequestguard()call()
M1M1
M2M2
M3M3
www.cs.wustl.edu/�schmidt/ Act-Obj.ps.gz
Intent
� Decouple thread ofrequest execution fromthread of requestreception
Forces Resolved
� Allow blockingoperations
� Permit flexibleconcurrency strategies
UC Irvine 90
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Active Object Pattern in TAO'&
$%
select()select()
2: NOTIFY
ORB CORE
serverendpoint
REACTIVE
1: ARRIVAL
3: READ
serverendpoint
select()
2: NOTIFY
ORB CORE
serverendpoint
1: ARRIVAL
3: READ
serverendpoint
THREAD-PERCONNECTION
'&
$%
ORB CORE
LEADER
FOLLOWERS
2: read()
Reactor
1: select()
3: release()
server endpoint
DISPATCHER
server endpoint
SunSoft IIOP TAO
� TAO supports several variants of Active Objects (e.g.,Thread-per-Connection, Thread-per-Request, Thread Pool, etc.)
UC Irvine 91
High-performance, Real-time ORBs Douglas C. Schmidt
Reducing Lock Contention and Priority Inversions withthe Thread-Specific Storage Pattern
� Problem
– It is important to minimize the amount of locking required toserialize access to resources shared by an ORB
� Forces
– Locks increase performance overhead– Locks increase potential for priority inversion– Different concurrency schemes yield different locking costs
� Solution
– Use the Thread-Specific Storage pattern to maximizethreading-model flexibility and minimize lock contention andpriority inversion
UC Irvine 92
High-performance, Real-time ORBs Douglas C. Schmidt
Minimizing ORB Locking withthe Thread-Specific Storage Pattern
THREAD ATHREAD A THREAD BTHREAD B
1: ACE_OS::thr_getspecific(key)
2: get_state(key)
ORB THREAD-SPECIFIC STATE
Reactor
Acceptor
Connector
Reactor
Acceptor
Connector
THREAD-SPECIFICOBJECT TABLES
INDEXED BY KEY
� Forces Resolved
– Minimizes overhead and priority inversion
Intent
� Allows multiplethreads to use onelogically globalaccess point toretrieve ORBthread-specificdata withoutincurring lockingoverhead for eachaccess
UC Irvine 93
High-performance, Real-time ORBs Douglas C. Schmidt
Using Thread-Specific Storage in TAO
CLIENTCLIENT ORBORB CORECORE
20 10 5 1
HZ HZ HZ HZ
CONNECTOR
20 10 5 1
HZ HZ HZ HZ
CONNECTOR
20 10 5 1
HZ HZ HZ HZ
CONNECTOR
SERVER ORB CORE
REACTOR
(20 HZ)
REACTOR
(10 HZ)
REACTOR
(5 HZ)
REACTOR
(1 HZ)
CC
OO
NN
NN
EE
CC
TT
22
AA
CC
CC
EE
PP
TT
OO
RR
CC
OO
NN
NN
EE
CC
TT
11
CC
OO
NN
NN
EE
CC
TT
33
II//OO SUBSYSTEMSUBSYSTEMII//OO SUBSYSTEMSUBSYSTEM
CC
OO
NN
NN
EE
CC
TT
22
AA
CC
CC
EE
PP
TT
OO
RR
CC
OO
NN
NN
EE
CC
TT
11
C
O
N
N
E
C
T
3
C
O
N
N
E
C
T
2
A
C
C
E
P
T
O
R
C
O
N
N
E
C
T
1
C
O
N
N
E
C
T
3
C
O
N
N
E
C
T
2
A
C
C
E
P
T
O
R
C
O
N
N
E
C
T
1
C
O
N
N
E
C
T
3
COMMUNICATION LINK
CLIENT APPLICATION
STUB STUB STUB
P1 P22 PP33 PP44
www.cs.wustl.edu/�schmidt/TSS-pattern.ps.gz
UC Irvine 94
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Flexibility Challenges� Problem
– Real-world ORBs must be flexible to satisfy the requirements ofmany different types of end-users and applications
� Forces
– Ad hoc schemes for ORB flexibility are too static andnon-extensible
– Flexibility often has many (related) dimensions
� Solution
– Use the Strategy pattern to support multiple transparently“pluggable” ORB strategies
UC Irvine 95
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing ORB Flexibility with the Strategy Pattern
StrategyStrategy
algorithm_interface()algorithm_interface()
ConcreteConcreteStrategy AStrategy A
algorithm_interface()algorithm_interface()
STRATEGY
ConcreteStrategy B
algorithm_interface()
ConcreteStrategy C
algorithm_interface()
Context
context_interface()
� Intent
– Factor out similarityamong algorithmicalternatives
� Forces Resolved
– Orthogonally replacebehavioral subsetstransparently
– Associating statewith an algorithm
UC Irvine 96
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Strategy Pattern in TAO
'&
$%
ORB COREORB CORE
OBJECTOBJECT ADAPTERADAPTER
DEMUXING
CODE
CONCURRENCY
CODE if (do_thread) // take lock...
...
if (do_thread) // releaselock...
CONNECTION
MANAGEMENT
CODE
if (do_thread)
// thread...
else
// single-threaded
'&
$%
Thread-Thread-SpecificSpecificConnectConnectStrategyStrategy
Cached CachedConnectConnectStrategyStrategy
Reactive ReactiveConcurrencyConcurrency
StrategyStrategy
Threaded ThreadedConcurrencyConcurrency
StrategyStrategy
SERVANT SERVANT 11 SERVANT NSERVANT N
IDLIDL
SKEL SKEL 22IDLIDL
SKEL NSKEL N
ORB COREORB CORE
SERVERSERVERCLIENTCLIENT
...... ............
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
NKK
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
N11
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
KK
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
22
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
11
(B) ACTIVE DEMUXING STRATEGY
index(object key)......
......
(A) PERFECT HASHING
DEMUXING
STRATEGY
OP
ER
AT
ION
OP
ER
AT
ION
KK
OP
ER
AT
ION
OP
ER
AT
ION
22
......
OP
ER
AT
ION
OP
ER
AT
ION
11
hash(operation)
hash(object key) OBJECTOBJECT
ADAPTERADAPTER
IDLIDL
SKEL SKEL 11
SERVANT SERVANT 22
ORB COREORB CORE
ORB COREORB CORE
StrategyStrategyAcceptorAcceptor
StrategyStrategyConnectorConnector
SunSoft IIOP TAO
UC Irvine 97
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Configurability Challenges
� Problem
– Aggressive use of Strategy pattern creates a configurationnightmare
� Forces
– Managing many individually configured strategies is hard– It’s hard to ensure that groups of semantically compatible
strategies are configured
� Solution
– Use the Abstract Factory pattern to consolidate multiple ORBstrategies into semantically compatible configurations
UC Irvine 98
High-performance, Real-time ORBs Douglas C. Schmidt
Centralizing ORB Configurability withthe Abstract Factory Pattern
AbstractAbstract
FactoryFactory
make_product_A()
make_product_B()
AbstractAbstract
Product_AProduct_A
ConcreteConcrete
Factory_1Factory_1
make_product_A()
make_product_B()
ConcreteConcrete
Factory_2Factory_2
make_product_A()
make_product_B()
AbstractAbstract
Product_BProduct_B
Product_B1Product_B1
Product_A2Product_A2
Product_B2Product_B2
Product_A1Product_A1
� Intent
– Integrate allstrategies used toconfigure an ORB
� Forces Resolved
– Consolidatescustomization ofmany strategies
– Ensuressemantically-compatiblestrategies
UC Irvine 99
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Abstract Factory Pattern in TAO
Dispatching
Strategy
Demuxing
Strategy
ORB
Server
Abstract
Factory
Concurrency
Strategy Thread-
per-
Connection
FIFO
Dispatching
Perfect
Hashing
Avionics AvionicsConcreteConcreteFactoryFactory
Rate-based Rate-based
DispatchingDispatching
Thread- Thread-
per-per-
PriorityPriority
ActiveActive
DemuxingDemuxing
Medical MedicalImagingImagingConcreteConcreteFactoryFactory
UC Irvine 100
High-performance, Real-time ORBs Douglas C. Schmidt
Addressing ORB Dynamic Configurability Challenges
� Problem
– Prematurely committing ourselves to a particular ORBconfiguration is inflexible and inefficient
� Forces
– Certain ORB configuration decisions can’t be made efficientlyuntil run-time
– Forcing users to pay for components they don’t use is undesirable
� Solution
– Use the Service Configurator pattern to assemble the desiredORB components dynamically
UC Irvine 101
High-performance, Real-time ORBs Douglas C. Schmidt
Enhancing Dynamic ORB Extensibilitywith the Service Configurator Pattern
init()fini()suspend()resume()info()
ServiceService
DispatchingDispatchingServiceService
ServiceServiceRepositoryRepository
services
ConcurrencyConcurrencyServiceService
DemuxingDemuxingServiceService
www.cs.wustl.edu/�schmidt/Svc-Conf.ps.gz
Intent
� Decouples ORB strategiesfrom time when they areconfigured
Forces Resolved
� Reduce resource utilization
� Support dynamic(re)configuration
UC Irvine 102
High-performance, Real-time ORBs Douglas C. Schmidt
Using the Service Configurator Pattern in TAO
dynamic ORB Service_Object * avionics_orb:make_orb() "-ORBport 2001"
Priority-basedPriority-basedDispatchingDispatching
TAOTAOPROCESSPROCESS
DLLDLLSS
Thread-per Thread-perRateRate
ConcurrencyConcurrency
AvionicsAvionicsConcreteConcreteFactoryFactory
PerfectPerfectHashingHashing
Service ServiceRepositoryRepository
ActiveActiveDemuxingDemuxing
MedicalMedicalImagingImagingConcreteConcreteFactoryFactory
FIFOFIFODispatchingDispatching
Thread-perThread-perConnectionConnection
ConcurrencyConcurrency
svc.confFILE
UC Irvine 103
High-performance, Real-time ORBs Douglas C. Schmidt
Quantifying the Benefits of Patterns
1 − 5 6 − 10 > 10MVG Range
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
% M
etho
ds in
Ran
ge
SunSoft IIOPTAO
Macabe Complexity Metric Scores forTAO and SunSoft IIOP
� Statistics
– Patterns greatlyreduce codecomplexity
� e.g., Most TAOcomponents have
v(G) < 10
– TAO components aresubstantially smallerthan SunSoft IIOP
� e.g., connectionmanagementreduced by a factorof 5
UC Irvine 104
High-performance, Real-time ORBs Douglas C. Schmidt
Lessons Learned Developing QoS-enabled ORBs
� Avoid dynamic connection management
� Minimize dynamic memory managementand data copying
� Avoid multiplexing connections fordifferent priority threads
� Avoid complex concurrency models
� Integrate ORB with OS and I/Osubsystem and avoid reimplementingOS mechanisms
� Guide ORB design by empiricalbenchmarks and patterns
'&
$%'
&
$%
UC Irvine 105
High-performance, Real-time ORBs Douglas C. Schmidt
Concluding Remarks
� Researchers and developers of distributed, real-time applicationsconfront many common challenges
– e.g., service initialization and distribution, error handling, flow control,scheduling, event demultiplexing, concurrency control, persistence, faulttolerance
� Successful researchers and developers apply patterns,frameworks, and components to resolve these challenges
� Careful application of patterns can yield efficient, predictable,scalable, and flexible middleware
– i.e., middleware performance is largely an “implementation detail”
� Next-generation ORBs will be highly QoS-enabled, though manyresearch challenges remain
UC Irvine 106 Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Cur
rent
Sta
tus
ofTA
O
SC
HE
DU
LIN
G
PR
OP
ER
TY
EV
EN
T/
NO
TIF
ICA
TIO
NT
RA
DIN
G
NA
MIN
GC
ON
CU
RR
EN
CY
TIM
E
A/V
ST
RE
AM
ING
NE
TW
OR
K
OR
B Q
OS
INT
ER
FA
CE
RID
LS
TU
BS
RID
LS
KE
LE
TO
N
CL
IEN
T
OS
KE
RN
EL
OS
I/
O S
UB
SY
ST
EM
NE
TW
OR
K A
DA
PT
ER
S
OS
KE
RN
EL
OS
I/
O S
UB
SY
ST
EM
NE
TW
OR
K A
DA
PT
ER
S
GIO
P/R
IOP
OB
JE
CT
(SE
RV
AN
T)
LO
GG
ING
CORBA SERVICES
IDL
CO
MP
ILE
R
RE
AL-T
IME
OR
B C
OR
E
AC
EC
OM
PO
NE
NT
S
LIF
EC
YC
LE
RE
AL-T
IME
OB
JE
CT
AD
AP
TE
R
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Synopsis of TAO’s Pattern-Oriented ORB Design
'&
$%
DIIDII ORBORBINTERFACEINTERFACE
IDLIDLSTUBSSTUBS
PORTABLEPORTABLE
OBJECTOBJECT
ADAPTERADAPTER
IDLIDLSKELETONSKELETON DSIDSI
CLIENTCLIENT
ORB COREORB CORE
SERVERSERVER
StrategyConnector
Cached
Connect
Strategy
CLIENT
5: REQUEST/RESPONSE
2: connect() 3: accept()
GIOP
Handler0..N
6:6: DISPATCH DISPATCH()()
4:4: CREATE CREATE && ACTIVATE ACTIVATE
1: operation()1: operation()
OBJECTOBJECT((SERVANTSERVANT))
Reactor Reactor
StrategyAcceptor
StrategyAcceptor
ConnectionHandler
ConnectionHandler
ConnectionHandler
ConnectionHandler
IDL COMPILER
'&
$%
......... ..................
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
NMM
SE
RV
AN
TS
ER
VA
NTN
::N
::O
PE
RA
TIO
NO
PE
RA
TIO
N11
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
MM
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
22
SE
RV
AN
TS
ER
VA
NT1::
1::
OP
ER
AT
ION
OP
ER
AT
ION
11
OBJECT ADAPTER
...
...IDL
SKEL 2
(A) LAYERED DEMUXING,PERFECT HASHING
SERVANT 1 SERVANT 2
OP
ER
AT
IONM
OP
ER
AT
ION2
...
OP
ER
AT
ION1
IDL
SKEL K
hash(operation)
IDL
SKEL 1
(C) DE-LAYERED ACTIVE
DEMUXING
hash(object key) index(object key/operation)
(B) LAYERED DEMUXING,DYNAMIC HASHING
SERVANT N
ORB CORE
OBJECT ADAPTER
ORB CORE
ORBCORE
GIOP/IIOP
DII ORBINTERFACE
operation()
IDLSTUBS OBJECT
ADAPTER
IDLSKELETON DSI
in argsin args
out args + return valueout args + return value
CLIENTCLIENT OBJECTOBJECT((SERVANTSERVANT))
UC Irvine 108
High-performance, Real-time ORBs Douglas C. Schmidt
Summary of TAO Research ProjectCompleted work
� First POA and first deployedreal-time CORBA schedulingservice
� Pluggable protocols framework
� Minimized ORB Core priorityinversion and non-determinism
� Reduced latency via demuxingoptimizations
� Co-submitters on OMG’sreal-time CORBA spec
Ongoing work
� Dynamic/hybrid scheduling
� Distributed QoS, ATM I/OSubsystem, & open signaling
� Implement CORBA Real-time,Messaging, and Fault Tolerancespecs
� Tech. transfer via DARPAQuorum program andwww.theaceorb.com
– Integration with Flick IDLcompiler, QuO, TMO, etc.
UC Irvine 109
High-performance, Real-time ORBs Douglas C. Schmidt
Summary: Real-time Optimizations in TAO
NETWORK
OS KERNEL
OS I/O SUBSYSTEM
NETWORK INTERFACES
ORBINTERFACE
ORBCORE
operation()
IDLSTUBS
OBJECT
ADAPTER
IDLSKELETON
in args
out args + return value
CLIENT
GIOP
OBJECT(SERVANT)
CONCURRENCY
MODELS
TRANSPORT
PROTOCOLS
I/OSUBSYSTEM
NETWORK
ADAPTER
PRESENTATION
LAYER
SCHEDULING,DEMUXING, &DISPATCHING
DATA COPYING
& MEMORY
ALLOCATION
CONNECTION
MANAGEMENT
OS KERNEL
OS I/O SUBSYSTEM
NETWORK INTERFACES
OBJ
REF
UC Irvine 110
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: New TAO Features and Optimizations
STANDARD
SYNCHRONIZERS
END-TO-END
PRIORITY
PROPAGATION
ORB CORE
OBJECT
ADAPTER
CLIENTOBJECT(SERVANT)
THREAD
POOLS
EXPLICIT
BINDING
NETWORK
OS KERNEL
NETWORK INTERFACE
OS I/O SUBSYSTEM
OS KERNEL
NETWORK INTERFACE
OS I/O SUBSYSTEM
PLUGGABLEPROTOCOLS &
PROPERTIES
GIOP GIOP
Forthcoming Features
� CORBA ComponentModel (CCM)
� Real-time and MinimumCORBA
� CORBA Messaging
� Fault-Tolerant CORBA
� Notification Service
www.cs.wustl.edu/
�schmidt/TAO-status.html
UC Irvine 111
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: Integrating QoS-EnabledCORBA Component Model with TAO
NETWORK
ORB RUN-TIMEIDLSTUBS
IDLSKELETON
OS KERNEL
HIGH-SPEED
NETWORK INTERFACE
REAL-TIME I/OSUBSYSTEM
OS KERNEL
HIGH-SPEED
NETWORK INTERFACE
REAL-TIME I/OSUBSYSTEM
ACE COMPONENTS
REAL-TIME ORB COREIOP
PLUGGABLEORB & XPORTPROTOCOLS
IOPPLUGGABLE
ORB & XPORTPROTOCOLS
QoS
ADAPTATION
HOME
COMPONENT
IMPLEMENTATION
CA
LL
BA
CK
S
REAL-TIME POASCHEDULER
IDL
SKELS
CLIENToperation ()
in args
out args + return value
REAL-TIME
PORTABLE OBJECT ADAPTER
CO
RB
A S
ER
VIC
ES
(SE
CU
RIT
Y,
EV
EN
TS
...)
Features
� Select optimalcommunication reflectively
� Re-factor component QoSaspects into their containers
� Dynamically load/unloadcomponentimplementations
UC Irvine 112
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: Integrating TAO with ATM I/O Subsystem
REACTOR
(20 HZ)
REACTOR
(10 HZ)
REACTOR
(5 HZ)
REACTOR
(1 HZ)
ORB CORE
APIC ATM DRIVER
I/OSUBSYSTEM
PER-VC
SOCKET
QUEUES
APIC PACING
QUEUES
Z
E
R
O
C
O
P
Y
B
U
F
F
E
R
S
�schmidt/RIO.ps.gz
Features
� Vertical integration of QoSthrough ORB, OS, and ATMnetwork
� Real-time I/O enhancementsto Solaris kernel
� Provides rate-based QoSend-to-end
� Leverages APIC features forcell pacing and zero-copybuffering
UC Irvine 113
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: Strategized Scheduling Framework ssttrruucctt RRTT__IInnffoo {{ wwcc__eexxeecc__ttiimmee__;; ppeerr iioodd__;; ccrr iittiiccaalliittyy__;; iimmppoorr ttaannccee__;; ddeeppeennddeenncciieess__;; }};;
SSCCHHEEDDUULLIINNGG
SSTTRRAATTEEGGYY
RRTT__IINNFFOO
RREEPPOOSSII TTOORRYY
OOFFFF--LL II NNEE
SSTTRRAATTEEGGIIZZEEDD
SSCCHHEEDDUULLEERR
OOFFFF--LLIINNEE
OONN--LLIINNEE
RRTT__IINNFFOO
RREEPPOOSSII TTOORRYY
RRUUNN--TTIIMMEE
SSCCHHEEDDUULLEERR
11.. SSPPEECCIIFFYY RRTT__OOPPEERRAATTIIOONN
CCHHAARRAACCTTEERRIISSTTIICCSS AANNDD
DDEEPPEENNDDEENNCCIIEESS
55.. AASSSSEESSSS SSCCHHEEDDUULLAABBIILL IITTYY
33.. AASSSSII GGNN SSTTAATTIICC PPRRIIOORRIITTYY AANNDD SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY
44.. MMAAPP SSTTAATTIICC PPRRIIOORRIITTYY,, DDYYNNAAMMII CC SSUUBBPPRRII OORRII TTYY,, AANNDD
SSTTAATTIICC SSUUBBPPRRIIOORRIITTYY II NNTTOO DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY
AANNDD DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY
66.. AASSSSIIGGNN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE CCOONNFFIIGGUURRAATTIIOONN
22.. PPOOPPUULLAATTEE
RRTT__IINNFFOO
RREEPPOOSSIITTOORRYY
77.. SSUUPPPPLLYY DDIISSPPAATTCCHHIINNGG QQUUEEUUEE
CCOONNFFIIGGUURRAATTIIOONN TTOO TTHHEE OORRBB
RRTTOOppeerraattiioonn
RRTTOOppeerraattiioonn
RRTTOOppeerraattiioonn
II //OO SSUUBBSSYYSSTTEEMM
OORRBB CCOORREE
OOBBJJEECCTT AADDAAPPTTEERR
99.. SSUUPPPPLLYY SSTTAATTIICC PPOORRTTIIOONNSS OOFF
DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY AANNDD
DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY
TTOO TTHHEE OORRBB
((SSCCHHEEDDUULLEERR''SS OOUUTTPPUUTT
IINNTTEERRFFAACCEE))
((SSCCHHEEDDUULLEERR''SS IINNPPUUTT
IINNTTEERRFFAACCEE))
88.. CCOONNFFIIGGUURREE QQUUEEUUEESS BBAASSEEDD
OONN DDIISSPPAATTCCHHIINNGG QQUUEEUUEE
CCOONNFFIIGGUURRAATTIIOONN
1100.. DDYYNNAAMMIICC QQUUEEUUEESS AASSSSIIGGNN
DDYYNNAAMMIICC PPOORRTTIIOONNSS OOFF
DDIISSPPAATTCCHHIINNGG SSUUBBPPRRIIOORRIITTYY
((AANNDD PPOOSSSSIIBBLLYY
DDIISSPPAATTCCHHIINNGG PPRRIIOORRIITTYY)) OORRBB EENNDDSSYYSSTTEEMM
www.cs.wustl.edu/�schmidt/dynamic.ps.gz
UC Irvine 114
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: Open ATM Signaling & Control
MANAGE
MIBTAO
RIO
APIC
NETWORK OPERATIONS
CENTER - ENDSYSTEM
GET STATSRECONFIG
SWITCH STATUS
CHANGE
RIO
APIC
TAO objB
objA
LOW LATENCY
CONTROLHIGH - BANDWIDTH
VIDEO/AUDIO DATA
ENDSYSTEM A
ENDSYSTEM B
HIGH - BANDWIDTH
BULK DATA
ENDSYSTEM C
HIGH -BANDWIDTH
VIDEO/AUDIO
MULTICAST
NETWORK
SERVICE
PROCESSOR
SPC
WUGS
SPC
SPC
SPC
SPC
SPC
SC
CLASSIFIER /ROUTER
ACTIVE NETWORK
RESOURCE CONTROLLER
NATIVE ATM BYPASSES THROUGH APIC
ATM PORT INTERCONNECT CONTROLLER (APIC)
USER
KERNEL
NSP
HIGH -PERFORMANCE
NETWORK ELEMENT
WEIGHTED
FAIR
QUEUING
IP FLOW
CLASSIFICATION
AND ROUTING
TAO
WUGS
OBJECT
SIGNALING
SERVERTAO
RESOURE
MANAGER
SWITCH
MANAGER
SWITCH CONTROLLER
HIGH -PERFORMANCE NETWORK ELEMENT
QOSMANAGER
TAO
PORT
CONTROLLER
SMART
PORT
CARD
UC Irvine 115
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Nex
tSte
ps:
Ada
ptiv
eM
iddl
ewar
e(
e.g.
,Q
uO/T
AO
)
INF
RA
ST
RU
CT
UR
EIN
FR
AS
TR
UC
TU
RE
MID
DL
EW
AR
EM
IDD
LE
WA
RE
DIS
TR
IBU
TIO
ND
IST
RIB
UT
ION
MID
DL
EW
AR
EM
IDD
LE
WA
RE
QQOO
S
S A
DA
PT
IVE
AD
AP
TIV
EL
AY
ER
LA
YE
R C
OM
MO
NC
OM
MO
NS
ER
VIC
ES
SE
RV
ICE
S
OP
ER
AT
ING
SY
ST
EM
OP
ER
AT
ING
SY
ST
EM
&& P
RO
TO
CO
LS
PR
OT
OC
OL
S
op
erati
on
()op
erati
on
()in
args
in args
ou
t a
rgs
+ r
etu
rn
valu
eou
t a
rgs
+ r
etu
rn
valu
eC
LIE
NT
CL
IEN
TO
BJE
CT
OB
JE
CT
((SE
RV
AN
TS
ER
VA
NT))
OP
ER
AT
ING
SY
ST
EM
OP
ER
AT
ING
SY
ST
EM
&& P
RO
TO
CO
LS
PR
OT
OC
OL
S
WIR
EL
ES
SW
IRE
LE
SS//W
IRE
LIN
E N
ET
WO
RK
WIR
EL
INE
N
ET
WO
RK
CL
IEN
T H
OS
TC
LIE
NT
H
OS
TS
ER
VE
R H
OS
TS
ER
VE
R H
OS
T
RESOURCE
RESOURCE
MANAGERS
MANAGERS
RESOURCE
RESOURCE
MANAGERS
MANAGERS
OB
JO
BJ
RE
FR
EF
STATUS
STATUS
COLLECTORS
COLLECTORS
BANDWIDTH
BANDWIDTH
CONTROL
CONTROL
PROPERTY
PROPERTY
MANAGERS
MANAGERS
POLICY
POLICY
MANAGERS
MANAGERS
QO
S M
EC
HA
NIS
MS
QO
S M
EC
HA
NIS
MS
...
QO
S M
AN
AG
ER
S
...
INF
RA
ST
RU
CT
UR
EM
IDD
LE
WA
RE
DIS
TR
IBU
TIO
NM
IDD
LE
WA
RE
QO
S A
DA
PT
IVE
LA
YE
R C
OM
MO
NS
ER
VIC
ES
ww
w.d
ist-
syst
ems.
bbn.
com
/pap
ers/
Key
The
mes
�
Dec
oupl
efu
nctio
nal
path
from
QoS
path
�
Em
phas
ize
inte
grat
ion
and
confi
gura
tion
UC
Irvi
ne
Hig
h-pe
rform
ance
,R
eal-t
ime
OR
Bs
Do
Nex
tSte
ps:
Aud
io/V
ideo
Str
eam
ing
End
-poi
ntFl
ow d
ata
Ada
ptor
Stre
am
(Sou
rce)
Flow
dat
a
Ada
ptor
Obj
ects
And
Man
agem
ent
Obj
ect
Con
trol
Inte
rfac
e
OR
B C
OR
E
POA
End
-poi
nt
Stre
am
Con
trol
Inte
rfac
e
Obj
ect
Stre
am
Stre
am POA
Con
trol
(Sin
k)
MU
LT
IME
DIA
STR
EA
M
ww
w.c
s.w
ustl.
edu/
�
schm
idt/a
v.ps
.gz
Effi
cien
cy
�
Soc
kets
for
data
tran
sfer
toge
thig
hpe
rfor
man
ce
Fle
xibi
lity
�
Use
sC
OR
BA
for
cont
rol
mes
sage
san
dpr
oper
ties
UC
Irvi
ne
High-performance, Real-time ORBs Douglas C. Schmidt
Next Steps: Distributed Interactive Simulations
NETWORKNETWORKOPERATIONSOPERATIONS
CENTERCENTER
HSMHSM
ARCHIVEARCHIVE
SERVERSERVER
AGENTAGENT
INTERACTIVEINTERACTIVE
AUDIOAUDIO//VIDEOVIDEO
AGENTAGENT ARCHITECTUREARCHITECTURE
SPCSPC
HARDWAREHARDWARE
EMBEDDEDEMBEDDED
TAOTAO
MIBMIB
AGENTAGENT
UC Irvine 118
High-performance, Real-time ORBs Douglas C. Schmidt
Web URLs for Additional Information� These slides: �schmidt/TAO4.ps.gz
� More information on CORBA: �schmidt/corba.html
� More info on ACE: �schmidt/ACE.html
� More info on TAO: �schmidt/TAO.html
� TAO Event Channel: �schmidt/JSAC-98.ps.gz
� TAO static scheduling: �schmidt/TAO.ps.gz
� TAO dynamic scheduling: �schmidt/dynamic.ps.gz
� ORB Endsystem Architecture: �schmidt/RIO.ps.gz
� Pluggable protocols: �schmidt/pluggable protocols.ps.gz
UC Irvine 119
High-performance, Real-time ORBs Douglas C. Schmidt
Web URLs for Additional Information (cont’d)
� Network monitoring, visualization, & control: �schmidt/NMVC.html
� Performance Measurements:
– Demuxing latency: �schmidt/COOTS-99.ps.gz
– SII throughput: �schmidt/SIGCOMM-96.ps.gz
– DII throughput: �schmidt/GLOBECOM-96.ps.gz
– ORB latency & scalability: �schmidt/ieee tc-97.ps.gz
– IIOP optimizations: �schmidt/JSAC-99.ps.gz
– Concurrency and connection models: �schmidt/RT-perf.ps.gz
– RTOS/ORB benchmarks:
�schmidt/RT-OS.ps.gz
�schmidt/words-99.ps.gz
UC Irvine 120