Java程序辅导

C++ Network Programming with Patterns,
Frameworks, and ACE
Douglas C. Schmidt
Professor Department of EECS
d.schmidt@vanderbilt.edu Vanderbilt University
www.cs.wustl.edu/schmidt/ (615) 343-8197
Sponsors
NSF, DARPA, ATD, BBN, Boeing, Cisco, Comverse, GDIS, Experian, Global MT,
Hughes, Kodak, Krones, Lockheed, Lucent, Microsoft, Mitre, Motorola, NASA, Nokia,
Nortel, OCI, Oresis, OTI, QNX, Raytheon, SAIC, Siemens SCR, Siemens MED,
Siemens ZT, Sprint, Telcordia, USENIX
Advanced ACE Tutorial Douglas C. Schmidt
Roadmap to Levels of Middleware
HOST INFRASTRUCTURE MIDDLEWARE
DISTRIBUTION MIDDLEWARE
COMMON MIDDLEWARE SERVICES
APPLICATIONS
HARDWARE DEVICES
WTS
HUD
Nav
AVIONICS
REPLICATION
SERVICE
DOMAIN-SPECIFIC MIDDLEWARE SERVICES
OPERATING SYSTEMS & PROTOCOLS
EVENT
CHANNEL
Cons
Cons
Cons
www.cs.wustl.edu/˜schmidt/PDF/
middleware-chapter.pdf
 Observations
– Historically, apps
built atop OS
– Today, apps built
atop middleware
– Middleware has
multiple layers
 Just like network
protocol stacks
Vanderbilt University 1
Advanced ACE Tutorial Douglas C. Schmidt
Motivation for Concurrency
SERVER
WORK
REQUEST
WORK
REQUEST
WORK
REQUEST
WORK
REQUEST
CLIENT
CLIENT
CLIENT CLIENT
(2) CONCURRENT SERVER
maxfdp1
read_fds
WORK
REQUEST
SERVER
CLIENT
WORK
REQUEST
WORK
REQUEST
WORK
REQUESTCLIENT
CLIENT CLIENT
(1) ITERATIVE SERVER
 Leverage hardware/software
– e.g., multi-processors and OS
thread support
 Increase performance
– e.g., overlap computation and
communication
 Improve response-time
– e.g., GUIs and network servers
 Simplify program structure
– e.g., sync vs. async
Vanderbilt University 2
Advanced ACE Tutorial Douglas C. Schmidt
Motivation for Distribution
PRINTER
FILE
SYSTEM
PRINTER FILE  SYSTEM
COMPUTER
(1)  STAND-ALONE  APPLICATION  ARCHITECTURE
(2)  DISTRIBUTED  APPLICATION  ARCHITECTURE
CD ROM
CD ROM
NETWORK
FI LE
SERVICE
CYCLE
SERVICE
DISPLAY
SERVICE
PRINT
SERVICE
NAME
SERVICE
TIME
SERVICE
 Collaboration! connectivity and
interworking
 Performance! multi-processing
and locality
 Reliability and availability!
replication
 Scalability and portability!
modularity
 Extensibility! dynamic
configuration and reconfiguration
 Cost effectiveness! open
systems and resource sharing
Vanderbilt University 3
Advanced ACE Tutorial Douglas C. Schmidt
Challenges and Solutions
 Developing efficient, robust, and extensible concurrent networking
applications is hard
– e.g., must address complex topics that are less problematic or not
relevant for non-concurrent, stand-alone applications
 OO techniques and OO language features help to enhance software
quality factors
– Key OO techniques include patterns and frameworks
– Key OO language features include classes, inheritance, dynamic
binding, and parameterized types
– Key software quality factors include modularity, extensibility,
portability, reusability, and correctness
Vanderbilt University 4
Advanced ACE Tutorial Douglas C. Schmidt
Caveats
 OO is not a panacea
– Though when used properly it helps minimize “accidental”
complexity and improve software quality factors
 It’s also essential to understand advanced OS features to enhance
functionality and performance, e.g.,
– Multi-threading
– Multi-processing
– Synchronization
– Shared memory
– Explicit dynamic linking
– Communication protocols and IPC mechanisms
Vanderbilt University 5
Advanced ACE Tutorial Douglas C. Schmidt
Tutorial Outline
 Brief overview of key OO networking and concurrency concepts and
OS platform mechanisms
– Emphasis is on practical solutions
 Examine a range of examples in detail
– Networked Logging Service
– Concurrent Web Server
– Application-level Telecom Gateway
– Call Center Manager Event Server
 Discuss general concurrent programming strategies
 Provide URLs for further reading on the topic
Vanderbilt University 6
Advanced ACE Tutorial Douglas C. Schmidt
Software Development Environment
 The topics discussed here are largely independent of OS, network,
and programming language
– Currently used successfully on UNIX/POSIX, Windows, and
RTOS platforms, running on TCP/IP networks using C++
 Examples are illustrated using freely available ADAPTIVE
Communication Environment (ACE) OO framework components
– Although ACE is written in C++, the principles covered in this
tutorial apply to other OO languages
– e.g., Java, Eiffel, Smalltalk, etc.
 In addition, other networks and backplanes can be used, as well
Vanderbilt University 7
Advanced ACE Tutorial Douglas C. Schmidt
Sources of Complexity
PRINTER
FILE
SYSTEM
PRINTER FILE  SYSTEM
COMPUTER
(1)  STAND-ALONE  APPLICATION  ARCHITECTURE
(2)  DISTRIBUTED  APPLICATION  ARCHITECTURE
CD ROM
CD ROM
NETWORK
FI LE
SERVICE
CYCLE
SERVICE
DISPLAY
SERVICE
PRINT
SERVICE
NAME
SERVICE
TIME
SERVICE
 Inherent complexity
– Latency
– Reliability
– Synchronization
– Deadlock
 Accidental Complexity
– Low-level APIs
– Poor debugging tools
– Algorithmic
decomposition
– Continuous
re-invention
Vanderbilt University 8
Advanced ACE Tutorial Douglas C. Schmidt
Sources of Inherent Complexity
Inherent complexity results from fundamental domain challenges,
e.g.:
Concurrent programming
 Eliminating “race conditions”
 Deadlock avoidance
 Fair scheduling
 Performance optimization
and tuning
Distributed programming
 Addressing the impact of latency
 Fault tolerance and high availability
 Load balancing and service
partitioning
 Consistent ordering of distributed
events
Vanderbilt University 9
Advanced ACE Tutorial Douglas C. Schmidt
Sources of Accidental Complexity
Accidental complexity results from limitations with tools and techniques
used to develop concurrent applications, e.g.,
 Lack of portable, reentrant, type-safe and extensible system call
interfaces and component libraries
 Inadequate debugging support and lack of concurrent and
distributed program analysis tools
 Widespread use of algorithmic decomposition
– Fine for explaining concurrent programming concepts and
algorithms but inadequate for developing large-scale concurrent
network applications
 Continuous rediscovery and reinvention of core concepts and
components
Vanderbilt University 10
Advanced ACE Tutorial Douglas C. Schmidt
OO Contributions to Concurrent
and Distributed Applications
Concurrent network programming is
traditionally performed using
low-level OS mechanisms, e.g.,
 fork/exec
 Shared memory and semaphores
 Memory-mapped files
 Signals
 sockets/select
 Low-level thread APIs
Patterns and frameworks elevate
development level to focus on
application concerns, e.g.,
 Service functionality and
policies
 Service configuration
 Concurrent event
demultiplexing and event
handler dispatching
 Service concurrency and
synchronization
Vanderbilt University 11
Advanced ACE Tutorial Douglas C. Schmidt
Overview of Patterns
 Patterns represent solutions to problems that arise when developing
software within a particular context
– i.e., “Patterns == problem/solution pairs within a context”
 Patterns capture the static and dynamic structure and collaboration
among key participants in software designs
– They are particularly useful for articulating how and why to
resolve non-functional forces
 Patterns facilitate reuse of successful software architectures and
designs
Vanderbilt University 12
A d v a n c e d A C E T u t o r i a l D o
E x a m p l e : t h e P r o x y P a t t e r n
N E T W O R K
C L I E N T
S E R V E R
2 :  F O R W A R D   R E Q U E S T
3 :   R E S P O N S E
:  Q U O T E R
1 :  M E T H O D   C A L L
4 :  M E T H O D   R E T U R N
:  Q U O T E R
P R O X Y
:  B R O K E R
I n t e n t : P r o v i d e a s u r r o g a t e f o r a n o t h e r o b j e c t t h a t
c o n t r o l s a c c e s s t o i t
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Overview of Frameworks and Components
 A framework is:
– “An integrated collection of components that collaborate to
produce a reusable architecture for a family of related applications”
 Frameworks differ from conventional class libraries:
1. Frameworks are “semi-complete” applications
2. Frameworks address a particular application domain
3. Frameworks provide “inversion of control”
 Frameworks facilitate reuse of successful networked application
software designs and implementations
– Applications inherit from and instantiate framework components
Vanderbilt University 14
Advanced ACE Tutorial Douglas C. Schmidt
Class Libraries versus Frameworks
NETWORKING
DATABASE
GUI
EVENT
LOOP
APPLICATION-
SPECIFIC
FUNCTIONALITY
EVENT
LOOP
EVENT
LOOP
CALL
BACKSINVOKES
(A) CLASS  LIBRARY  ARCHITECTURE
(B) FRAMEWORK  ARCHITECTURE
DATABASE
CLASSES
NETWORK
IPC
CLASSES
MATH
CLASSES
ADT
CLASSES
GUI
CLASSES
APPLICATION-
SPECIFIC
FUNCTIONALITY
EVENT
LOOP
GLUE
CODE
LOCAL
INVOCATIONS
ADT
CLASSES
MATH
CLASSES
Key distinctions
 Class libraries
– Reusable building blocks
– Domain-independent
– Limited in scope
– Passive
 Frameworks
– Reusable, “semi-complete”
applications
– Domain-specific
– Broader in scope
– Active
Vanderbilt University 15
Advanced ACE Tutorial Douglas C. Schmidt
The ADAPTIVE Communication Environment (ACE)
PROCESSES/
THREADS
DYNAMIC
LINKING
SHARED
MEMORY
SELECT/
IO COMP
FILE SYS
APIS
WIN32 NAMED
PIPES & UNIX
STREAM PIPES
UNIX
FIFOS
C
APIS
SOCKETS/
TLI
COMMUNICATION
SUBSYSTEM
VIRTUAL MEMORY & FILE
SUBSYSTEM
GENERAL OPERATING SYSTEM SERVICES
PROCESS/THREAD
SUBSYSTEM
FRAMEWORK
LAYER
ACCEPTOR CONNECTOR
NETWORKED
SERVICE
COMPONENTS
LAYER
NAME
SERVER
TOKEN
SERVER
LOGGING
SERVER
GATEWAY
SERVER
SOCK SAP/
TLI SAP
FIFO
SAP
LOG
MSG
SERVICE
HANDLER
TIME
SERVER
C++
WRAPPER
FACADE
LAYER SPIPE
SAP
CORBA
HANDLER
FILE
SAP
SHARED
MALLOC
THE ACE ORB
(TAO)
JAWS ADAPTIVE
WEB SERVER
STANDARDS-BASED MIDDLEWARE
REACTOR/
PROACTOR
PROCESS/
THREAD
MANAGERS
STREAMS
SERVICE
CONFIG-
URATOR
SYNCH
WRAPPERS
MEM
MAP
OS ADAPTATION LAYER
www.cs.wustl.edu/˜schmidt/ACE.html
Vanderbilt University 16
Advanced ACE Tutorial Douglas C. Schmidt
ACE Statistics
 ACE library contains  250,000
lines of C++
– Over 40 person-years of effort
 Ported to UNIX, Windows, MVS, and
RT/embedded platforms
– e.g., VxWorks, LynxOS, Chorus
 Large user and open-source
developer community
– ˜schmidt/ACE-users.html
 Currently used by
dozens of companies
– Bellcore, BBN,
Boeing, Ericsson,
Hughes, Kodak,
Lockheed, Lucent,
Motorola, Nokia,
Nortel, Raytheon,
SAIC, Siemens, etc.
 Supported commercially
by Riverace
– www.riverace.com
Vanderbilt University 17
Advanced ACE Tutorial Douglas C. Schmidt
The Key Frameworks in ACE
Acceptor-
Connector
Reactor Proactor
Service
Configurator Streams Task
 ACE contains a number of frameworks that can be used separately
or together
 This design permits fine-grained subsetting of ACE components
– Subsetting helps minimize ACE’s memory footprint
– $ACE_ROOT/doc/ACE-subsets.html
Vanderbilt University 18
Advanced ACE Tutorial Douglas C. Schmidt
Patterns for Communication Middleware
Event
Patterns
Concurrency
Patterns
External
Polymorphism
Wrapper
Facade
Connector
Thread
Pool
Thread-per
Session
Thread-per
Request
Asynchronous
Completion
Token
Thread
Specific
Storage
Active
Object
Half-Sync/
Half-Async
Leader/
Followers
Component
Configurator
Object
 Lifetime
Manager
Reactor
Proactor
Double
Checked
Locking
Thread-
Safe
Interface
Scoped
Locking
Strategized
Locking
Initialization
Patterns
Synchronization
Patterns
Acceptor
Observation
 Failures rarely result from
unknown scientific
principles, but from failing
to apply proven
engineering practices and
patterns
Benefits of Patterns
 Facilitate design reuse
 Preserve crucial design
information
 Guide design choices
Vanderbilt University 19
Advanced ACE Tutorial Douglas C. Schmidt
The ACE ORB (TAO)
NETWORK
ORB RUN-TIME
SCHEDULER
operation()
IDL
STUBS
IDL
SKELETON
in  args
out  args + return
value
CLIENT
OS  KERNEL
HIGH-SPEED
NETWORK  INTERFACE
REAL-TIME  I/O
SUBSYSTEM
OBJECT
(SERVANT)
OS  KERNEL
HIGH-SPEED
NETWORK  INTERFACE
REAL-TIME  I/O
SUBSYSTEM
ACE
COMPONENTS
OBJ
REF
REAL-TIME  ORB  CORE
IOP
PLUGGABLE
ORB & XPORT
PROTOCOLS
IOP
PLUGGABLE
ORB & XPORT
PROTOCOLS
REAL-TIME
OBJECT
ADAPTER
www.cs.wustl.edu/˜schmidt/TAO.
html
TAO Overview !
 A real-time,
high-performance
ORB
 Leverages ACE
– Runs on POSIX,
Windows,
RTOSs
Related efforts !
 QuO at BBN
 MIC/GME at
Vanderbilt
 XOTS
Vanderbilt University 20
Advanced ACE Tutorial Douglas C. Schmidt
TAO Statistics
 TAO order of magnitude
– Core ORB > 300,000 LOC
– IDL compiler > 200,000
LOC
– CORBA Object Services >
250,000 LOC
– Leverages ACE heavily
 Ported to UNIX, Windows, &
RT/embedded platforms
– e.g., VxWorks, LynxOS,
Chorus, WinCE
  50 person-years of effort
 Currently used by many
companies
– e.g., Boeing, BBN, Lockheed,
Lucent, Motorola, Raytheon,
SAIC, Siemens, etc.
 Supported commercially by OCI
and PrismTech
– www.ociweb.com
– www.prismtechnologies.com
Vanderbilt University 21
Advanced ACE Tutorial Douglas C. Schmidt
JAWS Adaptive Web Server
WWW
SERVER
2: index.html
1: GET ~schmidt
HTTP/1.0
COMMUNICATION  PROTOCOL
(E.G., HTTP)
GUI
HTML
PARSER
REQUESTER
GRAPHICS
ADAPTER
NETWORK
OS  KERNEL
OS  I/O  SUBSYSTEM
NETWORK  ADAPTERS
OS  KERNEL
OS  I/O  SUBSYSTEM
NETWORK  ADAPTERS
DISPATCHER
PROTOCOL
HANDLERS
WWW
CLIENT
www.cs.wustl.edu/˜jxh/
research/
 JAWS Overview
– A high-performance
Web server
 Flexible concurrency
and dispatching
mechanisms
– Leverages the ACE
framework
 Ported to most OS
platforms
– Used commercially by
CacheFlow
 www.cacheflow.com
Vanderbilt University 22
Advanced ACE Tutorial Douglas C. Schmidt
Java ACE
FRAMEWORKS
AND  CLASS
CATEGORIES
DISTRIBUTED
SERVICES  AND
COMPONENTS
NAME
SERVER
TOKEN
SERVER
LOGGING
SERVER
TIME
SERVER
JAVA
WRAPPERS
SYNCH
WRAPPERS
SOCK_SAP THREAD
MANAGER
LOG
MSG
TIMER
QUEUE
SERVICE
CONFIGURATOR
ADAPTIVE  SERVICE  EXECUTIVE  (ASX)
ACCEPTOR CONNECTOR
SERVICE
HANDLER
JAVA  VIRTUAL  MACHINE  (JVM)
www.cs.wustl.edu/˜schmidt/JACE.html
www.cs.wustl.edu/˜schmidt/C++2java.
html
www.cs.wustl.edu/˜schmidt/PDF/
MedJava.pdf
Java ACE
Overview
 A Java version
of ACE
– Used for
medical
imaging
prototype
Vanderbilt University 23
A d v a n c e d A C E T u t o r i a l D o
N e t w o r k e d L o g g i n g S e r v i c e
P
1
P
2
P
3
L O C A L   I P C
C L I E N T
L O G G I N G
D A E M O N
P
1
P
2
P
3
L O C A L   I P C
C L I E N T
L O G G I N G
D A E M O N
N E T W O R K
S T O R A G E
D E V I C E
H O S T   A H O S T   B
A B
S E R V E R   L O G G I N G
D A E M O N
S E R V E R
C L I E N T
H O S T
A
REMOTE  IPC
H O S T
B
R
E
M
O
TE
  I
P
C
C L I E N T
C O N S O L E
P R I N T E R
I n t e n t : Server logging daemon c o l l e c t s , f o r m a t s ,
a n d o u t p u t s l o g g i n g r e c o r d s f o r w a r d e d f r o m client
logging daemons r e s i d i n g t h r o u g h o u t a n e t w o r k o r
I n t e r n e t
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Networked Logging Service Programming API
The logging API is similar to printf(), e.g.:
ACE_ERROR ((LM_ERROR, "(%t) fork failed"));
Generates on logging server host:
Oct 31 14:50:13 1992@tango.ics.uci.edu@2766@LM_ERROR@client
::(4) fork failed
and
ACE_DEBUG ((LM_DEBUG,
"(%t) sending to server %s", server_host));
generates on logging server host:
Oct 31 14:50:28 1992@zola.ics.uci.edu@18352@LM_DEBUG@drwho
::(6) sending to server bastille
Vanderbilt University 25
Advanced ACE Tutorial Douglas C. Schmidt
Conventional Logging Server Design
Typical algorithmic pseudo-code for
networked logging server:
void logging_server (void) {
initialize acceptor endpoint
loop forever {
wait for events
handle data events
handle connection events
}
}
The “grand mistake:”
 Avoid the temptation to
“step-wise refine” this
algorithmically
decomposed
pseudo-code directly into
the detailed design and
implementation of the
logging server!
Vanderbilt University 26
A d v a n c e d A C E T u t o r i a l D o
T h e s e l e c t ( ) - b a s e d L o g g i n g S e r v e r
I m p l e m e n t a t i o n
N E T W O R K
S E R V E R
L O G G I N G   D A E M O N
m a x h a n d l e p 1
r e a d _ h a n d l e s
C O N N E C T I O N
R E Q U E S T
L O G G I N G
R E C O R D S
L O G G I N G
R E C O R D S
L O G G I N G
R E C O R D S
C L I E N T
C L I E N T
C L I E N T
C L I E N T
S E R V E R
a c c e p t o r
S e r i a l i z e s s e r v e r p r o c e s s i n g a t s e l e c t ( )
d e m u x i n g l e v e l
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C o n v e n t i o n a l L o g g i n g S e r v e r
I m p l e m e n t a t i o n
N o t e t h e e x c e s s i v e a m o u n t o f d e t a i l r e q u i r e d t o
p r o g r a m a t t h e s o c k e t l e v e l . . .
/ / M a i n p r o g r a m
s t a t i c c o n s t i n t P O R T = 1 0 0 0 0 ;
t y p e d e f u _ l o n g C O U N T E R ;
t y p e d e f i n t H A N D L E ;
/ / C o u n t s t h e # o f l o g g i n g r e c o r d s p r o c e s s e d
s t a t i c C O U N T E R r e q u e s t _ c o u n t ;
/ / A c c e p t o r - m o d e s o c k e t h a n d l e
s t a t i c H A N D L E a c c e p t o r ;
/ / H i g h e s t a c t i v e h a n d l e n u m b e r , p l u s 1
s t a t i c H A N D L E m a x h p 1 ;
/ / S e t o f c u r r e n t l y a c t i v e h a n d l e s
s t a t i c f d _ s e t a c t i v i t y _ h a n d l e s ;
/ / S c r a t c h c o p y o f a c t i v i t y _ h a n d l e s
s t a t i c f d _ s e t r e a d y _ h a n d l e s ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
M a i n E v e n t L o o p o f L o g g i n g S e r v e r
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] )
{
i n i t i a l i z e _ a c c e p t o r
( a r g c > 1 ? a t o i ( a r g v [ 1 ] ) : P O R T ) ;
/ / L o o p f o r e v e r p e r f o r m i n g l o g g i n g
/ / s e r v e r p r o c e s s i n g .
f o r ( ; ; ) {
/ / s t r u c t a s s i g n m e n t .
r e a d y _ h a n d l e s = a c t i v i t y _ h a n d l e s ;
/ / W a i t f o r c l i e n t I / O e v e n t s .
s e l e c t ( m a x h p 1 , & r e a d y _ h a n d l e s , 0 , 0 , 0 ) ;
/ / F i r s t r e c e i v e p e n d i n g l o g g i n g r e c o r d s .
h a n d l e _ d a t a ( ) ;
/ / T h e n a c c e p t p e n d i n g c o n n e c t i o n s .
h a n d l e _ c o n n e c t i o n s ( ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
I n i t i a l i z e A c c e p t o r S o c k e t
s t a t i c v o i d i n i t i a l i z e _ a c c e p t o r ( u _ s h o r t p o r t )
{
s t r u c t s o c k a d d r _ i n s a d d r ;
/ / C r e a t e a l o c a l e n d p o i n t o f c o m m u n i c a t i o n .
a c c e p t o r = s o c k e t ( P F _ I N E T , S O C K _ S T R E A M , 0 ) ;
/ / S e t u p t h e a d d r e s s i n f o . t o b e c o m e s e r v e r .
m e m s e t ( ( v o i d * ) & s a d d r , 0 , s i z e o f s a d d r ) ;
s a d d r . s i n _ f a m i l y = A F _ I N E T ;
s a d d r . s i n _ p o r t = h t o n s ( p o r t ) ;
s a d d r . s i n _ a d d r . s _ a d d r = h t o n l ( I N A D D R _ A N Y ) ;
/ / A s s o c i a t e a d d r e s s w i t h e n d p o i n t
b i n d ( a c c e p t o r ,
( s t r u c t s o c k a d d r * ) & s a d d r ,
s i z e o f s a d d r ) ;
/ / M a k e e n d p o i n t l i s t e n f o r c o n n e c t i o n r e q u e s t s .
l i s t e n ( a c c e p t o r , 5 ) ;
/ / I n i t i a l i z e h a n d l e s e t s .
F D _ Z E R O ( & r e a d y _ h a n d l e s ) ;
F D _ Z E R O ( & a c t i v i t y _ h a n d l e s ) ;
F D _ S E T ( a c c e p t o r , & a c t i v i t y _ h a n d l e s ) ;
m a x h p 1 = a c c e p t o r + 1 ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
H a n d l e D a t a P r o c e s s i n g
s t a t i c v o i d h a n d l e _ d a t a ( v o i d ) {
/ / a c c e p t o r + 1 i s t h e l o w e s t c l i e n t h a n d l e
f o r ( H A N D L E h = a c c e p t o r + 1 ; h < m a x h p 1 ; h + + )
i f ( F D _ I S S E T ( h , & r e a d y _ h a n d l e s ) ) {
s s i z e _ t n = h a n d l e _ l o g _ r e c o r d ( h , 1 ) ;
/ / G u a r a n t e e d n o t t o b l o c k i n t h i s c a s e !
i f ( n > 0 )
+ + r e q u e s t _ c o u n t ;
/ / C o u n t t h e # o f l o g g i n g r e c o r d s
e l s e i f ( n = = 0 ) {
/ / H a n d l e c o n n e c t i o n s h u t d o w n .
F D _ C L R ( h , & a c t i v i t y _ h a n d l e s ) ;
c l o s e ( h ) ;
i f ( h + 1 = = m a x h p 1 ) {
/ / S k i p p a s t u n u s e d h a n d l e s
w h i l e ( ! F D _ I S S E T ( - - h ,
& a c t i v i t y _ h a n d l e s ) )
c o n t i n u e ;
m a x h p 1 = h + 1 ;
}
}
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
R e c e i v e a n d P r o c e s s L o g g i n g R e c o r d s
s t a t i c s s i z e _ t h a n d l e _ l o g _ r e c o r d ( H A N D L E i n _ h ,
H A N D L E o u t _ h ) {
s s i z e _ t n ;
s i z e _ t l e n ;
L o g _ R e c o r d l r ;
/ / T h e f i r s t r e c v r e a d s t h e l e n g t h ( s t o r e d a s a
/ / f i x e d - s i z e i n t e g e r ) o f a d j a c e n t l o g g i n g r e c o r d .
n = r e c v ( i n _ h , ( c h a r * ) & l e n , s i z e o f l e n , 0 ) ;
i f ( n < = 0 ) r e t u r n n ;
l e n = n t o h l ( l e n ) ; / / C o n v e r t b y t e - o r d e r i n g
/ / T h e s e c o n d r e c v t h e n r e a d s < l e n > b y t e s t o
/ / o b t a i n t h e a c t u a l r e c o r d .
f o r ( s i z e _ t n r e a d = 0 ; n r e a d < l e n ; n r e a d + = n
n = r e c v ( i n _ h , ( ( c h a r * ) & l r ) + n r e a d ,
l e n - n r e a d , 0 ) ;
/ / D e c o d e a n d p r i n t r e c o r d .
d e c o d e _ l o g _ r e c o r d ( & l r ) ;
i f ( w r i t e ( o u t _ h , l r . b u f , l r . s i z e ) = = - 1 )
r e t u r n - 1 ;
e l s e r e t u r n 0 ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
H a n d l e C o n n e c t i o n A c c e p t a n c e
s t a t i c v o i d h a n d l e _ c o n n e c t i o n s ( v o i d )
{
i f ( F D _ I S S E T ( a c c e p t o r , & r e a d y _ h a n d l e s ) ) {
s t a t i c s t r u c t t i m e v a l p o l l _ t v = { 0 , 0 } ;
H A N D L E h ;
/ / H a n d l e a l l p e n d i n g c o n n e c t i o n r e q u e s t s
/ / ( n o t e u s e o f s e l e c t ’ s p o l l i n g f e a t u r e )
d o {
/ / B e w a r e o f s u b t l e b u g ( s ) h e r e . . .
h = a c c e p t ( a c c e p t o r , 0 , 0 ) ;
F D _ S E T ( h , & a c t i v i t y _ h a n d l e s ) ;
/ / G r o w m a x . s o c k e t h a n d l e i f n e c e s s a r y .
i f ( h > = m a x h p 1 )
m a x h p 1 = h + 1 ;
} w h i l e ( s e l e c t ( a c c e p t o r + 1 , & r e a d y _ h a n d l e s ,
0 , 0 , & p o l l _ t v ) = = 1 ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C o n v e n t i o n a l C l i e n t L o g g i n g
D a e m o n I m p l e m e n t a t i o n
T h e m a i n ( ) m e t h o d r e c e i v e s l o g g i n g r e c o r d s f r o m
c l i e n t a p p l i c a t i o n s a n d f o r w a r d s t h e m o n t o t h e
l o g g i n g s e r v e r
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] )
{
H A N D L E s t r e a m = i n i t i a l i z e _ s t r e a m _ e n d p o i n t
( a r g c > 1
? a t o i ( a r g v [ 1 ] )
: P O R T ) ;
L o g _ R e c o r d l r ;
/ / L o o p f o r e v e r p e r f o r m i n g c l i e n t
/ / l o g g i n g d a e m o n p r o c e s s i n g .
f o r ( ; ; ) {
/ / . . . g e t l o g g i n g r e c o r d s f r o m c l i e n t
/ / a p p l i c a t i o n p r o c e s s e s . . .
s i z e _ t s i z e = h t o n l ( l r . s i z e ) ;
s e n d ( s t r e a m , & s i z e , s i z e o f s i z e ) ;
e n c o d e _ l o g _ r e c o r d ( & l r ) ;
s e n d ( s t r e a m , ( ( c h a r * ) & l r ) , s i z e o f l r ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C l i e n t C o n n e c t i o n E s t a b l i s h m e n t
s t a t i c H A N D L E i n i t i a l i z e _ s t r e a m _ e n d p o i n t
( c o n s t c h a r * h o s t , u _ s h o r t p o r t )
{
s t r u c t s o c k a d d r _ i n s a d d r ;
/ / C r e a t e a l o c a l e n d p o i n t o f c o m m u n i c a t i o n .
H A N D L E s t r e a m = s o c k e t ( P F _ I N E T , S O C K _ S T R E A M , 0 ) ;
/ / S e t u p t h e a d d r e s s i n f o . t o b e c o m e c l i e n t .
m e m s e t ( ( v o i d * ) & s a d d r , 0 , s i z e o f s a d d r ) ;
s a d d r . s i n _ f a m i l y = A F _ I N E T ;
s a d d r . s i n _ p o r t = h t o n s ( p o r t ) ;
h o s t e n t * h p = g e t h o s t b y n a m e ( h o s t ) ;
m e m c p y ( ( v o i d * ) & s a d d r ,
h t o n l ( h p - > h _ a d d r ) ,
h p - > h _ l e n g t h ) ;
/ / A s s o c i a t e a d d r e s s w i t h e n d p o i n t
c o n n e c t ( s t r e a m ,
( s t r u c t s o c k a d d r * ) & s a d d r ,
s i z e o f s a d d r ) ;
r e t u r n s t r e a m ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Limitations with Algorithmic Decomposition
Algorithmic decomposition tightly couples application-specific
functionality and the following configuration-related characteristics:
 Application Structure
– The number of services per process
– Time when services are configured into a process
 Communication and Demultiplexing Mechanisms
– The underlying IPC mechanisms that communicate with other
participating clients and servers
– Event demultiplexing and event handler dispatching mechanisms
 Concurrency and Synchronization Model
– The process and/or thread architecture that executes service(s) at
run-time
Vanderbilt University 36
Advanced ACE Tutorial Douglas C. Schmidt
Overcoming Limitations via OO
 The algorithmic decomposition illustrated above specifies many
low-level details
– Moreover, the excessive coupling impedes reusability,
extensibility, and portability...
 In contrast, OO focuses on application-specific behavior, e.g.,
int Logging_Handler::handle_input (void)
{
ssize_t n = handle_log_record (peer ().get_handle (),
ACE_STDOUT);
if (n > 0)
++request_count; // Count the # of logging records
return n <= 0 ? -1 : 0;
}
Vanderbilt University 37
Advanced ACE Tutorial Douglas C. Schmidt
OO Contributions to Software
 Patterns facilitate the large-scale reuse of software architecture
– Even when reuse of algorithms, detailed designs, and
implementations is not feasible
 Frameworks achieve large-scale design and code reuse
– In contrast, traditional techniques focus on the functions and
algorithms that solve particular requirements
 Note that patterns and frameworks are not unique to OO!
– However, objects and classes are useful abstraction mechanisms
Vanderbilt University 38
Advanced ACE Tutorial Douglas C. Schmidt
Patterns in the Networked Logging Server
IteratorAdapter TemplateMethod
Factory
Method
Wrapper
Facade
TACTICAL  PATTERNS
STRATEGIC
PATTERNS
Acceptor
Active
Object
Component
Configurator
Reactor
 Strategic and tactical are relative to the context and abstraction level
Vanderbilt University 39
Advanced ACE Tutorial Douglas C. Schmidt
Summary of Pattern Intents
 Wrapper Facade ! “Encapsulates the functions and data provided
by existing non-OO APIs within more concise, robust, portable,
maintainable, and cohesive OO class interfaces”
 Reactor ! “Demultiplexes and dispatches requests that are
delivered concurrently to an application by one or more clients”
 Acceptor ! “Decouple the passive connection and initialization of a
peer service in a distributed system from the processing performed
once the peer service is connected and initialized”
 Component Configurator ! “Decouples the implementation of
services from the time when they are configured”
 Active Object ! “Decouples method execution from method
invocation to enhance concurrency and simplify synchronized
access to an object that resides in its own thread of control”
Vanderbilt University 40
Advanced ACE Tutorial Douglas C. Schmidt
Components in the OO Logging Server
 Application-specific components
– Process logging records received from clients
 Connection-oriented application components
– ACE_Svc_Handler (service handler)
 Performs I/O-related tasks with clients
– ACE_Acceptor factory
 Passively accepts connection requests
 Dynamically creates a service handler for each client and
“activates” it
 Application-independent ACE framework components
– Perform IPC, explicit dynamic linking, event demultiplexing, event
handler dispatching, multi-threading, etc.
Vanderbilt University 41
A d v a n c e d A C E T u t o r i a l D o
C l a s s D i a g r a m f o r O O L o g g i n g S e r v e r
S e r v i c e
C o n f i g u r a t o r
S t r e a m
C o n n e c t i o n
L o g g i n g
A c c e p t o r
L o g g i n g _ H a n d l e r
S O C K _ A c c e p t o r
L o g g i n g
H a n d l e r
S O C K _ S t r e a m
N u l l _ S y n c h
S V C _ H A N D L E R
P E E R _ A C C E P T O R
P E E R _ S T R E A M
S Y N C H _ S T R A T
C
O
N
N
E
C
T
IO
N
-
O
R
IE
N
T
E
D
C
O
M
P
O
N
E
N
T
S
A
P
P
L
IC
A
T
IO
N
-
S
P
E
C
IF
IC
C
O
M
P
O
N
E
N
T
S
A
C
E
F
R
A
M
E
W
O
R
K
C
O
M
P
O
N
E
N
T
S
I P C _ S A P
R e a c t o r
< < a c t i v a t e s > >
1
n
C o n c u r r e n c y
P E E R
A C C E P T O R
P E E R
S T R E A M
S v c
H a n d l e r
A c c e p t o r
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Addressing Robustness, Portability,
and Maintainability Challenges
 Problem
– Building distributed applications using low-level APIs is hard
 Forces
– Low-level APIs are verbose, tedious, and error-prone to program
– Low-level APIs are non-portable and non-maintainable
 Solution
– Apply the Wrapper Facade pattern to encapsulate low-level
functions and data structures
Vanderbilt University 43
Advanced ACE Tutorial Douglas C. Schmidt
The Wrapper Facade Pattern
Intent
 Encapsulates the functions
and data provided by
existing lower-level,
non-OO APIs within more
concise, robust, portable,
maintainable, and cohesive
higher-level OO class
interfaces
1: method_k()
2: function_k()
client
Functions
function_1()
...
function_n()
Wrapper
Facade
method_1()
...
method_m()
POSA2 (www.cs.wustl.edu/
˜schmidt/POSA/)
Forces Resolved
 Avoid tedious, error-prone, and non-portable system APIs
 Create cohesive abstractions
Vanderbilt University 44
A d v a n c e d A C E T u t o r i a l D o
M o t i v a t i n g t h e W r a p p e r F a c a d e P a t t e r n :
t h e S o c k e t A P I
S E R V E R
C L I E N T
s o c k e t ( )
b i n d ( )
( o p t i o n a l )
c o n n e c t ( )
s e n d ( ) / r e c v ( )
s o c k e t ( )
b i n d ( )
l i s t e n ( )
a c c e p t ( )
s e n d ( ) / r e c v ( )
2 :  A C T I V E
R O L E
3 :  S E R V I C E
P R O C E S S I N G
c l o s e ( )
c l o s e ( )
N E T W O R K
1 :  P A S S I V E
R O L E
S o c k e t s a r e t h e m o s t c o m m o n n e t w o r k
p r o g r a m m i n g A P I a n d a r e a v a i l a b l e o n m o s t O S
p l a t f o r m s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Problem with Sockets: Lack of Type-safety
int buggy_echo_server (u_short port_num)
{ // Error checking omitted.
sockaddr_in s_addr;
int acceptor =
socket (PF_UNIX, SOCK_DGRAM, 0);
s_addr.sin_family = AF_INET;
s_addr.sin_port = port_num;
s_addr.sin_addr.s_addr = INADDR_ANY;
bind (acceptor, (sockaddr *) &s_addr,
sizeof s_addr);
int handle = accept (acceptor, 0, 0);
for (;;) {
char buf[BUFSIZ];
ssize_t n = read (acceptor, buf, sizeof buf);
if (n <= 0) break;
write (handle, buf, n);
}
}
 I/O handles are
not amenable to
strong type
checking at
compile-time
 The adjacent
code contains
many subtle,
common bugs
Vanderbilt University 46
Advanced ACE Tutorial Douglas C. Schmidt
Problem with Sockets: Steep Learning Curve
Many socket/TLI API functions have complex semantics, e.g.:
 Multiple protocol families and address families
– e.g., TCP, UNIX domain, OSI, XNS, etc.
 Infrequently used features, e.g.:
– Broadcasting/multicasting
– Passing open file handles
– Urgent data delivery and reception
– Asynch I/O, non-blocking I/O, I/O-based and timer-based event
multiplexing
Vanderbilt University 47
Advanced ACE Tutorial Douglas C. Schmidt
Problem with Sockets: Portability
 Having multiple “standards,” i.e., sockets and TLI, makes portability
difficult, e.g.,
– May require conditional compilation
– In addition, related functions are not included in POSIX standards
 e.g., select(), WaitForMultipleObjects(), and poll()
 Portability between UNIX and Windows Sockets is problematic, e.g.:
– Header files
– Error numbers
– Handle vs. descriptor types
– Shutdown semantics
– I/O controls and socket options
Vanderbilt University 48
Ad
vanced
ACE
Tutorial
Douglas
C
.S
ch
m
idt
Problem
w
ith
S
o
ckets:P
o
o
rly
Structured
s o c k e t ( )
b i n d ( )
c o n n e c t ( )
l i s t e n ( )
a c c e p t ( )
r e a d ( )
w r i t e ( )
r e a d v ( )
w r i t e v ( )
r e c v ( )
s e n d ( )
r e c v f r o m ( )
s e n d t o ( )
r e c v m s g ( )
s e n d m s g ( )
s e t s o c k o p t ( )
g e t s o c k o p t ( )
g e t p e e r n a m e ( )
g e t s o c k n a m e ( )
g e t h o s t b y n a m e ( )
g e t s e r v b y n a m e ( )
Lim
itations

SocketAPIis
linear
ratherthan
hierarchical

There
is
n
o
co
n
sistency
a
m
o
ng
n
a
m
e
s
.
.
.

N
on-portable
V
anderbiltUniversity
49
A d v a n c e d A C E T u t o r i a l D o
S o c k e t T a x o n o m y
XF
ER
CO
NN
EC
TIO
N/
CO
MM
UN
ICA
TIO
N
RO
LE
L O C A L L O C A L / R E M O T E
S
T
R
E
A
M
AC
TIV
E
PA
SS
IVE
D
A
T
A
G
R
A
M
C
O
N
N
E
C
T
E
D
D
A
T
A
G
R
A
M
T
Y
P
E
  O
F
  C
O
M
M
U
N
IC
A
T
IO
N
  S
E
R
V
IC
E
C O M M U N I C A T I O N   D O M A I N
s e n d t o ( ) / r e c v f r o m ( )
s o c k e t ( P F _ U N I X ) / b i n d ( )  s o c k e t ( P F _ I N E T ) / b i n d ( )  
s e n d ( ) / r e c v ( )
s o c k e t ( P F _ U N I X )
b i n d ( ) / c o n n e c t ( )  
s o c k e t ( P F _ U N I X )
b i n d ( ) / l i s t e n ( ) / a c c e p t ( )
s o c k e t ( P F _ U N I X )
b i n d ( ) / c o n n e c t ( )
s e n d ( ) / r e c v ( )
s o c k e t ( P F _ I N E T )
b i n d ( ) / l i s t e n ( ) / a c c e p t ( )
s o c k e t ( P F _ I N E T )
b i n d ( ) / c o n n e c t ( )
s e n d ( ) / r e c v ( )
s o c k e t ( P F _ U N I X ) / b i n d ( )  
s e n d ( ) / r e c v ( )
s e n d t o ( ) / r e c v f r o m ( )
s o c k e t ( P F _ I N E T ) / b i n d ( )  
s o c k e t ( P F _ I N E T )
b i n d ( ) / c o n n e c t ( )  
s o c k e t ( P F _ U N I X )
b i n d ( ) / c o n n e c t ( )  
s o c k e t ( P F _ I N E T )
b i n d ( ) / c o n n e c t ( )  
T h e S o c k e t A P I c a n b e c l a s s i fi e d a l o n g t h r e e
d i m e n s i o n s
1 . C o n n e c t i o n r o l e
2 . C o m m u n i c a t i o n d o m a i n
3 . T y p e o f s e r v i c e
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
S o l u t i o n : A C E S o c k e t W r a p p e r F a c a d e s
XF
ER
L O C A L L O C A L / R E M O T E
S
T
R
E
A
M
AC
TIV
E
PA
SS
IVE
D
A
T
A
G
R
A
M
C
O
N
N
E
C
T
E
D
D
A
T
A
G
R
A
M
C O M M U N I C A T I O N   D O M A I N
L S O C K _ C o n n e c t o r  
 S O C K _ C o n n e c t o r
S O C K _ A c c e p t o r
L S O C K _ A c c e p t o r
S O C K _ D g r a m _ M c a s t
 L S O C K _ S t r e a m S O C K _ S t r e a m
 L S O C K _ C O D g r a m
 S O C K _ C O D g r a m
 S O C K _ D g r a m
 L S O C K _ D g r a m
 SO
CK
_D
gra
m
 S O C K _ D g r a m _ B c a s t
T
Y
P
E
  O
F
  C
O
M
M
U
N
IC
A
T
IO
N
  S
E
R
V
IC
E
CO
NN
EC
TIO
N/
CO
MM
UN
ICA
TIO
N
RO
LE
 LS
OC
K_
Dg
ram
T h e A C E C + + w r a p p e r f a c a d e s m o r e e x p l i c i t l y
m o d e l t h e k e y s o c k e t c o m p o n e n t s u s i n g O O
c l a s s e s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h e A C E C o n n e c t i o n - O r i e n t e d
S o c k e t W r a p p e r F a c a d e s
A C E _ I P C _ S A P A C E _ A d d r
A C E _ S O C K _ I O A C E _ S O C K
A C E _ S O C K _ A c c e p t o r A C E _ I N E T _ A d d r
A C E _ S O C K _ S t r e a m
A C E _ S O C K _ C o n n e c t o r
P a r t i c i p a n t s
 P a s s i v e a n d a c t i v e c o n n e c t i o n f a c t o r i e s
– A C E _ S O C K _ A c c e p t o r a n d A C E _ S O C K _ C o n n e c t o r
 S t r e a m i n g c l a s s e s
– A C E _ S O C K _ S t r e a m a n d A C E _ S O C K _ I O
 A d d r e s s i n g c l a s s e s
– A C E _ A d d r a n d A C E _ I N E T _ A d d r
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The ACE Connection-Oriented Socket
Wrapper Facade Factories
class ACE_SOCK_Connector
{
public:
// Traits
typedef ACE_INET_Addr PEER_ADDR;
typedef ACE_SOCK_Stream PEER_STREAM;
int connect
(ACE_SOCK_Stream &new_sap,
const ACE_INET_Addr &raddr,
ACE_Time_Value *timeout,
const ACE_INET_Addr &laddr);
// ...
};
class ACE_SOCK_Acceptor
: public ACE_SOCK
{
public:
// Traits
typedef ACE_INET_Addr PEER_ADDR;
typedef ACE_SOCK_Stream PEER_STREAM;
ACE_SOCK_Acceptor (const ACE_INET_Addr &);
int open (const ACE_INET_Addr &addr);
int accept
(ACE_SOCK_Stream &new_sap,
ACE_INET_Addr *,
ACE_Time_Value *);
//...
};
Vanderbilt University 53
Advanced ACE Tutorial Douglas C. Schmidt
ACE Connection-Oriented Socket Wrapper Facade
Streaming and Addressing Classes
class ACE_SOCK_Stream
: public ACE_SOCK {
public:
// Trait.
typedef ACE_INET_Addr PEER_ADDR;
ssize_t send (const void *buf,
int n);
ssize_t recv (void *buf,
int n);
ssize_t send_n (const void *buf,
int n);
ssize_t sendv_n (const iovec *iov,
int n);
ssize_t recv_n (void *buf, int n);
int close (void);
// ...
};
class ACE_INET_Addr
: public ACE_Addr
{
public:
ACE_INET_Addr (u_short port,
const char host[]);
u_short get_port_number (void);
ACE_UINT_32 get_ip_addr (void);
// ...
};
Vanderbilt University 54
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Motivating the
Socket Wrapper Facade Structure
 Q: Why decouple the ACE_SOCK_Acceptor and the
ACE_SOCK_Connector from ACE_SOCK_Stream?
 A: For the same reasons that ACE_Acceptor and
ACE_Connector are decoupled from ACE_Svc_Handler, e.g.,
– An ACE_SOCK_Stream is only responsible for data transfer
 Regardless of whether the connection is established passively
or actively
– This ensures that the ACE_SOCK* components aren’t used
incorrectly...
 e.g., you can’t accidentally read() or write() on
ACE_SOCK_Connectors or ACE_SOCK_Acceptors, etc.
Vanderbilt University 55
A d v a n c e d A C E T u t o r i a l D o
A n E c h o S e r v e r W r i t t e n u s i n g
A C E C + + S o c k e t W r a p p e r F a c a d e s
i n t e c h o _ s e r v e r ( u _ s h o r t p o r t _ n u m )
{
/ / L o c a l s e r v e r a d d r e s s .
A C E _ I N E T _ A d d r m y _ a d d r ( p o r t _ n u m ) ;
/ / I n i t i a l i z e t h e a c c e p t o r m o d e s e r v e r .
A C E _ S O C K _ A c c e p t o r a c c e p t o r ( m y _ a d d r ) ;
/ / D a t a t r a n s f e r o b j e c t .
A C E _ S O C K _ S t r e a m n e w _ s t r e a m ;
/ / A c c e p t a n e w c o n n e c t i o n .
a c c e p t o r . a c c e p t ( n e w _ s t r e a m ) ;
f o r ( ; ; ) {
c h a r b u f [ B U F S I Z ] ;
/ / E r r o r c a u g h t a t c o m p i l e t i m e !
s s i z e _ t n =
a c c e p t o r . r e c v ( b u f , s i z e o f b u f ) ;
n e w _ s t r e a m . s e n d _ n ( b u f , n ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A G e n e r i c V e r s i o n o f t h e E c h o S e r v e r
t e m p l a t e < c l a s s A C C E P T O R >
i n t e c h o _ s e r v e r ( u _ s h o r t p o r t )
{
/ / L o c a l s e r v e r a d d r e s s ( n o t e t r a i t s ) .
t y p e n a m e
A C C E P T O R : : P E E R _ A D D R m y _ a d d r ( p o r t ) ;
/ / I n i t i a l i z e t h e a c c e p t o r m o d e s e r v e r .
A C C E P T O R a c c e p t o r ( m y _ a d d r ) ;
/ / D a t a t r a n s f e r o b j e c t ( n o t e t r a i t s ) .
t y p e n a m e A C C E P T O R : : P E E R _ S T R E A M s t r e a m ;
/ / A c c e p t a n e w c o n n e c t i o n .
a c c e p t o r . a c c e p t ( s t r e a m ) ;
f o r ( ; ; ) {
c h a r b u f [ B U F S I Z ] ;
s s i z e _ t n =
s t r e a m . r e c v ( b u f , s i z e o f b u f ) ;
s t r e a m . s e n d _ n ( b u f , n ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Scope of the ACE IPC Wrapper Facades
ACE
IPC
SAP
A
SOCKET
API
SOCK
SAP
TLI
API
TLI
SAP
STREAM
PIPE API
SPIPE
SAP
NAMED
PIPE API
FIFO
SAP
SSL
SAP
SSL
API
MMAP
 API
MEM
SAP
SYSTEM V
IPC API
SysV
IPC
C++NPv1 (www.cs.wustl.edu/˜schmidt/ACE/book1/)
Vanderbilt University 58
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e W r a p p e r F a c a d e P a t t e r n
f o r t h e L o g g i n g S e r v e r
N o t e w e h a v e n ’ t i m p r o v e d t h e o v e r a l l d e s i g n ( y e t )
/ / . . . S a m e a s b e f o r e . . .
/ / A c c e p t o r - m o d e s o c k e t h a n d l e .
s t a t i c A C E _ S O C K _ A c c e p t o r a c c e p t o r ;
/ / S e t o f c u r r e n t l y a c t i v e h a n d l e s
s t a t i c A C E _ H a n d l e _ S e t a c t i v i t y _ h a n d l e s ;
/ / S c r a t c h c o p y o f a c t i v i t y _ h a n d l e s
s t a t i c A C E _ H a n d l e _ S e t r e a d y _ h a n d l e s ;
s t a t i c v o i d i n i t i a l i z e _ a c c e p t o r ( u _ s h o r t p o r t )
{
/ / S e t u p a d d r e s s i n f o . t o b e c o m e s e r v e r .
A C E _ I N E T _ A d d r s a d d r ( p o r t ) ;
/ / C r e a t e a l o c a l e n d p o i n t o f c o m m u n i c a t i o n .
a c c e p t o r . o p e n ( s a d d r ) ;
/ / S e t t h e < S O C K _ A c c e p t o r > i n t o n o n - b l o c k i n g m o d e .
a c c e p t o r . e n a b l e ( A C E _ N O N B L O C K ) ;
a c t i v i t y _ h a n d l e s . s e t _ b i t ( a c c e p t o r . g e t _ h a n d l e ( ) ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
M a i n E v e n t L o o p o f L o g g i n g S e r v e r
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] )
{
i n i t i a l i z e _ a c c e p t o r
( a r g c > 1 ? a t o i ( a r g v [ 1 ] ) : P O R T ) ;
/ / L o o p f o r e v e r p e r f o r m i n g l o g g i n g
/ / s e r v e r p r o c e s s i n g .
f o r ( ; ; ) {
/ / o b j e c t a s s i g n m e n t .
r e a d y _ h a n d l e s = a c t i v i t y _ h a n d l e s ;
/ / W a i t f o r c l i e n t I / O e v e n t s .
A C E : : s e l e c t ( i n t ( m a x h p 1 ) ,
/ / c a l l s o p e r a t o r f d _ s e t * ( ) .
r e a d y _ h a n d l e s ) ;
/ / F i r s t r e c e i v e p e n d i n g l o g g i n g r e c o r d s .
h a n d l e _ d a t a ( ) ;
/ / T h e n a c c e p t p e n d i n g c o n n e c t i o n s .
h a n d l e _ c o n n e c t i o n s ( ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
H a n d l i n g C o n n e c t i o n s a n d
D a t a P r o c e s s i n g
s t a t i c v o i d h a n d l e _ c o n n e c t i o n s ( v o i d ) {
i f ( r e a d y _ h a n d l e s . i s _ s e t ( a c c e p t o r . g e t _ h a n d l e ( ) ) )
A C E _ S O C K _ S t r e a m s t r ;
/ / H a n d l e a l l p e n d i n g c o n n e c t i o n r e q u e s t s .
w h i l e ( a c c e p t o r . a c c e p t ( s t r ) ! = - 1 )
a c t i v i t y _ h a n d l e s . s e t _ b i t ( s t r . g e t _ h a n d l e ( ) ) ;
}
}
s t a t i c v o i d h a n d l e _ d a t a ( v o i d ) {
A C E _ H A N D L E h ;
A C E _ H a n d l e _ S e t _ I t e r a t o r i t e r ( r e a d y _ h a n d l e s ) ;
w h i l e ( ( h = i t e r ( ) ) ! = A C E _ I N V A L I D _ H A N D L E ) {
A C E _ S O C K _ S t r e a m s t r ( h ) ;
s s i z e _ t n = h a n d l e _ l o g _ r e c o r d ( s t r , A C E _ S T D O U T ) ;
i f ( n > 0 ) / / C o u n t # o f l o g g i n g r e c o r d s .
+ + r e q u e s t _ c o u n t ;
e l s e i f ( n = = 0 ) {
/ / H a n d l e c o n n e c t i o n s h u t d o w n .
a c t i v i t y _ h a n d l e s . c l r _ b i t ( h ) ;
s . c l o s e ( ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
R e c e i v e a n d P r o c e s s L o g g i n g R e c o r d s
s t a t i c s s i z e _ t h a n d l e _ l o g _ r e c o r d ( A C E _ S O C K _ S t r e a m s ,
A C E _ H A N D L E o u t _ h )
A C E _ U I N T _ 3 2 l e n ;
A C E _ L o g _ R e c o r d l r ;
/ / T h e f i r s t r e c v r e a d s t h e l e n g t h ( s t o r e d a s a
/ / f i x e d - s i z e i n t e g e r ) o f a d j a c e n t l o g g i n g r e c o r d .
s s i z e _ t n = s . r e c v _ n ( ( c h a r * ) & l e n , s i z e o f l e n ) ;
i f ( n < = 0 ) r e t u r n n ;
l e n = n t o h l ( l e n ) ; / / C o n v e r t b y t e - o r d e r i n g
/ / P e r f o r m s a n i t y c h e c k !
i f ( l e n > s i z e o f ( l r ) ) r e t u r n - 1 ;
/ / T h e s e c o n d r e c v t h e n r e a d s < l e n > b y t e s t o
/ / o b t a i n t h e a c t u a l r e c o r d .
s . r e c v _ n ( ( c h a r * ) & l r , s i z e o f l r ) ;
/ / D e c o d e a n d p r i n t r e c o r d .
d e c o d e _ l o g _ r e c o r d ( & l r ) ;
i f ( A C E _ O S : : w r i t e ( o u t _ h , l r . b u f , l r . s i z e ) = = - 1 )
r e t u r n - 1 ;
e l s e r e t u r n 0 ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
O O C l i e n t L o g g i n g
D a e m o n I m p l e m e n t a t i o n
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] )
{
A C E _ S O C K _ S t r e a m s t r e a m ;
A C E _ S O C K _ C o n n e c t o r c o n ; / / E s t a b l i s h c o n n e c t i o n .
c o n . c o n n e c t ( s t r e a m , A C E _ I N E T _ A d d r ( a r g c > 1
? a t o i ( a r g v [ 1 ] ) : P O R T ) ) ;
A C E _ L o g _ R e c o r d l r ;
/ / L o o p f o r e v e r p e r f o r m i n g c l i e n t
/ / l o g g i n g d a e m o n p r o c e s s i n g .
f o r ( ; ; ) {
/ / . . . g e t l o g g i n g r e c o r d s f r o m c l i e n t
/ / a p p l i c a t i o n p r o c e s s e s . . .
A C E _ U I N T _ 3 2 s i z e = l r . s i z e ;
l r . s i z e = h t o n l ( l r . s i z e ) ;
e n c o d e _ l o g _ r e c o r d ( & l r ) ;
i o v e c i o v [ 2 ] ;
i o v [ 0 ] . i o v _ l e n = s i z e o f ( A C E _ U I N T _ 3 2 ) ;
i o v [ 0 ] . i o v _ b a s e = & l r . s i z e ;
i o v [ 1 ] . i o v _ l e n = s i z e ;
i o v [ 1 ] . i o v _ b a s e = & l r ;
/ / U s e s w r i t e v ( 2 ) ;
s t r e a m . s e n d v _ n ( i o v , 2 ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Evaluating the Wrapper Facade Solution
Benefits
 More concise
 More robust
 More portable
 More maintainable
 More efficient
Liabilities
 Potentially more indirection
 Additional learning curve
 Still haven’t solved the overall
design problem
– i.e., the overall design is
still based on step-wise
refinement of functions
Vanderbilt University 64
Advanced ACE Tutorial Douglas C. Schmidt
ACE C++ Wrapper Facade
Design Refactoring Principles
 Enforce typesafety at compile-time
 Allow controlled violations of typesafety
 Simplify for the common case
 Replace one-dimensional interfaces with hierarchical class
categories
 Enhance portability with parameterized types
 Inline performance critical methods
 Define auxiliary classes to hide error-prone details
Vanderbilt University 65
Advanced ACE Tutorial Douglas C. Schmidt
Enforce Typesafety at Compile-Time
Sockets cannot detect certain errors at compile-time, e.g.,
int acceptor = socket (PF_INET, SOCK_STREAM, 0);
// ...
bind (acceptor, ...); // Bind address.
listen (acceptor); // Make a acceptor-mode socket.
HANDLE n_sd = accept (acceptor, 0, 0);
// Error not detected until run-time.
read (acceptor, buf, sizeof buf);
ACE enforces type-safety at compile-time via factories, e.g.:
ACE_SOCK_Acceptor acceptor (port);
// Error: recv() not a method of .
acceptor.recv (buf, sizeof buf);
Vanderbilt University 66
Advanced ACE Tutorial Douglas C. Schmidt
Allow Controlled Violations of Typesafety
Make it easy to use the C++ Socket wrapper facades correctly, hard to
use it incorrectly, but not impossible to use it in ways the class
designers did not anticipate
 e.g., it may be necessary to retrieve the underlying socket handle:
ACE_SOCK_Acceptor acceptor;
// ...
ACE_Handle_Set ready_handles;
// ...
if (ready_handles.is_set (acceptor.get_handle ())
ACE::select (acceptor.get_handle () + 1, ready_handles);
Vanderbilt University 67
Advanced ACE Tutorial Douglas C. Schmidt
Supply Default Parameters
ACE_SOCK_Connector (ACE_SOCK_Stream &new_stream,
const ACE_Addr &remote_sap,
ACE_Time_Value *timeout = 0,
const ACE_Addr &local_sap = ACE_Addr::sap_any,
int protocol_family = PF_INET,
int protocol = 0);
The result is extremely concise for the common case:
ACE_SOCK_Stream stream;
// Compiler supplies default values.
ACE_SOCK_Connector con (stream, ACE_INET_Addr (port, host));
Vanderbilt University 68
A d v a n c e d A C E T u t o r i a l D o
D e fi n e P a r s i m o n i o u s I n t e r f a c e s
e.g. , u s e L S O C K t o p a s s s o c k e t h a n d l e s :
A C E _ L S O C K _ S t r e a m s t r e a m ;
A C E _ L S O C K _ A c c e p t o r a c c e p t o r ( " / t m p / f o o " ) ;
a c c e p t o r . a c c e p t ( s t r e a m ) ;
s t r e a m . s e n d _ h a n d l e ( s t r e a m . g e t _ h a n d l e ( ) ) ;
v e r s u s t h e l e s s p a r s i m o n i o u s B S D 4 . 3 s o c k e t c o d e
A C E _ L S O C K : : s e n d _ h a n d l e
( c o n s t A C E _ H A N D L E s d ) c o n s t {
u _ c h a r a [ 2 ] ; i o v e c i o v ; m s g h d r s e n d _ m s g ;
a [ 0 ] = 0 x a b , a [ 1 ] = 0 x c d ;
i o v . i o v _ b a s e = ( c h a r * ) a ;
i o v . i o v _ l e n = s i z e o f a ;
s e n d _ m s g . m s g _ i o v = & i o v ;
s e n d _ m s g . m s g _ i o v l e n = 1 ;
s e n d _ m s g . m s g _ n a m e = ( c h a r * ) 0 ;
s e n d _ m s g . m s g _ n a m e l e n = 0 ;
s e n d _ m s g . m s g _ a c c r i g h t s = ( c h a r * ) & s d ;
s e n d _ m s g . m s g _ a c c r i g h t s l e n = s i z e o f s d ;
r e t u r n s e n d m s g ( t h i s - > g e t _ h a n d l e ( ) ,
& s e n d _ m s g , 0 ) ;
N o t e t h a t S V R 4 a n d B S D 4 . 4 A P I s a r e d i f f e r e n t
t h a n B S D 4 . 3 !
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Combine Multiple Operations into One Operation
Creating a conventional acceptor-mode socket requires multiple calls:
int acceptor = socket (PF_INET, SOCK_STREAM, 0);
sockaddr_in addr;
memset (&addr, 0, sizeof addr);
addr.sin_family = AF_INET;
addr.sin_port = htons (port);
addr.sin_addr.s_addr = INADDR_ANY;
bind (acceptor, &addr, addr_len);
listen (acceptor);
// ...
ACE_SOCK_Acceptor combines this into a single operation:
ACE_SOCK_Acceptor acceptor ((ACE_INET_Addr) port);
Vanderbilt University 70
Advanced ACE Tutorial Douglas C. Schmidt
Create Hierarchical Class Categories
ACE
LSOCK
ACE
IPC
SAP
GROUP
COMM
DATAGRAM
COMM
STREAM
COMM
CONNECTION
ESTABLISHMENT
ACE
SOCK
ACE
SOCK
CODgram
 ACE
SOCK
Dgram
ACE
SOCK
Dgram
Bcast
ACE
SOCK
Acceptor
ACE
SOCK
Connector
ACE
SOCK
Stream
ACE
LSOCK
CODgram
ACE
LSOCK
Dgram
ACE
LSOCK
Connector
ACE
LSOCK
Acceptor
ACE
LSOCK
Stream
ACE
SOCK
Dgram
Mcast
Vanderbilt University 71
Advanced ACE Tutorial Douglas C. Schmidt
Enhance Portability with Parameterized Types
OS  KERNEL
PROTOCOL  MECHANISMS
(TCP/IP,  OSI,  ETC.)
USER
SPACE
DISTRIBUTED
APPLICATION 1
APPLICATION1 DISTRIBUTED
APPLICATION 3
APPLICATION3DISTRIBUTED
APPLICATION 2
APPLICATION2
KERNEL
SPACE
BSD SOCKET
API
COMMON  INTERFACE
(PARAMETERIZED  TYPES)
OCKET
A
SOCK_SAP
BSD SOCKET
API
SYSTEM  V
TLI   API
TLI_SAP
NETWORK
INTERFACE
// Conditionally select IPC mechanism.
#if defined (USE_SOCKETS)
typedef ACE_SOCK_Acceptor PEER_ACCEPTOR;
#elif defined (USE_TLI)
typedef ACE_TLI_Acceptor PEER_ACCEPTOR;
#endif // USE_SOCKETS.
int main (void)
{
// ...
// Invoke with appropriate
// network programming interface.
echo_server (port);
}
Switching wholesale between sockets and TLI simply requires
instantiating a different ACE C++ wrapper facade
Vanderbilt University 72
Advanced ACE Tutorial Douglas C. Schmidt
Inline Performance Critical Methods
Inlining is time and space efficient since key methods are very short:
class ACE_SOCK_Stream : public ACE_SOCK
{
public:
ssize_t send (const void *buf, size_t n)
{
return ACE_OS::send (this->get_handle (), buf, n);
}
ssize_t recv (void *buf, size_t n)
{
return ACE_OS::recv (this->get_handle (), buf, n);
}
};
Vanderbilt University 73
Advanced ACE Tutorial Douglas C. Schmidt
Define Auxiliary Classes to Hide Error-Prone Details
Standard C socket addressing is awkward and error-prone
 e.g., easy to neglect to zero-out a sockaddr_in or convert port
numbers to network byte-order, etc.
ACE C++ Socket wrapper facades define classes to handle details
class ACE_INET_Addr : public ACE_Addr { public:
ACE_INET_Addr (u_short port, long ip_addr = 0) {
memset (&this->inet_addr_, 0, sizeof this->inet_addr_);
this->inet_addr_.sin_family = AF_INET;
this->inet_addr_.sin_port = htons (port);
memcpy (&this->inet_addr_.sin_addr, &ip_addr, sizeof ip_addr);
}
// ...
private:
sockaddr_in inet_addr_;
};
Vanderbilt University 74
Advanced ACE Tutorial Douglas C. Schmidt
Demultiplexing and Dispatching Events
 Problem
– The logging server must process several different types of events
simultaneously from different sources of events
 Forces
– Multi-threading is not always available
– Multi-threading is not always efficient
– Multi-threading can be error-prone
– Tightly coupling event demuxing with server-specific logic is
inflexible
 Solution
– Use the Reactor pattern to decouple event demuxing/dispatching
from server-specific processing
Vanderbilt University 75
Advanced ACE Tutorial Douglas C. Schmidt
The Reactor Pattern
Reactor 
handle_events()
register_handler(h)
remove_handler(h)
select (handles);
foreach h in handles loop
     table[h].handle_event(type)
end loop
Event Handler
handle_event(type)
get_handle()
handlers
Handle ownsuses
notifies
Concrete
Event
Handler
Synchronous Event
Demultiplexer
select()
1 N
www.cs.wustl.edu/˜schmidt/
POSA/
Intent
 Demuxes & dispatches
requests that are
delievered concurrency
to an application by one
or more clients
Forces Resolved
 Serially demux events
synchronously &
efficiently
 Extend applications
without changing
demuxing code
Vanderbilt University 76
Advanced ACE Tutorial Douglas C. Schmidt
Collaboration in the Reactor Pattern
main
program
INITIALIZE
REGISTER  HANDLER
callback :
Concrete
Event_Handler
START  EVENT  LOOP
DATA  ARRIVES
OK TO  SEND
Reactor
handle_events()
FOREACH  EVENT  DO
handle_input()
select()
Reactor()
register_handler(callback)
handle_output()
SIGNAL  ARRIVES
TIMER  EXPIRES
handle_signal()
handle_timeout()
get_handle()
EXTRACT  HANDLE
REMOVE  HANDLER
remove_handler(callback)
I
N
I
T
I
A
L
I
Z
A
T
I
O
N
M
O
D
E
E
V
E
N
T
 
 
H
A
N
D
L
I
N
G
M
O
D
E
handle_close()
CLEANUP 
 Note inversion of
control
 Also note how
long-running
event handlers
can degrade
quality of service
since callbacks
“steal” Reactor’s
thread of
control...
Vanderbilt University 77
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e a n d I m p l e m e n t a t i o n s
o f t h e A C E R e a c t o r F r a m e w o r k
R e a c t o r f r a m e w o r k p a r t i c i p a n t s
A C E _ R e a c t o r
ACE_Event_HandlerACE_Timer_Queue
A C E _ T i m e _ V a l u e
A p p l i c a t i o n  E v e n t
H a n d l e r
0 . . 1
C o m m o n R e a c t o r i m p l e m e n t a t i o n s i n A C E
ACE_Reactor_Impl
ACE_Select_Reactor_Impl
A C E _ T P _ R e a c t o r
A C E _ W F M O _ R e a c t o r
A C E _ S e l e c t _ R e a c t o r _ T
T O K E N
A C E _ R e a c t o r
1
A C E _ S e l e c t _ R e a c t o r
« b i n d »
< A C E _ S e l e c t _ R e a c t o r _ T o k e n >
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E R e a c t o r F r a m e w o r k
i n t h e L o g g i n g S e r v e r
R E G I S T E R E D
O B J E C T S
F
R
A
M
E
W
O
R
K
L
E
V
E
L
K
E
R
N
E
L
L
E
V
E
L
A
P
P
L
IC
A
T
IO
N
L
E
V
E
L
1 :  h a n d l e _ i n p u t ( )
5 :  h a n d l e _ i n p u t ( )
6 :  r e c v ( m s g )
7 : p r o c e s s ( m s g )
2 :  s h  =  n e w  L o g g i n g _ H a n d l e r
3 :  a c c e p t  ( s h - > p e e r ( ) )
4 :  s h - > o p e n ( )
O S   E V E N T   D E M U L T I P L E X I N G   I N T E R F A C E
:  R e a c t o r
: T i m e r
Q u e u e
:  H a n d l e
T a b l e
:  E v e n t
H a n d l e r
:  L o g g i n g
H a n d l e r
:  E v e n t
H a n d l e r
:  L o g g i n g
H a n d l e r
:  E v e n t
H a n d l e r
:  L o g g i n g
A c c e p t o r
B e n e fi t s
 S t r a i g h t f o r w a r d t o
p r o g r a m
 C o n c u r r e n c y c o n t r o l
i s e a s y
L i a b i l i t i e s
 C a l l b a c k s a r e “ b r i t t l e ”
 C a n ’ t l e v e r a g e
m u l t i - p r o c e s s o r s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Addressing Acceptor Endpoint Connection
and Initialization Challenges
 Problem
– The communication protocol used between applications is often
orthogonal to its connection establishment and service handler
initialization protocols
 Forces
– Low-level connection APIs are error-prone and non-portable
– Separating initialization from processing increases software reuse
 Solution
– Use the Acceptor pattern to decouple passive connection
establishment and connection handler initialization from the
subsequent logging protocol
Vanderbilt University 80
A d v a n c e d A C E T u t o r i a l D o
T h e A c c e p t o r - C o n n e c t o r P a t t e r n
( A c c e p t o r R o l e )
S v c
H a n d l e r
p e e r _ s t r e a m _
o p e n ( )
A c c e p t o r
p e e r _ a c c e p t o r _
a c c e p t ( )
S v c  H a n d l e r
R e a c t o r
A P P L I C A T I O N -
D E F I N E D
A P P L I C A T I O N -
I N D E P E N D E N T
A C T I V A T E S
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P O S A /
I n t e n t o f A c c e p t o r R o l e
 Decouple the passive
connection and
initialization of a peer
service in a distributed
system from the
processing performed
once the peer service is
connected and
initialized
F o r c e s r e s o l v e d
 R e u s e p a s s i v e
c o n n e c t i o n s e t u p
a n d s e r v i c e
i n i t i a l i z a t i o n c o d e
 E n s u r e t h a t
a c c e p t o r - m o d e
h a n d l e s a r e n ’ t u s e d
t o r e a d / w r i t e d a t a
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e o f t h e A C E
A c c e p t o r - C o n n e c t o r F r a m e w o r k
ACE_Svc_Handler
P E E R _ S T R E A M ,
S Y N C H _ S T R A T E G Y
A C E _ A c c e p t o r
S V C _ H A N D L E R ,
P E E R _ A C C E P T O R
A C E _ C o n n e c t o r
S V C _ H A N D L E R ,
P E E R _ C O N N E C T O R
A p p l i c a t i o n
S e r v i c e
« b i n d »
ACE_Event_Handler
ACE_Task
S Y N C H _ S T R A T E G Y
F r a m e w o r k c h a r a c t e r i s t i c s
 U s e s C + + p a r a m e t e r i z e d t y p e s t o strategize I P C
a n d s e r v i c e aspects
 U s e s T e m p l a t e M e t h o d p a t t e r n t o s t r a t e g i z e
c r e a t i o n , c o n n e c t i o n e s t a b l i s h m e n t , a n d
c o n c u r r e n c y p o l i c i e s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E _ A c c e p t o r
i n t h e L o g g i n g S e r v e r
P A S S I V E
L I S T E N E R
A C T I V E
C O N N E C T I O N S
1 :  h a n d l e _ i n p u t ( )
2 :  s h  =  m a k e _ s v c _ h a n d l e r ( )
3 :  a c c e p t _ s v c _ h a n d l e r ( s h )
4 :  a c t i v a t e _ s v c _ h a n d l e r ( s h )
:  R e a c t o r
:  A c c e p t o r
:  L o g g i n g
A c c e p t o r
:  S v c
H a n d l e r
:  L o g g i n g
H a n d l e r
:  S v c
H a n d l e r
:  L o g g i n g
H a n d l e r
:  S v c
H a n d l e r
:  L o g g i n g
H a n d l e r
:  S v c
H a n d l e r
:  L o g g i n g
H a n d l e r
 T h e A C E _ A c c e p t o r i s a factory
– i.e., i t creates, connects, a n d activates a n
A C E _ S v c _ H a n d l e r
 T h e r e ’ s o f t e n o n e A C E _ A c c e p t o r
p e r - s e r v i c e / p e r - p o r t
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ A c c e p t o r C l a s s P u b l i c I n t e r f a c e
t e m p l a t e < c l a s s S V C _ H A N D L E R , / / S e r v i c e a s p e c t
c l a s s P E E R _ A C C E P T O R > / / I P C a s p e c t
c l a s s A C E _ A c c e p t o r : p u b l i c A C E _ S e r v i c e _ O b j e c t
{
/ / I n h e r i t s i n d i r e c t l y f r o m < A C E _ E v e n t _ H a n d l e r >
p u b l i c :
/ / I n i t i a l i z a t i o n .
v i r t u a l i n t o p e n
( t y p e n a m e c o n s t P E E R _ A C C E P T O R : : P E E R _ A D D R & ,
A C E _ R e a c t o r * = A C E _ R e a c t o r : : i n s t a n c e ( ) ) ;
/ / T e m p l a t e M e t h o d .
v i r t u a l i n t h a n d l e _ i n p u t ( A C E _ H A N D L E ) ;
p r o t e c t e d :
/ / F a c t o r y m e t h o d c r e a t e s a s e r v i c e h a n d l e r .
v i r t u a l S V C _ H A N D L E R * m a k e _ s v c _ h a n d l e r ( v o i d ) ;
/ / A c c e p t a n e w c o n n e c t i o n .
v i r t u a l i n t a c c e p t _ s v c _ h a n d l e r ( S V C _ H A N D L E R * ) ;
/ / A c t i v a t e a s e r v i c e h a n d l e r .
v i r t u a l i n t a c t i v a t e _ s v c _ h a n d l e r ( S V C _ H A N D L E R * ) ;
p r i v a t e :
/ / A c c e p t o r I P C c o n n e c t i o n s t r a t e g y .
P E E R _ A C C E P T O R p e e r _ a c c e p t o r _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ A c c e p t o r C l a s s I m p l e m e n t a t i o n
/ / S h o r t h a n d n a m e s .
# d e f i n e S H S V C _ H A N D L E R
# d e f i n e P A P E E R _ A C C E P T O R
/ / T e m p l a t e M e t h o d t h a t c r e a t e s , c o n n e c t s ,
/ / a n d a c t i v a t e s s e r v i c e h a n d l e r s .
t e m p l a t e < c l a s s S H , c l a s s P A > i n t
A C E _ A c c e p t o r < S H , P A > : : h a n d l e _ i n p u t ( A C E _ H A N D L E )
{
/ / F a c t o r y m e t h o d t h a t m a k e s a s e r v i c e h a n d l e r .
S H * s v c _ h a n d l e r = m a k e _ s v c _ h a n d l e r ( ) ;
/ / A c c e p t t h e c o n n e c t i o n .
a c c e p t _ s v c _ h a n d l e r ( s v c _ h a n d l e r ) ;
/ / D e l e g a t e c o n t r o l t o t h e s e r v i c e h a n d l e r .
a c t i v a t e _ s v c _ h a n d l e r ( s v c _ h a n d l e r ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The Template Method Pattern
Concrete
Class
primitive_operation1()
primitive_operation2()
...
primitive_operation1()
...
primitive_operation2()
...
Abstract
Class
template_method()
primitive_operation1()
primitive_operation2()
Intent
 Define the skeleton of
an algorithm in an
operation, deferring
some steps to
subclasses
Gamma et al., Design
Patterns: Elements of
Reusable Object-Oriented
Software AW, ’94
Vanderbilt University 86
Advanced ACE Tutorial Douglas C. Schmidt
Using the Template Method Pattern in
the ACE Acceptor Implementation
Acceptor
handle_input()
make_svc_handler()
accept_svc_handler()
activate_svc_handler()
...
make_svc_handler()
...
accept_svc_handler()
...
activate_svc_handler()
My
Acceptor
make_svc_handler()
activate_svc_handler()
Benefits
 Straightforward to
program via inheritance
and dynamic binding
Liabilities
 Design is “brittle” and
can cause “explosion” of
subclasses due to
“whitebox” design
Vanderbilt University 87
Advanced ACE Tutorial Douglas C. Schmidt
The Strategy Pattern
Strategy
algorithm_interface()
Concrete
Strategy A
algorithm_interface()
STRATEGY
Concrete
Strategy B
algorithm_interface()
Concrete
Strategy C
algorithm_interface()
Context
context_interface()
Intent
 Define a family of
algorithms, encapsulate
each one, and make
them interchangeable
Gamma et al., Design
Patterns: Elements of
Reusable Object-Oriented
Software AW, ’94
Vanderbilt University 88
Advanced ACE Tutorial Douglas C. Schmidt
Using the Strategy Pattern in
the ACE Acceptor Implementation
Thread
Strategy
Process
Strategy
Reactive
Strategy
Acceptor
handle_input()
sh = create_svc_handler ()
...
accept_svc_handler (sh)
...
1: activate_svc_handler(sh)
...
2: activate_svc_handler(sh)
<>
activate_svc_handler()
Concurrency
Strategy
Benefits
 More
extensible due
to “blackbox”
design
Liabilities
 More complex
and harder to
develop initially
Vanderbilt University 89
A d v a n c e d A C E T u t o r i a l D o
A C E _ A c c e p t o r T e m p l a t e M e t h o d
H o o k I m p l e m e n t a t i o n s
T e m p l a t e m e t h o d h o o k s c a n b e o v e r r i d d e n
/ / F a c t o r y m e t h o d f o r c r e a t i n g a s e r v i c e h a n d l e r .
t e m p l a t e < c l a s s S H , c l a s s P A > S H *
A C E _ A c c e p t o r < S H , P A > : : m a k e _ s v c _ h a n d l e r ( A C E _ H A N D L E )
r e t u r n n e w S H ; / / D e f a u l t b e h a v i o r .
}
/ / A c c e p t c o n n e c t i o n s f r o m c l i e n t s .
t e m p l a t e < c l a s s S H , c l a s s P A > i n t
A C E _ A c c e p t o r < S H , P A > : : a c c e p t _ s v c _ h a n d l e r ( S H * s h )
{
p e e r _ a c c e p t o r _ . a c c e p t ( s h - > p e e r ( ) ) ;
}
/ / A c t i v a t e t h e s e r v i c e h a n d l e r .
t e m p l a t e < c l a s s S H , c l a s s P A > i n t
A C E _ A c c e p t o r < S H , P A > : : a c t i v a t e _ s v c _ h a n d l e r ( S H * s h )
{
i f ( s h - > o p e n ( ) = = - 1 )
s h - > c l o s e ( ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ A c c e p t o r I n i t i a l i z a t i o n
I m p l e m e n t a t i o n
N o t e h o w t h e P E E R _ A C C E P T O R ’ s o p e n ( ) m e t h o d
h i d e s a l l t h e d e t a i l s a s s o c i a t e d w i t h p a s s i v e l y
i n i t i a l i z i n g c o m m u n i c a t i o n e n d p o i n t s
/ / I n i t i a l i z a t i o n .
t e m p l a t e < c l a s s S H , c l a s s P A > i n t
A C E _ A c c e p t o r < S H , P A > : : o p e n
( t y p e n a m e c o n s t P A : : P E E R _ A D D R & a d d r ,
A C E _ R e a c t o r * r e a c t o r )
{
/ / F o r w a r d i n i t i a l i z a t i o n t o t h e c o n c r e t e
/ / p e e r a c c e p t o r .
p e e r _ a c c e p t o r _ . o p e n ( a d d r ) ;
/ / R e g i s t e r w i t h R e a c t o r .
r e a c t o r - > r e g i s t e r _ h a n d l e r
( t h i s , A C E _ E v e n t _ H a n d l e r : : A C C E P T _ M A S K ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ S v c _ H a n d l e r C l a s s P u b l i c
I n t e r f a c e
N o t e h o w I P C a n d s y n c h r o n i z a t i o n aspects a r e
s t r a t e g i z e d
t e m p l a t e < c l a s s P E E R _ S T R E A M , / / I P C a s p e c t
c l a s s S Y N C H _ S T R A T > / / S y n c h a s p e c t
c l a s s A C E _ S v c _ H a n d l e r
: p u b l i c A C E _ T a s k < S Y N C H _ S T R A T >
/ / T a s k i s - a S e r v i c e _ O b j e c t ,
/ / w h i c h i s - a n E v e n t _ H a n d l e r
{
p u b l i c :
/ / C o n s t r u c t o r .
A C E _ S v c _ H a n d l e r ( R e a c t o r * =
A C E _ R e a c t o r : : i n s t a n c e ( ) ) ;
/ / A c t i v a t e t h e h a n d l e r ( c a l l e d b y t h e
. / / < A C E _ A c c e p t o r > o r < A C E _ C o n n e c t o r > ) .
v i r t u a l i n t o p e n ( v o i d * ) ;
/ / R e t u r n u n d e r l y i n g I P C m e c h a n i s m .
P E E R _ S T R E A M & p e e r ( v o i d ) ;
/ / . . .
p r i v a t e :
P E E R _ S T R E A M p e e r _ ; / / I P C m e c h a n i s m .
v i r t u a l ˜ A C E _ S v c _ H a n d l e r ( v o i d ) ;
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
ACE_Svc_Handler Implementation
#define PS PEER_STREAM
#define SS SYNCH_STRAT
template 
ACE_Svc_Handler::ACE_Svc_Handler
(ACE_Reactor *r): ACE_Service_Object (r)
{}
template 
int ACE_Svc_Handler::open
(void *) {
// Enable non-blocking I/O.
peer ().enable (ACE_NONBLOCK);
// Register handler with the Reactor.
reactor ()->register_handler
(this, ACE_Event_Handler::READ_MASK);
}
 By default, a
ACE_Svc_Handler
object is registered
with the singleton
ACE_Reactor
– This makes the
service “reactive”
so that no other
synchronization
mechanisms are
necessary
Vanderbilt University 93
A d v a n c e d A C E T u t o r i a l D o
O b j e c t D i a g r a m f o r O O L o g g i n g S e r v e r
 S e r v i c e
C o n f i g
S E R V E R
S E R V E R
L O G G I N G
D A E M O N
C O N N E C T I O N
R E Q U E S T
R E M O T E
C O N T R O L
O P E R A T I O N S
C L I E N T
L O G G I N G
R E C O R D S
C L I E N T
C L I E N T
C L I E N T
 L o g g i n g
H a n d l e r
 L o g g i n g
H a n d l e r
 L o g g i n g
A c c e p t o r
 S e r v i c e
M a n a g e r
 S e r v i c e
R e p o s i t o r y
 R e a c t o r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h e L o g g i n g _ H a n d l e r a n d
L o g g i n g _ A c c e p t o r C l a s s e s
/ / P e r f o r m s I / O w i t h c l i e n t l o g g i n g d a e m o n s .
c l a s s L o g g i n g _ H a n d l e r : p u b l i c
A C E _ S v c _ H a n d l e r < A C E _ S O C K _ A c c e p t o r : : P E E R _ S T R E A M ,
/ / T r a i t !
A C E _ N U L L _ S Y N C H >
{
p u b l i c :
/ / R e c v a n d p r o c e s s r e m o t e l o g g i n g r e c o r d s .
v i r t u a l i n t h a n d l e _ i n p u t ( A C E _ H A N D L E ) ;
} ;
/ / L o g g i n g _ H a n d l e r f a c t o r y .
c l a s s L o g g i n g _ A c c e p t o r : p u b l i c
A C E _ A c c e p t o r < L o g g i n g _ H a n d l e r , A C E _ S O C K _ A c c e p t o r >
{
p u b l i c :
/ / D y n a m i c l i n k i n g h o o k s .
v i r t u a l i n t i n i t ( i n t a r g c , c h a r * a r g v [ ] ) ;
v i r t u a l i n t f i n i ( v o i d ) ;
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Parameterizing IPC Mechanisms
 Q: How can you switch between different IPC mechanisms?
 A: By parameterizing IPC Mechanisms with C++ Templates, e.g.:
#if defined (ACE_USE_SOCKETS)
typedef ACE_SOCK_Acceptor PEER_ACCEPTOR;
#elif defined (ACE_USE_TLI)
typedef ACE_TLI_Acceptor PEER_ACCEPTOR;
#endif /* ACE_USE_SOCKETS */
class Logging_Handler : public
ACE_Svc_Handler
{ /* ... /* };
class Logging_Acceptor : public
ACE_Acceptor 
{ /* ... */ };
Vanderbilt University 96
Advanced ACE Tutorial Douglas C. Schmidt
Logging_Handler Input Method
Callback routine that receives logging records
int
Logging_Handler::handle_input (ACE_HANDLE)
{
// Call existing function to recv
// logging record and print to stdout.
ssize_t n =
handle_log_record (peer ().get_handle (),
ACE_STDOUT);
if (n > 0)
// Count the # of logging records
++request_count;
return n <= 0 ? -1 : 0;
}
 Implementation of
application-specific
logging method
 This is the main code
supplied by a
developer!
Vanderbilt University 97
A d v a n c e d A C E T u t o r i a l D o
L o g g i n g _ A c c e p t o r I n i t i a l i z a t i o n
a n d T e r m i n a t i o n
/ / A u t o m a t i c a l l y c a l l e d w h e n a L o g g i n g _ A c c e p t o r
/ / o b j e c t i s l i n k e d d y n a m i c a l l y .
L o g g i n g _ A c c e p t o r : : i n i t ( i n t a r g c , c h a r * a r g v [ ] )
{
A C E _ G e t _ O p t g e t _ o p t ( a r g c , a r g v , " p : " , 0 ) ;
A C E _ I N E T _ A d d r a d d r ( D E F A U L T _ P O R T ) ;
f o r ( i n t c ; ( c = g e t _ o p t ( ) ) ! = - 1 ; )
s w i t c h ( c ) {
c a s e ’ p ’ :
a d d r . s e t ( a t o i ( g e t o p t . o p t a r g ) ) ;
b r e a k ;
d e f a u l t :
b r e a k ;
}
/ / I n i t i a l i z e e n d p o i n t a n d r e g i s t e r
/ / w i t h t h e < A C E _ R e a c t o r > .
o p e n ( a d d r , A C E _ R e a c t o r : : i n s t a n c e ( ) ) ;
}
/ / A u t o m a t i c a l l y c a l l e d w h e n o b j e c t i s u n l i n k e d .
L o g g i n g _ A c c e p t o r : : f i n i ( v o i d ) { h a n d l e _ c l o s e ( ) ; }
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Putting the Pieces Together at Run-time
 Problem
– Prematurely committing ourselves to a particular logging server
configuration is inflexible and inefficient
 Forces
– It is useful to build systems by “scripting” components
– Certain design decisions can’t be made efficiently until run-time
– It is a bad idea to force users to “pay” for components they do not
use
 Solution
– Use the Component Configurator pattern to assemble the desired
logging server components dynamically
Vanderbilt University 99
Advanced ACE Tutorial Douglas C. Schmidt
The Component Configurator Pattern
Reactor1n
Concrete
Component
R
E
A
C
T
I
V
E
L
A
Y
E
R
C
O
N
F
I
G
U
R
A
T
I
O
N
L
A
Y
E
R
A
P
P
L
I
C
A
T
I
O
N
L
A
Y
E
R
1 1
Component
Config
n
Component
A
suspend()
resume()
init()
fini()
info()
1
Component
Repository
1
Event
Handler
Intent
 Decouples the
implementation of services
from the time when they are
configured
Forces Resolved
 Reduce resource utilization
 Support dynamic
(re)configuration
www.cs.wustl.edu/
˜schmidt/POSA/
Vanderbilt University 100
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e o f t h e A C E S e r v i c e
C o n fi g u r a t o r F r a m e w o r k
ACE_Service_Object
A C E _ S e r v i c e _ C o n f i g
A C E _ S e r v i c e _ R e p o s i t o r y
A C E _ S e r v i c e _ R e p o s i t o r y _ I t e r a t o r
ACE_Event_Handler
A p p l i c a t i o n  S e r v i c e
F r a m e w o r k c h a r a c t e r i s t i c s
 A C E _ S e r v i c e _ C o n f i g u s e s a v a r i a n t o f t h e
M o n o s t a t e p a t t e r n
 C a n b e a c c e s s e d e i t h e r v i a a s c r i p t o r
p r o g r a m m a t i c a l l y
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E S e r v i c e C o n fi g u r a t o r
F r a m e w o r k f o r t h e L o g g i n g S e r v e r
S E R V I C E
C O N F I G U R A T O R
R U N T I M E
 S e r v i c e
R e p o s i t o r y
S e r v i c e
O b j e c t
 T h r e a d
P o o l
L o g g e r
D L L S
S e r v i c e
O b j e c t
 T h r e a d
L o g g e r
d y n a m i c  L o g g e r  S e r v i c e _ O b j e c t  *
   l o g g e r : m a k e _ L o g g e r ( )  " - p  2 0 0 1 "
s v c . c o n f
F I L E
S e r v i c e
O b j e c t
R e a c t i v e
L o g g e r
R e a c t o r
 S e r v i c e
C o n f i g
 T h e e x i s t i n g L o g g i n g S e r v e r s e r v i c e i s
s i n g l e - t h r e a d e d
 O t h e r v e r s i o n s c o u l d b e m u l t i - t h r e a d e d
 N o t e h o w w e c a n s c r i p t t h i s v i a t h e s v c . c o n f
fi l e
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Dynamically Linking a Service
Dynamically linked factory function
that allocates a new
Logging_Acceptor
extern "C"
ACE_Service_Object *
make_Logger (void);
ACE_Service_Object *
make_Logger (void)
{
return new Logging_Acceptor;
// Framework automatically
// deletes memory.
}
 Application-specific factory
function used to dynamically
create a service
 The make_Logger() function
provides a hook between an
application-specific service
and the
application-independent ACE
mechanisms
– ACE handles all memory
allocation and deallocation
Vanderbilt University 103
Advanced ACE Tutorial Douglas C. Schmidt
Service Configuration
The logging service is configured
via scripting in a svc.conf file:
% cat ./svc.conf
# Dynamically configure
# the logging service
dynamic Logger
Service_Object *
logger:_make_Logger() "-p 2001"
# Note, .dll or .so suffix
# added to the logger
# automatically
Generic event-loop to
dynamically configure service
daemons
int main (int argc, char *argv[])
{
// Initialize the daemon and
// configure services
ACE_Service_Config::open (argc,
argv);
// Run forever, performing the
// configured services
ACE_Reactor::instance ()->
run_reactor_event_loop ();
/* NOTREACHED */
}
Vanderbilt University 104
Advanced ACE Tutorial Douglas C. Schmidt
State Chart for the Service Configurator Framework
INITIALIZED
CONFIGURE/
Service_Config::process_directives()
NETWORK  EVENT/
Reactor::dispatch()
RECONFIGURE/
Service_Config::process_directives()
SHUTDOWN/
Service_Config::close() AWAITING
EVENTS
CALL  HANDLER/
Event_Handler::handle_input()
IDLE
PERFORM
CALLBACK
START  EVENT  LOOP/
Reactor::run_event_loop()
Vanderbilt University 105
Advanced ACE Tutorial Douglas C. Schmidt
Advantages of OO Logging Server
 The OO architecture illustrated thus far decouples
application-specific service functionality from:
– Time when a service is configured into a process
– The number of services per-process
– The type of IPC mechanism used
– The type of event demultiplexing mechanism used
 We can use the techniques discussed thus far to extend applications
without:
– Modifying, recompiling, and relinking existing code
– Terminating and restarting executing daemons
 The remainder of the Logging Server slides examine a set of
techniques for decoupling functionality from concurrency
mechanisms, as well
Vanderbilt University 106
Advanced ACE Tutorial Douglas C. Schmidt
Concurrent OO Logging Server
 The structure of the Logging Server can benefit from concurrent
execution on a multi-processor platform
 This section examines ACE C++ classes and patterns that extend
the logging server to incorporate concurrency
– Note how most extensions require minimal changes to the
existing OO architecture...
 This example also illustrates additional ACE components involving
synchronization and multi-threading
Vanderbilt University 107
A d v a n c e d A C E T u t o r i a l D o
C o n c u r r e n t O O L o g g i n g
S e r v e r A r c h i t e c t u r e
N E T W O R K
S E R V E R
L O G G I N G   S E R V E R
L o g g i n g
H a n d l e r
L o g g i n g
H a n d l e r
L o g g i n g
A c c e p t o r
1 :  S O C K
    A c c e p t o r
2 :  a c c e p t ( )
4 :  h a n d l e _ i n p u t ( )
5 :  s p a w n ( )
3 :  c o n n e c t ( )
6 :  s e n d ( )
7 :  r e c v ( )
8 :  w r i t e ( )
C L I E N T
A
C L I E N T
B
6 :  s e n d ( )
7 :  r e c v ( )
8 :  w r i t e ( )
R e a c t o r
R u n s e a c h c l i e n t c o n n e c t i o n i n a s e p a r a t e t h r e a d
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Pseudo-code for Concurrent Server
 Pseudo-code for multi-threaded Logging_Handler factory
Logging Server
void handler_factory (void) {
initialize acceptor endpoint
foreach (pending connection event) {
accept connection
spawn a thread to handle connection and
run logging_handler() entry point
}
}
 Pseudo-code for logging_handler() function
void logging_handler (void) {
foreach (incoming logging records from client)
call handle_log_record()
exit thread
}
Vanderbilt University 109
Advanced ACE Tutorial Douglas C. Schmidt
Concurrency Overview
THREADS
PROCESS
SHARED  ADDRESS  SPACE
 A thread is a sequence of
instructions executed in one
or more processes
– One process !
stand-alone systems
– More than one process !
distributed systems
Traditional OS processes contain a single thread of control
 This simplifies programming since a sequence of execution steps is
protected from unwanted interference by other execution
sequences...
Vanderbilt University 110
Advanced ACE Tutorial Douglas C. Schmidt
Traditional Approaches to OS Concurrency
1. Device drivers and programs with signal handlers utilize a limited
form of concurrency
 e.g., asynchronous I/O
 Note that concurrency encompasses more than multi-threading...
2. Many existing programs utilize OS processes to provide
“coarse-grained” concurrency
 e.g.,
– Client/server database applications
– Standard network daemons like UNIX INETD
 Multiple OS processes may share memory via memory mapping
or shared memory and use semaphores to coordinate execution
 The OS kernel scheduler dictates process behavior
Vanderbilt University 111
Advanced ACE Tutorial Douglas C. Schmidt
Evaluating Traditional OS Process-based Concurrency
 Advantages
– Easy to keep processes from interfering
 A process combines security, protection, and robustness
 Disadvantages
– Complicated to program, e.g.,
– Signal handling may be tricky
– Shared memory may be inconvenient
 Inefficient
– The OS kernel is involved in synchronization and process
management
– Difficult to exert fine-grained control over scheduling and priorities
Vanderbilt University 112
Advanced ACE Tutorial Douglas C. Schmidt
Modern OS Concurrency
 Modern OS platforms typically provide a standard set of APIs that
handle
– Process/thread creation and destruction
– Various types of process/thread synchronization and mutual
exclusion
– Asynchronous facilities for interrupting long-running
processes/threads to report errors and control program behavior
 Once the underlying concepts are mastered, it’s relatively easy to
learn different concurrency APIs
– e.g., traditional UNIX process operations, Solaris threads, POSIX
pthreads, WIN32 threads, Java threads, etc.
Vanderbilt University 113
Advanced ACE Tutorial Douglas C. Schmidt
Lightweight Concurrency
 Modern operating systems provide lightweight mechanisms that
manage and synchronize multiple threads within a process
– Some systems also allow threads to synchronize across multiple
processes
 Benefits of threads
1. Relatively simple and efficient to create, control, synchronize, and
collaborate
– Threads share many process resources by default
2. Improve performance by overlapping computation and
communication
– Threads may also consume less resources than processes
3. Improve program structure
– e.g., compared with using asynchronous I/O
Vanderbilt University 114
A d v a n c e d A C E T u t o r i a l D o
E x a m p l e : S i n g l e - t h r e a d e d v s .
M u l t i - t h r e a d e d A p p l i c a t i o n s
M U L T I -
T H R E A D E D   R P C
C L I E N T
S E R V E RC T
U
S
E
R
K
E
R
N
E
L
T H R E A D
B L O C K E D
U
S
E
R
K
E
R
N
E
L
S E R V I C E
E X E C U T E S
R E Q U E S T
R E S P O N S E
S E R V E R
C L I E N T
U
S
E
R
K
E
R
N
E
L
U
S
E
R
K
E
R
N
E
L
S E R V I C E
E X E C U T E S
R E Q U E S T
R E S P O N S E
S E R V E R
U
S
E
R
K
E
R
N
E
L
S E R V I C E
E X E C U T E S
R E Q U E S T
R E S P O N S E
S I N G L E -
T H R E A D E D   R P C
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Hardware and OS Concurrency Support
THREAD
PE
PROCESSING
ELEMENT
LIGHTWEIGHT
PROCESS 
LWP
UNIX
PROCESS
PE PE PE PEPE PE PE PE
SHARED  MEMORY
K
E
R
N
E
L
-
L
E
V
E
L
U
S
E
R
-
L
E
V
E
L
LWP LWP LWPLWP LWP LWP LWP
Four typical
abstractions
1. Application
threads
2. Lightweight
processes
3. Kernel threads
4. Processing
elements
Vanderbilt University 116
Advanced ACE Tutorial Douglas C. Schmidt
Application Threads
Most process resources are
equally accessible to all threads
in a process, e.g.,
 Virtual memory
 User permissions and access
control privileges
 Open files
 Signal handlers
Each thread also contains unique
information, e.g.,
 Identifier
 Register set (e.g., PC and SP)
 Run-time stack
 Signal mask
 Priority
 Thread-specific data (e.g.,
errno)
Note, there is no MMU protection for threads in a single process
Vanderbilt University 117
Advanced ACE Tutorial Douglas C. Schmidt
Kernel-level vs. User-level Threads
 Application and system characteristics influence the choice of
user-level vs. kernel-level threading
 A high degree of “virtual” application concurrency implies user-level
threads (i.e., unbound threads)
– e.g., desktop windowing system on a uni-processor
 A high degree of “real” application parallelism implies lightweight
processes (LWPs) (i.e., bound threads)
– e.g., video-on-demand server or matrix multiplication on a
multi-processor
Vanderbilt University 118
Advanced ACE Tutorial Douglas C. Schmidt
Overview of OS Synchronization Mechanisms
 Threads share resources in a process address space
 Therefore, they must use synchronization mechanisms to coordinate
their access to shared data
 Traditional OS synchronization mechanisms are very low-level,
tedious to program, error-prone, and non-portable
 ACE encapsulates these mechanisms with wrapper facades and
higher-level patterns/components
Vanderbilt University 119
Advanced ACE Tutorial Douglas C. Schmidt
Common OS Synchronization Mechanisms
 Mutual exclusion (mutex) locks
– Serialize thread access to a shared resource
 Counting semaphores
– Synchronize thread execution
 Readers/writer (R/W) locks
– Serialize resources that are searched more than changed
 Condition variables
– Used to block threads until shared data changes state
 File locks
– System-wide R/W locks accessed by processes
Vanderbilt University 120
Advanced ACE Tutorial Douglas C. Schmidt
Additional ACE Synchronization Mechanism
 Events
– Gates and latches
 Barriers
– Allows threads to synchronize their completion
 Token
– Provides FIFO scheduling order
 Task
– Provides higher-level “active object” for concurrent applications
 Thread-specific storage
– Low-overhead, contention-free storage
Vanderbilt University 121
A d v a n c e d A C E T u t o r i a l D o
C o n c u r r e n c y M e c h a n i s m s i n A C E
To k e n
A D V A N C E D   S Y N C H
B a r r i e r
C o n d i t i o n
MUTEX
N u l l
C o n d i t i o n
C O N D I T I O N S
Ta s k
S Y N C H
A C T I V E
O B J E C T S
T h r e a d
M a n a g e r
M A N A G E R S
P r o c e s s
M a n a g e r
G u a r d
G U A R D S
R e a d
G u a r d
W r i t e
G u a r d
A t o m i c
O p
L O C K
T Y P E
T S S
TYPE
T h r e a d
M u t e x
N u l l
M u t e x
R W
M u t e x
E v e n t s
S e m a p h o r e
S Y N C H   W R A P P E R S
F i l e
L o c k
P r o c e s s
M u t e x
T h r e a d
M u t e x
T h r e a d
S e m a p h o r e
P r o c e s s
S e m a p h o r e
 A l l A C E C o n c u r r e n c y m e c h a n i s m s a r e p o r t e d t o
a l l O S p l a t f o r m s
 w w w . c s . w u s t l . e d u / ˜ s c h m i d t / A C E /
b o o k 1 /
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Addressing Logger Server Concurrency Challenges
 Problem
– Multi-threaded logging servers may be necessary when
single-threaded reactive servers inefficient, non-scalable, or
non-robust
 Forces
– Multi-threading can be very hard to program
– No single multi-threading model is always optimal
 Solution
– Use the Active Object pattern to allow multiple concurrent logging
server operations using an OO programming style
Vanderbilt University 123
A d v a n c e d A C E T u t o r i a l D o
T h e A c t i v e O b j e c t P a t t e r n
P r o x y
F u t u r e  m 1 ( )
F u t u r e  m 2 ( )
F u t u r e  m 3 ( )
H I D D E N
F R O M
C L I E N T S
V I S I B L E
T O
C L I E N T S
2 :  e n q u e u e ( M 1 )
A c t i v a t i o n
L i s t
e n q u e u e ( )
d e q u e u e ( )
S e r v a n t
n
1
l o o p  {
  m  =  a c t _ l i s t . d e q u e u e ( )
  i f  ( m . g u a r d ( ) )  m . c a l l ( )
  e l s e  a c t _ l i s t . e n q u e u e  ( m ) ;
}
S c h e d u l e r
d i s p a t c h ( )
e n q u e u e ( )
m 1 ( )
m 2 ( )
m 3 ( )
4 :  m 1 ( )
1 1
1 :  e n q u e u e ( n e w  M 1 )
3 :  d i s p a t c h ( )
M e t h o d
R e q u e s t
g u a r d ( )
c a l l ( )
M 1
M 2
M 3
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P O S A /
I n t e n t
 Decouples method
execution from method
invocation to enhance
concurrency and simplify
synchronized access to
an object that resides in
its own thread of control
F o r c e s R e s o l v e d
 A l l o w b l o c k i n g
o p e r a t i o n s
 P e r m i t fl e x i b l e
c o n c u r r e n c y
s t r a t e g i e s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E S u p p o r t f o r A c t i v e O b j e c t s
:  T A S K
S T A T E
:  M e s s a g e
Q u e u e
A C T I V E
t
2
 :
T a s k
2 :  e n q u e u e  ( m s g )
1 :  p u t  ( m s g )
t
1
 :
T a s k
t
3
 :
T a s k
6 :  p u t  ( m s g )
3 :  s v c  ( )
4 :  d e q u e u e  ( m s g )
5 :  d o _ w o r k ( m s g )
A C T I V E
A C T I V E
:  T A S K
S T A T E
:  M e s s a g e
Q u e u e
:  T A S K
S T A T E
:  M e s s a g e
Q u e u e
T h e A C E T a s k f r a m e w o r k c a n b e u s e d t o
i m p l e m e n t t h e c o m p l e t e A c t i v e O b j e c t p a t t e r n o r
l i g h t e r w e i g h t s u b s e t s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The ACE Task Framework
 An ACE_Task binds a separate thread of control together with an
object’s data and methods
– Multiple active objects may execute in parallel in separate
lightweight or heavyweight processes
 ACE_Task objects communicate by passing typed messages to
other ACE_Task objects
– Each ACE_Task maintains a queue of pending messages that it
processes in priority order
 ACE_Task is a low-level mechanism to support active objects
Vanderbilt University 126
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e o f t h e A C E T a s k F r a m e w o r k
A C E _ T h r e a d _ M a n a g e r ACE_Task
0 . . 1 *
S Y N C H
A C E _ M e s s a g e _ Q u e u e
S Y N C H
A C E _ M e s s a g e _ B l o c k
*
1
ACE_Service_ObjectACE_Event_Handler
F r a m e w o r k c h a r a c t e r i s t i c s
1 . A C E _ T a s k s c a n r e g i s t e r w i t h a n A C E _ R e a c t o r
2 . T h e y c a n b e d y n a m i c a l l y l i n k e d
3 . T h e y c a n q u e u e d a t a
4 . T h e y c a n r u n a s a c t i v e o b j e c t s i n 1 o r m o r e
t h r e a d s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The ACE_Task Class Public Interface
template 
// Synchronization aspect
class ACE_Task : public ACE_Service_Object {
public:
// Initialization/termination hooks.
virtual int open (void *args = 0) = 0;
virtual int close (u_long = 0) = 0;
// Transfer msg to queue for immediate processing.
virtual int put (ACE_Message_Block *, ACE_Time_Value * = 0) = 0;
// Run by a daemon thread for deferred processing.
virtual int svc (void) = 0;
// Turn task into active object.
int activate (long flags, int threads = 1);
Vanderbilt University 128
Advanced ACE Tutorial Douglas C. Schmidt
ACE_Task Class Protected Interface
Many of the following methods are used by put() and svc()
// Accessors to internal queue.
ACE_Message_Queue *msg_queue (void);
void msg_queue (ACE_Message_Queue *);
// Accessors to thread manager.
ACE_Thread_Manager *thr_mgr (void);
void thr_mgr (ACE_Thread_Manager *);
// Insert message into the message list.
int putq (ACE_Message_Block *, ACE_Time_Value *tv = 0);
// Extract the first message from the list (blocking).
int getq (ACE_Message_Block *&mb, ACE_Time_Value *tv = 0);
// Hook into the underlying thread library.
static void *svc_run (ACE_Task *);
Vanderbilt University 129
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Combining Threads & C++ Objects
 Q: What is the svc_run() function and why is it a static method?
 A: OS thread APIs require C-style functions as entry point
 The ACE Task framework encapsulates the svc_run() function
within the ACE_Task::activate() method:
template  int
ACE_Task::activate (long flags, int n_threads) {
if (thr_mgr () == NULL) thr_mgr (ACE_Thread_Manager::instance ());
thr_mgr ()->spawn_n (n_threads, &ACE_Task::svc_run,
(void *) this, flags);
}
1. ACE_Task::activate ()
2. ACE_Thread_Manager::spawn
    (svc_run, this);
3. _beginthreadex
    (0, 0,
     svc_run, this,
     0, &thread_id); RUN-TIME
THREAD STACK
4. template  void *
   ACE_Task::svc_run
     (ACE_Task *t) {
     // ...
     void *status = t->svc ();
     // ...
     return status; // Thread return.
   }
Vanderbilt University 130
Advanced ACE Tutorial Douglas C. Schmidt
The svc_run() Adapter Function
ACE_Task::svc_run() is static method used as the entry point to
execute an instance of a service concurrently in its own thread
template  void *
ACE_Task::svc_run (ACE_Task *t)
{
// Thread added to thr_mgr() automatically on entry.
// Run service handler and record return value.
void *status = (void *) t->svc ();
t->close (u_long (status));
// Status becomes "return" value of thread...
return status;
// Thread removed from thr_mgr() automatically on return.
}
Vanderbilt University 131
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Motivation
for the ACE_Thread_Manager
 Q: How can groups of collaborating threads be managed atomically?
 A: Develop the ACE_Thread_Manager class that:
– Supports the notion of thread groups
 i.e., operations on all threads in a group
– Implements barrier synchronization on thread exits
– Shields applications from incompatibilities between different OS
thread libraries
 e.g., detached threads and thread joins
Vanderbilt University 132
Advanced ACE Tutorial Douglas C. Schmidt
Using ACE Task Framework for Logging Server
Process remote logging records
by looping until the client
terminates connection
int
Thr_Logging_Handler::svc (void)
{
while (handle_input () != -1)
// Call existing function
// to recv logging record
// and print to stdout.
continue;
return 0;
}
 The OO implementation localizes
the application-specific part of
the logging service in a single
point, while leveraging off
reusable ACE components
 Compare with original, which
borrow’s the Reactor thread
int
Logging_Handler::handle_input (void)
{
handle_log_record
(peer ().get_handle (),
ACE_STDOUT);
// ...
}
Vanderbilt University 133
A d v a n c e d A C E T u t o r i a l D o
C l a s s D i a g r a m f o r C o n c u r r e n t
O O L o g g i n g S e r v e r
T h r
L o g g i n g
A c c e p t o r
T h r _ L o g g i n g _ H a n d l e r
S O C K _ A c c e p t o r
T h r
L o g g i n g
H a n d l e r
S O C K _ S t r e a m
N U L L _ S y n c h
S v c
H a n d l e r
A c c e p t o r
S V C _ H A N D L E R
P E E R _ A C C E P T O R
P E E R _ S T R E A M
S Y N C H
C
O
N
N
E
C
T
IO
N
-
O
R
IE
N
T
E
D
C
O
M
P
O
N
E
N
T
S
A
P
P
L
IC
A
T
IO
N
-
S
P
E
C
IF
IC
C
O
M
P
O
N
E
N
T
S
A
C
E
F
R
A
M
E
W
O
R
K
C
O
M
P
O
N
E
N
T
S
C o n c u r r e n c y
global
C o n n e c t i o n
R e a c t o r
< < a c t i v a t e s > >
1
n
P E E R
A C C E P T O R
P E E R
S T R E A M
I P C _ S A P
S t r e a m
S e r v i c e
C o n f i g u r a t o r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h r _ L o g g i n g _ A c c e p t o r a n d
T h r _ L o g g i n g _ H a n d l e r I n t e r f a c e s
T e m p l a t e c l a s s e s t h a t c r e a t e , c o n n e c t , a n d a c t i v a t e
a n e w t h r e a d t o h a n d l e e a c h c l i e n t
c l a s s T h r _ L o g g i n g _ H a n d l e r
: p u b l i c L o g g i n g _ H a n d l e r
/ / I n h e r i t s < h a n d l e _ i n p u t >
{
p u b l i c :
/ / O v e r r i d e d e f i n i t i o n i n < A C E _ S v c _ H a n d l e r >
/ / c l a s s t o s p a w n a n e w t h r e a d ! T h i s m e t h o d
/ / i s c a l l e d b y t h e < A C E _ A c c e p t o r > .
v i r t u a l i n t o p e n ( v o i d * ) ;
/ / P r o c e s s r e m o t e l o g g i n g r e c o r d s .
v i r t u a l i n t s v c ( v o i d ) ;
} ;
c l a s s T h r _ L o g g i n g _ A c c e p t o r : p u b l i c
A C E _ A c c e p t o r < T h r _ L o g g i n g _ H a n d l e r ,
A C E _ S O C K _ A c c e p t o r >
{
/ / S a m e a s < L o g g i n g _ A c c e p t o r > . . .
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h r _ L o g g i n g _ H a n d l e r
I m p l e m e n t a t i o n
O v e r r i d e d e fi n i t i o n i n t h e A C E _ S v c _ H a n d l e r c l a s s
t o s p a w n a n e w t h r e a d
i n t
T h r _ L o g g i n g _ H a n d l e r : : o p e n ( v o i d * )
{
/ / S p a w n a n e w t h r e a d t o h a n d l e
/ / l o g g i n g r e c o r d s w i t h t h e c l i e n t .
a c t i v a t e ( T H R _ D E T A C H E D ) ;
}
P r o c e s s r e m o t e l o g g i n g r e c o r d s b y l o o p i n g u n t i l
c l i e n t t e r m i n a t e s c o n n e c t i o n
i n t
T h r _ L o g g i n g _ H a n d l e r : : s v c ( v o i d )
{
w h i l e ( h a n d l e _ i n p u t ( ) ! = - 1 )
/ / C a l l e x i s t i n g f u n c t i o n t o r e c v
/ / l o g g i n g r e c o r d a n d p r i n t t o s t d o u t .
c o n t i n u e ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Dynamically Reconfiguring the Logging Server
The logging service is
configured via scripting in a
svc.conf file:
% cat ./svc.conf
# Dynamically reconfigure
# the logging service
remove Logger
dynamic Logger
Service_Object *
thr_logger:_make_Logger()
"-p 2002"
# .dll or .so suffix added to
# "thr_logger" automatically
Dynamically linked factory
function that allocates a new
threaded Logging_Acceptor
extern "C"
ACE_Service_Object *make_Logger (void);
ACE_Service_Object *
make_Logger (void)
{
return new Thr_Logging_Acceptor;
}
Logging service is reconfigured by changing the svc.conf file and
sending SIGHUP signal to server
Vanderbilt University 137
Advanced ACE Tutorial Douglas C. Schmidt
Caveats for the Concurrent Logging Server
 The concurrent Logging Server has several problems
– Output in the handle_log_record() function is not serialized
– The auto-increment of global variable request_count is also
not serialized
 Lack of serialization leads to errors on many shared memory
multi-processor platforms...
– Note that this problem is indicative of a large class of errors in
concurrent programs...
 The following slides compare and contrast a series of techniques
that address this problem
Vanderbilt University 138
Advanced ACE Tutorial Douglas C. Schmidt
Explicit Synchronization Mechanisms
 One approach for serialization uses OS mutual exclusion
mechanisms explicitly, e.g.,
// at file scope
mutex_t lock; // SunOS 5.x synchronization mechanism
// ...
handle_log_record (ACE_HANDLE in_h, ACE_HANDLE out_h)
{
// in method scope ...
mutex_lock (&lock);
if (ACE_OS::write (out_h, lr.buf, lr.size) == -1)
return -1;
mutex_unlock (&lock);
// ...
}
 However, adding these mutex calls explicitly causes problems...
Vanderbilt University 139
Advanced ACE Tutorial Douglas C. Schmidt
Problem: Explicit mutex_* Calls
 Inelegant ! “Impedance mismatch” with C/C++
 Obtrusive
– Must find and lock all uses of write()
– Can yield inheritance anomaly
 Error-prone
– C++ exception handling and multiple method exit points
– Thread mutexes won’t work for separate processes
– Global mutexes may not be initialized correctly
 Non-portable ! Hard-coded to Solaris 2.x
 Inefficient ! e.g., expensive for certain platforms/designs
Vanderbilt University 140
A d v a n c e d A C E T u t o r i a l D o
S o l u t i o n : S y n c h r o n i z a t i o n
W r a p p e r F a c a d e s
c l a s s A C E _ T h r e a d _ M u t e x
{
p u b l i c :
A C E _ T h r e a d _ M u t e x ( v o i d ) {
m u t e x _ i n i t ( & l o c k _ , U S Y N C H _ T H R E A D , 0 ) ;
}
˜ A C E _ T h r e a d _ M u t e x ( v o i d ) { m u t e x _ d e s t r o y ( & l o c k _ ) ;
i n t a c q u i r e ( v o i d ) { r e t u r n m u t e x _ l o c k ( & l o c k _ ) ; }
i n t t r y a c q u i r e ( v o i d )
{ r e t u r n m u t e x _ t r y l o c k ( & l o c k _ ) ; }
i n t r e l e a s e ( v o i d ) { r e t u r n m u t e x _ u n l o c k ( & l o c k _ ) ;
p r i v a t e :
/ / S u n O S 5 . x s e r i a l i z a t i o n m e c h a n i s m .
m u t e x _ t l o c k _ ;
v o i d o p e r a t o r = ( c o n s t A C E _ T h r e a d _ M u t e x & ) ;
A C E _ T h r e a d _ M u t e x ( c o n s t A C E _ T h r e a d _ M u t e x & ) ;
} ;
N o t e h o w w e p r e v e n t i m p r o p e r c o p y i n g a n d
a s s i g n m e n t b y u s i n g C + + a c c e s s c o n t r o l s p e c i fi e r s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P o r t i n g A C E _ T h r e a d _ M u t e x t o
W i n d o w s N T
c l a s s A C E _ T h r e a d _ M u t e x
{
p u b l i c :
A C E _ T h r e a d _ M u t e x ( v o i d ) {
l o c k _ = C r e a t e M u t e x ( 0 , F A L S E , 0 ) ;
}
˜ A C E _ T h r e a d _ M u t e x ( v o i d ) {
C l o s e H a n d l e ( l o c k _ ) ;
}
i n t a c q u i r e ( v o i d ) {
r e t u r n W a i t F o r S i n g l e O b j e c t ( l o c k _ , I N F I N I T E ) ;
}
i n t t r y a c q u i r e ( v o i d ) {
r e t u r n W a i t F o r S i n g l e O b j e c t ( l o c k _ , 0 ) ;
}
i n t r e l e a s e ( v o i d ) {
r e t u r n R e l e a s e M u t e x ( l o c k _ ) ;
}
p r i v a t e :
A C E _ H A N D L E l o c k _ ; / / W i n d o w s l o c k i n g m e c h a n i s m .
/ / . . .
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Using the C++ Mutex Wrapper Facade
 Using C++ wrapper facades improves portability and elegance
// at file scope.
ACE_Thread_Mutex lock; // Implicitly unlocked.
// ...
handle_log_record (ACE_HANDLE in_h, ACE_HANDLE out_h) {
// in method scope ...
lock.acquire ();
if (ACE_OS::write (out_h, lr.buf, lr.size) == -1)
return -1;
lock.release ();
// ...
 However, this doesn’t really solve the tedium or error-proneness
problems
– www.cs.wustl.edu/˜schmidt/PDF/ObjMan.pdf
Vanderbilt University 143
Advanced ACE Tutorial Douglas C. Schmidt
Automated Mutex Acquisition and Release
 To ensure mutexes are locked and unlocked, we’ll define a template
class that acquires and releases a mutex automatically
template 
class ACE_Guard
{
public:
ACE_Guard (LOCK &m): lock_ (m) { lock_.acquire (); }
˜ACE_Guard (void) { lock_.release (); }
// ... other methods omitted ...
private:
LOCK &lock_;
}
 ACE_Guard uses the Scoped Locking idiom whereby a constructor
acquires a resource and the destructor releases the resource
Vanderbilt University 144
Advanced ACE Tutorial Douglas C. Schmidt
The ACE_GUARD Macros
 ACE defines a set of macros that simplify the use of the
ACE_Guard, ACE_Write_Guard, and ACE_Read_Guard classes
– These macros test for deadlock and detect when operations on
the underlying locks fail
#define ACE_GUARD(MUTEX,OB,LOCK) \
ACE_Guard OB (LOCK); if (OB.locked () == 0) return;
#define ACE_GUARD_RETURN(MUTEX,OB,LOCK,RET) \
ACE_Guard OB (LOCK); if (OB.locked () == 0) return RET;
#define ACE_WRITE_GUARD(MUTEX,OB,LOCK) \
ACE_Write_Guard OB (LOCK); if (OB.locked () == 0) return;
#define ACE_WRITE_GUARD_RETURN(MUTEX,OB,LOCK,RET) \
ACE_Write_Guard OB (LOCK); if (OB.locked () == 0) return RET;
#define ACE_READ_GUARD(MUTEX,OB,LOCK) \
ACE_Read_Guard OB (LOCK); if (OB.locked () == 0) return;
#define ACE_READ_GUARD_RETURN(MUTEX,OB,LOCK,RET) \
ACE_Read_Guard OB (LOCK); if (OB.locked () == 0) return RET;
Vanderbilt University 145
A d v a n c e d A C E T u t o r i a l D o
T h r e a d - s a f e h a n d l e _ l o g _ r e c o r d ( )
F u n c t i o n
t e m p l a t e < c l a s s L O C K = A C E _ T h r e a d _ M u t e x > s s i z e _ t
h a n d l e _ l o g _ r e c o r d ( A C E _ H A N D L E i n , A C E _ H A N D L E o u t ) {
/ / b e w a r e s t a t i c i n i t i a l i z a t i o n . . .
s t a t i c L O C K l o c k ;
A C E _ U I N T _ 3 2 l e n ;
A C E _ L o g _ R e c o r d l r ;
/ / T h e f i r s t r e c v r e a d s t h e l e n g t h ( s t o r e d a s a
/ / f i x e d - s i z e i n t e g e r ) o f a d j a c e n t l o g g i n g r e c o r d .
s s i z e _ t n = s . r e c v _ n ( ( c h a r * ) & l e n , s i z e o f l e n ) ;
i f ( n < = 0 ) r e t u r n n ;
l e n = n t o h l ( l e n ) ; / / C o n v e r t b y t e - o r d e r i n g
/ / P e r f o r m s a n i t y c h e c k !
i f ( l e n > s i z e o f ( l r ) ) r e t u r n - 1 ;
/ / T h e s e c o n d r e c v t h e n r e a d s < l e n > b y t e s t o
/ / o b t a i n t h e a c t u a l r e c o r d .
s . r e c v _ n ( ( c h a r * ) & l r , s i z e o f l r ) ;
/ / D e c o d e a n d p r i n t r e c o r d .
d e c o d e _ l o g _ r e c o r d ( & l r ) ;
/ / A u t o m a t i c a l l y a c q u i r e m u t e x l o c k .
A C E _ G U A R D _ R E T U R N ( L O C K , g u a r d , l o c k , - 1 ) ;
i f ( A C E _ O S : : w r i t e ( o u t , l r . b u f , l r . s i z e ) = = - 1 )
r e t u r n - 1 ; / / A u t o m a t i c a l l y r e l e a s e m u t e x l o c k .
r e t u r n 0 ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Motivating the ACE_Guard Design
 Q: Why is ACE_Guard parameterized by the type of LOCK?
 A: since many different flavors of locking can benefit from the
Scoped Locking protocol
– e.g., non-recursive vs. recursive mutexes, intra-process vs.
inter-process mutexes, readers/writer mutexes, POSIX and
System V semaphores, file locks, and the null mutex
 Q: Why are templates used, as opposed to
inheritance/polymorphism?
 A: since they are more efficient and can reside in shared memory
 All ACE synchronization wrapper facades use the Adapter pattern to
provide identical interfaces to facilitate parameterization
Vanderbilt University 147
Advanced ACE Tutorial Douglas C. Schmidt
The Adapter Pattern
specific_request()
Adaptee1
client
1: request ()
2: specific_request()
Adapter
request()
Adaptee2
specific_request()Adaptee3
specific_request()
Intent
 Convert the interface
of a class into another
interface client
expects
Force resolved:
 Provide an interface
that captures
similarities between
different OS
mechanisms, e.g.,
locking or IPC
Vanderbilt University 148
Advanced ACE Tutorial Douglas C. Schmidt
Remaining Caveats
int Logging_Handler::handle_input (void)
{
ssize_t n = handle_log_record
(peer ().get_handle (), ACE_STDOUT);
if (n > 0)
// Count # of logging records.
++request_count;
// Danger, race condition!!!
return n <= 0 ? -1 : 0;
}
A more elegant solution incorporates
parameterized types, overloading, and the
Strategized Locking pattern, as discussed in
C++NPv1
 There is a race
condition when
incrementing the
request_count
variable
 Solving this problem
using the
ACE_Thread_Mutex
or ACE_Guard classes
is still tedious,
low-level, and
error-prone
Vanderbilt University 149
A d v a n c e d A C E T u t o r i a l D o
T r a n s p a r e n t l y P a r a m e t e r i z i n g
S y n c h r o n i z a t i o n U s i n g C + +
U s e t h e Strategized Locking p a t t e r n , C + +
t e m p l a t e s , a n d o p e r a t o r o v e r l o a d i n g t o d e fi n e
“ a t o m i c o p e r a t o r s ”
t e m p l a t e < c l a s s L O C K = A C E _ T h r e a d _ M u t e x ,
c l a s s T Y P E = u _ l o n g >
c l a s s A C E _ A t o m i c _ O p {
p u b l i c :
A C E _ A t o m i c _ O p ( T Y P E c = 0 ) { c o u n t _ = c ; }
T Y P E o p e r a t o r + + ( v o i d ) {
A C E _ G U A R D ( L O C K , g u a r d , l o c k _ ) ; r e t u r n + + c o u n t _ ;
}
o p e r a t o r T Y P E ( ) {
A C E _ G U A R D ( L O C K , g u a r d , l o c k _ ) ; r e t u r n c o u n t _ ;
}
/ / O t h e r a r i t h m e t i c o p e r a t i o n s o m i t t e d . . .
p r i v a t e :
L O C K l o c k _ ;
T Y P E c o u n t _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Final Version of Concurrent Logging Server
 Using the Atomic_Op class, only one change is made
// At file scope.
typedef ACE_Atomic_Op<> COUNTER; // Note default parameters...
COUNTER request_count;
 request_count is now serialized automatically
for (; ; ++request_count) // ACE_Atomic_Op::operator++
handle_log_record (get_handle (), ACE_STDOUT);
 The original non-threaded version may be supported efficiently as
follows:
typedef ACE_Atomic_Op COUNTER;
//...
for (; ; ++request_count)
handle_log_record
(get_handle (), ACE_STDOUT);
Vanderbilt University 151
Advanced ACE Tutorial Douglas C. Schmidt
Concurrent Web Client/Server Example
 The following example illustrates a concurrent OO architecture for a
high-performance Web client/server
 Key functional and non-functional system requirements are:
– Robust implementation of HTTP 1.0 protocol
 i.e., resilient to incorrect or malicious Web clients/servers
– Extensible for use with other protocols
 e.g., DICOM, HTTP 1.1, CORBA Simple Flow Protocol (SFP)
– Leverage multi-processor hardware and OS software
 e.g., Support various concurrency patterns
Vanderbilt University 152
A d v a n c e d A C E T u t o r i a l D o
G e n e r a l W e b C l i e n t / S e r v e r I n t e r a c t i o n s
W W W
S E R V E R
2 :  i n d e x . h t m l
1 :  G E T  ~ s c h m i d t
H T T P / 1 . 0
C O M M U N I C A T I O N   P R O T O C O L
( E . G . ,  H T T P )
G U I
H T M L
P A R S E R
R E Q U E S T E R
G R A P H I C S
A D A P T E R
N E T W O R K
O S   K E R N E L
O S   I / O   S U B S Y S T E M
N E T W O R K   A D A P T E R S
O S   K E R N E L
O S   I / O   S U B S Y S T E M
N E T W O R K   A D A P T E R S
D I S P A T C H E R
P R O T O C O L
H A N D L E R S
W W W
C L I E N T
w w w . c s . w u s t l . e d u / ˜ j x h / r e s e a r c h /
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P s e u d o - c o d e f o r C o n c u r r e n t W e b
S e r v e r
 P s e u d o - c o d e f o r m a s t e r s e r v e r
v o i d m a s t e r _ s e r v e r ( v o i d )
{
i n i t i a l i z e q u e u e a n d a c c e p t o r a t p o r t 8 0
s p a w n p o o l o f w o r k e r t h r e a d s
f o r e a c h ( p e n d i n g w o r k r e q u e s t f r o m c l i e n t s ) {
r e c e i v e a n d q u e u e r e q u e s t o n q u e u e
}
e x i t p r o c e s s
}
 P s e u d o - c o d e f o r t h r e a d p o o l w o r k e r s
v o i d w o r k e r ( v o i d )
{
f o r e a c h ( w o r k r e q u e s t o n q u e u e )
d e q u e u e a n d p r o c e s s r e q u e s t
e x i t t h r e a d
}
 A s u s u a l , m a k e s u r e t o a v o i d t h e “ g r a n d m i s t a k e ”
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Motivating a Request Queue
 Q: Why use a request queue to store messages, rather than directly
reading from I/O handles?
 A:
– Promotes more efficient use of multiple CPUs via load balancing
– Enables transparent interpositioning and prioritization
– Makes it easier to shut down the server correctly and portably
– Improves robustness to “denial of service” attacks
– Moves queueing into the application process rather than OS
 Drawbacks
– Using a message queue may lead to greater context switching
and synchronization overhead...
– Single point for bottlenecks
Vanderbilt University 155
A d v a n c e d A C E T u t o r i a l D o
T h r e a d E n t r y P o i n t
t y p e d e f A C E _ U n b o u n d e d _ Q u e u e < M e s s a g e > M E S S A G E _ Q U E U E ;
t y p e d e f u _ l o n g C O U N T E R ;
/ / T r a c k t h e n u m b e r o f r e q u e s t s
C O U N T E R r e q u e s t _ c o u n t ; / / A t f i l e s c o p e .
/ / E n t r y p o i n t i n t o t h e W e b H T T P 1 . 0 p r o t o c o l ,
/ / w h i c h r u n s i n e a c h t h r e a d i n t h e t h r e a d p o o l .
v o i d * w o r k e r ( M E S S A G E _ Q U E U E * m s g _ q u e u e )
{
M e s s a g e m b ; / / M e s s a g e c o n t a i n i n g H T T P r e q u e s t .
w h i l e ( m s g _ q u e u e - > d e q u e u e _ h e a d ( m b ) ) > 0 ) {
/ / K e e p t r a c k o f n u m b e r o f r e q u e s t s .
+ + r e q u e s t _ c o u n t ;
/ / P r i n t d i a g n o s t i c
c o u t < < " g o t n e w r e q u e s t "
< < A C E _ O S : : t h r _ s e l f ( )
< < e n d l ;
/ / I d e n t i f y a n d p e r f o r m W e b S e r v e r
/ / r e q u e s t p r o c e s s i n g h e r e . . .
}
r e t u r n 0 ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
M a s t e r S e r v e r D r i v e r F u n c t i o n
/ / T h r e a d f u n c t i o n p r o t o t y p e .
t y p e d e f v o i d * ( * T H R _ F U N C ) ( v o i d * ) ;
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] ) {
p a r s e _ a r g s ( a r g c , a r g v ) ;
/ / Q u e u e c l i e n t r e q u e s t s .
M E S S A G E _ Q U E U E m s g _ q u e u e ;
/ / S p a w n o f f N U M _ T H R E A D S t o r u n i n p a r a l l e l .
f o r ( i n t i = 0 ; i < N U M _ T H R E A D S ; i + + )
t h r _ c r e a t e ( 0 , 0 ,
T H R _ F U N C ( & w o r k e r ) ,
( v o i d * ) & m s g _ q u e u e ,
T H R _ B O U N D , 0 ) ;
/ / I n i t i a l i z e n e t w o r k d e v i c e a n d
/ / r e c v H T T P w o r k r e q u e s t s .
t h r _ c r e a t e ( 0 , 0 , T H R _ F U N C ( & r e c v _ r e q u e s t s ) ,
( v o i d * ) & m s g _ q u e u e ,
T H R _ B O U N D , 0 ) ;
/ / W a i t f o r a l l t h r e a d s t o e x i t ( B E W A R E ) !
w h i l e ( t h r _ j o i n ( 0 , & t _ i d , ( v o i d * * ) 0 ) = = 0 )
c o n t i n u e ; / / . . .
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P s e u d o - c o d e f o r r e c v _ r e q u e s t s ( )
v o i d r e c v _ r e q u e s t s ( M E S S A G E _ Q U E U E * m s g _ q u e u e )
{
i n i t i a l i z e s o c k e t a c c e p t o r a t p o r t 8 0
f o r e a c h ( i n c o m i n g r e q u e s t } )
{
u s e s e l e c t t o w a i t f o r n e w
c o n n e c t i o n s o r d a t a
i f ( c o n n e c t i o n )
e s t a b l i s h c o n n e c t i o n s u s i n g a c c e p t ( )
e l s e i f ( d a t a ) {
u s e s o c k e t s c a l l s t o
r e a d ( ) H T T P r e q u e s t s i n t o m s g
m s g _ q u e u e . e n q u e u e _ t a i l ( m s g ) ;
}
}
}
T h i s i s t h e “ s u p p l i e r ” t h r e a d
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Limitations with the Web Server
 The algorithmic decomposition tightly couples application-specific
functionality with various configuration-related characteristics, e.g.,
– The HTTP 1.0 protocol
– The number of services per process
– The time when services are configured into a process
 The solution is not portable since it hard-codes
– SunOS 5.x threading
– sockets and select()
 There are race conditions in the code
Vanderbilt University 159
Advanced ACE Tutorial Douglas C. Schmidt
Overcoming Limitations via OO
 The algorithmic decomposition illustrated above specifies too many
low-level details
– Moreover, the excessive coupling complicates reusability,
extensibility, and portability...
 In contrast, OO focuses on decoupling application-specific behavior
from reusable application-independent mechanisms
 The OO approach described below uses reusable framework
components and commonly recurring patterns
Vanderbilt University 160
Advanced ACE Tutorial Douglas C. Schmidt
Eliminating Race Conditions
 Problem
– A naive implementation of MESSAGE_QUEUE will lead to race
conditions
 e.g., when messages in different threads are enqueued and
dequeued concurrently
 Forces
– Producer/consumer concurrency is common, but requires careful
attention to avoid overhead, deadlock, and proper control
 Solution
– Utilize the Monitor Object pattern and condition variables
Vanderbilt University 161
Advanced ACE Tutorial Douglas C. Schmidt
The Monitor Object Pattern
Intent
 Synchronizes method execution to ensure
only one method runs within an object at a
time. It also allows an object’s methods to
cooperatively schedule their execution
sequences.
Monitor Object
+ synchronized_method_1()
...
+ synchronized_method_m()
# monitor_lock_
# monotor_condition_1_
...
# monitor_condition_n_
˜schmidt/POSA/
Forces Resolved
 Synchronization corresponds to methods
 Objects, not clients, are responsible for synchronization
 Cooperative method scheduling
Vanderbilt University 162
Advanced ACE Tutorial Douglas C. Schmidt
Overview of Condition Variables
 Condition variables (CVs) are used to “sleep/wait” until a particular
condition involving shared data is signaled
– CVs can wait on arbitrarily complex C++ expressions
– Sleeping is often more efficient than busy waiting...
 This allows more complex scheduling decisions, compared with a
mutex
– i.e., a mutex makes other threads wait, whereas a condition
variable allows a thread to make itself wait for a particular
condition involving shared data
Vanderbilt University 163
Advanced ACE Tutorial Douglas C. Schmidt
Condition Variable Usage Patterns
// Initially unlocked.
static ACE_Thread_Mutex lock;
static ACE_Condition_Thread_Mutex
cond (lock);
// synchronized
void acquire_resources (void) {
// Automatically acquire lock.
ACE_GUARD (ACE_Thread_Mutex, g, lock);
// Check condition in loop
while (condition expression false)
// Sleep.
cond.wait ();
// Atomically modify shared
// information.
// Destructor releases lock.
}
Note how the use of the Scoped
Locking idiom simplifies the
solution since we can’t forget to
release the lock!
// synchronized
void release_resources (void) {
// Automatically acquire lock.
ACE_GUARD (ACE_Thread_Mutex, g, lock);
// Atomically modify shared
// information...
cond.signal ();
// Could use cond.broadcast() here.
// guard automatically
// releases lock.
}
Vanderbilt University 164
Advanced ACE Tutorial Douglas C. Schmidt
ACE Condition Variable Interface
class ACE_Condition_Thread_Mutex
public:
// Initialize the CV.
ACE_Condition_Thread_Mutex
(const ACE_Thread_Mutex &);
// Implicitly destroy the CV.
˜ACE_Condition_Thread_Mutex (void);
// Block on condition, or until
// time passes. If time == 0 block.
int wait (ACE_Time_Value *time = 0);
// Signal one waiting thread.
int signal (void);
// Signal *all* waiting threads.
int broadcast (void) const;
private:
cond_t cond_; // Solaris CV.
const ACE_Thread_Mutex &mutex_;
};
The ACE_Condition_
Thread_Mutex class is
a wrapper for the native
OS condition variable
abstraction
 e.g., cond_t on
SunOS 5.x,
pthread_cond_t
for POSIX, and a
custom
implementation on
Windows and
VxWorks
Vanderbilt University 165
Advanced ACE Tutorial Douglas C. Schmidt
Overview of ACE_Message_Queue and
ACE_Message_Block
Message
Block
Message
Queue
head_
tail_
SYNCH
STRATEGY
Message
Block
next()
prev()
cont()Message
Block
next()
prev()
cont() Message
Block
next()
prev()
cont()
Data_Block
Data_Block
Data_Block
Data_Block
next()
prev()
cont()
 An ACE_Message_Queue is a list of
ACE_Message_Blocks
– Efficiently handles arbitrarily-large
message payloads
 An ACE_Message_Block is a
Composite
– Similar to BSD mbufs or SVR4
STREAMS m_blks
 Design parameterizes
synchronization and allocation
aspects
Vanderbilt University 166
A d v a n c e d A C E T u t o r i a l D o
T h e A C E _ M e s s a g e _ B l o c k C l a s s
#  b a s e _  :  c h a r  *
#  r e f c n t _  :  i n t
A C E _ D a t a _ B l o c k
A C E _ M e s s a g e _ B l o c k
+  i n i t  ( s i z e  :  s i z e _ t )  :  i n t
+  m s g _ t y p e  ( t y p e  :  A C E _ M e s s a g e _ T y p e )
+  m s g _ t y p e  ( )  :  A C E _ M e s s a g e _ T y p e
+  m s g _ p r i o r i t y  ( p r i o  :  u _ l o n g )
+  m s g _ p r i o r i t y  ( )  :  u _ l o n g
+  c l o n e  ( )  :  A C E _ M e s s a g e _ B l o c k  *
+  d u p l i c a t e  ( )  :  A C E _ M e s s a g e _ B l o c k  *
+  r e l e a s e  ( )  :  A C E _ M e s s a g e _ B l o c k  *
+  s e t _ f l a g s  ( f l a g s  :  u _ l o n g )  :  u _ l o n g
+  c l r _ f l a g s  ( f l a g s  :  u _ l o n g )  :  u _ l o n g
+  c o p y  ( b u f  :  c o n s t  c h a r  * , n  :  s i z e _ t )  :  i n t
+  r d _ p t r  ( n  :  s i z e _ t )
+  r d _ p t r  ( )  :  c h a r  *
+  w r _ p t r  ( n  :  s i z e _ t )
+  w r _ p t r  ( )  :  c h a r  *
+  l e n g t h  ( )  :  s i z e _ t
+  t o t a l _ l e n g t h  ( )  :  s i z e _ t
+  s i z e  ( )  :  s i z e _ t
#  r d _ p t r _  :  s i z e _ t
#  w r _ p t r _  :  s i z e _ t
#  c o n t _  :  A C E _ M e s s a g e _ B l o c k  *
#  n e x t _  :  A C E _ M e s s a g e _ B l o c k  *
#  p r e v _  :  A C E _ M e s s a g e _ B l o c k  *
#  d a t a _ b l o c k _  :  A C E _ D a t a _ B l o c k  *
*         1
C l a s s c h a r a c t e r i s t i c s
 H i d e m e s s a g i n g i m p l e m e n t a t i o n s f r o m c l i e n t s
A C E _ M e s s a g e
_ B l o c k
c o n t ( )
d a t a _ b l o c k ( )
w r _ p t r ( )
r d _ p t r ( )
PAYLOAD
A C E _ D a t a
_ B l o c k
A C E _ M e s s a g e
_ B l o c k
c o n t ( )
d a t a _ b l o c k ( )
w r _ p t r ( )
r d _ p t r ( )
A C E _ D a t a _ B l o c k
A C E _ M e s s a g e
_ B l o c k
c o n t ( )
d a t a _ b l o c k ( )
r d _ p t r ( )
w r _ p t r ( )
r e f e r e n c e _ c o u n t ( )  =  2
( 1 )  S I M P L E  M E S S A G E  S T R U C T U R E
( 2 )  C O M P O S I T E  M E S S A G E  S T R U C T U R E
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h e A C E _ M e s s a g e _ Q u e u e C l a s s
+  A C E _ M e s s a g e _ Q u e u e  ( h i g h _ w a t e r _ m a r k  :  s i z e _ t  =  D E F A U L T _ H W M ,
                     l o w _ w a t e r _ m a r k  :  s i z e _ t  =  D E F A U L T _ L W M ,
                     n o t i f y  :  A C E _ N o t i f i c a t i o n _ S t r a t e g y  *  =  0 )
+  open (high_water_mark : size_t = DEFAULT_HWM,
        low_water_mark : size_t = DEFAULT_LWM,
        notify : ACE_Notification_Strategy * = 0) : int
+  flush () : int
+  notification_strategy (s : ACE_Notification_Strategy *) : void
+  is_empty () : int
+  is_full () : int
+  enqueue_tail (item : ACE_Message_Block *,
                timeout : ACE_Time_Value * = 0) : int
+  enqueue_head (item : ACE_Message_Block *,
                timeout : ACE_Time_Value * = 0) : int
+  enqueue_prio (item : ACE_Message_Block *,
                timeout : ACE_Time_Value * = 0) : int
+  dequeue_head (item : ACE_Message_Block *&,
                timeout : ACE_Time_Value * = 0) : int
+  dequeue_tail (item : ACE_Message_Block *&,
                timeout : ACE_Time_Value * = 0) : int
+  high_water_mark (new_hwm : size_t) : void
+  high_water_mark (void) : size_t
+  low_water_mark (new_lwm : size_t) : void
+  low_water_mark (void) : size_t
+  close () : int
+  deactivate () : int
+  activate () : int
+  pulse () : int
+  state () : int
#  h e a d _  :  A C E _ M e s s a g e _ B l o c k  *
#  t a i l _  :  A C E _ M e s s a g e _ B l o c k  *
#  h i g h _ w a t e r _ m a r k _  :  s i z e _ t
#  l o w _ w a t e r _ m a r k _  :  s i z e _ t
A C E _ M e s s a g e _ Q u e u e
S Y N C H _ S T R A T E G Y
C l a s s c h a r a c t e r i s t i c s
 N o t e h o w t h e s y n c h r o n i z a t i o n a s p e c t c a n b e
s t r a t e g i z e d !
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h e A C E _ M e s s a g e _ Q u e u e P u b l i c
I n t e r f a c e
t e m p l a t e < c l a s s S Y N C H _ S T R A T = A C E _ M T _ S Y N C H >
/ / S y n c h r o n i z a t i o n a s p e c t
c l a s s A C E _ M e s s a g e _ Q u e u e
{
p u b l i c :
/ / D e f a u l t h i g h a n d l o w w a t e r m a r k s .
e n u m {
D E F A U L T _ L W M = 0 ,
D E F A U L T _ H W M = 4 0 9 6
} ;
/ / I n i t i a l i z e a M e s s a g e _ Q u e u e .
M e s s a g e _ Q u e u e ( s i z e _ t h w m = D E F A U L T _ H W M ,
s i z e _ t l w m = D E F A U L T _ L W M ) ;
/ / C h e c k i f f u l l o r e m p t y ( h o l d l o c k s )
i n t i s _ e m p t y ( v o i d ) c o n s t ;
i n t i s _ f u l l ( v o i d ) c o n s t ;
/ / E n q u e u e a n d d e q u e u e M e s s a g e _ B l o c k * ’ s .
i n t e n q u e u e _ p r i o ( A C E _ M e s s a g e _ B l o c k * , A C E _ T i m e _ V a l u e * ) ;
i n t e n q u e u e _ t a i l ( A C E _ M e s s a g e _ B l o c k * , A C E _ T i m e _ V a l u e * ) ;
i n t d e q u e u e _ h e a d ( A C E _ M e s s a g e _ B l o c k * & , A C E _ T i m e _ V a l u e * ) ;
i n t d e q u e u e _ t a i l ( A C E _ M e s s a g e _ B l o c k * & , A C E _ T i m e _ V a l u e * ) ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Parameterizing
Synchronization Strategies
 Q: What is ACE_MT_SYNCH and
how does it work?
 A: ACE_MT_SYNCH provides a
thread-safe synchronization strategy
for a ACE_Svc_Handler
– e.g., it ensures that an
ACE_Svc_Handler’s
ACE_Message_Queue is
thread-safe
– Any ACE_Task that accesses
shared state can use the
ACE_MT_SYNCH traits
Note the use of traits:
struct ACE_MT_SYNCH {
typedef ACE_Thread_Mutex
MUTEX;
typedef
ACE_Condition_Thread_Mutex
COND;
};
struct ACE_NULL_SYNCH {
typedef ACE_Null_Mutex
MUTEX;
typedef
ACE_Null_Condition COND;
};
Vanderbilt University 170
A d v a n c e d A C E T u t o r i a l D o
A C E _ M e s s a g e _ Q u e u e C l a s s P r i v a t e
I n t e r f a c e
p r i v a t e :
/ / C h e c k b o u n d a r y c o n d i t i o n s & d o n ’ t h o l d l o c k s .
i n t i s _ e m p t y _ i ( v o i d ) c o n s t ;
i n t i s _ f u l l _ i ( v o i d ) c o n s t ;
/ / R o u t i n e s t h a t a c t u a l l y d o t h e e n q u e u e i n g
/ / a n d d e q u e u e i n g a n d d o n ’ t h o l d l o c k s .
i n t e n q u e u e _ p r i o _ i ( A C E _ M e s s a g e _ B l o c k * ) ;
i n t e n q u e u e _ t a i l _ i ( A C E _ M e s s a g e _ B l o c k * ) ;
i n t d e q u e u e _ h e a d _ i ( A C E _ M e s s a g e _ B l o c k * & ) ;
i n t d e q u e u e _ t a i l _ i ( A C E _ M e s s a g e _ B l o c k * & ) ;
/ / . . .
/ / P a r a m e t e r i z e d t y p e s f o r s y n c h r o n i z a t i o n
/ / p r i m i t i v e s t h a t c o n t r o l c o n c u r r e n t a c c e s s .
/ / N o t e u s e o f C + + t r a i t s
t y p e n a m e S Y N C H _ S T R A T : : M U T E X l o c k _ ;
t y p e n a m e S Y N C H _ S T R A T : : C O N D n o t _ e m p t y _ c o n d _ ;
t y p e n a m e S Y N C H _ S T R A T : : C O N D n o t _ f u l l _ c o n d _ ;
s i z e _ t h i g h _ w a t e r _ m a r k _ ;
s i z e _ t l o w _ w a t e r _ m a r k _ ;
s i z e _ t c u r _ b y t e s _ ;
s i z e _ t c u r _ c o u n t _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Tips for Intra-class Locking
 Q: How should locking be performed in an OO class?
 A: Apply the Thread-Safe Interface pattern:
– “Interface functions should lock and do no work – implementation
functions should do the work and not lock ”
 This pattern helps to avoid intra-class method deadlock
– This is actually a variant on a common OO pattern that “public
functions should check, private functions should trust”
 Naturally, there are exceptions to this rule...
– This pattern avoids the following surprises
 Unnecessary overhead from recursive mutexes
 Deadlock if recursive mutexes aren’t used
 www.cs.wustl.edu/˜schmidt/POSA/
Vanderbilt University 172
A d v a n c e d A C E T u t o r i a l D o
A C E _ M e s s a g e _ Q u e u e C l a s s
I m p l e m e n t a t i o n
t e m p l a t e < c l a s s S Y N C H _ S T R A T >
A C E _ M e s s a g e _ Q u e u e < S Y N C H _ S T R A T > : : A C E _ M e s s a g e _ Q u e u e
( s i z e _ t h w m , s i z e _ t l w m )
: n o t _ e m p t y _ c o n d _ ( l o c k _ ) , n o t _ f u l l _ c o n d _ ( l o c k _ ) ,
. . . { }
t e m p l a t e < c l a s s S Y N C H _ S T R A T > i n t
A C E _ M e s s a g e _ Q u e u e < S Y N C H _ S T R A T > : : i s _ e m p t y _ i ( v o i d ) c o n s t
{ r e t u r n c u r _ b y t e s _ = = 0 & & c u r _ c o u n t _ = = 0 ; }
t e m p l a t e < c l a s s S Y N C H _ S T R A T > i n t
A C E _ M e s s a g e _ Q u e u e < S Y N C H _ S T R A T > : : i s _ f u l l _ i ( v o i d ) c o n s t
{ r e t u r n c u r _ b y t e s _ > h i g h _ w a t e r _ m a r k _ ; }
t e m p l a t e < c l a s s S Y N C H _ S T R A T > i n t
A C E _ M e s s a g e _ Q u e u e < S Y N C H _ S T R A T > : : i s _ e m p t y ( v o i d ) c o n s t
{
A C E _ G U A R D _ R E T U R N ( S Y N C H _ S T R A T : : M U T E X , g , l o c k _ , - 1 ) ;
r e t u r n i s _ e m p t y _ i ( ) ;
}
t e m p l a t e < c l a s s S Y N C H _ S T R A T > i n t
A C E _ M e s s a g e _ Q u e u e < S Y N C H _ S T R A T > : : i s _ f u l l ( v o i d ) c o n s t
{
A C E _ G U A R D _ R E T U R N ( S Y N C H _ S T R A T : : M U T E X , g , l o c k _ , - 1 ) ;
r e t u r n i s _ f u l l _ i ( ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
ACE_Message_Queue Operations
template  int
ACE_Message_Queue::
enqueue_tail (ACE_Message_Block *item,
ACE_Time_Value *tv) {
ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX,
guard, lock_, -1);
// Wait while the queue is full.
while (is_full_i ()) {
// Release the  and wait
// for timeout, signal, or space
// to become available in the list.
if (not_full_cond_.wait (tv) == -1)
return -1;
}
// Actually enqueue the message at
// the end of the list.
enqueue_tail_i (item);
// Tell blocked threads that
// list has a new item!
not_empty_cond_.signal ();
}
template  int
ACE_Message_Queue::
dequeue_head (ACE_Message_Block *&item,
ACE_Time_Value *tv) {
ACE_GUARD_RETURN (SYNCH_STRAT::MUTEX,
guard, lock_, -1);
// Wait while the queue is empty.
while (is_empty_i ()) {
// Release lock_ and wait for timeout,
// signal, or a new message being
// placed in the list.
if (not_empty_cond_.wait (tv) == -1)
return -1;
}
// Actually dequeue the first message.
dequeue_head_i (item);
// Tell blocked threads that list
// is no longer full.
if (cur_bytes_ <= low_water_mark_)
not_full_cond_.signal ();
}
Vanderbilt University 174
Advanced ACE Tutorial Douglas C. Schmidt
Overcoming Algorithmic Decomposition Limitations
 Previous slides illustrate tactical techniques and patterns that:
– Reduce accidental complexity e.g.,
 Automate synchronization acquisition and release (Scoped
Locking idiom)
 Improve synchronization mechanisms (Adapter, Wrapper
Facade, Monitor Object, Thread-Safe Interface, Strategized
Locking patterns)
– Eliminate race conditions
 Next, we describe strategic patterns, frameworks, and components
to:
– Increase reuse and extensibility e.g.,
 Decoupling service, IPC, and demultiplexing
– Improve the flexibility of concurrency control
Vanderbilt University 175
Advanced ACE Tutorial Douglas C. Schmidt
Selecting the Server’s Concurrency Architecture
 Problem
– A very strategic design decision for high-performance Web
servers is selecting an efficient concurrency architecture
 Forces
– No single concurrency architecture is optimal
– Key factors include OS/hardware platform and workload
 Solution
– Understand key alternative concurrency patterns
Vanderbilt University 176
Advanced ACE Tutorial Douglas C. Schmidt
Concurrency Patterns in the Web Server
 The following example illustrates the patterns and framework
components in an OO implementation of a concurrent Web Server
 There are various architectural patterns for structuring concurrency
in a Web Server
– Reactive
– Thread-per-request
– Thread-per-connection
– Synchronous Thread Pool
 Leader/Followers Thread Pool
 Half-Sync/Half-Async Thread Pool
– Asynchronous Thread Pool
Vanderbilt University 177
A d v a n c e d A C E T u t o r i a l D o
R e a c t i v e W e b S e r v e r
S E R V E R
C L I E N T
C L I E N T
C L I E N T
6 :  P R O C E S S   H T T P   R E Q U E S T
1 :  C O N N E C T
2 :  H A N D L E   I N P U T
3 :  C R E A T E   H A N D L E R
4 :  A C C E P T   C O N N E C T I O N
5 :  A C T I V A T E   H A N D L E R
 H T T P
H a n d l e r
 H T T P
H a n d l e r
 R e a c t o r
 H T T P
A c c e p t o r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h r e a d - p e r - R e q u e s t W e b S e r v e r
S E R V E R
C L I E N T
C L I E N T
C L I E N T
6 :  P R O C E S S   H T T P  R E Q U E S T
1 :  C O N N E C T
2 :  H A N D L E   I N P U T
3 :  C R E A T E   H A N D L E R
4 :  A C C E P T   C O N N E C T I O N
5 :  S P A W N   T H R E A D
H T T P
H a n d l e r
H T T P
H a n d l e r
H T T P
H a n d l e r
R e a c t o r
 H T T P
A c c e p t o r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h r e a d - p e r - C o n n e c t i o n W e b S e r v e r
S E R V E R
C L I E N T
C L I E N T
C L I E N T
 H T T P
A c c e p t o r
R e a c t o r
2 :  C R E A T E ,  A C C E P T ,
     A N D   A C T I V A T E
     H T T P _ H A N D L E R
1 :  H T T P
            R E Q U E S T
3 :  S P A W N  T H R E A D
    P E R   C O N N E C T I O N
4 :  P R O C E S S   H T T P   R E Q U E S T
 H T T P
H a n d l e r
 H T T P
H a n d l e r
 H T T P
H a n d l e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
L e a d e r / F o l l o w e r s S y n c h r o n o u s T h r e a d
P o o l W e b S e r v e r
4 :  P R O C E S S   H T T P   R E Q U E S T
S E R V E R
C L I E N T
C L I E N T
C L I E N T
2 :  A C C E P T   C O N N E C T I O N
3 :  M O R P H   I N T O   H A N D L E R
1 :  H T T P
              R E Q U E S T
E v e n t
D i s p a t c h e r
H T T P
H a n d l e r
H T T P
H a n d l e r
H T T P
H a n d l e r
H T T P
A c c e p t o r
 H T T P
A c c e p t o r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
H a l f - S y n c / H a l f - A s y n c S y n c h r o n o u s
T h r e a d P o o l W e b S e r v e r
S E R V E R
C L I E N T
C L I E N T
C L I E N T
4 :  D E Q U E U E  &
P R O C E S S
R E Q U E S T
M e s s a g e
Q u e u e
1 :  H T T P
             R E Q U E S T
5 :  P R O C E S S   H T T P   R E Q U E S T
 R e a c t o r
2 :  H A N D L E   I N P U T
3 :  E N Q U E U E   R E Q U E S T
 H T T P
H a n d l e r
 H T T P
A c c e p t o r
 H T T P
H a n d l e r
 H T T P
H a n d l e r
A c t i v e
O b j e c t
A c t i v e
O b j e c t
A c t i v e
O b j e c t
A c t i v e
O b j e c t
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A s y n c h r o n o u s T h r e a d P o o l W e b S e r v e r
7 :  P R O C E S S   H T T P   R E Q U E S T
S E R V E R
C L I E N T
C L I E N T
C L I E N T
6 :  D E Q U E U E   C O M P L E T I O N
&   P R O C E S S
R E Q U E S T
I / O
C o m p l e t i o n
P o r t
3 :  H T T P
       R E Q U E S T
1 :  I N I T I A T E   A S Y N C   A C C E P T
2 :  R U N   E V E N T   L O O P
4 :  A C C E P T   C O M P L E T E S
5 :  Q U E U E   C O M P L E T I O N
A s y n c
R e a d
A s y n c
R e a d
A s y n c
A c c e p t
A s y n c
A c c e p t
P r o a c t o r
H T T P
H a n d l e r
H T T P
H a n d l e r
H T T P
H a n d l e r
H T T P
H a n d l e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
W e b S e r v e r S o f t w a r e A r c h i t e c t u r e
H T T P
H a n d l e r
S o c k
S t r e a m
H T T P
H a n d l e r
S o c k
S t r e a m
H T T P
H a n d l e r
S o c k
S t r e a m
H T T P
A c c e p t o r
S o c k
A c c e p t o r
E v e n t
D i s p a t c h e r
 E v e n t D i s p a t c h e r
– E n c a p s u l a t e s W e b s e r v e r c o n c u r r e n c y a n d
d i s p a t c h i n g s t r a t e g i e s
 H T T P H a n d l e r s
– P a r s e s H T T P h e a d e r s a n d p r o c e s s e s
r e q u e s t s
 H T T P A c c e p t o r
– A c c e p t s c o n n e c t i o n s a n d c r e a t e s H T T P
H a n d l e r s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P a t t e r n s i n t h e W e b
S e r v e r I m p l e m e n t a t i o n
A c c e p t o r
C o n n e c t o r
T h r e a d
P o o l
T h r e a d - p e r
R e q u e s t
T h r e a d - p e r
C o n n e c t i o n
H a l f - S y n c /
H a l f - A s y n c
S t r a t e g y A d a p t e r
W r a p p e r
F a c a d e
S i n g l e t o n
A b s t r a c t
F a c t o r y
T A C T I C A L   P A T T E R N S
S T R A T E G I C   P A T T E R N S
D o u b l e
C h e c k e d
L o c k i n g
C o m p o n e n t
C o n f i g u r a t o r
R e a c t o r /
P r o a c t o r
A s y n c h r o n o u s
C o m p l e t i o n
T o k e n
A c t i v e
O b j e c t
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Patterns in the Web Client/Server (cont’d)
 The Web Client/Server uses same patterns as distributed logger
– i.e., Reactor, Component Configurator, Active Object, and
Acceptor
 It also contains patterns with the following intents:
– Connector ! “Decouple the active connection and initialization of
a peer service in a distributed system from the processing
performed once the peer service is connected and initialized”
– Double-Checked Locking Optimization ! “Allows atomic
initialization, regardless of initialization order, and eliminates
subsequent locking overhead”
– Half-Sync/Half-Async ! “Decouples synchronous I/O from
asynchronous I/O in a system to simplify concurrent programming
effort without degrading execution efficiency”
Vanderbilt University 186
A d v a n c e d A C E T u t o r i a l D o
A r c h i t e c t u r e o f O u r W e b S e r v e r
s v c _ r u n
R E Q U E S T  P R O C E S S I N G  L A Y E R
 O p t i o n s
s
 H T T P
H a n d l e r
 H T T P
H a n d l e r
 H T T P
H a n d l e r
 H T T P
A c c e p t o r
 R e a c t o r
 H T T P
P r o c e s s o r
 M s g
Q u e u e
s
s v c _ r u n
s v c _ r u n
s v c _ r u n
 Q U E U E I N G
 L A Y E R
 I / O  D E M U X I N G
 L A Y E R
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P D F / H P L . p d f
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A n I n t e g r a t e d R e a c t i v e / A c t i v e
W e b S e r v e r
 :  R e a c t o r
R E G I S T E R E D
O B J E C T S
F
R
A
M
E
W
O
R
K
L
E
V
E
L
K
E
R
N
E
L
L
E
V
E
L
A
P
P
L
IC
A
T
IO
N
L
E
V
E
L
O S   E V E N T   D E M U L T I P L E X I N G   I N T E R F A C E
1 :  h a n d l e _ i n p u t ( )
s v c _ r u n
4 :  g e t q ( m s g )
5 : s v c ( m s g )
2 :  r e c v _ r e q u e s t ( m s g )
3 :  p u t q ( m s g )
 H T T P
P r o c e s s o r
:  H a n d l e
T a b l e
s v c _ r u n
s v c _ r u n
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
A c c e p t o r
W e ’ r e f o c u s i n g o n t h e R e a c t i v e l a y e r h e r e
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
HTTP_Handler Public Interface
template 
class HTTP_Handler : public
ACE_Svc_Handler {
public:
// Entry point into ,
// called by .
virtual int open (void *)
{
// Register with 
// to handle input.
reactor ()->register_handler
(this, ACE_Event_Handler::READ_MASK);
// Register timeout in case client
// doesn’t send any HTTP requests.
reactor ()->schedule_timer
(this, 0, ACE_Time_Value (CLIENT_TIMEOUT));
}
The HTTP_Handler is
the Proxy for
communicating with
clients (e.g., Web
browsers like Netscape
or i.e.,)
 It implements the
asynchronous
portion of Half-
Sync/Half-Async
pattern
Vanderbilt University 189
Advanced ACE Tutorial Douglas C. Schmidt
HTTP_Handler Protected Interface
protected:
// Reactor dispatches this
// method when clients timeout.
virtual int handle_timeout
(const ACE_Time_Value &, const void *)
{
// Remove from the Reactor.
reactor ()->remove_handler
(this,
ACE_Event_Handler::READ_MASK);
}
// Reactor dispatches this method
// when HTTP requests arrive.
virtual int handle_input (ACE_HANDLE);
// Receive/frame client HTTP
// requests (e.g., GET).
int recv_request (ACE_Message_Block *&);
};
These methods are
invoked by callbacks
from ACE_Reactor
 Reactor
REGISTERED
OBJECTS
1: handle_timeout()
2: remove_handler(this)
 : Reactor: TimerQueue
 Event
Handler
 HTTP
Handler
Vanderbilt University 190
Advanced ACE Tutorial Douglas C. Schmidt
Integrating Multi-threading
 Problem
– Multi-threaded Web servers are needed since Reactive Web
servers are often inefficient and non-robust
 Forces
– Multi-threading can be very hard to program
– No single multi-threading model is always optimal
 Solution
– Use the Active Object pattern to allow multiple concurrent server
operations in an OO-manner
Vanderbilt University 191
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A c t i v e O b j e c t P a t t e r n a n d
A C E T a s k F r a m e w o r k i n t h e W e b S e r v e r
 :  R e a c t o r
R E G I S T E R E D
O B J E C T S
F
R
A
M
E
W
O
R
K
L
E
V
E
L
K
E
R
N
E
L
L
E
V
E
L
A
P
P
L
IC
A
T
IO
N
L
E
V
E
L
O S   E V E N T   D E M U L T I P L E X I N G   I N T E R F A C E
1 :  h a n d l e _ i n p u t ( )
s v c _ r u n
4 :  g e t q ( m s g )
5 : s v c ( m s g )
2 :  r e c v _ r e q u e s t ( m s g )
3 :  p u t q ( m s g )
 H T T P
P r o c e s s o r
:  H a n d l e
T a b l e
s v c _ r u n
s v c _ r u n
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
A c c e p t o r
W e ’ r e f o c u s i n g o n t h e A c t i v e O b j e c t l a y e r h e r e
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The HTTP_Processor Class
class HTTP_Processor
: public ACE_Task {
private: HTTP_Processor (void);
public:
// Singleton access point.
static HTTP_Processor *instance (void);
// Pass a request to the thread pool.
virtual int put (ACE_Message_Block *,
ACE_Time_Value *);
// Entry point into a pool thread.
virtual int svc (void)
{
ACE_Message_Block *mb = 0;
// Wait for messages to arrive.
for (;;) {
getq (mb); // Inherited from 
// Identify and perform HTTP
// Server request processing...
 Processes HTTP
requests using the
“Thread-Pool”
concurrency model
 This method
implements the
synchronous task
portion of the Half-
Sync/Half-Async
pattern
Vanderbilt University 193
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e S i n g l e t o n P a t t e r n
/ / S i n g l e t o n a c c e s s p o i n t .
H T T P _ P r o c e s s o r *
H T T P _ P r o c e s s o r : : i n s t a n c e ( v o i d )
{
/ / B e w a r e o f r a c e c o n d i t i o n s !
i f ( i n s t a n c e _ = = 0 )
/ / C r e a t e t h e S i n g l e t o n " o n - d e m a n d . "
i n s t a n c e _ = n e w H T T P _ P r o c e s s o r ;
r e t u r n i n s t a n c e _ ;
}
/ / C o n s t r u c t o r c r e a t e s t h e t h r e a d p o o l .
H T T P _ P r o c e s s o r : : H T T P _ P r o c e s s o r ( v o i d )
{
/ / I n h e r i t e d f r o m c l a s s T a s k .
a c t i v a t e ( T H R _ B O U N D ,
O p t i o n s : : i n s t a n c e ( ) - > t h r e a d s ( ) ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Subtle Concurrency Woes with the Singleton Pattern
 Problem
– The canonical Singleton implementation has subtle “bugs” in
multi-threaded applications
 Forces
– Too much locking makes Singleton too slow...
– Too little locking makes Singleton unsafe...
 Solution
– Use the Double-Checked Locking optimization pattern to minimize
locking and ensure atomic initialization
Vanderbilt University 195
Advanced ACE Tutorial Douglas C. Schmidt
The Double-Checked Locking Optimization Pattern
HTTP
Processor
static instance()
static instance_
if  (instance_ == NULL)  {
    mutex_.acquire ();
    if  (instance_  == NULL)
        instance_ = new HTTP_Processor;
    mutex_.release ();
}
return instance_;
Mutex
www.cs.wustl.edu/
˜schmidt/POSA/
Intent
 Allows atomic initialization,
regardless of initialization order,
and eliminates subsequent
locking overhead
Forces Resolved:
 Ensures atomic object
initialization
 Minimizes locking overhead
Caveat!
 This pattern assumes atomic
memory access
Vanderbilt University 196
Advanced ACE Tutorial Douglas C. Schmidt
The ACE Singleton Template
template 
class ACE_Singleton : public ACE_Cleanup {
public:
static TYPE *instance (void) {
// Memory barrier could go here...
if (s_ == 0) {
ACE_GUARD_RETURN (LOCK, g,
ACE_Object_Manager
::get_singleton_lock (), -1);
if (s_ == 0)
s_ = new ACE_Singleton;
// Memory barrier could go here.
ACE_Object_Manager::at_exit (s_);
}
return s_->instance_;
}
virtual void cleanup (void *param = 0);
protected:
ACE_Singleton (void);
TYPE instance_;
static ACE_Singleton *s_;
};
Features
 Turns any class into
a singleton
 Automates
Double-Checked
Locking Optimization
 Ensures automatic
cleanup when
process exits
www.cs.wustl.edu/
˜schmidt/PDF/
ObjMan.pdf
Vanderbilt University 197
Advanced ACE Tutorial Douglas C. Schmidt
Integrating Reactive and Multi-threaded Layers
 Problem
– Justifying the hybrid design of our Web server can be tricky
 Forces
– Engineers are never satisfied with the status quo ;-)
– Substantial amount of time is spent re-discovering the intent of
complex concurrent software design
 Solution
– Use the Half-Sync/Half-Async pattern to explain and justify our
Web server concurrency architecture
Vanderbilt University 198
Advanced ACE Tutorial Douglas C. Schmidt
The Half-Sync/Half-Async Pattern
Q
U
E
U
E
I
N
G
L
A
Y
E
R
A
S
Y
N
C
H
R
O
N
O
U
S
 
 
T
A
S
K
 
 
L
A
Y
E
R
S
Y
N
C
H
R
O
N
O
U
S
 
T
A
S
K
 
 
L
A
Y
E
R
SYNC
TASK 2
1, 4: read(data)
3: enqueue(data)
2: interrupt
MESSAGE  QUEUES
SYNC
TASK 1
SYNC
TASK 3
ASYNC
TASK
EXTERNAL
EVENT  SOURCES
Intent
 Decouples synchronous I/O
from asynchronous I/O in a
system to simplify concurrent
programming effort without
degrading execution efficiency
Forces Resolved:
 Simplify programming
 Ensure efficient I/O
www.cs.wustl.edu/
˜schmidt/POSA/
Vanderbilt University 199
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e H a l f - S y n c / H a l f - A s y n c
P a t t e r n i n t h e W e b S e r v e r
AS
YN
C 
 T
AS
K
LE
VE
L
SY
N
C 
 
TA
SK
LE
VE
L
QU
EU
EI
NG
LE
VE
L
 :  R e a c t o r
1 :  h a n d l e _ i n p u t ( )
2 :  r e c v _ r e q u e s t ( m s g )
3 :  p u t q ( m s g )
:  H a n d l e
T a b l e
 E v e n t
H a n d l e r
 H T T P
A c c e p t o r
s v c _ r u n
4 :  g e t q ( m s g )
5 : s v c ( m s g )
 H T T P
P r o c e s s o r
s v c _ r u n
s v c _ r u n
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
 E v e n t
H a n d l e r
 H T T P
H a n d l e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
J o i n i n g A s y n c a n d S y n c T a s k s
i n t h e W e b S e r v e r
/ / T h e f o l l o w i n g m e t h o d s f o r m t h e b o u n d a r y
/ / b e t w e e n t h e A s y n c a n d S y n c l a y e r s .
t e m p l a t e < c l a s s P A > i n t
H T T P _ H a n d l e r < P A > : : h a n d l e _ i n p u t ( A C E _ H A N D L E h )
{
A C E _ M e s s a g e _ B l o c k * m b = 0 ;
/ / T r y t o r e c e i v e a n d f r a m e m e s s a g e .
i f ( r e c v _ r e q u e s t ( m b ) = = H T T P _ R E Q U E S T _ C O M P L E T E ) {
r e a c t o r ( ) - > r e m o v e _ h a n d l e r
( t h i s , A C E _ E v e n t _ H a n d l e r : : R E A D _ M A S K ) ;
r e a c t o r ( ) - > c a n c e l _ t i m e r ( t h i s ) ;
/ / I n s e r t m e s s a g e i n t o t h e Q u e u e .
H T T P _ P r o c e s s o r < P A > : : i n s t a n c e ( ) - > p u t ( m b ) ;
}
}
i n t H T T P _ P r o c e s s o r : : p u t ( A C E _ M e s s a g e _ B l o c k * m s g ,
A C E _ T i m e _ V a l u e * t i m e o u t )
{
/ / I n s e r t t h e m e s s a g e o n t h e M e s s a g e _ Q u e u e
/ / ( i n h e r i t e d f r o m c l a s s T a s k ) .
p u t q ( m s g , t i m e o u t ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Optimizing Our Web Server for
Asynchronous Operating Systems
 Problem
– Synchronous multi-threaded solutions are not always the most
efficient
 Forces
– Purely asynchronous I/O is quite powerful on some OS platforms
 e.g., Windows NT 4.x or UNIX with aio_()* calls
– Good designs should be adaptable to new contexts
 Solution
– Use the Proactor pattern to maximize performance on
Asynchronous OS platforms
Vanderbilt University 202
Advanced ACE Tutorial Douglas C. Schmidt
The Proactor Pattern
Intent
 Demultiplexes and
dispatches service
requests that are
triggered by the
completion of
asynchronous
operations
Resolves same forces
as Reactor
HTTP
Acceptor
Proactor
handle_events()
handle_accept()
handle_read_file()
handle_write_file()
handle_timeout()
get_handle()
Timer_Queue
schedule_timer(h)
cancel_timer(h)
1
overlapped_result =
  GetQueuedCompleteStatus();
overlapped_result->complete()
1
APPLICATION-
DEPENDENT
APPLICATION-
INDEPENDENT
n
A
Async
Op
open()
cancel()
Completion
Handler
HTTP
Handler
Async
Write
Async
Accept
Handles
www.cs.wustl.edu/˜schmidt/POSA/
Vanderbilt University 203
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e o f t h e A C E P r o a c t o r
F r a m e w o r k
A C E _ A s y n c h _ R e a d _ S t r e a m A C E _ A s y n c h _ W r i t e _ S t r e a m
A C E _ P r o a c t o r
ACE_Handler
ACE_Service_Handler
A C E _ A s y n c h _ A c c e p t o r
A C E _ A s y n c h _ R e s u l t
ACE_Timer_Queue
A C E _ A s y n c h _ C o n n e c t o r
F r a m e w o r k c h a r a c t e r i s t i c s
 S i m i l a r t o t h e A C E R e a c t o r f r a m e w o r k , e x c e p t
b e h a v i o r i s “ i n v e r s e ”
 P o r t a b l e t o W i n d o w s a n d v a r i o u s U N I X
p l a t f o r m s t h a t s u p p o r t a i o _ * ( ) f a m i l y o f
m e t h o d s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E P r o a c t o r F r a m e w o r k f o r
t h e W e b S e r v e r
C o n n e c t i o n S e t u p P h a s e
4 :  c o n n e c t
W e b  S e r v e r
W e b
B r o w s e r
A c c e p t o r
C o m p l e t i o n
D i s p a t c h e r
H T T P
H a n d l e r
O p e r a t i n g
S y s t e m
3 :  h a n d l e
e v e n t s
5 :  a c c e p t
c o m p l e t e
1 :  a c c e p t
c o n n e c t i o n s
2 :  a c c e p t
( A c c e p t o r ,
D i s p a t c h e r )
6 :
a c c e p t
c o m p l e t e
7 :  c r e a t e
8 :  r e a d  ( H a n d l e r ,
D i s p a t c h e r )
D a t a T r a n s f e r P h a s e
W e b  S e r v e r
W e b
B r o w s e r
F i l e
S y s t e m
C o m p l e t i o n
D i s p a t c h e r
H T T P
H a n d l e r
O p e r a t i n g
S y s t e m
1 :  G E T
/ e t c / p a s s w d
2 :  r e a d  c o m p l e t e
3 :  r e a d
c o m p l e t e
4 :  p a r s e  r e q u e s t
6 :  w r i t e  ( F i l e ,  C o n n . ,
H a n d l e r ,  D i s p a t c h e r )
7 :  w r i t e
c o m p l e t e
8 :  w r i t e
c o m p l e t e
5 :  r e a d  ( F i l e )
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Structuring Service Initialization
 Problem
– The communication protocol used between clients and the Web
server is often orthogonal to the initialization protocol
 Forces
– Low-level connection establishment APIs are tedious, error-prone,
and non-portable
– Separating initialization from use can increase software reuse
substantially
 Solution
– Use the Acceptor and Connector patterns to decouple passive
service initialization from run-time protocol
Vanderbilt University 206
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E _ A c c e p t o r
i n t h e W e b S e r v e r
P A S S I V E
L I S T E N E R
A C T I V E
C O N N E C T I O N S
1 :  h a n d l e _ i n p u t ( )
2 :  s h  =  m a k e _ s v c _ h a n d l e r ( )
3 :  a c c e p t _ s v c _ h a n d l e r ( s h )
4 :  a c t i v a t e _ s v c _ h a n d l e r ( s h )
:  R e a c t o r
:  A c c e p t o r
:  H T T P
A c c e p t o r
:  S v c
H a n d l e r
:  H T T P
H a n d l e r
:  S v c
H a n d l e r
:  H T T P
H a n d l e r
:  S v c
H a n d l e r
:  H T T P
H a n d l e r
:  S v c
H a n d l e r
:  H T T P
H a n d l e r
T h e H T T P _ A c c e p t o r i s a factory t h a t creates,
connects, a n d activates a n H T T P _ H a n d l e r
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
HTTP_Acceptor Class Interface
template 
class HTTP_Acceptor :
public ACE_Acceptor,
// Note use of a "trait".
ACCEPTOR>
{
public:
// Called when  is
// dynamically linked.
virtual int init (int argc, char *argv[]);
// Called when  is
// dynamically unlinked.
virtual int fini (void);
// ...
};
The HTTP_Acceptor
class implements the
Acceptor role
 i.e., it accepts
connections/initializes
HTTP_Handlers
Vanderbilt University 208
A d v a n c e d A C E T u t o r i a l D o
H T T P _ A c c e p t o r C l a s s I m p l e m e n t a t i o n
/ / I n i t i a l i z e s e r v i c e w h e n d y n a m i c a l l y l i n k e d .
t e m p l a t e < c l a s s P A > i n t
H T T P _ A c c e p t o r < P A > : : i n i t ( i n t a r g c , c h a r * a r g v [ ] )
{
O p t i o n s : : i n s t a n c e ( ) - > p a r s e _ a r g s ( a r g c , a r g v ) ;
/ / I n i t i a l i z e t h e c o m m u n i c a t i o n e n d p o i n t a n d
/ / r e g i s t e r t o a c c e p t c o n n e c t i o n s .
p e e r _ a c c e p t o r ( ) . o p e n ( t y p e n a m e
P A : : P E E R _ A D D R ( O p t i o n s : : i n s t a n c e ( ) - > p o r t ( ) ) ,
R e a c t o r : : i n s t a n c e ( ) ) ;
}
/ / T e r m i n a t e s e r v i c e w h e n d y n a m i c a l l y u n l i n k e d .
t e m p l a t e < c l a s s P A > i n t
H T T P _ A c c e p t o r < P A > : : f i n i ( v o i d )
{
/ / S h u t d o w n t h r e a d s i n t h e p o o l .
H T T P _ P r o c e s s o r < P A > : : i n s t a n c e ( ) - >
m s g _ q u e u e ( ) - > d e a c t i v a t e ( ) ;
/ / W a i t f o r a l l t h r e a d s t o e x i t .
H T T P _ P r o c e s s o r < P A > : : i n s t a n c e ( ) - >
t h r _ m g r ( ) - > w a i t ( ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Using the ACE Service Configurator
Framework in the Web Server
SERVICE
CONFIGURATOR
RUNTIME
 Service
Repository
Service
Object
 TPR
Web
Server
DLLS
Service
Object
 Reactive
Web
Server
svc.conf
FILE
Service
Object
TP Web
Server
Reactor
 Service
Config
dynamic Web_Server Service_Object *
   web_server:make_Web_Server() "-port 80"
Vanderbilt University 210
Advanced ACE Tutorial Douglas C. Schmidt
Component Configurator Implementation in C++
The concurrent Web Server is
configured and initialized via a
configuration script
% cat ./svc.conf
dynamic Web_Server
Service_Object *
web_server:_make_Web_Server()
"-p 80 -t $THREADS"
# .dll or .so suffix added to
# "web_server" automatically
Factory function that dynamically
allocates a Half-Sync/Half-Async
Web Server object
extern "C" ACE_Service_Object *
make_Web_Server (void);
ACE_Service_Object *
make_Web_Server (void)
{
return new
HTTP_Acceptor;
// ACE dynamically unlinks and
// deallocates this object.
}
Vanderbilt University 211
Advanced ACE Tutorial Douglas C. Schmidt
Main Program for the Web Server
int main (int argc, char *argv[])
{
// Initialize the daemon and
// dynamically configure services.
ACE_Service_Config::open (argc,
argv);
// Loop forever, running services
// and handling reconfigurations.
ACE_Reactor::instance ()->
run_reactor_event_loop ();
/* NOTREACHED */
}
 The main()
function is totally
generic!
 Dynamically
configure & execute
Web Server
 Make any
application
“Web-enabled”
Vanderbilt University 212
Advanced ACE Tutorial Douglas C. Schmidt
Optimizing the JAWS Framework
Protocol
Filter
Handler
Protocol
Framework
Strategy
Concurrency
Protocol Pipeline
Framework
Framework
I/O Strategy
Filesystem
Cached Virtual
Expander
Tilde
~
/home/...
Event Dispatcher
A
cceptor
A
c
t
i
v
e
 
O
b
j
e
c
t
Asynchronous Completion Token
Reactor/Proactor Singleton
Adapter
Streams
Strategy
Service Configurator
State
S
t
r
a
t
e
g
y
www.cs.wustl.edu/˜jxh/research/
 Use lightweight
concurrency
 Minimize locking
 Apply file caching
and memory
mapping
 Use “gather-write”
mechanisms
 Minimize logging
 Pre-compute HTTP
responses
 Avoid excessive
time() calls
 Optimize the
transport interface
Vanderbilt University 213
Advanced ACE Tutorial Douglas C. Schmidt
Application-level Telecom Gateway Example
WIDE  AREA
NETWORK
SATELLITES
TRACKING
STATION
PEERS
STATUS  INFO
COMMANDS BULK  DATA
TRANSFER
LOCAL  AREA  NETWORK
GROUND
STATION
PEERS
GATEWAY
 This example explores the
patterns and reusable
framework components
for an application-level
Gateway
 The Gateway routes
messages between Peers
 Gateway and Peers are
connected via TCP/IP
Vanderbilt University 214
A d v a n c e d A C E T u t o r i a l D o
O O S o f t w a r e A r c h i t e c t u r e
o f t h e G a t e w a y
C O N N E C T I O N
R E Q U E S T
C O N N E C T I O N
R E Q U E S T
O U T G O I N G
M E S S A G E S
I N C O M I N G
M E S S A G E S
G A T E W A Y
 R o u t i n g
T a b l e
 A c c e p t o r
 C o n n e c t o r
 S u p p l i e r
H a n d l e r
C o n s u m e r
H a n d l e r
 S u p p l i e r
H a n d l e r
R e a c t o r
C o n s u m e r
H a n d l e r
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P D F /
T A P O S - 0 0 . p d f
A l l c o m p o n e n t s i n t h i s a r c h i t e c t u r e a r e b a s e d o n
p a t t e r n s f r o m A C E
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Gateway Behavior
 Components in the Gateway behave as follows:
1. Gateway parses configuration files that specify which Peers to
connect with and which routes to use
2. Proxy_Handler_Connector connects to Peers, then creates
and activates Proxy_Handler subclasses
(Supplier_Handler or Consumer_Handler)
3. Once connected, Peers send messages to the Gateway
– Messages are handled by an Supplier_Handler
– Supplier_Handlers work as follows:
 Receive and validate messages
 Consult a Routing_Table
 Forward messages to the appropriate Peer(s) via
Consumer_Handlers
Vanderbilt University 216
A d v a n c e d A C E T u t o r i a l D o
P a t t e r n s i n t h e G a t e w a y
A c c e p t o r -
C o n n e c t o r
C o m p o n e n t
C o n f i g u r a t o r
N o n - b l o c k i n g
B u f f e r e d  I / O
H a l f - S y n c /
H a l f - A s y n c
T A C T I C A L
P A T T E R N S
S T R A T E G I C
P A T T E R N S
R e a c t o r
A c t i v e  O b j e c t
T e m p l a t e
M e t h o d
I t e r a t o r
F a c t o r y
M e t h o d
P r o x y
W r a p p e r
F a c a d e
T h e G a t e w a y c o m p o n e n t s a r e b a s e d u p o n a
c o m m o n pattern language
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C l a s s D i a g r a m f o r S i n g l e - T h r e a d e d
G a t e w a y
P r o x y
H a n d l e r
C o n n e c t o r
P r o x y _ H a n d l e r
S O C K _ C o n n e c t o r
S O C K _ S t r e a m
N u l l _ S y n c h
S v c
H a n d l e r
C o n n e c t o r
S V C _ H A N D L E R
P E E R _ C O N N E C T O R
P E E R _ S T R E A M
S Y N C H
C
O
N
N
E
C
T
IO
N
-
O
R
IE
N
T
E
D
C
O
M
P
O
N
E
N
T
S
A
P
P
L
IC
A
T
IO
N
-
S
P
E
C
IF
IC
C
O
M
P
O
N
E
N
T
S
A
C
E
F
R
A
M
E
W
O
R
K
C
O
M
P
O
N
E
N
T
S
S t r e a m
S e r v i c e
C o n f i g u r a t o r
C o n c u r r e n c y
global
C o n n e c t i o n
R e a c t o r
1
< < a c t i v a t e s > >
S u p p l i e r / C o n s u m e r
H a n d l e r
n
I P C _ S A P
P E E R
C O N N E C T O R
P E E R
S T R E A M
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
OO Gateway Architecture
 Application-specific components
– Proxy_Handlers route messages among Peers
 Connection-oriented application components
– ACE_Svc_Handler
 Performs I/O-related tasks with connected clients
– ACE_Connector factory
 Establishes new connections with clients
 Dynamically creates an ACE_Svc_Handler object for each
client and “activates” it
 Application-independent ACE framework components
– Perform IPC, explicit dynamic linking, event demultiplexing, event
handler dispatching, multi-threading, etc.
Vanderbilt University 219
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E R e a c t o r
F r a m e w o r k f o r t h e G a t e w a y
C O N C R E T E   E V E N T
H A N D L E R S
F
R
A
M
E
W
O
R
K
L
E
V
E
L
K
E
R
N
E
L
L
E
V
E
L
A
P
P
L
IC
A
T
IO
N
L
E
V
E
L
4 :  s e n d ( m s g )
2 :  r e c v ( m s g )
3 :  r o u t e ( m s g )
O S   E V E N T   D E M U L T I P L E X I N G   I N T E R F A C E
T i m e r
Q u e u e
 :  R e a c t o r
1 :  h a n d l e _ i n p u t ( )
 H a n d l e
T a b l e
 E v e n t
H a n d l e r
 C o n s u m e r
H a n d l e r
 E v e n t
H a n d l e r
 C o n s u m e r
H a n d l e r
 E v e n t
H a n d l e r
 S u p p l i e r
H a n d l e r
B e n e fi t s
 S t r a i g h t f o r w a r d t o
p r o g r a m
 C o n c u r r e n c y c o n t r o l
i s t r i v i a l
L i a b i l i t i e s
 D e s i g n i s “ b r i t t l e ”
 C a n ’ t l e v e r a g e
m u l t i - p r o c e s s o r s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Addressing Active Endpoint Connection
and Initialization Challenges
 Problem
– Application communication protocols are often orthogonal to their
connection establishment and service initialization protocols
 Forces
– Low-level connection APIs are error-prone and non-portable
– Separating initialization from processing increases software reuse
– Asynchronous connections are important over long-delay paths
 Solution
– Use the Acceptor-Connector pattern to decouple connection and
initialization protocols from the Gateway routing protocol
Vanderbilt University 221
A d v a n c e d A C E T u t o r i a l D o
T h e A c c e p t o r - C o n n e c t o r P a t t e r n
( C o n n e c t o r R o l e )
S v c
H a n d l e r
p e e r _ s t r e a m _
o p e n ( )
C o n n e c t o r
c o n n e c t ( s h ,  a d d r )
c o m p l e t e ( )
S v c  H a n d l e r
R e a c t o r
A P P L I C A T I O N -
D E F I N E D
A P P L I C A T I O N -
I N D E P E N D E N T
A C T I V A T E S
H A N D L E   A S Y N C
C O N N E C T I O N   C O M P L E T I O N
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P O S A /
I n t e n t o f C o n n e c t o r R o l e
 Decouple the active
connection and
initialization of a peer
service in a distributed
system from the
processing performed
once the peer service is
connected and initialized
F o r c e s R e s o l v e d :
 R e u s e c o n n e c t i o n
c o d e
 E f fi c i e n t l y s e t u p
c o n n e c t i o n s w i t h
m a n y p e e r s o r o v e r
l o n g d e l a y p a t h s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Structure of the Acceptor-Connector Pattern in ACE
ACE_Svc_Handler
PEER_STREAM,
SYNCH_STRATEGY
ACE_Acceptor
SVC_HANDLER,
PEER_ACCEPTOR
ACE_Connector
SVC_HANDLER,
PEER_CONNECTOR
Application
Service
«bind»
ACE_Event_Handler
ACE_Task
SYNCH_STRATEGY
Additional features of the
ACE_Connector
 Uses C++ parameterized
types to strategize IPC and
service aspects
 Uses Template Method
pattern to strategize
creation, connection
establishment, and
concurrency policies
Vanderbilt University 223
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E _ C o n n e c t o r
i n t h e G a t e w a y
P E N D I N G
C O N N E C T I O N S
A C T I V E
C O N N E C T I O N S
 S v c
H a n d l e r
 S v c
H a n d l e r
 S v c
H a n d l e r
H a s h _ M a p
 C o n n e c t o r
 R e a c t o r
 S v c
H a n d l e r
 S v c
H a n d l e r
 S v c
H a n d l e r
 T h e A C E _ C o n n e c t o r i s a factory
– i.e., i t connects a n d activates a n
A C E _ S v c _ H a n d l e r
 T h e r e ’ s t y p i c a l l y 1 A C E _ C o n n e c t o r p e r - s e r v i c e
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ C o n n e c t o r C l a s s P u b l i c I n t e r f a c e
A r e u s a b l e t e m p l a t e f a c t o r y c l a s s t h a t e s t a b l i s h e s
c o n n e c t i o n s w i t h c l i e n t s
t e m p l a t e < c l a s s S V C _ H A N D L E R ,
/ / T y p e o f s e r v i c e
c l a s s P E E R _ C O N N E C T O R >
/ / C o n n e c t i o n f a c t o r y
c l a s s A C E _ C o n n e c t o r : p u b l i c A C E _ S e r v i c e _ O b j e c t
{
p u b l i c :
/ / I n i t i a t e c o n n e c t i o n t o P e e r .
v i r t u a l i n t c o n n e c t
( S V C _ H A N D L E R * & s v c _ h a n d l e r ,
t y p e n a m e c o n s t P E E R _ C O N N E C T O R : : P E E R _ A D D R & r a ,
A C E _ S y n c h _ O p t i o n s & s y n c h _ o p t i o n s ) ;
/ / C a n c e l a < s v c _ h a n d l e r > t h a t w a s
/ / s t a r t e d a s y n c h r o n o u s l y .
v i r t u a l i n t c a n c e l ( S V C _ H A N D L E R * s v c _ h a n d l e r ) ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
D e s i g n I n t e r l u d e : M o t i v a t i o n f o r t h e
A C E _ S y n c h _ O p t i o n s C l a s s
 Q : What is the ACE_Synch_Options class?
 A : T h i s a l l o w s c a l l e r s t o d e fi n e t h e
s y n c h r o n y / a s y n c h r o n y p o l i c i e s , e.g.,
c l a s s A C E _ S y n c h _ O p t i o n s {
/ / O p t i o n s f l a g s f o r c o n t r o l l i n g
/ / s y n c h r o n i z a t i o n .
e n u m { U S E _ R E A C T O R = 1 , U S E _ T I M E O U T = 2 } ;
A C E _ S y n c h _ O p t i o n s
( u _ l o n g o p t i o n s = 0 ,
c o n s t A C E _ T i m e _ V a l u e & t i m e o u t
= A C E _ T i m e _ V a l u e : : z e r o ,
c o n s t v o i d * a c t = 0 ) ;
/ / T h i s i s t h e d e f a u l t s y n c h r o n o u s s e t t i n g .
s t a t i c A C E _ S y n c h _ O p t i o n s s y n c h ;
/ / T h i s i s t h e d e f a u l t a s y n c h r o n o u s s e t t i n g .
s t a t i c A C E _ S y n c h _ O p t i o n s a s y n c h ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ S y n c h _ O p t i o n s a n d
A C E _ C o n n e c t o r S e m a n t i c s
R e a c t o r T i m e o u t B e h a v i o r
Ye s 0 , 0 R e t u r n  1 w i t h e r r n o
E W O U L D B L O C K ; s e r v i c e h a n d l e r
i s c l o s e d v i a r e a c t o r e v e n t l o o p .
Ye s t i m e R e t u r n  1 w i t h e r r n o
E W O U L D B L O C K ; w a i t u p t o s p e c i fi e d
a m o u n t o f t i m e f o r c o m p l e t i o n u s i n g
t h e r e a c t o r .
Ye s N U L L R e t u r n  1 w i t h e r r n o
E W O U L D B L O C K ; w a i t f o r c o m p l e t i o n
i n d e fi n i t e l y u s i n g t h e r e a c t o r .
N o 0 , 0 C l o s e s e r v i c e h a n d l e r d i r e c t l y ; r e t u r n
 1 w i t h e r r n o E W O U L D B L O C K .
N o t i m e B l o c k i n c o n n e c t _ s v c _ h a n d l e r ( )
u p t o s p e c i fi e d a m o u n t o f t i m e f o r
c o m p l e t i o n ; i f s t i l l n o t c o m p l e t e d ,
r e t u r n  1 w i t h e r r n o E T I M E .
N o N U L L B l o c k i n c o n n e c t _ s v c _ h a n d l e r ( )
i n d e fi n i t e l y f o r c o m p l e t i o n .
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ C o n n e c t o r C l a s s P r o t e c t e d
I n t e r f a c e
p r o t e c t e d :
/ / M a k e a n e w c o n n e c t i o n .
v i r t u a l S V C _ H A N D L E R * m a k e _ s v c _ h a n d l e r ( v o i d ) ;
/ / A c c e p t a n e w c o n n e c t i o n .
v i r t u a l i n t c o n n e c t _ s v c _ h a n d l e r
( S V C _ H A N D L E R * & s h ,
t y p e n a m e c o n s t P E E R _ C O N N E C T O R : : P E E R _ A D D R & a d d r ,
A C E _ T i m e _ V a l u e * t i m e o u t ) ;
/ / A c t i v a t e a s e r v i c e h a n d l e r .
v i r t u a l i n t a c t i v a t e _ s v c _ h a n d l e r ( S V C _ H A N D L E R * ) ;
/ / D e m u l t i p l e x i n g h o o k s .
v i r t u a l i n t h a n d l e _ o u t p u t ( A C E _ H A N D L E ) ; / / S u c c e s s .
v i r t u a l i n t h a n d l e _ i n p u t ( A C E _ H A N D L E ) ; / / F a i l u r e .
v i r t u a l i n t h a n d l e _ t i m e o u t ( A C E _ T i m e _ V a l u e & ,
c o n s t v o i d * ) ;
/ / T a b l e m a p s I / O h a n d l e t o a n A C E _ S v c _ T u p l e * .
H a s h _ M a p _ M a n a g e r < A C E _ H A N D L E , A C E _ S v c _ T u p l e * ,
A C E _ N u l l _ M u t e x > h a n d l e r _ m a p _ ;
/ / F a c t o r y t h a t e s t a b l i s h e s c o n n e c t i o n s a c t i v e l y .
P E E R _ C O N N E C T O R c o n n e c t o r _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ C o n n e c t o r C l a s s I m p l e m e n t a t i o n
/ / I n i t i a t e c o n n e c t i o n u s i n g s p e c i f i e d
/ / b l o c k i n g s e m a n t i c s .
t e m p l a t e < c l a s s S H , c l a s s P C > i n t
A C E _ C o n n e c t o r < S H , P C > : : c o n n e c t
( S H * & s h ,
c o n s t P C : : P E E R _ A D D R & r _ a d d r ,
A C E _ S y n c h _ O p t i o n s & o p t i o n s )
{
A C E _ T i m e _ V a l u e * t i m e o u t = 0 ;
i n t u s e _ r e a c t o r =
o p t i o n s [ A C E _ S y n c h _ O p t i o n s : : U S E _ R E A C T O R ] ;
i f ( u s e _ r e a c t o r )
t i m e o u t = & A C E _ T i m e _ V a l u e : : z e r o ;
e l s e
t i m e o u t =
o p t i o n s [ A C E _ S y n c h _ O p t i o n s : : U S E _ T I M E O U T ]
? ( T i m e _ V a l u e * ) & o p t i o n s . t i m e o u t ( ) : 0 ;
/ / H o o k m e t h o d s .
i f ( s h = = 0 )
s h = m a k e _ s v c _ h a n d l e r ( ) ;
i f ( c o n n e c t _ s v c _ h a n d l e r ( s h , r a d d r ,
t i m e o u t ) ! = - 1 )
a c t i v a t e _ s v c _ h a n d l e r ( s h ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ C o n n e c t o r H o o k M e t h o d
I m p l e m e n t a t i o n s
t e m p l a t e < c l a s s S H , c l a s s P C > S H *
A C E _ C o n n e c t o r < S H , P C > : : m a k e _ s v c _ h a n d l e r ( v o i d ) {
r e t u r n n e w S H ;
}
t e m p l a t e < c l a s s S H , c l a s s P C > i n t
A C E _ C o n n e c t o r < S H , P C > : : c o n n e c t _ s v c _ h a n d l e r ( S H & * s h ,
t y p e n a m e c o n s t P E E R _ C O N N E C T O R : : P E E R _ A D D R & a d d r ,
A C E _ T i m e _ V a l u e * t i m e o u t ) {
/ / P e e r _ C o n n e c t o r f a c t o r y i n i t i a t e s c o n n e c t i o n .
i f ( c o n n e c t o r _ . c o n n e c t ( s h , a d d r , t i m e o u t ) = = - 1 )
/ / I f t h e c o n n e c t i o n h a s n ’ t c o m p l e t e d , t h e n
/ / r e g i s t e r w i t h t h e R e a c t o r t o c a l l u s b a c k .
i f ( u s e _ r e a c t o r & & e r r n o = = E W O U L D B L O C K )
/ / C r e a t e < A C E _ S v c _ T u p l e > f o r < s h > & r e t u r n - 1
} e l s e
/ / A c t i v a t e i m m e d i a t e l y i f w e ’ r e c o n n e c t e d .
a c t i v a t e _ s v c _ h a n d l e r ( s h ) ;
}
t e m p l a t e < c l a s s S H , c l a s s P C > i n t
A C E _ C o n n e c t o r < S H , P C > : : a c t i v a t e _ s v c _ h a n d l e r ( S H * s h )
{ i f ( s h - > o p e n ( ( v o i d * ) t h i s ) = = - 1 ) s h - > c l o s e ( ) ; }
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
S p e c i a l i z i n g A C E _ C o n n e c t o r a n d
A C E _ S v c _ H a n d l e r
A P P L I C A T I O N -
I N D E P E N D E N T
A P P L I C A T I O N -
S P E C I F I C
P r o x y
H a n d l e r
S u p p l i e r
H a n d l e r
C o n s u m e r
H a n d l e r
S v c
H a n d l e r
M e s s a g e
Q u e u e
 P r o d u c i n g a n a p p l i c a t i o n t h a t m e e t s G a t e w a y
r e q u i r e m e n t s i n v o l v e s specializing A C E
c o m p o n e n t s
– A C E _ C o n n e c t o r !
A C E _ P r o x y _ H a n d l e r _ C o n n e c t o r
– A C E _ S v c _ H a n d l e r !
A C E _ P r o x y _ H a n d l e r !
A C E _ S u p p l i e r _ H a n d l e r a n d
A C E _ C o n s u m e r _ H a n d l e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ P r o x y _ H a n d l e r C l a s s P u b l i c
I n t e r f a c e
/ / D e t e r m i n e t h e t y p e o f t h r e a d i n g m e c h a n i s m .
# i f d e f i n e d ( A C E _ U S E _ M T )
t y p e d e f A C E _ M T _ S Y N C H S Y N C H ;
# e l s e
t y p e d e f A C E _ N U L L _ S Y N C H S Y N C H ;
# e n d i f / * A C E _ U S E _ M T * /
/ / U n i q u e c o n n e c t i o n i d t h a t d e n o t e s P r o x y _ H a n d l e r .
t y p e d e f s h o r t C O N N _ I D ;
/ / T h i s i s t h e t y p e o f t h e R o u t i n g _ T a b l e .
t y p e d e f A C E _ H a s h _ M a p _ M a n a g e r < P e e r _ A d d r ,
R o u t i n g _ E n t r y ,
S Y N C H : : M U T E X >
R O U T I N G _ T A B L E ;
c l a s s P r o x y _ H a n d l e r
: p u b l i c A C E _ S v c _ H a n d l e r < A C E _ S O C K _ S t r e a m , S Y N C H > {
p u b l i c :
/ / I n i t i a l i z e t h e h a n d l e r ( c a l l e d b y t h e
/ / < A C E _ C o n n e c t o r > o r < A C E _ A c c e p t o r > ) .
v i r t u a l i n t o p e n ( v o i d * = 0 ) ;
/ / B i n d a d d r e s s i n g i n f o t o R o u t e r .
v i r t u a l i n t b i n d ( c o n s t A C E _ I N E T _ A d d r & , C O N N _ I D ) ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Parameterizing Synchronization
into the ACE_Hash_Map_Manager
 Q: What’s a good technique to implement a Routing Table?
 A: Use a ACE_Hash_Map_Manager container
– ACE provides a ACE_Hash_Map_Manager container that
associates external ids with internal ids, e.g.,
 External ids (keys) ! URI
 Internal ids (values) ! pointer to memory-mapped file
 Hashing provides O(1) performance in the average-case
Vanderbilt University 233
Advanced ACE Tutorial Douglas C. Schmidt
Applying the Strategized Locking pattern
to the ACE_Hash_Map_Manager Class
template 
class ACE_Hash_Map_Manager { public:
bool bind (EXT_ID, INT_ID *);
bool unbind (EXT_ID);
bool find (EXT_ID ex, INT_ID &in)
{ // Exception-safe code...
ACE_READ_GUARD (LOCK, g,
lock_, false);
// lock_.read_acquire ();
if (find_i (ex, in)) return true;
else return false;
// lock_.release ();
}
private:
LOCK lock_;
bool find_i (EXT_ID, INT_ID &);
// ...
};
ACE_Hash_Map_Manager
uses the template-based
Strategized Locking pattern
to
 Enhance reuse
 Parameterize different
synchronization
strategies, e.g.:
– ACE_Null_Mutex,
ACE_Thread_Mutex,
ACE_RW_Mutex, etc.
Vanderbilt University 234
A d v a n c e d A C E T u t o r i a l D o
D e t a i l e d O O A r c h i t e c t u r e
o f t h e G a t e w a y
C O N N E C T I O N
R E Q U E S T
C O N N E C T I O N
R E Q U E S T
O U T G O I N G
M E S S A G E S
I N C O M I N G
M E S S A G E S
G A T E W A Y
 R o u t i n g
T a b l e
 S u p p l i e r
H a n d l e r
 S O C K
S t r e a m
 A c c e p t o r
 C o n n e c t o r
R e a c t o r
C o n s u m e r
H a n d l e r
 S O C K
S t r e a m
 M e s s a g e
Q u e u e
C o n s u m e r
H a n d l e r
 S O C K
S t r e a m
 M e s s a g e
Q u e u e
 S u p p l i e r
H a n d l e r
 S O C K
S t r e a m
H a s h  M a p
M a n a g e r
H a s h  M a p
M a n a g e r
S O C K
C o n n e c t o r
S O C K
A c c e p t o r
N o t e t h e u s e o f o t h e r A C E c o m p o n e n t s , s u c h a s
t h e s o c k e t w r a p p e r f a c a d e s a n d t h e
A C E _ H a s h _ M a p _ M a n a g e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ S u p p l i e r _ H a n d l e r I n t e r f a c e
c l a s s S u p p l i e r _ H a n d l e r : p u b l i c P r o x y _ H a n d l e r
{
p u b l i c :
S u p p l i e r _ H a n d l e r ( v o i d ) ;
p r o t e c t e d :
/ / R e c e i v e a n d p r o c e s s P e e r m e s s a g e s .
v i r t u a l i n t h a n d l e _ i n p u t ( A C E _ H A N D L E ) ;
/ / R e c e i v e a m e s s a g e f r o m a P e e r .
v i r t u a l i n t r e c v _ p e e r ( A C E _ M e s s a g e _ B l o c k * & ) ;
/ / A c t i o n t h a t r o u t e s a m e s s a g e f r o m a P e e r .
i n t r o u t e _ m e s s a g e ( A C E _ M e s s a g e _ B l o c k * ) ;
/ / K e e p t r a c k o f m e s s a g e f r a g m e n t .
A C E _ M e s s a g e _ B l o c k * m s g _ f r a g _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A C E _ C o n s u m e r _ H a n d l e r I n t e r f a c e
c l a s s C o n s u m e r _ H a n d l e r : p u b l i c P r o x y _ H a n d l e r
{
p u b l i c :
C o n s u m e r _ H a n d l e r ( v o i d ) ;
/ / S e n d a m e s s a g e t o a G a t e w a y
/ / ( m a y b e q u e u e d ) .
v i r t u a l i n t p u t ( A C E _ M e s s a g e _ B l o c k * ,
A C E _ T i m e _ V a l u e * = 0 ) ;
p r o t e c t e d :
/ / P e r f o r m a n o n - b l o c k i n g p u t ( ) .
i n t n o n b l k _ p u t ( A C E _ M e s s a g e _ B l o c k * m b ) ;
/ / F i n i s h s e n d i n g a m e s s a g e w h e n
/ / f l o w c o n t r o l a b a t e s .
v i r t u a l i n t h a n d l e _ o u t p u t ( A C E _ H A N D L E ) ;
/ / S e n d a m e s s a g e t o a P e e r .
v i r t u a l i n t s e n d _ p e e r ( A C E _ M e s s a g e _ B l o c k * ) ;
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
ACE_Proxy_Handler_Connector Class Interface
class Proxy_Handler_Connector :
public ACE_Connector

// Connection factory
{
public:
// Initiate (or reinitiate)
// a connection on
// the Proxy_Handler.
int initiate_connection
(Proxy_Handler *);
}
 ACE_Proxy_Handler_
Connector is a concrete
factory class that:
– Establishes connections with
Peers to produce
ACE_Proxy_Handlers
– Activates
ACE_Proxy_Handlers,
which then route messages
 ACE_Proxy_Handler_
Connector also ensures
reliability by restarting failed
connections
Vanderbilt University 238
A d v a n c e d A C E T u t o r i a l D o
A C E _ P r o x y _ H a n d l e r _ C o n n e c t o r
I m p l e m e n t a t i o n
/ / ( r e ) i n i t i a t e a c o n n e c t i o n t o a P r o x y _ H a n d l e r
i n t
P r o x y _ H a n d l e r _ C o n n e c t o r : : i n i t i a t e _ c o n n e c t i o n
( P r o x y _ H a n d l e r * p h )
{
/ / U s e a s y n c h r o n o u s c o n n e c t i o n s . . .
i f ( c o n n e c t ( p h ,
p h - > a d d r ( ) ,
A C E _ S y n c h _ O p t i o n s : : a s y n c h ) = = - 1 ) {
i f ( e r r n o = = E W O U L D B L O C K )
/ / N o e r r o r , w e ’ r e c o n n e c t i n g a s y n c h r o n o u s l y .
r e t u r n - 1 ;
e l s e
/ / T h i s i s a r e a l e r r o r , s o r e s c h e d u l e
/ / o u r s e l v e s t o r e c o n n e c t .
r e a c t o r ( ) - > s c h e d u l e _ t i m e r
( p h , 0 , p h - > t i m e o u t ( ) ) ;
}
e l s e / / W e ’ r e c o n n e c t e d s y n c h r o n o u s l y !
r e t u r n 0 ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
The Non-blocking Buffered I/O Pattern
Reactor1n
R
E
A
C
T
I
V
E
L
A
Y
E
R Event
Handler
Routing
Table
find()
Supplier
Handler
handle_input()
1
n
R
O
U
T
I
N
G
L
A
Y
E
R
Consumer
Handler
handle_output()
put()
Message
Queue
www.cs.wustl.edu/˜schmidt/PDF/
TAPOS-00.pdf
Intent
 Decouple multiple input
sources from multiple output
sources to prevent blocking
Forces Resolved:
 Keep misbehaving
connections from disrupting
the QoS for well-behaved
connections
 Different concurrency
strategies for
Supplier_Handlers and
Consumer_Handlers
Vanderbilt University 240
A d v a n c e d A C E T u t o r i a l D o
C o l l a b o r a t i o n i n S i n g l e - t h r e a d e d
G a t e w a y R o u t i n g
 S u p p l i e r
H a n d l e r
6 :  p u t  ( m s g )
1 :  h a n d l e _ i n p u t ( )
2 :  r e c v _ p e e r ( m s g )
3 :  f i n d ( )
R O U T E
I D
C o n s u m e r
H a n d l e r s
4:
 p
ut
 (m
sg
)
7 :  s e n d _ p e e r ( m s g )
8 :  e n q u e u e ( m s g )
9 :  s c h e d u l e _ w a k e u p ( )
- - - - - - - - - - - - - - -
1 0 :  h a n d l e _ o u t p u t ( )
1 1 :  d e q u e u e ( m s g )
1 2 :  s e n d _ p e e r ( m s g )
M e s s a g e
Q u e u e
 C o n s u m e r
H a n d l e r
 R o u t i n g
T a b l e
M e s s a g e
Q u e u e
 C o n s u m e r
H a n d l e r
5 :  s e n d _ p e e r ( m s g )
N o t e t h e c o m p l e x c o o p e r a t i v e s c h e d u l i n g l o g i c
r e q u i r e d t o h a n d l e o u t p u t fl o w c o n t r o l c o r r e c t l y
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
S u p p l i e r _ H a n d l e r a n d
C o n s u m e r _ H a n d l e r I m p l e m e n t a t i o n s
i n t S u p p l i e r _ H a n d l e r : : h a n d l e _ i n p u t ( A C E _ H A N D L E ) {
A C E _ M e s s a g e _ B l o c k * r o u t e _ a d d r = 0 ;
i n t n = r e c v _ p e e r ( r o u t e _ a d d r ) ;
/ / T r y t o g e t t h e n e x t m e s s a g e .
i f ( n < = 0 ) {
i f ( e r r n o = = E W O U L D B L O C K ) r e t u r n 0 ;
e l s e r e t u r n n ;
}
e l s e
r o u t e _ m e s s a g e ( r o u t e _ a d d r ) ;
}
/ / S e n d a m e s s a g e t o a P e e r ( q u e u e i f n e c e s s a r y ) .
i n t C o n s u m e r _ H a n d l e r : : p u t ( A C E _ M e s s a g e _ B l o c k * m b ,
A C E _ T i m e _ V a l u e * ) {
i f ( m s g _ q u e u e _ - > i s _ e m p t y ( ) )
/ / T r y t o s e n d t h e m e s s a g e * w i t h o u t * b l o c k i n g !
n o n b l k _ p u t ( m b ) ;
e l s e / / M e s s a g e s a r e q u e u e d d u e t o f l o w c o n t r o l .
m s g _ q u e u e _ - > e n q u e u e _ t a i l
( m b , & A C E _ T i m e _ V a l u e : : z e r o ) ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
S u p p l i e r _ H a n d l e r M e s s a g e R o u t i n g
/ / R o u t e m e s s a g e f r o m a P e e r .
i n t S u p p l i e r _ H a n d l e r : : r o u t e _ m e s s a g e s
( A C E _ M e s s a g e _ B l o c k * r o u t e _ a d d r )
{
/ / D e t e r m i n e d e s t i n a t i o n a d d r e s s .
C O N N _ I D r o u t e _ i d =
* ( C O N N _ I D * ) r o u t e _ a d d r - > r d _ p t r ( ) ;
c o n s t A C E _ M e s s a g e _ B l o c k * c o n s t d a t a =
r o u t e _ a d d r - > c o n t ( ) ;
R o u t i n g _ E n t r y * r e = 0 ;
/ / D e t e r m i n e r o u t e .
R o u t i n g _ T a b l e : : i n s t a n c e ( ) - > f i n d ( r o u t e _ i d , r e ) ;
/ / I n i t i a l i z e i t e r a t o r o v e r d e s t i n a t i o n ( s ) .
S e t _ I t e r a t o r < P r o x y _ H a n d l e r * >
s i ( r e - > d e s t i n a t i o n s ( ) ) ;
/ / M u l t i c a s t m e s s a g e .
f o r ( P r o x y _ H a n d l e r * o u t _ p h ;
s i . n e x t ( o u t _ p h ) ! = - 1 ;
s i . a d v a n c e ( ) ) {
A C E _ M e s s a g e _ B l o c k * n e w m s g = d a t a - > d u p l i c a t e ( ) ;
i f ( o u t _ p h - > p u t ( n e w m s g ) = = - 1 ) / / D r o p m e s s a g e .
n e w m s g - > r e l e a s e ( ) ; / / D e c r e m e n t r e f c o u n t .
}
d e l e t e r o u t e _ a d d r ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P e e r _ M e s s a g e S c h e m a
/ / P e e r a d d r e s s i s u s e d t o i d e n t i f y t h e
/ / s o u r c e / d e s t i n a t i o n o f a P e e r m e s s a g e .
c l a s s P e e r _ A d d r {
p u b l i c :
C O N N _ I D c o n n _ i d _ ; / / U n i q u e c o n n e c t i o n i d .
u _ c h a r l o g i c a l _ i d _ ; / / L o g i c a l I D .
u _ c h a r p a y l o a d _ ; / / P a y l o a d t y p e .
} ;
/ / F i x e d s i z e d h e a d e r .
c l a s s P e e r _ H e a d e r { p u b l i c : / * . . . * / } ;
/ / V a r i a b l e - s i z e d m e s s a g e ( s d u _ m a y b e
/ / b e t w e e n 0 a n d M A X _ M S G _ S I Z E ) .
c l a s s P e e r _ M e s s a g e {
p u b l i c :
/ / T h e m a x i m u m s i z e o f a m e s s a g e .
e n u m { M A X _ P A Y L O A D _ S I Z E = 1 0 2 4 } ;
P e e r _ H e a d e r h e a d e r _ ; / / F i x e d - s i z e d h e a d e r .
c h a r s d u _ [ M A X _ P A Y L O A D _ S I Z E ] ; / / M e s s a g e p a y l o a d .
} ;
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Tips on Handling Flow Control
 Q: What should happen if put() fails?
– e.g., if a queue becomes full?
 A: The answer depends on whether the error handling policy is
different for each router object or the same...
– Strategy pattern: give reasonable default, but allow substitution
 A related design issue deals with avoiding output blocking if a Peer
connection becomes flow controlled
Vanderbilt University 245
A d v a n c e d A C E T u t o r i a l D o
S u p p l i e r H a n d l e r M e s s a g e R e c e p t i o n
/ / P s e u d o - c o d e f o r r e c v ’ i n g m s g v i a n o n - b l o c k i n g I / O
i n t S u p p l i e r _ H a n d l e r : : r e c v _ p e e r
( A C E _ M e s s a g e _ B l o c k * & r o u t e _ a d d r )
{
i f ( m s g _ f r a g _ i s e m p t y ) {
m s g _ f r a g _ = n e w A C E _ M e s s a g e _ B l o c k ;
r e c e i v e f i x e d - s i z e d h e a d e r i n t o m s g _ f r a g _
i f ( e r r o r s o c c u r ) c l e a n u p
e l s e
d e t e r m i n e s i z e o f v a r i a b l e - s i z e d m s g _ f r a g _
} e l s e
d e t e r m i n e h o w m u c h o f m s g _ f r a g _ t o s k i p
n o n - b l o c k i n g r e c v o f p a y l o a d i n t o m s g _ f r a g _
i f ( e n t i r e m e s s a g e i s n o w r e c e i v e d ) {
r o u t e _ a d d r = n e w M e s s a g e _ B l o c k
( s i z e o f ( P e e r _ A d d r ) , m s g _ f r a g _ )
P e e r _ A d d r a d d r ( i d ( ) ,
m s g _ f r a g _ - > r o u t i n g _ i d _ , 0 ) ;
r o u t e _ a d d r - > c o p y ( & a d d r , s i z e o f ( P e e r _ A d d r ) ) ;
r e t u r n t o c a l l e r a n d r e s e t m s g _ f r a g _
}
e l s e i f ( o n l y p a r t o f m e s s a g e i s r e c e i v e d )
r e t u r n e r r n o = E W O U L D B L O C K
e l s e i f ( f a t a l e r r o r o c c u r s ) c l e a n u p
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Design Interlude: Using the ACE_Reactor
to Handle Flow Control
 Q: How can a flow controlled Consumer_Handler know when to
proceed again without polling or blocking?
 A: Use the ACE_Event_Handler::handle_output()
notification scheme of the Reactor
– i.e., via the ACE_Reactor’s methods schedule_wakeup() and
cancel_wakeup()
 This provides cooperative multi-tasking within a single thread of
control
– The ACE_Reactor calls back to the handle_output() hook
method when the Proxy_Handler is able to transmit again
Vanderbilt University 247
Advanced ACE Tutorial Douglas C. Schmidt
Performing a Non-blocking put() of a Message
int Consumer_Handler::nonblk_put
(ACE_Message_Block *mb) {
// Try sending message
// via non-blocking I/O
if (send_peer (mb) != -1
&& errno == EWOULDBLOCK) {
// Queue in *front* of the
// list to preserve order.
msg_queue_->enqueue_head
(mb, &ACE_Time_Value::zero);
// Tell Reactor to call us
// back it’s ok to send again.
reactor ()->schedule_wakeup
(this, ACE_Event_Handler::WRITE_MASK);
}
}
This method is called
in two situations:
1. When first trying
to send over a
connection
2. When flow control
abates
Vanderbilt University 248
A d v a n c e d A C E T u t o r i a l D o
S e n d i n g a M e s s a g e t o a C o n s u m e r
i n t
C o n s u m e r _ H a n d l e r : : s e n d _ p e e r ( A C E _ M e s s a g e _ B l o c k * m b )
{
s s i z e _ t n ;
s i z e _ t l e n = m b - > l e n g t h ( ) ;
/ / T r y t o s e n d t h e m e s s a g e .
n = p e e r ( ) . s e n d ( m b - > r d _ p t r ( ) , l e n ) ;
i f ( n < = 0 )
r e t u r n e r r n o = = E W O U L D B L O C K ? 0 : n ;
e l s e i f ( n < l e n )
/ / S k i p o v e r t h e p a r t w e d i d s e n d .
m b - > r d _ p t r ( n ) ;
e l s e / * i f ( n = = l e n g t h ) * / {
/ / D e c r e m e n t r e f e r e n c e c o u n t .
m b - > r e l e a s e ( ) ;
e r r n o = 0 ;
}
r e t u r n n ;
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
F i n i s h S e n d i n g w h e n
F l o w C o n t r o l A b a t e s
/ / F i n i s h s e n d i n g a m e s s a g e w h e n f l o w c o n t r o l
/ / c o n d i t i o n s a b a t e . T h i s m e t h o d i s a u t o m a t i c a l l y
/ / c a l l e d b y t h e R e a c t o r .
i n t
C o n s u m e r _ H a n d l e r : : h a n d l e _ o u t p u t ( A C E _ H A N D L E )
{
A C E _ M e s s a g e _ B l o c k * m b = 0 ;
/ / T a k e t h e f i r s t m e s s a g e o f f t h e q u e u e .
m s g _ q u e u e _ - > d e q u e u e _ h e a d
( m b , & A C E _ T i m e _ V a l u e : : z e r o ) ;
i f ( n o n b l k _ p u t ( m b ) ! = - 1
| | e r r n o ! = E W O U L D B L O C K ) {
/ / I f w e s u c c e e d i n w r i t i n g m s g o u t c o m p l e t e l y
/ / ( a n d a s a r e s u l t t h e r e a r e n o m o r e m s g s
/ / o n t h e < A C E _ M e s s a g e _ Q u e u e > ) , t h e n t e l l t h e
/ / < A C E _ R e a c t o r > n o t t o n o t i f y u s a n y m o r e .
i f ( m s g _ q u e u e _ - > i s _ e m p t y ( )
r e a c t o r ( ) - > c a n c e l _ w a k e u p
( t h i s , A C E _ E v e n t _ H a n d l e r : : W R I T E _ M A S K ) ;
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
T h e G a t e w a y C l a s s
S U P P L I E R   H AN D L E R
C O N S U M E R   H AN D L E R
S e r v i c e
O b j e c t
A P P L I C A T I O N -
I N D E P E N D E N T
A P P L I C A T I O N -
S P E C I F I C
C o n n e c t o r
H a s h  M a p
M a n a g e r
C o n f i g
T a b l e
P r o x y
H a n d l e r
C o n n e c t o r
R o u t i n g
T a b l e
G a t e w a y
T h i s c l a s s i n t e g r a t e s o t h e r a p p l i c a t i o n - s p e c i fi c a n d
a p p l i c a t i o n - i n d e p e n d e n t c o m p o n e n t s
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Dynamically Configuring Gateway into an Application
Parameterized by proxy handler
template

class Gateway
: public Service_Object
{
public:
// Perform initialization.
virtual int init
(int argc, char *argv[]);
// Perform termination.
virtual int fini (void);
Example of the Component
Configurator pattern
int main (int argc, char *argv[])
{
// Initialize the daemon and
// dynamically configure services.
ACE_Service_Config::open (argc,
argv);
// Run forever, performing the
// configured services.
ACE_Reactor::instance ()->
run_reactor_event_loop ();
/* NOTREACHED */
}
Vanderbilt University 252
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E S e r v i c e C o n fi g u r a t o r
F r a m e w o r k f o r t h e G a t e w a y
d y n a m i c  G a t e w a y  S e r v i c e _ O b j e c t  *
   g a t e w a y : m a k e _ G a t e w a y ( )  " - p  2 0 0 1 "
s v c . c o n f
F I L E
S E R V I C E
C O N F I G U R A T O R
R U N T I M E
 S e r v i c e
R e p o s i t o r y
S e r v i c e
O b j e c t
 T h r e a d
P o o l
G a t e w a y
D L L S
S e r v i c e
O b j e c t
 T h r e a d - p e r
C o n n e c t i o n
G a t e w a y
S e r v i c e
O b j e c t
R e a c t i v e
G a t e w a y
R e a c t o r
 S e r v i c e
C o n f i g
W e c a n r e p l a c e t h e s i n g l e - t h r e a d e d G a t e w a y w i t h
a m u l t i - t h r e a d e d G a t e w a y
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Dynamic Linking a Gateway Service
The Gateway service is
configured via scripting in a
svc.conf file:
% cat ./svc.conf
static Svc_Manager
"-p 5150"
dynamic Gateway
Service_Object *
gateway:_make_Gateway()
"-d -p $PORT"
# .dll or .so suffix
# added to "gateway"
# automatically
Dynamically linked factory function
that allocates a new
single-threaded Gateway
extern "C"
ACE_Service_Object *make_Gateway (void);
ACE_Service_Object *make_Gateway (void)
{
return new
Gateway;
// ACE automatically deletes memory.
}
Vanderbilt University 254
Advanced ACE Tutorial Douglas C. Schmidt
Concurrency Strategies for Patterns
 The Acceptor-Connector pattern does not constrain the concurrency
strategies of a ACE_Svc_Handler
 There are three common choices:
1. Run service in same thread of control
2. Run service in a separate thread
3. Run service in a separate process
 Observe how our patterns and ACE framework push this decision to
the “edges” of the design
– This greatly increases reuse, flexibility, and performance tuning
Vanderbilt University 255
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A c t i v e O b j e c t P a t t e r n
f o r t h e G a t e w a y
C O N C R E T E
E V E N T
H A N D L E R S
F
R
A
M
E
W
O
R
K
L
E
V
E
L
K
E
R
N
E
L
L
E
V
E
L
A
P
P
L
IC
A
T
IO
N
L
E
V
E
L
1 :  h a n d l e _ i n p u t ( )
4 :  s e n d ( m s g )
2 :  r e c v ( m s g )
3 :  r o u t e ( m s g )
 R e a c t o r
T i m e r
Q u e u e
 S i g n a l
H a n d l e r s
 H a n d l e
T a b l e
O S   E V E N T   D E M U L T I P L E X I N G   I N T E R F A C E
 E v e n t
H a n d l e r
S u p p l i e r
H a n d l e r
 E v e n t
H a n d l e r
S u p p l i e r
H a n d l e r
 E v e n t
H a n d l e r
S u p p l i e r
H a n d l e r
 E v e n t
H a n d l e r
 C o n s u m e r
H a n d l e r
 E v e n t
H a n d l e r
 C o n s u m e r
H a n d l e r
E a c h C o n s u m e r _ H a n d l e r i s i m p l e m e n t e d a s a n
A c t i v e O b j e c t
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C o l l a b o r a t i o n i n M u l t i - t h r e a d e d
G a t e w a y R o u t i n g
 S u p p l i e r
H a n d l e r
6 :  p u t  ( m s g )
1 :  h a n d l e _ i n p u t ( )
2 :  r e c v _ p e e r ( m s g )
3 :  f i n d ( )
R O U T E
I D
C o n s u m e r
H a n d l e r s
4:
 p
ut
 (m
sg
)
M e s s a g e
Q u e u e
 C o n s u m e r
H a n d l e r
 R o u t i n g
T a b l e
M e s s a g e
Q u e u e
 C o n s u m e r
H a n d l e r
5 :  s e n d _ p e e r ( m s g )
5 :  s e n d _ p e e r ( m s g )
N o t e t h a t t h i s d e s i g n i s m u c h s i m p l e r s i n c e t h e O S
t h r e a d s c h e d u l e r h a n d l e s b l o c k i n g
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Using the Half-Sync/Half-Async Pattern in the Gateway
Q
U
E
U
E
I
N
G
L
A
Y
E
R
A
S
Y
N
C
H
R
O
N
O
U
S
 
 
T
A
S
K
 
 
L
A
Y
E
R
S
Y
N
C
H
R
O
N
O
U
S
 
T
A
S
K
 
 
L
A
Y
E
R
1: dequeue(msg)
2: send(msg)
2: recv(msg)
3: get_route(msg)
4: enqueue(msg)
1: dispatch()
 Reactor
MESSAGE  QUEUES
Consumer
Handler 
Consumer
Handler 
Consumer
Handler 
 Supplier
Handler
 Supplier
Handler
 Supplier
Handler
 ACE_Reactor plays the
role of “async” layer
 ACE_Task active object
plays the role of “sync” layer
 This particular configuration
is a common variant of the
Half-Sync/Half-Async
pattern, as described in
POSA2
Vanderbilt University 258
A d v a n c e d A C E T u t o r i a l D o
C l a s s D i a g r a m f o r M u l t i - T h r e a d e d
G a t e w a y
P r o x y
H a n d l e r
C o n n e c t o r
P r o x y _ H a n d l e r
S O C K _ C o n n e c t o r
S O C K _ S t r e a m
M T _ S y n c h
S v c
H a n d l e r
C o n n e c t o r
S V C _ H A N D L E R
P E E R _ C O N N E C T O R
P E E R _ S T R E A M
S Y N C H
C
O
N
N
E
C
T
IO
N
-
O
R
IE
N
T
E
D
C
O
M
P
O
N
E
N
T
S
A
P
P
L
IC
A
T
IO
N
-
S
P
E
C
IF
IC
C
O
M
P
O
N
E
N
T
S
A
C
E
F
R
A
M
E
W
O
R
K
C
O
M
P
O
N
E
N
T
S
S t r e a m
S e r v i c e
C o n f i g u r a t o r
C o n c u r r e n c y
global
C o n n e c t i o n
R e a c t o r
1
< < a c t i v a t e s > >
S u p p l i e r / T h r _ C o n s u m e r
H a n d l e r
n
I P C _ S A P
P E E R
C O N N E C T O R
P E E R
S T R E A M
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Thr_Consumer_Handler Class Interface
#define ACE_USE_MT
#include Proxy_Handler.h
class Thr_Consumer_Handler
: public Consumer_Handler
{
public:
// Initialize the object and
// spawn new thread.
virtual int open (void *);
// Send a message to a peer.
virtual int put
(ACE_Message_Block *,
ACE_Time_Value *);
// Transmit peer messages
// in separate thread.
virtual int svc (void);
};
New subclass of
Proxy_Handler uses the
Active Object pattern for the
Consumer_Handler
 Uses multi-threading and
synchronous I/O (rather than
non-blocking I/O) to transmit
message to Peers
 Transparently improve
performance on a
multi-processor platform and
simplify design
Vanderbilt University 260
Advanced ACE Tutorial Douglas C. Schmidt
Thr_Consumer_Handler Class Implementation
Override definition in the
Consumer_Handler class
int
Thr_Consumer_Handler::open (void *)
{
// Become an active object by
// spawning a new thread to
// transmit messages to Peers.
activate (THR_DETACHED);
}
 The multi-threaded version
of open() is slightly
different since it spawns a
new thread to become an
active object!
 activate() is a
pre-defined method on
ACE_Task
Vanderbilt University 261
A d v a n c e d A C E T u t o r i a l D o
T h r _ C o n s u m e r _ H a n d l e r C l a s s
I m p l e m e n t a t i o n
/ / Q u e u e u p a m e s s a g e f o r t r a n s m i s s i o n .
i n t
T h r _ C o n s u m e r _ H a n d l e r : : p u t ( A C E _ M e s s a g e _ B l o c k * m b ,
A C E _ T i m e _ V a l u e * )
{
/ / P e r f o r m n o n - b l o c k i n g e n q u e u e .
m s g _ q u e u e _ - > e n q u e u e _ t a i l ( m b ,
& A C E _ T i m e _ V a l u e : : z e r o ) ;
}
/ / T r a n s m i t m e s s a g e s t o t h e p e e r ( n o t e
/ / s i m p l i f i c a t i o n r e s u l t i n g f r o m t h r e a d s . . . )
i n t
T h r _ C o n s u m e r _ H a n d l e r : : s v c ( v o i d )
{
A C E _ M e s s a g e _ B l o c k * m b = 0 ;
/ / S i n c e t h i s m e t h o d r u n s i n i t s o w n t h r e a d i t
/ / i s O K t o b l o c k o n o u t p u t .
w h i l e ( m s g _ q u e u e _ - > d e q u e u e _ h e a d ( m b ) ! = - 1 )
s e n d _ p e e r ( m b ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Dynamic Linking a Threaded Gateway Service
% cat ./svc.conf
remove Gateway
dynamic Gateway
Service_Object *
thr_gateway:_make_Gateway()
"-d"
# .dll or .so suffix added
# to "thr_Gateway"
# automatically
Dynamically linked factory
function that allocates a
multi-threaded Gateway object
extern "C"
ACE_Service_Object *make_Gateway (void);
ACE_Service_Object *make_Gateway (void)
{
return new
Gateway;
// ACE automatically deletes memory.
}
Vanderbilt University 263
A d v a n c e d A C E T u t o r i a l D o
C a l l C e n t e r M a n a g e r ( C C M )
E v e n t S e r v e r E x a m p l e
E V E N T
S E R V E R
S U P E R
V I S O R
C C M
S t r e a m
A C E
R U N - T I M E
T E L E C O M
S W I T C H E S
S e s s i o n   R o u t e r
M o d u l e
E v e n t   F i l t e r
M o d u l e
S w i t c h   A d a p t e r
M o d u l e
E v e n t   A n a l y z e r
M o d u l e
S U P E R
V I S O R
S U P E R
V I S O R
M I B
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P a t t e r n s i n t h e C C M E v e n t S e r v e r
C o m p o n e n t
C o n f i g u r a t o r
P i p e s   &
F i l t e r s
T A C T I C A L
P A T T E R N S
S T R A T E G I C
P A T T E R N S
R e a c t o r
C o m p o s i t eI t e r a t o r
F a c t o r y
M e t h o d
P r o x y
W r a p p e r
F a c a d e
L a y e r s
P u b l i s h e r
S u b s c r i b e r
A c c e p t o r -
C o n n e c t o r
 T h e E v e n t S e r v e r c o m p o n e n t s a r e b a s e d u p o n
a c o m m o n pattern language
 w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P D F /
D S E J - 9 4 . p d f
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Overview of the ACE Streams Framework
 An ACE_Stream allows flexible configuration of layered processing
modules
 It is an implementation of the Pipes and Filters architecture pattern
– This pattern provides a structure for systems that process a
stream of data
– Each processing step is encapsulated in a filter ACE_Module
component
– Data is passed through pipes between adjacent filters, which can
be re-combined
 The CCM Event Server was design and implemented using ACE
Streams
Vanderbilt University 266
A d v a n c e d A C E T u t o r i a l D o
S t r u c t u r e o f t h e A C E S t r e a m s
F r a m e w o r k
A C E _ M o d u l e A C E _ S t r e a mACE_Task
S Y N C HS Y N C H
2
S Y N C H
2 . . *
F r a m e w o r k c h a r a c t e r i s t i c s
 A n A C E _ S t r e a m c o n t a i n s a s t a c k o f
A C E _ M o d u l e s
 E a c h A C E _ M o d u l e c o n t a i n s t w o A C E _ T a s k s
– i.e., a read t a s k a n d a write t a s k
 E a c h A C E _ T a s k c o n t a i n s a n
A C E _ M e s s a g e _ Q u e u e a n d a p o i n t e r t o a n
A C E _ T h r e a d _ M a n a g e r
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
I m p l e m e n t i n g a S t r e a m i n A C E
N E T W O R K   I N T E R F AC E
O R   P S E U D O -D E V I C E S
S T R E A M
T a i l
M u l t i p l e x o r
A P P L I C A T I O N
S t r e a m
S T R E A M
H e a d
A P P L I C A T I O N
S t r e a m
U
P
S
T
R
E
A
MD
O
W
N
S
T
R
E
A
M
M E S S A G E
W R I T E
T A S K
R E A D
T A S K
M O D U L E
o p e n ( ) = 0
c l o s e ( ) = 0
p u t ( ) = 0
s v c ( ) = 0
N o t e s i m i l a r i t i e s t o S y s t e m V S T R E A M S
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A l t e r n a t i v e C o n c u r r e n c y M o d e l s
f o r M e s s a g e P r o c e s s i n g
M o d u l e
A
P R O C E S S   O R
T H R E A D
W R I T E   T A S K
O B J E C T
R E A D   T A S K
O B J E C T
M O D U L E
O B J E C T
M o d u l e
B
M o d u l e
C
M o d u l e
A
M o d u l e
B
M o d u l e
C
2 :  s v c ( )
1 :  p u t ( )
4 :  s v c ( )
3 :  p u t ( )
A C T I V E
A C T I V E
A C T I V E
A C T I V E
2 :  p u t ( )
1 :  p u t ( )
A C T I V EA C T I V E
T A S K - B A S E D
P R O C E S S   A R C H I T E C T U R E
M E S S A G E - B A S E D
P R O C E S S   A R C H I T E C T U R E
M E S S A G E
O B J E C T
T a s k - b a s e d m o d e l s a r e m o r e i n t u i t i v e b u t l e s s
e f fi c i e n t t h a n M e s s a g e - b a s e d m o d e l s
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
U s i n g t h e A C E S t r e a m s F r a m e w o r k
f o r t h e C C M E v e n t S e r v e r
S e s s i o n  R o u t e r
M o d u l e
P r e s e n t a t i o n
M o d u l e
E v e n t  F i l t e r
M o d u l e
E v e n t  A n a l y s i s
M o d u l e
P r e s e n t a t i o n
M o d u l e
S w i t c h  A d a p t e r
M o d u l e
MD110 ERICSSON
T E L E C O M
S W I T C H E S
S U P E R
V I S O R S
MD110 ERICSSON
MD110 ERICSSON
S U P E R
V I S O R S
S U P E R
V I S O R S
S w i t c h  I O
S e s s i o n  I O
R e a c t o r
w w w . c s . w u s t l . e d u / ˜ s c h m i d t / P D F /
D S E J - 9 4 . p d f
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Broader Context: External OS for Telecom Switches
NETWORK
SWITCHES
SERVER
CLIENT
APPLICATION
FRAMEWORK
CALL  CENTER
MANAGER
DIRECTORY
MANAGER
EXTENSION
MANAGER
CLIENT CLIENT
CLIENT
Features
 Allow clients to manage
various aspects of telecom
switches without modifying
the switch software
 Support reuse of existing
components based on a
common architectural
framework
Vanderbilt University 271
Advanced ACE Tutorial Douglas C. Schmidt
Applying ACE Streams to External OS
DATABASE
 Session
Router
 Event
Analyzer
 Reactor
 Switch
Adapter
SWITCHES
SERVER
CLIENT
CLIENT
CLIENT
CALL  CENTER
MANAGER
DIRECTORY
MANAGER
EXTENSION
MANAGER
NETWORK
ACE
FRAMEWORK
Vanderbilt University 272
Advanced ACE Tutorial Douglas C. Schmidt
ACE Stream Example: Parallel I/O Copy
Consumer
Module
active 2: svc()
3: put()
active
Producer
Module
1: read()
4: svc()
5: write()  Program copies stdin
to stdout via the use of
a multi-threaded
ACE_Stream
 Stream implements a
“bounded buffer”
 Since the data flow is
uni-directional the
“read” ACE_Task is
always ignored
Vanderbilt University 273
A d v a n c e d A C E T u t o r i a l D o
P r o d u c e r C l a s s I n t e r f a c e
t y p e d e f s h o r t - h a n d s f o r t e m p l a t e s
t y p e d e f A C E _ S t r e a m < A C E _ M T _ S Y N C H > M T _ S t r e a m ;
t y p e d e f A C E _ M o d u l e < A C E _ M T _ S Y N C H > M T _ M o d u l e ;
t y p e d e f A C E _ T a s k < A C E _ M T _ S Y N C H > M T _ T a s k ;
D e fi n e t h e P r o d u c e r i n t e r f a c e
c l a s s P r o d u c e r : p u b l i c M T _ T a s k
{
p u b l i c :
/ / I n i t i a l i z e P r o d u c e r .
v i r t u a l i n t o p e n ( v o i d * )
{
/ / a c t i v a t e ( ) i s i n h e r i t e d f r o m c l a s s T a s k .
a c t i v a t e ( T H R _ B O U N D ) ;
}
/ / R e a d d a t a f r o m s t d i n a n d p a s s t o c o n s u m e r .
v i r t u a l i n t s v c ( v o i d ) ;
/ / . . .
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
P r o d u c e r C l a s s I m p l e m e n t a t i o n
R u n s a s a n a c t i v e o b j e c t i n a s e p a r a t e t h r e a d
i n t P r o d u c e r : : s v c ( v o i d ) {
f o r ( ; ; ) {
A C E _ M e s s a g e _ B l o c k * m b ;
/ / A l l o c a t e a n e w m e s s a g e .
A C E _ N E W _ R E T U R N ( m b ,
A C E _ M e s s a g e _ B l o c k ( B U F S I Z ) ,
- 1 ) ;
/ / K e e p r e a d i n g s t d i n , u n t i l w e r e a c h E O F .
s s i z e _ t n = A C E _ O S : : r e a d ( A C E _ S T D I N ,
m b - > w r _ p t r ( ) ,
m b - > s i z e ( ) ) ;
i f ( n < = 0 ) {
/ / S e n d s h u t d o w n m e s s a g e t o o t h e r
/ / t h r e a d a n d e x i t .
m b - > l e n g t h ( 0 ) ;
t h i s - > p u t _ n e x t ( m b ) ;
b r e a k ;
} e l s e {
m b - > w r _ p t r ( n ) ; / / A d j u s t w r i t e p o i n t e r .
/ / S e n d t h e m e s s a g e t o t h e o t h e r t h r e a d .
t h i s - > p u t _ n e x t ( m b ) ;
}
}
}
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C o n s u m e r C l a s s I n t e r f a c e
D e fi n e t h e C o n s u m e r i n t e r f a c e
c l a s s C o n s u m e r : p u b l i c M T _ T a s k
{
p u b l i c :
/ / I n i t i a l i z e C o n s u m e r .
v i r t u a l i n t o p e n ( v o i d * )
{
/ / < a c t i v a t e > i s i n h e r i t e d f r o m c l a s s T a s k .
a c t i v a t e ( T H R _ B O U N D ) ;
}
/ / E n q u e u e t h e m e s s a g e o n t h e M e s s a g e _ Q u e u e
/ / f o r s u b s e q u e n t p r o c e s s i n g i n < s v c > .
v i r t u a l i n t p u t ( A C E _ M e s s a g e _ B l o c k * ,
A C E _ T i m e _ V a l u e * = 0 )
{
/ / < p u t q > i s i n h e r i t e d f r o m c l a s s T a s k .
r e t u r n p u t q ( m b , t v ) ;
}
/ / R e c e i v e m e s s a g e f r o m p r o d u c e r
/ / a n d p r i n t t o s t d o u t .
v i r t u a l i n t s v c ( v o i d ) ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
C o n s u m e r C l a s s I m p l e m e n t a t i o n
C o n s u m e r d e q u e u e s a m e s s a g e f r o m t h e
A C E _ M e s s a g e _ Q u e u e , w r i t e s t h e m e s s a g e t o t h e
s t d e r r s t r e a m , a n d d e l e t e s t h e m e s s a g e
i n t
C o n s u m e r : : s v c ( v o i d ) {
A C E _ M e s s a g e _ B l o c k * m b = 0 ;
/ / K e e p l o o p i n g , r e a d i n g a m e s s a g e f r o m t h e q u e u e ,
/ / u n t i l w e g e t a 0 l e n g t h m e s s a g e , t h e n q u i t .
f o r ( ; ; ) {
i n t r e s u l t = g e t q ( m b ) ;
i f ( r e s u l t = = - 1 ) b r e a k ;
i n t l e n g t h = m b - > l e n g t h ( ) ;
i f ( l e n g t h > 0 )
A C E _ O S : : w r i t e ( A C E _ S T D O U T , m b - > r d _ p t r ( ) ,
l e n g t h ) ;
m b - > r e l e a s e ( ) ;
i f ( l e n g t h = = 0 ) b r e a k ;
}
}
T h e P r o d u c e r s e n d s a 0 - s i z e d m e s s a g e t o i n f o r m
t h e C o n s u m e r t o s t o p r e a d i n g a n d e x i t
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
M a i n D r i v e r F u n c t i o n f o r t h e S t r e a m
C r e a t e P r o d u c e r a n d C o n s u m e r M o d u l e s a n d
p u s h t h e m o n t o t h e S t r e a m
i n t m a i n ( i n t a r g c , c h a r * a r g v [ ] )
{
/ / C o n t r o l h i e r a r c h i c a l l y - r e l a t e d
/ / a c t i v e o b j e c t s .
M T _ S t r e a m s t r e a m ;
/ / A l l p r o c e s s i n g i s p e r f o r m e d i n t h e
/ / S t r e a m a f t e r < p u s h > ’ s c o m p l e t e .
s t r e a m . p u s h ( n e w M T _ M o d u l e
( " C o n s u m e r " , n e w C o n s u m e r ) ;
s t r e a m . p u s h ( n e w M T _ M o d u l e
( " P r o d u c e r " , n e w P r o d u c e r ) ) ;
/ / B a r r i e r s y n c h r o n i z a t i o n : w a i t f o r
/ / t h e t h r e a d s , t o e x i t , t h e n e x i t
/ / t h e m a i n t h r e a d .
A C E _ T h r e a d _ M a n a g e r : : i n s t a n c e ( ) - > w a i t ( ) ;
}
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Evaluation of the ACE Stream Framework
 Structuring active objects via an ACE_Stream allows
“interpositioning”
– i.e., similar to adding a filter in a UNIX pipeline
 New functionality may be added by “pushing” a new processing
ACE_Module onto an ACE_Stream, e.g.:
stream.push (new MT_Module ("Consumer", new Consumer))
stream.push (new MT_Module ("Filter", new Filter));
stream.push (new MT_Module ("Producer", new Producer));
 Communication between ACE_Modules is typically anonymous
Vanderbilt University 279
Advanced ACE Tutorial Douglas C. Schmidt
Concurrency Strategies
 Developing correct, efficient, and robust concurrent applications is
challenging
 Below, we examine a number of strategies that addresses
challenges related to the following:
– Concurrency control
– Library design
– Thread creation
– Deadlock and starvation avoidance
Vanderbilt University 280
Advanced ACE Tutorial Douglas C. Schmidt
General Threading Guidelines
 A threaded program should not arbitrarily enter non-threaded (i.e.,
“unsafe”) code
 Threaded code may refer to unsafe code only from the main thread
– e.g., beware of errno problems
 Use reentrant OS library routines (‘ r’) rather than non-reentrant
routines
 Beware of thread global process operations, such as file I/O
 Make sure that main() terminates cleanly
– e.g., beware of pthread_exit(), exit(), and “falling off the
end”
Vanderbilt University 281
Advanced ACE Tutorial Douglas C. Schmidt
Thread Creation Strategies
 Use threads for independent jobs that must maintain state for the life
of the job
 Don’t spawn new threads for very short jobs
 Use threads to take advantage of CPU concurrency
 Only use “bound” threads when absolutely necessary
 If possible, tell the threads library how many threads are expected to
be active simultaneously
– e.g., use thr_setconcurrency()
Vanderbilt University 282
Advanced ACE Tutorial Douglas C. Schmidt
General Locking Guidelines
 Don’t hold locks across long duration operations (e.g., I/O) that can
impact performance
– Use ACE_Token instead...
 Beware of holding non-recursive mutexes when calling a method
outside a class
– The method may reenter the module and deadlock
 Don’t lock at too small of a level of granularity
 Make sure that threads obey the global lock hierarchy
– But this is easier said than done...
Vanderbilt University 283
Advanced ACE Tutorial Douglas C. Schmidt
Locking Alternatives
 Code locking
– Associate locks with body of functions
 Typically performed using bracketed mutex locks
– Often called a Monitor Object
 Data locking
– Associate locks with data structures and/or objects
– Permits a more fine-grained style of locking
 Data locking allows more concurrency than code locking, but may
incur higher overhead
Vanderbilt University 284
Advanced ACE Tutorial Douglas C. Schmidt
Single-lock Strategy
 One way to simplify locking is use a single, application-wide mutex
lock
 Each thread must acquire the lock before running and release it
upon completion
 The advantage is that most legacy code doesn’t require changes
 The disadvantage is that parallelism is eliminated
– Moreover, interactive response time may degrade if the lock isn’t
released periodically
Vanderbilt University 285
Advanced ACE Tutorial Douglas C. Schmidt
Monitor Object Strategy
 A more OO locking strategy is to use a Monitor Object
– www.cs.wustl.edu/˜schmidt/POSA/
 Monitor Object synchronization mechanisms allow concurrent
method invocations
– Either eliminate access to shared data or use synchronization
objects
– Hide locking mechanisms behind method interfaces
 Therefore, modules should not export data directly
 Advantage is transparency
 Disadvantages are increased overhead from excessive locking and
lack of control over method invocation order
Vanderbilt University 286
Advanced ACE Tutorial Douglas C. Schmidt
Active Object Strategy
 Each task is modeled as an active object that maintains its own
thread of control
 Messages sent to an object are queued up and processed
asynchronously with respect to the caller
– i.e., the order of execution may differ from the order of invocation
 This approach is more suitable to message passing-based
concurrency
 The ACE_Task class can be used to implement active objects
– www.cs.wustl.edu/˜schmidt/POSA/
Vanderbilt University 287
Advanced ACE Tutorial Douglas C. Schmidt
Invariants
 In general, an invariant is a condition that is always true
 For concurrent programs, an invariant is a condition that is always
true when an associated lock is not held
– However, when the lock is held the invariant may be false
– When the code releases the lock, the invariant must be
re-established
 e.g., enqueueing and dequeueing messages in the
ACE_Message_Queue class
Vanderbilt University 288
Advanced ACE Tutorial Douglas C. Schmidt
Run-time Stack Problems
 Most threads libraries contain restrictions on stack usage
– The initial thread gets the “real” process stack, whose size is only
limited by the stacksize limit
– All other threads get a fixed-size stack
 Each thread stack is allocated off the heap and its size is fixed
at startup time
 Therefore, be aware of “stack smashes” when debugging
multi-threaded code
– Overly small stacks lead to bizarre bugs, e.g.,
 Functions that weren’t called appear in backtraces
 Functions have strange arguments
Vanderbilt University 289
Advanced ACE Tutorial Douglas C. Schmidt
Deadlock
 Permanent blocking by a set of threads that are competing for a set
of resources
 Caused by “circular waiting,” e.g.,
– A thread trying to reacquire a lock it already holds
– Two threads trying to acquire resources held by the other
 e.g., T
1
and T
2
acquire locks L
1
and L
2
in opposite order
 One solution is to establish a global ordering of lock acquisition (i.e.,
a lock hierarchy)
– May be at odds with encapsulation...
Vanderbilt University 290
Advanced ACE Tutorial Douglas C. Schmidt
Avoiding Deadlock in OO Frameworks
 Deadlock can occur due to properties of OO frameworks, e.g.,
– Callbacks
– Inter-class method calls
 There are several solutions
– Release locks before performing callbacks
 Every time locks are reacquired it may be necessary to
reevaluate the state of the object
– Make private “helper” methods that assume locks are held when
called by methods at higher levels
– Use an ACE_Token or ACE_Recursive_Thread_Mutex
Vanderbilt University 291
A d v a n c e d A C E T u t o r i a l D o
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x
I m p l e m e n t a t i o n
H e r e i s p o r t a b l e i m p l e m e n t a t i o n o f r e c u r s i v e t h r e a d
m u t e x e s a v a i l a b l e i n A C E :
c l a s s A C E _ R e c u r s i v e _ T h r e a d _ M u t e x
{
p u b l i c :
/ / I n i t i a l i z e a r e c u r s i v e m u t e x .
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x ( v o i d ) ;
/ / I m p l i c i t l y r e l e a s e a r e c u r s i v e m u t e x .
˜ A C E _ R e c u r s i v e _ T h r e a d _ M u t e x ( v o i d ) ;
/ / A c q u i r e a r e c u r s i v e m u t e x .
i n t a c q u i r e ( v o i d ) ;
/ / C o n d i t i o n a l l y a c q u i r e a r e c u r s i v e m u t e x .
i n t t r y a c q u i r e ( v o i d ) ;
/ / R e l e a s e s a r e c u r s i v e m u t e x .
i n t r e l e a s e ( v o i d ) ;
p r i v a t e :
A C E _ T h r e a d _ M u t e x n e s t i n g _ m u t e x _ ;
A C E _ C o n d i t i o n _ T h r e a d _ M u t e x m u t e x _ a v a i l a b l e _ ;
A C E _ t h r e a d _ t o w n e r _ ;
i n t n e s t i n g _ l e v e l _ ;
} ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
A c q u i r i n g a n
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x
i n t A C E _ R e c u r s i v e _ T h r e a d _ M u t e x : : a c q u i r e ( v o i d )
{
A C E _ t h r e a d _ t t _ i d = A C E _ T h r e a d : : s e l f ( ) ;
A C E _ G U A R D _ R E T U R N ( A C E _ T h r e a d _ M u t e x , g u a r d ,
n e s t i n g _ m u t e x _ , - 1 ) ;
/ / I f t h e r e ’ s n o c o n t e n t i o n , g r a b m u t e x .
i f ( n e s t i n g _ l e v e l _ = = 0 ) {
o w n e r _ = t _ i d ;
n e s t i n g _ l e v e l _ = 1 ;
}
e l s e i f ( t _ i d = = o w n e r _ )
/ / I f w e a l r e a d y o w n t h e m u t e x , t h e n
/ / i n c r e m e n t n e s t i n g l e v e l a n d p r o c e e d .
n e s t i n g _ l e v e l _ + + ;
e l s e {
/ / W a i t u n t i l n e s t i n g l e v e l d r o p s
/ / t o z e r o , t h e n a c q u i r e t h e m u t e x .
w h i l e ( n e s t i n g _ l e v e l _ > 0 )
m u t e x _ a v a i l a b l e _ . w a i t ( ) ;
/ / N o t e t h a t a t t h i s p o i n t
/ / t h e n e s t i n g _ m u t e x _ i s h e l d . . .
o w n e r _ = t _ i d ;
n e s t i n g _ l e v e l _ = 1 ;
}
r e t u r n 0 ;
V a n d e r b i l t U n i v e r s i t y
A d v a n c e d A C E T u t o r i a l D o
R e l e a s i n g a n d I n i t i a l i z i n g a n
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x
i n t A C E _ R e c u r s i v e _ T h r e a d _ M u t e x : : r e l e a s e ( v o i d )
{
A C E _ t h r e a d _ t t _ i d = A C E _ T h r e a d : : s e l f ( ) ;
/ / A u t o m a t i c a l l y a c q u i r e m u t e x .
A C E _ G U A R D _ R E T U R N ( A C E _ T h r e a d _ M u t e x , g u a r d ,
n e s t i n g _ m u t e x _ , - 1 ) ;
n e s t i n g _ l e v e l _ - - ;
i f ( n e s t i n g _ l e v e l _ = = 0 ) {
/ / P u t t h e m u t e x i n t o a k n o w n s t a t e .
o w n e r _ = A C E _ O S : : N U L L _ t h r e a d ;
/ / I n f o r m w a i t e r s t h a t t h e m u t e x i s f r e e .
m u t e x _ a v a i l a b l e _ . s i g n a l ( ) ;
}
r e t u r n 0 ;
}
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x : :
A C E _ R e c u r s i v e _ T h r e a d _ M u t e x ( v o i d )
: n e s t i n g _ l e v e l _ ( 0 ) ,
o w n e r _ ( A C E _ O S : : N U L L _ t h r e a d ) ,
m u t e x _ a v a i l a b l e _ ( n e s t i n g _ m u t e x _ ) { }
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Avoiding Starvation
 Starvation occurs when a thread never acquires a mutex even
though another thread periodically releases it
 The order of scheduling is often undefined
 This problem may be solved via:
– Use of “voluntary pre-emption” mechanisms
 e.g., thr_yield() or Sleep()
– Using an ACE “Token” that strictly orders acquisition and release
Vanderbilt University 295
Advanced ACE Tutorial Douglas C. Schmidt
Drawbacks to Multi-threading
 Performance overhead
– Some applications do not benefit directly from threads
– Synchronization is not free
– Threads should be created for processing that lasts at least
several 1,000 instructions
 Correctness
– Threads are not well protected against interference
– Concurrency control issues are often tricky
– Many legacy libraries are not thread-safe
 Development effort
– Developers often lack experience
– Debugging is complicated (lack of tools)
Vanderbilt University 296
Advanced ACE Tutorial Douglas C. Schmidt
Lessons Learned using OO Patterns
 Benefits of patterns
– Enable large-scale reuse of software architectures
– Improve development team communication
– Help transcend language-centric viewpoints
 Drawbacks of patterns
– Do not lead to direct code reuse
– Can be deceptively simple
– Teams may suffer from pattern overload
Vanderbilt University 297
Advanced ACE Tutorial Douglas C. Schmidt
Lessons Learned using OO Frameworks
 Benefits of frameworks
– Enable direct reuse of code (cf patterns)
– Facilitate larger amounts of reuse than stand-alone functions or
individual classes
 Drawbacks of frameworks
– High initial learning curve
 Many classes, many levels of abstraction
– The flow of control for reactive dispatching is non-intuitive
– Verification and validation of generic components is hard
Vanderbilt University 298
Advanced ACE Tutorial Douglas C. Schmidt
Lessons Learned using C++
 Benefits of C++
– Classes and namespaces modularize the system architecture
– Inheritance and dynamic binding decouple application policies
from reusable mechanisms
– Parameterized types decouple the reliance on particular types of
synchronization methods or network IPC interfaces
 Drawbacks of C++
– Some language features are not implemented
– Some development environments are primitive
– Language has many dark corners and sharp edges
 Purify helps alleviate many problems...
Vanderbilt University 299
Advanced ACE Tutorial Douglas C. Schmidt
Lessons Learned using OOD
 Good designs can be boiled down to a few key principles:
– Separate interface from implementation
– Determine what is common and what is variable with an interface
and an implementation
– Allow substitution of variable implementations via a common
interface
 i.e., the “open/closed” principle & Aspect-Oriented
Programming (AOP)
– Dividing commonality from variability should be goal-oriented
rather than exhaustive
 Design is not simply drawing a picture using a CASE tool, using
graphical UML notation, or applying patterns
– Design is a fundamentally creative activity
Vanderbilt University 300
Advanced ACE Tutorial Douglas C. Schmidt
Software Principles for Distributed Applications
 Use patterns/frameworks to decouple policies/mechanisms
– Enhance reuse of common concurrent programming components
 Decouple service functionality from configuration
– Improve flexibility and performance
 Use classes, inheritance, dynamic binding, and parameterized
types
– Improve extensibility and modularity
 Enhance performance/functionality with OS features
– e.g., implicit and explicit dynamic linking and multi-threading
 Perform commonality/variability analysis
– Identify uniform interfaces for variable components and support
pluggability of variation
Vanderbilt University 301
Advanced ACE Tutorial Douglas C. Schmidt
Conferences and Workshops on Patterns
 Pattern Language of Programs Conferences
– PLoP, September, 2002, Monticello, Illinois, USA
– OOPSLA, November, 2002, Seattle, USA
– hillside.net/patterns/conferences/
 Distributed Objects and Applications Conference
– Oct/Nov, 2002, UC Irvine
– www.cs.wustl.edu/˜schmidt/activities-chair.html
Vanderbilt University 302
Advanced ACE Tutorial Douglas C. Schmidt
Patterns, Frameworks, and ACE Literature
 Books
– Gamma et al., Design Patterns: Elements of Reusable
Object-Oriented Software AW, ’94
– Pattern Languages of Program Design series by AW, ’95-’99.
– Siemens & Schmidt, Pattern-Oriented Software Architecture,
Wiley, volumes ’96 & ’00 (www.posa.uci.edu)
– Schmidt & Huston, C++ Network Programming: Mastering
Complexity with ACE and Patterns, AW, ’02
(www.cs.wustl.edu/˜schmidt/ACE/book1/)
– Schmidt & Huston, C++ Network Programming: Systematic
Reuse with ACE and Frameworks, AW, ’03
(www.cs.wustl.edu/˜schmidt/ACE/book2/)
Vanderbilt University 303
A d v a n c e d A C E T u t o r i a l D o
H o w t o O b t a i n A C E S o f t w a r e a n d
T e c h n i c a l S u p p o r t
 A l l s o u r c e c o d e f o r A C E i s f r e e l y a v a i l a b l e
– w w w . c s . w u s t l . e d u / ˜ s c h m i d t / A C E .
h t m l
 M a i l i n g l i s t s
– a c e - u s e r s @ c s . w u s t l . e d u
– a c e - u s e r s - r e q u e s t @ c s . w u s t l . e d u
– a c e - a n n o u n c e @ c s . w u s t l . e d u
– a c e - a n n o u n c e - r e q u e s t @ c s . w u s t l . e d u
 N e w s g r o u p
– c o m p . s o f t - s y s . a c e
 C o m m e r c i a l s u p p o r t f r o m R i v e r a c e a n d O C I
– w w w . r i v e r a c e . c o m
– w w w . t h e a c e o r b . c o m
V a n d e r b i l t U n i v e r s i t y
Advanced ACE Tutorial Douglas C. Schmidt
Concluding Remarks
 Developers of networked application software confront recurring
challenges that are largely application-independent
– e.g., service configuration and initialization, distribution, error
handling, flow control, event demultiplexing, concurrency,
synchronization, persistence, etc.
 Successful developers resolve these challenges by applying
appropriate patterns to create communication frameworks
containing components
 Frameworks and components are an effective way to achieve
systematic reuse of software
Vanderbilt University 305