a new simulation tool for the modeling and control of...

A New Simulation Tool for the Modelingand Control of Distributed SystemsFernando G. GonzalezDepartment of Electrical and Computer EngineeringUniversity of Central FloridaOrlando, FL [email protected]

Wayne J. DavisDepartment of General EngineeringUniversity of Illinois at Urbana-ChampaignUrbana, IL 61801

This paper discusses the construction of a new simulation and control tool that is designed to facilitatethe modeling of distributed systems. This tool can be employed to design and implement distributedcontrol architectures for managing large-scale, discrete event systems. Current commercial simula-tion tools cannot adequately model distributed control architectures. They can only model the systemusing a single model and therefore cannot adequately predict the performance constraints that thesedistributed architectures impose on the system. Furthermore, current tools are not designed to con-trol the systems they model. Thus, in modeling and controlling such distributed systems, one mustcurrently write the models using a general-purpose language such as C++. In simulation, the individ-ual models are tied together using a model of the communications network included as part of thetool. This paper discusses this new tool, which is capable of addressing these concerns.

Keywords: PLS. PROVIDE 3-5 KEYWORDS

1. Introduction

The problem being addressed in this paper is the simula-tion and control of distributed discrete event systems. Thispaper describes the creation of a software tool that is de-signed to facilitate the modeling, simulation, and controlof distributed systems. This tool is designed to perform thesimulation and control of the discrete event system using adistributed collection of models. It is assumed that the sys-tem, or actually the discrete event aspects of the system,responds to messages passing between the controller andthe hardware. These large distributed manmade systemsinclude flexible manufacturing systems, intelligent vehic-ular traffic control systems, airport traffic control systems,and even military decision-making systems as just a fewexamples. With manufacturing potentially benefiting themost, simulation and control of distributed manufacturingsystems are the main focus of this work.

When writing a simulation model, developers canemploy either a dedicated simulation language such asARENA [1] or a general-purpose programming languagesuch as C++. The simulation language has the advantage of

|||||

Submission Date: xxxAccepted Date: xxx

SIMULATION, Vol. 78, Issue 9, September 2002 xxx-xxx©2002 The Society for Modeling and Simulation International

facilitating modeling development, provided that the simu-lation tool provides an appropriate paradigm for modelingthe system. For example, ARENA provides a set of ba-sic modeling elements that can quickly be assembled toconstruct the model. It also includes the essential softwareelements that are needed to execute the model and to per-form the requisite statistical analyses.

If the modeling paradigm is appropriate to the situa-tion, then a simulation language represents the ideal choice.Even when the paradigm does not quite fit, minor model-ing deficiencies can sometimes be addressed by providinguser-defined patches to the software. In general, though,simulation languages will not provide the programmingflexibility provided by a general-purpose programminglanguage, and therefore their scope of applicability is lim-ited. When the modeling paradigm imposed by the simula-tion tool is simply inappropriate, its use should be rejected.For example, to assess the consequences that an includeddistributed control architecture has on the performance ofa complex system such as a flexible manufacturing system(FMS), the set of modeling elements provided by most sim-ulation languages is simply insufficient. In fact, the adoptedmodeling paradigm employed by most commercial simu-lation tools (i.e., entity flow through a stochastic queu-ing network) is totally inappropriate for modeling suchsystems. Davis et al. [2] and Davis [3] discuss, in detail,

Gonzalez and Davis

the numerous restrictions that entity flow-based simula-tion languages impose on the modeling of an FMS. Mizeet al. [4] further discuss the inaccuracies that ensue fromthe use of current simulation languages to model an FMS.Today, developing models of complex systems, in whichthe consequences that the control architecture imposes onthe system are to be assessed, requires the use of a general-purpose programming language such as C/C++.

An evolving application for simulation is to employ themodel to control the system. This approach increases thevalidity of the model since the model of the system andthe implemented controllers are one and the same. In ad-dition, if a simulation tool could provide software objectsthat are capable of controlling elements of the consideredsystem, then the software tool could provide a computer-aided design tool for specifying the control architecturesthat are needed to manage these systems. Today, such toolsdo not exist, and most control architectures for these com-plex systems are being designed on an ad hoc basis. Amajor thrust of our recent research has been to develop amodeling paradigm that can address both the modeling andcontrol requirements.

The design goal for our simulation tool is to providethe modeling convenience of a commercial simulation lan-guage while providing the additional modeling capabilityneeded to control real-world systems such as distributedsystems. A basic set of C++ objects was developed to pro-vide the minimum necessary modeling elements that mustbe employed in specifying most simulation and controllermodels, regardless of the modeling paradigm being used.However, our simulation tool also includes additional ob-jects that are needed to support control and communica-tion concerns. In addition, it permits the addition of user-programmed C++ objects in a manner that is not currentlysupported by most commercial languages.

This paper first introduces the simulation tool. We showhow the modeling elements are implemented and used ina simulation along with the simulation executive function.Next, new modeling elements, as well as added features forthe executive function that provide it with the capability tocontrol a system, are discussed. Finally, we demonstratehow to use this tool to simulate and control distributed sys-tems by introducing the concept of a coordination function.

For discussion purposes in this paper, an event repre-sents the action taken when a given task is implementedin either the simulated or actual physical system. For ex-ample, a DELAY event models the passing of time while agiven processing task is being executed, whereas a QUE-SEIZE event models the system dynamics associated withthe allocation of the resources that are needed for execut-ing a processing task. Also note that throughout the paper,we will be considering two modes of operation for thediscussed tool: a simulation mode and a control mode. Insimulation mode, time advances as each event is being ex-ecuted, but in control mode, execution is tied to a real-timeclock.

2. Other Related Work

In much the same way that we use differential equationsto model continuous systems, much work has been donein modeling discrete event systems using a set of equa-tions. In the early 1970s, Zeigler [5, 6] introduced DEVS(Discrete Event System Specification) as a way to describediscrete event systems. This effort, like the effort by Ra-madge and Wonham [7] and Wonham and Ramadge [8],focuses on modeling discrete event systems using a formallanguage like a finite state machine. More recent researchin this area can be found in Chow, Zeigler, and Kim [9];Passino and Burgess [10]; Chung, Lafortune and Lin [11];and Rudie, Lafortune, and Feng [12]. The objective of theresearch in these references is to examine control-theoreticideas such as controllability, observability, stability, aggre-gation, and decentralized and hierarchical control of DES(see Ramadge and Wonham [13] and Passino and Burgess[10]). Discrete event system control has also been stud-ied in this context. Song and Kim [14] showed how theDEVS framework can be used to design a DES controller.However, the controller their method produces is a theoret-ical model described in the form of finite state automata.Yalcin and Boucher [15] also used finite state automata indeadlock avoidance.

The main difference between our work and the one justdescribed is that we produce a software program that cancontrol and simulate the system. This program can be exe-cuted on a computer. The control-theoretic work only pro-duces a formal description of the system. A formal descrip-tion of a discrete event system serves as a way to analyzeand understand the dynamics of the system. Our work as-sumes one already has an understanding of the system. Alogical sequence may be to use the control-theoretic re-search to understand the system and then use this knowl-edge to build a software model of the system using a soft-ware tool such as the one presented here. For example,Zeigler et al. [16] introduced DEVS-C++ as a simulationtool that is based on using DEVS models. A DEVS-C++model is created from the DEVS specification and executedon a computer. This model is used to simulate a continu-ous system using discrete event simulation as opposed tousing discrete time simulation. It serves as a good exampleof using a control-theoretic model to gain an understand-ing of the system and then create a software model for thesimulation based on this model.

The following research efforts are more closely relatedto our work. Peters et al. [17], Smith et al. [18], and Smithand Peters [19] have adapted ARENA to control their ex-perimental FMS. However, unlike the modeling approachto be discussed in this paper, their control architecture islimited to only one hierarchical level in which a single su-pervisor, the cell controller, manages a set of subordinateprocesses. In their modification of ARENA, they have in-cluded special events to facilitate message passing amongthe controllers. Narayanan et al. [20], Govindaraj et al.[21], and Bodner and Reveliotis [22] have also developed

2 SIMULATION Volume 78, Number 9

A NEW SIMULATION TOOL FOR DISTRIBUTED SYSTEMS

simulation tools that are capable of controlling a system.They claim that the typical approach to address the com-plexity of FMS control systems is a hierarchical decompo-sition. A widely used decomposition features tree layers,strategic decisions, tactical decisions, and operational de-cisions [23, 24]. Bodner and Reveliotis, Govindaraj et al.,and Narayanan et al. claim that to handle lower level is-sues involving real-time control, two additional layers areneeded. The structural and equipment control layers ad-dress issues such as deadlock avoidance. In their research,hierarchical decomposition refers to the logical decompo-sition of the decision-making process, while we refer toit as the actual decomposition of the controller that yieldsa distributed controller. Bodner and Reveliotis do not ac-tually consider distributed systems. Incidentally, they alsoclaim that FMS exist but that there are no controllers forthem.

3. The Concepts

The following are two of the concepts on which our tool isbased. The first one, the hierarchical architecture, describeshow a control architecture can be decomposed to simplifyeach component. The second concept, the distributed mod-eling methodology, tells how to simulate such a distributedsystem.

3.1 The Hierarchical Architecture

The concept of the hierarchical architecture was originallypublished in Tirpak et al. [25]. The architecture introducesthe coordinated object, CO, as the basic element for mod-eling an FMS or other DESs. Each coordinated objectrepresents a basic hierarchical element in which intelli-gent control or integrated planning and control are to beaddressed.

Using this architecture gives rise to multiresolutionalcontrol. At the topmost level, the coordinated object’sscope of control includes the most aggregate level of deci-sion. The planning horizon may be very long, perhaps look-ing several months or years ahead. This coordinated objectmakes high-level decisions. This object can be thought ofas the CEO of a company. The controller in the middlelevels has a smaller planning horizon but is still makingmanagerial-type decisions. This object is similar to themiddle-level managers at a company. At the bottommostlevel, the controller has the smallest planning horizon andis limited to the most immediate concerns. These decisionsare primarily concerned with the adjustment of machine-related parameters such as the speed of a certain motor orthe amount of pressure at a valve. This object is similar tothe factory workers at a company.

3.2 The Distributed Modeling Methodology

The simulation of distributed systems discussed here isbased on the distributed modeling methodology—referred

to as the Hierarchical Object-Oriented ProgrammableLogic Simulator (HOOPLS)—in Davis, Macro, and Set-terdahl [26]; Davis et al. [2]; and Davis [3]. This method-ology is based on the belief that the interactions among thecontrollers must be considered by the simulation modelto accurately model a system with a distributed controlarchitecture. The single most important characteristic ofthe methodology and what separates it from other object-oriented simulation approaches is its attention to modelingthe flow of messages among the controllers included withinthe architecture. By modeling the flow of messages, themethodology allows the simulation to accommodate dis-tributed models. This has two advantages: the simulationis more accurate since it includes all of the communicationamong the distributed controllers, and the same distributedmodel can be used for simulation as well as control. Beingable to use the same model for both control and simulationhas several advantages. First, the model has the maximumfidelity since the same model is actually used to control thesystem as well as for simulation. Second, the model doesnot have to be written since the model used for control canbe used for simulation without modification. Using thismodeling methodology, the state definition of the systemis easy to determine. It is simply the contents of the vari-ables and data structures in the model and the software tool.Since both the model and the software tool are the same forboth simulation and control, the states are compatible [2].

4. The Modeling Framework

Each included modeling element is represented as an ob-ject in programmed C++. As with most discrete event sim-ulation approaches, the simulation is event driven, and anexecutive object manages sequential processing of events.A scheduled-event list is maintained in which the sched-uled events are stored in chronological order based on theevent times when they are scheduled to occur.

Our simulation tool, like most simulation tools, consistsof three basic parts:

1. the executive function, which actually manages thesimulation while operating under the constraints im-posed by the system model;

2. the data structures, which include the scheduled-event list (a list of events scheduled to occur in thefuture), the pending-event list (a list of events await-ing a response from controlled hardware to occurbefore they can be processed), and the resource list;

3. the set of modeling elements, which define the sys-tem model.

The simulation executive function manages the chrono-logically ordered scheduled-event list, controls the pro-cessing of the events as they are removed from thescheduled-event list, schedules new events to be placed

Volume 78, Number 9 SIMULATION 3

Gonzalez and Davis

on the scheduled-event list, and manages the allocation ofresources. After each event is processed, the executive ob-ject removes the event with the smallest event time from thescheduled-event list and then invokes the proper functionfrom the model to manage the event’s execution. Given thestreamlined approach that we have adopted, there are twobasic types of events that occur: (1) QUESEIZE, in whichthe event occurs as a result of resources becoming avail-able for assignment to a requesting entity, and (2) DELAY,in which the event occurs when a processing task is fin-ished. Other types of events that also provide an expandedmodeling flexibility for this simulation environment are notconsidered in most simulation tools, including the PEND-ING, INPUT, ERROR, and TIMEOUT elements.

In this tool, we tried to stick with the traditional hold-seize-release modeling elements used by the popular com-mercial simulation languages such as ARENA instead ofusing a more natural mapping. Since negotiations are be-ing made to produce a commercial product from this tool,it is important to the success of the tool that the modelingelements show some familiarity to that found in the cur-rent simulation tools. It is our goal to appeal to the currentmodeling community.

4.1 The System Model

The system model function must be specified to allow theexecutive object to implement the simulation. The execu-tive object assigns a unique event number for each possibleevent that can occur. When it calls the system model, itpasses the event number to the system model. The modelfunction next uses this event number to determine whichsubfunction of the model to execute. Thus, as defined here,the model function simply represents a large case statementor switch statement with the event number being the switchvariable. The included model also carefully delineates theactions taken by the system’s controller when a given eventactually occurs during the execution of the real system. Be-cause the time it takes the controller to process an eventis negligible when compared to the time that the systemrequires to perform a typical task, time does not advancewhile executing an event.

Logically, an event occurs when an entity residing inthe system needs to change its state. For example, a parthas finished processing and needs to be removed from themachine. Our simulator changes the state of the entity byexecuting events as specified in the model, until the en-tity can no longer change its state without either waitingfor some resource or for a given task to be completed.When no further events can be processed on the given en-tity, program control is returned to the executive function.Modeling elements are used to model the waiting periodsthat occur in the real system. The QUESEIZE modelingelement is used to model the waiting period that occurs asa result of an entity waiting in a queue for one or morerequired resources. The DELAY element models the delay

period involved in the execution of task on an entity by amachine or other server.

Discrete event systems usually have entities flowingamong their physical elements whose flow must be bothexplained and coordinated. The entity may contain a serialnumber, locations to store attributes that describe its state,a list of elements to store the process plan if the entity rep-resents a part, and an address stack that stores the returnaddress for program control purposes. This return addressdefines which segment of code the program should returnto once the currently selected segment of code has beenprocessed.

Each modeling element object also contains two pub-lic functions. The first function is the insert function. Thisfunction permits the designer to insert an entity into anymodeled element object within the model through the useof C++ code. Inserting an entity into an element causesthe element to execute by either scheduling an event on thescheduled event list or declaring itself as needing a set of re-sources. Either way, the current stream of event processingon an entity can no longer continue executing, and pro-gram control must be returned to the executive function.The entity that was being processed thus remains storedinside of the modeling element into which it was inserteduntil it is removed at some event time that is scheduled tooccur in the future. When this event occurs, the entity isthen removed from the modeling element using the sec-ond public function, the remove function. Processing ofthe event resumes and continues until the entity is againinserted into another modeling element. Therefore, the en-tity flows from one modeling element to another over thecourse of the simulation.

In our simulation tool, the designer can also insert a setof C++ statements in-line between any two modeling ele-ments. The C++ code can then perform any function on theentity that is required, including tallying statistics from itsattribute list or returning resources allocated to the entity.Note that neither of these actions have consequences onthe evolution of time. The modeling element that followsthe last statement of the inserted C++ code will then re-ceive the entity after the intervening C++ statements havebeen executed on it. The processing of an event ends whenthe next modeling element is encountered. At this point,the entity is stored inside of the modeling element’s objectuntil the event associated with the entity residing withinthe modeling element occurs.

Each modeling element object is instantiated from anobject subclass for the element type. A feature of all ofthese element subclasses is their ability to store (queue)the pointer of the entity that enters the element. Within asystem’s model, a given modeling element may be usedseveral times. For example, a particular model may haveseveral DELAY elements. Each use of an element withina model will assign a unique label, which will also be at-tached to every event that is instantiated using that ele-ment. For example, the seventh QUESEIZE employed in



the model would be assigned the label QUESEIZE7. A DE-LAY32 label would be attached to the 30-second DELAYelement included in the model.

The following is an example of a DELAY37 event. Thisevent is executed when the cart arrives at the machine witha part; the delay models the movement of the part from itsprevious location to the machine. Notice that the first stepin the event is to remove the entity from the DELAY37element, and the last step is to store the entity inside of theQUESEIZE42 element’s object. By inserting the entity intothe QUESEIZE42 element, the entity requests the machineand waits until the machine becomes available.

case DELAY37:ent = MoveCart_d37.remove();

.

.

.GetMachine_q42.insert(ent);break;

Note: MoveCart_d37 is the name of the DELAY37modeling element’s object; likewise, GetMachine_q42 isthe name of the QUESEIZE42 element’s object. The vari-able ent is the pointer to the entity’s object.

4.2 The Basic Simulation Engine

The simulation engine is implemented as an object, a keyfeature that allows the simulation tool to address distributedsystems. That is, distributed systems are simulated by in-cluding the simulation object for each distributed controllerinto a single program. Therefore, it is necessary to haveeach simulation engine encapsulated into an object.

The simulation executive object contains several datastructures that are used to maintain the simulation’s stateinformation. The first is the scheduled-event list. It is achronologically ordered list of events that are scheduled tooccur in the future. The scheduled-event list objects containthe type of event (e.g., DELAY), an identification index(e.g., 37 for DELAY37), and the time that the event isscheduled to occur. Second, there is a pending-event list,which stores events requiring a response from controlledequipment. Next, there is the resource object, which storesthe resources as a table of resource entities in which eachentity represents a type of resource. The user defines thelist of resources as part of the model specification. Thus,the resource object contains the resource name and thequantity for each resource type that is available.

Finally, a simulation executive object contains a pointerto its model function and a pointer to its communicationfunction, which will be discussed later. Given these datastructures, each executive object manages the scheduled-event list and the pending-event list, controls the process-ing of the events, and manages the allocation of resources.Finally, the executive object contains an executive func-

tion, which uses the data structure objects, as well as thetwo pointers to the model and communication functions,to manage the simulation (see Fig. 1).

The executive function runs in a cycle. The cycle be-gins by pulling the next event—the event with the earli-est scheduled time—off the scheduled-event list. It thensets the current simulation time to the event’s time of oc-currence. Next, the event is executed by calling the modelfunction provided by the user and passing it the event num-ber. Once control is given back to the executive function,it checks each of the entities that are waiting for resourcesto see if the required resources for any requesting entitybecame available with the execution of the last event.

The entities waiting for resources are stored inside ofthe QUESEIZE modeling elements. The executive objectcontains a list of all of these modeling elements. To checkto see whether any entity can be allocated its needed re-sources, the executive function traverses the list of QUE-SEIZE elements, checking each element until it finds anentity to which it can allocate all of the resources that it hasrequested. At this point, the corresponding QUESEIZE ele-ment is executed. Because of the possibility that processingof an event might release one or more allocated resources,the list of QUESEIZE elements must be checked wheneverany event is executed. To eliminate any QUESEIZE ele-ment priority over another for the allocation of resources,every time the executive function checks the list of QUE-SEIZE elements, it begins with the last element that waspreviously checked. Once every QUESEIZE element inthe list is traversed without any additional events occur-ring, the executive function completes the checking of thelist of QUESEIZE elements and continues in its primaryexecution cycle.

Note that all of the subsequent events that were executedas a result of resources becoming available were triggeredby the initial event that was pulled off the scheduled-eventlist at the beginning of the cycle. Thus, all of these eventsoccur at the same instant in time. That is, the system clockhas not been incremented. When no more events can occurat the current instant of time, the current event processingcycle is finished and a new cycle begins.

The following is the pseudo-code for the executivefunction:

While the scheduled-event list is not emptyGet the next event from the scheduled-event list.Update the system clock with the event’s time.Execute the event by calling the model function andpassing it the event number.Visit each QUESEIZE element that has at least oneentity in it.

If this element can allocate the resources requestedthen

Execute the corresponding QUESEIZE event.Until every QUESEIZE element in the list is checkedwithout causing an event to execute.


Gonzalez and Davis

Simulaton Executive Object

Event List• Schedule event• Get next event• Get next event

time

Pending Event List

• Insert event• Check and

remove event

System Clock

Resources• Check and Seize• Release• Query resources• Statistics

Pointer to modeling function

Pointer to communication function

• Initialize resources

• Run Simulation

• End Simulation

Modeling

Function

Communication

Function

Figure 1. The executive object. The circles represent the data structure objects, the square represents the executive function, andthe flat rectangles represent pointer variables.

4.3 The Basic Set of Modeling Elements

In the development of this modeling environment, only aminimal set of modeling elements has been included. Thedecision as to which elements were to be provided wasbased on whether the object required an event to be sched-uled on the scheduled-event list or required the allocationof resources. Because our modeling environment will pro-vide an increased capacity to include C++ programmingelements, we decided to provide for the other modelingrequirements, such as the TALLY element, by defining alibrary of C++ functions. Furthermore, some of the simplermodeling elements, such as COUNT, BRANCH, and AS-SIGN, are not included in the library of functions becausethey can be trivially implemented in C++.

• DELAY: This element is used to model task processing inthe real system. As an object, the DELAY element con-sists of a priority queue used to store the entities’ pointer,a pointer to a random-number generator that can generaterandom numbers from a prespecified probability distribu-tion, and an event label. The priority among the entitiesin the queue is ordered by the entities’ remaining delaytime. This way, the order of the entities in the queue issynchronized with the order of the DELAY events that are

scheduled to occur for that particular DELAY element.The random-number generator is used to compute the de-lay time. When an entity is inserted into a DELAY ele-ment, several things happen. First, the element generatesthe delay time, computes the time when the delay will fin-ish, and stores the pointer to the entity in its queue usingthe time when it will finish its delay as the priority. TheDELAY element then schedules a delay event for itself inthe scheduled-event list.

• CREATE: This element is similar to the DELAY element,but it functions automatically without an entity flowinginto the element. When a CREATE event occurs, it auto-matically creates a new CREATE event and schedules iton the scheduled-event list. This element can be employedto model the continued flow of entities entering a system.

• QUESEIZE: This element is used to model a physicalqueue in the real system. It contains a queue to store thepointers of the involved entity objects and an element la-bel. When an entity is inserted into the QUESEIZE ele-ment, the element simply puts the entity into the rear of thequeue. No event is scheduled because the entity is waitingfor resources to become available.

• RELEASE: This element releases the allocated resourcesso that other entities can have access to the released re-sources. Care must be taken not to release resources that



were not allocated. If this situation should occur, an errorcondition will cause the simulation to halt.

• SEIZE: Unlike the QUESEIZE element, this element doesnot contain a queue. It simply seizes the requested re-sources immediately. The resources must be available oran error will occur, causing the simulation to halt. This el-ement is used to initialize the status of the resources in thecase when some of the resources are seized at the initialstate of the simulation.

• CREATECOPY: This element creates an identical copy ofthe entity passed to it, except the created entity is issued anew serial number.

• TALLY: This element is used to find statistical informa-tion on certain variables. Every time a TALLY element iscalled, the variable passed to it is averaged into the averageof all of the variables sent to that TALLY element so far.Other statistics are also maintained. When the simulationfinishes, the TALLY element gives the average, the stan-dard deviation, the maximum value, the minimum value,the final value, and the number of observations of all thevariables that the TALLY element was given.

In addition to the other elements discussed, two ele-ments provide recursive function-call capabilities. Theseelements are presented in detail in Gonzalez [27] and willnot be discussed here.

5. The Control and Communication Support

The original desire was to develop a tool that would bothsimulate and control the modeled system. To accomplishthis goal, the controller must communicate with the phys-ical machine as well as with a supervisory agent. There-fore, two new modeling elements have been added. ThePENDING element has been included to allow the modelto synchronize itself with the hardware, and the INPUTelement allows messages to enter the controller from anexternal agent.

• PENDING: The PENDING element uses a new event listcalled the pending-event list, which has been included tosupport control applications. The pending-event list is anunordered list of events that are awaiting responses fromthe hardware before they can occur. There are no specifiedevent times associated with these events. When the hard-ware finishes a task, it responds to the controller with amessage. A communication function, which must be sup-plied by the modeler, retrieves a key from this message andsends this key to the executive object, where it is comparedto each PENDING event in the list. If a PENDING eventwith the same key is found, the event is removed fromthe list by the executive function and executed by pass-ing it to the model function. If a match is not found, thenthe message is assumed to be new, and an INPUT eventis executed instead. An INPUT event occurs when a newunexpected message enters the system.

• INPUT: The INPUT element is executed when no matchis found in the pending-event list for an incoming mes-sage. The message is assumed to be a new message andnot a response from the hardware. The INPUT modeling

element creates a new entity. This is needed because a newmessage may represent the assignment of a new entity ortask to the controller.

In control mode, the executive function cycles in muchthe same way as in simulation mode, except for the follow-ing changes. First, the clock must run in real time instead ofsimulated time. That is, instead of extracting the next eventfrom the scheduled-event list and advancing the simulatedtime to the time that event is to occur, as is done in thesimulation mode, the controller waits until the next eventtime occurs before it executes the event. So, if the nextevent is to occur at 5:00, the controller waits until 5:00before it executes it, as opposed to simply setting the cur-rent simulated time to 5:00. The controller gets the currenttime from the system clock. Note that while the executivefunction is waiting for the time to execute the next event onthe scheduled-event list, a message from the hardware mayarrive. This may cause an event from the pending-event listto be executed before the next event on the scheduled-eventlist is processed. So the next event on the scheduled-eventlist may not be the next event to occur.

One may wonder why there is a scheduled-event list incontrol mode. One of the reasons is that some of the oper-ations may not be implemented and need to be simulatedusing this event list. This is the case when the hardware rep-resents a physical emulator (pilot plant) for another real-world system. Also, even if the system being controlledis a real system with no parts being simulated, there maystill be a need to schedule events into the future. It may bethe controller’s responsibility to determine when a processneeds to finish. For example, one may want to bake a partin an oven for a prescribed amount of time. The scheduled-event list can then be used in a fashion that is similar to anegg timer.

The following is the pseudo-code for the executive func-tion running in control mode. Note that at the point wherethe cycle would normally finish in simulation mode, thecommunication lines are checked for incoming messagesby calling the communication function. This function mustbe supplied by the user along with the system model. Be-cause executing a PENDING event might release someresources, once control is given back to the executive func-tion, all of the QUESEIZE elements must be rechecked.

While the scheduled-event list and the pending-eventlist are not emptyGet the next event’s time from the scheduled-event list.If the event’s time is earlier than or equal to the currenttime

Execute the event by calling the model function andpassing it the event number.

Visit each QUESEIZE element that has at least oneentity in it.

If this element can allocate the resources requestedthen

Execute the corresponding QUESEIZE event.


Gonzalez and Davis

Until every QUESEIZE element in the list is checkedwithout causing an event to execute.Execute the communication function.If a key is returned

Search the pending-event list for a key match.If a match is found

Execute the PENDING event.Else

Execute the INPUT event.Visit each QUESEIZE element that has at least oneentity in it.

If this element can allocate the resourcesrequested then

Execute the corresponding QUESEIZEevent.

Until every QUESEIZE element in the list ischecked without causing an event to execute.

For an example of how this algorithm works, considerthe case in which the next event in the scheduled-event listis a DELAY event. The executive function first pulls thisevent from the list. Next suppose the event’s time is earlierthan the current time, and therefore the executive functioncalls the modeling function with the DELAY event. Whilethe model executes the DELAY event, it releases the oven.Once the model returns control to the executive function,the executive function then checks all of the QUESEIZE el-ements searching for an event that is waiting for resourcesthat are now available. Continuing the example, supposethat a QUESEIZE module is waiting for the oven. Whenthis QUESEIZE element is checked, the oven is seized, andthe executive function calls the model to execute the QUE-SEIZE event. In this event, the model releases the hold-ing spot of the part waiting for the oven. After the eventis executed, the executive function continues checking allof the QUESEIZE elements, including the ones alreadychecked, until all of them have been checked and none cancontinue execution until further resources are released. Itrechecks from the beginning since now a holding spot isavailable instead of the oven. The next step for the exec-utive function is to check the communication lines for anincoming message. Suppose that a message arrived and itwas a response to a PENDING event. The executive func-tion finds the matching event in the pending-event list andcalls the modeling function with the corresponding PEND-ING event. While executing this event, resources may bereleased, so once control is given back to the executivefunction, it must recheck all of the QUESEIZE elementsagain.

In the simulation mode, the PENDING element is pro-cessed as a DELAY. Recall that the PENDING element isused to wait for the physical hardware to finish a task. Insimulation mode, the actual processing does not occur, anda DELAY element is employed to model the delay asso-ciated with processing a given task. This feature providesone essential key that allows the same model to be executed

in both a simulation and control mode. The user need onlyset the mode of operation, and the model will either con-trol or simulate the system. The primary difference in theexecution of the model under each mode of operation isthat in the control mode, the PENDING element insertsa PENDING event into the pending-event list, while inthe simulation mode, DELAY events that are generated bya corresponding PENDING element are inserted into thescheduled-event list. Note also that in control mode, theDELAY element can still operate as a DELAY element.However, the delay is measured in real time as opposed tosimulated time as when the model is in simulation mode.

The following is an example of a QUESEIZE9 event.The event is executed when the machine requested by theentity is seized and processing is to begin. If running in con-trol mode, the controller sends a message to the machineto begin processing. It is the responsibility of the modelerto include the code that is needed to send commands to thehardware and provide checks to ensure that the commandsare not sent in simulation mode. These requirements canbe easily addressed in C++. The entity is stored into thePENDING35 element until the machine responds to thecontroller indicating that it has finished processing. At thispoint, a PENDING35 event occurs. If running in simu-lation mode, a message is not sent to the hardware, andthe PENDING35 element acts as a DELAY element byscheduling an event into the scheduled-event list.

case QUESEIZE9:ent = Machine_q9.remove();if (control_mode)

send_msg(“PROCESS”);Process_p35.insert(ent);break;

case PENDING35:ent = Process_p35.remove();

5.1 The Error Handling Support

Two types of errors can occur: time-out errors that are dueto a device not responding and error messages sent by adevice.

The time-out error is an optional feature that can be usedin conjunction with the PENDING element. The PEND-ING element handles this type of error by using the chrono-logically ordered scheduled-event list. The time-out erroroccurs when a pending message does not arrive within agiven time limit. If the modeler wants to use the time-outerror-handling feature, then when inserting the entity intoa PENDING element, a time limit must be passed as well.The PENDING element then schedules a TIMEOUT eventinto the scheduled-event list. This event behaves like a DE-LAY event, but it is used only for time-out error handling.

If the executive function executes the TIMEOUT event,this means that the pending controller did not respondwithin the given time limit. The user must then retrieve



the entity from the corresponding PENDING element byusing its time-out public function instead of its removefunction. This causes the PENDING element to removethe pending event from the pending-event list instead ofremoving the TIMEOUT event from the scheduled-eventlist, as is done by the remove function. Like the removefunction, it also returns the entity stored in the PENDINGelement.

On the other hand, if the pending response arrives withinthe allowed time limit, then the PENDING element mustremove the TIMEOUT event from the scheduled-event listto prevent it from occurring at its scheduled event time.This is done by the PENDING element’s remove functionwhen the time-out feature is used. The TIMEOUT eventthat corresponds to a PENDING element has the same iden-tification number. That is, a PENDING37 element maycause a TIMEOUT37 event to occur.

The other type of error handling occurs when an errormessage arrives from another controller or from a physicaldevice. These error messages do not correspond to a par-ticular job. For example, it may be a message indicatingthe machine is out of some resource. The error messagesdirected to a particular job will use the regular communica-tion channels but will contain the error message instead ofthe response message that it normally sends. The modelercan prepare for the arrival of an error message by insertingan ERROR event into the pending-event list along with anerror key using the ERROR element’s insert function. Noentity is inserted because these messages do not correspondto an entity. This insert function also does not determinethe end of an event. Instead, this function is called dur-ing model initialization. Once the ERROR event occurs,no entity is removed from the element; instead, anotherentity (not representing a job) may be created, if needed,to handle the error. The executive function searches thepending-event list with a key extracted from the messageevery time a message arrives. The modeler must determinea key for each error message and use this key when insert-ing the ERROR event into the pending-event list. Using theerror message itself as the key is recommended. The serialnumber can be set to NULL to direct the communicationfunction to use the message itself as a key. This techniqueis used to communicate with the physical devices that donot keep track of job serial numbers.

• ERROR: This element works much like the PENDINGelement, with the exception that the ERROR event in thepending-event list is not inserted or removed dynamicallyduring execution. Instead, the ERROR events are insertedduring system initialization and remain in the pending-event list throughout the simulation. They are also notremoved when an error occurs. Furthermore, the pending-error messages do not count as messages waiting to arrive.This fact becomes important in determining when to quitexecution.

In the following example, the use of the TIMEOUTerror-handling option is added to the previous example.

If the machine responds within the time limit of 50 min-utes, the PENDING event occurs; otherwise, the TIME-OUT event occurs, and an error message is sent to thesupervisor indicating that the device did not respond. Notethat the TIMEOUT event number must be the same as thecorresponding PENDING element’s number.

case QUESEIZE9:ent = Machine_q9.remove();if (control_mode)

send_msg(“PROCESS”);Process_p35.insert(ent, 50);break;

case PENDING35:ent = Process_p35.remove();...break;

case TIMEOUT35:ent = Process_p35.time_out();supervisor.send_msg(“Device not responding”);...break;

6. An Example

The following is an example using this developed simula-tion tool. The model on the left simulates or controls, de-pending on the mode of operation, a simple M/M/1 queuewith an arrival rate of 4.4 time units and a service rate of 4.2time units, both from an exponential distribution. On theright is the equivalent simulation written in SIMAN (Sam-ple Program 3.1 from Introduction to Simulation UsingSIMAN [28]). Note that since the SIMAN language doesnot have the capability to control a system, this SIMANexample is equivalent to the model using our tool only ifyou consider our tool operating in simulation mode.

The public function get_TNOW() returns the simulationtime. The simulation starts by calling the function sim(). Itsparameter is the time when the first create event will occur.Before the simulation can begin, the resources must be ini-tialized using a function provided by the executive object.An array consisting of the resource list must be passed toit. The executive object also provides a function to printthe results of the simulation. The RandomExp object gen-erates exponentially distributed random numbers used forthe arrival and service times. The QUESEIZE, DELAY,and CREATE objects are the modeling elements that areassociated with events. The RELEASE and TALLY objectsare supporting modeling elements. The model is simply aswitch statement along with the declaration of the model-ing elements.


Gonzalez and Davis

Note that in simulation mode, this model will performa simulation of the M/M/1 queue exactly as the SIMANmodel does because in this mode, the INPUT element func-tions as a CREATE element, the PENDING element func-tions as DELAY, and the TIMEOUT element is ignored.

#define MACHINE 0int JobsDone = 0;ResourceStruct resources[] = “Machine,” 1; // 0RandomExp arrival (1 / 4.4);RandomExp service_time ( 1 / 4.2 );QueSeize machine_q1 (“Machine,”

MACHINE, 1);Release machine_r1 (MACHINE, 1);Pending service_p1 ( &service_time );Input arrivals_i1 ( &arrival );Tally TIS_t1 (“Time in System.”);voidperform( int EventNumber)

{Entity *ent;switch (EventNumber)

{case INPUT1:

if (get_TNOW() > 480)break;

ent = arrivals_i1.create();ent->attr[0] = get_TNOW();machine_q1.insert(ent);break;

case QUESEIZE1:ent = machine_q1.remove();if (control_mode)

machine.send_msg(“Start”);service_p1.insert(ent, 50);break;

case PENDING1:ent = service_p1.remove();machine_r1.release();TIS_t1.tally(get_TNOW() - ent->attr[0]);JobsDone_ent++;delete ent;break;

case TIMEOUT1:ent = service_p1.timeout();supervisor.send_msg(“TimeOut”);delete ent;break;

default:cout « “Oops. Did a boo boo\n”;break;

}}

void

main(){InitializeResources(resources);sim( arrival.get() );PrintResults();}

BEGIN1 CREATE: EXPONENTIAL( 4.4 );2 QUEUE, Buffer;3 SEIZE: Machine;4 DELAY: EXPONENTIAL(4.2 );5 RELEASE: Machine;6 COUNT: JobsDone: DISPOSE;

END;

BEGIN1 PROJECT, SampleProblem3.1, SM2 DISCRETE, 100;3 QUEUES: Buffer;4 RESOURCES: Machine;5 COUNTERS: JobsDone;6 REPLICATE, 1,0,480;

END;

7. Simulation of Distributed Systems

This section discusses the features that are contained withinthe simulation tool that allow distributed models to be usedfor simulation. Please note that although we are simulatinga distributed system, the simulation itself is not distributed.

To simulate distributed systems, the simulation of eachsubsystem must continue to be managed by its executivefunction using its own data structures as mentioned above.All of these components are encapsulated into a single ob-ject, called the simulation object. This feature allows theinclusion of several instantiations of the simulation objectinto a single program, possibly one for each modeled sub-system. This feature is essential because a distributed sys-tem is typically managed by a hierarchy of controllers. Tobuild the simulation model, one simply includes all of thecontrollers into a single program without considering theneed to duplicate variable and function names. Obviously,only one of the simulation objects can be executed at agiven time because there is only one computational threadin the programming approach that we have adopted. To em-ulate all of the simulation objects operating concurrently,the HOOPLS function executes one simulation object foran appropriate time and then switches to another object.The HOOPLS function uses its global event list and anincluded message relay to determine which simulation ob-ject will execute next. The individual simulation objects,on the other hand, determine when to return control backto the HOOPLS function.

Recall that in the simulation of a nondistributed system,the executive function constantly cycles in a loop. This loopstarts by removing the next event from the event list andends when there are no messages waiting to be processed



and all queued events are waiting for resources. To permitseveral executive objects to operate concurrently, the ex-ecutive function was modified so that it cycles through theloop only once every time it is called. This modificationwas accomplished by adding two functions: one to executea single cycle in the loop excluding the processing of mes-sages and the other to process an incoming message. TheHOOPLS function can then call either function dependingon whether the object needs to process an event from theglobal event list or a message from the message relay.

In the simulation of distributed systems, a distinctionbetween the simulation and control modes does not ex-ist. The simulation objects are always executed in controlmode. In simulation mode, used for nondistributed sys-tems, there is no other object in the program so no commu-nication occurs. The PENDING elements behave like theDELAY elements. However, when simulating distributedsystems, since other objects are included, communicationmust occur. Recall that including these messages in thesimulation is the main objective of the distributed model-ing methodology. Here the PENDING elements functionas they do in control mode by waiting for a message toarrive before executing a pending event. So even thoughthe objects are being used for simulation, they are runningin control mode. The other aspect of control mode is theway the clock advances. The current time is given to eachobject when it is requested to execute an event. The objectupdates its internal clock using this time, checks the timeof the next event, and executes it if the event time is earlierthan or equal to the current time. Of course, the current timeand the event time will always be equal since the HOOPLSfunction updates its clock to the time of the next event be-fore calling the object. The ability of the HOOPLS functionto perform this check results from the fact, discussed later,that the HOOPLS function has a global event list that in-cludes the events that are contained within the event list ofeach individual executive object. Mechanically, however,each object treats the time as it does in control mode bywaiting for the clock to advance by itself. The HOOPLSfunction is the one that updates the time in discrete incre-ments, as is commonly done in simulation, and then tellseach object the new time.

Another special feature of our simulation tool is themanner by which messages enter the model. Instead ofchecking the communication lines every time the execu-tive function reaches the end of an event-processing cycle,the lines are not checked at all. Rather, the HOOPLS func-tion manages the flow of messages. Whenever a message ispresent, the HOOPLS function calls the executive object’sfunction to process a message and provides it with the mes-sage. The simulation object will then handle the messageby executing the proper events in the same way that it pre-viously handled a message when it was responsible for de-tecting the message arrival. We adopted this approach to en-sure that a single entity—namely, the HOOPLS function—was responsible for monitoring the arrival of all messages.

In this manner, the HOOPLS function also serves as themessage server for handling communication among thecontrollers.

The last feature in our simulation software is the DE-LAY element. When a DELAY element is executed, in ad-dition to scheduling a DELAY event onto the submodel’sscheduled-event list, it also automatically schedules a DE-LAY event onto the HOOPLS global event list. Using thisapproach, the HOOPLS function maintains a chronologi-cally ordered, global event list with all of the events asso-ciated with every simulation object. The HOOPLS func-tion uses this global event list to determine which simula-tion object will be invoked next. Since the scheduling ofevents into this global event list occurs automatically, theuser does not need to consider this global event list whenmodeling a local delay. From the user’s point of view, thedelay is modeled in the same way as if the model werenot a member of a collection of distributed simulation ob-jects. As a matter of fact, no part of the submodel has tobe modified to accommodate the distributed simulation.Each submodel executes as if it were running on its owndedicated computational thread. This is important becauseit allows modelers to implement a distributed model us-ing the knowledge they already have in building singlemodels.

7.1 The HOOPLS Executive Function

The HOOPLS function is a supervisory function that man-ages the execution of all of the submodels’ simulation ob-jects. The HOOPLS function contains a global event listand a message relay. This function runs in a cycle muchthe same way the simulation objects do when running in-dependently (i.e., not as part of a distributed simulation).The cycle starts by removing the next event from the globalevent list and passing it to the appropriate submodel forexecution. While a submodel is being executed, the mes-sage relay, which acts as the network, receives all of themessages that are outputted by the submodel that is cur-rently executing. These messages are stored in the rearof the message queue, which is used to implement themessage relay. Once the submodel finishes executing itscurrent processing cycle, the HOOPLS function removesthe message that is at the front of the message queue andpasses it to the recipient submodel. The submodel then re-ceives the message and executes the appropriate functionsthat are needed to handle the message. Additional mes-sages may then be generated. These messages are againinserted at the rear of the message queue. Once control isreturned to the HOOPLS function, it removes the next mes-sage from the front of the message queue and recycles itthough the message-processing loop. This procedure is re-peated until no messages remain in the message queue. Atthis time, the HOOPLS executive function finishes its cycleand begins the next cycle by removing the next event in theglobal event list and passing it to the appropriate simulationobject.


Gonzalez and Davis

In the executive function cycle of an individual sim-ulation object, all of the events that are executed in thatcycle occur in an instant of simulated time. For the caseof the HOOPLS executive function cycle, all of the eventsthat are executed in this cycle also occur in an instant ofsimulated time. Note that this cycle may include several ofthe individual object’s executive function cycles, so evenmore can happen in an instant of simulated time. If exe-cuting an event produces a message, then that message issent to another object for execution. This happens withinthe same HOOPLS executive function cycle; therefore, italso occurs in the same instant of simulated time. This isacceptable if the time it takes a message to travel in the net-work from the source object to the destination is negligiblewhen compared to the duration times of the other events.This is the case for our physical emulator that is used asa test bed for our simulator/controller. We use a local-areanetwork that transmits messages at a very fast rate.

We have also made provisions for modeling situationsin which the message-passing time is not negligible. Ifthese message-passing times are significant, the modelercan schedule the arrival of a message onto the global eventlist. In this case, the HOOPLS executive function’s cyclewill not get the next message from the message relay atthe end of its cycle. Instead, it skips that part of the cycleand continues with the beginning of the next cycle, when itpulls the next event from the global event list. This event,however, may be a message arrival event, in which case thefunction gets the next message from the message relay andpasses it to the destination object for execution. Thus, sim-ulating network delays simply introduces message arrivalevents into the global event list and leaves the execution ofmessages to be managed by the global event list.

The events in the global event list tell which submodelwill address the event and the time at which the event oc-curs. When an event is pulled off the list, the simulated timeis then advanced to the next event’s event time. The eventtype is not recorded on the global event list, as this infor-mation is contained within the submodel’s local scheduledevent list and need not be duplicated. The HOOPLS func-tion then calls the appropriate submodel and passes it thecurrent simulated time. The submodel, knowing that it is re-sponsible for executing the next event, pulls the next eventoff its local scheduled event list and executes it. As statedabove, it executes its executive function’s event-processingcycle only up to the point when the communication linesare to be checked. At this point, program control is returnedto the HOOPLS function, which then checks the messagerelay and continues its cycle (see Fig. 2).

8. An Example Using the Tool for DistributedControl of a Physical Emulator

The following section illustrates an example of a real phys-ical system that is controlled using a distributed model. A

simulation using the same set of distributed models for theconsidered system is also performed. Each of the submod-els for the included controllers was programmed using thesimulation tool as discussed above. In Davis et al. [29] andGonzalez and Davis [30], the physical system is discussedin greater detail, while in Gonzalez and Davis [31], theemployed simulation model is discussed.

The physical system is a model of a flexile manufactur-ing system. This emulator provides a test bed for the de-velopment of the simulation and control tool that is neededto manage the system. Note that since we are assuming thesystem is a true real FMS, we thus have modeled it using acollection of independent models. Simulating this systemwith a commercial simulation software such as ARENAwill be impossible. In fact, there is no tool that can sim-ulate the system using the collection of models. The typ-ical method that is commonly used to model distributedsystems is to create a new equivalent model that is notdistributed.

The schematic for the constructed FMS emulator and aphotograph is depicted in Figure 3.

The emulator has four processing centers, numbered 1through 4. Each processing center (PC) contains one pri-mary subordinate processing resource (the unit process)and a dedicated material handling system (MHS). Withinthe emulated FMS (see Fig. 3), another subordinate processis the fixturing center (FC). The structure and emulationof the FC are very similar to that of a PC. The FC has adedicated MHS consisting of a primary carousel capable ofholding 16 jobs and two smaller carousels for loading andunloading jobs from the automatic guided vehicle (AGV).The movement of these carousels is controlled by a dedi-cated PLC[PLS. DEFINE]. The FC has two fixturing po-sitions that represent the subordinate unit processes. Thefinal subordinate process is the cell’s MHS. AGVs are em-ployed as the primary material handlers at the cell level andare emulated with an HO-scale electric train. In this layout,more than 40 track segments can be individually powered.Sensory switches are provided on each track segment todetect the presence of an AGV.

In constructing this control architecture, 25 indepen-dent copies of the simulation tool were employed, eachwith its own model of the subsystem that it is addressingand all running concurrently (see Fig. 4). The only thingthat ties them together into a single-control architecture isthe communication among them. The communication isperformed across a local-area network (LAN) connectingseven computers where the cell controller and each of thecell’s six subordinate controllers are situated on their owncomputer. Additional communication links are providedvia RS-232 links between the FC and PC controllers andtheir dedicated PLCs.

Note that when using the model to control the system,the cell’s supervisor is represented by an independent agentthat assigns tasks to the cell controller. Thus, the cell’ssupervisor is not considered to be part of the modeled



Global Event List

Message Relay

1

2

3

Figure 2. An example of a HOOPLS (Hierarchical Object-Oriented Programmable Logic Simulator) simulation of a controllerdecomposed into three subcontrollers. Each of the three subcontrollers is simply included in the simulation without any modification.

Figure 3. Schematic for the constructed flexible manufacturing system (FMS) emulator (left) and a photograph of the emulator(right).


Gonzalez and Davis

cellmhs

pc1mhs pc1proc

pc1plc

pc1cell pc2cell

pc2mhs pc2proc

pc2plc

pc3cell

pc3mhs pc3proc

pc4cell

pc4mhs pc4proc

pc4plcpc3plc

fsmhs fsproc1 fsproc2

fsplc

fmscell

fmssup

Supervisor

Emulator Control Hierarchy

Physical Equipment

fscell

mhsplc

Figure 4. The implemented ROOCH PLS. DEFINE] control hierarchy for the physical flexible manufacturing system (FMS)emulator. The submodels that are presented between the two horizontal lines are the actual controllers used both to control andsimulate the system. The rest of the models were created to provide a complete model for simulation purposes only; they are notused in control mode.

control architecture. The PLC controllers that are manag-ing the basic processes at the lowest level of the hierarchyare not implemented as models either. Rather, the modeledcontrollers that serve as supervisors to the PLCs that aremanaging this basic process communicate directly with theprocess’s PLCs using their dedicated RS-232 communica-tion links.

In this example, the developed model controlled thephysical system. The controller was given a total of threejobs, each with two processing steps. Each processing steprequired the part to be moved to a different machine. Inaddition to the processing of the part, before and after eachprocessing step, the part had to be moved to the fixturingcenter for fixturing. The communication was performedusing a local-area network. There were four Sun Work sta-tions and six PLC controllers. The software controllerswere distributed among the four workstations. Each PLCcontrolled the hardware that was attached to it. The samemodel was then employed to run the simulation to projectthe future performance of the system given its current state.The message transcript for both the actual run and the sim-ulation run was similar. The only difference was due tothe discrepancy between the estimated and actual durationtimes. No changes had to be made to any of the models toswitch from using them for control or simulation except forthe inclusion of a “main( )” function within each controller.This main( ) function was needed when using the modelfor control since each model runs as its own independentprogram. These main( ) functions are disabled when usingthe models for simulation.

9. Conclusion and Future Work

We have shown a method for modeling discrete event sys-tems using a collection of independent models. To simulatea distributed discrete event system with a nondistributedsimulation, several features must be present in a simula-tion tool. The first is the capability of including severalsimulation objects into a single program. This is accom-plished by enclosing the simulation tool into an object sothat several instantiations of the tool can exist in a singleprogram. The second feature that a simulation tool musthave is the capability to execute only one event along withthe events that immediately occur as a reaction to the oc-currence of the event before returning control back to thesupervisory process.

We have described a simulation tool written in C++ hav-ing these features. In addition, we have shown how the toolcan control the system using the same model by simply al-tering the manner by which time is updated and the orderin which the events are executed. We presented a supervi-sory function called the HOOPLS function that, when usedin conjunction with the distributed models created underour simulation tool, permits the simulation of a distributedsystem.

The advantages of simulating a distributed system usingthe HOOPLS methodology were shown to be as follows:

1. increased accuracy by taking into consideration theinteraction among the controllers;

2. increased fidelity by using the actual controller mod-els for simulation;



3. decreased development time since one may use thesame distributed model for simulation, eliminatingthe need to create a new model;

4. increased ability to transfer the system state fromthe controller to the simulation model. Transferringthe system state is simply a matter of transferringthe data from one program to an identical copy ofthe same program without the need for conversionor analysis of the data.

The distributed system was successfully tested on ourphysical FMS emulator (see Davis et al. [32] and Gonzalezand Davis [30]). Real-time simulations of the distributedcontroller using the HOOPLS methodology were concur-rently executed along with the controller that controlled thephysical emulator using the same simulation model. Thesimulations were initialized to the current state of the phys-ical emulator before starting. This allowed the simulationsto project future predictions given its current state evenwhile it was operating. This feature will ultimately permitthe incorporation of predictive control strategies into themanagement of the system.

The next step in this research is to develop the intelli-gence that will perform the real-time scheduling and con-trol functions for each coordinated object. Our immedi-ate research plans will address the development of a real-time scheduling capability for the cell controller. An im-plemented HOOPLS simulation will be employed withinthis scheduler to perform the on-line simulations that canproject future system performance to assist in the schedul-ing process (see Davis [3, 33], Davis et al. [29], Gonzalezand Davis [34, 35], and Tirpak et al. [25]). The cell con-troller will still control the system, but it will able to dynam-ically change its control strategy to implement the currentplan developed by the real-time scheduler. A new physi-cal FMS emulator that is currently under development willserve as the test bed for the algorithmic developments. Thisemulator will be distributed across the nation with subor-dinate systems in Illinois and Florida (see Gonzalez andDavis [36] and Gonzalez et al. [37]).

10. References

[1] Kelton, W. D., R. P. Sadowski, and D. A. Sadowski. 1998. Simulationwith ARENA. New York: McGraw-Hill.

[2] Davis, W. J., D. Setterdahl, J. Macro, V. Izokaitis, and B. Bauman.1993. Recent advances in the modeling, scheduling and control offlexible automation. Proceedings of the 1993 Winter SimulationConference, pp. 143-55.

[3] Davis, W. J. 1998. On-line simulation: Need and evolving researchrequirements. In Simulation handbook, edited by J. Banks, 465-516. New York: John Wiley.

[4] Mize, J. H., H. C. Bhuskute, and M. Kamath. 1992. Modeling ofintegrated manufacturing systems. IIE Transactions 24 (3): 14-26.

[5] Zeigler, B. P. 1976. Theory of modeling and simulation. New York:Wiley-Interscience.

[6] Zeigler, B. P. 1984. Multifaceted modeling and discrete event simu-lation. London: Academic Press.

[7] Ramadge, P. J., and W. M. Wonham. 1987. Supervisory control of aclass of discrete event processes. SIAM Journal of Control andOptimization 25 (1): 000-000.

[8] Wonham, W. M., and P. J. Ramadge. 1987. On rhe supremal control-lable sublanguage of a given language. SIAM Journal of Controland Optimization 25 (3): 000-000.

[9] Chow, C. A., B. P. Zeigler, D. H. Kim. 1994. Abstract simulator forthe parallel DEVS formalism. Proceedings of the Fifth AnnualConference on AI, Simulation, and Planning in High AutonomySystems, pp. 157-63.

[10] Passino, K. M., and K. L. Burgess. 1998. Stability analysis of discreteevent systems. New York: John Wiley.

[11] Chung, S. L., S. Lafortune, and F. Lin. 1992. Limited lookaheadpolicies in supervisory control of discrete event systems. IEEETransaction on Automatic Control 37 (12): 1921-35.

[12] Rudie, K., S. Lafortune, and L. Feng. 1999. Minimal communicationin a distributed discrete-event control system. Proceedings of the1999 American Control Conference 3:1965-70.

[13] Ramadge, P. J., and W. M. Wonham. 1989. The control of discreteevent systems. Proceedings of the IEEE 77 (1): 81-98.

[14] Song, H. S., and T. G. Kim. 1994. The DEVS framework for discreteevent systems control. Proceedings of the Fifth Annual Confer-ence on AI, Simulation, and Planning in High Autonomy Systems,pp. 228-34.

[15] Yalcin, A., and T. O. Boucher. 2000. Deadlock avoidance in flexiblemanufacturing systems using finite automata. IEEE Transactionson Robotics and Automation 16 (4): 424-29.

[16] Zeigler, B. P., Y. Moon, D. Kim, and G. Ball. 1997. The DEVS en-vironment for high-performance modeling and simulation. IEEEComputational Science and Engineering 4 (3): 61-71.

[17] Peters, B. A., J. S. Smith, J. Curry, and C. LaJimodiere. 1996. Ad-vanced tutorial: Simulation based scheduling and control. Pro-ceedings of the 1996 Winter Simulation Conference, pp. 194-98.

[18] Smith, J. S., R. A. Wysk, D. T. Sturrock, S. E. Ramaswamy, G. D.Smith, and S. B. Joshi. 1994. Discrete event simulation for shopfloor control. Proceedings of the 1994 Winter Simulation Confer-ence, pp. 962-69.

[19] Smith, J. S., and B. A. Peters. 1998. Simulation as a decision-makingtool for real-time control of flexible manufacturing systems. Pro-ceedings of the 1998 International Conference on Robotics andAutomation, pp. 586-90.

[20] Narayanan, S. D., A. Bodner, U. Sreekanth, S. J. Dilley, T. Govin-daraj, L. F. McGinnis, and C. M. Mitchell. 1992. Object-orientedsimulation to support operator decision making in semiconductormanufacturing. Proceedings of the 1992 International Conferenceon Systems, Man and Cybernetics, pp. 1510-19.

[21] Govindaraj, T., L. F. McGinnis, C. M. Mitchell, D. A. Bodner,S. Narayanan, and U. Sreekanth. 1993. OOSIM: A tool for sim-ulating modern manufacturing systems. Proceedings of the 1993National Science Foundation Grantees in Design and Manufac-turing Conference, pp. 1055-62.

[22] Bodner, D. A., and S. A. Reveliotis. 1997. Virtual factories: Anobject-oriented simulation-based framework for real-time FMScontrol. Proceedings of the 6th International Conference onEmerging Technologies and Factory Automation, pp. 208-13.

[23] Arango, G., and R. Prieto-Diaz. 1991. Domain analysis: Conceptsand research directions. In Domain analysis and software systemsmodeling, edited by EDITOR?, 9-33. Los Alamitos, CA: IEEEComputer Society Press.

[24] Stecke, K. E. 1985. Design, planning scheduling and control prob-lems of flexible manufacturing systems. Annals of Operation Re-search 3:000-000.

[25] Tirpak, T. M., S. M. Daniel, J. D. LaLonde, and W. J. Davis. 1992.A fractal architecture for modeling and controlling flexible man-ufacturing systems. IEEE Transactions on Systems, Man and Cy-bernetics 22 (5): 564-67.

[26] Davis, W. J., J. Macro, and D. Setterdahl. 1997. An integratedmethodology for the modeling, scheduling and control of


Gonzalez and Davis

flexible automation. Journal on Robotics and Intelligent ControlVOL:000-000.

[27] Gonzalez, F. G. 1996. A simulation-based controller builder for flex-ible manufacturing systems. Proceedings of the 1996 Winter Sim-ulation Conference, pp. 1068-75.

[28] Pegden, D., R. Shannon, and R. Sadowski. 1990. Introduction tosimulation using SIMAN. New York: McGraw-Hill.

[29] Davis, W. J., J. G. Macro, A. L. Brooks, M. S. Lee, and G. S. Zhou.1996. Developing a real-time emulation of multiresolutional con-trol architectures for complex, discrete-event systems. Proceed-ings of the Second Semiotic Modeling and Simulation AnalysisConference, National Institute of Standards and Technology, pp.274-79.

[30] Gonzalez, F. G., and W. J. Davis. 1997. A simulation-based con-troller for distributed discrete-event systems with application toflexible manufacturing. Proceedings of the 1997 Winter Simula-tion Conference, pp. 845-53.

[31] Gonzalez, F. G., and W. J. Davis. 1997. A simulation-based con-troller for a flexible manufacturing cell. Proceedings of the 1997International Conference on Systems, Man and Cybernetics.

[32] Davis, W. J., B. Bauman, J. Macro, and D. Setterdahl. 1994. Con-structing an emulator for research and education in the control offlexible automation. Proceedings of the ORSA Technical Sectionon Manufacturing Management Conference, pp. 151-57.

[33] Davis, W. J. 1992. A concurrent computing algorithm for real-timedecision making. ORSA Computer Science and Operation Re-search: New Developments in Their Interfaces Conference, pp.247-66.

[34] Gonzalez, F. G., and W. J. Davis. 1998. An architecture for thedistributed intelligent control of flexible manufacturing systems.Proceedings of the 1998 Conference on Simulation Methods andApplications, pp. 106-13.

[35] Gonzalez, F. G., and W. J. Davis. 1999. An intelligent control ar-chitecture distributed across the network, for the control of large-

scale discrete-event system. Proceedings of the 1999 Joint Confer-ence of the Third World Multiconference on System, Cyberneticsand Informatics (SCI 99) and the Fifth International Conferenceon Information Systems Analysis and Synthesis (ISAS 99).

[36] Gonzalez, F. G., and W. J. Davis. 1998. Developing a physical emu-lator for a flexible manufacturing system. Proceedings of the 1998International Conference on Systems, Man and Cybernetics.

[37] Gonzalez, F. G., A. Helton, D. Helton, J. Smith, E. Thompson, andG. Walterscheild. 2000. The design of a solid-state physical modelof an automated system to be used as a test bed for control appli-cations. Proceedings of the 2000 Winter Simulation Conference.

Fernando G. Gonzalez is an assistant professor of electrical andcomputer engineering at the University of Central Florida in Or-lando, Florida. He received his Ph.D. in electrical engineeringat the University of Illinois at Urbana-Champaign. Earlier, hereceived his B.S. in computer science and M.S. in electrical engi-neering at Florida International University. His current researchaddresses the real-time management of distributed discrete eventsystems.

Wayne J. Davis is a professor of general engineering at the Uni-versity of Illinois at Urbana-Champaign. He received his degreesin engineering sciences from Purdue University. His current re-search addresses the intelligent control architectures for large-scale discrete event systems. In this effort, he is collaboratingwith the Intelligent Systems Division at the National Institute ofStandards and Technology. He is also developing several newsimulation languages to support this development.


a new simulation tool for the modeling and control of...

Documents