Simulation Interoperability Workshop, Paper 06S-SIW-017

1

Real Time Estimation and Prediction Using Optimistic Simulation and Control Theory Techniques

Jeffrey S. Steinman, Ph.D. WarpIV Technologies, Inc. 5449 Caminito Exquisito

San Diego, CA 92130 (858) 531-0643

Steinman@WarpIV.com

Timothy Busch, Ph.D. Air Force Research Labs, Rome Labs

AFRL/IFSB 525 Brooks Road Rome, NY 13441-4505

(315) 330-1486 Timothy.Busch@rl.af.mil

Keywords:

WarpIV, Estimation, Prediction, Optimistic Simulation, Kalman Filters, Optimization, Control Theory, PDES

ABSTRACT: This concept paper provides a theoretical foundation for developing high performance parallel and distributed simulations that can be used to (1) estimate the state of a modeled system based on real-time inputs, (2) predict future multi-hypothesis outcomes based on those estimates, and (3) perform estimation and prediction capabilities in a parallel and distributed computing environment while achieving computational speedup. Kalman Filtering techniques are first introduced to provide the necessary technical background on estimation and prediction control theory used in the rest of the paper. Following the discussion of Kalman Filters, this paper then shows how these control theory concepts can be applied to rollback-based parallel and distributed simulations to facilitate real-time estimation and prediction. Three very important research topics are then discussed: (1) handling multiple hypotheses branching within the simulation, (2) using statistical algebra to represent state variables as statistical distributions rather than relying on random number generation and multiple independent replications to support Monte Carlo models, and (3) metrics for defining raw and relative measures of effectiveness and performance. Finally, challenging related issues and research topics are discussed, such as where bottlenecks might arise and how using Field Programmable Gate Arrays (FPGAs) can alleviate those performance problems through the use of specialized hardware acceleration.

1 Estimation and Prediction Many real-world problems would benefit from the ability to estimate and predict [5, 13] the state of complex systems in real-time [6]. Examples include: (1) monitoring and estimating freeway traffic conditions while predicting future congestion problems based on knowledge of the system and its estimated state, (2) monitoring and estimating the current state of a nuclear power facility while predicting potential problems and exploring potential what-if solutions, and (3) monitoring and estimating the real-world picture of a complex military battlefield operation while predicting the future outcome of the current plan and other alternative plans [2].

Real-time estimates are always obtained by feeding imperfect noisy measurements into the estimator. Because the measurements are not exact, a weighting scheme must be applied to balance the believability of each measurement with respect to previous estimates of the state. Measurement errors must be taken into account.

For example, if one were to measure the distance from point A to point B with a measuring device 100 times, each with the same measurement accuracy, the best estimate at any point in time would simply be determined by averaging the set of measurements.

A simple way to estimate the distance between the two points as each measurement is obtained over time is to appropriately weigh the previous estimate with the new measurement. In this manner, a best estimate of the distance is always maintained while each new measurement applies a correction to the previous estimate. This is shown in equation (2.1), where Mi represents the ith measurement used to estimate the distance D.1

1 Equation 2.1 is simply another way of representing the estimate as the

sum of each measurement divided by the total number of measurements.

2

€

ˆ D i =i −1

i

ˆ D i−1 +

1i

M i (2.1)

Note that this simple example assumed various properties that are rarely true in more complex and realistic estimation and prediction problems. First, it was assumed that the two points A and B were stationary. Suppose A actually represents an aircraft in aggressive combat with an enemy aircraft represented by B. Assume that the distance measurements are obtained by a radar sensor, where among other factors, the uncertainty is a function of the returned signal strength that will vary over time. Ground clutter, signal propagation as a function of their separation distance, weather conditions, chaff or decoys, noise jamming, reception of other competing signals, radar pulse width, temperature of the receiver, antenna pointing error, beamwidth, and radar cross-section profiles of the target all affect the accuracy of each measurement.

In addition, both aircraft are likely maneuvering to avoid being shot down by the other, while simultaneously deploying offensive air-to-air combat tactics to engage their opponent. As time evolves, each aircraft knows its motion through the use of internal gyros and GPS data, so there is very little uncertainty by each aircraft to estimate their own spatial position as they maneuver. However, estimating the location of the other aircraft as it maneuvers is much more challenging because the opposing aircraft gyro and GPS data are obviously not provided. In fact, electronic counter measures are often used to deceive opponents during such engagements. Both aircraft must use estimation and prediction techniques to track their opponent.

In most estimation/prediction systems, the new state estimate of the system at time ti is obtained by optimally combining noisy measurements with the previously predicted state. Then, the new state estimate is extrapolated to time ti+1 to form the next prediction of the system. The estimation/prediction cycle repeats as time evolves and new measurements become available. This iterative process is shown in Figure 1 where X represents the abstract state of the system.

Figure 1: Estimation and Prediction.

Accurate estimators must balance the uncertainty of available measurements with unknown or unpredictable behaviors associated with the model such as enemy aircraft maneuvering. The most recent measurements are usually given more weight in the estimation process when the enemy aircraft is maneuvering, but the weighting must also carefully take the accuracy of the measurement into account as well. Very noisy measurements are not given as much weight in the estimation process because they would lead to less accurate and more volatile estimates of the state. Thus, the accuracy of the measurements and the unpredictability of the system counterbalance to

determine the weighting of each measurement as estimates are determined.

A good state estimator used to represent the location of moving objects will likely include additional variables such as velocity and acceleration that can be directly measured or inferred from the measurements. More complex models may be constructed to represent the state of a moving object when their particular flight characteristics are well known. An example of this might be the trajectory of a ballistic missile during free flight while undergoing Keplarian motion.

Complex models typically take several measurement cycles for the estimator/predictor to take on meaningful state values. For example, at least three measurements are required to infer velocity and acceleration state values when only position is measured. In addition, the state is not always directly measured, so a conversion process between the raw measurements and meaningful state variables may be required.

These concepts will be developed further in the next section as the Kalman filter formalism is derived.

2 Kalman Filters This section provides a self-contained derivation of the Kalman Filter [17] equations from first principles. It is not assumed that the reader is familiar with Kalman Filters. Important concepts from this section are used later in the paper to discuss estimation and prediction using optimistic simulation techniques.

2.1 Kalman Filter Terms and Assumptions

To begin the discussion of Kalman Filters, it is important to define various terms and assumptions. First, it is assumed that the true theoretical equations describing a system with n state variables can be represented as a linear discrete matrix equation of the following form:

€

X (ti+1) = φ (ti+1,ti )X (ti ) + L(ti ) +U (ti ) (3.1)

Where,

€

X (ti ) Represents the true state of the system at time ti. This is characterized as an n x 1 vector.

€

φ (ti+1,ti ) Represents the transition matrix that extrapolates the state of the system from time ti to ti+1. This is characterized as an n x n matrix.

€

L(ti ) Represents the “aiding” term that describes known inputs to the system between time ti and ti+1. This is characterized as an n x 1 vector.

€

U (ti ) Represents the unknown inputs to the system between time ti and ti+1. This is characterized as an n x 1 vector.

The true state of the system is never actually known, which is why it is important to perform estimations and predictions based on input measurements collected

3

regularly over time. Kalman Filters can have different representations depending on whether the estimation and prediction steps are performed separately or collectively in a single step. This paper uses the representation of the Kalman Filter that performs the estimation and prediction in a single step. The estimator/predictor form of the Kalman Filter is written as follows:

€

ˆ X (ti+1) = φ (ti+1,ti ) ˆ X (ti ) + L(ti ) +

K (ti ) Z (ti )−H (ti ) ˆ X (ti )[ ] (3.2)

Where,

€

ˆ X (ti ) Represents the estimated/predicted state of the system at time ti. This is implemented as an n x 1 vector

€

Z (ti ) Represents a vector of measurements of the system taken at time ti. Assuming that there are m measurements, this is implemented as an m x 1 vector.

€

H (ti ) Is known as the measurement matrix. It translates the state of the system to the set of measurements. This is implemented as an m x n matrix.

€

K (ti ) Is called the Kalman Gain. It weighs the correction to the state obtained from each measurement. The Kalman Gains are implemented as an n x m matrix.

A few more terms are needed before beginning the derivation of the Kalman Filter. First, all of the measurements include a noise term, represented as V(ti). Thus, the measurement can be written as follows.

€

Z (ti ) = H (ti )X (ti ) +V (ti ) (3.3)

In other words, the measurement can be thought of as a linear combination of the true state, plus a noise term that introduces error to the measurement. Equation 3.4 then represents the estimate of the state at times ti and ti+1 as the truth plus an error in the estimation.

€

ˆ X (ti ) = X (ti ) + ˜ X (ti )ˆ X (ti+1) = X (ti+1) + ˜ X (ti+1)

(3.4)

One further assumption is made concerning all measurements and unknown inputs. It is assumed that all measurements of the system are unbiased and uncorrelated with each other and with the unknown U(ti) system noise vector.

2.2 Covariance Matrix

The derivation of the Kalman Filter begins by identifying the estimate error in terms of what is already known. This is described in equation (3.5) by rearranging equation (3.4).

€

˜ X (ti+1) = ˆ X (ti+1)− X (ti+1) (3.5)

Plugging equations (3.1) and (3.2) into equation (3.5) produces the following expression.

€

˜ X (ti+1) = φ (ti+1,ti ) ˆ X (ti ) + L(ti )

+K (ti ) Z (ti )−H (ti ) ˆ X (ti )[ ]−φ (ti+1,ti ) ˆ X (ti ) + L(ti )−U (ti )

(3.6)

Equation 3.7 is obtained after rearranging the terms in equation 3.6 and then using equation 3.3 to represent the measurement.

€

˜ X (ti+1) = φ (ti+1,ti ) ˆ X (ti )− X (ti ){ }+K (ti ) H (ti )X (ti ) +V (ti )−H (ti ) ˆ X (ti )[ ]−U (ti )

(3.7)

Again, using equation 3.4 to simplify the expression, equation 3.7 can be written as follows:

€

˜ X (ti+1) = φ (ti+1,ti ) ˜ X (ti ) + K (ti )H (ti ) − ˜ X (ti )[ ]+K (ti )V (ti )−U (ti )

(3.8)

Finally, after combining and simplifying terms in 3.8, the predicted state error can be written as:

€

˜ X (ti+1) = φ (ti+1,ti )−K (ti )H (ti )[ ] ˜ X (ti )+K (ti )V (ti )−U (ti )

(3.9)

The next step in the Kalman Filter derivation defines the covariance matrix as the expectation value of the state error vector in equation 3.9 multiplied by its transpose. This results in the symmetric n x n matrix shown below.

€

P(ti+1) = E ˜ X (ti+1) ˜ X T (ti+1)[ ] (3.10)

Many of the cross terms in equation 3.l0 vanish. This is because the measurement and system errors are assumed to be both uncorrelated and unbiased. The resulting covariance matrix can be considerably simplified and written as:

€

P(ti+1) = β(ti )P(ti )β(ti )T

+K (ti )R(ti )K (ti )T +Q(ti )

(3.11)

Where,

€

β(ti ) = φ (ti+1,ti )−K (ti )H (ti )

€

P(ti ) = E ˜ X (ti ) ˜ X (ti )T[ ]

€

R(ti ) = E V (ti )V (ti )T[ ]

€

Q(ti ) = E U (ti )U (ti )T[ ]

The R matrix is an m x m symmetric matrix that represents the measurement noise. Its diagonal elements are the variances of the measurements. The Q matrix is an n x n symmetric matrix. It is sometimes called the system

4

noise matrix because it represents the uncertainty of the modeled system.

2.3 Kalman Gains

The next step in the derivation is to compute the Kalman Gains. The goal is to determine the optimal values for K(ti) that minimize the covariance matrix. This is accomplished by setting the partial derivatives of the covariance matrix with respect to K(ti) equal zero and then solving for K(ti).

€

∂P(ti+1)∂K (ti )

= −H (ti )P(ti ) φ (ti+1,ti )−K (ti )H (ti )[ ]T

− φ (ti+1,ti )−K (ti )H (ti )[ ]P(ti )H (ti )T

+R(ti )K (ti )T +K (ti )R(ti )

(3.12)

Setting this to zero and simplifying leads to the following expression in equation 3.13.

€

0 = R(ti )K (ti )T −H (ti )P(ti )β(ti )

T{ }+ K (ti )R(ti )− β(ti )P(ti )H (ti )

T{ } (3.13)

This can be rearranged as,

€

0 = K (ti )R(ti )− β(ti )P(ti )H (ti )T{ }

+ K (ti )R(ti )− β(ti )P(ti )H (ti )T{ }

T (3.14)

Since each bracketed term is the transpose of the other term, setting either of the terms to zero automatically sets the other term to zero. So, the expression in 3.14 reduces to the following.

€

0 = K (ti )R(ti )

− φ (ti+1,ti )−K (ti )H (ti )[ ]P(ti )H (ti )T (3.15)

Rearranging the terms once again leads to the final result shown in equation 3.16 for the Kalman gains.

€

K (ti ) = φ (ti+1,ti )P(ti )H (ti )T

R(ti ) +H (ti )P(ti )H (ti )T[ ]−1

(3.16)

Note that large values for R(ti) cause the gains to decrease because the measurements are noisy. However, this is countered by the P(ti) covariance term that can be strongly influenced by the system noise, Q(ti). As Q(ti) gets large, the covariance term gets large which increases the gains. In the simplest one-dimensional case where H(ti) and φ(ti+1, ti) are equal to one, the gains approach one during steady state when Q(ti) is much larger than the measurement noise R(ti). Conversely, the gains approach zero during steady state when Q(ti) is much smaller than the measurement noise R(ti).

2.4 Simplifying the Covariance Matrix

Equation 3.11 can be simplified through the expansion of each term and then carefully applying equation 3.16. First, expanding 3.11 leads to equation 3.17 below.

€

P(ti+1) = φ (ti+1,ti )P(ti )φ (ti+1,ti )T

− φ (ti+1,ti )P(ti )H (ti )T K (ti )

T{ }−K (ti )H (ti )P(ti )φ (ti+1,ti )

T

+K (ti ) H (ti )P(ti )H (ti )T +R(ti )[ ]K (ti )T

+Q(ti )

(3.17)

Plugging 3.16 into the K(ti) term in the fourth line of equation 3.17 cancels the term inside the square brackets to form a simpler expression for the covariance matrix. This leads to equation 3.18 below.

€

P(ti+1) = φ (ti+1,ti )−K (ti )H (ti )[ ]P(ti )φ (ti+1,ti )T

+Q(ti )(3.18)

2.5 Kalman Filter Summary

This section derived the predictor/estimator form of the Kalman Filter. Three important equations were used to implement Kalman Filters. First, equation 3.16 was used to compute the Kalman gains. Then, equation 3.2 was used to update the state. Finally, equation 3.18 was used to update the covariance matrix to prepare for the next computation in the estimation/prediction cycle.

The key ingredients of the Kalman Filter algorithm that can be applied to simulation for the use of estimation and prediction are the notions of (1) state, which is the representation of the modeled system through the collection of attributes that change over time, (2) transition matrix that allows the state of the system to be transformed from one time to another time, (3) aiding terms that represent accurately known inputs that are fed into the modeled system, (4) system noise that represents unknown factors in the model such as opponent plans and maneuvering tactics, (5) measurements that are collected over time and have a degree of noise or uncertainty, and (6) gains that balance measurement noise with system noise to obtain the best estimation and prediction of the state.

3 Techniques of Optimistic Simulation This section provides a high-level description of optimistic parallel discrete event simulation techniques [4, 8, 15]. It then shows how these techniques relate to the estimation and prediction problem.

The fundamental challenge of executing a simulation in parallel on multiple processors is to ensure that each object modeled in the simulation processes its events in causally correct ascending time order, while also achieving maximal processing efficiency and concurrency. Optimistic processing techniques are primarily used to support complex parallel simulations

5

that place no constraints on how modeled objects that are distributed across multiple processors interact. Optimistic simulations allow any object to interact with any other object on any other processing node at any time scale. This is in contrast to conservative techniques that either (1) limit which objects can interact with which other objects, (2) place constraints on how tightly in time objects on remote nodes can interact with one another, and/or (3) place event ordering rules on inter-object interactions.

Rollback techniques allow objects on any compute node to process their events optimistically assuming that the processing of the next current event will not become invalid due to the arrival of an earlier event scheduled by a simulated object residing on another node. When such straggler messages are received, all optimistically processed events for the receiving object that were processed with time tags greater than the time tag of the straggler event, are rolled back. When using incremental state saving techniques, the events are rolled back in reverse order much like the familiar undo mechanisms that are provided by many commercial software applications.

Events perform two basic operations. They arbitrarily (1) modify state variables and (2) generate new events. Rollbacks must therefore undo those two operations. This means that the rollback infrastructure of the optimistic simulation engine must be capable of (1) undoing the state changes made by each event and (2) retract any new events that were scheduled.

Both incremental state saving and full state saving techniques have been used in the WarpIV Simulation Kernel and other optimistic simulation engines such as SPEEDES to restore state as rollbacks occur. Full state saving techniques supported in other systems save the entire state of a modeled object before each event is processed. This can be very efficient for objects having small states. However, this approach can become highly inefficient both in terms of memory consumption and processing overheads for complex objects having large states that are fragmented over large numbers of memory segments.

This paper focuses on incremental state saving techniques that capture each change made to state as they occur. The rollback infrastructure simply undoes those changes in reverse order. For optimistic simulations requiring dynamic memory allocation, the rollback infrastructure must not only support the restoration of primitive state variables, but it also must support generic dynamic memory allocation, deallocation, and container classes such as trees or lists that store the dynamically allocated memory.

When events are scheduled for objects residing on other nodes, antimessages are sent to retract the event-scheduling messages if the event is rolled back. This can cause cascading rollbacks and further antimessages if those retracted events were processed before they were retracted.

Various optimistic flow control techniques have been developed in WarpIV and in its predecessor SPEEDES to keep the number of rollbacks and antimessages stable. These techniques include (1) holding back high-risk messages until it is safe (or safer) to release them and (2) limiting event-processing optimism on runaway nodes. A good rollback infrastructure only rolls back those events that are associated with each individual object, and not the entire state of the simulation on a node. Object-based (as opposed to node-based) rollback techniques significantly reduce the number of rollbacks experienced by an application as it executes in parallel.

Optimistic simulation techniques are ideal for estimation and prediction of real-time systems because the transition matrix goes both forward and backward in time. A forward transition is normally accomplished through the rapid processing of events in parallel. A backward transition in time is simply accomplished by rolling back the state of the objects that are of interest. In some cases, a rollback may be followed by a rollforward much like the redo operations found in many commercial software applications.

As the simulated system transitions forward in time, events that were rolled back may either rollforward or be reprocessed as needed. One would only reprocess an event if its input parameters and/or state values have changed in a manner that would significantly affect its outcome. So for example, if an object were rolled back due to the arrival of a straggler query event that does not modify any state variables, it would be safe to rollforward instead of reprocessing all of the events that were rolled back. For practical reasons, tolerances can be established to determine if rolled back events need to be reprocessed or if they can be quickly rolled forward.

3.1 Estimation and Prediction Formalism

As previously mentioned, fundamental to optimistic simulation is the notion of state, forward processing of events, rollbacks, and rollforward. These notions can be used to define X(t) as the state of the system, and φ(t2, t1) as the transition matrix that evolves the system from one time to another time. At a finer level of granularity, these concepts can also be used to describe the evolution of individual objects and/or their internal components within the simulation. Again, optimistic techniques allow the transition matrix to evolve the models both forward and backward in time. Rollback and rollforward capabilities are critical to allow the simulation to predict far into the future, while being able to rollback as real-time inputs are received and processed to form current estimates of the state. In this mode of operation, time-tagged events are only committed when the wall time exceeds their logical time stamp.

To complete the formalism, it is important to define the other terms in equation 3.2 using optimistic simulation concepts.

Aiding terms, L(t), are in many ways similar to measurements, Z(t), in that they provide external input events into the simulated system. However, the most important difference between aiding terms and

6

measurements is that aiding terms are assumed to provide information without error, while measurements are partially corrupted by some degree of observation noise. For example, in a battlefield simulation used to estimate the current state of a military operation and predict future outcomes, aiding terms might represent known actions, accurate GPS locations of assets, or outcomes of specific situations, while the measurements are provided through the detection of enemy forces using imperfect sensors and uncertain intelligence reports.

Detections may come from various types of sensors including (1) radar, (2) sonar, (3) infrared, (4) visual, (5) passive reception of signals, etc. Note that in all of these cases, a degree of uncertainty is included in the measurements. The Kalman filter equations work very well in estimating the state of such systems where the physical model is known, measurements are collected, but there is uncertainty in knowing the tactics of the opponent.

The system noise, Q(t) in equation 3.18 often drives the covariance matrix and must be estimated in order to correctly compute gains in equation 3.16. Without measurements, the covariance matrix grows exponentially over time. As the covariance matrix grows, the measurements are weighted more strongly by the Kalman gains. Unfortunately, most simulations do not propagate their error estimates. This makes it difficult to assess predicted results. It also impacts the ability to perform Verification Validation and Accreditation (VV&A).

In steady state, the system noise and measurement noise offset each other to provide stable values for the Kalman gains. However, it is also important for a good estimator to actually estimate the system noise. For example, the system noise should grow when an enemy aircraft begins maneuvering in an unpredictable manner to avoid being shot down or to perform an offensive air-to-air tactic. Maneuver detection is a critical component of modern air-to-air tracking systems.

Part of the simulated state may involve the direct use of Kalman filters to estimate the opponent’s state, while aiding terms are used to calibrate the state of the known system. Prediction of the opponent is more difficult because the tactics deployed may be unpredictable. In some cases, it may be necessary to split opponent actions into different hypothesis to model radically different courses of action. Each hypothesis may have a different probability associated with its likelihood. In any case, the covariance matrix of the system, which represents the uncertainty of the system state, grows exponentially as predictions are made further out in time. This is consistent with the familiar notion that it is harder to accurately predict what will happen in the far future rather than what will happen in the near future. Predicting the weather is a good example of this principle.

State discontinuities may occur as various predicted outcomes arise. For example, an aircraft may be fired upon with a specified probability of kill or damage. At least three very different possible situations must be handled. Either the aircraft was (1) destroyed, (2) impaired, (3) or unharmed. If the aircraft is destroyed,

then it can no longer participate in its battle plan. An impaired aircraft may be repaired and continue to engage the enemy later on in the simulation. An unharmed aircraft will continue to engage with the enemy according to its planned strategy. These three very distinct outcomes would require completely different state-variable representations and must be accomplished through either splintering multiple Monte Carlo replications of the simulation, or through multiple hypotheses branching. Multiple hypothesis branching is discussed later in this paper.

3.2 Conceptual Operation

The conceptual operation of using optimistic simulation to support real-time estimation and prediction is straightforward. The simulation must always be running and predicting the future up to a specified time or time window, while rolling back affected models occasionally to process real-time inputs. The real-time inputs are received and processed as externally generated events. They can be in the form of aiding terms used to describe known changes to the system, or in the form of noisy measurements that require balancing the believability of the measurements with system noise in the model and its covariance matrix. In any case, the objects affected by the inputs are rolled back to the current real-world time. Processing these real-time inputs can be thought of as calibrating, or estimating the real-time state of the simulation. After the inputs have been received and processed, the system must quickly rollforward and/or reprocesses those events that were rolled back due to the new inputs to continue predicting the future.

Because many complex systems such as modeling military operations [11] can take on very different states at critical decision points in the simulation, it is necessary to support branching, either through the generation of separate Monte Carlo replications, or through the use of multiple hypotheses branching techniques that support multiple hypotheses within a single execution. Multiple hypotheses branching can also be used to explore what-if possibilities or alternative plans. Because all hypotheses share the processing of events not affected by the branching, this approach has tremendous appeal for computationally intensive simulation applications. Computation sharing between multiple branches is one of the key enabling features necessary for supporting real-time operation.

4 Multiple Hypotheses Branching Analytic simulations are often independently executed many times with different parameter settings to determine the optimal configuration of a complex system. Monte Carlo simulations that model critical decisions using probabilistic distributions and random number generation may require large numbers of end-to-end simulation replications to determine the statistical measures of effectiveness and/or performance of a complex system. Real-time estimation and prediction Decision Aid Tools project the future through simulation using real-time data to calibrate the current state and require multiple-hypothesis what-if branching capabilities to

7

simultaneously explore the effectiveness of various decision options.

These three different kinds of simulation applications are commonly implemented by running multiple executions of the same simulation using (1) different parameter settings, (2) different starting random number seeds, or (3) different critical decision options and/or battle plans. Simplistic parallel processing is achieved by farming the replications to multiple processors. However, this simple approach ignores the fact that each of the replicated simulation executions may redundantly repeat many of the same time-consuming calculations.

Some improvements to the brute force multiple-replication Monte Carlo approach [9] have been explored. One example is to replicate the simulation only at branch points [3]. This has the appeal of sharing calculations up to the branch point, but it does not take full advantage of computation sharing afterwards. Another approach clones objects at critical branch points [7]. While this offers a higher degree of computation sharing, it still does not take full advantage of the potential computation sharing for complex objects with many components.

A more efficient approach to support these kinds of simulations is to automatically detect and reuse redundant calculations shared between the multiple branches at the state attribute level. Unique calculations special for a particular hypothesis branch would only be performed when required. As a further optimization, this more efficient computational approach should also take advantage of parallel processing when multiple processors are available through the use of advanced optimistic time management and multi-replication event processing algorithms.

Multiple hypotheses simulation applications can be supported with minimal computational overhead on single-processor machines, multi-processor machines, and or networks of such machines. Special-purpose hardware such as Field Programmable Gate Arrays (FPGAs) can also be provided to speed up certain highly parallel computations. This will be briefly discussed towards the end of this paper.

4.1 Multiple Hypothesis State Management

The key to supporting parallel replications within a simulation is to automatically manage hypothesis-dependent state data within models of the system. So, instead of running independent replications of the simulation with different initial state values, the multiple-hypothesis simulation engine actually manages the different state values for each replication within the models of the simulated system. To automate this capability, three new data types have been prototyped within the WarpIV Simulation Kernel to represent integers, doubles, and Boolean flags. These techniques are generalizable to other data types. Replication-specific model states are comprised of these data types. Depending on the desired setting, up to 32 or 64 simultaneous replications can be supported in the current prototype. The concepts described here may be easily extended to larger replication sets.

Each event is associated with a replication set stored as a 32 or a 64 bit-field integer value. The bits set in the integer represent the collection of replications associated with the event. For example, a 32-bit integer value representing the collection of replications with hex value FFFFFFFF would represent all replications {0, 1, 2, … 31}. An event with replication set hex value 0000000F would represent the subset of replications {0, 1, 2, 3}.

Normally, all initial events start out representing the full set of replications. However, events may split into multiple events with smaller replication sets at key decision points or when state variables are accessed having different replication values. In this manner, an event naturally splits into multiple replication sets and may be processed several times as state variables are accessed with different replication values. Table 1 illustrates how a replicated state variable is implemented as an array of 32 values with a corresponding replication mask indicating which replications have the same value.

Table 1: An example of a replicated integer with 32 replications. The value for each replication and the replication bit-mask are stored in the array. The bit-mask identifies which replications have the same value. So, since the value 0 is stored in replications {0, 9, 31}, the bit mask has three 1’s in the corresponding bits. The rest of the bits are set to 0.

Rep Value Mask (binary) 0 0 1000 0000 0000 0000 0000 0010 0000 0001 1 5 0000 0000 1000 0000 0100 0001 1000 0010 2 1 0100 0000 0100 1100 0011 0000 0111 0100 3 2 0011 0001 0011 0001 1000 0000 0000 1000 4 1 0100 0000 0100 1100 0011 0000 0111 0100 5 1 0100 0000 0100 1100 0011 0000 0111 0100 6 1 0100 0000 0100 1100 0011 0000 0111 0100 7 5 0000 0000 1000 0000 0100 0001 1000 0010 8 5 0000 0000 1000 0000 0100 0001 1000 0010 9 0 1000 0000 0000 0000 0000 0010 0000 0001 10 3 0000 1100 0000 0000 0000 0100 0000 0000 11 4 0000 0010 0000 0000 0000 1000 0000 0000 12 1 0100 0000 0100 1100 0011 0000 0111 0100 13 1 0100 0000 0100 1100 0011 0000 0111 0100 14 5 0000 0000 1000 0000 0100 0001 1000 0010 15 2 0011 0001 0011 0001 1000 0000 0000 1000 16 2 0011 0001 0011 0001 1000 0000 0000 1000 17 7 0000 0000 0000 0010 0000 0000 0000 0000 18 1 0100 0000 0100 1100 0011 0000 0111 0100 19 1 0100 0000 0100 1100 0011 0000 0111 0100 20 2 0011 0001 0011 0001 1000 0000 0000 1000 21 2 0011 0001 0011 0001 1000 0000 0000 1000 22 1 0100 0000 0100 1100 0011 0000 0111 0100 23 5 0000 0000 1000 0000 0100 0001 1000 0010 24 2 0011 0001 0011 0001 1000 0000 0000 1000 25 4 0000 0010 0000 0000 0000 1000 0000 0000 26 3 0000 1100 0000 0000 0000 0100 0000 0000 27 3 0000 1100 0000 0000 0000 0100 0000 0000 28 2 0011 0001 0011 0001 1000 0000 0000 1000 29 2 0011 0001 0011 0001 1000 0000 0000 1000 30 1 0100 0000 0100 1100 0011 0000 0111 0100 31 0 1000 0000 0000 0000 0000 0010 0000 0001

Before each event is processed, its replication-set bitfield is stored in a globally accessible variable called CurrentReplicationSet. This variable represents the current set of replications that need to be processed by the event. The first replication, called FirstReplication, in the CurrentReplicationSet is also stored in another globally accessible variable. It corresponds to the first bit set in the CurrentReplicationSet.

8

Whenever the event accesses a replicated state variable, the bit-field mask stored in the FirstReplication array element is ANDed with the CurrentReplicationSet. This may reduce the collection of replications in the CurrentReplicationSet if the associated replicated values in the model’s state have different replication values. When the event processing completes, the CurrentReplicationSet is XORed with the remaining replication set. The result is stored back in the CurrentReplicationSet. If the resulting value is non-zero, then the event is processed again with a new CurrentReplicationSet and FirstReplication. This continues until all of the bits stored in the CurrentReplicationSet are zero. In this manner, event splitting is automated.

Note that this approach does not require examining each replicated state variable as it is accessed to determine which ones have the same value. The bit-field mask automates this very efficiently. However, when an event modifies a replicated value, a check is required to determine if the new value is shared by other replication sets, or if it constitutes a new set. For large numbers of replications, this can be computationally intensive. Fortunately, several optimizations are possible to reduce this computation. One very promising approach is to use FPGAs to perform the operation in parallel using programmable hardware.

As events are potentially processed multiple times, new events may be generated for different replication sets. These generated events can be merged if their time tags and associated event parameters are identical.2 This is done before the generated events are released to the simulation engine for scheduling. Furthermore, two identical events with different replication sets on the same model can also be merged before processing. In this manner, event joining is automated.

An event may independently split into multiple replications. For example, an aircraft fired upon may split its replication set into three sets (or hypothesis) to indicate whether it was destroyed, impaired, or unharmed. When forming new replication branches, it is important to ensure that the replications are uncorrelated. So, a random assignment of bits to replication sets is required. There are various ways to randomly assign bits to replication sets such as pre-computing random numbers in tables. However, this problem also lends itself to FPGA hardware acceleration where the random numbers can be generated in parallel using fast hardware.

As time evolves, the outcome of a predicted decision point will become known. Consider the previous example of an entity possibly being destroyed, impaired, or unharmed. At most, only one of these cases can be true when the actual time of the branch is reached in the real world. So, the branches that do not match reality must be pruned. The false branches are rolled back and retracted

2 These constraints can be loosened. Identical event types for the same

object with nearly the same time tags could be merged into a single event. In the same manner, tolerances can be established for state variables so that they are considered to be the same if they are close enough in value.

causing all of their generated outcomes to be removed from the simulation as well. Then, the true branch is reassigned to all replications in its original set before the branching occurred. The bookkeeping in this step is very complex and will require careful testing to verify its correctness.

The benefit of (1) managing multiple replications in state variables, (2) automating event splitting, and (3) automating event joining is that computations that are common to groups of replications are now shared while also facilitating the unique computations that are specific to individual or subset replications. Dramatic reductions in computations may be achieved by automatically sharing redundant work between replications. Of course, there is the potential that the overhead in supporting this capability will outweigh the benefits gained. This must be investigated carefully.

Furthermore, by integrating this technology with an optimistic simulation engine, parallel and distributed event processing can be achieved through the use of rollback-based time management synchronization algorithms, which is necessary to support the real-time estimation and prediction techniques described in this paper. Parallel performance should be exceptional for large systems.

5 Statistical Algebra A critical component of the real-time estimation and prediction strategy described in this paper is to use multiple hypotheses branching to split the event processing into a series of replications that are internally managed within the parallel simulation execution. If random numbers [10] are directly used in the models, then the execution will quickly splinter into large numbers of replications because the state variables rapidly diverge. Therefore, it is important to only branch when critical decisions that have very different state representations are required. To handle statistical models, the statistical algebra [16] capabilities described in this section are used to represent statistical state variables as distributions, not just random values.

Convolution operations are required to obtain new distributions from algebraic expressions involving two independent distributions. If A(x) and B(y) each represent distributions over independent variables x and y, then P(z) for a function F(A,B) can be represented as follows in equation 6.1.

€

P(z) = A(x)dx B(y)δ (z −F(x,y))dyy=−∞

y= +∞∫

x=−∞

x= +∞∫

= A(x)B(F−1(x,z))dxx=−∞

x= +∞∫

(6.1)

Where F-1(x,z) is the inverse of F(x,y) that is determined by solving z=F(x,y) for y. Note that δ(z-F(x,y)) represents the Dirac Delta Function whose value is zero when its argument is non-zero, is infinite when its argument is zero, and is normalize to have the area under its curve equal to one.

9

For the simple case where F(A,B) = A + B, the inverse is found by solving z = x + F-1(x,z). The inverse is simply z - x, which is the familiar convolution shown in equation 6.2.

€

P(z) = A(x)B(−∞

+∞∫ z − x)dx (6.2)

The current WarpIV Statistical Algebra implementation represents distributions through the use of discrete bins ranging over their independent variable. Applications may define the number of bins and the domain of the independent variable. The sum of the binned values is normalized to one. Numerically computing algebraic expressions involving distributions is accomplished by summing the weighted contribution to the appropriate new bin. Special care over binning and sub-binning must be taken into account to ensure that the resulting distributions obtained from algebraic operations are smooth. Otherwise, potential spikes and gaps can arise in the resulting distributions.3

The WarpIV Statistical Algebra capability supports basic algebraic operations such as +, -, *, and /. However, for complete generality, it also supports transcendental functions such as sqrt(), sin(), cos(), tan(), arcsin(), arcos(), arctan(), exp(), log(), pow(), etc. Special care has been taken to ensure that the range of independent variables does not contain any singularities or undefined values. For example, if the independent variable x for distribution A ranges from -1.0 to +1.0, then singularities would arise in the expression B=1/A. In a similar manner, square roots of negative numbers are not allowed.4

Handling correlations in more complex equations is also important. For example in the current implementation, there is a difference between the expressions Y = pow(X, 2) verses Y = X*X. In the first expression involving the power transcendental function, X is correlated, while in the second expression all algebraic operations are treated as independent and uncorrelated distributions. Thus, these two expressions yield very different resulting distributions. Perhaps future implementations will detect such correlations.

An example showing the use of the WarpIV Statistical Algebra package is shown in Code Segment 1.

Code Segment 1: Statistical Algebra.

// Overview.C tutorial demonstrates how to // use the basic features of the statistical // algegra package. #include "WpStatAlgebra.H" int main() {

3 The initial implementation of the WarpIV Statistical Algebra

capability suffered from gaps and spikes until the integration algorithm was modified to use appropriate resolution sub-binning techniques.

4 The current WarpIV Statistical Algebra capability may someday be extended to support complex numbers.

// Create a Gaussian distribution named “x” // with the following settings: // lo = 0.0 // hi = 10.0 // numBins = 50 // mean = 5.0 // sigma = 1.0 WpStatGaussian x(0.0, 10.0, 50, 5.0, 1.0, “x”); x.GraphDistribution(); // Create an exponential distribution named // “y” with the following settings: // lo = 0.0 // hi = 10.0 // numBins = 100 // timeConstant = 1.0 WpStatExp y(0.0, 10.0, 50, 1.0, “y”); y.GraphDistribution(); // Define a distribution named “z” that will // be used to store the result WpStatDouble z(“z”); // Compute z as a complex mathematical // function of x and y z = log(sqrt(x) + 2.0 * cos(y) + 5.0) / 3.14159; z.GraphDistribution(); // Exit with status 0 return 0; }

The output distributions of this program are shown in the figures below.

Figure 2: Gaussian distribution for X.

10

Figure 3: Exponential distribution for Y.

Figure 4: Distribution for Z.

Additional capabilities are provided by the WarpIV Statistical Algebra package to allow applications to smooth and/or re-bin distributions, fit standard statistical functions to numerical distributions, obtain statistical metrics such as the mean and standard deviation from distributions, use built-in services for a wide variety of common statistical functions, and perform correlated operations on user-defined functions involving multiple distributions.

6 Measures of Effectiveness The primary purpose of the estimation and prediction techniques discussed in this paper is to accurately assess the potential outcomes of actions taken by decision makers who interact with the system. As previously mentioned, estimation techniques fuse live data with simulated data to best determine the state of the system. Based on the current estimated state of the system, actions are chosen according to some policy. Prediction is used to determine the utility of these actions. Policy implementation will be the subject of a future paper.

It is also important to define metrics that characterize the performance and effectiveness of the overall system. These metrics provide the feedback to the user indicating the quality of the outcome based on real-time estimation and prediction. In this section, all metrics are normalized between zero and one. A score of one would indicate

perfect operation, while a score of zero would indicate complete failure of the system or planned outcome.

It is important to distinguish the difference between the terms effectiveness and performance. Usually the term effectiveness relates to the overall results of the simulation and is used to indicate how well the objectives were met. The term performance relates to how well systems or subsystems performed their function. For example, an aircraft might perform poorly while still effectively meeting its mission objectives.

It is possible for the simulated system to not perform according to plan, while still producing an effective result. A good example of this in sports is a broken football play that still results in scoring a touchdown. In this case, the play was poorly executed (low score for the performance) while the end result still produced a touchdown (high score for the effectiveness).

Battle plans in a complex military operation evolve over time. Anticipated gains and losses therefore evolve. It is important to not only look at the raw effectiveness metric at any point in time to characterize the operation of the plan. For example, the invasion at Normandy on D-Day started out very poor in terms of losses to the allied forces, yet these losses were anticipated. Of course the plan in the long run was very effective and concluded with a complete victory and the liberation of Europe. Like a chess game, it is sometimes advantageous to sacrifice a pawn early on to obtain a better position that later wins the battle. This leads to defining two terms: (1) raw effectiveness and (2) relative effectiveness. These terms will be described in the context of a battlefield simulation involving red and blue opposing forces.

6.1 Raw Effectiveness

Raw effectiveness is determined by assigning an intrinsic value to each blue or red asset in the battle. The normalized raw effectiveness score is determined by summing these values in a meaningful way. A score of one would indicate that all red assets have been destroyed without any blue losses. Similarly, a score of zero would indicate that all blue assets have been destroyed without any red losses.

The intrinsic value for each entity (or asset) in the battle can be defined at a given point in time. Normally, the intrinsic value does not change for each entity. However, it is possible for new entities to enter the battle, for damaged entities to be repaired, and/or for new information to be provided indicating changes to the intrinsic value of an entity.

Intrinsic values can be defined as follows:

€

I iB (t) = Intrinsic value for blue entityi at time t

€

I iR (t) = Intrinsic value for red entityi at time t

The actual value for each entity (or asset) in the battle can be defined at a given point in time. The actual value ranges from zero (meaning that the entity/asset has been destroyed) to the full intrinsic value (meaning that the

11

entity/asset has perfect health, has not diminished its capacity to engage in battle, has all of its weapon systems in tact, and has a full fuel supply).

€

AiB (t) = Actual value for blue entityi at time t

€

AiR (t) = Actual value for red entityi at time t

The total intrinsic values for Blue and Red entities/assets in the battle can be specified at a given point in time:

€

I B (t) = I iB

i

NB∑ (t) (7.1)

€

I R (t) = I iR

i

N R∑ (t) (7.2)

The total actual values for Blue and Red entities/assets in the battle can also be specified at a given point in time:

€

AB (t) = AiB

i

NB∑ (t) (7.3)

€

AR (t) = AiR

i

N R∑ (t) (7.4)

As previously mentioned, raw effectiveness is a value between zero and one indicating the raw effectiveness of the plan outcome. A value of one indicates complete success (i.e., all Red entities/assets destroyed and no Blue entities/assets destroyed). A value of zero indicates complete failure (i.e., all Blue entities/assets destroyed and no Red entities/assets destroyed). The raw effectiveness is not an indication of how accurately the plan has been followed. It is simply a measure of the overall outcome.

€

RawEffectiveness(t) =AB (t) + I R (t)−AR (t)[ ]

I B (t) + I R (t) (7.5)

A successful plan normally results in the raw effectiveness value generally increasing over time. However, it is possible for a plan to accomplish its mission, even though the final raw effectiveness decreases. This would happen if the cost of completing a mission turns out to be higher than the overall gain.

Because of the uncertain nature of predicting the outcome of plans, it is important to execute multiple replications or support multiple-hypothesis branch points and statistically analyze the results of the different replications. The mean and standard deviation of these replications may be provided at regular time increments during the simulation. These mean values and their standard deviations can be fitted using χ2 analyses to obtain time-based visual plots that provide trend analysis visualization to users.

The raw effectiveness can be thought of as the overall measure of effectiveness of the plan. This is different from the relative effectiveness, which is a measure of the plan performance (i.e., how accurately the plan was followed).

6.2 Relative Effectiveness

The state of each entity/asset in the plan at any point in time can be defined as an abstract vector of values. These state values can be projected forward in time either from simulation or from the actual planned expectations. In air combat operations, this can come directly from the Air Tasking Order (ATO). As discussed earlier in this paper, estimated state values are provided through the processing of live data feeds that are provided as inputs into the system. These three different kinds of state vectors are defined below.

The simulation predicts the state vectors for each entity/asset in the scenario.

€

X i

B (t) = Predicted state vector of blue entityi at time t

€

X i

R (t) = Predicted state vector of red entityi at time t

The expected state vectors for each entity/asset in the plan is projected from the plan to predict the outcome of the engagement.

€

Y i

B (t) = Projected state vector of blue entityi at time t

€

Y i

R (t) = Projected state vector of red entityi at time t

The real-time estimated state vector of each entity is defined as follows.

€

Z i

B (t) = Estimated state vector of blue entityiat current time t

€

Z i

R (t) = Estimated state vector of red entityiat current time t

The relative effectiveness is a measure of how accurately the plan is being performed with respect to the expectations or of the plan. Two types of relative effectiveness are computed: (1) simulated predictions in time vs. planed expectations from the plan, and (2) simulated estimation of the system at the current time with respect to the real-time operational picture.

In the first case, the relative effectiveness provides a prediction of the uncertainty of the plan performance over time. It predicts when the plan might fall apart, and when new planning may be required. It provides insight on the chaos that may ensue during the fog of war. Some replications may deviate from the plan due to statistical and/or uncertainties in the scenario, while other replications produce the anticipated outcome. A statistical analysis of the relative effectiveness is used to help characterize the plan dispersion. Mean and sigma values for the relative effectiveness are computed for each regular time increment in the simulation. Like the raw effectiveness, the mean and standard deviations for the relative effectiveness can also support χ2 curve fits and trend analysis plots.

In the second case, the relative effectiveness allows simulation replications that began their execution in the past to verify that they match the current real-time picture. The relative effectiveness for this second case is used to

12

prune those replications that do not match the real world picture. These two cases are shown below:

Case 1 – Simulated predictions:

€

RelEffectiveness(t) = 1− 1I B + I R

X i (t)−

Y i (t)

i

N∑ (7.6)

Case 2 – Simulated estimates:

€

RelEffectiveness(t) = 1− 1I B + I R

X i (t)−

Z i (t)

i

N∑ (7.7)

Note that for each of these cases, the magnitude of the vector difference is weighted for each vector value. The overall magnitude for each term in the sum is normalized and lies between zero and its intrinsic value. Computing the normalized state vector difference between the expected state and the predicted or estimated state is beyond the scope of this paper. It would be a rich topic for future R&D. However, characterizing how much a plan has deviated will likely involve issues such as: (1) location of the entity with respect to where it is supposed to be, (2) deviation of the plan and allowing for time considerations such as being behind or ahead of schedule, and (3) health of the entity with respect to the planned health.

A score of one would indicate that the plan is being executed as perfectly. It is important to remember that losses to assets may be expected, so a score of one does not imply the best possible outcome. A score of zero would indicate complete chaos, meaning that the plan has fallen apart and is no longer valid. Like a broken football play, it is still possible to achieve the objectives even for a low relative effectiveness score. For Case 2, a low score would indicate that the replication does not agree with reality and should therefore be pruned.

7 Summary and Conclusions This paper described how traditional estimation and prediction control theory can be combined with optimistic simulation techniques to support the estimation and prediction of complex real-time systems. To support this capability in an efficient manner, several important new concepts were introduced to support (1) multiple replication branching at key decision points within a single parallel execution of the simulation, (2) statistical algebra to operate on actual distributions instead of relying on random number generation, and (3) a formal mechanism to define raw and relative effectiveness metrics that characterize the outcome of a given plan.

Many of the techniques described in this paper have been successfully developed and/or prototyped in the WarpIV Simulation Kernel [15], which is an open source implementation of the proposed Standard Simulation Architecture [14]. The next step in this research is to extend the capabilities of WarpIV to fully support these capabilities and then to demonstrate these estimation and prediction techniques in a simple, but functional simulation.

7.1 Future Research

Future research will likely involve several steps. First, the techniques discussed in this paper must be fully developed and tested. Critical computationally intensive bottlenecks must be identified and optimized. Interfaces must be defined that are easy to use for model developers. Second, hardware acceleration techniques should be explored to improve performance when necessary. There appears to be a very good fit for FPGAs to speed up many of the algorithms described in this paper. Third, new modeling techniques must be explored to develop simulations having estimated and predicted state values based on inputs from external sources while having statistical properties without generating random numbers. Other estimation and prediction techniques such as extended Kalman Filters [5], Bayesian estimation theory [13], and Particle Filters [1] should be explored to address non-linear systems, stability/convergence issues, and situations where critical measurements are missing or unavailable. Fourth, analysis techniques and tools must be developed to characterize the different measures of effectiveness and performance. Fifth, optimization techniques can be developed to automatically analyze outcomes of branches and prune those that do not produce desired outcomes while generating new ones. These optimization techniques will likely break the traditional rules of event scheduling in discrete-event simulations by allowing events to schedule new events in the past. Rollback/rollforward and Global Virtual Time (GVT) management techniques in real-time will be required to enable this capability.

7.2 From Concept to Reality

Once all of the research issues are completely worked out, the final step is to develop validated models and to then integrate this capability into real-world systems. Live intelligence data feeds coming from the Global Information Grid (GIG) [12] will drive the estimation and prediction processes described in this paper. Analysis tools will give rapid feedback to the users to describe effectiveness and performance of the system. They will help users understand the current estimated state of the system as real-time data is received. The analysis tools will also provide feedback to the users to help them understand the predicted outcomes of the system.

7.3 Benefits

The benefits of developing these concepts are far-reaching and can be used to estimate, predict, and optimize many real-world systems. Examples where this technology would have a significant impact include: urban traffic management for the Department of Transportation, air traffic control for the Federal Aviation Administration, energy grid and individual power plants for the Department of Energy, space exploration missions for the National Aeronautics and Space Administration, and military battlefield operations for the Department of Defense.

13

8 Author Biography JEFFREY S. STEINMAN received his Ph.D. in 1988 from the University of California Los Angeles in High-Energy Particle Physics. Between 1988 and 1995, Dr. Steinman led several high-performance computing R&D activities at JPL/Caltech in support of Strategic Defense, Air Defense, Ballistic Missile Defense, and various NASA space exploration missions.

While at JPL/Caltech, Dr. Steinman pioneered the technology and software development of the Synchronous Parallel Environment for Emulation and Discrete-Event Simulation (SPEEDES) framework. This work resulted in five patents and more than fifty technical papers in high-performance computing, optimistic discrete-event simulation, data structures, message-passing algorithms, object-oriented design, parallel and distributed multi-resolution interest management, parallel high-speed communications, and HLA.

Dr. Steinman directed software engineering teams for several mainstream simulation programs including the Parallel and Distributed Computing Simulation (PDCS), the Parallel Naval Simulation System (NSS), Wargame 2000 (WG2K), the Joint Simulation System (JSIMS), the Joint Modeling and Simulation System (JMASS), and the High-Performance Computing Run Time Infrastructure (HPC-RTI). He has also provided consulting and software support many other programs including the Joint Warfare System (JWARS), the Extended Air Defense Test Bed (EADTB), Enhanced Naval Warfare Gaming System (ENWGS), and several others.

Dr. Steinman was an original member of the Time Management and Data Distribution Management working groups during the formation of the HLA standard. Dr. Steinman was the lead architect for the JSIMS Common Component Simulation Engine, and is currently overseeing the development of the next-generation open source WarpIV Simulation Kernel.

More recently, Dr. Steinman was the lead architect of the Dynamic Situation Assessment and Prediction (DSAP) Multi-Replication Framework (MRF) tool developed for the Air Force Research Laboratory, Rome Labs.

TIMOTHY BUSCH received his B.S. degree in electrical engineering from the State University of New York at Buffalo in 1986, the M.S.E.E. from Syracuse University in 1991, and the Ph.D. degree from Binghamton University in 1997 where he concentrated in model predictive control applied to beam stabilization for laser communication systems. He is a senior engineer with the Air Force Research Laboratory’s information directorate working on projects that bring a control theoretic approach to solving problems associated with the Air Force Command and Control System.

Current interests include the use of simulations to implement roll out strategies for determining policy in partially observed Markov decision processes (POMDP). Dr. Busch is a research advisor to the National Research Council and an assistant professor in the electrical engineering department of the State University of New

York Institute of Technology. He has authored/co-authored more than 25 papers.

9 References 1. Bolic Miodrag, 2004. “Architectures for Efficient

Implementation of Particle Filters.” Ph.D. dissertation, Stony Brook University.

2. Busch Timothy, "Modeling of Air Operations for Course of Action Determination" in Enabling Technologies for Simulation Science VI, Alex Sisti and Dawn Trevisani Editors, Proceedings of SPIE Vol. 4716, pp 35-40, (2002).

3. Chen Dan, et. al., 2004. “Incremental HLA-Based Distributed Simulation Cloning.” In proceedings of the 2004 Winter Simulation Conference, pages 386-394.

4. Fujimoto Richard, 2000. “Parallel and Distributed Simulation Systems.” John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012.

5. Gelb A., 1974. “Applied Optimal Estimation.” MIT Press.

6. Katsuhiko Ogata, 2005. “Discrete-Time Control Systems (2nd Edition).” ISBN/UPC: 0130342815. Pearson Education.

7. Hybinette, M. and R. M. Fujimoto. 2001. Cloning parallel simulations. ACM Transactions on Modeling and Computer Simulation, 11: 378-407.

8. Jefferson David, 1985. “Virtual Time.” ACM Transactions on Programming Languages and Systems, Vol. 7, No. 3, pages 404-425.

9. Lammers Craig, et. al., “Applying a multi-replication framework to support dynamic situation assessment and predictive capabilities.” Proc. SPIE Vol. 5805, p. 257-268, Enabling Technologies for Simulation Science IX; Dawn A. Trevisani, Alex F. Sisti; Eds.

10. Papoulis Athanasios, 1965. “Probability, Random Variables, and Stochastic Processes.” McGraw-Hill, Inc.

11. Phister Paul, Busch Timothy, and Plonisch Igor, 2003. “Joint synthetic battlespace: Cornerstone for predictive battlespace awareness.” In Proceedings of the Eighth International Command and Control Research and Technology Symposium.

12. Piccerillo Robert and Brumbaugh David, 2004. “Predictive battlespace awareness: Linking intelligence, surveillance and reconnaissance operations to effects based operations.” In Proceedings of the 2004 Command and Control Research and Technology Symposium, San Diego, California, June 2004.

13. Sage Andrew and Melsa James, 1971. “Estimation Theory with Applications to Communications and Control.” Robert E. Krieger Publishing Company, Inc.

14

14. Steinman Jeff and Hardy Doug, 2004. “Evolution of the Standard Simulation Architecture.” In proceedings of the Command and Control Research Technology Symposium, paper 067.

15. Steinman Jeff, 2005. “The WarpIV Simulation Kernel.” In proceedings of the 2005 Workshop on Principles of Advanced and Distributed Simulation (PADS).

16. Steinman Jeff, 2005, “Statistical Algebra Package - Programming Guide: Version 1.4.” Copyright © 2005, WarpIV Technologies, Inc.

17. Zarchan Paul and Musoff Howard, 2005. “Fundamentals of Kalman Filtering: A Practical Approach.” AIAA.