SIMULATION PRACTICE = THEORY

Simulation Practice and Theory 2 (1995) 291-301

End-user evaluation procedures associated with the interactive F-l 11C Pave Tack Simulation (FPTS)

Mike Davies *, Fred D.J. Bowden

Information Technology Diuision, Electronics & Surveillance Research Laboratory, Building 79 Labs Area, Defence Science and Technology Organisation, PO Box 1500, Salisbury SA 5108, Australia

Received 27 June 1994; revised 24 January 1995

Abstract

Development of any interactive system simulation is greatly assisted by end-user evaluation, particularly so at a concept demonstration stage. Evaluation of any individual aspect of the man-machine interface can sometimes be difficult, however, if the remaining features of the interface are underdeveloped or crude such that the general environment portrayed by the simulation differs significantly from that of the real system. The use of an additional, intermediary interface, taking the form of an experienced simulation operator, can help alleviate these difficulties; having the ability to make some of the operating procedures automatic can also be quite powerful. This paper discusses how such evaluation procedures were used in the development of the F-l 11C Pave Tack Simulation (FPTS) which is a large, event-stepping, interactive simulation of the F-l 11C navigation and weapon delivery system. The operator of the simulation takes the role of the F-l 11C navigator, flies a pre-set mission and interacts with that mission through the use of emulated Pave Tack and other cockpit controls and associated displays. The FPTS is being used in the training of F-l 11C navigators.

Keywords: Evaluation; Interactive; Pave Tack; Simulation; Training

1. Introduction

End-user evaluation is a critical factor in the development of an interactive system simulation. Such evaluation can involve, for example, manual control calibration and the assessment of system functionality, system response, and visual display characteristics. Evaluation of any individual aspect of the man-machine interface can sometimes be difficult, however, if the remaining features of the interface are

underdeveloped or crude such that the general environment portrayed by the simula- tion differs significantly from that of the real system. In the case of comparing the

* Corresponding author. Email: mcd@itd.dsto.gov.au

Elsevier Science B.V.

SSDZO928-4869(95)00005-4

292 M. Davies, F. D. J. BowdenlSimulation Practice and Theory 2 (199.5) 291-301

sensitivities of a personal computer (PC) and an aircraft joystick control for target tracking, for example, it is important that the end-user can provide feedback quickly before getting conditioned to using the PC-based emulation. There is a danger, then, in forcing the end-user to become proficient in the simultaneous use of other interface mechanisms that are not physically comparable with controls in the real system, since the general discomfort that this causes can impair or mar that end-user’s judgement. This paper reports on the use of particular evaluation procedures that helped alleviate such difficulties in the development of the F-111C Pave Tack Simulation (FPTS).

Pave Tack is an all-weather, day and night, navigation and weapon delivery system [l] installed on the F-1llC aircraft and used by the F-l 11C human navigator during F-11 1C missions. It encompasses a number of analog and digital systems which include sensors and avionics equipment. The FPTS is a large, interactive, event-stepping simulation, written in conventional Turbo Pascal and running on a PC, of the Pave Tack and associated systems. The simulation is considerably detailed and models the information flow between the central Pave Tack operational flight program and peripheral components, including the analog-to-digital and digital-to- analog interfaces. Owing to this level of algorithmic fidelity, the FPTS can be used as a tool in pre- and post-mission analysis, mission effectiveness studies, and also to study the effect of system error propagation. The simulation can also be used for training purposes in which case the operator would be an actual F-111C navigator (or trainee); it is with such an application of the FPTS that this paper is associated. The F-111C navigator is, therefore, referred to as the end-user. Configuration of the man-in-the-loop environment necessitated a detailed inspection of the nature, usage and effect of the numerous controls and display systems available to the human navigator. Such systems are adequately emulated in the FPTS through the use of a number of standard PC keyboard and joystick facilities, and computer graphics. The emulated interface allows actual navigator tactics to be employed during a simulated F-111C mission and the effect of such tactics to be observed and studied. The simulation offers, therefore, considerable training benefits.

Similar evaluation procedures to those presented in this paper could be employed in the development of other interactive simulations.

2. The F-l 11C Pave Tack and associated systems

The Pave Tack system is mainly composed of a forward looking infrared (FLIR) sensor and display; a laser; and an operational flight program which resides in the Pave Tack pod itself. The Pave Tack system interfaces with other aircraft sensors (such as the attack radar) which also have associated displays and numerous avionics systems such as the Inertial Navigation System (INS) and the Navigation Computer Unit (NCU) [l].

The main information that is calculated by the Pave Tack system during a mission is navigational data, such as the present position of the aircraft and aircraft-to-target (mission destination) range components. Such components are used to compute

M. Davies, ED. J. BowdenlSimulation Practice and Theory 2 (1995) 291-301 293

information such as the time to weapon release or impact. Owing to possible INS drift errors [4] and other sources of relevant inaccurate data, the navigational information can be somewhat unreliable. For this reason, the navigator is encouraged to carry out navigation updates en route to the target using known landmarks (offsets) which give good infrared (IR) and/or radar returns and for which their geographical location is known with some satisfactory confidence. The target itself can provide a means of updating.

The FLIR sensor allows the navigator to locate, view and designate ground-based features for the purpose of navigation updating, weapon delivery and damage assess- ment. Updating of the navigation system can also be achieved through the use of the attack radar. Lasing can be used to both get accurate range information during an update and also for guidance of laser guided bombs.

Despite the high-tech nature of the Pave Tack system, it is evident that realisation of the full capabilities of the system is very much reliant on the human navigator’s ability to carry out a number of duties.

3. The role of the F-111C human navigator

To illustrate the role of the navigator in an F-111C mission, consider the process of navigation updating and weapon delivery in the vicinity of the target. The following gives an overview of actual navigator practices in order to illustrate the basic interface characteristics of the FPTS.

The sensor displays available to the navigator can be presented using the primary (larger) and secondary screens of the Virtual Image Display (VID) unit. The navigator can select which sensor return resides on the primary screen and directly influence primary display features such as field-of-view (FOV) (magnification); mode changes; and image brightness and contrast. In their default modes of operation, both the sightlines of the FLIR and radar systems are slave to the stored location of the mission destination or target.

Consider the attack radar image as forming the primary display and the FLIR image the secondary. Assuming that the target radar return is visible, any navigation error will be manifested by the target being displaced from the desired location directly beneath the radar display crosshairs. The navigator can correct errors by depressing the Antenna Indicator Control (AIC) handle trigger to half-action and adjusting the radar azimuth and range cursors such that the desired state is achieved. The NCU uses the new position of the radar sightline to correct either the perceived present position of the aircraft or the stored position of the target - whichever is deemed by the navigator to be in error.

This form of navigation updating is, however, coarse. To achieve a more precise update, the navigator places the FLIR image on the primary display with the use of the Video Switching Control on the VID where the default mode of operation (cue mode) is assumed. A navigation update is initiated by again depressing the AIC trigger to half-action (coarse cue mode) and altering the pointing angles of the FLIR sightline. Navigation corrections are generated in this mode but are not used by the

294 M. Davies, F.D. J. BowdenlSimulation Practice and Theory 2 (1995) 291-301

NCU until truck mode is taken up by momentarily depressing the AIC trigger to full-action. In truck mode the navigator is required to maintain target designation (to “track”) via a thumb tracking joystick which sends corresponding acceleration (and deceleration) signals for use in updating automatically generated sightline slewing rates that provide assistance to the navigator. During this tracking phase, the navigator can decide to also lase the target in order to make use of accurate range information which results in further precision to the navigation update gener- ated by the NCU.

A similar procedure is adopted prior to and during weapon delivery. Target designation is particularly important in this phase of the mission. Tracking must be adequately maintained at all times even whilst the aircraft is making severe weapon delivery and egress manoeuvres such as toss bombing. The importance of the slewing rate assistance is quite evident in such situations.

Track mode can be terminated by the navigator again momentarily depressing the AIC trigger to full-action. Cue mode is again assumed and, following weapon delivery, can be used to conduct damage assessment.

The steps undertaken to achieve a navigation update using an offset aimpoint en route to the target are identical to those discussed above. A pre-stored offset is selected for viewing and sensors can be slaved to this landmark in preference to the target.

4. The FPTS and its interfaces

It is convenient to describe the general features of the FPTS with reference to the following four sections.

4.1. Main processing and assumptions

Since the actual Pave Tack and associated aircraft systems are a combination of analog and digital processes, each component can be viewed as having a certain processing rate varying from a fixed cycle rate of 1 Hz to being infinite (ie a continuous (analog) process). (The actual cycle rates are 1, 4, 8, 10, 16, 32, 40 and approx 240 Hz.) An event stepping process was found to be the best way to simulate this system.

The Pave Tack system is simulated through the use of 17 main events which are managed by an appropriate simulation controller or event handler. An event corres- ponds typically to the completion of a cycle rate mentioned above. An event that becomes activated at the head of the event queue is immediately re-scheduled (according to the time interval corresponding to its processing rate) and hence the number of main events in the queue is always fixed at 17. Continuous processes are modelled as discrete events with appropriate time intervals such that associated information required by the truly digital systems has no intrinsic staleness. Special considerations also had to be given to the ordering of “simultaneous” events. Real time processing, essential to the interactive requirements of the simulation, is achieved

M. Davies, F. D. J. BowdenlSimuhtion Practice and Theory 2 (1995) 291-301 295

through deliberate pausing of central event processing and limiting the refresh rate of displays. The discrete nature of such an event-stepping approach is not detrimental to the realism of the simulation owing to the small time steps associated with the main events.

The PC screen was used to emulate the display features of the primary VID display; no secondary screen was replicated. This does not degrade realism, since in the actual system it is mainly the primary screen that is interfaced.

4.2. The FPTSJlight path modelfor an F-l 11 C

The F-111C flight model is a combination of a six degree-of-freedom model designed by the Aeronautical Research Laboratory of the DSTO and a manoeuvre controller developed at the University of Newcastle [3]. The program is written in Fortran and runs on a Vax. The model is executed by carrying out a sequence of user defined manoeuvres (eg level acceleration; pull-up; turn; altitude change; dive and climb) resulting in an F-111C flight path consisting of latitude, longitude and altitude; inertial velocity components; and aircraft orientation data. Such data is output at a rate of 160 Hz in order to satisfy the sampling requirements of the FPTS events. The resulting flight path data files are down loaded from the Vax to the PC and the FPTS uses this information with the assumption that it is true, ie exact aircraft information about which system errors (such as those due to navigation) can be purposely introduced or generated.

4.3. Operator inputs

The F-111C navigator input interface is emulated in the FPTS through the use of standard PC keyboard and joystick inputs. Keyboard keys are used to emulate the many push-button, spring-loaded lever and rotary controls that are important to the navigator’s role in a typical mission. Some examples of such controls are given in Table 1, which is limited to the controls mentioned in the navigation update process of Section 3. In this table is an indication of the distributed location of the actual controls, their varied nature, and the simplicity of the FPTS implementation. Also indicated in Table 1 is the use of the PC joystick to provide AIC-related functions. The joystick is self-centring and hence suitable to represent both the AIC handle and the thumb tracker.

4.4. Display features

Emulation of the primary VID display is given on the PC screen. Owing to the requirement for real time execution, the amount of detail that can be presented is limited. Typical features of the FLIR and radar displays generated by the FPTS are illustrated in Figs. 1 and 2 respectively. The alphanumeric information is presented in a form identical to the actual F-111C system and covers such data as aircraft latitude and longitude, elevation and bearing along with other mission information [ 11, details of which are irrelevant to this paper. The sensor imagery, however, is

Tab

le

1 E

xam

ples

of

F-l

11

C

cock

pit

cont

rol

emul

atio

n in

th

e FP

TS

Rea

l F-

111C

en

viro

nmen

t Si

mul

ated

Coc

kpit

cont

rol

Func

tion

Nat

ure

of c

ontr

ol

Loc

atio

n FP

TS

equi

vale

nt

Dat

a m

ode

switc

h Se

lect

s m

ode

of F

LIR

di

spla

y

(nor

mal

, st

atus

, na

viga

tion)

Spri

ng-l

oade

d le

ver

VID

ho

usin

g (s

ide)

Rad

ar

rang

e sw

itch

Vid

eo

switc

hing

co

ntro

l

AIC

ha

ndle

AIC

tr

igge

r

FOV

co

ntro

l

Las

er

fire

bu

tton

Thu

mb

trac

ker

Off

set

sele

ctio

n sw

itch

Adj

usts

ra

dar

scre

en

mag

nifi

catio

n

Sele

cts

sens

or

imag

e fo

r pr

imar

y

disp

lay

Use

d fo

r co

arse

ta

rget

desi

gnat

ion

or

trac

king

Sele

cts

mod

es

of o

pera

tion.

C

an

be

at

half

or

fu

ll-ac

tion

(lat

ter

mom

enta

rily

)

Sele

cts

wid

e or

na

rrow

FL

IR

FOV

Star

ts

or

stop

s la

ser

firi

ng

Use

d fo

r fi

ne

targ

et

desi

gnat

ion

or

trac

king

du

ring

na

viga

tion

upda

ting

and

wea

pon

deliv

ery

Sele

cts

pre-

set

offs

ets

for

view

ing

(0

to

6)

Gra

duat

ed

rota

ry

cont

rol

Spri

ng-l

oade

d le

ver

(tog

gle)

H

and

held

ha

ndle

Fing

er

activ

ated

tr

igge

r

Thu

mb

activ

ated

fl

at-

surf

aced

sw

itch

(tog

gle)

Thu

mb

pres

sed

butto

n

(tog

gle)

T

hum

b ac

tivat

ed

cent

ralis

ing

joys

tick

Gra

duat

ed

rota

ry

cont

rol

VID

ho

usin

g (f

ront

)

VID

ho

usin

g (s

ide)

AIC

ha

ndle

AIC

ha

ndle

as

sem

bly

AIC

ha

ndle

as

sem

bly

AIC

ha

ndle

as

sem

bly

AIC

ha

ndle

as

sem

bly

Mul

ti O

ffse

t A

impo

int

Pane

l

“L”

key

(cyc

les

thro

ugh

thre

e

mod

es)

“ <

” ke

y ~

decr

ease

“ >

” ke

y ~

incr

ease

“P”

key

(tog

gle)

Joys

tick

‘7”

key

~ at

ha

lf-

actio

n or

re

leas

ed

(tog

gle)

“Y

” ke

y -

at

full-

actio

n an

d th

en

rele

ased

“w”

key

(tog

gle)

“F”

key

(tog

gle)

Joys

tick

“o”,

“l”

to

“6”

keys

M. Davies, F. D. J. BowdenlSimulation Practice and Theory 2 (1995) 291-301 297

Fig. 1. FPTS simulated FLIR display

SEQ 99

OAP 0

ALT 000

900

TLT 024

RNG 028 800

N35 46 38

E 117

47 37

TG 26

BRG

I R

EBL

R/M 006

GA

Fig. 2. FPTS simulated radar display.

298 M. Davies, I? D. J. BowdenlSimulation Practice and Theory 2 (1995) 291-301

quite basic and consists typically of a small circular target in the radar display, and flat filled rectangles for runways with a stick figure for a building and a straight- lined horizon in the FLIR display.

The graphics capability of Turbo Pascal was used initially but found to be too slow, even with the limited image fidelity. A graphics card using a Am95C60 quad pixel dataflow manager chip [ 23 was therefore used and provided considerable speed advantages. Definitive points on the FLIR Image, described in terms of ranges from the target and coordinate transformation techniques, are employed to give perspective qualities on the two-dimensional screen. As the aircraft approaches the target, the perspective and size of the image change realistically, with image features being clipped using an adapted Sutherland-Hodgeman algorithm [ 51. Alphanumeric infor- mation and other peripheral data is updated during a mission using system data from associated modules of the sophisticated FPTS events.

5. Operation and evaluation of the FPTS

In a given simulated mission, the aircraft is considered to follow a fixed flight path and the operator of the simulation, playing the role of the F-l 11C navigator, interacts with features of the Pave Tack system. Through the use of the emulated cockpit controls the operator can carry out such practices as navigation updating in order to attempt to eradicate, for example, navigation error purposely introduced at simulation initialisation. A typical time series for an error correction process is given in Fig. 3, which shows the correction signals generated by a manual navigation update procedure conducted in response to an initial error of 340 ft in target location (east range to target from aircraft). Numerous filters and checks are carried out on the correction signals and the conditions under which they were derived, and in this particular example corrections were only actually used after the firing of the laser. The visual effect of different modes of operation and the impact on tracking difficulty of aircraft manoeuvres are also important aspects that can be experimented with. A “hands free” perfect tracker option is also incorporated within the FPTS.

Development of the FPTS necessitated considerable research of the associated F-l 11C systems, including painstaking inspection of detailed system specifications and assessment of system behaviour. The use of footage from on-board video record- ing equipment, capable of recording the primary VID display plus audio comments by both pilot and navigator for a select period during the mission, also featured heavily in this process. Despite the continual evaluation that occurred during devel- opment, obtaining feedback on the simulation by actual F-l 11C navigators was still a critical requirement.

In terms of such end-user evaluation, there were a number of main items that needed to be addressed; namely, joystick sensitivity and system response during tracking; representation of modes of operation; sensitivity to aircraft manoeuvres and approach to target; and fidelity of sensor imagery displays.

The FPTS keyboard-based interface is a severe simplification of the actual cockpit controls available to the F-111C navigator. It can be expected, therefore, that a

M. Davies, F. D. J. BowdenlSimulation Practice and Theory 2 ( 1995) 291-301 299

500

-..-....._ Pod correction signal Coarse Cue Mode

Entered At 6.1s / \h ; *

-250 - Laser Fired At 29.3s

Track Mode Entered At 16.9s

-500’ ’ , ’ * . ’ . s ’ * . . ’ 0 20 40 60 60

Time (s)

Fig. 3. Example navigation update.

significant period of familiarisation would be necessitated before normal operations could be conducted without impairment. It is too much, therefore, to expect an end- user to operate all aspects of the FPTS interface at any one time. It was possible, however, to conduct an evaluation of the items required through the combined use of an experienced FPTS operator, automatic operations, and the end-user.

To evaluate joystick sensitivity and system response during tracking, the experi- enced FPTS operator would operate the keyboard and, in this regard, act as an intermediary interface between the navigator and the FPTS. This arrangement is shown pictorially in Fig. 4. The navigator would carry out a variety of tracking tasks and issue verbal commands that the keyboard operator would translate with minimal response time. For example, at a particular stage leading to weapon delivery the navigator might voice “Pave Tack prime!, Coarse Cue!” in which case the keyboard operator would hit the “P” key followed by the “T” key. This approach left the navigator free to carry out the tracking task and, through observations of the primary VID display displayed on the PC screen, assess system response characteristics and joystick sensitivity. Joystick sensitivity could be altered during run time in response to the end-user’s requests.

Making the tracking task automatic, through the use of the perfect tracker facility, also proved invaluable to the evaluation process. An identical configuration to that used for joystick and system response evaluation could be utilised, but this time the navigator would be relieved of the burden of tracking. This gave the end-user the ability to comfortably observe the basic FPTS emulation of the numerous modes of operation, eg cue, coarse cue and track, and the associated display characteristics

300 AL Davies, F. D. J. BowdenlSimuhtion Practice and Theory 2 (199.5) 291-301

F-l 11C Flight Path

\

program data (downloaded prior to FPTS execution)

Fig. 4. FPTS configuration for end-user evaluation.

for a variety of aircraft missions. The FPTS can also be made fully automatic in that, for a given aircraft mission, standard Air Force operating procedures could be incorporated. Such procedures involve adopting certain modes of operation at specific instances during an F-111C attack profile. Pre setting such mode changes to occur automatically in the FPTS again permitted a more comfortable evaluation of system characteristics during the missions concerned. As an example, target imagery on the Pave Tack display is known to exhibit distinct orientation patterns when the aircraft flies directly over a target which are not present if the aircraft does a fly-by. Replication, by the FPTS, of such phenomena could be inspected.

Overall, through the use of such procedures and facilities, the navigator was not inhibited by the FPTS interface and his knowledge and experience could be best utilised to help achieve the desired realism and to calibrate aspects of the simulation such as joystick sensitivity. It was clear from such trials that the very basic fidelity

M. Davies, F D. J. BowdenlSimulation Practice and Theory 2 (1995) 291-301 301

of the display imagery was by far adequate and it was concluded by both the developers and navigators that the FPTS satisfactorily emulates the fundamental features of the F-111C Pave Tack system.

6. Conclusions

An indication of the confidence placed in the FPTS by the Royal Australian Air Force (RAAF), and the success of this project, is that the computer simulation has been incorporated as a vital inclusion to an F-l 11C simulator used for the training of both pilots and navigators. The FPTS provides Pave Tack capability to the simulator and offers a means of “hands-on” training otherwise only achievable through flying hours in an actual F-111C aircraft. The training benefits of the FPTS are, therefore, being realised. In including it as part of the simulator, the RAAF have taken the interactive element of the FPTS further by interfacing it to cockpit hardware associated with the F-111C navigator. With reference to Fig. 4, the PC processor is still employed but with the VID display passed to an actual VID unit; operator input achieved through use of the cockpit hardware indicated in Table 1 rather than the PC keyboard and joystick; and flight path data being fed during run-time from the simulator. Interest has also been expressed in using the FPTS as a stand-alone PC-based training tool in a one-to-many classroom environment which would provide additional training benefits.

This paper has described the interactive aspects of the FPTS which include emulation, in generally a basic manner, of F-111C navigator cockpit controls and sensor displays. The operator interface was achieved through the use of standard keyboard and joystick facilities. The employment of particular evaluation procedures meant that such facilities were sufficient to convey clearly the navigator training benefits offered by the simulation. The use of an intermediary interface and automatic control operation is detailed and proved to be an essential aspect of the FPTS evaluation procedure. It is expected that such evaluation procedures could be employed in the development of other interactive simulations that require some form of assessment by an associated domain expert.

References

[l] AN/AVQ-26 Pave Tack system description, Part 1, Ford Aerospace and Communications

Corporation, Aeronautronic Division, PT-78-FS-001, Newport Beach, CA, February 1978.

[2] T. Crawford, S. Tindall, E. Dupuis, W. Reis and A. Strupat, Am95C60 QPDM Quad Pixel Datajow Manager (Advanced Micro Devices, Sunnyvale, CA, 1988).

[3] P.W. Gibbens, Manoeuvre Controller Design for an F-111C Flight Dynamic Model, Aeronautical

and Maritime Research Laboratory ARL-FLIGHT-MECH-TM-463, Melbourne, to be published.

[4] G.R. Pitman Jr, Inertial Guidance (John Wiley & Sons, New York, 1962).

[S] D.F. Rogers, Procedural Hementsfor Computer Graphics (McGraw-Hill, Singapore, 1985).