human factors integration and human centred design concepts
TRANSCRIPT
Diploma in Aviation Medicine:Human Performance
Revision June 11
Objectives of Aviation Psychology
• to enhance flight safety
• to improve effectiveness
Why is There Interest in Human Performance in Aviation?
• Aviation is a safety-critical operation• Aircrew are subjected to many sources of stress• High levels of human performance must be achieved (e.g., fast jet pilots)• Human error is heavily implicated in aviation accidents
Main fatal accident causes in 1994
Accident cause Accidents (%) Fatalities (%)
Air crew error Controlled Flight into Terrain Weather Loss of Control Engine Failure/Fire Structural/System Failure
65 37 30 14 16 11
68 43 24 34 7 13
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System
Content of Human Performance Module
General Principles • Introduction to Human Performance Module• Fundamentals of Human Performance• Individual Differences• Social Psychology and Aviation
• A small amount of basic theory to help you to interpret the practical studies
Content of Human Performance Module
New this year! • An early session on human error
• …to provide a context for the module
Content of Human Performance Module
Personal & Environmental Factors Stress & Workload in Aviation I Stress & Workload in Aviation II Perceptual Issues in Aviation Situation Awareness Selection of Aviation Personnel
Content of Human Performance Module
Training and Simulation Simulation and Training Fundamentals of CRM Training Practical Aspects of CRM & LOFT
Content of Human Performance Module
Systems Factors Aviation Ergonomics I Aviation Ergonomics II
Content of Human Performance Module
The Human Factor in Aviation AccidentsSeminar: Flight Safety
– Prof Peter Jorna, former head of division at NLR Amsterdam
Also an accident module at Henlow, providing a context for this moduleSee also lectures on Sleep, Fatigue and Shift-Working
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System: Relevance of Module Topics
SelectionIndividual
Differences
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System: Relevance of Module Topics
ErgonomicsWorkload
Technical Training
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System: Relevance of Module Topics
StressPerception
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System: Relevance of Module Topics
CRM Training
Equipment& tasks
Environment
Otherpersonnel
Humanoperator
The Human in the Aviation System: Relevance of Module Topics
‘SituationAwareness’and ‘Human
Error’ encompass all these interactions
Human Information Processing
Cognition
Processes involved in the input, storage, transformation, and output of information by humans
Main topics:
• Memory
• Attention
• Skills
Memory
Three major memory systems:
• Sensory memory
• Short-term memory
• Long-term memory
Summary of properties of memory systems (inferred from classicexperiments on memory) learn this!
Sensory Short-term Long-term
Capacity High 72 chunks no knownlimit
Duration 1 sec 10-15 sec permanent?
Type of storage Physical acoustic semanticcharacteristics
Nature of retrieval parallel serial hierarchy?
Nature of forgetting decay/maskinginterference failure ofretrieval
Attention
Selective attention: attend to one of several competing sources of information
Divided attention: attempt to attend to more than one information sourceor task at the same time
Shadowing taskCan detect physical changes on the unattended channel, but not semantic content
Dichotic listening taskPerformance very poorSubjects tended to organise their recall by ear, not by recency of presentation
Led Broadbent to propose Filter Theory. But, later shown that• subjects tend to hear their own name on the unattended channel• subjects tend to follow the message, even if it switches ears• hence, there is semantic processing on the unattended channel
However, we can assume that recognition of unattended information is less likely than recognition of attended material
Make sure that you understand this!
Divided attention
Key question:
Does man have a single information-processing channel (all tasks compete for the same ‘resources’ or ‘capacity’)
or
specialised resources for particular types of activity? (tasks performed concurrently compete only if they draw upon the same resources)
Some support for the multiple resource theory: often, the degree oftask interference depends upon the similarity of the tasks
But: sometimes tasks that are dissimilar are found to interfere
Baddeley’s working memory model is a compromise between extremesingle-channel and multiple-resource views
Central Executive
Visuo-spatial scratchpad
Articulatory loop
Skills (obviously relevant to training lectures!)Characteristics• typically a sequence of activities• goal-directed behaviour• use of feedback
Skill acquisition
Three phases are sometimes distinguished:• Early or cognitive phase• Intermediate or associative phase• Final or autonomous phase
In the final phase, behaviour becomes automatic; delegated to thecontrol of ‘motor programs’ that do not require conscious attentionand do not place heavy demands for mental resources
Many everyday errors (actions not as planned) are associated with overlearned behaviourThese errors involve well-practised behaviour, but are inappropriateSome aircrew errors are of this type
Issues in skill acquisition
• Whole versus part learning
• Massed versus spaced learning
• Transfer of training very important aspect of simulator-based training
Individual Differences
Two major types of individual difference covered: Intelligence/ability/aptitude
Personality
Factor Analysis: make sure that you have a good intuitivegrasp of this: you don’t need to know the underlying mathematics!
Basic psychometric criteria
Intelligence/ability/aptitude These are the key issues
Intelligence: Innate or learned?
Intelligence: How many abilities?
• There is evidence for a general ability factor• However, specific abilities also appear to exist
Intelligence: The Intelligence Quotient (IQ)
Aptitudes
Test Fairness
Intelligence: Are IQ tests valid?
Personality key issues
Types of personality test: Interview (not reliable) Projective tests Personality questionnaires
(discussed examples of each)
Is there a ‘pilot personality’?
Does personality influence success in flying training?
Is there an ‘accident-prone’ personality’?
Social Psychology
Types of social influence (can use this info for CRM questions)
Compliance: behaviour consistent with direct request• foot-in-the-door phenomenon• door-in-the-face phenomenon Conformity: behaviour consistent with group norms• size of group (up to about four)• attractiveness and status of group members• Informational influence (trusting others’ judgements) and normative influence (seeking group acceptance) Obedience to authority • Milgram experiment• 62.5% of the 40 subjects administered shocks to the highest level• factors affecting obedience, such as status of experimenter, proximity to ‘student’
Think about how this applies to small groups of interest to us,such as flight crew or teams of maintenance engineers
Group Decision Making: Polarisation
• Was thought that group decision making was more risky than individual DM (‘risky shift’) — but became apparent that there is a shift in the direction of the pole that, on average, the group favours as individuals (polarisation)
• Stoner’s experiments
• Normative and informational influences produce group polarisation
Group Decision Making: Groupthink• Work of Janis. Based on real-life examples such as Bay of Pigs (or, more
recently, UK MPs’ expenses!)• Desire for consensus overrides group members’ motivation to assess risk
and consider alternative courses of action
• Groupthink occurs under the following conditions:• High cohesiveness of the group• Uncertainty of approval• Insulation of the group• Directive leadership• High stress situations
• Symptoms include• Illusion of invulnerability• Stereotypes of out-group• ‘Mindguards’• Direct pressure on dissenters• Collective rationalisation
• Effects on decision making:• Incomplete survey of alternatives• Incomplete survey of objectives• Incomplete analysis of risks associated with course of action• No contingency plans
Aeronautical Decision Making (ADM)• Work of Jensen: decision error is cause of most fatal aviation accidents;
argued that decision making can be improved through training
• Decisions have two components:• Rational judgement (‘Headwork’)• Motivational judgement (‘Attitudes’)
• Hazardous attitudes:• Anti-authority• Resignation• Impulsivity• Invulnerability• Macho
• ADM courses aim to provide:• Ability to recognise hazardous attitudes• Knowledge of effects of these attitudes • Skills to overcome the effects
• Methods• Self assessment tools• Examination of case studies• Practical Exercises
Perceptual Issues in Aviation
Perception is the process of acquiring, selecting, and organising sensory information
The most important perceptual processes for aviation are those associated with vision and hearing
1 The ear and the auditory system– balance and the vestibular system– localisation of sound and identification of source
2 The visual system– bottom-up processing– top-down processing– cues to depth perception
The ear and the auditory system
Balance. The vestibular system of the inner ear detects angular and linear accelerations of the head
Hearing. To detect sounds, to determine the location of their sources and to recognise the identity of these sources
The ear serves two main functions:
Practical implicationsBalance and the vestibular system
Accelerating aircraft
The situation is aggravated if the pilot attempts to compensate for an incorrect percept. Although feedback from the vestibular system can be compelling, a pilot needs to learn to trust instrumentation.
With regard to the otolith, the weight force in a climbing aircraft operates similarly to the resultant force in an accelerating aircraft. Without visual feedback, pilots can mistake acceleration for pitch.
Ascending aircraft
Weight force
Inertia force
Resultant
Localisation of soundAuditory perception
Interaural differences:•Intensity. Most suited to localising high frequencies
•Time/phase. Most suited to localising low frequencies.
Sounds emanating from directly in front and behind the head produce the same interaural differences.
Practical implications Cockpit design
– The cockpit relies heavily on the presentation of visual information. Adoption of auditory signals may reduce the workload experienced by pilots in the visual domain.
Localisation of auditory warnings– Similar sounding warnings emanating from similar
areas may cause confusion– Adoption of white noise bursts within ambulance
sirens
Auditory perception
What you see is what you get?Visual modality is obviously extremely important in aviation. But can we always trust our eyes?
The visual scene is captured by the eye as a poor quality, two-dimensional representation
What is perceived is determined by:‘Bottom-up’ processes. The percept of a stimulus is determined by features of the stimulus as processed by the visual cortex‘Top-down’ processes. The interpretation (consciously or not) of a stimulus can be determined by our experience and knowledge
Important distinction!
Visual perception
If the percept is generated deterministically (bottom-up processing) from the visual cortex . . .
. . . how can one distal (real world) stimulus produce two percepts?
By a mental model: Our own experience and expectations help to determine what we see (top-down processing)
Old or young women?
Thirteen or ‘B’?Necker cube
Lincoln or women?
Top-down processesVisual perception
Convergence– of the eyes.
Stereopsis– disparity between the
two images. Accommodation
– of the lens. Retinal versus actual size
– for known objects. Overlap
– a near object will occlude the view of a far object.
Position in visual field– objects nearer the horizon
are farther away. Aerial Perspective
– clarity of objects is reduced at distance.
Relative motion– angular velocity greater for
near objects.
All require both bottom-up & top-down processing.
Depth perception: learn this!Visual perception
44
• Position in visual field• objects nearer the horizon are farther
away
• Textual Gradient• Surfaces will have a finer texture with
distance
• Stereopsis• Binocular disparity between the two
images
• Convergence• of the eyes
• Occlusion• a near object will occlude the view of a
far object
• Perceptual constancy• Retinal versus actual size
• Relative motion• angular velocity greater for near objects
• All require both bottom-up & top-down processing.
How we perceive depth
Know this
Featureless surfaces, or those with textures of unknown sizes, can produce inaccurate judgements of size.
– Sea.– Beehives for caravans.
Can produce an inaccurate mental model of the situation which overrides the correct perception of the instruments.
– Top-down influences.– Exacerbated by fatigue and workload.
Some perceptual problemsVisual perception
Pilots may have to visually judge the glide slope without any cues other than those from the surface of the world.
The ‘aspect’ (retinal shape) of the runway is not very useful.
However, the visual touchdown point is a constant and unchanging cue, relative to the horizon.
If the horizon cannot be seen, its location must be implied,– The runway’s sides meet at the horizon.– The terrain’s texture gradients.– The relative position of the aircraft’s canopy.
Practical implications: visual approach
Visual perception: know the practical implications (next few slides)
Visualtouchdownpoint
HORIZON
A
B
Angle of Approach A = B
Practical implications: visual approach (2)Visual perception
= angle of approach
Visualimpactpoint
Actualtouchdownpoint
Practical implications: visual approach (3)Visual perception
Identification of a colliding aircraft is confounded by;– Constant relative bearing.
• Unique characteristic.• Periphery of retina detects sensitive to movement.
– Non-linear increase in retinal size.• Retinal image doubles with each halving of closure
distance.– Uneven visual acuity across the retina.
• Maximal acuity at the fovea.• Detection only if pilot is looking directly at it.• Implications for visual scanning to acquire proximal
image on the fovea.
Practical implications: mid-air collisionsVisual perception
Aircraft A
Impact
Aircraft BRelative Bearing
Practical implications: mid-air collisions (2)Visual perception
3 secs / 0.5 degree
1.5 secs / 1 degree
0.1 secs / VERY BIG
Practical implications: mid-air collisions (3)Visual perception
Bottom-up (information from our senses) and top-down (expectations and experiences) processes affect the way we perceive the world.
The resultant perception is often not a true reflection of the external world.
This can be advantageous when it is in our interest for differences between features in the external world to be exaggerated but potentially catastrophic when perceptual illusions lead us to take inappropriate behaviour.
SummaryPerception
Ergonomics
Why ‘Ergonomics’? – Murrell
The HSI Framework – seven domains– Manpower– Personnel– Training– Human Factors Engineering (aka Ergonomics)
• Workplace design• Anthropometry• Critical Dimensions
– System Safety– Health Hazards– Social & Organisational
HSI often called Human Factors Integration (HFI) – HFI is really the process by which HSI is applied to equipment procurement
Some people adopt a strict definition of ergonomics; others treat all of HSI as being within the scope of ergonomics. You would not be penalised for adopting the latter definition!
Tragic consequences
Kegworth
USS Vincennes
Herald of Free Enterprise
Chernobyl
Three Mile Island
HSI Domains KNOW THESE
Manpower: numbers of personnel required to operate, maintain, sustain, & train to deliver capability (e.g. aircrew complement)
Personnel: cognitive/physical capabilities required to train for, operate, maintain, sustain system
Training: instruction/education/ training to provide job skills, knowledge, values, and attitudes (different methods summarised)
Human Factors Engineering (aka Ergonomics)Workplace designAnthropometryCritical Dimensions
Systems Safety: applying HF expertise into programme Safety Management Process
Health hazards: conditions inherent in the system that may cause injury or reduce performance or well-being
Social/organisational factors: applying techniques from organisational psychology, social sciences, information science, and system of systems
Human Factors Engineering(aka Ergonomics)
focused on the integration of human characteristics into system definition, design, development, and evaluation to optimise human machine performance under operational conditions.
Workplaces & interfaces Cockpits Workstations Control rooms Offices Transport systems Factories
Controls Displays Computer hardware Computer software Protective clothing Other people
Physical workplace design Inputs required from
– EHFA– Task analysis– Link analysis– Allocation of function
Consider– Operational and environmental context– Human dimensions– Biomechanics and physiology
Functional factors Task issues
– Procedures– Critical elements
Responsibilities of organisation and individuals Communications
– Verbal– Non-verbal
Visual issues, such as sight lines Flows of materials and personnel Access and clearance
– Normal– Emergency– Maintenance
Protection– Protective clothing & equipment– Barriers & guards
Anthropometry Physical human dimensions Population specific Linear dimensions, for example:
– Stature– Functional reach– Sitting height
Girth dimensions, for example:– Waist – Head circumference
Each dimension is expressed in terms of percentile
Be careful with percentiles when applying anthropometry
Requirements often state …must accommodate the 5th percentile and the 95th percentile human…
But, these people do NOT exist!
Critical dimensions Choose dimensions relevant to the workstation,
posture, and task– Sitting, standing, reach, fit, walking, crouching
5th percentile (smaller) dimensions considered for:– Seat adjustment, reach, vision, control movement, foot
rests 95th percentile (larger) dimensions considered for:
– Seat adjustment, ingress, fit, access, clearance
Clothing Clothing increases most dimensions through the
addition of bulk– e.g. stature, sitting height, chest depth, shoulder
breadthBut Decreases the reach dimensions due to restriction of
movement– e.g. functional reach, vertical functional reach
When to integrate Human Factors
(Eurocontrol, 1999)
HSI “Designed” to Fit MoD’s Acquisition Operating Framework (AOF) Policy and Good Practice
CADMID cycle System Readiness Levels
(DEF STAN 00-250. May 2008; http://www.aof.mod.uk ; www.hfidtc.com )
MoD JSP to be introduced later this year
Concept Assessment Demonstration Manufacture In service Disposal
Initial Gate Main Gate System Acceptance
Has now happened
Summary HSI covers all aspects of applied human factors Human Factors Engineering is just one element that
needs to be integrated HSI comprises tools and processes that fit with
systems engineering HSI is widely applicable Early inclusion is so much better than late
intervention
Stress and Workload
Types of stress
• Life stress less important than the others in this context, but be aware of it • Environmental stress • Cognitive stress
Life stress
Typically measured by questionnaire
Some correlation between questionnaire scores and illness
Some evidence that life stress is associated with accidents
Level of arousal
Performance
Yerkes-Dodson law: Know this
Inverted U relation between arousal and performance
Performance declines as arousal increases or decreases from the optimal level
The optimal arousal level is inversely related to task difficulty
Difficulttask
Easytask
Environmentalstress
Fear• disruption of manual dexterity• disruption of secondary task performance
Noise• greater effect on difficult tasks• effect on error• increased attentional selectivity• effect on arousal (increases initially, then returns to normal)
Sleep loss• periodic lapses• decreased attentional selectivity• greater decrement on ‘easy’ tasks• decreased arousal
Hypoxia• performance affected at over 10,000 ft • some evidence that task learning is affected at only 8,000 ft
Combined stressorsSleep loss and noise each impair performance in isolationHowever, noise improves the performance of sleep-deprived individuals
Know this
Patterns of effects of stressors (from Hockey)
Stressor Arousal Selectivity Speed Accuracy STM
Noise + + 0 - -Anxiety + + 0 - -Incentive + + + + +Stimulants + + + 0 -Heat + + 0 - 0Alcohol - + - - -Sleep loss - - - - 0Fatigue- + - - 0Depressants - - - - -
+ increase- decrease0 no effect
no need to memorise all this, but know that each stressor has its own pattern of effects (cannot be explained by Yerkes-Dodson law)
Personality and stressKnow this
Two major dimensions of personality are:neuroticism (trait anxiety)introversion-extraversion
Introverts are chronically over-arousedExtraverts are chronically under-arousedAn arousing stressor (caffeine) has different effects on these individuals
Trait anxiety comprises worry and emotionalityWorry appears to interfere with task performancePerformance of high-anxiety subjects impaired under high workload
Evidence that personality influences success in flying training
Is the Yerkes-Dodson law adequate? Know this
For:
• Can explain effects of combined stressors
• Can explain some effects of personality
• Can explain some effects of task difficulty (e.g. greater effect of sleep loss on easy tasks)
Against:
• Does not explain specific patterns of effects of individual stressors
• Does not explain effects on attention
• Too flexible: does not lead to firm predictions
Factors influencing the effects of stressors know this
• Task difficulty
• Task duration
• Personality
• Intensity of the stressor
• Motivation
• Importance of the task component
• Presence of other stressors
Workload (cognitive stress) know thisTypes of workload measure
Subjective Example: NASA Task Load Index • easy to obtain• face valid• unobtrusive• subjects can readily quantify their experience
Physiological Example: heart rate variability• do not disrupt performance• often provide continuous record
Performance-based Primary task or secondary task (e.g. time estimation)• provide direct measure of operator performance
• difficult to establish which questions to ask (dimensions of workload)• difficult to compare different types of task• ratings may not be correlated with task performance
• equipment may be physically intrusive• only indirect indication of performance
• operator may invest more effort to maintain primary-task performance • choice of secondary task is important
Effects of high workload
• operator is prone to actions not as planned: unable to monitor activity fully• increased attentional selectivity• may respond quickly but inaccurately• may shed some sub-tasks completely
Strategies for workload reduction [know this]
Change the task:apply sound ergonomic principlesautomate some functionsuse new technologies
Change the operatorprovide extensive training, to produce motor programs (overlearning)
Personnel selectionfor example, low trait anxiety may confer betterability to cope with high task demands
Selection
Aims:
Deciding • What to measure• How to measure• Effectiveness of measures
Stages in Selection System
Specify selection criteria
Specify assessment methods
Evaluate
(After Hunter & Burke 1995)
The Systems Approach to developing selection processes.
1. Job / Competency analysis — Identify Knowledge, Skills and Attitudes (KSAs) required
2. Use KSAs to identify appropriate selection methods.
3. Establish the reliability and fairness of the process
4. Validate
Job Analysis
PersonRequirements
Aptitudes
Predictors
PersonSpecification
Job Requirements
TaskCompetencies
Criteria
JobDescription
Prediction
(After Hunter & Burke 1995)
Job Analysis
• Aim — Identify critical competencies required for successful job
performance
• Outputs — What does the job holder do?• Inputs — what skills, knowledge, abilities does the job holder need?
• Result is a competency framework identifying critical success factors associated with successful performance
• Why?1. To achieve the best possible prediction of job performance (put the
right people in the job)2. Legal requirement – test fairness
Types of Job Analysis
1. Hierarchical task analysis (Annett, Duncan et al 1971)
2. Functional Analysis (Fletcher 1991)
Techniques: Critical incidence technique (CIT) (Flanagan 1954*)
• Identify key roles and functions of job• Identify critical behaviours (related to success or failure)• Classify into similar behaviours• Summarise• Validate using other SME
Other techniques include Repertory grid
Classifying Aptitudes
Fleishman’s Taxonomy of Skills
Abilities were classified into:• Cognitive: Information processing and problem solving• Perceptual/spatial: Attention and spatial orientation• Physical: Flexibility, strength and stamina• Psychomotor: Coordination and reaction time
NATO Study: Aptitude dimensions for military fast-jet pilots (Bydorf 1993)• Situational awareness: Perceptual closure + reaction time• Spatial orientation• Time sharing• Aggressiveness• Divided attention• Psychomotor coordination• Perceptual speed• Selective attention• Visualisation
Weighting Aptitudes
Determining priorities
• Need to identify relative importance of aptitudes in job performance
• DIF Analysis. Ratings of:• Difficulty• Importance• Frequency
Choice of Measure
LifeExperiences
Ability Motivation Temperament
Performance
CV; BiodataInterview
Personality QuestionnaireGroup exercises
•Occupational Interest Inventories; •Measures of
personal values•Interview
• Psychometric tests• Work sample tests
• Physical tests
Types of aptitude measure• Paper & pencil measures
• Computer-based testing: BARB (British Army); OASC (RAF); MicroPat (AAC, RN, BA, Cathay); TASKOMAT (Commercial); BAT (USAF)
• Ease of administration• Experimental testing• Dynamic measures possible• Measure processing capacity• Multi-tasks• Sophisticated measures such as response latency
• Work sample • RAF Flying Grading
• Simulation based•Advantages of CBT & work sample• Lower costsExample: Canadian Automated Pilot Selection System
• Biodata
• Personality measures see other lectures
Effectiveness of measures
Evaluating Selection: Reliability and Validity
Reliability Accuracy and stability of the test• Internal consistency reliability• Split-half reliability• Parallel forms• Test-retest reliability • Inter-rater reliability
Validity Does the test really measure what it claims to measure?• Construct validity • Content validity • Predictive validity
See other lectures as well!
Predictor score
Perf
orm
ance
sco
re
FalsePositivesTrue
Negatives
TruePositivesFalse
Negatives
Cut-off score
‘Pass Mark’
Error in allocation
Predictor score
Perf
orm
ance
sco
re TP
FPFN
TN
Higher Correlation reduces error
Predictor score
Perf
orm
ance
sco
re
Cut-off score1
‘Pass Mark’
Cut-off score2
Effect of setting Cut-off scores
Great 8 Competency Factors Overall job performance Leading and deciding 0.35 Supporting and co-operating 0.23 Interacting and presenting 0.30 Analysing and reporting 0.35 Creating and conceptulising 0.42 Organising and executing 0.45 Adapting and coping 0.25 Enterprising and performing 0.37
Average Correlation between competency ratings and job performance
Interpreting scores• Norm referenced — most cognitive/ability tests
• Self referenced — Attitude/Personality measures• Criterion-referenced — job skills
Validity of Different Methods
Selection Method Mean Validity Co-efficient
Interview - Unstructured 1
Interview - Structured 2
.14
.35
Biodata 1 .37
References 3/1 .17 to .26
Cognitive ability testing 4/1 .25 to .53
Personality testing 1/5
Work-sample tests 1
Trainability tests 6
.10 to .33
.54
.46
Example: RAF Aircrew Selection
READY TO CHANGE YOUR LIFE?Your visit to OASC will take several days and includes:• an initial briefing;• aptitude tests;• an aptitude test review;• the exercise phase;• an interview;• an occupational medical; and• fitness assessments. (from OASC brochure)
Example: RAF Aircrew Selection
ADPO10369 EVOLUTION OF APTITUDE TESTING IN THE RAF M. Bailey, RAF Cranwell
Before 1940: main method was unstructured interview
About 50% pilot training failure rate at start of WWII
First set of Aircrew Selection Board tests included• Essay writing• Elementary maths• General intelligence
• Early developments• need recognised for separate tests of skills and personality• shift to testing for specific roles (e.g. electromechanical coordination)• 1944: With help from USAAF, series of objectives measures (e.g. 24 aptitude tests for six aircrew categories) – waste down from 48% to 25%• use of specially trained staff
Example: RAF Aircrew Selection
1944–84Many more tests created – but at the end of this period tests were not markedly different
Preliminary Flying School closed 1974 – selection then relied purely on aptitude tests. For various reasons validities dropped; for example, to .14 for training results
Second generation selection tests: • exploited increased computing capability:
• at first, computerised versions of existing tests• later, new tests (based on abilities required, using Fleishman’s system) – Air Traffic and Fighter Controller Test Battery produced
• Nine weighted test scores used• Good predictive validity
Issues• No formal job analysis• Tests driven by theory and test availability
Hence 1990s:• Shift to domain-centred framework
Example: RAF Aircrew Selection
:
Simulation and Training
Information from
• skill lecture
• simulation and training lecture (technical skills)
• CRM lecture (non-technical skills)
• Human error lecture
• etc
Key Issues
• Training needs analysis (organisational, occupational, individual) — focus on Knowledge, Skills, Abilities/ Attitudes (KSAs) Design of training programme
• Develop Instruction by Objective• Select Instructional Strategy• Select/ organise element to be trained• Identify training aids• Organise materials/resources• Apply learning principles• Develop Evaluation Instruments
• Implementation (who, where, when)
• Evaluation (Reaction, Learning, Behaviour, Results) • Note DIF analysis (difficulty, importance, frequency) — helps to decide whether not to train, to train, or to over-train
Training issues (see also Human Information Processing lecture):
• Massed versus distributed practice
• Whole- versus part-task approach
• Phases of learning
• Feedback Media and technology
• Simulation — very important in aviation, where the objective is to maximise the transfer of learning from simulator to aircraft; fidelity is a key issue: do not need physical fidelity (simulator does not need to resemble the aircraft), but functional fidelity is important
• Internet-based — increasingly important; can be accessed even in the field
• Traditional — still some role for classroom-based instruction
Situation Awareness
Topics
Definitions Models Theory Metrics Applications Limitations
Leading causal factor in a review of 175 aviation mishaps
Hartel, Smith and Prince
(1991)
Major causal factor in 88% of accidents associated with human error in a review of major aircraft carrier accidents (1989-1992)
Endsley(1994)
Controlled Flight Into terrain (CFIT) accidents killed 5000 people between 1978 and 1992. 74% of these accidents were due to loss of flight crew SA
Woodhouseand
Woodhouse(1995)
Why is Studying Situation Awareness Important?
SA popularised to describe the psychological processes of:– Attention– Memory– Perception– Prediction– Pattern Matching
Historical Origins of SA
‘‘Mental Model’Mental Model’of the situationof the situation
Definitions of SA 1
“ . . . Knowledge of current and near-term disposition of both friendly and enemy forces within a volume of airspace.” McMillan McMillan (1994)(1994)
“. . . One’s ability to remain aware of everything that is happening at the same time and to integrate that sense of awareness into what one is doing at that moment.” Haines & Flateau (1992)Haines & Flateau (1992)
“. . . A pilot’s continuous perception of self and aircraft in relation to the dynamic environment of flight, threats, and mission, and the ability to forecast, then execute tasks based on that perception.” Hamilton (1987)Hamilton (1987)
Situation Awareness is . . .
A Working Definition of SA
Situation Awareness is . . .
– “The perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” Endsley (1988)Endsley (1988)
– It is derived from the aircraft instrumentation, the out-the-window view, and his or her senses
– The quality of an operator’s SA is moderated by individual capabilities, training, experience, objectives, and the ability to respond to task workload
– The term ‘SA’ should only ever be applied to dynamic environments
Summary know this
When all is said and done we know that Situation Awareness refers to an operator’s knowledge and Understanding of the dynamic environment in which he/she is operating
It is knowledge of the ‘Big Picture’ SA provides the basis for subsequent decision
making and performance in the operation of complex, dynamic systems
PERCEPTION OF PERCEPTION OF ENVIRONMENTENVIRONMENT
COMPREHENSIONCOMPREHENSIONOF CURRENT SITUATIONOF CURRENT SITUATION
PROJECTION OFPROJECTION OFFUTURE STATUSFUTURE STATUS
A Model of SA
Level 3Level 3
Level 2Level 2
Level 1Level 1
Endsley (1995)Endsley (1995)
PERCEPTION OF PERCEPTION OF ENVIRONMENTENVIRONMENT
‘‘Perceive Perceive relevantrelevant
information’information’
COMPREHENSIONCOMPREHENSIONOF CURRENT SITUATIONOF CURRENT SITUATION
‘‘Integrate withIntegrate withtask goals’task goals’
PROJECTION OFPROJECTION OFFUTURE STATUSFUTURE STATUS
‘‘Predict futurePredict futureevents / statesevents / states
based on based on understanding’understanding’
SITUATION AWARENESS
Abilities Experience Training
System Capability Interface Design Stress & Workload Complexity Automation
Information ProcessingMechanisms
Long–term MemoryStores
Automaticity
Goals & Objectives Preconceptions
(Expectations)
Feedback
State of theEnvironment
Decision Performance ofActions
Task/System Factors
Individual Factors
Comprehensionof CurrentSituation
Level 2
Perceptionof Elementsin CurrentSituationLevel 1
Projectionof FutureStatus
Level 3
Endsley’s Modelbe aware of main elements
Model of SA in dynamic decision making (from Endsley, 2000)
SA
Environmental State
E.g. TemperatureTime Pressures
Day/nightNoise
Lethality
Operator Traits
E.g. IQCognitive AbilitiesConscientiousness
ExperiencePersonality
TrainingRisk Taking
Real World/SystemInformation
Salience of infoAvailability of info
Info complexityInfo quantityAutomation
Quality of HMI
Operator State
E.g. FrightenedConfused
High WorkloadFatigued
Aggressive
Goals
Mission GoalsSystem Goals
Personal Goals
QinetiQ’s Model of The SA Process
The SA ‘PROCESS’
The SA Process A series of complex cognitive
processes, including: Perception, Working Memory, Pattern Matching, Attention and Long Term Memory
NOT ‘task’ or ‘individual’ specific Also referred to as Situation
Assessment (SAS) Will be influenced by a multitude of
‘SA Factors’
These factors WILL BE ‘task’ AND ‘individual’ specific
Each factor will have different weightings or importance attached to it for differing military domains
The number of such factors is vast
Operator Traits
E.g. IQCognitive AbilitiesConscientiousness
ExperiencePersonality
TrainingRisk Taking
Environmental State
E.g. TemperatureTime Pressures
Day/nightNoise
Lethality
Goals/Doctrine/SOPs
Mission GoalsSystem Goals
Personal GoalsDirectives
ROECommander Intent
Knowledge/Information
Salience of infoAvailability of info
Info complexityInfo quantityAutomation
Quality of HMI
Factors Affecting the SA Process
Know the main headings!
The output of the SA PROCESS will be a number of ‘Situation Models’ (or dynamic mental models)
These situation models are essentially knowledge and understanding
The quality of a person’s SA is defined by the match between these situation models and reality Situation Model
Real World
The difference betweenthese represents the quality of one’s SA
SA as a Product
The person will have a situation model for each of the relevant ‘SA Information Domains’ associated with a specific task or job
Each SA information domain will comprise a number of ‘SA Elements’
Example: Endsley (2001) illustrates this for the task/job of piloting a civil aviation aircraft
Geographical SA•own aircraft•other aircraft•terrain features •airports•cities•waypoints•navigation fixes•position relative to
designated features•path to desired location•runway and taxiway assignments•climb/descent points
System SA•system status•functioning and settings
•radio•altimeter•transponders•flight modes and automation
•deviations from correct settings•ATC communications present•fuel•impact of degrades and settings
•on performance•time and distance available on fuel
Spatial/Temporal SA•attitude•altitude•heading•velocity•vertical velocity•Gs•flight path•actual values relative to assigned•projected flight path•projected landing time
Environmental SA•weather formations and movement•temperature•icing•ceilings•fog•Turbulence, winds•sun•visibility•IFR/VFR conditions•areas to avoid•flight safety•projected weather conditions
SA Elements
In Summary The development and
maintenance of SA occurs within an individual’s head
The SA process (or SAS) is a generic continuous process/cycle that is impacted upon by many factors
These factors will vary in their importance and influence depending upon the specific task and the individual undertaking that task
An individual will continuously cycle through the SA process for each SA Information Domain, developing a situation model for each
These situation models will be task-specific
All situation models will be continually updated and revised as new information becomes available or as the factors affecting the SA process change in importance or in state
Team SA SA can be applied to teams as
well as to individuals Caution needed here, as SA
cannot be shared (it resides inside the individual’s head), but information can be shared
We could be talking about:– 1) The overlap in SA for the team– 2) The SA of the team as moderated by
the primary decision maker– 3) The collective SA of the entire team
Measuring Situation Awareness
SA has become a major design driver– Developing operator interfaces to enhance SA– Developing automated systems without resulting in a loss of
SA– Training techniques are designed to develop better SA
Development of SA metrics for evaluation purposes– Development of metrics since the late 1980s– Varying degrees of maturity / validation – Various forms of metrics
• Subjective Vs Objective• Self-report Vs Third-Party rating• Simulator-based Vs Test flight
SA Metrics Crew SA SA Global Assessment technique (SAGAT) Snapshots SA Flight Training Evaluator (SAFTE) China Lake SA Scale (CLSA) SA Rating Technique (SART) SA Supervisory Rating Form (SASRF) Physiological Measures: Eye Activity
SA Metrics – Summary
SALevel
CrewSA
SAGAT SnapShots
SAFTE CLSA SART SASRF EyePOG
PerceiveEnvironment
ComprehendCurrentSituation
ProjectFutureStatus
Know at least SAGAT/SART in a little detail, plus names of a few others
SA Metrics – Summary
Most SA measures have been designed using a particular SA definition, and with a specific application in mind– Keep this in mind when selecting an SA measure
In practice, 2 of the SA measures outlined previously are used far more than the others:– SART (subjective)– SAGAT (objective)
This is probably due to the extensive validity data that accompanies these measures (we ‘know’ they are measuring SA)
There are three main military applications for SA research:– 1) System/interface design, development, assessment and
evaluation• Operator interfaces designed to enhance SA• Automated systems must switch without losing operator
SA– 2) Training operators to have better SA– 3) Selecting operators who are predisposed to having high SA
So, of What Use is SA Research?
Limitations of SA
SA is a theoretical construct– Practical difficulties in measuring and predicting SA– For those who do not understand the theoretical basics of what SA is all about, there can
be an element of perceived circularity
‘‘Why did the aircraft Why did the aircraft crash?’crash?’
Because of lack Because of lack of pilot SAof pilot SA
‘‘How do we know there How do we know there was a lack of pilot SA?’was a lack of pilot SA?’
Because heBecause hecrashedcrashed
Immature concept– Still much debate over definitions and measures
Crew Resource Management (CRM)
Introduction Need for effective interaction Aviation accidents: most have human error component
CRM Evolution Evolution of CRM to fifth-generation
CRM Training
Objectives of CRM
Knowledge, skills, attitudes to promote safe, efficient operations:
– Effective decision making– Good crew communication– Understanding/acceptance of role and responsibilities
CRM focuses broadly on training transportable teamwork skills
CRM Training
Types of CRM course
Foundation Course– Wide range of topics covered– Focus on discussion and video
Continuation Courses– In depth coverage of topic areas– Skills practice (low fidelity)
LOFT/MOST– Skills practice (high fidelity)– Crew-centred debrief
CRM Training
Topics in typical CRM courses– Human information processing– Personality and attitudes– Communications– Teamwork structures– Teamwork behaviours– Leadership style– Decision making– Stress management– Human error– Situation awareness– Automation on the flight deck– Fatigue and workload– Case studies & research findings– Be able to list the main topics
CRM Training
Leadership issues
Effects of captain’s attitudes
Authority Gradient
CRM Training
Communication– US ASRS: most accidents involved failure of information transfer– Low-error crews demonstrate different patterns of comms
Communication skills know them!– Inquiry– Advocacy– Listening– Conflict resolution– Critique– Feedback
Barriers to communication– Physical; word usage; interpersonal; mental– Cultural and language barriers– Subordination problems– ‘Power-distance’ barriers
CRM Training
Core teamwork behaviours
– Monitoring
– Feedback
– Backing up
CRM Training
Ad hoc teams– Frequently arise in airline ops– 73% of accidents occur on first day crew flying together
Situation awareness– Important topic in CRM– See lecture on SA!
CRM Training
CRM training resources– Self-study– Classroom awareness training– Modelling– Classroom skills training– Skills practice in simulators– Practice/coaching during flying
CRM Training
LOFT– Run in a high fidelity simulator– Realistic sortie/real time– Crew and facilitator ‘in role’– Few failures – Non-technical focus – Focus on choice dilemmas– Non jeopardy– Crew-centred debrief using video
CRM Training
CRM Issues– What is ‘best practice’?– Does it work? – Those needing most help from CRM most resistant to
change– May change attitudes but not behaviour– Needs management commitment– CRM skill fade occurs over time– Cultural issues should be considered
CRM Training
Error and Accidents(See Accident module)
The following may help you structure yourKnowledge of this topic – drawn from workof John Chappelow
Perception
Intention
Action
Task
Disruptive factors
Enabling factors
Predispositions
ENVIRONMENT
SYSTEM
OPERATOR
personalitytalent
fatiguealcoholoverarousalunderarousal
ergonomics
trainingbriefingsocial context
noiseheatcoldvibration
threattask demand
Be able to list/describe themain factors
Error type Disruptivefactors
Enablingfactors
Predisposingfactors
Perceptionerrors
Stressors Physical Trait
Visual illusion Physiological Ergonomics PersonalityDisorientation Noise Handling
characteristicsLack of talent
Undetected threat Time pressure System logic InexperienceMisinterpretation Operational
pressureExcess zeal
High task demand Lack of airmanship
Intention errors Low task demand Sensory limitationsRule violation Threat Cognitive
limitationsInappropriatemodel
Distraction Motor limitations
Action errors Organisational StateCognitive failure Training AlcoholSlow response Briefing FatiguePrecipitate response Administrative
supportHypoglycaemia
Disorganisedresponse
Social context Life stress
Mishandling Low moraleUnderarousalOverarousalSocial factors
Error type Disruptivefactors
Enablingfactors
Predisposingfactors
Perceptionerrors
Stressors Physical Trait
Visual illusion Physiological Ergonomics PersonalityDisorientation Noise Handling
characteristicsLack of talent
Undetected threat Time pressure System logic InexperienceMisinterpretation Operational
pressureExcess zeal
High task demand Lack of airmanship
Intention errors Low task demand Sensory limitationsRule violation Threat Cognitive
limitationsInappropriatemodel
Distraction Motor limitations
Action errors Organisational StateCognitive failure Training AlcoholSlow response Briefing FatiguePrecipitate response Administrative
supportHypoglycaemia
Disorganisedresponse
Social context Life stress
Mishandling Low moraleUnderarousalOverarousalSocial factors
Social context9%Social factors
11%
Personality21%
Cognitive failure17%
Disorganisedresponse 26%
Ergonomics
TrainingBriefingAdmin.support
~ 40%
Inexperience
23%
Summary: major factors
IntrovertedExtraverted
Stable
NeuroticAnxious Impulsive
You’ve seen this before...
Expanded data set
Major causal factors: Human factors
0 5 10 15 20 25 30 35
E rgonom ic s
Lac k of airm ans hip
P ers onality
S ens ory lim itations
S oc ial fac tors
Dis trac t ion
Overarous al
A dm inis trative s upport
High tas k dem and
Inex perienc e
Num be r of m isha ps
Expanded data set
Sensitivity: Human factors
-15 -10 -5 0 5 10 15
Social context
Sensory limitations
Supervision
Lack of airmanship
Briefing
Administrative support
Inexperience
High task demand
Distraction
Social factors
This graph showsbenefit of eliminatingthe factor, and costof an increase in itsseverity
Social factors areseen to be moreimportant when weconduct sensitivityanalysis: these problemsare soluble
Error: Examples of Possible Remedies know this
Administrative support• Organisational interventions
Inexperience• More (or improved) training
High task demand• Selection (e.g., some personality types cope better with high workload)• Training creates more ‘spare capacity’• The system can be modified to reduce workload (automation, better ergonomics etc)
Distraction• May be able to select individuals less prone to distraction
Social factors• Personnel selection• CRM training