line oriented safety audit losa
DESCRIPTION
Auditorias en lineaTRANSCRIPT
Line OperationsSafety Audit (LOSA)
Robert L. Helmreich, PhD, Robert L. Helmreich, PhD, FRAeSFRAeS
James Klinect, PhD Cand.James Klinect, PhD Cand.
North American Aviation Safety North American Aviation Safety ConferenceConference
Atlanta, February 2003Atlanta, February 2003
Human Factors Research ProjectHuman Factors Research ProjectThe University of Texas at AustinThe University of Texas at Austin
Data to Identify Safety IssuesData to Identify Safety Issues Accident investigationAccident investigation
Limited, non-representative sampleLimited, non-representative sample
Incident reportsIncident reports (ASRS and ASAP) (ASRS and ASAP) Data slanted to events resulting from system and flight crew failuresData slanted to events resulting from system and flight crew failures UT HF developing data category system with AA & COUT HF developing data category system with AA & CO
Formal checkrides (Line and Proficiency)Formal checkrides (Line and Proficiency) Data show crew capability and procedural knowledgeData show crew capability and procedural knowledge
Flight Data Recorders – FOQAFlight Data Recorders – FOQA Data show “what happened” in terms of flight parametersData show “what happened” in terms of flight parameters
Non-jeopardy observation of normal flightsNon-jeopardy observation of normal flights Give data on why things happen and how they are managedGive data on why things happen and how they are managed Provides realistic baseline of safety dataProvides realistic baseline of safety data Is proactiveIs proactive
LOSALOSA Jump seat observations of flight crew Jump seat observations of flight crew
performance during regular scheduled flightsperformance during regular scheduled flights Observers unobtrusive – collecting Observers unobtrusive – collecting
data not participating in flightdata not participating in flight Team of observers from different Team of observers from different
backgrounds backgrounds Line pilots / Union representativesLine pilots / Union representatives Check airmenCheck airmen Safety and Training pilotsSafety and Training pilots UT observersUT observers
All data are DE-IDENTIFIED and All data are DE-IDENTIFIED and CONFIDENTIALCONFIDENTIAL
Scientific BackgroundScientific Background
LOSA uses systematic observation of LOSA uses systematic observation of behaviorbehavior
Systematic observation is a validated Systematic observation is a validated methodology using observers trained and methodology using observers trained and calibrated to high reliabilitycalibrated to high reliability
Systematic observation has been employed Systematic observation has been employed in scientific studies of crew performance in in scientific studies of crew performance in demanding environmentsdemanding environments For example, Aquanauts in undersea habitats For example, Aquanauts in undersea habitats
during Project SEALAB and Project Tektiteduring Project SEALAB and Project Tektite
Purpose of LOSAPurpose of LOSA Collect valid empirical data on:Collect valid empirical data on:
1.1. Crew performanceCrew performance – strengths and – strengths and weaknessesweaknesses ProficiencyProficiency Decision-MakingDecision-Making CRM skillsCRM skills Procedural complianceProcedural compliance Threat and error managementThreat and error management
2.2. System performanceSystem performance – strengths and – strengths and weaknessesweaknesses CultureCulture Airspace System – airports and navigational AidsAirspace System – airports and navigational Aids Aircraft design / automationAircraft design / automation Standards / Training / Safety / MaintenanceStandards / Training / Safety / Maintenance Crew support – ATC, Cabin, Ground, and DispatchCrew support – ATC, Cabin, Ground, and Dispatch
History of LOSAHistory of LOSA
Formal LOSA developed at request of Delta Formal LOSA developed at request of Delta Airlines to validate operational impact of Airlines to validate operational impact of Human Factors (CRM) trainingHuman Factors (CRM) training
The focus of initial LOSA was systematic The focus of initial LOSA was systematic assessment of CRM-related crew behaviorsassessment of CRM-related crew behaviors
In 1997, collaborating with Continental In 1997, collaborating with Continental Airlines, LOSA was expanded to record Airlines, LOSA was expanded to record threats and errors and their management threats and errors and their management during flight operationsduring flight operations
• Delta domesticDelta domestic• Delta internationalDelta international• TWATWA• AmericanAmerican• Continental Continental • USAirwaysUSAirways
CRM LOSA 1992 - 1996
Threat and Error Management Threat and Error Management LOSA (1997-2002)LOSA (1997-2002)
• Continental Latin Continental Latin AmericaAmerica
• Continental Continental ExpressExpress
• Gulfstream Gulfstream ExpressExpress
• Air New ZealandAir New Zealand• Air MicronesiaAir Micronesia• ContinentalContinental• DeltaDelta
• USAirwaysUSAirways• Cathay PacificCathay Pacific• EVA AirEVA Air• Uni Air (Taiwan)Uni Air (Taiwan)• FrontierFrontier• QANTASQANTAS• BraathensBraathens• Singapore Singapore
(forthcoming)(forthcoming)
1900+ flight segments. All foreign carriers are U.S. codeshares
LOSA ComponentsLOSA Components Part 1. Flight crew surveyPart 1. Flight crew survey Part 2. Flight crew interviewPart 2. Flight crew interview Part 3. Descent and approachPart 3. Descent and approach Part 4. CRM CountermeasuresPart 4. CRM Countermeasures
Behavioral markersBehavioral markers Part 5. Threat and error mgtPart 5. Threat and error mgt
LOSA Data: Flight CrewLOSA Data: Flight Crew
General Flight Crew General Flight Crew InformationInformation
DemographicsDemographics Attitudes / perceptions / Attitudes / perceptions /
safety culture (FMAQ)safety culture (FMAQ) Safety interview Safety interview
commentscomments
Flight DescriptionFlight Description Observer narrativeObserver narrative Overt threatsOvert threats Operational complexityOperational complexity
Flight Crew PerformanceFlight Crew Performance Behavioral markersBehavioral markers Crew errors and violationsCrew errors and violations Undesired aircraft statesUndesired aircraft states Technical data for Technical data for
approachesapproaches Type and stabilityType and stability
Threat managementThreat management Error managementError management Undesired state Undesired state
managementmanagement
For each flight segment, observers collect data on:
ThreatThreat
Threat ManagementThreat Management
Adverse weatherAdverse weather TerrainTerrain TrafficTraffic Airport conditionsAirport conditions A/C malfunctionsA/C malfunctions Automation eventsAutomation events Communication eventsCommunication events
Operational time pressuresOperational time pressures Non-normal operationsNon-normal operations ATC command events / errorsATC command events / errors Cabin events / errorsCabin events / errors MX events / errorsMX events / errors Dispatch events / errorsDispatch events / errors Ground crew events / errorsGround crew events / errors
• Threats - Originate outside the flight crew’s influence but require active management to prevent them from becoming consequential to safety
Representative Threat FindingsRepresentative Threat Findings
90% of flight segments had one or 90% of flight segments had one or more threatsmore threats
Organizational range – 84% to 97%Organizational range – 84% to 97% Most frequent threatsMost frequent threats
Adverse weather – 19%Adverse weather – 19% ATC clearances/late changes 16%ATC clearances/late changes 16%
ErrorError
Error AvoidanceError Avoidance
Complete error avoidance is Complete error avoidance is impossible – errors are inevitableimpossible – errors are inevitable
63% of flights observed had one or 63% of flights observed had one or more errorsmore errors Must look for sources of error to Must look for sources of error to
strengthen system defensesstrengthen system defenses
Human Limitations Lead to Human Limitations Lead to ErrorError
Limited memory capacityLimited memory capacity Limited processing capacityLimited processing capacity
multi-tasking capabilitymulti-tasking capability Limits imposed by stressors Limits imposed by stressors
tunnel visiontunnel vision Limits imposed by fatigue and other Limits imposed by fatigue and other
physiological factorsphysiological factors Poor group dynamicsPoor group dynamics Cultural influencesCultural influences
Flight Crew Error TypesFlight Crew Error TypesIntentional NoncomplianceIntentional Noncompliance – violations – violations
ex) Performing a checklist from memoryex) Performing a checklist from memory
ProceduralProcedural – Followed procedures but wrong – Followed procedures but wrong executionexecution
ex) Wrong altitude setting dialed into the MCP ex) Wrong altitude setting dialed into the MCP
CommunicationCommunication – Missing information or – Missing information or misinterpretation within cockpitmisinterpretation within cockpit
ex) Miscommunication by crew with ATCex) Miscommunication by crew with ATC
DecisionDecision – Discretionary decision that – Discretionary decision that unnecessarily increases riskunnecessarily increases risk
ex) Unnecessary navigation through adverse weatherex) Unnecessary navigation through adverse weather
Decision ErrorDecision Error
Choice increasing risk in a situation withChoice increasing risk in a situation with multiple courses of action possiblemultiple courses of action possible time available to evaluate alternatives time available to evaluate alternatives no discussion of consequences of no discussion of consequences of
alternate courses of actionalternate courses of action no formal procedures to followno formal procedures to follow
Error ResponseError Response
45% Undetected45% Undetected 29% Detected and action taken29% Detected and action taken 5% Detected and ignored5% Detected and ignored
Error OutcomesError Outcomes
Inconsequential – 71%Inconsequential – 71% Consequential – 29%Consequential – 29%
Additional Errors – 8% (Error Chains)Additional Errors – 8% (Error Chains) Undesired Aircraft States – 21%Undesired Aircraft States – 21%
Undesired Aircraft StateUndesired Aircraft State
• Lateral deviation
• Vertical deviation
• Speed too high
• Speed too low
• Incorrect aircraft configuration– Flight controls– Systems– Fuel– Automation
• Unstable approach
• Abrupt aircraft control
• Long landing – no go
around
• Firm landing
• Forced landing
• Runway incursion
• Wrong taxiway, ramp,
runway, country
A compromised situation placing the flight at increased risk
Phase of Flight EffectsPhase of Flight Effects
Phase of FlightPhase of Flight Threats by Threats by PhasePhase
Errors by Errors by PhasePhase
Pre-Departure / TaxiPre-Departure / Taxi 42%42% 2626%%
Takeoff / ClimbTakeoff / Climb 14%14% 23%23%CruiseCruise 8%8% 4%4%
Descent / Approach / Descent / Approach / LandLand 32%32% 42%42%
Taxi / ParkTaxi / Park 4%4% 5%5%
Descent / approach / land phase contains the most Descent / approach / land phase contains the most variability in crew performance and consequential errorsvariability in crew performance and consequential errors
Organizational DifferencesOrganizational DifferencesAirline AAirline A Airline BAirline B Airline CAirline C
Mean Mean ThreatsThreats
2.252.25 4.644.64 3.273.27
Mean ErrorsMean Errors 0.930.93 1.971.97 4.324.32
MeanMean
Und. StateUnd. State0.190.19 0.420.42 0.950.95
Threat and ErrorThreat and Error
Relationship between number of Relationship between number of threats and errors is relatively threats and errors is relatively weak, although statistically weak, although statistically significant (r=.19)significant (r=.19)
A model of Threat and Error A model of Threat and Error Management derived from Management derived from
LOSA dataLOSA data
The model is being used as The model is being used as framework for the analysis of framework for the analysis of ASAP reports, incidents, and ASAP reports, incidents, and
accidentsaccidents
Threat and Error Management Threat and Error Management ModelModel
Th re ats:Late n t an d Ov e rt
Th re at M an ag e me n t
In co n se q ue n tial In cide n t / Accide n t
Erro rs
Erro r M an age me n t
Un de sire d AircraftState
Un de sire d AircraftState M an age me n t
LOSA and CRMLOSA and CRM
Interpreting LOSA Data for Interpreting LOSA Data for CRMCRM
Procedural errorsProcedural errors - may indicate poor - may indicate poor workload management and/or poor workload management and/or poor proceduresprocedures
Communications errorsCommunications errors - may reflect - may reflect inadequate CRM (monitoring and challenging) inadequate CRM (monitoring and challenging) or complacencyor complacency
Proficiency errorsProficiency errors - suggest pressures to train - suggest pressures to train and/or need for higher standards and better and/or need for higher standards and better monitoring by check airmenmonitoring by check airmen
Decision errorsDecision errors - may indicate need for more - may indicate need for more CRM training on expert decision making and CRM training on expert decision making and risk assessmentrisk assessment
ViolationsViolations - suggest poor procedures, weak - suggest poor procedures, weak captain leadership and/or a culture of non-captain leadership and/or a culture of non-compliancecompliance
““[LOSA] acquires direct, first-hand data on the successful [LOSA] acquires direct, first-hand data on the successful recovery from errors by flight crews during normal line recovery from errors by flight crews during normal line flights. [It] is aimed at collecting data on successful flights. [It] is aimed at collecting data on successful human performance; and this is indeed a first in our human performance; and this is indeed a first in our industry, since aviation has traditionally collected data industry, since aviation has traditionally collected data on failed human performance, such as an accident or on failed human performance, such as an accident or incident investigation.” incident investigation.”
- - Costa Pereira, ICAO Secretary General in May 8, 2000 correspondence Costa Pereira, ICAO Secretary General in May 8, 2000 correspondence to Jane F. Garvey, FAA Administrator to Jane F. Garvey, FAA Administrator
““It is recommended that, IATA and Operators together It is recommended that, IATA and Operators together further promote the use of Flight Operations Quality further promote the use of Flight Operations Quality Assurance, the Line Operations Safety Audit (LOSA)Assurance, the Line Operations Safety Audit (LOSA)……”……”
--IATA Safety Report (Jet) 2000IATA Safety Report (Jet) 2000 Executive Summary Executive Summary, p. vi, p. vi
Endorsed as safety priority by ALPA and IFALPAEndorsed as safety priority by ALPA and IFALPA
Support for LOSASupport for LOSA
Recognition and Extension of Recognition and Extension of LOSALOSA
Flight Safety Foundation/Aviation Week and Flight Safety Foundation/Aviation Week and Space Technology Laurels, 2001Space Technology Laurels, 2001
The UT Threat and Error Management Model The UT Threat and Error Management Model adopted as framework for analysis of adopted as framework for analysis of accidents by IATAaccidents by IATA
Dispatch LOSA (with Continental) underwayDispatch LOSA (with Continental) underway
The University of Texas The University of Texas Human Factors Research ProjectHuman Factors Research Project
www.psy.utexas.edu/HumanFactors