presented to: by: date: federal aviation administration operational suitability evaluations...
TRANSCRIPT
![Page 1: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/1.jpg)
Presented to:
By:
Date:
Federal AviationAdministrationOPERATIONAL
SUITABILITY EVALUATIONS
Weather-in-the-Cockpit Workshop
Robert Ruiz, AFS-430
August 9, 2006
![Page 2: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/2.jpg)
2 2Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Introduction• Background• Activities• Objectives• Study• Approach• Evaluation
![Page 3: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/3.jpg)
3 3Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Background• WX Products emerge from research as technically sound
products – Good Science• Operational implementation (AFS definition) historically
not part of the R&D effort• Lack of operational suitability evaluation on newly
developed graphical products• Some products issued as supplementary with restrictions• CIP/FIP 2005 Hazard analysis highlighted safety issues
and the need for a structured Safety Assessment of WX products
![Page 4: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/4.jpg)
4 4Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
FY-06 Implementation Activities
• AFS-400 able to secure funding to develop and implement an operational suitability evaluation
• FAA Tech Center Weather Sensors Group was selected to conduct the evaluation
![Page 5: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/5.jpg)
5 5Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Definitions – HBAT 05-01 and AIM
• Primary Weather Product.– An aviation weather product that meets all the regulatory
requirements and safety needs for use in making flight-related, aviation weather decisions.
• Supplementary Weather Product. – An aviation weather product that may be used for enhanced
situational awareness. If used, a supplementary weather product must only be used in conjunction with one or more primary weather products. In addition, the FAA may further restrict the use of supplementary weather products through limitations described in the product label to conduct the evaluation
![Page 6: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/6.jpg)
6 6Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Objectives – Operational Suitability Evaluation• Determine pilot understanding/intuitiveness of the product• Determine that the information presented is not hazardous
misleading information• Determine the level of safety of the product; provide an
equivalent level of safety (Supplementary-Situational Awareness)
• Determine the level of training required for product use• Determine if any training and guidance materials will be
needed• Provide a roadmap for AVS to move the product from
Supplementary to Primary
![Page 7: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/7.jpg)
7 7Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Study Outline• Study 1: Supplementary Weather Product
– Assess CIP operational suitability for pilot use as Supplementary Product with no restrictions
– Assess operational suitability using: • Situational Awareness• CIP Supplementary Weather Product Questionnaire • Interview questions• Observer ratings
• Study 2: Primary Weather Product– Identify CIP operational suitability issues for pilot use as Primary
Product– Identify issues using:
• Situational Awareness• CIP Primary Weather Product Questionnaire • Interview questions• Observer ratings
![Page 8: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/8.jpg)
8 8Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Overview
•Evaluate CIP Severity and Probability products•Evaluate from pilot perspective•Use in flight planning scenarios
– Preflight– In-flight*
*Note: While “In-Flight” use is being examined, the evaluation will not yield data appropriate for cockpit use.
![Page 9: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/9.jpg)
9 9Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Approach
• Simulate route planning with focus on in-flight icing– Preflight – In-flight
• Active pilot participants (n=32/48)– General Aviation (Expert and Novice)– Business Jet– Regional Airline
![Page 10: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/10.jpg)
10 10Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Approach
• Flight Planning Scenarios– Archived weather examined to determine flight routes– Flight routes developed so that icing is encountered– Eight GA Routes– Four FAR Part 121 (SkyWest) Routes– Flight route information includes:
• Departure, Destination, Waypoints
• ETD
• ETE
• Preferred Flight Level
• Distance
• Equipage
![Page 11: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/11.jpg)
11 11Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Conduct• Experimental Conditions
– Study 1: CIP Supplementary Weather Product • AIRMET/SIGMETs (No CIP)
• AIRMET/SIGMETs (With CIP)
– Study 2: CIP Primary Weather Product• CIP (No AIRMET/SIGMETs)
• CIP (With AIRMET/SIGMETs)
![Page 12: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/12.jpg)
12 12Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Conduct• Experimental Conditions
– Pilots• Expert
• Novice
– Equipage • De/anti-icing equipment
• No de/anti-icing equipment
– Icing Severity• Trace/Light
• Moderate or Greater (MOGR)
![Page 13: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/13.jpg)
13 13Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Conduct
• Pretest Briefing– Objectives of evaluation– Training
• CIP information and use
• Evaluation platform
• Evaluation data collection tools
![Page 14: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/14.jpg)
14 14Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Conduct• Preflight
– Assigned preferred route• Trace/Light Icing
• Moderate or Greater (MOGR)
– Standard weather briefing – Depending on experimental condition, provide icing products
for Supplementary Study or Primary Study – Pilot decides route/altitude adjustments/no adjustments– Interview questions– Assess Situational Awareness
• Pilot
• CFI
![Page 15: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/15.jpg)
15 15Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Conduct• In-flight
– Assigned preferred route• Trace/Light Icing • Moderate or Greater (MOGR)
– Perform preflight self brief– Pilot is asked to imagine he/she is about 90 minutes into flight– In-flight weather update provided– Depending on experimental condition, provide icing products for
Supplementary Study or Primary Study – Pilot decides route/altitude adjustment/no adjustment – Interview questions– Assess Situation Awareness
• Pilot• CFI
![Page 16: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/16.jpg)
16 16Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Data Elements
• Situation Awareness (SA) rating– Pilot– Observer (CFI)
• Operational suitability questionnaire responses• Interview comments• Potential pilot errors/poor decisions• General observations
![Page 17: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/17.jpg)
17 17Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Criteria: Supplementary
• Statistically significant increase in SA when CIP is used with AIRMETs/SIGMETs
Evaluation Criteria: Primary
• No decline in SA when CIP is used without AIRMET/SIGMETs
![Page 18: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/18.jpg)
18 18Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Results (Expected)
• Supplementary Weather Product Evaluation– Operational suitability confirmation data
• Data to support or not support CIP use as supplementary product without restrictions
• Primary Weather Product Evaluation– Identify operational suitability issues
• Issues to be resolved in transitioning CIP to Primary Product• Interview responses• Pilot Error• Situation Awareness• Operational Suitability Comments
![Page 19: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/19.jpg)
19 19Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Evaluation Timeline
• Data collection (7/21 – 9/05)
• Evaluation Reports– Quick Look (9/25/2006); NBAA/FPAW– Final Report (11/03/2006)
• CIP-Severity and Probability Operational Dec 06.
![Page 20: Presented to: By: Date: Federal Aviation Administration OPERATIONAL SUITABILITY EVALUATIONS Weather-in-the-Cockpit Workshop Robert Ruiz, AFS-430 August](https://reader030.vdocument.in/reader030/viewer/2022032612/56649efa5503460f94c0bd53/html5/thumbnails/20.jpg)
20 20Federal AviationAdministration
Operational Suitability EvaluationAugust 9, 2006
Acknowledgement
• FAA Tech Center Weather Sensors Group in providing assistance in developing some of the slides that were used in the presentation
Contact InformationRobert M. Ruiz
AFS-430
202-385-4578