sas_08_automation_for_system_safety_analysis_malin 1 automation for system safety analysis jane t....
TRANSCRIPT
SAS_08_Automation_for_System_Safety_Analysis_Malin1
Automation for System Safety Analysis
Jane T. Malin, Principal Investigator
Project: Automated Tool and Method for System Safety Analysis
Software Assurance SymposiumSeptember, 2008
Complex systems typically fail because of the unintended consequences of their design, the things they do that were not intended to be done. - M. Griffin, System Engineering and the “Two Cultures” of Engineering, March 28, 2007
SAS_08_Automation_for_System_Safety_Analysis_Malin2
Overview
• Problem/NASA Relevance• Technical Approach and Overview• 2008 Target Capability • Crew Exploration Vehicle (CEV) Launch Abort System Case• Data – Constellation (Cx) failure modes and effects
analysis/critical items lists (FMEA/CILs)• Technical Challenges
– Information Extraction – Semi-Automated Model Construction– Analysis and Test Case Generation
• 2009 Planned Capability• Potential Applications
SAS_08_Automation_for_System_Safety_Analysis_Malin3
Problem and NASA Relevance• NASA needs early evaluation of software (SW)
requirements and design, to reduce software-system integration risks– Assess system failures and anomalous conditions that may
challenge software in system integration testing– Identify robustness issues early (and often)– Identify requirements gaps early (and often)
• Project test case: NASA Constellation (Cx) Launch Abort System (LAS) for Pad Abort
PLANT and Environment SOFTWARE‘Activate’ Faults and
Influence Failures
‘Activate’ Faults and Influence Failures
Operations and Stresses
FAULTS/ReliabilityFAULTS/Reliability
SAS_08_Automation_for_System_Safety_Analysis_Malin4
Technical Approach
Systematic semi-automated extraction and analysis for early evaluation and rapid update– Capture model of the controlled system architecture
• Abstract physical architecture models with subsystems, functions, interfaces, connections
• Extract directly from requirements and design text and data– Capture risks and hazards in model
• Constraints, hazards, risks from requirements and design • Risk and failure libraries
– Analyze and simulate to identify risks and constraints• Analyze and simulate hazard/risk propagation in the system• Use operational and off-nominal scenarios and
configurations– Identify possible test scenarios for virtual system
integration testing
SAS_08_Automation_for_System_Safety_Analysis_Malin5
Technology Overview
Extract and Model
Analyze, Simulate and Test Early
- Identify interaction-propagation paths
- Investigate influence of timing
- Perform Virtual Tests
Requirements Text
Hazard Identification Tool (HIT)
CONFIG Hybrid Simulation
Virtual System Integration Laboratory (VSIL)
Information Extractor Identify Test Cases
Aerospace Ontology
SAS_08_Automation_for_System_Safety_Analysis_Malin6
Modeler: Architecture Model and Visualization of a Set of Requirements
[C.1] Telecommunication Subsystem (TeleSub)• [C.1.1] The CDHC sends the TeleSub a compressed
picture. [FG.1] [TeleSub C.1.4]• [C.1.2] The CDHC sends the TeleSub telemetry. [FG.2]
[FR.1] [FR.5] [TeleSub C.1.5] • [C.1.3] The CDHC sends In View of Ground alerts to the
TeleSub. [DP.5.6] [TeleSub C.1.6]• [C.1.4] The CDHC receives plan files from the TeleSub.
[FR.3] [TeleSub C.1.3]• [C.1.5] The CDHC receives ground commands from the
TeleSub. [FR.3] [TeleSub C.1.2]
• [C.1.6] The CDHC receives the TeleSub operating state
from the TeleSub. [DP.5.5] [TeleSub C.1.1] …
[C.2] Camera Subsystem• [C.2.1] The CDHC sends the Camera a "take picture"
command. [FG.2] [FR.1] [FR.3] • [C.2.2] The CDHC sends the Camera x, y and z gimballing
coordinates. [FG.2] [FR.1] [FR.3] • [C.2.3] The CDHC sends a turn on command to the
Camera. [DP.5.3] [H Constraint 1.1.4]• [C.2.4] The CDHC sends a turn off command to the
Camera. [DP.5.3] • [C.2.5] The CDHC receives a compressed picture file from
the Camera. [FG.1] [FG.2] [FR.1]
…
[C.4] Attitude Determination Subsystem (ADS)• [C.4.1] The CDHC receives an In View of Ground alert from
the ADS. [DP.5.6] [ADS]• [C.4.2] The CDHC receives the ADS operating state from
the ADS. [DP.5.5] [ADS]
Note: CDHC is Command and Data Handling ComputerPhysical/Functional Architecture Visualization
SAS_08_Automation_for_System_Safety_Analysis_Malin7
CONFIG Simulation: Assess Timed Scenarios
CONFIG simulation tool used for software virtual validation testing for NASA 1997 90-day manned Lunar Life Support Test
• Software: Intelligent control for gas storage and transfer• Models: Gas volumes and processing systems controlled by software; mixed fidelity, discrete and continuous • Testing: Simulated failures and imbalances that would not be tested in hardware-software integration
• Too slow to develop, too expensive, too destructive• Results: Identified software requirements deficiency due to unintended consequences of integrating gas processing systems
SAS_08_Automation_for_System_Safety_Analysis_Malin8
Virtual System Integration Lab (VSIL)
• Triakis has used VSIL in >25 avionics verification projects
• Project Output to VSIL: Models and test definitions
Models and Test Definitions
DE: detailed executable, the simulation of the embedded controller hardware RAM/ROM: memories
ES: executable specifications I/O: input/output
V&V: verification and validation CPU: processor
SAS_08_Automation_for_System_Safety_Analysis_Malin9
2008 Target Capability• Integration: Information extraction, architecture modeling and test
generation– Model parts extracted from requirements and FMEA/CIL texts
• XML output, including reference traces• Components, physical hierarchy, connections, interface components,
flows/resources, time or phase context• Functions, vulnerabilities, limits, failures, causes
– Ontology for model extraction and semi-automated modeling• Identify types of components, functions, problems, resources• Paths: A provides power to B; C receives command data from B• Functions and failures: B processes command data; B failure mode is No Output
command to D; cause of no output is B does not receive power.– Semi-automated model development from extracted model parts
• Component model library: Resource producer; Data processor…• Generic functions, failures and influences: Resource problem, Stressor, Data rate
problem, Data Integrity problem…– Model visualization for overview and completeness checking– Simulation and path analysis to identify hazardous configurations, scenarios
and test cases • Where failure or degradation of required functions results from unintended system
interactions
• Project Participants– CEV Flight Software Engineering, Abort Decision Logic, Abort Sequence– Orion Software Safety and Mission Assurance
SAS_08_Automation_for_System_Safety_Analysis_Malin10
CEV Launch Abort System (LAS) Case
Crew Exploration Vehicle (CEV) Pad Abort Sequence - notional
SAS_08_Automation_for_System_Safety_Analysis_Malin11
CEV Launch Abort System Case
• CEV Crew Module (CM) software controls the Pad Abort Sequence – LAS events, trajectory– No direct command feedback
• Components for the case– Paths and interactions for commands firing separation pyros during
pad abort sequence• CM computer → Remote Interface Unit → LAS Pyros• Possible Addition Case: Inertial measurement unit (IMU) → GN&C →
Abort motor– Summer Systems Engineering Intern manually built TEAMS models
of pyros and Remote Interface Unit (RIU) from FMEA/CILs• Need to perform simulation rather than pure path analysis on
the LAS case– Timing is important in aborts– Hazard Identification Tool (HIT) path analysis models do not
capture timing– CONFIG simulations can use timing
SAS_08_Automation_for_System_Safety_Analysis_Malin12
Data – ConOps, Requirements, Safety Analyses
• Developing tools and methods using documents and data from CEV sources, both NASA and Orion contractor
• Met with NASA expert on Orion software that controls Launch Aborts– Identified key CEV documents and confirmed analysis
approach• Orion Contractor’s Concept of Operations
– Best guess at the Abort Sequence• Interface Requirements Documents• Interface Control Documents, when they become available• Project Orion Flight System Safety Hazard Analysis• FMEA/CILs (preliminary now)
– Determined that much key information (sensors, feedback) is TBD, and being defined by the Cx Integrated Abort Team
SAS_08_Automation_for_System_Safety_Analysis_Malin13
Technical Challenges
• Limitations of early life-cycle requirements, design, hazard analysis and FMEA/CIL as sources for– Automatic extraction of model information from requirements
and design text– Semi-automatic construction of models from extracted
information, for simulation and visualization
• Combining graph analysis and simulation to identify possible hazard paths and off-nominal test scenarios for complex system interaction models
• Maturation Challenge: Develop mature software prototypes that can be used to develop products for broader use
SAS_08_Automation_for_System_Safety_Analysis_Malin14
Information Extraction• Objective: Extract information from CEV sources for semi-automated model
construction • Information Extraction Evaluation
– Success Criterion: % of available model information extracted, compared to % of model information available
• Types of Extractions (from text to XML)– Interface Requirements → Components, connections, flows/resources,
reference trace • Some interface components, vulnerabilities, functions, limits, context (time or phase)
– FMEA/CILs → Components, system to component hierarchy, interfaces, subcomponents, functions, failures, causes, effects
– Architecture Descriptions → Components, interfaces, hierarchy, functions; some design parameters, acronyms
• Challenges– Multiple document formats require definition of data structures for each
document, some difficult sentence parsing– Indirect access to Cradle requirements via PDF documents
• Progress: – Parser improved by incorporating NESC-funded parser from Univ. Central
Florida– Experience coding multiple document data structures, leading to format
specification approach– Successful extraction from Cradle-based PDF docs.
SAS_08_Automation_for_System_Safety_Analysis_Malin15
Example Information Extraction Benchmarking
LAS ComponentsComponent: Nose cone
Component: Canard Section Function: ? Component Enables Function: LAS: reorient the CM Function Enables Function: LAS?: deploy
parachute (following an abort) Component Group: Three propulsive motors Component: attitude control motor
Enables Function: LAS?: control attitude … Component: A bi-conic adapter Connection
Connector: bi-conic adapterFrom: LASTo: CMType: Structural
Component: Boost protective cover Acronym: Boost protective cover = BPC Design parameter: size
Determined by: ascent heating Function: protect CM thermal protection system coatings
Agent: Boost protective coverAction: protectOperand: CM thermal protection system coatings
Other: CM Component: Thermal protection system
Acronym: thermal protection system = TPS
LAS Description: The LAS consists of a nose cone, a canard section which enables the LAS to reorient the CM for parachute deployment following an abort, three propulsive motors (attitude control, jettison, and abort), a bi-conic adapter which provides the structural interface to the CM, and a boost protective cover (BPC) sized for ascent heating to protect CM thermal protection system (TPS) coatings.
Ideal Model Extraction Benchmark: Top Level: LAS Function: reorient the CM Agent: LAS
Action: reorientOperand: CM
Function: control attitude Agent: LAS?
Action: controlOperand: ?
Variable: attitude … Function: deploy parachute Agent: LAS?
Action: deployOperand: parachute
SAS_08_Automation_for_System_Safety_Analysis_Malin16
Model Construction• Objective: Use extracted model information
for semi-automatic model construction • Challenges
– LAS case with timing issues requires CONFIG simulation, not just HIT architecture model
– Missing information in pre-PDR documents• Operating modes, vulnerabilities, side effects etc.
• Progress– CONFIG provides visualization and supports
libraries of generic components– Concept for use of Aerospace Ontology
hierarchies with CONFIG library for generic components, operating modes, functions, side effects, problems
SAS_08_Automation_for_System_Safety_Analysis_Malin17
Analysis and Test Case Generation• Objectives:
– Identify and evaluate failure and hazard propagation in the system model• Unintended system interactions and unanalyzed propagation of failure
effects– Generate corresponding off-nominal test cases (configurations, scenarios)
• Evaluations – Hazard Analysis Success Criterion: Are new hazards and failures identified,
compared to standard method?– Test Case Success Criterion: Does model-based hazards and failures
analysis make test generation easier than current methods?• Challenges
– Path analysis algorithm for HIT models needs to be adapted, because LAS case will use CONFIG simulation rather than HIT
– Extracting information for operational and failure scenarios has not yet been addressed
• FMEA/CIL effects information can provide parts of failure scenarios• Progress
– Concept for combining HIT and CONFIG models, using CONFIG Inner Models for subsystem details
SAS_08_Automation_for_System_Safety_Analysis_Malin18
2009 Planned Capability
• Capabilities should be valuable from pre-PDR through operations
• Continue tool enhancements focusing on – Off-nominal test scenario discovery and evaluation– Component model library and generic defaults
• Use on a new CEV case – more complex interactions and more complete system information
• Deliver– Tool prototype files – Information extraction tool, Aerospace
ontology, HIT graph modeling and analysis, CONFIG simulation modeling, model libraries
– Documentation – methods, tools, user manuals
SAS_08_Automation_for_System_Safety_Analysis_Malin19
Future Applications
• Improve efficiency and repeatability of system and software risk analysis – Reduce time spent reanalyzing when specifications and designs
change• Visualize integrated requirements
– Combined success and failure spaces– Combined system and operation/event spaces
• Validate requirements and perform integration tests early with low-fidelity and multi-fidelity simulation
• Validate FMEAs and fault trees• Evaluate completeness and consistency of requirements
and risk– Support requirements traceability evaluations– Enhance analysis with reliability and event probability
information