Formal Model-Based Development in Aerospace Systems:Challenges to Adoption
Mats P. E. HeimdahlUniversity of Minnesota Software Engineering Center
Critical Systems Research GroupDepartment of Computer Science and Engineering
University of Minnesota
and a Plea for Help
Domain of Concern
How we Develop SoftwareConcept
Formation
Requirements Specification
Design
Implementation
Integration
System
Unit Test
Integration Test
System Test
Object Code
Test
Analysis
Model-Based Development
SpecificationModel
Visualization PrototypingTesting
Code
Analysis
Properties
Model-Based Development Tools
• Commercial Products– Esterel Studio and
SCADE Studio from Esterel Technologies
– Rhapsody from I-Logix– Simulink and Stateflow
from Mathworks Inc.– Rose Real-Time from
Rational– Etc. Etc.
SystemSpecification/Model
How we Will Develop SoftwareConcept
Formation
Requirements
Implementation
Integration
PropertiesAnalysi
s
Integration Test
System
Test
Specification Test
What Does Industry Want?
Better / SaferCheaperFaster
Model-Based Development Examples
Company Product Tools Specified & Autocoded Benefits Claimed
Airbus A340 SCADE With Code Generator
70% Fly-by-wire Controls 70% Automatic Flight Controls 50% Display Computer 40% Warning & Maint Computer
20X Reduction in Errors Reduced Time to Market
Eurocopter EC-155/135 Autopilot
SCADE With Code Generator
90 % of Autopilot
50% Reduction in Cycle Time
GE & Lockheed Martin
FADEDC Engine Controls
ADI Beacon Not Stated
Reduction in Errors 50% Reduction in Cycle Time Decreased Cost
Schneider Electric
Nuclear Power Plant Safety Control
SCADE With Code Generator
200,000 SLOC Auto Generated from 1,200 Design Views
8X Reduction in Errors while Complexity Increased 4x
US Spaceware
DCX Rocket MATRIXx Not Stated
50-75% Reduction in Cost Reduced Schedule & Risk
PSA Electrical Management System
SCADE With Code Generator
50% SLOC Auto Generated 60% Reduction in Cycle Time 5X Reduction in Errors
CSEE Transport
Subway Signaling System
SCADE With Code Generator
80,000 C SLOC Auto Generated Improved Productivity from 20 to 300 SLOC/day
Honeywell Commercial Aviation Systems
Primus Epic Flight Control System
MATLAB Simulink
60% Automatic Flight Controls 5X Increase in Productivity No Coding Errors Received FAA Certification
Problem 1Believing Testing Can be Eliminated
Testing will always be a crucial (and costly) component
How we Develop SoftwareConcept
Formation
Requirements Specification
Design
Implementation
Integration
System
Unit Test
Integration Test
System Test
AnalysisObject Code
Test
SystemSpecification/Model
Testing Does not go AwayConcept
Formation
Requirements
Implementation
Integration
Properties
Extensive Testing (MC/DC)
SystemSpecification/Model
It Simply MovesConcept
Formation
Requirements
Implementation
Integration
Properties
Extensive Testing (MC/DC)
SystemSpecification/Model
Do it the Right WayConcept
Formation
Requirements
Implementation
Integration
PropertiesAnalysi
s
Integration Test
System
Test
Specification Test
Unit Test
Example: ADGS-2100 Adaptive Display & Guidance System
RequirementDrive the Maximum Number of Display Units
Given the Available Graphics Processors
Counterexample Found in 5 Seconds!
Checking 573 PropertiesFound 98 Errors
883 Subsystems
9,772 Simulink Blocks
2.9 x 1052 Reachable States
Remedy
• Be honest about the capabilities of model-based development and formal methods– Done right, provides outstanding requirements,
models, analysis, etc., etc. – May greatly reduce the effort spent in testing
Problem 2Believing the Model is Everything
The model is never enough
Modeling is so much
fun
Properties
Specification/Model
Modeling FrenzyConcept
Formation
Requirements
Implementation
IntegrationHow do we know the model
is “right”?
Headfirst into m
odeling
System
SystemSpecification/Model
Do it the Right WayConcept
Formation
Requirements
Implementation
Integration
PropertiesAnalysi
s
Integration Test
System
Test
Specification Test
Unit Test
Remedies
• Recognize the Role of Software Requirements– The model is not everything
• Development Methods for Model-Based Development Badly Needed– Model-Based Software Development Process
• Develop Tools and Techniques for Model, Properties, and Requirements Management
• Develop Inspection Checklists and Style Guidelines for Models
Problem 3Trusting Verification
To really mess things up,you need formal verification
Model Checking Process
Does the systemhave property X?
Model
Engineer
SMV
Automatic TranslationSMV Properties
Properties
Automated Check
Yes!
SMVSpec.
Automatic Translation
Model Checking Process
Does the systemhave property X?
Model
Engineer
SMV
Automatic TranslationSMV Properties
Properties
SMVSpec.
Automatic Translation
Counter Example
Automated Check
No!
Property or Model: Who is Right?
AG(Onside_FD_On -> Mode_Annunciations_On)
The Mode Annunciations shall be turned onwhen the Flight Director is turned on
AG( (Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On)
If this side is active, the Mode Annunciations shall be turned on when the Flight Director is turned on
If this side is active and the Mode Annunciations are off, the Mode Annunciations shall be turned on when the Flight Director is turned on
AG( ! Mode_Annunciations_On -> AX ((Is_This_Side_Active & Onside_FD_On) -> Mode_Annunciations_On)))
Translated All the “Shalls” into SMV Properties
Analysis Process Steps• All properties verified (!),
or…• Counterexamples found for
some properties • Simulate counterexample in
MBD environment and make corrections to:– model
– properties
– requirements
– assumptions (invariants)
Formal Analysis Model
MBD Model
Shall Statements
CTL Properties
Corrections Corrections
CorrectionsCorrections
Create Model(Manual)
Formalize Properties (Manual)
Merge(Automated)
Translate(Automated)
Simulation / Corrections
Formal Verification
Remedies• Develop techniques to determine adequacy of model and
property set– How do we know they are any “good”
• Techniques for management of invariants– How do we validate the assumptions we make
• Methodology and guidance badly needed – Tools with training wheels– “Verification for Dummies”
All we need is one high-profile verified systemto fail spectacularly to set us back
a decade or more
Model Checking ProcessWhy? Guru
Does the systemhave property X?
Model
Engineer
SMV
Automatic TranslationSMV Properties
Properties
SMVSpec.
Automatic Translation
Out to Lunch?
Problem 4Believing One Tool Will Be Enough
To be effective, we need a suite of notations and analysis tools
(and the ability to continually integrate new ones)
Original Tool Chain
RSML-e
NuSMV Model Checker
PVS Theorem Prover
Rockwell Collins/U of Minnesota
SRI International
RSML-e to NuSMVTranslator
RSML-e to PVSTranslator
Conversion to SCADE
DesignVerifier
SCADE
Lustre
NuSMV
PVSSafe StateMachines
Simulink
SimulinkGateway
StateFlow
SPY
Esterel Technologies
MathWorks
University of Minnesota/Rockwell Collins (NASA LaRC Funded)
University of Minnesota (NASA IV&V Funded)
Reactive Systems
Esterel Technologies
MathWorks
SRI International
University of Minnesota/Rockwell Collins (NASA LaRC)
University of Minnesota (NASA IV&V)
Current(?) Tool Status
DesignVerifier
SCADE
Lustre
NuSMV
PVS
Safe StateMachines
SAL
ICS
SymbolicModel Checker
BoundedModel Checker
Infinite Model Checker
Simulink
SimulinkGateway
StateFlow
Reactis
SPY
Three Conjectures
• No one modeling language will be universally accepted, nor universally applicable
• No one verification/validation tool will satisfy the analysis needs of a user
• Languages and tools must be tested on real world problems by practicing engineers– Preferably in commercial tools
Translation – with no ILEffort = m * n High quality translations
Lustre ++
polytables
SCADE
RSML-e
PVS
poly’
SMV
C
m modeling languages n target languages
poly
Translation – with ILEffort = m + n Low quality translations
Lustre IL
Lustre ++
polytables
SCADE
RSML-e
PVS
poly’
SMV
C
m modeling languages n target languages
poly
A Proposed Framework (Van Wyk)
• Based on techniques from extensible programming languages, specifically attribute grammars extended with forwarding.
• Hypothesis: – An extensible language may serve as a host language for
domain specific extensions (to construct new modeling languages),
– while forwarding enables the feasible construction of high quality translations from source specification languages to target analysis languages.
• Provided to spur discussion only! There may be better solutions.
Translation – with lang. exts.Effort = m + n + Σ t I High quality translations
Lustre Host
Lustre ++
polytables
SCADE
RSML-e
PVS
poly’
SMV
C
m modeling languages n target languages
forwarding
poly
pvs_trans (t2)
pvs_trans (t1)
c_trans (t3)
forwarding
forwarding
c_trans
smv_trans
pvs_trans
Remedies
• Next generation tools must allow easy extension and modification of notations to meet domain specific needs
• They must allow easy construction of high-quality translations from modeling notations to analysis tools
• They also must enable controlled reuse of tool infrastructure to make tool extensions cost effective
Problem Summary
• Believing Testing Can be Eliminated
• Believing the Model is Everything
• Trusting Verification• Believing One Tool Will
Be Enough
Thank You
• Rockwell Collins– Steven Miller– Michael Whalen– Alan Tribble– Michael Peterson
• NASA Langley– Ricky Butler– Kelly Hayhurst– Celeste Bellcastro
• NASA Ames– Michael Lowry
• NASA IV&V Facility– Kurt Woodham (L3-Titan)
• My Students at Minnesota– Anjali Joshi– Ajitha Rajan– Yunja Choi,– Sanjai Rayadurgam– Devaraj George– Dan O'Brien
Opinions in talk are mine.Do not blame the innocent.
Discussion
For More Information
• Michael W. Whalen et. al., Formal Validation of Avionics Software in a Model-Based Development Process, Formal Methods in Industrial Critical Systems (FMICS’2007), July 2007.
• Steven P. Miller, Alan C. Tribble, Michael W. Whalen, Mats P. E. Heimdahl, Providing the Shalls, International Journal on Software Tools for Technology Transfer (STTT), Feb 2006.
• Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner, ADGS-2100 Adaptive Display & Guidance System, NASA Contractor Report NASA-2006-CR213952, Feb. 2006. Available at http://hdl.handle.net/2002/16162.
• A lot of good reading at http://shemesh.larc.nasa.gov/fm/fm-collins-intro.html
• Eric Van Wyk and Mats Heimdahl. Flexibility in modeling languages and tools: A Call to Arms. To appear in Software Tools for Technology Transfer.