bridging the gap between model-based development and …7) bridging the gap... · scade lustre...
Post on 01-Feb-2018
222 Views
Preview:
TRANSCRIPT
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
Bridging the Gap Between Model-Based Development and Model Checking
AFRL Safe & Secure Systems & Software Symposium Dr. Steven P. Miller
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
2
Acknowledgements
• NASA Langley Research Center (Ricky Butler)
• Air Force Research Labs
• University of Minnesota (Dr. Mats P. E. Heimdahl)
• Lockheed Martin
• Dr. Mike Whalen
• Dr. Darren Cofer
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
3
Presentation Overview
Who Are We?
What Problem are We Solving?
Overview of Our Approach
Case Studies
Challenges and Future Directions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
4
Rockwell Collins Headquartered in Cedar Rapids, Iowa
20,000 Employees Worldwide
2008 Sales of $4.77 Billion
AfricaJohannesburg, South Africa
AsiaBangkok, ThailandBeijing, ChinaHong KongHyderabad, IndiaKuala Lumpur, MalaysiaManila, PhilippinesMoscow, RussiaOsaka, JapanShanghai, ChinaSingaporeTokyo, Japan
AustraliaAuckland, New ZealandBrisbane, AustraliaMelbourne, AustraliaSydney, Australia
CanadaMontrealOttawa
EuropeAmsterdam, NetherlandsFrankfurt, GermanyHeidelberg, GermanyLondon, EnglandLyon, FranceManchester, EnglandParis, FranceReading, EnglandRome, ItalyToulouse, France
MexicoMexicali
South AmericaSantiago, ChileSao Jose dos Campos, BrazilSao Paulo, Brazil
MinnesotaMinneapolis
MissouriKansas CitySt. Louis
New YorkNew York
North CarolinaCharlotteRaleigh
OklahomaMidwest CityTulsa
OregonPortland
PennsylvaniaPhiladelphiaPittsburgh
TexasDallasFort WorthRichardson
UtahSalt Lake City VirginiaSterlingWarrenton
WashingtonKirklandRentonSeattle
Washington, DC
CaliforniaCarlsbadCypressIrvineLos AngelesPomonaPowaySan FranciscoSan JoseTustin
FloridaMelbourneMiamiOrlando
GeorgiaAtlantaWarner Robins
HawaiiHonolulu
IllinoisChicago
IowaBellevueCoralvilleDecorahManchester
KansasWichita
MarylandWhite Marsh
MassachusettsBoston
MichiganAnn ArborDetroit
InternationalDomestic
AfricaJohannesburg, South Africa
AsiaBangkok, ThailandBeijing, ChinaHong KongHyderabad, IndiaKuala Lumpur, MalaysiaManila, PhilippinesMoscow, RussiaOsaka, JapanShanghai, ChinaSingaporeTokyo, Japan
AustraliaAuckland, New ZealandBrisbane, AustraliaMelbourne, AustraliaSydney, Australia
CanadaMontrealOttawa
EuropeAmsterdam, NetherlandsFrankfurt, GermanyHeidelberg, GermanyLondon, EnglandLyon, FranceManchester, EnglandParis, FranceReading, EnglandRome, ItalyToulouse, France
MexicoMexicali
South AmericaSantiago, ChileSao Jose dos Campos, BrazilSao Paulo, Brazil
MinnesotaMinneapolis
MissouriKansas CitySt. Louis
New YorkNew York
North CarolinaCharlotteRaleigh
OklahomaMidwest CityTulsa
OregonPortland
PennsylvaniaPhiladelphiaPittsburgh
TexasDallasFort WorthRichardson
UtahSalt Lake City VirginiaSterlingWarrenton
WashingtonKirklandRentonSeattle
Washington, DC
CaliforniaCarlsbadCypressIrvineLos AngelesPomonaPowaySan FranciscoSan JoseTustin
FloridaMelbourneMiamiOrlando
GeorgiaAtlantaWarner Robins
HawaiiHonolulu
IllinoisChicago
IowaBellevueCoralvilleDecorahManchester
KansasWichita
MarylandWhite Marsh
MassachusettsBoston
MichiganAnn ArborDetroit
InternationalDomestic
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
5
• Commercial/Military Avionics Systems
• Communications
• Navigation & Landing Systems
• Flight Control
• Displays
• Weapon Data Links
“Working together creating the most trusted source of communication and aviation electronic solutions”
Rockwell Collins’ core business is based on the delivery of High Assurance Systems
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
6
Advanced Technology Center
Identify, acquire, develop and transition value-driven technologies to support the continued growth of Rockwell Collins.
Technologists: 173 Administrators: 10 Technicians: 31
Automated Analysis Section
Applies mathematical tools and reasoning to the production of high assurance systems.
Technologists: 10 Administrators: 1
27%9%
64%PhD
MS
BABA46%
37%17%PhD
Masters
Bachelors
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
7
AAMP5 Microcode Verification
(PVS)
AAMP-FV Microcode Verification
(PVS)
AAMP5 Partitioning
(PVS)
JEM Java μProc (PVS)
FGS Mode Confusion
Study (PVS)
FCP 2002 Microcode
(ACL2) AAMP7 Separation
Kernel (ACL2)l
FGS Mode Confusion (RSML-e,
PVS) FGS Safety
Analysis (RSML-e, NuSMV)
ADGS 2100 (Simulink, NuSMV)
NASA Aviation Safety
vFaat (ACL2, PVS)
NSA
SHADE (ACL2)
Turnstile (SPARK)
Guardol (ACL2, Prover)
AFRL
Greenhills Integrity
RTOS (ACL2)
CerTA FCS
(NuSMV, Prover)
Greenhills Integrity
Gen4 (ACL2)
Mixed Criticality
Architectures
1994 1996 1998 2000 2002 2004 2006 1992 2008 2010
Formal Methods at Rockwell Collins
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
8
Presentation Overview
Who Are We?
What Problem are We Solving?
Overview of Our Approach
Case Studies
Challenges and Future Directions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
9
1
10
100
1000
10000
100000
1965 1970 1975 1980 1985 1990 1995
K W
ords
INS4K
A300B
200K
A300FF
A3102M
A3204M
A330/A340
10M
23K
J.P. Potocki De Montalk, Computer Software in Civil Aircraft, Sixth Annual Conference on Computer Assurance (COMPASS ’91), Gaithersberg, MD, June 24-27, 1991.
Airborne Software Doubles Every Two Years
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
10
Complexity Size
1995 1970
No. of Signals
Object Code
(Mbytes)
230K
0
100
0 1995 1970
Year
747-200 757/767
747-400
777
747-200 757/767
747-400
777
Year
Similar Growth Has Been Seen by Boeing
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
13
Presentation Overview
Who Are We?
What Problem are We Solving?
Overview of Our Approach
Case Studies
Challenges and Future Directions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
14
Exploit the Convergence of Two Trends
• Model-Based Development – Domain specific graphical notations – MATLAB Simulink®, Esterel Technologies SCADE Suite™ – Enable early simulation and debugging – Automated generation of code and tests
• Model-Checking – Prove properties about a model – Explore all possible inputs and states – Highly automated
Reduce Costs and Improve Quality by Using Analysis to Find Errors During Early Design
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
15
Company Product Tools Specified & Autocoded Benefits Claimed Airbus A340 SCADE
With Code Generator
• 70% Fly-by-wire Controls • 70% Automatic Flight Controls • 50% Display Computer • 40% Warning & Maint Computer
• 20X Reduction in Errors • Reduced Time to Market
Eurocopter EC-155/135 Autopilot
SCADE With Code Generator
• 90 % of Autopilot
• 50% Reduction in Cycle Time
GE & Lockheed Martin
FADEDC Engine Controls
ADI Beacon • Not Stated
• Reduction in Errors • 50% Reduction in Cycle Time • Decreased Cost
Schneider Electric
Nuclear Power Plant Safety Control
SCADE With Code Generator
• 200,000 SLOC Auto Generated from 1,200 Design Views
• 8X Reduction in Errors while Complexity Increased 4x
US Spaceware
DCX Rocket MATRIXx • Not Stated
• 50-75% Reduction in Cost • Reduced Schedule & Risk
PSA Electrical Management System
SCADE With Code Generator
• 50% SLOC Auto Generated • 60% Reduction in Cycle Time • 5X Reduction in Errors
CSEE Transport
Subway Signaling System
SCADE With Code Generator
• 80,000 C SLOC Auto Generated • Improved Productivity from 20 to 300 SLOC/day
Honeywell Commercial Aviation Systems
Primus Epic Flight Control System
MATLAB Simulink
• 60% Automatic Flight Controls • 5X Increase in Productivity • No Coding Errors • Received FAA Certification
Model-Based Development
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
16
• Breakthrough Technology of the 1990’s • Widely Used in Hardware Verification (Intel, Motorola, IBM, …) • Several Different Types of Model Checkers
– Explicit, Symbolic, Bounded, Infinite Bounded (SMT), …
• Exhaustive Search of the Global State Space – Consider All Combinations of Inputs and States – Equivalent to Exhaustive Testing of the Model – Produces a Counter Example if a Property is Not True
• Easy to Use – “Push Button” Formal Methods – Very Little Human Effort Unless You’re at the Tool’s Limits
• Limitations – State Space Explosion (10100 – 10200 States)
What Are Model Checkers?
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
17
Even Small Systems Have Trillions (of Trillions) of Possible Tests!
Testing Checks Only the Values We Select Model Checker Tries Every Possible Value!
Finds every exception to the property being checked!
Advantage of Model Checking
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
18
Rockwell Collins Translation Framework
SCADE
Lustre
NuSMV
PVSSafe StateMachines
SAL Symbolic Model Checker
SAL
Simulink SimulinkGateway
StateFlow
Reactis ACL2
Prover
SimulinkGateway
C, Ada
SAL Infinite Model Checker
SAL Bounded Model Checker
Rockwell Collins/U of Minnesota
MathWorks
SRI InternationalReactive Systems
Esterel Technologies
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
19
• Many small Lustre-to-Lustre translation passes
• Each pass refines closer to the target language
• Each pass deals with one change
• Pre/Post conditions define when a pass is valid
Pretty Print
Lustre Lustre Lustre
Lustre
Lustre
Lustre
Lustre C Code
Pretty Print
Lustre Lustre Ada Code
Pretty Print
Lustre PVS
Pretty Print
Lustre Lustre
Lustre NuSMV
Pretty Print
Lustre Lustre Prover
Lustre
RDV
LustreREPRNC
RDV
SCA
RNC IPS
RC
REN
FNH
PTL
IASRCRFBY
RACT
RNST
A Product Family of Translators
• Last step pretty prints to the target language
• Extensive reuse of passes
• New translators can be developed quickly (usually in less than a week)
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
20
Model
CPU Time (For NuSMV to Compute
Reachable States)
Improvement
Before After
Mode1 > 2 hours 11 sec > 650x
Mode2 > 6 hours 169 sec > 125x
Mode3 > 2 hours 14 sec > 500x
Mode4 8 minutes < 1 sec 480x
Arch 34 sec < 1 sec 34x
WBS 29+ hours 1 sec 105,240x
Translators Optimize for Specific Analysis Tools
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
21
Presentation Overview
Who Are We?
What Problem are We Solving?
Overview of Our Approach
Case Studies
Challenges and Future Directions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
22
ADGS-2100 Adaptive Display & Guidance System
Example Requirement: The Cursor Shall Never be
Positioned on an Inactive Display
Counterexample Found in 5 Seconds
Checked 563 Properties - Found and Corrected 98 Errors
in Early Design Models
Modeled in Simulink
Translated to NuSMV
4,295 Subsystems
16,117 Simulink Blocks
Over 1037 Reachable States
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
23
Translation Time: 1-4 Hours Turnaround: 1 Day to 1 Week
Iteration 1
Simulink R14 Model
Simulink R13 Model
SCADE Model
NuSMV Model
Translation Time: 10 Minutes Turnaround: 3 Hours to 2 Days
Iteration 2
Simulink R14 Model
Reactis Model
NuSMV Model
Translation Time: 10 Minutes Turnaround: 10 Minutes
Iteration 3
Simulink R14 Model
Reactis Model
NuSMV Model
ATC Group (Beige)
Dev. Group (Blue)
ADGS-2100 Technology Transfer
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
24
• Sponsored by the Air Force Research Labs – Air Vehicles (RB) Directorate - Wright Patterson
• Investigate Roles of Testing and Formal Verification – Can formal verification complement or replace some testing?
• Example Model – Lockheed Martin Adaptive UAV Flight Control System – Redundancy Management Logic in the Operational Flight Program (OFP) – Well suited for verification using the NuSMV model-checker
Lockheed Martin Aero Rockwell Collins
• Enhanced During CerTA FCS
• Based on Testing
– Graphical Viewer of Test Cases – Support for XML/XSLT Test Cases – Added C++ Oracle Framework
• Developed Tests from Requirements
• Executed Tests Cases on Test Rig
• Developed Properties from Requirements
– Support for Simulink blocks – Support for Stateflow – Support for Prover model-checker
• Enhanced During CerTA FCS
• Based on Model-Checking
• Proved Properties using Model-Checking
CerTA FCS Phase I
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
25
For Each of Ten Control Surfaces
• Triplex Voter – Input monitor, sensor fusion, and
failure isolation
• Failure Processing – Logs failures into a data store
• Reset Manager – Reset logic for sensors and control
surfaces (not shown)
Subsystems / Blocks
Charts / Transitions
Truth Table Cells
Reachable State Space Properties
Triplex voter 10 / 96 3 / 35 198 6.0 * 1013 48
Failure processing 7 / 42 0 / 0 0 2.1 * 104 6
Reset manager 6 / 31 2 / 26 0 1.32 * 1011 8
Total 23 / 169 5 / 61 198 N/A 62
CerTA FCS Phase I - OFP Redundancy Management Logic
4input_sel
3totalizer_cnt
2persistence_cnt
1failure_report
pc
trigger
input_a
input_b
input_c
DST_index
input_sel
triplex_input_selector
input_a
input_b
input_c
trip_lev el
persist_lim
MS
f ailreport
pc
tc
triplex_input_monitor
trip_leveltrip_level1
persist_limpersistence limit
[DSTi]
[C]
[B]
[status_c]
[status_b]
[status_a]
[A]
[trigger]
[DSTi][MS]
[MS]
[DSTi][A]
[prev_sel]
[prev_sel]
[DSTi]
[trigger]
[trigger]
[status_c]
[status_b]
[status_a]
[A]
[A]
IndexVector
[C]
[B]
[C]
[B]
[C]
[B]
f ailure_report
dst_index
Failure_Processing
mon_f ailure_report
status_a
status_b
status_c
prev _sel
input_a
input_b
input_c
f ailure_report
Failure_Isolation
Extract Bits[0 3]
Extract Bits
DOCText
double
DST
Data StoreRead
8dst_index
7status_c
6status_b
5status_a
4input_c
3input_b
2input_a
1sync
persist_lim
totalizer_cnt<tc>
trip_lev el
persistence_cnt<pc>
sy nc<>
f ailreport
Input Monitor
Failure Processing
Failure Isolation
Sensor Fusion
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
26
Errors Found in Redundancy Manager Model Checking Testing
Triplex Voter Failure Processing Reset Manager
Total
3 5
4 12
0 0 0 0
• Model-Checking Found 12 Errors that Testing Missed
• Spent More Time on Testing than Model-Checking
– 60% of total on testing vs. 40% on model-checking
Model-checking was more cost effective
than testing at finding design errors.
CerTA FCS Phase I – Errors Found
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
27
• Sponsored by the Air Force Research Labs – Air Vehicles (RB) Directorate - Wright Patterson
• Can Model-Checking be Used on Infinite State Systems? – Large, numerically intensive, non-linear systems
CerTA FCS Phase II
• Example Model – Lockheed Martin Adaptive UAV
Flight Control System – Effector Blender (EB) – Generates actuator commands
for aircraft control surfaces – Matrix arithmetic of floating
point numbers
• Challenges – Identifying the right properties to verify – Verification of floating point numbers – Verification of Stateflow flowcharts with cyclic transition paths – Compositional verification to scale to entire Effector Blender
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
28
• Generates Actuator Commands – Six control surfaces – Adapts its behavior as aircraft
state changes – Iterative algorithm that
repeatedly manipulates a 3 x 6 matrix of floating point numbers
• Large Complex Model – Inputs
• 32 floating point inputs • 3 x 6 matrix of floating point values
– Outputs • 1 x 6 vector of floating point values
– 166 Simulink subsystems – 2000+ basic Simulink blocks – Huge reachable state space
• Completely Functional – No internal state
Surf1 left vertical tail surf2 right vertical tail surf3 left flap surf4 right flap surf5 left outboard spoiler surf6 right outboard spoiler
Control EffectorArrangement
Spoilers (L&R)
V-Tail Rudders (L&R)
Flaps (L&R)
CerTA FCS Phase II – Effector Blender
1
Effector Blender
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
29
• No Explicit Requirements for the Effector Blender Model – Requirements defined for Effector Blender + aircraft model – Addition of aircraft model pushes verification beyond current tools
• Avoid Properties Verifiable by Other Means – Control theory – stability, tracking performance, feedback design … – Simulation – design validation – Implementation – code generation/compilation, scheduling, …
• Focus on the Consistency of the Effector Blender Model – Relationships the model should always maintain – Partial requirements specification
• Preservation of Control Surface Limits – EB computes upper and lower limits for each control surface command – Function of aircraft design, aircraft state, and max extension per cycle – Commanded extension should always be between these limits
CerTA FCS Phase II – What to Verify?
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
30
• Floating Point Numbers – Fixed number of bits with a movable decimal (radix) point – No decision procedures for floating point numbers available
• Real Numbers – Real numbers have unbounded size and precision – Would hide errors caused by limitations of floating point arithmetic – Control theory problems are inherently non-linear – Decision procedures for non-linear real numbers have exponential cost
• Solution - Translate Floating Point Numbers into Fixed Point – Extended translation framework to automate this translation – Convert floating point to fixed point (scaling provided by user) – Convert fixed point into integers (use bit shifting to preserve magnitude) – Shift from NuSMV (BDD-based) to Prover (SMT-solver) model checker
• Advantages & Issues – Use bit-level integer decision procedures for model checking – Results unsound due to loss of precision – Highly likely to find errors – very valuable tool for debugging
CerTA FCS Phase II – Verification of Floating Point Numbers
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
31
Typical Specification – Models are typically organized in a hierarchy of subsystems – Subsystems are often nested several levels deep – Most of the complexity is in the leaf subsystems – Leaf subsystems can often be verified through model checking
CerTA FCS Phase II – Compositional Verification
1 Out1
1 In1
2 In2
In_B1
In_B2 Out_B
Subsystem B
In_A1
In_A2 Out_A
Subsystem A
P2 & P3 -> Q1
Q2
P1 & Q1 -> Q2
Q1
Composition of Subsystems – Tends to be simple – Lends itself well to theorem proving
P2 & P3 => Q1 P1 & Q1 => Q2
P1 & P2 & P3 => Q =>
Q
P1
P2 & P3
Issues – Need to avoid circular reasoning to ensure soundness – Can be ensured by eliminating cyclic dependencies between atomic subsystems – Identifying the right leaf level invariants to support composition – Complexity of the proof obligations for the intermediate levels – Lack of a unified automated verification system
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
32
• Can Model-Checking be Used on Infinite State Systems? – Large, numerically intensive, non-linear systems
CerTA FCS Phase II - Results
• Effector Blender – Inputs
• 32 floating point inputs • 3 x 6 matrix of floating point values
– Outputs • 1 x 6 vector of floating point values
– 166 Simulink subsystems – 2000+ basic Simulink blocks
• Errors Found – Five previously unknown errors that would drive actuators past their limits – Several implementation errors were being masked by defensive programming
WPAFB 08-5183 RBO-08685 8/20/2008
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
33
Presentation Overview
What Problem are We Solving?
Who Are We?
What are Formal Methods?
Examples of Using Formal Methods
Challenges and Future Directions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
34
Theorem Provers
Arbitrary Models Labor Intensive
Non Linear Arithmetic
Transcendental Functions
Floating Point
Decision Procedures
Extending the Verification Domain
• Very Large or Infinite State Systems – SMT-Solvers – Large integers and reals – Limited to linear arithmetic – Ease of use is a concern
SMT-Solvers
Infinite State Models using k - Induction
Implicit State
< 10 200 Reachable States
Model Checkers
• Non-Linear Arithmetic – Multiplication/division of real variables – Transcendental functions (trigonometric, …) – Essential to navigation systems
• Large Finite Systems (<10200 States) – Implicit state (BDD) model checkers – Easy to use and very effective
• Theorem Provers – Deal with arbitrary models – Concerns are ease of use and labor cost
• Floating Point Arithmetic – Most modeling languages use floating
point (not real) numbers
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
35
setDesiredSpeedboolean
2
modeuint32
1
setEvent
safetyCondition
cancel
brakePedal
carGear
carSpeed
validinputs
safetyCondition
resumeEvent
mode _logic
onOff
decel
set
accel
resume
safetyCondition
mode
setDesiredSpeed
Delay = 1 Sec
Delay = 1 Sec
validInputsboolean
true_false
8
carSpeeddouble
miles_per_hour
7
carGearuint32
enumerated
6
brakePedalbooleanon_off
5
cancelboolean
true_false
4
accelResumebooleanon_off
3
decelSetbooleanon_off
2
onOffbooleanon_off
1
cruiseThrottledouble
miles_per_hour
1
throttleDelta%_per_step
thottleDelta%_per_second
1.00
isCruiseActive ?
<Init = 0.0>
z
1
StepsPerSec
<U=10.0><L=-10.0>
<U=100.0><L=0.0>
NO THROTTLEdouble
0.0
doublecarSpeed
doublemiles_per_hour
3
desiredSpeeddouble
miles_per_hour
2
modeuint32
enumerated
1
desiredSpeeddouble
miles_per_hour
3
cruiseThrottledouble
percentage
2
modeuint32enumerated
1
[carSpeed ]
[carSpeed ]
SetThrottle
mode
desiredSpeed
carSpeed
cruiseThrottle
SetDesiredSpeed
mode
carSpeed
setDesiredSpeed
desiredSpeed
ModeLogic
onOff
decelSet
accelResume
cancel
brakePedal
carGear
carSpeed
validInputs
mode
setDesiredSpeed
Goto[carSpeed ]
validInputsboolean
true_false
8
carSpeeddouble
miles_per_hour
7
carGearuint32
enumerated
6
brakePedalbooleanon_off
5
cancelboolean
true_false
4
accelResumebooleanon_off
3
decelSetbooleanon_off
2
onOffbooleanon_off
1
cruiseThrottledouble
percentage
3
desiredSpeeddouble
miles_per_hour
2
modeuint32
enumerated
1
isBrakePressed?[brakePosition ]
CruiseController
onOff
decelSet
accelResume
cancel
brakePedal
carGear
carSpeed
validInputs
mode
cruiseThrottle
desiredSpeed
validInputsboolean
true_false
7
carSpeeddouble
miles_per_hour
6
carGearuint32
enumerated
5
cancelboolean
true_false
4
accelResumeboolean
true_false
3
decelSetboolean
true_false
2
onOffboolean
true_false
1
Composition of Subsystems – Tends to be simple – Well suited for theorem proving
Typical Model-Based Specification – Models are organized in a
hierarchy several levels deep – Most of the complexity is in the
leaf models – Leaf models can often be verified
through model checking
What Should the User Interface Be? – Not emacs! – Integrated with the model – Simple theorem proving – Powerful model checking
Combining Theorem Proving and Model Checking For Compositional Verification
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
36
• How Many Properties Do I Need? – When am I done? – Are there coverage metrics like there are for testing? – How do I convince the certification authorities I’m done?
• Are Some Properties Better than Others? – Properties related to safety – Cross cutting properties find the most important errors – Simple, local properties find a surprising number of errors
• Is There a Process or Heuristics for Defining Properties? – Prove a property about each discontinuity in each output
• What is the Relationship Between Proof and Testing – Can proving replace some testing? – Can testing replace some proving?
Finding the Right Properties
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
37
Verifying Asynchronous Systems
• Occur Frequently in System Designs – Implement fault tolerance or meet performance requirements
• No Global Clock - Each Node Has Its Own Clock
• Quasi-Synchronous System – Clocks have similar (but not identical) periods, drift, jitter
• Interleaving Leads to State Space Explosion – Makes model checking difficult
• Need to Exploit the Constraints Imposed by Quasi-Synchrony
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
38
System Architectural Modeling & Analysis
Target Hardware
Separation Kernel
Reusable Trusted Middleware(RTOS, I/O , RT-CORBA)
Sys Specific Middleware(Schedule, Communication Routes)
App A App B App C
Common Computing Resource 1Common Computing Resource 2
Common Computing Resource 3
IMA BUS
PerformanceAnalysis
SafetyAnalysis
SecurityAnalysis
IMA Cabinet
ADL
SystemArchitecture
Model
AutoGenerate
Logical
Physical
SimulinkModel
VAPSModel
CCode
AdaCode
CCode
Level BClassified Level C
Unclassified
Level ATop Secret
Implements
Abstra
cts
System Architecture Development
Software C
omponent D
evelopment
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
39
• Formal Methods Are Practical and Are Being Widely Used – Model Based Development is the industrial face of formal methods – The engineers get to pick the modeling tools! – Semantics of some of the commercial tools could be improved
• Formal Verification Tools Are Being Used in Industry – Key is to verify the models the engineers are already building – Large portions of existing systems can be verified with model checkers – Need to make model checking accessible to the average engineer
• Directions for the Future Work – Making verification tools more powerful and easier to use – Integration of theorem proving and model checking – Finding the right properties – Verification of asynchronous systems – Modeling and analysis of system architectural models
Conclusions
© Copyright 2009 Rockwell Collins, Inc. All rights reserved.
40
http://shemesh.larc.nasa.gov/fm/fm-collins-intro.html
• Whalen, M., Cofer, D., Miller, S., Krogh, B., Storm, W.: Integration of Formal Analysis into a Model-Based Software Development Process. In 12th International Workshop on Formal Methods for Industrial Critical Systems (FMICS2007), Berlin, Germany (2007).
• Whalen, M., Innis, J., Miller, S., Wagner, L.: ADGS-2100 Adaptive Display & Guidance System Window Manager Analysis, CR-2006-213952, NASA (2006).
• Mats P.E. Heimdahl, Michael W. Whalen, Ajitha Rajan, and Steven P. Miller, Testing Strategies for Model-Based Development, NASA Contractor Report NASA-2006-CR214307, April 2006. Available at http://hdl.handle.net/2002/214307.
• Miller, S., Tribble, A., Whalen, M., Heimdahl, M., Proving the Shalls, International Journal on Software Tools for Technology Transfer (STTT), Feb 2006.
• Michael W. Whalen, John D. Innis, Steven P. Miller, and Lucas G. Wagner, ADGS-2100 Adaptive Display & Guidance System, NASA Contractor Report NASA-2006-CR213952, Feb. 2006. Available at http://hdl.handle.net/2002/16162.
• Steven P. Miller, Mike W. Whalen, Dan O’Brien, Mats P.E. Heimdahl, and Anjali Joshi, A Methodology for the Design and Verification of Globally Asynchronous/Locally Synchronous Architectures, NASA Contractor Report NASA/CR-2005-213912, Sept. 2005. Available at http://hdl.handle.net/2002/15934.
• Miller, S., Anderson, E., Wagner, L., Whalen, M., Heimdahl, M.: Formal Verification of Flight Critical Software. In AIAA Guidance, Navigation and Control Conference and Exhibit, AIAA-2005-6431, American Institute of Aeronautics and Astronautics (2005).
For More Information
top related