increasing design confidence: model and code verification...dlr autonomous humanoid robot –...
TRANSCRIPT
![Page 1: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/1.jpg)
1© 2015 The MathWorks, Inc.
Increasing Design Confidence: Model and Code Verification
Chuck Olosky
Application Engineer
Nishaat Vasi
Product Manager
![Page 2: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/2.jpg)
2
Finding errors late is expensive and risky
Examples
1996: Ariane 5 rocket destroyed, software defect due to faster horizontal
drifting speed of new rocket – $850 million lost
1999: Loss of Titan IV, incorrect software with wrong roll rate filter constant -
$1.23 billion lost
2013: Recall of 344,000 minivans, software defect may cause application of
brake without driver action
2013: Recall 7,100 cars, software fault results in automatic transmission to
shift out of park without brake press
![Page 3: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/3.jpg)
3
Cruise Control
Module (MBD)
System
InputsOutputsFuel Rate Control
Module
Shift Logic
Control Module
ECU
system
Le
ga
cy c
od
e
Application: Cruise Control
![Page 4: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/4.jpg)
4
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Code
integration
checks
Field tests
Early prototyping
Dead logic due to float-to-fixed model conversion
Identify requirement gaps, assess model
coverage, model-code equivalence
Changing analog-to-digital
converter from 14 to 12-bit
results in dead code
While going downhill, target
speed increases with “reduce
speed” button
![Page 5: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/5.jpg)
5
Demo: Cruise Control Overview
Simulink, Simulink Design Verifier
![Page 6: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/6.jpg)
6
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Code
integration
checks
Field tests
![Page 7: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/7.jpg)
7
Finding Unintended Behavior
Converting floating-point model to integer calibrations, signals…
Dead logic due to “uint8” operation
![Page 8: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/8.jpg)
8
Finding Unintended Behavior
Dead logic due to “uint8”
operation on incdec/holdrate*10
Fix change the order of operation
10*incdec/holdrate
Condition can never be false
![Page 9: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/9.jpg)
9
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Code
integration
checks
Field tests
![Page 10: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/10.jpg)
10
Simulation Testing Workflow
Structural coverage
report
Did we completely
test our model?
Did we meet
requirements?
Review functional
behavior
Design
Requirements
![Page 11: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/11.jpg)
11
Did We Completely Test our Model?
Model Coverage
Analysis
Potential causes of less
than 100% coverage:
Missing requirements
Over-specified design
Design errors
Missing tests
![Page 12: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/12.jpg)
12
Demo: Simulation Based Testing
Simulink Test, Simulink Verification and Validation
![Page 13: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/13.jpg)
13
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Code
integration
checks
Field tests
![Page 14: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/14.jpg)
14
ECU System Architecture
Cruise_onoff
Brake
Speed
Coast set
Accel reset
EGO Sensor
MAP Sensor
Inputs
Gear
Engaged
Target speed
Fuel Rate
Outputs
Fuel Rate Control
Module
Shift Logic
Control Module
Cruise Control
Module (MBD)
ECU
system
Le
ga
cy c
od
e
![Page 15: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/15.jpg)
15
Demo: Code Integration Errors
Polyspace Code Prover
![Page 16: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/16.jpg)
16
Code Integration Issues
Dead code
Maximum target speed = 90Target speed parameter
propagated to “Cruise_ctrl.c”
[0 … 40]
![Page 17: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/17.jpg)
17
Search for Parameter in Upstream Source
![Page 18: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/18.jpg)
18
Use Call Graphs for Multi-File Traceability
![Page 19: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/19.jpg)
19
Root Cause for Dead Code
Changing analog-to-digital converter from 14 to 12-bit results in dead code
MASK – accounts for
scaling down for new ADC
from 14-bit to 12-bit
CONV_FACTOR –
accounts for translating
sensor input to miles/hrt
Overlooked changing
CONV_FACTOR for new
ADC
![Page 20: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/20.jpg)
20
Cruise Control
Module (MBD)
Find Dead Code During Integration
Cruise_onoff
Brake
Speed
Coast set
Accel reset
EGO Sensor
MAP Sensor
Inputs
Fuel Rate Control
Module
Shift Logic
Control Module
Cruise Control
Module (MBD)
ECU
system
Le
ga
cy c
od
e
Inaccurate
scaling for
speed
Dead c
ode
Gear
Engaged
Target speed
Fuel Rate
Outputs
![Page 21: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/21.jpg)
21
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Code
integration
checks
Field tests
![Page 22: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/22.jpg)
22
Field Calibration Tests Uncover Error
Problem: While going downhill, target speed increases with “reduce speed”
button and assumes random values
– Functional tests pass for model
– No redundancies in model (100% coverage achieved)
– Nominal signal and parameter values worked in simulation
Debug Options:
1. Create test to reach this Cal condition
2. Use static analysis tools to identify/ prove correctness
![Page 23: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/23.jpg)
23
Construct a model of field issue
Constrain inputs to represent field issue
Create model of field issue behavior
Ask tool to prove whether errant condition can occur
Using Model-Based Design to Reproduce Field Issue
![Page 24: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/24.jpg)
24
Field Issue Behavior Model
Target speed increases with “reduce speed” button
a) I set the target speed to the
vehicle speed (40 mph) while
going downhill on the track
b) I was pulsing the “reduce
speed” button until it decreased
the target speed to the 20 mph
limit
c) The next time I hit the “reduce
speed” button it increased the
target speed from 20 mph to 33
mph
![Page 25: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/25.jpg)
25
Demo: Reproducing Field Issues
Simulink Design Verifier
![Page 26: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/26.jpg)
26
Gaining Confidence in our Design
Effort / Time
Confidence
Ad-hoc
tests
Functional
& structural
tests
Design error
detection
Requirement
proofs
Code
integration
checks
Field tests
![Page 27: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/27.jpg)
27
Industry Examples of Finding Errors Early
Lear Body Control Electronics
– Found >95% of requirements issues before
implementation (compared to 30% prior)
DLR Autonomous Humanoid Robot
– Functional defects reduced by 80%
Weichai Common-Rail Diesel Engine
– Detect 60% - 70% bugs before integration
Airnamics Unmanned Aerial System
– Found 95% control software bugs before first flight
![Page 28: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/28.jpg)
28
MathWorks V&V Product Portfolio
Product Capabilities
Simulink Test Author, execute, and manage simulation-based
tests for models and generated code
Simulink Verification &
Validation
Trace to requirements, check model standards,
perform coverage analysis
Simulink Design
Verifier
Identify design errors, automatically generate test
vectors, verify designs against requirements
Polyspace Bug Finder Find software bugs and check compliance to
MISRA
Polyspace Code
Prover
Prove the absence of run-time errors in software
![Page 29: Increasing Design Confidence: Model and Code Verification...DLR Autonomous Humanoid Robot – Functional defects reduced by 80% ... Validation Trace to requirements, check model standards,](https://reader033.vdocument.in/reader033/viewer/2022050518/5fa1fe2860a9ac66716a4e5d/html5/thumbnails/29.jpg)
29
Thank You