1© 2015 The MathWorks, Inc.
Automating Best Practices to
Improve Design Quality
Daniel Martins
2
Poor Requirements Management
Sources: Christopher Lindquist, Fixing the Requirements Mess, CIO Magazine, Nov 2005
Why do 71% of Embedded Projects Fail?
3
Key Takeaways
▪ Author, manage requirements in Simulink
▪ Early verification to find defects sooner
▪ Automate manual verification tasks
▪ Workflow that conforms to safety standards
4
Requirements
Challenge with Traditional Development Process
Specification C/C++
Hand code
5
Simulink Models for Specification
Requirements C/C++Executable
Specification
Hand code
6
Complete Model Based Design
Code
Generation
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
7
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Model Based Design Verification Workflow
Component
and system
testing
Equivalence
testing
Equivalence
checking
Review and
static analysis
8
Challenges with Requirements
Where are
requirements
implemented?
How are
they tested?
Is design and
requirements
consistent?
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Component
and system
testing
9
Gap Between Requirements and Design
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
10
Simulink Requirements
Author
Track Manage
11
Import
Import Requirements from External Sources
IBM Rational DOORS
Simulink Requirements EditorMicrosoft Word
12
Link Requirements, Designs and Tests
Verified
By
Test Case
x
REQ 3.1 ENABLING CRUISE CONTROL
Cruise control is enabled
when …..
ENABLE SWITCH DETECTION
If the Enable switch is
pressed ……
Implemented
By
Derives
13
Track Implementation and Verification
Passed
Failed
No Result
Missing
Verification Status
Implemented
Justified
Implementation Status
Missing
14
Respond to Change
If the switch is pressed and the counter reaches 50then it shall be recognized as a long press of the switch.
If the switch is pressed and the counter reaches 75then it shall be recognized as a long press of the switch.
ImplementsOriginal Requirement
Updated Requirement
15
Verify Design to Guidelines and Standards
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Is the design
built right?
Is it too
complex?
Is it ready
for code
generation?Review and
static analysis
16
Automate verification with static analysis
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Check for:
• Readability and Semantics
• Performance and Efficiency
• Clones
• And more……Model Advisor Analysis
17
Generate reports for reviews and documentation
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Model Advisor Analysis Model Advisor Reports
18
Navigate to Problematic Blocks
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
19
Guidance Provided to Address Issues or Automatically Correct
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
20
Built in checks for industry standards and guidelines
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
• DO-178/DO-331
• ISO 26262
• IEC 61508
• IEC 62304
• EN 50128
• MISRA C:2012
• CERT C, CWE, ISO/IEC TS 17961
• MAAB (MathWorks Automotive Advisory Board)
• JMAAB (Japan MATLAB Automotive Advisory Board)
21
Configure and customize analysis
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
22
Static
Analysis
Checks for standards and guidelines are often performed late
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Rework
23
Static
Analysis
Edit-Time
Checking
Shift Verification Earlier With Edit-Time Checking
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
• Highlight violations as you edit
• Fix issues earlier
• Avoid rework
25
Assess Quality with Metrics Dashboard
• Consolidated view of
metrics
• Size
• Compliance
• Complexity
• Identify where problem
areas may be
26
Grid Visualization for Metrics
▪ Visualize Standards
Check Compliance
– Find Issues
– Identify patterns
– See hot spots
Red: Fail
Orange: Warning
Green: Pass
Gray: Not run
Legend:
27
Detect Design Errors with Formal Methods
▪ Find run-time design errors:• Integer overflow
• Dead Logic
• Division by zero
• Array out-of-bounds
• Range violations
▪ Generate counter example to reproduce error
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
28
Prove That Design Meets Requirements
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
▪ Prove design properties using formal requirement models
▪ Model functional and safety requirements
▪ Generates counter example for analysis and debugging
29
Functional Testing
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Does the
design meet
requirements?
Is it functioning
correctly?
Is it
completely
tested?
30
Test Case
Main Model
Systematic Functional Testing
AssessmentsInputs
Test Sequence
Signal Builder
MAT file (input) MAT file (baseline)
Test Assessment
MATLAB Unit Test
Excel file (input) Excel file (baseline)
Test Harness
31
Manage Testing and Test Results
32
Coverage Analysis to Measure Testing
Simulink• Identify testing gaps
• Missing requirements
• Unintended Functionality
• Design Errors
Stateflow
Generated Code
Coverage Reports
33
Test Case Generation for Functional Testing
▪ Specify functional test
objectives– Define custom objectives that signals
must satisfy in test cases
▪ Specify functional test
conditions– Define constraints on signal values to
constrain test generator
Test Condition
Test Objective
Test Objective
34
C/C++
Static Code Analysis
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Is the code
compliant
to MISRA?
Is integrated
code free of
run-time
errors?
Other code
Is interface between
generated and other
code fully tested?
The Generated Code is integrated
with Other Code (Handwritten)
35
Static Code Analysis with Polyspace
▪ Code metrics and standards
– Comment density, cyclomatic complexity,…
– MISRA and Cybersecurity standards
– Support for DO-178, ISO 26262, ….
▪ Bug finding and code proving
– Check data and control flow of software
– Detect bugs and security vulnerabilities
– Prove absence of runtime errors
Results from Polyspace Code Prover
36
Equivalence Testing
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Is the code
functionally
equivalent to
model?
Is all the
code tested?
Equivalence
checking
Equivalence
testing
37
Equivalence Testing
▪ Processor in the Loop (PIL)
– Numerical equivalence, model to target code
– Execute on target board
▪ Re-use tests developed for model to test code
▪ Collect code coverage
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Target
Board
▪ Software in the Loop (SIL)
– Show functional equivalence, model to code
– Execute on desktop / laptop computer
Desktop
Computer
PIL
SIL
38
Qualify tools with IEC Certification Kit and DO Qualification Kit
▪ Qualify code generation and verification products
▪ Includes documentation, test cases and procedures
BAE Systems Delivers DO-178B Level A Flight
Software on Schedule with Model-Based Design
KOSTAL Asia R&D Center Receives ISO 26262
ASIL D Certification for Automotive Software
Developed with Model-Based Design
39
Lear Delivers Quality Body Control Electronics Faster Using
Model-Based Design
ChallengeDesign, verify, and implement high-quality automotive
body control electronics
Solution
Use Model-Based Design to enable early and
continuous verification via simulation, SIL, and HIL
testing
Results
▪ Requirements validated early. Over 95% of
issues fixed before implementation, versus 30%
previously
▪ Development time cut by 40%. 700,000 lines of
code generated and test cases reused
throughout the development cycle
▪ Zero warranty issues reported
“We adopted Model-Based Design not only to deliver better-
quality systems faster, but because we believe it is a smart
choice. Recently we won a project that several of our
competitors declined to bid on because of its tight time
constraints. Using Model-Based Design, we met the original
delivery date with no problem."
- Jason Bauman, Lear Corporation
Link to user story
Lear automotive body electronic control
unit.
40
Customer References and Applications
LS Automotive Reduces Development Time for Automotive Component Software with Model-Based Design
Specification errors detected early
Continental Develops Electronically Controlled Air Suspension for Heavy-Duty Trucks
Verification time cut by up to 50 percent
Airbus Helicopters Accelerates Development of DO-178B Certified Software with Model-Based Design
Software testing time cut by two-thirds
More User Stories: www.mathworks.com/company/user_stories.html
41
Summary
1. Author and manage requirements within Simulink
2. Find defects earlier
3. Automate manual verification tasks
4. Reference workflow that conforms to safety standards
RequirementsExecutable
Specification
Model used for
production code
generation
Simulink Models
C/C++
Generated code
Component
and system
testing
Equivalence
testing
Equivalence
checking
Review and
static analysis