Collins Advanced Technology Center / Iowa State University / SEC
PM/PI: David Statezni, co-PI: Gary Daugherty, Rockwell CollinsSubcontract: Dr. Suresh Kothari, Iowa State University
Contract: F33615-00-C-1624
Advanced Computing Systems
Slide 2
Pattern driven analysis & adaptation of HA SW
Problem Description & Program Objectives
Technical Approach
Schedule
Milestones in next 6 months
Technology transition
Slide 3
Project web site
This presentation and other information is available from:
http://dirac.ee.iastate.edu/sec/index.htm
User Name: sec
Password: skgd5103
Slide 4
Problem Description*
Large body of complex software (Boeing OCP)
High assurance application (SEC)
Software is critical (faults may result in mission failure, loss of aircraft, equivalent to DO-178B level A)
Concern over a number of issues (83 in Issues List)
Complete manual review impractical for small team, esp. with respect to global issues
Testing is expensive, and does not cover all cases
Slide 5
Program Objectives*
Cost effective, automated approach (driven by static analysis)
Analyze/adapt OCP to establish confidence, eliminate problems, avoid potential faults in flight
Address the most important issues
Flexible with respect to definition of problems, specification of changes, ability to keep developer in the loop - based on user specified “Software Patterns”
Support for multiple modeling and programming languages (to amortize costs, address a broader user base)
Test cases generated from interface specs to meet MC/DC coverage criteria – for changed modules, "high risk” modules
Slide 6
Technical Approach
Catalog
of patterns for high assurance software
KCStoolset
(for translation, analysis, and adaptation)
Modeling tools
(GME, Rose, etc.)
OCP and other software source (in C++, Java, other languages)
HA SW
XML/XMI models
Patterns in XML/XMI
Adapted OCP, and other software source (in C++, Java, other languages.)
Models
SW
HA Models
Adapted XML/XMI models
Developer
Review/editIssues List
T-VEC
Regression testing
T-VEC formal specs
Slide 7
Issues List: Complete
Compiled from a variety of sources
Identifies 83 issues related to software for high assurance systems
Includes both design level and language level issues
And issues related to verification and testing List available from the Issues page of our web site
http://dirac.ee.iastate.edu/sec/HA-SW-Issues.htm
Slide 8
Issues List: Sources*
Compiled from a wide variety of sources, including: Handbook for Object-Oriented Technology in Aviation (OOTiA),
Proceedings of FAA/NASA Workshops 1 & 2 on Object-Oriented Technology in Aviation, 2003
Guide to the use of the Ada Programming Language in high integrity systems, ISO/IEC TR 15942, 2000
John Barnes. High Integrity Ada: The SPARK Approach, Addison-Wesley, Harlow, England, ISBN 0-201-17517-7, 1997
Use of the C++ Programming Language, Certification Authorities Software Team (CAST), position paper, CAST-8, January, 2002
David W. Binkley. C++ in Safety Critical Systems, NIST, November 1995
Boeing, BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801, 2002
Slide 9
Bertrand Meyer. “Applying design by contract.” IEEE Computer 25(10):40-51, October 1992.
Barbara Liskov and Jeanette Wing. “A Behavioral Notion of Subtyping”, ACM Transactions on Programming Languages and Systems, 16(6): 1811-1841, November 1994.
Scott Meyers. Effective C++, 2nd edition, Addison-Wesley, Reading, MA, 1998
Scott Meyers. More Effective C++, Addison-Wesley, Reading, MA, 1996
Marshall P. Cline and Greg A. Lamow. C++ FAQs, Addison-Wesley, Reading, MA, 1995.
Jeff Offutt, et al. A Fault Model for Subtype Inheritance and Polymorphism, Twelfth IEEE International Symposium on Software Reliability Engineering (ISSEE ’01), 2001
Robert V. Binder. Testing Object-Oriented Systems: Models, Patterns, and Tools, Addison-Wesley, Reading, MA, 2000.
Issues List: Sources*
Slide 10
XCIL: Complete*
eXtensible Common Intermediate Language (XCIL) Target language independent XML representation
Based on UML, with actions defined by Microsoft Intermediate Language (MS-IL) and the Java Virtual Machine (JVM)
Extended to include full C++
Precise semantics
Supports UML models and all .NET and JVM languages
Basis for integration with other tools
See http://dirac.ee.iastate.edu/sec/XCIL.htm
Slide 11
XPSL: Complete*
eXtensible Pattern Specification Language (XPSL) Patterns described in GOF-like format
But formal (XPSL) – in order to drive underlying tools
An extension of Aspect-Oriented concepts
But at the level of actions, with support for transformations (in addition to insert/replace)
And semantically defined pointcuts identified by analysis
Anticipates OMG Query/View/Transformation standard, which we plan to use as a foundation
See http://dirac.ee.iastate.edu/sec/XPSL.htm
Slide 12
Pattern catalogs: Complete
40 patterns (10 abstract, 30 concrete)
Addressing 26 of the 83 issues
Build on FAA/NASA, AVSI patterns defined by RC, Boeing, Honeywell, Goodrich and others
Few other high assurance software patterns available
Developed incrementally
Most important issues first
See http://dirac.ee.iastate.edu/sec/PatternCatalogs.htm
Slide 13
KCS toolset: To date
C++ to XCIL translator complete
Entire OCP (version 2.2) translated to XCIL
Pattern analysis/adaptation is also being developed incrementally
Initial focus on finding problems
Analysis support for 8 patterns, addressing 10 issues
Working on adaptation framework for two of the patterns
With more to follow, based on ranking of problems
Slide 14
Ranking of problems*
Hard run time error A fault may occur when the erroneous code is executed
Lurking run time error A hard run time error may result if additional correct code is
introduced
Obstacle to analysis (potential masking of existing errors) The code cannot be analyzed in some manner typically required
for “certification”
Maintenance risk (potential source of future errors) The code is correct, but subsequent changes are likely to introduce
errors
Slide 15
Analysis results: To date*
For the 8 patterns and 10 issues addressed so far: Matches on 9 of the 10 issues were found when the patterns
were applied to the OCP
Typically several hundred matches per issue
Problems range from maintenance risks to lurking and potential run time errors
A few of these are discussed in the following slides
Summary given by OCP Results link on home page of web site http://dirac.ee.iastate.edu/sec/HomePage.htm
Detailed results (with mapping to source code) given on OCP Results page
Slide 16
Pattern catalog
Increment one
Development of eXtensible Pattern Specification Language (XPSL/XML)
Development of patterns for:
Control flow,
Data integrity,
Pointer integrity,
Unintended side effects,
Multiple inheritance
Slide 17
Tool support
Increment one
Development of XCIL
Translation of C++ to XCIL,
Analysis framework (queries and views),
Analysis support for selected increment one patterns,
Focus on “local” patterns, affecting individual OCP components
Slide 18
Pattern: Safe Type Conversion
Addresses issue 4: Implicit type conversion Implicit type conversion may involve a potential loss of data or
precision, or semantics [OOTiA Handbook]
“Type conversions shall not be made from a longer to a shorter (number of bits) type. Type conversion shall not be made from an unsigned type to a signed type of the same size” - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801
Matches
No implicit conversions between integer and float
58,675 implicit conversions of a larger value to a smaller value
352 implicit conversions of a signed value to an unsigned value
87 implicit conversions of an unsigned value to a signed value
Matches represent potential run time errors
See OCP Results and Increment One Patterns for details
Slide 19
Adaptations
Subject to review
Need report on types converted from/to
Need report on writes to variables involved in conversions
Need to consider whether the actual range of values of the source type fits within the range of values permitted by the destination type
Need to consider whether the converted values are treated as numeric values (whether signed/unsigned matters)
Introduce user defined types or typedefs
Use bool instead of integer
Use single size, avoid conversions
Use explicit conversions
Introduce run-time range checks (not performed by C++)
Slide 20
Pattern: Compiler generated operations
Addresses issue 25: Compiler generated operations Compiler generated operations have default meanings that may be
inappropriate for a given class
Analysis results 1,451 classes with compiler generated default constructors
2,078 classes with compiler generated copy constructors
2,100 classes with compiler generated “=” (assignment) operators
2,218 classes with compiler generated “&” (address of) operators
Matches are subject to application of the pattern’s rules
Violations of these rules represent likely run time errors and lurking run time errors
Slide 21
Pattern: Compiler generated operations
Rules Law of The Big Three: “If a class needs a destructor, or a copy
constructor, or an assignment operator, it needs them all” … “During code reviews and debugging sessions, we have traced many core dumps back to violations of The Law” [Cline and Lomow, C++ FAQs]
“Declare a copy constructor and an assignment operator for classes with dynamically allocated memory” [Scott Meyers, Effective C++, 2nd edition, item 11]
Declare a constructor for all classes with an user defined invariant [OOTiA Handbook]
All fields without initial values must be set by a user defined constructor [OOTiA Handbook]
See OCP Results for details, associated adaptations
Slide 22
Adaptations
The user is responsible for introducing the necessary operations, while the tool is responsible for checking the user supplied code
In addition the tool may introduce run-times checks of constructor postconditions, to ensure the class invariant is established
Checks of user supplied constructor
Run time check of invariant as postcondition
Check that all fields are initialized
Checks of user supplied copy constructor
Run time check of invariant as postcondition
Check that all dynamically allocated components are replicated (deep copy)
Check of user supplied assignment operator
Check that all dynamically allocated components are replicated (deep copy)
Check of user supplied destructor
Check that all dynamically allocated components are deallocated
Slide 23
Pattern: No pointer arithmetic
Addresses issue 30: Use of pointer arithmetic “Operational software shall not include pointer arithmetic. In general, there
is little justification for going to the effort of designing code to do pointer arithmetic. The performance gained by doing pointer arithmetic is seldom worth the increased risk of error.” - Explicit pointer value calculation is prohibited for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801
Matches
470 instances of pointer arithmetic
Matches represent potential run time errors and obstacles to analysis
Rules A pointer can be used only as a substitute for an opaque reference
See OCP Results and Increment One Patterns for details/adaptations
Slide 24
Adaptations
Replace each array element pointer with an array base pointer and index, performing all arithmetic on indices rather than pointer values
Selectively introduce run-time bounds checks (typically not performed by C++ compilers)
Restrict other uses of pointers in XCIL to the cases given (in which the use of a pointer is compatible with the use of an opaque reference), requiring the user to make associated changes
Slide 25
Pattern: Goto-free execution
Addresses issue 19: Use of goto’s “The use of goto is regarded as both unnecessary, and undesirable
compared to the use of other control structures. It should not appear in well written code.” – SPARK
The use of goto should be excluded because it complicates flow analysis (FA), symbolic analysis (SA), timing analysis (TA), and structural testing (ST). - Guide for the Use of the Ada Programming Language in High Integrity Systems, ISO/IEC TR 15942
Matches 1,744 gotos
Matches represent obstacles to analysis
See OCP Results and Increment One Patterns for details/adaptations
Slide 26
Adaptations
Subject to review
Unless used as a substitute for exceptions, refactor code to:
Replace goto with structured control flow (conditional, loop, break, continue, error return, other common forms)
If used as a substitute for exceptions, apply exception/error handling patterns:
Enforcing the disciplined use of exceptions (per Bertrand Meyer)
Ensuring all reported errors (thrown exceptions) have handlers – as in Java
Eliminating macros in favor of pattern substitution, allowing a variety of error reporting/error handling strategies
Slide 27
Pattern: Fall through switch case
Addresses issue 17: Break in switch “Each case in a switch, which contains any executable code, shall be
terminated with a break. This eliminates the dubious practice of falling through a case to use code in a succeeding case for common processing. It is too easy to lose track of the correct program flow in code maintenance.” Mandatory for DO-178B software levels A through D - Boeing. BCA Technical Standard for the Use of C++ in Airborne Software, D6-82801
Matches 461 fall through cases
Matches represent maintenance risks
See OCP Results and Increment One Patterns for details/adaptations
Slide 28
Adaptations
Refactor code to either:
introduce a method called by both cases (when the shared code is cohesive)
introduce an inlined call (when the shared code is short and cohesive)
introduce a conditional (if statement or nested case) when the shared code is not cohesive
introduce replicated code (when the shared code is short but not cohesive)
Slide 29
Other increment one patterns
Non-discriminated union (Issue 47)
Implicit method return (Issue 56)
Loop termination (Issue 44)
Enforce invariant (Issue 13)
Closed argument list (Issue 15)
No dangling pointers to local variables
No assignment statement arguments (Issue 40)
No user thrown exceptions (Issue 49)
Top level handler for predefined exceptions (Issue 49)
Slide 30
Pattern catalog
Increment two
Development of patterns for:
Contracts and subtyping,
Coupling and cohesion,
OO metrics,
Dynamic resource allocation,
Safety critical language subsets
Slide 31
Tool support
Increment two
Adaptation framework (transformations),
Adaptation support for selected increment one patterns,
Analysis support for selected increment two patterns,
Focus on “global” patterns, requiring an analysis of the entire OCP
Slide 32
Pattern: Formal interfaces*
Addresses Issue 1: Lack of precise requirements/interface specifications “If we don’t state what a module should do, there is little likelihood that it will
do it (The law of excluded miracles)” – Bertrand Meyer
Programming by contract: “The relationship between a class and its clients [can be] viewed as a formal agreement [contract]. Only through such a precise definition of every module’s claims and responsibilities can we hope to attain a significant degree of trust in large software systems” [Bertrand Meyer, Object-oriented Software Construction]
MC/DC test coverage criteria [DO-178B], Unit level subtyping tests
Matches All class interfaces
See Increment Two Patterns for details, discussion of generation of pre/post from code
Slide 33
Pattern: Subtyping*
Addresses Issue 6: Supertype/subtype compatibility Issue 23: Use of dynamic dispatch Issue 5: Dynamic dispatch during object construction
In particular “The single largest source of design errors is improper inheritance” [C+
+ FAQs, Issue 119]
Errors may result from Faulty intuition Failure to implement to inherited specifications Misuse of inheritance to share code/data Programmer specified optimizations Accidental overriding/hiding Missing override
Slide 34
Pattern: Subtyping*
Analysis results 938 subtyping relationships
1,392 cases in which an operation is implemented by multiple methods (OMM pointcut)
3,112 instances of “true dynamic dispatch” (TDD pointcut)
Review results (sample of ~40 subtyping relationships) 24 violations of subtyping rules/LSP (including conflicting postconditions,
less visible subclass operations, disabled methods, null method implementations, and weaker postconditions) - overlapping
Violations represent either lurking or hard run time errors
See Increment Two Patterns and OCP Subtyping Relationships for details/adaptations
Slide 35
Pattern: Multiple implementation inheritance*
Addresses issue 8: Ambiguity resulting from the use of multiple implementation inheritance
Multiple inheritance session of OOTiA workshop unattended because no one thought MII appropriate for high assurance applications
Review results (sample of ~40 subtyping relationships)
4 instances of multiple implementation inheritance
Matches represent maintenance risks and potential run time errors
Rules Replace with delegation for DO-178B levels A through C
See OCP Results and Increment One Patterns for details/adaptations
Slide 36
Other increment two patterns*
Safety critical language subsets (“Safe” C++?) (Issue 71)
Synchronization (Issue 3)
Multiple interface inheritance (Issue 7)
Replacement of multiple implementation inheritance with delegation (Issue 8)
No dynamic allocation after initialization (Issue 9)
Initialization before use (Issue 12)
Class coupling (Issues 13, 67, 68)
Dynamic dispatch test coverage
Method extension
Deep hierarchy
Slide 37
Regression/subtype testing*
Formal interface specifications (pre/post) are being developed for all classes we expect to change
And for all classes involved in subtype relationships that require LSP compliance testing
Test cases are generated from these formal interface specs by T-VEC
This approach helps us address issue 1: Lack of precise requirements/interface specifications
As well as providing a basis for regression testing
With MC/DC coverage
Slide 38
Lessons learned*
Tool support has been invaluable in identifying potential problems Much more effective than reviews in terms of hours spent,
accuracy, and counts of problems found
Valuable as a driver/assistant in the review process
Necessary due to size of OCP
Need to be able to interactively refine problem description, tune patterns
Slide 39
Schedule
Slide 40
Next Milestones*
Regression testing of 1st pass adapted software (pointer arithmetic, type conversion, etc.) 6/27/2003
Complete work on analysis and adaptation (strategy = most important first) 7/31/2003
Regression testing of 2nd pass adapted software 8/29/2003
Final Report 12/19/2003
Slide 41
Technology Transition*
Web site has latest versions of the Issues list, XCIL papers, XPSL papers, Pattern catalogs, and the OCP analysis results
Reports identifying problems found (by both reviews and the toolset) will be posted to the web site
Change log + T-VEC interface specs, test cases, and test results for changed modules will be provided
Revised OCP code will be posted when all regression testing is complete
The tools developed by ISU and the contents of the web site will be distributed on CD
The KCS tools will be made commercially available through Ensoft Corporation at ISU Research Park
Slide 42
Program Issues
Maximize benefit in time remaining Extended subcontract with ISU to end of July
Space and time required to run KCS analysis and adaptations Faster machine, parallelization, incremental
Use of the ISU toolset by others requires licensing of EDG parser (from Edison Software) University agreement with Edison (similar to ISU)
Licensing from Ensoft (as part of XCIL translator for C++)