Toward Replacing Toward Replacing Structure-Based Testing Structure-Based Testing

with Analysiswith Analysis

Vdot SanthanamVdot Santhanam

BoeingBoeing

Structure-Based TestingStructure-Based Testing

DO-178B measures efficacy of software DO-178B measures efficacy of software verification through structural coverage verification through structural coverage measuresmeasures• Level C – Statement CoverageLevel C – Statement Coverage• Level B – Decision CoverageLevel B – Decision Coverage• Level A – Modified Condition/Decision CoverageLevel A – Modified Condition/Decision Coverage

Often this leads to significant amount of Often this leads to significant amount of time spent achieving structural coverage time spent achieving structural coverage for the sake of coveragefor the sake of coverage

Certification Certification

Contrary to EASA and MoD practices, the Contrary to EASA and MoD practices, the FAA has not embraced analysis as a FAA has not embraced analysis as a means of software verificationmeans of software verification• There are no requirements to perform even There are no requirements to perform even

simple forms of analysissimple forms of analysis• We’d like to see that culture change because it We’d like to see that culture change because it

willwill Reduce the cost of verificationReduce the cost of verification Enhance the reliability of the process Enhance the reliability of the process Improve our confidence in the softwareImprove our confidence in the software

• With DO-178C planning starting, the time is With DO-178C planning starting, the time is rightright

Where Do We Want To Go?Where Do We Want To Go?

Would like to see Would like to see • Varying degrees of analysis by Varying degrees of analysis by

assurance level, e.g.,assurance level, e.g., Set-use analysis for level CSet-use analysis for level C Range-constraint analysis for level BRange-constraint analysis for level B Exception and memory leak analysis; RMA Exception and memory leak analysis; RMA

for level Afor level A

• Alternate form of credit for verification-Alternate form of credit for verification-by-analysis (in lieu of structural by-analysis (in lieu of structural coverage testing)coverage testing)

Research NeededResearch Needed

Devise analysis techniques that achieve Devise analysis techniques that achieve the same/similar objectives as testingthe same/similar objectives as testing

Devise new, more powerful techniques Devise new, more powerful techniques that can surpass testing in cost-that can surpass testing in cost-effectivenesseffectiveness

Stream-line research toward more Stream-line research toward more pragmatic applications for existing formal pragmatic applications for existing formal techniques (rather than pursue the elusive techniques (rather than pursue the elusive goal of proving programs correct)goal of proving programs correct)