adacore paris tech day 2016: eric perlade - verification solutions

Eric Perlade

Verification Solutions

Technical Account Manager

C/C++ C#/Java Ada

RobustnessAchieved

DeveloperResponsibility

ToolResponsibility

LanguageResponsibility

Assembly

Software that matters

Leverage the increase in tool responsibility offered by Ada and SPARK

Find software bugs and vulnerabilities earlier in the development process

Ideally we’d like the developers to do this as part of their every day workflow

Verification can be made easyAda

Verification

Assure that software fully satisfies all the expected requirements

Non-FunctionalRequirements

FunctionalRequirements

Functional Requirements

A refinement from System Requirements -> Software Requirements

”The software shall…”

High Level and Low Level – DO-178

Require verification

FunctionalRequirements

Non-Functional Requirements

Can be from outside the SRD refinement path

Coding Standards

Complexity Requirements

Coverage Requirements

Target resource usage Requirements

Non-FunctionalRequirements

SoftwareRequirementsSpecification

SoftwareArchitecture

SoftwareDetailedDesign

Code

UnitTest

IntegrationTest

SystemTest

GNATmetricGNATcheckSPARKProCodePeer

GNATtestGNATemulatorGNATcoverage

GNATstackGNATcoverage(TargetTrace)

Know Your Code

• Embrace Programming by Contract• Continuous Unit Testing• Measure your software• Keep your subprograms at a sensible length and level of

complexity• Ensure your software is readable and understandable by all the

developers

Static Verification

GNATmetric

Variety of different metrics are available

• Complexity• Syntax Elements• Line Metrics• Coupling Metrics

Output into an XML file which can be post-processed

GNAT Front End

Objective is to leverage the increased formality of the language to allow tools to be more responsible.

Compiler switches for the GNAT front end can do a great job of catching coding problems early in the development process

Many customers turn on ALL warnings and ensure warnings are treated as ERRORS which will halt compilation

Produces better developers ??

Style Checking

The key word here is consistency

Developers can unintentionally introduce code constructs that are considered un-helpful in reducing defects

General guidelines for good Ada can accompany company coding standards

GNATcheck and the GNAT front end itself are available to help

GNATcheck

A set of implemented rules from which a subset can be selected as a coding standard

Specified in a file read by GNATcheck and the output can be written to a textual report file or an XML file for post-processing

GNAT Programming Studio has a fantastic GUI helper for rule selection

Static Stack Analysis

Generation of the basic stack consumption and call-graph information.

Performed during compilation-fcallgraph-info=su,daGenerates a .ci file per object file

Analysis and report generation

Pitfalls

Cycles, unbounded frames, external calls, dynamic variables or indirect calls

Use “-Wa” and “-v” to get additional information

Encourage developers to run GNATstack and try to rework the code to enable full stack analysis

Build your own tools !

libadalang

Dynamic Verification

SoftwareRequirementsSpecification

SoftwareArchitecture

SoftwareDetailedDesign

Code

GNATtestGNATemulatorGNATcoverage

SystemTest

Integration Test

Desktop Dynamic Verification

TARGETEMULATIONGNATemulator COVERAGEANALYSISGNATcoverage

UNITTESTGENERATIONGNATtest

HARNESS

TESTCASES

UUT

package Simple is

procedure SubP(Loop_Count : in Integer;Even_Count : out Integer;Odd_Count : out Integer

) withPre => (Loop_Count > 0),Post => ((Even_Count >= 0) and (Odd_Count >= 0));

end Simple;

GNATtest

Unit Test Harness Generation

Included in GNAT Pro

Based on the Open Source AUnit Framework

Available for Native and Cross Compilers

Code Generation = Lower Costs

GNATtest Work Flow

gnattest -v --RTS=ravenscar-full-prep -Pex4.gpr

Generates Test Case Skeletons and a Harness

Very Flexible and allows for CM of generated code

Aware of User Defined Test Case Code

Clearly identifies what will NOT survive a re-generation

GNATtest –Test Case Generationpackage Simple.Test_Data is

-- begin read onlytype Test is new AUnit.Test_Fixtures.Test_Fixture

-- end read onlywith null record;

procedure Set_Up (Gnattest_T : in out Test);procedure Tear_Down (Gnattest_T : in out Test);

end Simple.Test_Data;

-- This package has been generated automatically by GNATtest.-- Do not edit any part of it, see GNATtest documentation for more details.

-- begin read onlywith Gnattest_Generated;

package Simple.Test_Data.Tests is

type Test is new GNATtest_Generated.GNATtest_Standard.Simple.Test_Data.Testwith null record;

procedure Test_SubP_75ecda (Gnattest_T : in out Test);-- simple.ads:3:4:SubP

end Simple.Test_Data.Tests;-- end read only

GNATtest –Test Case Generation

-- This package is intended to set up and tear down the test environment.-- Once created by GNATtest, this package will never be overwritten-- automatically. Contents of this package can be modified in any way-- except for sections surrounded by a 'read only' marker.

package body Simple.Test_Data is

X : constant Integer := 20;

procedure Set_Up (Gnattest_T : in out Test) ispragma Unreferenced (Gnattest_T);

beginnull;

end Set_Up;

procedure Tear_Down (Gnattest_T : in out Test) ispragma Unreferenced (Gnattest_T);

beginnull;

end Tear_Down;

end Simple.Test_Data;

-- This package has been generated automatically by GNATtest.-- You are allowed to add your code to the bodies of test routines.-- Such changes will be kept during further regeneration of this file.-- All code placed outside of test routine bodies will be lost. The-- code intended to set up and tear down the test environment should be-- placed into Simple.Test_Data.

with AUnit.Assertions; use AUnit.Assertions;

package body Simple.Test_Data.Tests is-- begin read only

procedure Test_SubP (Gnattest_T : in out Test);procedure Test_SubP_75ecda (Gnattest_T : in out Test) renames Test_SubP;

-- id:2.2/75ecda11d3241da6/SubP/1/0/procedure Test_SubP (Gnattest_T : in out Test) is-- simple.ads:3:4:SubP

-- end read only

pragma Unreferenced (Gnattest_T);

begin

AUnit.Assertions.Assert(Gnattest_Generated.Default_Assert_Value,"Test not implemented.");

-- begin read onlyend Test_SubP;

-- end read only

end Simple.Test_Data.Tests;

Even_Count, Odd_Count : Integer;

begin

Simple.SubP(Loop_Count => 21,Even_Count => Even_Count,Odd_Count => Odd_Count

);

Assert(((Even_Count = 10) and (Odd_Count = 10)),"Loop_Count => 21");-- White box analysis identified that no odd numbers above 20 will be counted

Simple.SubP(Loop_Count => 22,Even_Count => Even_Count,Odd_Count => Odd_Count

);

Assert(((Even_Count = 11) and (Odd_Count = 10)),"Loop_Count => 22");

GNATtest – Coverage Analysis

GNATemulator

QEMU Open Source Processor EmulatorNot a Simulator

I/O Connections

No InstrumentationActual Target Object CodeNo expensive native re-host

PowerPC/ARM/SPARCExecutable Image

PowerPC/ARM/SPARC -> x86 Translation

x86 Host Platform

GNATemulator Platform Support

VxWorks 6VxWorks 653

PowerPC ELF Bare Metal

LEON ELF Bare Metal

ARM ELF Bare Metal

GNATemulator Debugging

Puts a verification environment on the developers desk

Rapid re-test

No target hardware

Actual object code EXE

GNATemulator

GDBSERVER

TCP

GPS/GDB

GNATcoverage

• Coverage Analysis - Multiple Modes of Operation

• Run and Capture Execution Trace Data

• Conversion of IEEE-ISTO 5001-2003 (Nexus) Trace Data

• Coverage Analysis of Execution Trace Data• Source Code Level• Object Code Level

Native Intel

PowerPC

LEON 2 and LEON 3

ARM(Target Trace Port)

GNATcoverage Platform Support

Build Considerations

-gDebug data

-fpreserve-control-flowControl Optimizers for precise SLOC info

-fdump-scosSource Coverage Obligation in *.ali files

Build Considerations

Support for Optimizations (up to -O1)

Inlining Allowed (-gnatn)

No External Libraries Needed

All can be achieved using GNAT Project file scenario variables.

GNATcoverage Analysis Levels

Source Levelgnatcov coverage --level=stmtgnatcov coverage --level=stmt+decisiongnatcov coverage --level=stmt+mcdc

Object Levelgnatcov coverage --level=insngnatcov coverage --level=branch

GNATcoverage Analysis Formats

xcov - Annotated Sources in Text Format

report - Textual Summary

HTML - Colours, Sortable Columns and Per-project indexes

Conclusion

AdaCore has verification tools that are applicable to the different phases in the classic V-model.

Developers can leverage the increase in tool responsibility to continuously verify and catch defects early.

Easy to integrate into the developers work environment through the GPS IDE but also command line driven for non-interactive use cases.

Advance through these slides for more instruction

Getting Started

This template has been setup with a variety of slide layouts to give you a solid foundation that you can build on and adapt as necessary. Get started by clicking the Insert tab and selecting the New Slide dropdown to choose from a variety of slide designs. Each design is available in Dark, Blue, and Light background themes.

Resources

A repository of AdaCore acquired Stock Photography, product screenshots, logos, and other assets can be found at dropboxat the following link : http://bit.ly/1SMRRXT

Please note that any stock photography may be used for the purposes of company PowerPoint presentations. For any other use case, please get in touch with us first at design@adacore.com

The following icons have been useful for illustrating various industries. Visit http://www.thenounproject.com for a good

resource beyond what’s available here.

Rail Naval Drone Medical Financial

Security Auto Space Air ATM Submarine

Grid

The following icons have been useful for illustrating various industries. Visit http://www.thenounproject.com for a good

resource beyond what’s available here.

Rail Naval Drone Medical Financial

Security Auto Space Air ATM Submarine

Grid