Parasoft ADP solutions

Automated Defects PreventionAutomated Defects Preventionforfor

Embedded Systems Software DevelopmentEmbedded Systems Software Development

by Wiktor Grodowskiby Wiktor Grodowski

• Mostly focuses on Functional Testing tied back to Requirements

• Design – Code – “Integration and Test”

• Early Testing is minimal and short-lived

• Testing efforts are focused on the isolation and subsequent fixing of “bugs” – Error Detection

• “Failure to Communicate” between Development, Test and Quality Assurance (QA)

Observations on Software Testing

.. Not much has changed in 25 years ..

The Cost of Waiting

• ADP = Automated Defects Prevention

• Defects Prevention != Error Detection

• ADP = Defects Prevention + Automation

• ADP is not a replacement for CMM / CMMI and alike. It is a change in a software development approach.

• ADP is practical, flexible, down to earth, based on over 20 years of experience

• http://www.adpqb.org

What is ADP all about?

• ADP originates from Deming's and Jurand's works

• Attempting to introduce Deming's methodology into SDLC

• Follow the basic steps:•Identify a defect•Find its cause•Locate where and why it was introduced•Implement preventive methods•Monitor the process

• Start preventing errors instead of detecting them!

A brief history

Deming's wheel

Plan – establish objectives and processesDo – implement new processCheck – measure the results and compare against expected outcomeAct – analyze the differences, determine where to apply changes to include improvement

ADP principles

Principle 1: Establishment of Infrastructure Build a strong foundation through integration of people and

technology Principle 2: Application of General Best Practices

Learn from others’ mistakes MISRA, JSF, other coding standards

Principle 3: Customization of Best Practices Learn from your own mistakes

Principle 4: Measurement and Tracking of Project Status Understand the past and present to make decisions about

the future Principle 5: Automation

Let the computer do it! Principle 6: Incremental Implementation of ADP’s Practices

and Policies

Establishment of infrastructure

People – extending traditional roles Must promote communication, efficiency, productivity and

job satisfaction Each role must include defects prevention tasks

Understanding roles All must understand how to adhere to the roles they are

given Define group behavior

Technology – minimum and more SCMS Build system BTS Reporting system

Application of best practices

Most software projects share same characteristics Best practices can be used to limit:

Human errors Common software defects

Where do the best practices come from? Different granularity of best practices:

Organizational-level (EVMS – Earned Value Management System, general guidelines for use of CMS)

Design-level (suggestions for usage of specific technologies or techniques, such as design patterns, Ajax or SOA)

Code-construction level (procedures for peer programming, writing unit tests, code review etc.)

Customization of best practices

Must address project-specific problems Is based on the Deming's principles

Monitor process Identify a defect Find cause of a defect Locate part of a process that let the defect slip through Modify process to prevent the defect from reappearing Monitor process (again)

Static analysis tool should be implemented to monitor both practices automatically

Measurement and tracking

Measures must be quantitative Absolute values (number of defects) Metrics (possession of a given attribute, confidence factor)

Measurement: What and how can be measured? What does the measure or metric indicate? What decisions can be based on these information?

Tracking: What data can be tracked? How is the data tracked? What do we gain from tracking this data?

Automation

What is automation? „Activity that can be performed completely without human

intervention” “Automation of mundane and repetitive tasks improves people

satisfaction and effectiveness” Not to substitute people but to improve working conditions for

people Automation improves product quality Automation facilitates human communication Automation helps to implement and enforce best practices and

organizational standards Automation improves people productivity Automation helps to collect measurement data

Code reviews are time-intensive and human-intensive

Review process can be done with a “static analysis” tool

Quality focus is typically NOT built into product schedules so there is

typically little time to do proper testing.

Automation is required to leverage these precious cycles

Developers write more testing procedures/code than the code they are

developing for delivery!

ADP “automates” much of the driver, stubs and infrastructure

creation as well as provide metrics, like code coverage

Developers typically do NOT like testing.

Some problems will never be solved!

Why Automation is Important !

Incremental implementation of ADP

Continuous change through: Learning Adaptation Innovation

Change, however, can be overwhelming ADP needs to be implemented gradually, on a group-by-group

basis Pilot group(s) Division(s) Organization

Best practices should also be introduced one-by-one

Incremental implementation of ADP

ADP is both iterative and incremental – the phases do not have to follow sequentially

SCM

Source Control

code & tests

TCM

Team ConfigurationManager

Test Results

Scheduled Build and TestServer

Reporting

Program ManagerArchitect

Exemplary ADP Deployment

Professional Editions

Server Edition

Architect Edition

Nightly Results

DeveloperMachines

Architect / Technical lead

Company Standards

depl

oym

ent Target device

High costs of defects

Number of defects in a final product must be kept at minimal due to

a high recall cost

Possibility of having human life or human safety at stake,

dependent on a stability of an embedded device

Important time to market

Late product = lost money

Mandatory compliance

MISRA, JSF, DO-178B, others

Contractor's internal standards

Relation to Embedded Systems

Exemplary ROI

Unit testing caused drop in residual errors almost by a factor of 2.

Is it worth it?

Revenues ($)

Time (months)

On-time Delayedentry entry

Peak revenue

Peak revenue from delayed entry

Market rise Market fall

W 2W

Time

D

On-time

Delayed

Rev

enue

s ($

)

Is it really worth it?

ADP

Non-ADP

C++Test Coding Standards analysis Flow Analysis Unit Testing - Coverage analysis Code Review Authoring and Automation

Concerto Work Planning and Tracking Reporting and Analytics Acceptance testing Data correlation Process Visibility

Insure++

Runtime memory analysis

Parasoft Tools supporting ADP

XUnderstands and anticipates common coding mistakes that can lead to poor software quality

XUses testing as a process measurement technique

XXUses testing results to fix errors

XLearns from test results and improves processes and prevents future errors

XReduces probability of errors reoccurring

XAddresses the critical roles in creating software

XProvides tools and processes to keep team members code at same level

XUses testing results to fix the development process

XXFind Errors

ADPTestingBenefits

Benefits of ADP

Questions & Answerswww.parasoft-embedded.com