parasoft .test, write better c# code using data flow analysis

Parasoft .TEST Write better C# Code Using Data Flow Analysis www.parasoft.com

Data Flow Analysis is not Dynamic testing! So What is Dynamic testing?

Dynamic testing (or dynamic analysis) is a term used in software engineering to describe the testing of the dynamic behavior of code.

That is, dynamic analysis refers to the examination of the physical response from the system to variables that are not constant and change with time. In dynamic testing the software must actually be compiled and run; Actually Dynamic Testing involves working with the software, giving input values and checking if the output is as expected.

Dynamic tools , Redgate Ants profiler, dotTrace, Boundschecker, Glowcode etc…

Dynamic Testing – Disadvantages

Dynamic testing can take place only after compilation

and linking. It may involve running several test cases

each of which may take longer than compilation.

It finds bugs only in parts of the code that are actually

executed.

Furthermore such testing often touches less

than half the code.

Data Flow Analysis is Not Regular Static Code Analysis that we usually use

What is Static Code Analysis?

Static Code Analysis tools , Reshaper, FxCop, CodeRush Static analysis, also called static code analysis, is a method

of computer program debugging that is done by examining the code without executing the program. The process provides an understanding of the code structure, and can help to ensure that the code adheres to industry standards. Automated tools can assist programmers and developers in carrying out static analysis.

Static Code Analysis is divided into 2 methods Pattern matching Data Flow Analysis

Pattern matching example

Avoid using the unary + operator

class UnaryPlus { int _a = 0; int _b;

public int DoSomething(int x) { _b = +x; // VIOLATION - the user meant _b += x /*...*/ } }

So What is it Data Flow Analysis?

Data-flow analysis is a technique for gathering information about the possible set of values calculated at various points in a computer program. A program's control flow graph (CFG) is used to determine those parts of a program to which a particular value assigned to a variable might propagate. The information gathered is often used by compilers when optimizing a program. A canonical example of a data-flow analysis is reaching definitions.

Why Do we need Data Flow Analysis when Analyzing C# Code?

Many defects may arise due to interactions among different methods and classes, and also depend on the actual path of execution.

These defects do not get covered by pattern-based static analysis.

Moreover, many of these defects are difficult to uncover via testing because many

of the exceptional conditions are hard to reproduce. Even with 100% statement coverage, there will be many paths that do not get covered.

Thus, it is helpful to have an automated tool that simulates a large number of paths through the code, looking for potential defects.

C++test - Static analysis

Bug DetectiveArray out of boundaries detection using Reaching Definition

d1 i = 0d2 j = nd3 k = a[i]

B1

B2

B3

B4

d4 i = i + 1d5 j = j - 1

d6 i = 0

d7 k = a[i]

DEDef = 4,5DEFKill = 1,2,7

DEDef = 1,2,3DEFKill = 4,5,6,7

DEDef = 7DEFKill = 3DEDef = 6

DEFKill = 1,4

int a[10]

8From Course 236800 Technion - Parasoft® C++test by Alon Bialik

Data Flow Analysis Vs Pattern matching

Avoid unreachable code in condition

This defects can be found with simple Pattern Matching tool public class MySimpleConditions

{

public const bool CONST_VALUE = true;

public void SimpleMethod(string path)

{

const bool localConst = false;

if((CONST_VALUE && localConst) || IsValid(path)) // validation, it will always be false

{

return true;

}

else

{

return false;

}

}

public bool IsValid(string path)

{ ... }

}

Data Flow Analysis Vs Pattern matching

class Example

{

//This example triggers a violation due to the presence of a dead code fragment

// This need A Data Flow Analysis capabilities to be discovered

public static void checkRange(char ch)

{

if ((ch < '0') || (ch > '9'))

{

throw new ArgumentException("Only digits are permitted", "ch");

}

// obviously dead code

if ((ch >= 'a') && (ch <= 'f')) // VIOLATION

{

processHexValue(ch);

}

}

public static void processHexValue(char ch)

{

// some code

}

}

Real life …

The Problem / Symptom Application passed development and most QA

testing Problem found during late stages or after

shipping Application crashes or hangs sometimes – not

easy to reproduce

Real life …

The Bug Exceptions are thrown from event handlers – This bug is

from .TEST code base! public delegate void ChosenItemChanged( IChooserItem selectedItem); public event ChosenItemChanged ChosenItemChanged;

private void lowerListSelectionChanged( object sender, System.EventArgs e)

{ if (ChosenItemChanged != null) {

ChosenItemChanged(SelectedItem); }

}

Real life …

How could I have prevented this? Implement a best practice to ensure all event-raising methods are enclosed in a

try/catch block Observed big improvement in stability after enforcing this

private void lowerListSelectionChanged(object sender, System.EventArgs e)

{

if (ChosenItemChanged != null)

{

try

{

ChosenItemChanged(SelectedItem);

}

catch (Exception ex)

{

LOGGER.error("Failed to invoke ChosenItemChanged.", ex);

}

}

}

.TEST – Multi-pronged approach

• Enforcement of best

practices

• Static detection of run-time

errors

• Advanced testing features

• Code review

• Proven workflow for regular

runs

What is .TEST?

• Enforcement of best practices

• Static detection of run-time errors


• Code review

• Proven workflow for regular runs

• Identifies code implementation flaws early in a non-intrusive way

• Enforces coding patterns to prevent bugs

• Makes code easier to maintain

• Scales to support the group, project, division, as well as corporate standards

• Supports all “stages” of development

• Educates developers on potential pitfalls

• Computes important software metrics – points out code that is hard to maintain

What is .TEST?


• Static detection of run-time

errors


• Code review


BugDetective finds run-time errors without executing code

Good at catching errors in exceptional situations – these are missed by normal testing

Displays potential execution path leading to error – easy to understand and fix

What is .TEST?




– Test execution

– Tracer

– Test generation

• Code review


• Unit Test execution:– Robust execution engine

addressing practical issues

– Automatic updating of assertions when code changes

– User level stubs help in automating tests and enable testing of complex classes

– Run tests from within the actual application – makes tests more realistic and maintainable

– Code coverage views that can be used as a guide to improving the tests

– Flexible data sources support

What is .TEST?




– Test execution

– Tracer

– Test generation

• Code review


• What?– High quality tests that are

maintainable

– Insight into how the classes get used

– Realistic values and calling sequences

• How?– Run your app and collect info

on important method calls

– View the events in a friendly GUI and gain insight

– Select calling sequence for tests

What is .TEST?




– Test execution

– Tracer

– Test generation

• Code review


• Automatically generate simple regression suite

• All tests are in NUnit format allowing easy modification

• Snapshot of current behavior

• Improve the test harness over time

What is .TEST?

• Enforcement of Best Practices


• Automated Test Case Generation

• Code Review

• Proven Workflow for Regular Runs

• Best way to find complicated logic errors

• Organizes code review tasks– All check-ins get reviewed

• Creates healthy interaction among developers

• Works in pre-commit and post-commit modes

What is .TEST?

• Enforcement of Best Practices


• Automated Test Case Generation

• Code Review

• Proven Workflow for Regular Runs

• Easily configured for nightly builds

• Easy to share rules and configurations

• Import violations from nightly runs into Visual Studio

• Click of button runs tests or static analysis

• Maintains quality of your code as it evolves

• High quality customizable reports that help in record keeping and standards compliance

• Going live now….

Team-based Workflow

CVS / VSS / ClearCase

code & tests

Concerto

Team Server

Scheduled TestServer

Global Reporting System

Visual Studio

.TEST CLI(Batch Mode)

Visual Studio Team Practices

DeveloperMachines

Architect / Technical lead

TestResults

Test Results

Now we moving to live demonstration

Developer Workflow

1. Developer … before check-in Scan and clean code before check-in

2. Automated .TEST command-line Typically overnight Code analysis + Unit testing (execution) Data sent to GRS (optionally) Tasks uploaded to TCM for developers

3. Developer … next day Open Project in Visual Studio Download results from the TCM Fix problems before developing new code

Summery:

Prevention is always cheaper than cure If you are using neither static nor dynamic test

tools, static tools offer greater marginal benefits. Static testing achieves 100% statement coverage

in a relatively short time Typically dynamic testing takes longer than static testing yet

finds fewer bugs. Even if you achieve 100% statement coverage with Dynamic

testing it doesn’t mean that you have 100% path Coverage

If timescales are tight, use of dynamic testing tools might be omitted, but tool-supported static testing should never be omitted.

How to get an Evaluation Copy?

Contact me 09-8855803 [email protected] Or go to:

http://www.parasoft.com/jsp/products/dottest.jsp

לייזרוביץ – דניאל ההקשבה על 'תודה

parasoft .test, write better c# code using data flow analysis

Technology