Software Testing

Prepared by:Raghav

Target Audience

Beginners Intermediate

What do we cover?

Basics of testing Levels of Testing Testing techniques

Software testing

25

50

75

100

125

Relative costper defect

Req Design Code Test Production

Software testing

Points to Ponder: 80% of all defects occur in 20% of the work Test strategies impact test costs Conversion of testing from craft to process can

give extensive benefits

Software Testing

What hinders effective testing?

Aspects that hinder Effective testing

Optimism Belief that the system works Negative attitude towards effective testing Conflict between testers and developers Testing is expensive Delivery commitments

The V Model

What is V model?

The V ModelThe V Modelvalid

ation

Customer Requirements

Software Requirements

Architecture (HLD)

Detailed Design (LLD)

Acceptance

System testing

Integration testing

Unit testing

Code

Testing v/s Life Cycle

Stage Test Activity

Requirements Acceptance Test ,System Test

Design (HL) Integration Test

Design (LL) Unit Test

Code Unit test / Debug

Integration Integration test / debug

System test / debug

Acceptance Acceptance test / debug

Why Test?

Developers not fallible Bugs in compilers, languages, DBs, OS Certain bugs easier to find in testing Don’t want customers to find bugs Post release debugging is expensive

Test team

Why separate testing team?

Is Testing everything?

There are other approaches possible to create good software

Effective methods are•Inspection•Design style•Static analysis•Language checks•Development environment

Can Testing be replaced?

NO!! Even if we use other methods, cannot do away

with testing Review, inspect, read, walkthrough, better

methodologiesAnd then TEST

TESTING

Definition: Testing is the process of exercising or evaluating a system or system component by manual or automated means to verify that it satisfies specified requirements. (IEEE 83a)

Testing objective

Process of executing a program with an intent of finding an error

A good test case is one that has a high probability of finding an as yet undiscovered error

A successful test is one that uncovers an as yet undiscovered error

Testing Involves

•Test plan•Generate test conditions, cases•Create required test environment•Execute tests by

Initiating application under testApplying inputs that are specified in the already generated test case

Compare actual results with expected resultsResults: Test passed or failed!

A ‘FAILED’ Test

The term ‘Test Failed’ is used when the program has failed to give the expected result

Actually, the test has succeeded in discovering a fault!

Definitions

•Defect

A deviation from specification or standard

Anything that causes customer dissatisfaction

Verification

All QC activities throughout the life cycle that ensure that interim deliverables meet their input specification

Validation

The ‘test phase’ of the life cycle which assures that the end product meets the user’s needs

Debugging

Debugging is the act of attempting to

determine the cause of the symptoms

of malfunctions detected by testing or

by frenzied user complaints. (Beizer)

Testing: static and dynamic

• Static Testing: Verification performed without executing the system’s code

•Code Inspection•Reviews

• Dynamic Testing: Verification or validation performed by executing the system’s code

Testing Techniques

• Black Box• White Box• Basis path testing

Black box testing

What is black box testing?

Black box testing

Also called Behavioural testing, focuses on functional requirement of the software

Black box testing

Detects following categories of errors

Incorrect or missing functions Interface errors Errors in data structures and external database

access Behaviour or performance errors Initialisation and termination errors

Types of Black box testing

Boundary value analysisEquivalence partitioningSpecial value

BVA(Boundary Value Analysis)

What is BVA?

Boundary value analysis (two variables)Boundary value analysis (two variables)

a<=x1 <=b

c<=x2 <=d

X2

X1

d

c

a b

BVA

Great number of errors tends to occur at the boundaries of the input domain rather than in the ‘center’

BVA leads to a selection of test cases that exercise bounding values

Equivalence partitioning

What is equivalence partitioning?

Equivalence partitioning testingEquivalence partitioning testing •EC partitions the data to partition

of a set• Partition refers to collection of

mutually disjoint subsets whose

union is the entire set.•Choose one data element from

each partitioned set.• EC based testing allows

- to have a sense of complete

testing.

- helps avoid redundancy• The KEY is the choice of

equivalence relation.

Special value

What is special value testing?

Special value testingSpecial value testing

• Select test data on the basis of features of a function to be computed

• Tester uses her/his domain knowledge, experience with similar programs

• Ad-hoc/seat of pants/skirt testing

• No guidelines, use best engineering judgment

• Special test cases/Error guessing

• Is useful – don't discount effectiveness !

Levels of testingLevels of testing

What does each level of testing uncover?

• Unit testing– implementation deficiencies– Code / LLD

• Integration testing– interface deficiencies– architecture / HLD

• System testing– requirement deficiencies– SRS

• Regression testing– fix has not broken existing

functionality - product / code

• Alpha / Beta testing• Acceptance testing

– customer expectations– notion of stability (time)

• Certification testing – external compliance– third party

Unit Testing

What is Unit testing?

Unit Testing

Unit- Smallest testable piece of software A unit can be compiled / assembled / linked /

loaded; and put under a test harness Unit testing is done to show that the unit does not

satisfy the functional specification and/or its implemented structure does not match the intended design structure

Integration Testing

What is integration testing?

Integration Testing

Integration is the process of aggregating components to create larger components.

Integration testing done to show that even though components were individually satisfactory, the combination is incorrect or inconsistent

Integration problems

What are the general integration problems?

Types of Integration problems

Configuration / version control I/O format, protocol mismatches Conflicting data views/ usage Data integrity violated Wrong parameters

Integration test approachIntegration test approach

• Top down

• Bottom up

• Sandwich

M23

M12

M22

M31

M0

M11

M21

M13

Top down integrationTop down integration

• Modules are integrated by moving downward

• Depth-first or breadth-first manner

• Verifies major control or decision points early in the test process

• Stubs replace low-level modules

M23

M12

M22

M31

M0

M11

M21

M13

Bottom – up integrationBottom – up integration

• Construction and testing with atomic modules

• Stubs are NOT needed

• Drivers are needed

Integration testing – some thoughtsIntegration testing – some thoughts

• How to choose integration order – incremental

• May be based on criticality of interfaces/module

• Risk reduction principle may be used

• Critically based on ..

- Module used by lot of other modules

- Module has high cyclomatic complexity

- Has special non-functional (e.g, performance) requirements

What to check for ….

• Interface integrity

• Functional validity

• Information content

• Performance

System testing

What is system testing?

System Testing

A system is the big component System testing is aimed at revealing bugs that

cannot be attributed to a component as such, to inconsistencies between components or planned interactions between components

Concern: issues, behaviours that can only be exposed by testing the entire integrated system (e.g., performance, security, recovery)

System testing is…System testing is…

• Functionality Testing

• Stress testing

• Usability testing

• Security testing

• Performance testing

• Storage testing

• Internationalization testing

• Compatibility testing

• Installation testing

• Reliability testing

• Recovery testing

• Supportability testing

• Compliance testing

• …

Acceptance Testing

Aims at uncovering ‘implied requirements’ Aims at evaluating ‘fitness for use’ Should not find bugs which should have been

found in earlier testing phases

Regression Testing

Testing after changes been made to ensure that no unwanted changes were made

Test Strategy

This defines the scope and general directions for testing for a project

Should be developed for each project Should be reviewed and approved with the

project plan

Test Plan

Test plans specify the test conditions, features, functions that will be tested for a specific level of testing. Plans also contain other information about resources, schedules, etc

Test plans should be prepared as soon as the corresponding document in the development life cycle is produced

Test specification

What is test specification?

Test Case

A sequence of steps, sub steps, and other actions, performed serially, in parallel, or in some combination of consecution, that creates the desired test conditions that the test case is designed to evaluate

Is complete testing possible?

To prove that a program is free of bugs is•Practically impossible•Theoretically a mammoth exercise

Stop Criteria

When to stop testing?

STOP Criteria

Time runs out – Poor criteria A certain number of errors found Require a certain test coverage Stop when testing becomes unproductive

Software testing

ExercisePrepare a checklist for testing

IV & V

What is Independent Verification & Validation?