integration test

In The Name Of Allah

System Integration TestMOHAMMAD SADEGH SALEHI

FARDIN KAVYANI

Fall 2015

System Integration Test - Fall 2015

Agenda

3/20

Introduction

The Concept of Integration Testing

Component / Module testing

Why do we do integration testing?

The major advantages of conducting SIT

Granularity of System Integration Testing

Drivers and Stubs

Approaches in Integration Testing

Integration test tools


Introduction

4/20

Why Test?

No matter how well software has been designed and coded, it will inevitably still contain defects.

Testing is the process of executing a program with the intent of finding faults (bugs).

A “successful” test is one that finds errors, not one that doesn’t find errors.


Introduction-cont

5/20

Why Test?

Testing can “prove” the presence of faults, but can not “prove” their absence

But can increase confidence that a program “works”


Introduction-cont

What is Test?

6/20

Unit test – test of small code unit: file, class, individual method or subroutine.

Integration test – test of several units combined to form a (sub)system, preferably adding one unit at a time.

System (alpha) test – test of a system release by “independent” system testers.

Acceptance (beta) test – test of a release by end-users or their representatives.


Introduction-contWhen to Test?

7/20

Early “Agile programming” developers write unit test cases before coding each unit Many software processes involve writing (at least) system/acceptance tests in parallel

with development

Often Regression testing: rerun unit, integration and system/acceptance tests

After refactoring Throughout integration Before each release


Introduction-contDefining a Test?

8/20

Goal – the aspect of the system being tested.

Input – specify the actions and conditions that lead up to the test as well as the input (state of the world, not just parameters) that actually constitutes the test.

Outcome – specify how the system should respond or what it should compute, according to its requirements .


The Concept of Integration Testing

9/20

Testing in which software components, hardware components, or both together are combined and tested to evaluate interactions between them.

Integration testing usually go through several realword business scenarios to see whether the system can successfully complete workflow tasks.

Integration plan specifies the order of combining the modules into partial systems.

Integration is …Making different modules work together


for each

The Concept of Integration Testing-cont

10/20

readFile String[]String

getCharacterFreqString Frequency[]


Component / Module testing

11/20

A unit is the smallest testable piece of software. A unit is defined as a database trigger, stored procedure, function, report, form, batch load program or PL/SQL program.

Unit testing is the testing that is performed to validate that the unit does satisfy its functional specification and/or that its implementation structure does match the intended design structure.

Module /Component consists of units integrated into a module that performs a specific business function.


do we do integration testing?

12/20

Unit tests only test the unit in isolation.

Many failures result from faults in the interaction of subsystems.

Often many Off-the-shelf components are used that cannot be unit tested.

Without integration testing the system test will be very time consuming.

Failures that are not discovered in integration testing will be discovered after the system is deployed and can be very expensive.


Granularity of System Integration Testing

13/20

System Integration testing is performed at different levels of granularity

Intra-system testing This form of testing constitutes low-level integration testing with the objective of combining the

modules together to build a cohesive system

Inter-system testing It is a high-level testing phase which requires interfacing independently tested systems

Pairwise testing In pairwise integration, only two interconnected systems in an overall system are tested at a time The purpose of pairwise testing is to ensure that two systems under consideration can function

together, assuming that the other systems within the overall environment behave as expected


Integration testing approaches

14/20

Common approaches to perform system integration testing

Incremental Top-down Bottom-up Sandwich Big-bang


Drivers and Stubs

15/20

Driver: A program that calls the interface procedures of the module

being tested and reports the results. A driver simulates a module that calls the module currently being

Tested.

Stub: A program that has the same interface procedures as a module

that is being called by the module being tested but is simpler. A stub simulates a module called by the module currently being

tested

Driver

Tested

Unit

Stub

Module under test

Procedure call

Procedure call


Example: A 3-Layer-Design(SpreadSheet)

16/20

Layer I

Layer II

Layer III

Spread

SheetView

A

Calculator

C

BinaryFile

Storage

E

XMLFile

Storage

F

Currency

DataBase

G

Currency

Converter

D

Data

Model

B

A

C

E F G

DB

Spread

SheetView

BinaryFile

Storage

Entity

Model

A

E F

Currency

DataBase

G

Currency

Converter

DB

Calculator

C

XMLFile

Storage


Big-Bang Approach

17/20

A

C

E F G

DB

Test A

Test B

Test G

Test F

Test E

Test C

Test DTest

A, B, C, D,

E, F, G


Bottom-up Testing Strategy

18/20

The subsystems in the lowest layer of the call hierarchy are tested individually

Then the next subsystems are tested that call the previously tested subsystems

This is repeated until all subsystems are included

Drivers are needed.


Bottom-up Testing Strategy

19/20

A

C

E F G

DB

A

Test

A, B, C, D,

E, F, G

E

Test E

F

Test F

BTest B, E, F

C

Test C

D

Test D,G

G

Test G


Pros and Cons of Bottom-Up Integration Testing

20/20

Con: Tests the most important subsystem (user interface) last, Drivers needed.

Pro: No stubs needed. Useful for integration testing of the following systems,

• Object-oriented systems,• Real-time systems,• Systems with strict performance requirements.


Top-down testing strategy

21/20

Test the top layer or the controlling subsystem first

Then combine all the subsystems that are called by the tested subsystems and test the resulting collection of subsystems

Do this until all subsystems are incorporated into the test

Stubs are needed to do the testing.


Top-down Integration

22/20

Test

A, B, C, D,

E, F, G

All LayersLayer I + II

Test A, B, C, D

Layer I

Test A

A

E F

B C D

G


Pros and Cons of Top-down Integration Testing

23/20

Pro Test cases can be defined in terms of the functionality of the system

(functional requirements) No drivers needed

Cons Writing stubs is difficult: Stubs must allow all possible conditions to be

tested. Large number of stubs may be required, especially if the lowest level of the

system contains many methods. Some interfaces are not tested separately.


Sandwich Testing Strategy

24/20

Combines top-down strategy with bottom-up strategy

The system is viewed as having three layers A target layer in the middle A layer above the target A layer below the target

Testing converges at the target layer.


Sandwich Testing Strategy

25/20

Test

A, B, C, D,

E, F, G

Test B, E, F

Test D,G

Test A

Test E

Test F

Test G

Test A,B,C, D

A

E F

B C D

G


Pros and Cons of Sandwich Testing

26/20

Top and Bottom Layer Tests can be done in parallel

Problem: Does not test the individual subsystems and their interfaces thoroughly before integration

Solution: Modified sandwich


Modified Sandwich Testing Strategy

27/20

Test in parallel:

Middle layer with drivers and stubs Top layer with stubs Bottom layer with drivers

Test in parallel:

Top layer accessing middle layer (top layer replaces drivers) Bottom accessed by middle layer (bottom layer replaces

stubs).


Modified Sandwich Testing

28/20

Test F

Test E

Test B

Test G

Test D

Test A

Test C

Test B, E, F

Test D,G

Test A,C

Test

A, B, C, D,

E, F, G

A

E F

B C D

G


Continuous Testing

29/20

Continuous build:Build from day oneTest from day oneIntegrate from day one System is always runnable

Requires integrated tool support:Continuous build serverAutomated tests with high coverageTool supported refactoringSoftware configuration managementIssue tracking.


Continuous Testing Strategy

30/20

Spread

SheetView

BinaryFile

Storage

Data

Model

Layer I

Layer II

Layer III

A

E F

Currency

DataBase

G

Currency

Converter

DB

Calculator

C

XMLFile

Storage

Sheet View+ Cells

+ Addition+ File Storage


Steps in Integration Testing

31/20

4. Test subsystem decomposition: Define test cases that exercise all dependencies

5. Test non-functional requirements: Execute performance tests

6. Keep records of the test cases and testing activities.

7. Repeat steps 1 to 7 until the full system is tested.

The primary goal of integration testing is to identify failures with the (current) component configuration.

1. Based on the integration strategy, select a component to be tested. Unit test all the classes in the component.

2. Put selected component together; do any preliminary fix-up necessary to make the integration test operational (drivers, stubs)

3. Test functional requirements: Define test cases that exercise all uses cases with the selected component


Integration Test tool

32/20


Resource

33/20

Software Testing and Quality Assurance: Theory and Practice, Chapter 7 Kshirasagar Naik, Priyadarshi Tripathy, 2008. ISBN: 978-0-471-78911-6

Object-Oriented Software Engineering Using UML, Patterns, and Java, 3e, Chapter11, Bernd Bruegge, Allen H. Dutoit, 2009, ISBN:0136061257

Martin Fowler about CI http://www.martinfowler.com/articles/continuousIntegration.html

http://www.qatestingtools.com/integration_testing_tools

http://www.martinfowler.com/articles/continuousIntegration.html

Thank You …

Questions….

[email protected]@gmail.com

mailto:[email protected]

mailto:[email protected]


The major advantages of conducting SIT

35/20

The major advantages of conducting SIT(System Integration Testing) are as follows:

Defects are detected early

It is easier to fix defects detected earlier

We get earlier feedback on the health and acceptability of the individual modules and on the overall system

Scheduling of defect fixes is flexible, and it can overlap with development


Integration Challenges

36/20


Integration Challenges-cont

37/20

Broken Integration

You have a broken integration when: Source code server does not build successfully, Shared component works in one system, but breaks others, Unit tests fail, Code quality fails (coding conventions, quality metrics), Deployment fails.

The earlier you can detect problems, the easier it is to resolve them.


Integration Challenges-cont

38/20

Manual Integration

Integration becomes expensive if made manual (build, test, deployment, …) with too less checkin’s (hours or days…) If integration problems and bugs are detected too late

Reduces desire to refactor Long time between integration increases risk of merge

IDE makes many cross-cutting changes easy


Different Types of Interfaces

39/20

Three common paradigms for interfacing modules:

Procedure call interface Shared memory interface Message passing interface

The problem arises when we “put modules together” because of interface errors.

Interface errors

Interface errors are those that are associated with structures existing outside the local environment of a module, but which the module uses.


Different Types of Interface Errors

40/20

Inadequate error processing

Additions to error processing

Inadequate post-processing

Inadequate interface support

Initialization/value errors

Validation od data constraints

Timing/performance problems

Coordination changes

Construction

Inadequate functionality

Location of functionality

Changes in functionality

Added functionality

Misuse of interface

Misunderstanding of interface

Data structure alteration

Hardware/software interfaces