info 637lecture #81 software engineering process ii integration and system testing info 637 glenn...

INFO 637 Lecture #8 1

Software Engineering Process IIIntegration and System Testing

INFO 637

Glenn Booker


Testing & Documentation

This week covers integration and system testing, and user documentation


Testing Principles

The purpose of testing is to assess the quality of the product

While defects found in testing need to be fixed, few defects should be left by the beginning of testing

If you find a lot of defects in testing, there are probably lots of others hiding!


Testing Principles

Even good software engineers, following a poor process for development, will tend to produce poor quality software

Testing is not a substitute for using a good process!


Testing Strategy

Build the system from the unit-tested partsIntegration test the system to ensure it

is correctly built, and all parts function together

System test the product to demonstrate it meets system requirements


Testing Strategy

While doing all of this: Identify low quality components and return them

to the quality/process manager for assessment and cleanup

If they still cause problems, return them to the quality/process manager for rework or replacement

In later cycles, regression tests show you didn’t break anything that worked earlier


Build and Integration Strategies

Several approaches may be used for building the system

Integration testing is to verify that all parts are present and communicating with each other – testing for functionality is done by system testing


Build and Integration Strategies

There are four main integration strategies Big Bang Strategy One-at-a-time Strategy Cluster Strategy Flat-system Strategy

For good quality code, almost any strategy could work well


Big Bang Strategy

The simplest integration strategy is to simply put all of the pieces together and see if everything works

Rarely works for complex or low quality systems – only try if you have known high quality components


One-at-a-time Strategy

Another simple approach is to start with one component, and test it

Then add another component, and test them

Keep repeating one component at a time until the entire system is present

Good for identifying problems, but very time intensive


Cluster Strategy

Another common strategy is to implement groups of related modules together

Choices of groups are either by type of function (printing, file management, reports, etc.) or by portion of the system (ATM balance verification, then withdrawal, then deposit)


Flat-system Strategy

This is a top-down integration strategyFirst build the highest level modulesThen add the next level modules called

by themThen keep adding lower and lower level

modules until the system is completeGood for finding system-level issues


System Test Strategy

System testing seeks to answer some fundamental questions about the system Does the system perform its

intended functions? Does the system meet its quality goals? Will the system work under normal conditions? Will the system work under adverse conditions?



Other key concerns include: Can the system be installed? Does it start properly? Will it recover from software or

hardware failures? Is its performance adequate? Can users use the system easily?



System testing cannot be complete for any complex system, so test cases need to be chosen to address the most important concerns

Test planning should address several related functions or characteristics with each test



Most common priorities for system testing include: System functionality Stress testing Usability testing Performance testing


Second System Test Strategy

Another approach is to do all aspects of testing for one functional area at a time

Do function testing, stress testing, usability, and performance testing for one set of functions

Then repeat for each functional areaLoses overall feel for the system


Third System Test Strategy

Combine previous two strategiesDo normal and stress testing for lower

level functionsThen test higher and higher level functions

until the entire system is coveredGood for poor quality systems, but

time consuming


Fourth System Test Strategy

Do a top down version of the previous strategy

Test highest level functions under normal and stressed conditions

Work down to lower and lower level functions until whole system tested

Only works with high quality product


Test Planning

We have seen test planning appear in many phases of the TSP

Test plans need to identify The materials needed for testing (including

the system configuration tested) What tests will be run What order the tests will be run


Test Completeness

TSP focuses on measuring system testing by the percent of requirements which were tested

Other methods focus on the percent of logical paths which were tested, or the percent of code exercised


Test Planning

Test plans also need to: Name each test Predict the outcome of each test Predict how long the test should take Scripts and test cases will be needed for

most testsEstimate the number of defects to be

found in each phase, and their repair time


Test Tracking

Test effectiveness is measured by the number of defects found per hour of running tests

A test log (e.g. p. 172) records the details of testing


Defect-prone Modules

Modules (components) which have a lot of defects in testing have a lot of defects after release

Hence it is critical for testing to identify parts of the system which need to be fixed or reworked before they become a problem later


Defect Data

SUMP records the defects found and removed for each (life cycle) phase

SUMQ captures quality for each moduleHence these two forms are key to

identifying defect-prone modules, and knowing when they first became that way


Test Scripts

To start testing you need to have: Completed SRS and SDS documents You have some components which have been

implemented and passed unit testing Later cycles also need prior versions of

the product


Test Scripts

Test script TEST1 (p. 178) includes: Develop build, integration test, and system

test procedures Determine size, time, and materials needed for

each test Build the product Conduct integration testing Conduct system testing


Test Scripts

Produce user documentation, review and fix it

Outputs include An integrated and tested product for this cycle Completed LOGD and LOGTEST forms for

all tests Completed user documentation Time, size, and defect data in TSP system


Documentation

Documentation for a system may include manuals for various purposes and audiences Installation User Maintenance (troubleshooting, upgrades, etc.) Training Marketing

Here we focus on end user documentation


Documentation

Documentation captures the mindset of the design – its philosophy or train of thought

This helps the user know what should be expected next

Documentation can also capture little used features, and ones whose usage are hard to guess correctly


Documentation Approach

Documentation needs to be written from the user’s viewpoint, not the designer’s

Good user documentation focuses on groups of functions which the user may wish to use, generally increasing in complexity

Best to start writing documentation right after design of each type of function


Documentation Features

Good user documentation includes: A glossary for unusual terminology Error messages, troubleshooting, and

recovery information Index of key topics And a detailed table of contents

Good to outline the documentation first, and make sure key functions are accounted for


Documentation Features

Documentation should use short, simple paragraphs and sentences

Use lots of bulleted lists to break up concepts

Write to a consistent level of detail, and make sure documentation uses a consistent expectation of what the user knows


Documentation Review

Check documentation for: Organization Terminology Content Accuracy Readability Understandability


Why Here?

Why is documentation discussed with integration and system testing? User documentation is often the basis for some

functionality (system) testing Need to make sure the examples provided

in the documentation really do work exactly as stated!

info 637lecture #81 software engineering process ii integration and system testing info 637 glenn...

Documents

system testing info

system testing slide

testing strategy

system integration testing

entire system

system requirements

testing documentation

testing principles