Software Development

Software Testing

Testing Definitions

There are many tests going under various names. The following is a general list to get a feel for the different types of testing required by during typical high quality software development.

Acceptance Test

The test performed by users of a new or changed system in order to approve the system and go live

Usually carried out at the customer’s premises or using customer’s data with the customer being present

This is usually the final phase of development and results in the handover of the software product to the customer

Active / Passive Testing

An active test introduces specific test data and the results of processing that data are observed for correctness

Passive testing works on real data and the outputs are observed for correctness

Alpha and Beta Testing

Alpha testing is the first test phase carried out by the developers in the lab

Beta testing is carried out when alpha testing is complete and the developers can find no errors (ideally). Beta testing is carried out by selected real users who report back to the developers. This enables the users to get early exposure to the software and the developers get usability and error feedback

Automated Testing

This is when software is used to test software. This enables test repeatability and is faster as the user is not generally required to enter lots of data manually

If the code is changed in a single module, all the tests would be re-run on the whole application to ensure that the changes have not impacted on any other modules. Retesting everything in this way is often referred to as “regression testing”

White and Black Box Testing

White box testing requires knowledge of the internal operation of the software. This enables the test to chack all paths through a function for example

Black box testing assumes no knowledge of the internal operation and based on the specification will input test data at a system level and observe the correctness of the output

Dirty or Negative Testing

This method requires data to be input such that it will maximise the chances of failure so is used to test the application’s error recovery effectiveness

Examples could be user input validation through to memory allocation failure

Functional Testing

Does the application do what it has been specified to do?– Check all menu and tool bar options– Check all keyboard input

Likely to take the form of a task list which a tester goes through to check all options and navigation paths. This is not a usability test but initial feedback on usability could be collected at the same time

Recovery Testing

How well does the application recover from hardware and software failure?– Reading a data file which you know is always

available will still need error recovery code in case the disk you are reading from fails

– How well does the system recover from the dirty test phase?

– What happens if the device your application is trying to communicate with does not respond?

Test Case Testing

This consists of a set of test data, test programs (test harnesses) and a set of expected results

Typically a set of test cases is prepared (this is a test scenario). A file of input test data and expected results for example is read by the test harness (test driver program). A function (unit of code so is often referred to as a unit test) for example is called with the test data and the results are compared with the expected results. The success or failure of the test is reported and the next test is carried out. This method of testing tends to be totally automated if possible

Test Suite

This is a collection of test scenarios which themselves are a collection of test cases

In oop for example, a test suite may consist of a test scenario for each class. The test scenario may consist of a series of tests for each of the member functions

Usability Testing

A series of tests carried out by users or subject experts to determine the ease of use

Various techniques such as– Task driven– Observation– Keystroke and mouse recording– Interview– Think aloud – Questionnaires

Test Driven Development

This is not a test method as such but is test oriented. Based on the system requirements, a series of tests is specified which, when they pass, mean that a piece of working code has been produced. i.e. the test is specified before the software is written and the software is written to pass the test.

Often used in Extreme Programming (XP) but can be used in its own right as a development method

Compatibility Testing

Ensures that the software is compatible with the hardware, operating systems, and other software packages that it will be working with such as:

– Works on required software platforms such as Windows and Linux

– Works on required hardware platform such as 16, 32 and 64 bit systems

– Works with various software releases if required such as Win 95, XP, Vista

Performance Testing

Ensures that the software runs fast enough to meet speed requirements e.g. executing algorithms in time specified

Ensures that the software does not consume more memory than anticipated

Function Stubs

When developing an application as part of a team, you may need to call functions (or objects) that do not yet exist (because someone is still writing them– Write a function which does nothing other than

enable the caller to call it and return a value or series of values depending on the function spec.

– The caller can test the calling function’s response to a variety of return values from the stubs

Test Harness Example

Performs 16 and 32 bit compatibility tests Performs Test Case Testing on a function Uses Automated Testing Uses Active Testing Data and results taken from a Test Scenario Function under test is a C function to convert a

String to an Integer (like atoi but with extra features). Errors found would need source header updating

with mod date and author and an overview of the problem and fix