CSIS3600 Systems Analysis and Design

System Implementation

and Testing

TestingOnce coding has begun, the testing process can also begin. Testing is an important part of development. Initial testing plans are usually devised during the analysis stage to be carried out during the implementation phase. Besides testing the software for errors and defects, testing serves as a way to improve communications. Analyst serves as an advisor and coordinator for program testing.

Quality Assurance and Testing

QA activities are often shortchanged during design and especially during implementation.

Schedule pressures can build with each successive phase. QA and testing activities may be bypassed in an

ill-fated attempt to speed up the project.

QA activities require development personnel to open their work to thorough examination and criticism by others. Many people are reluctant to do this.

Many people view testing and test personnel as the bearers of bad news. They mistakenly believe that no news is good

news and that bypassing testing is a way of avoiding bad news.

How to Overcome the Attitude Towards QA

Integrate QA into the project and schedule it from the beginning and never abandon it.

What Must Be Tested?

Code (application programs, etc.)

Procedural manuals

New Hardware

System Interfaces

When should testing really begin?

Ideas?

When Should Testing be Done?

Done throughout the systems development process Best to test beforehand than to fail after

installation Testing can be done on 'parts' and then in

totality

Categories of Tests

Testing can be divided several ways.

Generally there are two categories of tests. Static testing in which the code being tested is

not executed Technical Reviews, Walkthroughs, Inspections and

Desk checking are static Dynamic testing in which the software code is

executed.

Formal but less Intimidating Quality Assurance Procedures

Technical review Formal or informal review of design or

construction details by a group of developers Sometimes it just helps for someone else to

‘look at it’

Walkthrough A review by two or more people of accuracy and completeness.Tend to be informalThe focus is on what the code doesMost often used during analysis and design but can also be used during implementationIf done during analysis, done without actual code (can be done using models)[somewhat what I do when I review your group deliverables]

DeskChecking Testing technique in which the program code is sequentially executed manually by the reviewer

Usually someone who understands program logic reviews the code with a pencil and paper

Inspection More formal version of a walkthrough Participants review and analyze materials before they

meet as a group When group meets, participants play specific roles

Presenter – summarized material being inspected Critics – describe errors or concerns found Secretary – records notes

Additional errors or problems may be uncovered during the discussion

Usually the goal is to examine program code for predictable language-specific errors

Participants discuss possible solution strategies and, before the meeting is over, agree on approaches to problem resolution.

Formal QA Procedures

Testing usually follows four phases: Modeling the software's environment Selecting test scenarios Running and evaluating test scenarios Measuring testing progress

Test Cases

A test case is a formal description of a starting state one or more events to which the software must

respond the expected response or ending state

Starting state and ending state are represented by a set of test data. Starting state may represent a particular set of

database contents existence of a particular customer and order for

that customer Event may be represented by a set of input data customer account number and order number

used to query order status Expected response may be a described behavior

(display of certain information) or a specific state of stored data (canceled order)

More on Test Cases

Preparing test cases and test data is tedious and time-consuming.

At the program or module level, every instruction must be executed at least once.

There are automated tools based on proven mathematical techniques that can be used to generate a complete set of test cases.

What Tests to Select

Set of tests that cause each source statement to be executed at least onceSet of tests that cause each branching structure (If, CASE, While, etc) to be evaluated with each of its possible actions.Set of tests that cause each data structure to be initialized and then subsequently usedSet of tests that expose the seeded faults. Seeded faults are errors intentionally inserted into the

source code. Ideally by finding seeded errors, the tester will also find real errors.

Tests

Unit Testing

Integration Testing

Stub Testing

System Testing

Build and Smoke Test

Regression Testing

Unit Testing

Testing of each module in isolation in an attempt to discover any errors in its code.Programmers are generally responsible for unit-testing their own code prior to integrating it with modules written by other programmers.Some organizations assign “testing buddies” to help programmers test their own code.Having a different programmer test the code usually finds more errors.

Integration Testing Process of bringing together modules for test purposesObjective is to identify errors that were not or could not be detected by unit-testing such as: Interface compatibility Parameter values Run-time exception Unexpected state interactions

It is a gradual process, You being with the root and add coordinating modules

one at a time

Stub Testing

Process used in testing modules

Stubs are 2-3 lines of code that stand in for a missing module

Effective where modules operate top-down

During testing the coordinating module calls a stub who ‘stands’ in for the missing module.

System Testing Similar to integration testing but extends the bringing together of program modules to bringing together components.Follows the same incremental logic as integration testing.More than simply expanding integration testing attempts to demonstrate whether the system has met its

objectiveInvolves the use of non-live test data in a non-live testing environment that simulates the real environment as closely as possible. Usually test case scenarios are developed to form the basis of system testing

Build and Smoke Test System test that is performed dailyThe system is completely compiled and linked and a battery of tests is performed to see whether anything malfunctions in an obvious way (“smokes”)Commonly associated with interactive or rapid developmentIdentify problems results from code modification Daily testing ensure that errors are found quickly and can be easily tracked to their source

Regression Testing

After testers submit successfully reproduced failures to development, developers generally create a new version of the software (in which the bug[s] have been removed).

Testing progresses through subsequent software versions until one is determined to be fit for release.

This retesting is called regression testing.

Acceptance Test

Performed to determine whether the system fulfills user requirements

Last round of testing before system is handed over to users

Formal activity

Versioning

Alpha

Beta

Limited Availability

General Availability

Alpha

Test version that is incomplete but ready for some level of rigorous testing.

Tests performed during alpha include: Recovery testing – force software to fail to insure it can

recover Security testing – verify protection mechanisms Stress testing – tries to break the system Performance testing – determines how the system

performs under a range of possible environments.

Beta

A system that is stable enough to be tested by end users

Run in an end users environment using their own data

Sometimes thought of as a rehearsal of the installation phase

Production version

Limited Availability – A production version considered final but is released to a limited number of end users for additional user acceptance testing

General Availability – A production version is considered a final product that can be formally distributed to general population of users

Numbering of Versions Alpha usually starts at 0.# Beta is also numbered beginning with 0.# First Production version starts with 1.0 Versions progress from here sometimes with

alpha and beta versions of major upgrades

Example Timeline – Test and ProductionDate    

January 2000 Alpha version 0.1  

April 2000 Alpha version 0.2  

July 2000 Beta version 0.3  

September 2000 Beta version 0.4  

January 2001 Production version 1.0

 

April 2001   Planned version 2.0

September 2001 Production version 1.1

 

January 2002   Alpha version 1.91.

April 2002   Beta version 1.9.2

January 2003   Production version 2.0

Complexity of Testing OO Methods can be called by many other methods and the calling methods may be distributed across many classesClasses may inherit methods and state variables from other classesThe specific method to be called is dynamically determined at run time based on the number and type of message parametersObjects can retain internal variable values (that is, the object state) between calls. The response to two identical calls may be different due

to state changes that result from the first call or occur between calls.

Testing Summation

Testing and Technical reviews find between 50 and 75 percent of errors.

Some errors are more easily detected using one method while other by using a different method.

Thus, the two techniques are more effective used jointly rather than individually.

References:Types of Testing Coverage:

http://www.kaner.com/coverage.htm

Step-by-Step Test Design

http://www.stqemagazine.com/

Number of testers to developers:

http://www.jrothman.com/Papers/ItDepends.html

Testing and Java Technology

http://developer.java.sun.com/developer/technicalArticles/InnerWorkings/testing/

Site for Testers to Share Techniquies

http://www.testingcraft.com/exploratory-robinson.html

Great Bugs Site

http://www.bugnet.com/