Chapter 8&9System Testing and Acceptance Testing322235 Software Testing

ByWararat Songpan(Rungworawut), Ph.D.

Department of Computer Science, Faculty of Science, Khon Kaen University

System Testing

Requirements Specification

PreliminaryDesign

DetailedCoding

Coding

Unit Testing

IntegrationTesting

SystemTesting

Acceptance Testing

System Testing• The behavior of whole system/product is tested

as defined by the scope of the development project or product.

• System testing is most often the final test to verify that the system to be delivered meets the specification and its purpose.

• System testing is carried out by specialists testers or independent testers.

• System testing should investigate both function and non-functional requirement of the testing.

Type of System Testing

• Functional Testingo Unit Testing (Black box & White box)o Integration Testing (Black box & White box)o System Testing (Black box)o Acceptance Testing (Black box)o Regression Testing (Black box & White box)

• Non-Functional Testingo Usability Testingo Performance Testingo Load Testingo Stress Testingo Recovery Testingo Security Testingo Compatibility Testing

Entry Criteria for System Testing

• Unit Testing should be finished.• Integration of modules should be fully integrated.• As per the specification document software

development is completed.• Testing environment is available for testing

(Similar to Staging environment)

How to do System Testing

Step 1) First & important step is preparation of system Test planStep 2) Second step is to creation Test Scenario&test cases.Step 3) Creation of test data which used for System testing.Step 4) Manual/Automated test case execution.Step 5) Execution of normal test case.Step 6) Bug reporting, Bug verification & Regression testing.Step 7) Repeat testing life cycle (if required).

System Testing• Document: Test Scenario & test cases.• Test Scenario: This is a brief story about the

flow/path followed by the system/application under test to achieve a business process flows that is a document describe the input, Action/Event and expected response to determine whether the particular functionality of the application is working fine.

Example of Test Scenario

1. Input UserName and

Password2. Click Log In

3. Show message

“Welcome to UserName”

Test Scenario Design

Step Test Action Expected Results

Pass/Fail Approval

1 Input User and Password

User and Password is

not more than 6

characters

Pass Ok

2 Click Log In Log In Button is ok

Pass Ok

3 Show Message “Welcome to UserName”

Show Message

Fail Ok

Test Scenario ID: TS_Login

Test Scenario & Test case Design (Combination)

Test ID Test Action&Test case Expected Results

Pass/Fail Approval

Login01 1. Input User: wararat Input Password: comsci2. Click Log In

Show Message Welcome to

wararat

Pass Ok

Login02 1. Input User: Input Password: comsci2. Click Log In

Login page is not display

Pass Ok

Login03 1. Input User: wararat Input Password: 2. Click Log In

Login page is not display

Pass Ok

Login04 1. Input User: wararat2 Input Password: wararat2. Click Log In

Login page is not display

Pass Ok

Test Scenario ID: TS_Login

Requirement Validation Matrix

Req_01

Req_04

TC

001

TC

002

TC

003

TC

007

Req_02

Req_03

TC

004

TC

005

TC

006

Traceability Requirement to System Testing

• Requirement of Hotel Reservation System 1. Login 2. Reserve Booking 3. Manage Booking 4. Payment

12

Requirement No.

Requirement Description

Test Scenario ID

Test Case ID Status

Req_01 Login TS001 TC001, TC002, TC003, TC004

Fail

Req_02 Reserve Booking

TS002 TC005 Pass

Req_03 Manage Booking

TS003 TC006 Pass

Req_04 Payment TS004 TC007 Fail

Example: Requirement Validation Matrix Tool

Ref: http://www.lifeheed.com/article.php?article=72

Non-Functional Testing

• Usability Testing• Performance/Load Testing• Stress Testing• Recovery Testing• Security Testing• Compatibility Testing

Usability Testing• Usability means the software s capability to be

learned and understood easily and how attractive it looks to the end user.

• Testing toolso http://www.conceptfeedback.como Click Heato Google Analyticso Google Website Optimizer

Performance Testing/Load Testing

• Performance Testing is done to determine the software characteristics like response time, throughput or MIPS (Millions of instruction per second) / virtual users at which the system/software operates.

• Testing Toolso Microsoft Visual Studio 2008 and aboveo Load2Testo Site Blastero Piloto Testinganywhereo Neoload

Stress Testing• Testing of software/hardware for its effectiveness

in giving consistent or satisfactory performance under extreme and unfavorable conditions such as heavy network traffic, heavy processes load, under or over clocking of underlying hardware, working under maximum requests for resource utilization of the peripheral or in the system.

• Testing Toolso Microsoft Visual Studio 2008 and above.o Load2Testo Site Blastero Pilot

Security Testing• Security is all about finding out all the potential

loopholes and weakness of the system, which might result into loss/theft of highly sensitive information or destruction of the system by an intruder/outsider.

• Testing toolso Web Scarabo Wire Shark

Compatibility Testing• Compatibility Testing is a type of testing used to

ensure compatibility of the system built with various other objects such as other web browsers, hardware platforms, user and operating system.

• Testing Toolso Adobe Browser Labo Browser Shotso Super Preview

Acceptance Testing

Requirements Specification

PreliminaryDesign

DetailedCoding

Coding

Unit Testing

IntegrationTesting

SystemTesting

Acceptance Testing

Acceptance Testing• Acceptance Testing or User Acceptance Testing

(UAT) is the software testing process where system tested for acceptability & validates the end to end business flow. Such type of testing executed by client in separate environment (Similar to production environment) & confirm whether system meets the requirements as per requirement specification or not.

• Acceptance Testing is “Black Box” tests. UAT does not aware of internal structure of the code.

Environment Setup

System(Operating in Production)

New System(As a run

test)

Same input

Output Output

Match

Defect analysis Installation

YesNo

Type of Acceptance Testing

• As the software application may use different users on different way.

• End user will use and how customer actually use the software application.

• Most of software venders are use the term:o Alpha testingo Beta Testing

Alpha Testing• Alpha Testing is conducted by Customer at the

developer s site.• It is performed by potential users like developer,

end users or organization users before it is release to external customer & report the defect found while Alpha Testing.

Beta Testing• The beta testing is to be carried out without any

help of developers at the end of user site by the end user.

• It is performed under uncontrolled environment. • This testing is conducted by limited users & all

issues found during this testing are reported on continuous basis which help to improve the system.

• The is last stage of the testing where product is sent outside the company of for trial offer to download.

How difference Alpha & Beta Testing

•Alpha Testing

During Development

•Beta Testing

Customer-end Site

Example: Beta Testing

Example: Beta Testing

Example: Beta Testing

Regression Testing

Requirements Specification

PreliminaryDesign

DetailedCoding

Coding

Unit Testing

IntegrationTesting

SystemTesting

Acceptance Testing

Regression Testing

Regression Testing• Regression Testing is one of the most importance

type of software testing, in which it checks whether a small change in any component of the application does not affect the unchanged component.

• Testing Tools.o Winrunnero Quick Test Proo Regression Testero Selenium