test execution and defect management

8/3/2019 Test Execution and Defect Management

1/36

Software Testing Lifecycle Practice-

Test Execution and Defect Management

Yujuan Dou


2/36

Information Before Class

Document from Joe To answer the

questions, it will be

helpful for final exam

Microsoft WordDocument

Microsoft Word

Document


3/36

Identifier

HW_TC&TRPT_Group Number_yymmdd


4/36

Agenda

Test Execution Defect Management


5/36

Test Ex ec ut ion


6/36

TEST Execution This phase involves actual test execution. The testers

must execute the documented test cases and capturethe results in a test report.

The test leader should divide the test cases amongthe different testers and consolidate the results oftesting.

In case the application under test is expected to havemultiple builds then a mechanism for identifying a

correct build for testing should be decided.


7/36

Unit Testing Integration Testing

System Testing Acceptance Testing

Levels of Test Execution


8/36

Test Log

Each Test Log should be made up of a series

of entries that present an audit trail for variousaspects of the test execution including, but

not limited to, the following:

the date and time stamp of when the eventoccurred

a description (usually brief) of the event logged

some indication of the observed status

additional contextual information where relevant

additional details relating to any anomalous or

erroneous condition detected


9/36

Test Execution

Test Environment

Test Data Test Tools


10/36

Test Execution

A Test Execution Run Plan could also beprepared depending upon the followingcriteria:

Execution rate

TC priority Execution

Order

Complexity

ImpactedFunctionality

Unstablefunctionality

CustomerPriority

Test ExecutionRun Plan

FunctionalityDependency

Execution

OrderTC Inter-

dependency


11/36

RUP-Test Execution

Setup test environment to known state

Set execution tool options

Schedule Test Suite execution

Execute Test Suite

Evaluate execution of Test Suite Recover from halted tests

Inspect the Test Logs for completeness and accuracy

Restore test environment to known state Maintain traceability relationships

Evaluate and verify your results


12/36

DEFECT MANAGEMENT


13/36

What are Defects

is a failure to comply with a requirement


14/36

Defect Management

This process goes in parallel with the testexecution phase.

Defect reporting should be clear and concise

and should make it easy for the developer toreproduce the failure scenario.


15/36

Defect Entry & Severity Classification:

Typically, the bugs are logged into a defectdatabase with the following information:

Synopsis - One line description of the bug.

Bug environments - Info about OS, Application

server, browser, Database, builds used with all theversion information where the bug occurs arespecified.

Other environment on which the bug does not occur

are also mentioned Detailed description with exact error message or the

observed behavior


16/36

Defect Information

Exact steps to reproduce the bugExpected behavior

Regression information if the bug was not

occurring on any of the recent builds.This information is often used to determinewhether the bug is to be fixed or not.

Severity of the bug

Priority of the bug


17/36

Defect Severity

Severity 0: Customers production use of the solutionis stopped or so severely impacted that the customer

cannot reasonably continue work. There is noworkaround or the provided workaround is notacceptable to the customer because of its business

impact. Severity 1: Implementation is operational but its

functionality is seriously affected. If a workaround has

been provided, the loss of functionality can only besustained for a short time. Or there is a problempreventing roll-out / go-live / implementation.


18/36

Defect Severity

Severity 2: Implementation is operational buta problem has been identified and a specificportion of the system either provides incorrectresults or is not operating as documented. A

workaround is available and is acceptable tothe customer.

Severity 3: Customer has a question on theproduct or a request for a product modificationor enhancement.


19/36

Severity in Bugzilla

This field describes the impact of a bug.

Blocker Blocks development and/or testing work

Critical crashes, loss of data, severe memoryleak

Major major loss of function Minor minor loss of function, or other problem

where easy workaround is present

Trivial cosmetic problem like misspelled words

or misaligned text

Enhancement Request for enhancement


20/36

Priority

This field describes the importance andorder in which a bug should be fixed. Thisfield is utilized by the

programmers/engineers to prioritize theirwork to be done.

The available priorities range from P1(most important) to P5 (least important.)


21/36

Bug Tracking Tool

A bug tracking tool must be used for tracking

the bugs. The tool must at minimum have the

following features:

Bug logging and automatic updating into the defectdatabase

Automatic generation of bug id which is used for all

further tracking of the bug Connecting to the appropriate release based on

the product and the release version id


22/36

Bug Tracking Tool Promotion and demotion of bugs

Notifications to the respective groups or memberswhenever there are promotions or demotions.

Querying the database for bugs based on acriterion. E.g. On Originators Name or Assigneesname, based on a build in which fix is available,based on states, list of open bugs, test bugs and

closed bugs for a given release.


23/36

Decision regarding Bug Fixing

The decision as to whether the bug needs to be fixed forthe current release or whether it can be postponed to afuture release is taken by the Bug Assessment teamwhich comprises of the SQA team, development team

and concerned third party groups.

The meeting is convened on a daily basis during the

testing cycle. The list of currently open bugs forms theagenda for the meeting. The main purpose of themeeting is as follows:


24/36

Review of all the currently open product defects anddetermine the appropriate vehicle for delivery. Only thebugs that are needed to be fixed for the current releasewill be connected to this release for fixing. The typical

criteria considered while deciding this are: Severity of the bug

Whether it is a regression bug or not

Criticality of the bug from the point of view of the end- Customer

Verification of duplicate bugs, and to connect themaccordingly through the bug tracking tool.


25/36

Defect Management

Reject the bug andupdate thru Bugtracker tool

Yes

Are there anymore open bugs?

End

No

Connect this bug to theappropriate release

Start

Is Bug Valid?

Is this bug to be fixedfor the current release?

Connect this bug to thecurrent release

Assign the bug todeveloper and send mail

Yes

No

No

Yes


26/36

Defect ManagementFixing of bugs and subsequent testing The developer who is assigned the bug, fixes it, and

adds the fix description through the bug tracking tooland also the build in which the fix would be available.The bug is then promoted to the Test state, andassigned to the QE manager for testing the fix.

A new build containing a list of bug fixes is then

released to SQA. The SQA team then verifies the fixby testing it.


27/36

Defect Management

If the bug still exists, then it is demoted to

Open/Re-open state

If the bug is fixed, then it is promoted to

Closed stateSQA also performs the regression testing to

ensure that the fixes have no side-effects.


28/36

Create bug report step by step


29/36

Step 1:Login bugzilla with your account


30/36

Step2: Click the link New at the

bottom of the page


31/36

Defect Tracking Process

Defect Tracking Process
http://defect%20tracking%20process.doc/http://defect%20tracking%20process.doc/http://defect%20tracking%20process.doc/


32/36

APPENDIX A: State Definition

ParentState DefinitionSubmitted Openingstateforadefectwhenthedefectparentiscreated.Assigned Defecthasbeentriagedandassignedtoadevelopmentteamlead.Opened Defecthasbeenassignedtoanownerforresolution.Pending

Build

DefecthasbeenfixedandisawaitingaBaseline&BuildintheappropriateUCMProjectIntegrationstream.

Validation

Defecthasbeenresolved,deliveredandbuilt

andisavailabletobedeployedtoatestenvironmentfortestingbyappropriatetestresource.

Closed DefectiscompletedandValidated asClosed.Deferred Defectresolutionispostponedandwillbeacteduponlater.Duplicate DefectalreadyexistsinClearQuest.

APPENDIX B: Origin Category


33/36

APPENDIX B: Origin CategoryOriginCategory DefinitionConfiguration Tablechangeenablessystemtoperformtospecification.Development Itemnotfunctioningasdefined.Environment Issueoccurredduetothetestenvironment.OutofScope Itemcouldnotreasonablybeconsideredpartofspecificationsortestscope.ProductionOnly Itemfoundinproductionhowevercouldnotberecreatedinthetest

environment.

Requirements NotincludedorerrorinbusinessrequirementsRequirements/

InterfaceAnalysisdidnotincorporateup/downstreamimpacts.

Specifications Notincludedorerrorinfunctionalspecifications/usecases.TechnicalChange Changeimprovessystemprocessinghowevercannotbedetectedasaproblemthroughtesting.TestIssue Notincludedintestscopeorplanordeterminedtobenotabug

(NAB).

ToBeDetermined Defectunderevaluation. Interimcategory.VendorSoftware Vendor/3rd Party software


34/36

Guideline for Lab

Guideline for Lab Operation.doc
http://guideline%20for%20lab%20operation.doc/http://guideline%20for%20lab%20operation.doc/http://guideline%20for%20lab%20operation.doc/


35/36


36/36

test execution and defect management

Documents