test estimation challenges

7/27/2019 Test Estimation Challenges

1/5

Test Estimation Challenges

1. Successful test estimation is a challenge for most organizations coza. No standard Formulae/Methods for Test Estimation

b. Test Effort Estimates also includes the Debugging Effortc. Difficult to attempt testing estimates without first having detailed information about a

project

d. Software Testing Myth that Testing can be performed at the Ende. Difficult to attempt testing estimates without an understanding of what should be

included in a 'testing' estimation for a project (functional testing? unit testing? reviews?

inspections? load testing? security testing?)

Traditional Practices

1. Current Test Planning Methods include

a. Using Percentage of Development Efforti. Depends on the accuracy of the Development Effort

ii. Does not account revisit of Development Effortb. Using Tester Developer Ratio

i. May not be same for all types of Projects

ii. Does not consider the size of the Projectc. Using KLOC

i. Does not consider Complexity, Criticality & Priority of the Project

Whats the Best Approach to Test Estimation

There is no simple answer for this. The 'best approach' is highly dependent on theparticular organization and project and the experience of the personnel involved.

For example, given two software projects of similar complexity and size, the appropriate

test effort for one project might be very large if it was for life-critical medical equipmentsoftware, but might be much smaller for the other project if it was for a low-cost

computer game. A test estimation approach that only considered size and complexity

might be appropriate for one project but not for the other.

Approaches to Test Estimation

Implicit Risk Context Approach:

Metrics-Based Approach:

Test Work Breakdown Approach:

Iterative Approach:


2/5

Percentage-of-Development Approach:

Test Estimation Process A Practical Approach

1. Combination of all the Approaches

2. Considers Risk & Complexity Factors3. Based on Previous History i.e. Organization or Project Metrics

4. Based on Work Breakdown Structure

5. Based on Iterative Model

Elements of Test Estimation Process

1. Break sizing into smaller, easier to estimate tasksa. Decompose the test project into phases.

i. System Test

ii. Unit Test

b. Decompose each phase into constituent activitiesi. System Test Planning

ii. Test Executionc. .Decompose each activity into tasks and subtasks until each task or subtask at the

lowest level of composition

i. Executing a test scenarioii. Writing a defect

2. Taking risk priority into account

3. Set up dependencies

a. Dependent tasks internal to the test subproject.b. Document dependencies, resources, and tasks external to the test subproject (i.e.,

those that involve collaborative processes)

1. Consider type of code (complex, reused, etc.)

2. Augment professional judgment and gut instinct with previous project data, industry

metrics, and so forth.3. Identify and, if possible, resolve discrepancies between the test subproject schedule

and the project schedule.

4. Use the work-breakdown-structure and schedule to develop a budget. Extract from

your work-breakdown-structure a complete list of resources. For each resource,determine the first and last day of assignment to the project.

5. If you have resources shared across multiple test projects within a given time period,

understand the percentage allocation of each resources assignment to each project duringvarious time periods.

6. Revisit the Estimation continuously in order to reflect any change in the Project

Requirements or Schedule7. Be Repeatable preferably Automat able

A Sample Estimation Process


3/5

Total work = Test case time + Defect time

Test Case Time = Test Case Development time + Test Case Execution Time

Defect Time = (Hours/Defect * # Defects)

Note: Consider the defect severities while arriving at Hours/Defect and also that

Hours/Defect should take into account the Defect creation time, Debugging time and

Defect Retesting time till the closure.

Test Case Time

Test Case Development Time = (Hours/Test case development* #Test cases)Test Case Execution Time = (Hours/Test case Execution * #Test Cases)

Note: Consider the Risk, Complexity while arriving at the Hours/Test case Development

and Hours/Test case Execution

Software Test Estimation - 9 General Tips on How to Estimate Testing Time Accurately

Posted In | Testing Life cycle, Test strategy

This is a guest article by Author N. Sandhya Rani.

For success of any project test estimation and proper execution is equally important as thedevelopment cycle. Sticking to the estimation is very important to build good reputation

with the client.

Experience play major role in estimating software testing efforts. Working on varied

projects helps to prepare an accurate estimation for the testing cycle. Obviously one

cannot just blindly put some number of days for any testing task. Test estimation should

be realistic and accurate.

In this article I am trying to put some points in a very simple manner, which are helpful

to prepare good test estimations. I am not going to discuss the standard methods for testestimations like testing metrics, instead I am putting some tips on - How to estimate

testing efforts for any testing task, which I learned from my experience.

Factors Affecting Software Test Estimation, and General Tips to Estimate Accurately:

1) Think of Some Buffer Time


4/5

The estimation should include some buffer. But do not add a buffer, which is not realistic.

Having a buffer in the estimation enables to cope for any delays that may occur. Having a

buffer also helps to ensure maximum test coverage.

2) Consider the Bug Cycle

The test estimation also includes the bug cycle. The actual test cycle may take more daysthan estimated. To avoid this, we should consider the fact that test cycle depends on the

stability of the build. If the build is not stable, then developers may need more time to fix

and obviously the testing cycle gets extended automatically.

3) Availability of All the Resources for Estimated Period

The test estimation should consider all the leaves planned by the team members (typically

long leaves) in the next few weeks or next few months. This will ensure that theestimations are realistic. The estimation should consider some fixed number of resources

for test cycle. If the number of resources reduces then the estimation should be re-visited

and updated accordingly.

4) Can We Do Parallel Testing?

Do you have some previous versions of same product so that you can compare theoutput? If yes, then this can make your testing task bit easier. You should think the

estimation based on your product version.

5) Estimations Can Go Wrong - So re-visit the estimations frequently in initial stages

before you commit it.

In early stages, we should frequently re-visit the test estimations and make modification

if needed. We should not extend the estimation once we freeze it, unless there are majorchanges in requirement.

6) Think of Your Past Experience to Make Judgments!Experiences from past projects play a vital role while preparing the time estimates. We

can try to avoid all the difficulties or issues that were faced in past projects. We can

analyze how the previous estimates were and how much they helped to deliver product ontime.

7) Consider the Scope of Project

Know what is the end objective of the project and list of all final deliverables. Factors tobe considered for small and large projects differ a lot. Large project, typically include

setting up test bed, generating test data, test scripts etc. Hence the estimations should be

based on all these factors. Whereas in small projects, typically the test cycle include testcases writing, execution and regression.

8 ) Are You Going to Perform Load Testing?If you need to put considerable time on performance testing then estimate accordingly.

Estimations for projects, which involve load testing, should be considered differently.

9) Do You Know Your Team?


5/5

If you know strengths and weaknesses of individuals working in your team then you can

estimate testing tasks more precisely. While estimating one should consider the fact that

all resources may not yield same productivity level. Some people can execute fastercompared to others. Though this is not a major factor but it adds up to the total delay in

deliverables.

And finally tip number 10.

Over To You!

This test estimation tip is purposefully left blank so that you can comment your bestestimation techniques in below comment section.

test estimation challenges

Documents