test case point analysis

Click here to load reader

Post on 11-Aug-2014

4.870 views

Category:

Automotive

0 download

Embed Size (px)

DESCRIPTION

Approaches to estimating size and effort of software testing activities.

TRANSCRIPT

  • Agenda Objectives Background and Motivation Existing Estimation Methods Test Case Point Analysis Effort Estimation Methods using TCP Conclusion and Future Work 2
  • Objectives Discuss importance of software estimation, especially for testing projects Discuss existing methods for estimating testing activities Introduce an approach to estimating testing projects Test Case Point Analysis Effort Estimation using TCP 3
  • Agenda Objectives Background and Motivation Existing Estimation Methods Test Case Point Analysis Effort Estimation Methods using TCP Conclusion and Future Work 4
  • Software Estimation and Its Importance Software estimation process of determining the cost, time, staff, and other related attributes of software projects, often before work is performed Estimation is important for the success or failure of software projects Provides inputs for Making investment decisions Budget and staff allocation Tradeoff and risk analysis Project planning Stakeholder negotiation etc. 5
  • Popular Software Estimation Methods Sizing Methods Source Lines of Code (SLOC) Function Points Analysis Use Case Points Effort Estimation Methods Expert judgment/experience Productivity index COCOMO, SEER-SIM, SLIM models Software testing estimation methods Using a test distribution percentage Test Case Points Analysis (by Cognizant Technology Solutions) [2] Mainly based on the number of steps 6
  • Motivation Testing accounts for up to 50% of project effort [1] Current problems estimates are done for the whole project rather than testing specific lack of reliable methods designed for estimating size and effort of software testing vague definitions of testing productivity due to the lack of a size measure for software testing There are needs of accurately estimating effort of testing activities measuring size and productivity of testing activities measuring effectiveness and efficiency of software testing 7
  • Motivation (contd) Our aim to Defining a method for estimating the size of testing activities Defining methods to estimate testing effort, schedule, and staff accurately using this size measure Using this size measure as a basis for measuring productivity effectiveness other testing metrics 8
  • Agenda Objectives Background and Motivation Existing Estimation Methods Test Case Point Analysis Effort Estimation Methods using TCP Conclusion and Future Work 9
  • Test Case Point Analysis Principles Size must reflect the mass and complexity of the testing project Size should correlate with testing effort Test case point is measured using test cases as main input Test case complexity is based on Checkpoints Precondition Test Data Type of test case 10
  • Test Case Point Analysis (contd) Process overview Project Attributes Test Case Estimate EstimatedTest Case Test Case Points Testing Effort Point Analysis Effort 11
  • Test Case Point Analysis (contd) TCPA estimates the size of testing projects using test cases as input Steps 1. Measure test case complexity Count checkpoints Measure the complexity of precondition Measure the complexity of data 2. Adjust Test Case Point by type of test Consider 11 types of test case o Each type has a weight o Adjust TCP using the weight Use functional test cases as baseline with the weight of 1.0 12
  • Checkpoint Checkpoint Is the condition in which the tester verifies whether the result produced by the target function matches the expected criterion One test case consists of one or many checkpoints Counting rule 1 One checkpoint is counted as one Test Case Point 13
  • Test Case Precondition Precondition specifies the condition to execute the test case mainly affects the cost to execute the test case may be related to data prepared for the test case Four levels of complexity None Low Medium High 14
  • Test Case Precondition (contd) None The precondition is not applicable or important to execute the test case Or, the precondition is just reused from the previous test case to continue the current test case Low The condition for executing the test case is available with some simple modifications required Or, some simple set-up steps are needed Medium Some explicit preparation is needed to execute the test case The condition for executing is available with some additional modifications required Or, some additional set-up steps are needed High Heavy hardware and/or software configurations are needed to execute the test case 15
  • Test Case Precondition (contd) Counting rule 2A: Counting Unadjusted Test Case Points for Precondition: Each complexity level of precondition is assigned a number of Test Case Points Based on our survey of 18 senior QA engineers 16
  • Test Data Test Data used to execute the test case can be generated at the test case execution time, sourced from previous tests, or generated by test scripts Test data is test case specific, or general to a group of test cases, or for the whole system Four levels of complexity None Low Medium High 17
  • Test Data (contd) None No test data preparation is needed Low Simple test data is needed and can be created during the test case execution time Or, the test case uses a slightly modified version of existing test data and requires little or no effort to modify the test data Medium Test data is deliberately prepared in advance with extra effort to ensure its completeness, comprehensiveness, and consistency High Test data is prepared in advance with considerable effort to ensure its completeness, comprehensiveness, and consistency This could include using support tools to generate data and a database to store and manage test data Scripts may be required to generate test data 18
  • Test Data (contd) Counting rule 2B: Counting Unadjusted Test Case Points for Test Data: Each complexity level of Test Data is assigned a number of Test Case Points Based on our survey of 18 senior QA engineers 19
  • Adjust Test Case Point Test Case Point counted till this point is considered Unadjusted Test Case Point (UTCP) UTCP is adjusted by considering types of test case Each type of test case is assigned a weight Adjusted Test Case Point (ATCP): n ATCP = UTCPi * Wi i=1 UTCPi - the number of UTCP counted for the test case ith. Wi - the weight of the test case ith, taking into account its test type 20
  • Weight by Type of Test Based on our survey of 18 senior QA engineers 21
  • Weight by Type of Test (contd) 22
  • Test Case Point Analysis - Summary Process overview Count Checkpoints DetermineTest Case UTCP Adjust with TCP Precondition Test Type Complexity Determine Test Data Complexity 23
  • Agenda Objectives Background and Motivation Existing Estimation Methods Test Case Point Analysis Effort Estimation Methods using TCP Conclusion and Future Work 24
  • Estimate Testing Effort Estimate testing effort using TCP Test effort distribution, four phases Test Planning Test Analysis and Design Each of these phases may be performed multiple times Test Execution Test Tracking and Reporting Effort estimation methods Productivity index Regression 25
  • Estimate Testing Effort (contd) When estimating TCP, assume that each phase is performed ONCE Effort is estimated dependent on how many times a phase is performed 26
  • Productivity Index Effort is computed using productivity index of completed project Productivity index is measured as person-month per TCP Effor