[hcmc stc jan 2015] making it count – agile test metrics

25

Upload: ho-chi-minh-city-software-testing-club

Post on 17-Jul-2015

545 views

Category:

Technology


1 download

TRANSCRIPT

Row Labels Count of Priority Note:

Blocker 13 until approx. end-May 2014,

Critical 59 Major was Priority default

Major 141

Minor 125

Trivial 17

Grand Total 355

Row Labels Count of Urgency Note 1:

Low [must be if Deferred or Won't Fix] 17 Medium is Urgency default

Medium 196

Soon - High 108 Note 2:

Urgent [must be if Pri=Blocker] 34 24 bugs don't have Urgency,

Grand Total 355 opend before it became

mandatory

4%

16%

40%

35%

5%

Blocker

Critical

Major

Minor

Trivial

Priority

Count of Priority

5%

55%

30%

10%

Low [must be if Deferred orWon't Fix]

Medium

Soon - High

Urgent [must be if Pri=Blocker]

Urgency

Count of Urgency

Note 1:

Some defects,

presumably

created before

it became mandatory,

don’t have a value

Note 2:

Code dev …

is the default for

Fault Source

1%

0% 1%

1%

3%

0%2%

3%

7%

1%

65%

4%

0%4%

0%

5%

2%0%

Resolution, Weighted, by Found During

Cannot Reproduce Story / Systemtest (by QCA)Duplicate CIS internal Acceptancetest (by/for PO)Duplicate Story / System test (byQCA)Duplicate Walkthrough

Fixed CIS internal Acceptance test(by/for PO)Fixed Code review

Fixed Customer Acceptance Test

Fixed Integration test (by QCA)

Fixed Production

Fixed Regression test

Fixed Story / System test (by QCA)

Fixed Walkthrough

Rejected - Test Error Production

Rejected - Test Error Story / Systemtest (by QCA)Won't Fix Regression test

Resolution

Found During

Sum of Weighting

73%

3%

1%

11%

2%

2%

0% 0%

4%

1%

2%

0%

1%

Fault Source, Weighted

Code development

Configuration parameter

Deployment process

No real fault; Bug rejected

Production data

Testing - Data

Testing - Environment

Fault Source

Sum of Weighting

84%

0%

1%

2%

13%

Quality Char., Weighted

Functionality

Integration - interfaces with othersystems

Performance (response orthroughput) / resource efficiency (e.g.

memory leak, CPU or disk usage)

Security (authentication,authorisation, penetration etc.)

Usability (ease of use, appearance,on-line help, user guide etc.)

Quality Characteristic

Sum of Weighting

76%

4%

1% 7%

2% 2%

0%

0%4%

1%

2%0%

1%

Fault Source, Weighted

Code development

Configuration parameter

Deployment process

No real fault; Bug rejected

Production data

Testing - Data

Testing - Environment

(blank)

Fault Source

Sum of Weighting

2%

85%

8%

3% 2% Error Type, Weighted

Ambiguity - spec had >1way to interpret

Inaccuracy - code / spec /parameter value incorrect

No real fault; Bug rejected

Omission - s/thing left outof code / spec / config.

(blank)

Error Type

Sum of Weighting

3% 0%3%

8%

77%

4%1%

4%

Found During, Weighted

CIS internal Acceptance test(by/for PO)Code review

Customer Acceptance Test

Production

Story / System test (by QCA)

Walkthrough

Regression test

Integration test (by QCA)

Found During

Sum of Weighting

1%

8% 0%

43%

4%

4%

3%

3%

1%

5%

5%

2%0%

0%

0%

1%

1%

0%

0%

1% 0%

0%

0%

0%

1%

1%

1%

1%0%

1%

0%0%

1%

1%

0%

0%

0% 1%

Components, Weighted CIPA

Monitoring

Translations

(blank)

Manual Editor

Back Office

Credit Reports

Batch Processing

Statistics

Data Upload

Searching

Administration

Back Office, Matching

Component/sSum of Weighting

Product A Release 1 82%

Product A Release 2 87%

Product A Release 3 84%

Product B Release 1 91%

Product B Release 1 96%

Product C Release 1 86%

Product C Release 2 79%

Product C Release 3 92%

%