testing metrics - making your tests visible

TESTING METRICSMAKING YOUR TESTS

VISIBLEAlper Mermer

WHY WOULD YOU CARE ABOUT METRICS ?

Make your team aware of the trends

WHY WOULD YOU CARE ABOUT METRICS ?

Share the pain

WHY WOULD YOU CARE ABOUT METRICS ?

Watch production

WHY WOULD YOU CARE ABOUT METRICS ?

Improve ownership

WHY WOULD YOU CARE ABOUT METRICS ?

Prevent issues

WHY WOULD YOU CARE ABOUT METRICS ?

Not all metrics are needed in every project

WHICH METRICS SHOULD YOU TRACK ?

Measure every service performance seperately

Define sets of scenarios to run in every build

Performance Measurements Through Time

WHICH METRICS SHOULD YOU TRACK ?

Build mechanisms to warn about unexpected variances

in servicesWatch overall system performance as well

Keep an eye on trends

Performance Measurements Through Time

WHICH METRICS SHOULD YOU TRACK ?

Break builds with high security risks

For lesser security risks, watch the trend

Number Of Security Warnings

WHICH METRICS SHOULD YOU TRACK ?

Test coverage (?)Cyclomatic Complexity

Code Quality Metrics

WHICH METRICS SHOULD YOU TRACK ?

Depth of InheritanceDistance from Main

Code Quality Metrics

WHICH METRICS SHOULD YOU TRACK ?

Tests that cover your requirements are listed hereIncreased time in Run Time affects feedback time

Serves as a motivation for testers

API + GUI Test Count And Run Time

WHICH METRICS SHOULD YOU TRACK ?

Should help to increase focus on quality

Decreasing number of higher level bugs boosts

up morale

Defects By Priority/Severity

WHICH METRICS SHOULD YOU TRACK ?

Lesser number of bugs increases the confidence in the product

An increase in the number of higher level bugs should start an

investigation

Defects By Priority/Severity

WHICH METRICS SHOULD YOU TRACK ?

Trend in number of defects from production is importantIt affects product and company image

Should be closely monitored and analyzed

Number of Defects From Production

WHICH METRICS SHOULD YOU TRACK ?

Serves as an indicator of problematic areasShould initiate discussions around prioritization

Defects From Production by Module

WHICH METRICS SHOULD YOU TRACK ?

A team internal metricA nice indicator of your testing process

It is expected to be very high in a mature shift left testing culture

Mean Time Between Build Failures

LAST WORDS

Metrics Are There to Help Your Brain, Not Replace It

THANK YOUQuestions ?

amermer@thoughtworks.com

@alpermermer

@test_hive