test smarter: efficient coverage metrics that won't leave you exposed
TRANSCRIPT
Efficient Coverage Metrics That Won’t Leave You Exposed
Who are we?
2
Nikhil Kaul Rick Almeida
On our plate today
3
• Why use coverage metrics?
• Different types of coverage metrics
• Where these coverage metrics may go wrong
• Coverage goals and where to start?
Coverage metrics can come in handy in a variety of ways
4
1 Areas not covered by tests
Indirect measure of quality 2
3
Identifying redundant test cases
Variety of coverage metrics are available to achieve these results
5
Coverage Metrics
Statement Branch Path
Let’s start with Statement Coverage
6
No of statements run Statement Coverage = -------------------------------- * 100
Total number of statements
Statement Coverage
Branch Coverage
Path
Coverage
Source Code
100% statement coverage doesn’t mean effective testing
7
Statement Coverage
Branch Coverage
Path
Coverage
TestCase
Statement Coverage
100%
Statement coverage comes with other challenges
8
Terminal condition Do-While Switch Statements
Statement Coverage
Branch Coverage
Path
Coverage
Branch Coverage
9
Statement Coverage
Branch Coverage
Path
Coverage
• Each branch (if statements, loops) has been executed at least once
• Every “if statement” has two distict branches
• No branches lead to abnormal behavior
Source Code
To cover 100% of branches, we need one more test case
10
Statement Coverage
Branch Coverage
Path
Coverage
TestCase
Statement Coverage
100%
Branch Coverage
50%
However, branch coverage can ignore short circuit operators
11
Statement Coverage
Branch Coverage
Path
Coverage
No call to function1
Test Case 1: Condition 1 = true Condition 2 = true
Test Case 2: Condition 1 = false Condition 2 = false
Next comes path coverage
12
Statement Coverage
Branch Coverage
Path
Coverage
T F
T F T F
1 2 3 4
Source Code
50% path coverage with TT and FF
13
Statement Coverage
Branch Coverage
Path
Coverage
TestCases
Branch Coverage
100%
Path Coverage
50%
You are missing 2 paths
14
Statement Coverage
Branch Coverage
Path
Coverage
T F
T F T F
1 2 3 4
Path coverage is not always optimal
15
Statement Coverage
Branch Coverage
Path
Coverage
Exponential increase in branches with added conditionals
End up creating extra test cases
Where to start with coverage
16
Find evident issues early and often
Don’t focus on hitting high coverage metrics for individual parts of you app
Test some of each part of you application
What to do once you get started?
17
Call at least one method in every file or class
Attain 80% to 90% branch coverage in each method
Invoke 8 out of 10 methods, ignoring branch coverage
TestComplete for automated functional testing
18
AQTime for code coverage
Optimizes code for developers & testers
Supports multiple languages & frameworks Integrates with IDEs and TestComplete
DEMO
20
QUESTIONS?
21