development testing
TRANSCRIPT
Development testing Yury Kisliak @ EIS Group
- What is testing- Kinds of testing- Kinds of coverage- Asserts
What is testing?
What is testing?
What is testing?
What is testing?
What is testing?
What is testing?
What is testing?
Good testing consist of selecting a good set of test inputs and designing a good acceptability checks.
What is going on when we test?
What is going on when we test?
-A lot!
What is going on when we test?
What is going on when we test?
What is going on when we test?
What is going on when we test?
Why testing?
- Testing shows presence of defects (but cannot prove that there are no defects).
- Appropriate testing reduces the probability for defects.
- What is testing- Kinds of testing- Kinds of coverage- Asserts
Types of testinghttps://en.wikipedia.org/wiki/Software_testing#Testing_types
Types of testinghttps://en.wikipedia.org/wiki/Software_testing#Testing_types
Testing Levels
Testing Levels
Testing Levels
Testing Methods
Testing Methods
Static
- Reviews, walkthroughs, inspections.
- Static program analysis.
Testing Methods
Static
- Reviews, walkthroughs, inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a given set of test cases
Testing Methods
Static
- Reviews, walkthroughs, inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a given set of test cases
Black box
- Examining functionality without any knowledge of internal implementation.
White box
- Internal perspective of the system, as well as programming skills, are used to design test cases.
Testing Methods
Static
- Reviews, walkthroughs, inspections.
- Static program analysis.
Dynamic
- Executing programmed code with a given set of test cases
Black box
- Examining functionality without any knowledge of internal implementation.
White box
- Internal perspective of the system, as well as programming skills, are used to design test cases.
- What is testing- Kinds of testing- Kinds of coverage- Asserts
Coverage? What coverage?
Coverage? What coverage?
Coverage? What coverage?
Coverage? What coverage?
Coverage? What coverage?
Equivalence partitioning
Equivalence partitioning
Equivalence partitioning
Equivalence partitioning
Test Coverage
Test Coverage
Test Coverage
Test Coverage
Test Coverage - is proportion of a program, exercised during testing.
+ gives objective score
+ when coverage is <100%, we are given a meaningful task
- not god in finding “errors of omission”
- difficult to interpret scores <100%
- 100% coverage does mean that all bugs are found
Coverage metrics
Coverage metricsHow many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
Coverage metricsHow many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Coverage metricsStatement (line) coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Coverage metricsStatement (line) coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
100%
Coverage metricsStatement (line) coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
100%
Coverage metricsStatement (line) coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
100% 50%
Coverage metricsBranch coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Coverage metricsBranch coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
50%
Coverage metricsBranch coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
50% 50%
Coverage metricsBranch coverage: How many coverage metrics
are out there?- A lot!
http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
50% 50%+ = 100%
Coverage metrics ✓ Line coverage Branch coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Loop coverage:
Execute each loop zero, one and more than one times.
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
a, !b, c => true
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
a, !b, c => true!a, !b, c => false
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
a, !b, c => true!a, !b, c => false!a, b, c => true
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
a, !b, c => true!a, !b, c => false!a, b, c => true!a, b, !c => false
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Modified condition/decision coverage:
MC/DC = branch coverage + conditions takes all possible values + each condition independently affects the outcome of a decision
a, !b, c => true!a, !b, c => false!a, b, c => true!a, b, !c => false
MC/DC is used in avionics software to ensure adequate testing of the most critical (flight and landing related) parts.
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Path coverage:
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Path coverage:
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Path coverage:
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
Path coverage:
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓ Path coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
What else:
Coverage metrics ✓ Line coverage Branch coverage✓ Loop coverage✓ MC/DC✓ Path coverage✓
How many coverage metrics are out there?
- A lot!http://www.badsoftware.com/coverage.htm (list of 101 coverage metrics)
How many of them do we need to care about?
- Very few.
What else:
- boundary value coverage- synchronization coverage- … other 94 and more coverages
How to use coverage
How to use coverage
How much testing is enough?
Tools
Tools
- What is testing- Kinds of testing- Kinds of coverage- Asserts
Assertions
Assertions
- makes code self-checking, leading to more effective testing
- make code fail early
- document assumptions, preconditions, postconditions and invariants.
- are not for error handling
- must not have side effects
Assertions
- makes code self-checking, leading to more effective testing
- make code fail early
- document assumptions, preconditions, postconditions and invariants.
- are not for error handling
- must not have side effects
Use error handling code for conditions you expect to occur. Use assertions for conditions that should never occur.
Assertions
- makes code self-checking, leading to more effective testing
- make code fail early
- document assumptions, preconditions, postconditions and invariants.
- are not for error handling
- must not have side effects
Use error handling code for conditions you expect to occur. Use assertions for conditions that should never occur.
The end.
Testing by itself does not improve software quality. Test results are an indicator of quality, but in and of themselves they don’t improve it. Trying to improve software quality by increasing the amount of testing is like trying to lose weight by weighing yourself more often. What you eat before you step on to the scale determines how much you will weigh, and the software-develompent techniques you use determine how many errors testing will find. If you want to lose weight, don’t buy a new scale; change your diet. If you want to improve your software, don’t just test more; develop better.
-Steve McConnell