Download - Testing 2007
-
8/9/2019 Testing 2007
1/8
CO531: Software Engineering Practice:
Testing
or some of the things you should
know about testing but never knew
you had to ask!
Books
Amazon returns 400+ hits for softwaretesting in the title
Recommended:How to Break SoftwarebyJames A. Whittaker, Addison Wesley, 2003.Price: 25.19 (from Amazon)
Google returns about 175,000,000 hits to asearch on software testing
Truth vs Anecdotes?
Testing
Why do we do it?
Because we aint perfect
When do we do it? All the time
How do we do it?
Systematically and repeatedly
When do we stop doing it?
When the software isnt used any longer
Simple Example
CO522 Example class
p(x) = 4096x6 36864x5 + 129280x4
222720x3 + 194224x2 78096x + 10395
Did anyone test their calculation method?
How did/do you test this?
Balance?
-
8/9/2019 Testing 2007
2/8
Exhaustive testing
Exhaustively test a method that takes two
integer parameters. Assume
32-bit integers
runs a test every 50 nanoseconds (10-9 secs)
How long will this take?
Exhaustive testing .
is generally not an option.
General Introduction
More next term
What is the point of testing?
XP is very hot on testing
What does testing actually tell you?
What cant testing tell you?
Confidence building but dont forget
that Oh fcuk moment
Dijkstra says
Program testing can be used to show the presenceof bugs, but never to show their absence!
You must not give the world what it asks for, butwhat it needs.
We must not put mistakes into programs becauseof sloppyness, we have to do it systematically andwith care.
Object-oriented programming is an exceptionallybad idea which could only have originated inCalifornia.
-
8/9/2019 Testing 2007
3/8
Errors --Nomenclature
looking for bugs, errors, defects, faults,
failures, features, showstoppers
lost marks, wrong answers, dead people,
crashed aircraft, no passports, blue screens,
any of the above (Microsoft)
Debugging fixing it safely!
Removal cost increases with time
Why/How Do Errors/Bugs Occur
Designed/written/tested/used by
fallible/malicious people.
You made a silly error a simple oversight
Youre being pushed to complete a project
Unforeseeable circumstances occur
Difficult/impossible to test some
circumstances for real
Types of Testing
Just talking about testing code
Unit testing vs System testing
White/Glass box, grey box, black box
Iterative testing vs Big bang testing
-
8/9/2019 Testing 2007
4/8
Story
Understand
Add one failing test
Add production
code for that test
Run ALL tests
Rework OK Story
Complete
Next
Story
Never write a line of
functional code without
a broken test. Kent Beck
The Test First (TDD) Scheme
Why Test-First
Feedback: instant on code changes
Task orientation: focuses on problem
decomposition, maintains focus, provides
steady and measurable progress
Quality: built-in and maintained via testing
Low-level design: context for decisions
(classes/methods, names, interfaces, )
Advocates and Sceptics
Advocates: increased productivity andbetter quality
Sceptics: Hard to learn, counter-productive Quality: Interleaving tests with coding
Productivity: Not so obvious
Studies: inconclusive
Other factors: team dynamics, self-esteem,courage
What else is it?
Test first coding is not a testing technique
Ward Cunningham (being provocative!)
Analysis technique
Design technique
and, I believe, its also a testing technique!
-
8/9/2019 Testing 2007
5/8
Unit Testing
Usually tests just one particular method
Need confidence that low-level routines workbefore adding high level code
Not concerned with formal validation of code
Still need other forms of testing
Wish to avoid the ripple effect
Good unit testing
Better code design
Less time in the debugger
Unit Testing contd
Answers the question Is the code doing exactly what I think it should?
Note that this isnt the question Does the code do what the requirements demand?
You need to ensure your code does what you want ALLTHE TIME Regression tests all tests must pass NOT JUST THE NEW
ONES
Unit testing helps with documentation Tests are visible
Test are up-to-date and correct (because you keep running them!)
Code documentation (on the other hand) tends to drift
Unit Testing
Testing isnt free but ...
Invest up-front to save time later
Typical excuses It takes too long to write tests
It takes too long to run the tests
Its not my job to test my code
I dont know what the code is supposed to do so I canttest it
I feel guilty putting testers and QA staff out of work
Im not allowed to run unit tests on a live system
JUnit
Nice to have help with unit tests
BlueJ provides a neat interface to the JUnit
testing framework. Dont have to write drivers
You have to know the answer to each test
You can build the tests up
You can run all the tests easily (and batches ifyou want to do a bit more work.
-
8/9/2019 Testing 2007
6/8
Unit Tests vs Function Tests
Unit tests are local
Include integration tests
Written by (and for) programmers
Function testing = higher order testing
Does code do what the user expects?
XP has continual customer involvement
Iterative function testing = minimal surprises
Done by end user (as well as programmers)
Communication
Others: stress, performance, security, installability
Generating Test Cases
Problem dependent -- experience
Some general rules
equivalence classes one example covers many
edge cases what happens at the limits (< |
-
8/9/2019 Testing 2007
7/8
Basic Guide to Debugging
1. Recognise that there is a defect inyourcode
Check input data (program should already do this?)
Reproducibility
Identify the symptoms
2. Isolate the source of the problem
What part of the system is causing the problem
Check inputs and outputs over smaller parts of the
system
Experience counts here
Basic Guide (contd.)
3. Why has the problem occurred?
Incorrect code may be a long way from the problemsappearance (code distance/time)
E.g., incorrect value generated but problem notdetected until it is used much later
Are there similar sections of code elsewhere?
4. Determine the fix
Sometimes simple one line
Sometimes catastrophic major rewrite
Quick fix may be necessary
Basic Guide (contd.)
5. Fix, TEST, TEST and TEST
Apply the fix
Check it does correct the problem (TEST)
Check nothing else breaks (TEST)
Add new test (which one???) to regression
(and unit ??) tests
Useful things to do
1. Have the correct mind-set Your code probably is broken
2. Add assert/debug statements into code so they can be
activated when necessary3. Use log files
4. Only ever make one substantial source change at a time Helps prevent adding new bugs
5. Back out of changes that dont fix the problem Problem not where you think it is
Modified code never called
Use a source code control system (Subversion/CVS)
-
8/9/2019 Testing 2007
8/8
Do I Only Add Failing Tests
Never write a line of functional code
without a broken test. Kent Beck.
Doesnt say I cant add tests that dont fail!
Add test if
they show a failure (need to write code)
there is something special about the test
Document all tests
Testing GUIs
Is a form of function testing
XP approach is not as successful (Kent
Beck says so so it must be true)
Its hard to do
Its hard to automate
Screen shots are worth a thousand words.
Whittakers Book
Read his book for a really useful list of
things to try when you perform both unit
and function testing.
Try some of these attacks out on your
project code.