implementation and test derived from dr. fawcett’s slides phil pratt-szeliga cse 784 fall 2009
Post on 21-Jan-2016
214 Views
Preview:
TRANSCRIPT
Implementation and Test
Derived from Dr. Fawcett’s SlidesPhil Pratt-SzeligaCSE 784Fall 2009
Overview
Good Neighbor PolicyChanges to codeProving CorrectnessTestingDebugging
Good Neighbor Policy
When designing programs remember you do not have exclusive access to: Memory, Disk space, Screen formats I/O channel formats, peripherals public names, files, database locks operating system mutexes, semaphores, handles communication lines and environment variables
Release system resources as soon as you can. Don’t use more than you need
Assign memory and files dynamically when you need them and release them when done
If you change iostream formats (C++) save the original and restore to the original quickly.
Use namespaces to reduce the names in the public domain
Changes
If you modify functions in a server’s public interface You break the design of clients requiring redesign,
implementation, building and test If you modify the internal private implementation of a
server or add public functions You break the code of clients requiring recompilation
If you modify the private implementation of a class that clients use through an abstract interface You break the build of every client requiring relinking
If you modify the private implementation of a module packaged as a dynamic link library and exposing only public interfaces You break nothing, just copy the dll into the directory of any
component that needs the service
Proving Correctness
Definition – Correctness Ability of a program to perform exactly in accord to
its specifications If a component’s inputs satisfy a given set of
preconditions then a correct program guarantees its outputs satisfy its specification
To prove a function correct we verify two assertions The function calls only correct functions The current function meets its specifications when
preconditions are met
Inspections
Code Inspections To verify correctness of a component we need:
Specification of processing Required preconditions Statement of side-effects
Support for inspections Each module shall have a prologue that provides a
brief description of the module’s operations Each function in each module shall have a
prologue: Processing sections A list of inputs and preconditions A list of outputs and global data effected
Function Inspection Process
Enumerate all possible states of a function For each state determine if the function works
correctly If a state is found that the function is
confusingly complex, the function is not inspectable and the inspection is done
If a state is found that the function is incorrect, the inspection is done
If an invoked function is incorrect or not inspectable, so is the invoker
If all states pass then the function passes
White Box and Black Box Testing
White Box Testing Test implementation, showing that each block of
code operates as the designer intends Black Box Testing
Test requirements independent of implementation
White Box Tests:
Construction Tests
Unit Tests
Black Box Tests
Performance Tests
Validation Tests
Qualification Tests
Regression Tests
Integration Tests
White Box Testing
For each function in a component: Every program statement is executed at least once (100%
coverage) Each relational expression is tested for true and false values Every boundary condition is tested within and at the boundary State changes are predicted and verified All errors are predicted and error handling tested
Functions that are small, functionally cohesive and have narrow coupling are much easier to test
This is not exhaustive testing. In exhaustive testing every combination of paths within the function is exercised
White Box Tests
Construction Preliminary testing as component is being built Purpose is to prepare for unit testing
Unit Full white box testing as described in previous slide Purpose is to test that the module is correct
Integration Part white box and part black box Purpose is to ensure components operate together
Unit Test Procedure
•Test Name
•Author
•Preconditions
•Derived Requirements
•Test Procedure
•Expected Results
•Actual code
Black Box Tests
Integration Tests Part white box part black box Demonstrates that major blocks satisfy requirements
allocated to them (Helps debug qualification tests) Validation Tests
Does the system crash? Instrument the system to show that all inputs within
specifications produce outputs within specifications Inputs out of spec should produce error messages
Performance Tests Build test drivers and instrumentation software that exercises
timing key threads Analyze memory and disk usage
Black Box Tests
Qualification TestsStep by step demonstration that each A and
B spec requirement is metPass/fail criteria developed for each
requirement and reviewed with customerTest data notebook maintained throughout
test. Each test is not complete until the customer signs the notebook for that test
Test data notebook is supplied to the customer (one of a kind products)
Black Box Tests
Qualification Tests Inspection – Visually inspect documentation, source code.
“shall use the C# programming language”
Demonstration – Run software normally. “shall provide operator with a command menu for selections”
Analysis – Collect test data and analyze offline. “shall report 99.999% of targets”
Test – Instrument the software and provide special test inputs.
“shall initialize the system and enter operational mode within 30 seconds”
Qualification Test Procedures
Look at Qualification Test Procedure Handout
Black Box Tests (continued)
Regression TestsDoes it still work?Summary level tests to give quick
verification that a previously tested software product still works:
After minor code changesAfter changes in the platformAt the beginning of a new maintenance phase
Often used to substitute for re-qualification after changes are made to meet specifications during qualification
Bugs
You need to care about bugs for two reasons: In the short term, customers contact you for help, forcing you
to spend time and money on the current product instead of working on new ones
In the long term, customers start buying alternatives to your buggy product
What are bugs? Inconsistent user interface Poor performance Unmet specifications Crashes or data corruption
(Debugging Applications, John Robbins, Microsoft Press)
Bugs
Common sources of bugs:Short or impossible schedules“Code first, think later”Misunderstood requirementsEngineer ignorance or improper trainingLack of commitment to quality
Debugging Advice Debugging is hard, so learn from your mistakes Some language syntax is prone to errors, so minimize their use
Gotos Global variables Void * pointers Automatic Type Conversions
Explain your code to someone else Make the bug reproducible
Don’t start intensive debugging until you can reliably reproduce the problem Divide and conquer Display output to localize your search Write self-checking code Write to a log file Keep records
If the search is long, without records you will forget what you have already done
(from debugging book #1)
Debugging Books
The Practice of Programming, Kernighan & Pike, Addison-Wesley, 1999
Debugging Applications for Microsoft .Net and Windows, John Robbins, Microsoft Press, 2003
Unit Tests
John Robin’s advice for unit tests is as follows: Only the paranoid survive “I have many friends who are excellent engineers,
but when it comes to having them interface with my code, I verify their data down to the last bit”
“…I even have a healthy skepticism about myself” “Assertions, tracing and commenting are how I start
verifying my fellow developers who are calling my code”
“Unit testing is how I verify myself” Unit test drivers are scaffolds you use to call your
code outside the normal program environment
Unit Tests (continued)
Start writing unit tests as soon as you start writing code
Test each incremental change in isolation and spread test development over the whole code development phase
Think about how you are going to test your code before you write it
“While you are coding you should be running your unit tests all the time”
The key to effective unit tests is Code Coverage – the percent of lines you’ve executed during test
“The simple fact is that a line not been tested is a line waiting to crash”
End of presentation
top related