March 25, 2004

Announcements Friday – recruiting event

12-6 Dean Smith Center Bloomberg LP, New York City full-time positions and summer internships programmers C/C++ and UNIX

New section of 190 next fall: Logic and Automated Theorem Proving Prof. Plaisted

Final Presentations Logistics

11 presentations @ 20 minutes Need 4 hours Exam scheduled for noon on April

27 Propose that we run from 11:30 –

3:30 Problems?

Reminder: Data to Collect

Your expertise level for (3 point scale) Language and platform Technology

Time spent Education Meetings Code and test Documentation

U.S. Students Shun Computer Sciences San Jose Mercury News (03/24/04) CRA’s Taulbee Survey

U.S. undergraduate enrollment in CS fields declined by 19 percent in 02-03

newly declared CS majors dropped 23 percent

concern about offshore outsourcing lowered career expectations

"most of the solutions to the complex problems that companies need to solve haven't been written yet…

that's what we should be preparing our students to do"

Four Parts of Testing

Model Select test cases Execute test cases Measure

Model

Basic Software Model

capabilities

environment

User interfaces

APIs

Operating system

Files

Input

Output

Storage

Processing

Test Case Selection

From the User Interface: Inputs

Error messages Default values Character sets and data types Overflow input buffers Input interactions Repeated inputs

From the User Interface: Outputs Concept: What inputs create interesting

outputs? REQUIRES DOMAIN EXPERTISE

State-based outputs Invalid outputs

Changeable outputs Screen refreshes

Bubble diagrams: Reverse state exploration

Define a failure state What would have happened to get you

there? Repeat

Find a way to force it down that path

Let’s try it: remote access keys

Capabilities – Storage and Processing

Same input, different initial conditions Too many or too few values in a data

structure Alternative ways to modify constraints Invalid operator/operand combinations Recursive functions Overflow/underflow computations Feature interaction

How to select specific cases

Data based Boundary conditions Equivalence classes

Control based State transitions

Let’s try it

Word PowerPoint calculator

Execution

Execution tools

Automation – often primarily scripts

Critical for regression testing GUI tools are abundant, but

marginal

Measurement

Test Coverage Metrics Statement coverage

basic block coverage Decision coverage

Each Boolean expression Condition coverage

Each entity in Boolean expressions Path coverage

loops

Advantages of different models?

Estimating how many bugs are left

Historical data Capture-recapture model from

biology

References

Whittaker, How to Break Software

Kaner, The Impossibility of Complete Testing at www.kaner.com (Articles)

Your Test Plan and Testing

Unit System User