test specification

Test Specification

Sabesan

Info

rmat

ions

tekn

olog

i

Institutionen för informationsteknologi | www.it.uu.se

Why test? Improve quality

Find bugs (no crashes, erroneous calc) Verify the requirements?

To put it another way: Testing is a verification process to improve

the confidence that the system conforms to a specification

Your job is to “prove” that the specification is implemented correct

Info

rmat

ions

tekn

olog

i


Fulfill requirements

Think of how to test each requirement Some requirements should be considered

in all test cases (e.g. write to output file) Others have specific tests Nothing unexpected (if spec. is not open)

Info

rmat

ions

tekn

olog

i


Writing a test case Numbered Refers to requirement(s)

Prove valid test case (against spec) Shows that all requirements are addressed (Suite) Tip: A requirement that would have generated an

error earlier does not need to be mentioned over and over again

Explains HOW to execute the test For dummies, include all required info Explains the in-parameters and the expected output.

User (or script) able to determine the verdict PASS or FAIL

Info

rmat

ions

tekn

olog

i


Reference

The purpose of the Test Specification is to verify a system according to a referenced Requirement Specification, i.e., add VERSION number of the specification

Most of the formalism as RS

Info

rmat

ions

tekn

olog

i


Test Specification A. Part 2

Test case A.1 • Tested requirements:

- Req. A.3.10 *) (boundary testing: value inside the range) - Req. A.3.14 *) (boundary testing: value inside the range) - Req. A.3.15 *) (boundary testing: value inside the range) - Req. A.1.27 *) mainly targeted requirements for this test case

• Test data: - Change to a directory where you have write permissions. For example: cd ~/ - Create a simple XML-Input file containing a camera (location:[0,0,0] sky:[0,1,0] look_at:[0,0,1]) and a red sphere (radius: 1.2 location:[0,0,12] surface:pigment:color[1,0,0]) and name the file “simple_scene1.xml”. - Write the following command to the shell: tracer -z 500 -x 500 -y 500 -w 1 -h 1 test_a1.ppm <simple_scene1.xml - Write the following command to the shell to view the output picture: xview test_a1.ppm • Expected result: - The program should terminate without error message.

Info

rmat

ions

tekn

olog

i


Tests for Non-Functional Req How to “test” a req. “Code should be

written in a nice and structured way”? Refer to code review

Verdict PASS if code review PASS If code not available write N/A “not applicable”

This should be in the Test Plan (not in TS) How to “test” a req. “Use getopt lib”

Query supplier e.g. ask developer/manager This is a simple YES/NO question

Info

rmat

ions

tekn

olog

i


Testing limitations

A crashed program is “not an option”. Parameter range e.g. if the specified

range is [1..1000] this gives the tests Inside range close to bound, i.e., 1 and1000 Common or other value picked “random”

inside the range, e.g., 100 Outside range close to bound, i.e., 0 and

1001 WHAT will happen here…

Info

rmat

ions

tekn

olog

i


Combinations You cannot test all cases, e.g.,

1000*1000*1000 values Vary upper and lower bound Vary object hidden each other (partly or fully) Vary object not in view of the camera You can discuss functionality with you MP2

group, e.g., how the camera works or how the XML file works, but NOT the test cases

Info

rmat

ions

tekn

olog

i


Discussion

Is it good to slim the test suite?

Info

rmat

ions

tekn

olog

i


Equivalent tests

For function “f(x)” Test f=(0 if x=0),(x if 0<x<10), and (0 if x=10) If range for x is [0..10]

f(0) f(some in [1..9]) f(10) must be tested Classes {0} {1,...,9}{10}

Is f() affected by something else? Can we suspect problems if something

else changes?

Info

rmat

ions

tekn

olog

i


Report

Test Verdict Comment

1 PASS

2 FAIL No error mess

Info

rmat

ions

tekn

olog

i


Report

Which system is tested (code version) Who has tested Which test specification used (version) RS

test specification

Documents