Kaspar Loog

kaspar.loog@knowit.ee

”Testing is execution of software in order to show that it doesn’t work” - Glenford Myers, 1979

“Testing is measuring quality” - William Hetzel, 1988

Testing reduces our risks

2010-11-22 2

o

”Measure” quality (”Testing is to knowing everything”)

Release the right product in time

NOT to lose money

Stay in the market

2010-11-22 4

o

Enhance, and help the development of, the product

Verifiy requirements/functionality

Measure the Quality of the Product

2010-11-22 6

2010-11-22 7

Many defects found Few defects found

Few defects found Few defects found

High

High Low

Low

Test Quality

SW Quality

You think you are here

..but you are here

o

”Testing is expensive” ◦ Manhours for test

◦ Test environment

Compared to: ◦ What does it cost to recall a release?

◦ What does it cost to get unhappy customers?

◦ What does it cost to lose market volume because the quality is too poor?

◦ etc...

2010-11-22 8

o

Economical

Communication

Ethical

2010-11-22 9

Exhaustive-testning

What is Exhaustive-testning?

◦ We test all combinations of in parameters, prerequisites and environments.

How long time does it take? (Yes/No)

◦ Infinitely long time

◦ Short time

◦ Impractically long time

2010-11-22 10

o

It depends of the risk of:

◦ Miss serious defects

◦ The defects we will find are expensive to correct.

◦ Need to recall a release.

◦ Releasing bad code

◦ Losing market volume

◦ Not be first on the market

◦ Execute effective testing

2010-11-22 11

How is risk normally defined?

Probability * Impact

Risk = S * K o

2010-11-22 12

Prioritise the testing so that

whenever you stop testing

you have done the best

possible testing on the given

time.

o

ISO/IEC 12119 ◦ Information technology - Software packages -

Quality requirements and testing

ISO/IEC 14598 ◦ Information technology - Software product

evaluation

IEEE Std 829 ◦ Standard for test documentation

SIGIST ◦ Specialist Interest Group in Software Testing

ISTQB ◦ International Software Testing Qualifications Board

2010-11-22 13

o

Verification

Validation

Testing

Bug

Error, Defect, Failure

Reliability

2010-11-22 14

o

”The process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of the phase”

[ BS 7925-1]

2010-11-22 15

That the system is correct

according to the written

requirements.

o

”Determination of the correctnesss of the products of software development with respect to the user needs and requirements”

[ BS 7925-1]

2010-11-22 16

That the system is correct

to the real requirements.

o

”The process of exercising software to verify that it satisfies specifed requirements and to detect defects”

[ BS 7925-1]

2010-11-22 17

o

2010-11-22 18

o

2010-11-22 19

A person does a

programming error.

ERROR

The Error creates

a defect in the code.

DEFECT

The defect can create a

unwanted reaction.

FAILURE

2010-11-22 20

Test closure activities

Evaluating exit criteria and reporting

Implementation and

execution

Analysis and Design

Planning and

control

Delivery

START

o

Unit test: The lowest level of testing, often executed by a developer.

Functionality test: Test of the functionality of a controlled part of the system.

System test: Tests the system.

Acceptance test: ◦ User Acceptance Testing

◦ Operational testing

◦ Alpha- & Beta-test

2010-11-22 21

Entry criteria ◦ The purpose of entry criteria is

to prevent a task from starting which would entail more (wasted) effort compared to the effort needed to remove the failed entry criteria

Exit criteria ◦ The purpose of exit criteria is to

prevent a task from being considered completed when there are still outstanding parts of the task which have not been finished.

2010-11-22 23

Static testing

Formal / informal review Static

analysis

o

2010-11-22 24

Effective -Finds defects

Well constructed -represents other TC

Maintanable -easy to maintain

Economical -easy to use

o

System with 2 input fields

◦ Choose a method for black-box testing

◦ How many test cases would you write?

2010-11-22 25

o

Age

Shoe size

Statical (does not execute any code)

Whitebox (glassbox)

Blackbox

Other (ex. experience based)

2010-11-22 26

Goto 10 for i=1 to 100

o

Black-box methods: ◦ Equivalence partitioning (EP) ◦ Boundary value analysis (BVA) ◦ Random input ◦ etc...

White box methods ◦ Statements, decisions, test path coverage, dataflow,

branch coverage etc...

Experience based ◦ Error guessing ◦ Exploratory testing

System with 2 input fields

◦ Choose a method for black-box testing

◦ How many test cases would you write?

2010-11-22 28

o

Age

Shoe size

2010-11-22 29

UC1-1 UC2-1 .. ..

Requirement spec.

TEST CASE 1 UC1-1

TEST CASE2 UC2-1

Test spec..

TestCase1 UC1-1 OK

TestCase2 UC2-1 NOK

Test Protocol

UC2-1 NOK Testcase2, Step X

Incident report

o

2010-11-22 30

Examples of risk criterias: ◦ Test where the defect has the most impact

◦ Test where the defects are most visible

◦ Test where defects are most probable

◦ Ask the customer to prioritise

◦ Test where many changes have been done

◦ Test where many defects have been found

◦ Test where most complexity or complicated functions exist

2010-11-22 31

Evil messsenger, ”your baby is ugly”

32

A creative work

Learn more and deeper than most people

In the middle of the action

Wide technical skills

Deep technical skills

Hard, a challenge

May sometimes pretend that he is a customer

33

ISTQB – www.istqb.org -- downloads

TMap & TMap Next

TPI (Test Process Improvement)

www.stickyminds.com

2010-11-22 34

Creating a test plan

Management doc not a techincal doc

What (tested items, deliverables)

Why? – give reasoning for decisions

How? – give indications for methods

Who? – how many people, what kind of people?

When? – at what stage, when to stop

Where? – which environment, technical requirements etc.

Define the purpose of testing

Define test items ◦ Funcional

◦ Non-functional Performance

Response-time

Scalability

Security and authentication

Robustness, handling errors, wrong intput etc.

Tested or not tested

Give reasoning for each!

Approach – white-box, black-box

Acceptance criteria

Deliverables – test spec, instructions, protocols, reports, scripts, programs

Envirionmental needs – OS, browser etc.

Testing tasks – writing spec, install, test data creation, execution, bug reports, reporting

Roles/responsibilities - creator-reviewer

Staff & training

Schedule

Risks

1. Write a test plan with your group 1. Purpose & exit criteria 2. Test items 3. Environment needs 4. Tasks 5. Risks (project/product risks not testing risks!)

2. Write a review of your neighbours test plan 3. Submit when done ◦ Put your names on!!!

4. ASK QUESTIONS!

An application or documentation?

The Seaga Mechanical Combo is a great way to start vending in any location! Super simple - just insert exact coins and turn the knob to vend. The attached bill

changer is a great convenience and increases sales! Offer nine selections of snacks - 3 snack/chip and 6 candy - for a total of 89

snack items. Also provide five selections of cold beverages - cans only - with a great 155 can

capacity! The bill changer holds 120 USD worth of quarters, dimes and nickels. Some

assembly is required. Shipping is included, as well as operation manuals and price decals

Specifications 3 chip/snack, 6 candy selections 5 cold beverage selections (12 oz. cans only) Capacity: 89 items 155 cold beverages For indoor use only Electicity needed for refrigerating Vend is mechanical - exact coins required