system testing process in iterative environment · 2019-07-25 · system testing process in...

International Conference On

Software Testing, Analysis & ReviewNovember 19 - 23 Stockholm, Sweden

P r e s e n t a t i o n

T3Thursday 22nd November, 2001

System Testing Process inIterative Environment

Salia S Laitinen

Thursday 22 November 2001 T3

System Testing Process in an Iterative Development Environment Saila Laitinen •Education: M.Sc. (Comp. Eng.), University of Oulu, Finland Lic. Tech. student in Helsinki University of Technology •Working experience: –1994-1995 R&D trainee in a small medical technology company –1995-1998 Trainee / Software engineer in Nokia Telecommunications –1998-2000 Group Manager in Nokia Networks, –01/2001-06/2001 System testing manager in Nokia Ventures Organization –07/2001-> Testing Project manager in Nokia Mobile Phones.

1 © NOKIA EuroStarPresentation.PPT/SLa

��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� !�� "��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� ��

� !�� "��


Introduction• Speaker: Saila Laitinen• Education: M.Sc. (Comp. Eng.), University of Oulu, Finland

Lic. Tech. student in Helsinki University of Technology

• Working experience:– 1994-1995 R&D trainee in a small medical technology

company– 1995-1998Trainee / Software engineer in Nokia

Telecommunications– 1998-2000 Group Manager in Nokia Networks,– 01/2001-06/2001 System testing manager in Nokia Ventures

Organization– 07/2001-> Testing Project manager in Nokia Mobile Phones.


Facts Problems• BU was established in

order to ’create’ a totally new product (1999)

• Highly experienced management and software engineers

• Young and inexperienced (yet eager) integration & testing stuff

• We were on a fragile ice

• Management had some conflicting assumptions on how-to-do things.

• Low competence in product testing


Facts / 2 Problems / 2• Bad experience from the

previous project (prototype)• Bad quality

• No documentation

• No generally accepted rules for how to do things.

• Committed management

• People were critical about everything<-> they were open to any suggestions

• ’Jungle-rules’ applied


Problems Medication• We were on a fragile ice

• Management had some conflicting assumptions on how-to-do things.

• Low competence in product testing.

• Studying new technologies/strategy work (WAP, WLan, Bluetooth,…)

• Communication, communication, communication and once again communication!

• Arranged a private tailor-made software testing seminar for the personnel


Problems/ 2 Medication / 2• People were critical

about anything. <->They were open to any suggestions.

• Jungle-rules applied

• Talking with the people. Asking their opinions and solution suggestions

• Investigation of the tools available


Medication Results• Studying new

technologies/strategy work (WAP, WLan, Bluetooth,…)

• Communication, communication, communication and once again communication!

• Arranged a private tailor-made software testing seminar for the personnel

• General understanding of the product

• Less misunderstandings

• Software Testing competence increased, the role of testing was understood


Medication / 2 Results / 2• Talking with the people.

Asking their opinions and solution suggestions

• Investigation of the tools available

• Getting to know other people in the same organization

• ReqPro & ClearCase & ClearQuest from the Rational


ResultsHow long it took to get results

• General understanding of the product

• Less misunderstandings

• Software Testing competence increased, the role of testing was understood

• 1-2 months

• Continuous

• 1 week course and lots of reading

10 © NOKIA EuroStar Presentation.PPT/SLa

Results / 2How long it took to get results / 2

• Getting to know other people in the same organization

• ReqPro & ClearCase & ClearQuestfrom the Rational

• Continuous

• Basic knowledge already existed


The V-Model

R e q u ire m e n ts p e c if ic a t io n

(U s e C a s e l is tF u n c . s p e c .)

S y s te m d e s ig n(A rc h ite c tu a l

d e s ig nin te rn a l in te r fa c e

s p e c .)

D e ta ile d d e s ig n(M o d u le d e s c .)

C o d e

M o d u le T e s tE x e c u t io n

In te g ra t io n T e s tE x e c u t io n

S y s te m T e s tE x e c u t io n

R e q u ire m e n t'ste s tin g

S ys te nd e s ig nte s tin g

D e s ig nte s tin g

S ys te m T e s t C a se d e s ig n

In te g ra tio n T e s t C a se D e s ig n

M odule TestCase Design


Integration and system testing• Extreme coding

• Integration testing in a ‘medium’ (big)

• Exit criteria for the IT

• Entry criteria for the system testing

• A person called ’Integrator works with each two coders and is responsible for:

• integration between two modules

• System level integrators do integration testing and are responsible for:

• IT of all existing interfaces between modules

• Integration test manager provides a tested system, which fulfils the exit criteria for SYTE people


System Testing


System Testing /2

TO DOTO DO NOT TO DONOT TO DO

Find faults not only in the system but also in the specifications

Create confidence

Create a strong process

Save time and money but NOT in the expense of the quality

Have any useless bureaucracy

Create a strict process, which does not allow the use of a common sense


HOW?• By having a plain and clear instructions (general policy)

• By having willingness to change the instructions if needed

• By having willingness to change own habits, and attitudes if needed

• By gathering experience in how to design test cases, which really do find faults


How? / 2• By having strong enough management, who knows that ”no authorities-no

responsibilities”

• By continuous competence development

• By remembering what is our main goal: developing a fault poor product

All test cases are prioritised so that whenever the product isdecided to be released, we can be sure that we have done the besttesting in the time available. [ISEB SW Testing Certificate Training,Grove Consultant, UK]


Testing phases• Documentation testing

• The idea of documentation testing is to eliminate collisions within one product description or between two or more product descriptions. //[Chechik Marshal, Gannon John (1994) Automatic Verification of Requirement Implementation. In: International Symposium of Software Testing and Analysis, August 17-19, Seattle, Washington, USA, 2. 1-5, New York]

• Functional testing, based of use cases• Prioritised• No 1 priority cases = Smoke test• No 2 priority cases = Base functional testing• No 3 priority cases = Executed if schedule allows

• State transition testing• Based on state transition diagrams• Prioritised

• Usability testing• High level usability observation first

• Regression testing• Planning just before executing

• Acceptance testing• Product validating


Documentation testing in a V-ModelR e q u ire m e n ts p e c if ic a t io n

(U s e C a s e l is tF u n c . s p e c .)

S y s te m d e s ig n(A rc h ite c tu a l

d e s ig nin te rn a l in te r fa c e

s p e c .)

D e ta ile d d e s ig n(M o d u le d e s c .)

C o d e

M o d u le T e s tE x e c u t io n

In te g ra t io n T e s tE x e c u t io n

S y s te m T e s tE x e c u t io n

R e q u ire m e n t'ste s tin g

S ys te nd e s ig nte s tin g

D e s ig nte s tin g

S ys te m T e s t C a se d e s ig n

In te g ra tio n T e s t C a se D e s ig n

M odule TestCase Design

D ocum enta t ion tes ting

D ocum enta tion tes ting

In te g ra to rs


More problems• The process is quite confusing• Many parallel activities• No need for useless documentation

Unavoidable management


The process model

T im e scale

T est P lanning

Feature lis t

T est P lanning T est P lanningT est P lanningT est P lanning

T est scenariodes ign

H W /SW R eq.Spec.

T est casedes ign

Im plem enting

T est Execu tionand reporting





C orrec tiveim plem enting





T est casedes ign

T est casedes ign

T est casedes ign

T est casedes ign

C hange C ontro l B oard

Pro

ject

Man

agem

ent

Arc

hite

ctua

l des

ign

F ea ture lis t Fea ture lis tFea ture lis tFea ture lis t

F irs t R eleaseR eady

R elease T esting

P rogram S teering G roup

Second R eleaseR eady

R elease T esting

Im plem entingIm plem entingIm plem entingIm plem enting





H W /SW R eq.Spec.

H W /SW R eq.Spec.

H W /SW R eq.Spec.

H W /SW R eq.Spec.

Change Control Board reviews test reports, decides about changes in the productdecides when the product is released

Project Management does the documentation testing, manages resources

Program steering group makes the market analysis


Case Study

• The business program was closed right after the first experimental project was started:

• No real data available


Q & A

system testing process in iterative environment · 2019-07-25 · system testing process in...

Documents