Successful Introduction of a Practical Test

StrategyJurian van de Laar,

Improve Quality Services, The Netherlands

Europe’s Premier Software Testing EventStockholmsmässan, Sweden

WWW.EUROSTARCONFERENCES.COM

“Testing For Real, Testing For Now”

A practical Test Strategy

Jurian van de Laar

jla@improveqs.nl

EuroSTAR Conference 2009

Stockholm

Accredited provider ISTQB

Foundation & all Advanced modules

Training provider IREB

Requirements Engineering

Accredited Lead Assessors

formal TMMi assessments

Improve Quality Services

Leading in Testing & Quality Management

Consultancy, Subcontracting and Training

Founded in 1998 by Erik van Veenendaal

Eindhoven

Improve Quality Services

Waalre (near Eindhoven)

The Netherlands

www.improveqs.nl

Policy? Strategy?

What does it mean ??

Do you need this ??

Case study

Critical success factors

Sounds familiar ?

Definitions Test Policy & Strategy

Test Policy – High level description of principles,

approach and major objectives of the organization

regarding testing.

Test Strategy – High level description of test levels for

an organization or programme, and the testing to be

performed within each level.

Test Policy

Test Strategy

Master Test Plan

Generic risks

Business goals Test goals

Blueprint

Projects

Organization

Test Policy 20XX

Definition of testing

Mission / Ambition based on

business goals

EmployeesTest processCustomers

Test Process Improvement

- objectives, performance indicators, model

Test targets

Test Strategy

Based on the test policy

Addresses the generic product risks

Contains description of test levels:

Purpose, responsibility

Level of independence (organizational)

Entry and exit criteria

Test process and deliverables

Approach (top-down, bottom-up)

Test design techniques

Tools to be used

Standards to be used

Test environment

Typical non-functionals

Test policy & strategy are important …

Provide common basis for projects

What is really important for business / managers

Leading in models like TMMi, TPI, and TMap Next

… but also difficult to realize !

‘We have no business goals or policy’

‘We don’t have time / interest / priority’

The challenge

Practical case study

Principal: IT-Solutions Mgr. Solutions Architecture

Draft Test Strategy guidelines as starting point

My assignment: coaching creation of Test Strategy

Themes: strategy, terminology, organization

Scope: IT Express BeNeLux

Business, Business-IT and IT-services

Roadmap and planning

Organize meeting structure

Kick-off with management group

Training day for work group

Strategy workshops (4 sessions)

Elaboration Test Policy & Strategy

Reporting to management group

Approach

3

5

6

7

1

2

4

Kick-off with management

Purpose: get feedback and constraints

Presentation using provocative arguments

Results:

Feedback:

How ‘testing’ is perceived in the organization

Effects of insufficient testing (= business case for improvement)

Important quality characteristics

Constraints

Realistic (acceptable) costs and organization structure

3

Strategy workshops

1. Agree on purpose and scope

using management feedback and constraints as input

2. Determine policy themes

reliability, customer facing, usability, performance, efficient testing, time to market

3. Definition of testing

4. Business goals not our responsibility, proactive thinking is (rationale)

7 values DHL taken as starting point

5. Test strategy V-model, determine test levels

RACI*, responsibilities, who is ‘main contractor’

5

* = RACI: Responsible, Accountable, Consulted, Informed

Elaboration: results

Test Policy document

Quality goals + rationale (2 A4)

Test policy (2 A4)

Test Strategy document

Principles

V-model

Characteristics per level

Risk based testing

Management presentation

Results and recommendations

6

Elaboration: Test Policy6

Elaboration: V-model6

Full Roll-OUT

Management presentation

Recommendations

Agree on common test strategy

Apply risk based testing

Install a test competence cell

Start a test process improvement project

Agree on common test terminology (e.g. ISTQB)

Recommendations have been implemented

Test competence cell: competence manager installed

Interviews / meetings with stakeholders

Plan for deployment written

7

Lessons and success factors

It requires time to determine a good test strategy!

If there is no policy available, then propose one.

Involve management (feedback, constraints).

Broad representation of organization as workgroup.

Foundation test knowledge in workgroup (training).

Bring sufficient flexibility into your V-model.

Who is responsible for what (RACI) ?

Pioneer for implementation from own organization.

Epilogue (1)

Follow-up by test competence manager:

Further elaboration and deployment of test strategy (templates and tools)

Organized sessions/training on test awareness

Assistance in projects to apply test strategy

Has set up method to manage test environment

The assignment ended in 2008, what happened afterwards…

Epilogue (2)

Current situation (mid 2009):

Competence cell (0.5 FTE*): central organization of testing expertise.

Close cooperation helps to get clear requirements

Clearly defined test levels are helpful

Test strategy currently applied in a project for European standardisation of one of DHL’s most important product lines

The assignment ended in 2008, what happened afterwards…

* = FTE: Full-time equivalent

Conclusions

A sound Test Policy and Test Strategy …

are the important starting points for projects

help to have good discussions …and decisions

can be very clear and specific

give answers to relative questions

are the first step in test process improvement

… are worthwhile to give a try in your organization ?!?!