When a Test Manager Makes a Difference

Practitest Webinar

Derk-Jan de Grood / Valori

26 September 2017

2

“The only constant I am sure of

Is this accelerating rate of

change”

Source : [Peter Grabriel–downside up]

3

Goals

Find out where Test managers can add value in todays

context

4

The Traditional Test manager

What does the test manager do?

5

Traditional test managers spend time on…

6

Test Policy

Test strategy (for the team, the project, the

organization)

Budget & Resourcing

PRA

Environment

Planning new releases to test

Release Advise

Test report

Progress report

Fight for quality (with e.g. the project manager)

Connect (with e.g. Release management)

Defect Triage

Staff tasks like Team lead, Education, etc.

TPI

Tooling and Automation

Let’s go Agile

7

Part of

Development

Done = Tested

Team is

responsible for

testing

… Continues Integration and

Deployment

8

Part of

Development

Done = Tested

Team is

responsible for

testing

Part of Delivery

Pipeline

Automated

Agile testing differs

Continues Involvement

Tools

Essential

MultiDimensional

Skills

Com-munication

Quick Feedback

9

Nishi Grover Garg

Set your Course

10

Workshop TestNet

11

How did your work change

over the last 5 years?

Workshop TestNet

12

How did your work change

over the last 5 years?

More Less Started Stopped Keep

• Teamwork

• Exploratory

testing

• Automation

• Feedback

customers

• Global TS

• Releases

• Security

• PRA

• E2E testing

• Cloud

• Working

individually

• Detailed scripts

• Manual Testing

• Formal

acceptance

• Reporting and

Logging

• Test planning

• Uniform tests

• Test automation

• Using tools

• Real time test

results

• Integrated tests

• Using

Production Data

• Value driven

decision making

• Data driven

decision making

• Agile SCRUM

• Auditing

• Open source +

own build tools

• Testers that can

only test

• Detailed test

plan

• Manual written

test reports

• PRA (while

doing devops)

• Quality Focus

• Specifications

• Drafting test

cases based on

specifications

• Hardcore

Testing

• Keep learning

• Waterfall

• UAT

13

Test management tasks

gone sour

How did your work change

over the last 5 years?

More Less Started Stopped Keep

• Teamwork

• Exploratory

testing

• Automation

• Feedback

customers

• Global TS

• Releases

• Security

• PRA

• E2E testing

• Cloud

• Working

individually

• Detailed scripts

• Manual Testing

• Formal

acceptance

• Reporting and

Logging

• Test planning

• Uniform tests

• Test automation

• Using tools

• Real time test

results

• Integrated tests

• Using

Production Data

• Value driven

decision making

• Data driven

decision making

• Agile SCRUM

• Auditing

• Open source +

own build tools

• Testers that can

only test

• Detailed test

plan

• Manual written

test reports

• PRA (while

doing devops)

• Quality Focus

• Specifications

• Drafting test

cases based on

specifications

• Hardcore

Testing

• Keep learning

• Waterfall

• UAT

14

Test management tasks

with value

How did your work change

over the last 5 years?

15

THE TESTMANAGERThere is no formal role for

Test management in Agile

BEWARE

Exploring in addition

to

checking

How Quality is achieved in agile

16

Integration testing

using virtualization

Feature teams

complete integrated

features

Testing is done in the

sprint

Traditional test managers spend time on…

17

Test Policy

Test strategy (for the team, the project, the

organization)

Budget & Resourcing

PRA

Environment

Planning new releases to test

Release Advise

Test report

Progress report

Fight for quality (with …)

Connect (with e.g. Release management)

Defect Triage

Staff tasks like Team lead, Education, etc

TPI

Tooling and Automation

Honesty Call

18

Workshop with Jan Jaap Cannegieter

19

Results of the dot voting

20

Developing Craftmanship

Multiple teams

Acceptance manager

Integration

Guarding Quality

Specific tests like Performance/Security

Compliance officer

Manager Test pool

Domain expert

0 5 10 15 20 25 30 35 40

Settings where TM adds valueMeeting compliance and

regulations

Coordinating teams

working on the same

project

Responsible for

acceptance

Responsible for the

integration test

Manager of a test pool Development of test

craftsmanship

21

Meet our crew

22

Acceptance Multiple teamsIntegration

As overall test

manager

I ensure that all the

teams in the project

are working on the

right tests

So that we reduce

overlap, prevent gabs

and the teams work

efficiently on the

overall quality

23

Multiple teams

Tasks and activities

• Define the overall quality strategy

• Watch over the boundaries between teams

• Perform risk analysis and monitor risks

• Make sure the DoD’s of teams are in line with each other

• Monitor test coverage

• Monitor that all teams do (non-functional) testing

professionally

• Help teams to reach an acceptable level of testing

• Tune test approach of the teams

• Make sure end-to-end testing is done

Agile Quality Strategy

26

What needs to be tested

Steering committee Dashboard/ Roadmapping

Auditing the test work

Coaching the testing team members

Organizing tests that do not fit the sprint

We still need a

plan to know

were complete

There is a need

feedback and

traceability

(comfort &

adaptivety)

Testing needs

be of quality to

justify itself

Testing needs

to add value to

hold up an

release

Benefits

• Enhance overall quality (E2E)

• More risk control

• Create transparency (quality and progress)

• Prevent double work or gaps

• Take away bottlenecks

• Efficiency – teams become more efficient

• (NOT: more uniformity in the way teams test)

Acceptance

28

As Acceptance

manager

I ensure that we all

parties focus on the

right things and know

what is needed for the

formal acceptance of

the solution

So that we succeed

our mission without

late surprises

29

Acceptance

Acceptance manager

When applicable

• One system that support different parts of the organization

• Not one PO that has the authority to decide

• When working with different suppliers

• Political environment

Benefits

• Different parts of the organization and suppliers are involved

• Working towards a clear goal

• Clearer acceptance criteria upfront

Tasks and activities

• Elicitate the acceptance criteria

• Make sure acceptance criteria are taken into

account

• Make sure acceptance criteria are used in

testing

• Stakeholder management

• Keeping track on progress during the project

• Stimulate collaboration between parties

• Become a trusted wing partner

Experience: building an mobile

network

32

33

It ain’t done until its accepted

34

Witness

DevelopmentenTes ng

Testedsolu on

Witnessreport Customer

Witness proces

35

36

Transparency

As integration manager

I ensure that we

maintain a focus on

integration of

component and

systems

So that customer

journeys can be

completed and we

actually deliver value

38

Integration

Integration Manager

When Applicable

• Project or changes that involve multiple systems

• Organization is not yet completely on DevOps

Benefits

• Better Quality and user experience

• Early detection of Integration problems

• Sharing knowledge on system landscape

• Bridging between teams

• Better efficiency and saving money

39

Tasks and activities

• Managing environments

• Advising tooling and virtualisation

• Version and configuration management

• Scrum of Scrums

• Communication and advising teams

• Drafting customer journeys and scenario’s

• Quality ambassador

• Tracking progress, auditing test quality

41Experience story: Energy Market

Architecture

• What are the business processes?

• What are the components?

• What are the interfaces?

Acceptance criteria

• What is the Minimal Viable Product?

• What integrations are needed to make it work?

Requirements traceability

• When are we complete?

• How do test results add up to acceptance?

42

Missing

What should a car minimally

do?

43

Planned Integration Tests

44

Integration

tests

Release Date

Ensuring Integration (rough sketch)

45

Organization

Component

System

Service

Continuously

(in the sprint)

Occasionally

(e.g. prior to a

release)

Synergy

46

47

Do’s and Don’ts (workshop results)

48

Do

• Coach the teams

• Build bridges

• Give teams responsibilities

• Create transparency

• Align with e.g. PO’s

• Create clear quality requirements

• Talk with testers (1:1)

• Be approachable

• Help and Advice

• Audit

• Helicopter view

• Be a quality ambassador

Don't

• Micro management

• Telling people what to do

• Interfere with team internal affairs

• Short term focus

• Tunnel vision

• Combined functions (tester + manager)

• Difficult complex reports

49

Develop Craftsmanship

50

Develop Craftsmanship

As Practice lead

I ensure that we

exchange experiences

and develop our

profession

So that we keep

adding value and

excel as testers

Development craftmanship

When applicable

• Big organizations with different testers

• Testing important for the business

• Organizations that invest in their employees

Benefits

• Enhanced craftmanship

• Better, more professional testing

• Happy testers

• More happy customers

• Save money and time

51

Tasks and activities

• Coach testers

• Organize knowledge sessions

• Organize courses

• Stimulate that testers visit conferences

• Keep your own knowledge up-to-date

• Test when necessary

• Manage knowledge

• Organize that testers share knowledge

• Manage test community

• Promote the importance of testing

Other crew members

Compliance officer and Manager of the test pool

53

Finale

54

Quality is

embedded

in agile

development

55

Role is changing

56

Traditional test managers spend time on…

15

Test Policy

Test strategy (for the team, the project, the

organization)

Budget & Resourcing

PRA

Environment

Planning new releases to test

Release Advise

Test report

Progress report

Fight for quality (with e.g. the project manager)

Connect (with e.g. Release management)

Defect Triage

Staff tasks like Team lead, Education, etc

TPI

Tooling and Automation

Value as a test manager

5757

Acceptance Multiple teamsIntegration Develop Craftsmanship

58

Show your worth

We need less of them…

Test manager

Test Engineer

59

60

Related roles

61

So be good…. or grow into a new role

62

Benefits of this webinar

We looked at

the activities of the Test manager

the settings where a Test manager has value

how to make a difference

alternative roles

63

Derk-Jan

Valori

Coltbaan 4a

3439 NG NIEUWEGEIN

The Netherlands

• derkjandegrood@valori.nl

• +31(0)651807878

• www.valori.nl

• @DerkJanDeGrood

• http://djdegrood.wordpress.com

64