agile testing in a formal organisation · mapping cobit at the agile/scrum process 39 • include...

Agile Testing in aFormal Organisation

Egbert Bouman & Robin Esman

Twitter: @egbertbouman

Valori & De Nederlandsche Bank

About us

Robin Esman

• 20+ years in IT, 9 years at DNB

• Test community lead

• Business Analist

[email protected]

Egbert Bouman

• Valori

• Author SmarTEST

• Certified risk auditor

• Test automation consultant

• Agile test coach & trainer

[email protected]

mailto:[email protected]

mailto:[email protected]

Our message today

YES…

… formal organisations do agile

• Because it’s mainstream now

• And rightly so: agility is good for everyone

BUT…

… this poses extra challenges

• Be aware of that challenges

• And adopt your agile strategy accordingly

SO…

… take advantage of our story

• With seven best practices we’ve selected in retrospective

• They worked for us, and may work for you

3

In the context of better testing!

DNB INTRO

In

Behind Closed Doors

Collectors value >> Gold value Collectors value >> Gold value

DNB Working on trust

6

Het Agile Manifesto

77

1. Our highest priority is to satisfy the

customer with early and continuous

delivery of valuable software.

2. Welcome changing requirements, even

late in development.

3. Deliver working software frequently,

from a couple of weeks to a couple of

months.

4. Business people and developers must

work together daily.

5. Build teams with motivated

individuals. Give them the

environment and support, and trust

them.

6. Most efficient and effective is face-

to-face conversation.

7. Working software is the primary

measure of progress.

8. Promote sustainable development. Agile

teams should be able to maintain a

constant pace indefinitely.

9. Continuous attention to technical

excellence and good design.

10.Simplicity: the art of maximizing the

amount of work not done.

11. The best results emerge from self-

organizing teams.

12.Regularly, the team reflects on how

to become more effective, then adjusts

its behavior accordingly.

The 12

‘A

gile Principles’

How to merge agile into

a formal organisation’s DNA?

9

vs

Risk

aversion

& control

How did we

address the

challenge?

We implemented a lot of bright

(and not so bright) idea’s

10

And selected the top-7 for You

Our advice: 7 ‘agile test essentials’

11

1. No scrumdamentalism

2. Use a framework for 'challenged self-assessment'

3. Respect separation of duties: DTC model

4. Make visual WoW Cards

5. Manage your test automation

6. Improve your improvement capacity

7. Take advantage from the next (Cobit) audit

Beware of scrumdamentalism

Maximise agility, but stick to ‘good old’ best test practices

• Risk analysis

• Refinement: critical thinking, ‘what if’

• Exceptions, rainy days, negative testing

• Sound defect management

• Overall test management

These and other proven practices are OK

• Don’t treat them with contempt

• Beware of short sighted labels like SINO, AINO, Zombie Scrum, …

12

But be clear about the tough choices

More trust, less ‘governance’

More exploratory, less detail in advance

More collaboration, less toll gates

13


14








Started with the

‘Agile Growth Accelerator’

and Agile Scorecard

MaakMaakBuild

Architecture

Six improvement

areas

Testing still the bottleneck

Gartner:

“Testing repeatedly emerges as the most formidable constraint

impacting velocity”

Forrester:

“As organizations adopt (…) Agile and DevOps, testing has

become the biggest constraint impacting quality and speed ”

16

Continued with the

‘Test Growth Accelerator’,

and ‘SmarTEST scorecard

Seven improvement areas:

1.Agile test strategy

2.Scope, risks and test goals

3.Test design & execution

4.People & culture

5.Test environments

6.Automated testing

7.Planning & control

The SmarTEST scorecard in more detail

Best of breed ‘waterfall’ frameworks

• Test Process Scorecard (SmarTEST®)

• TPI (TMap®), CMMi, TMMi

Made agile

• Address all 12 ‘agile principles’

• Kniberg self assessment checklist

Fit for non-hardcore software companies

• Fully agile

• Respecting pre-agile proven practices

‘Challenged Self Assessment’

• Team decides upon improvement focus and required maturity level

• Supported and challenged!! by agile coach18

Real Growth AccelerationPresented 2016Q4 by the pilot teams themselves

Deposito guarantee system

WSS Wall Street System F&A Peoplesoft(no improvement data)

Cash Operations(late boarding)

CRM Financial Supervision

Scorecard process retrospective

Management happy

• Control

• Direction

• Clear & concise

Teams happy

• Challenging whilst helpful

• Freedom of choice

• ‘Do-able action items’

20


21








DeveloperQuality & business aware

Tester(More or less) independent

CustomerFinal proof & verdict

Respect separation of duties: DTC model

No walls, only roles!!

Design&

Build

Operations

Developer Tester Customer

The test ‘Shift Left’

Unit & integration test (CI)

Acceptancetest

Independent functional (chain)test new & existing

(ST, SIT, FT)

Developer Tester

Unit Test

Customer

A succesfull shift left path

Test New & OldAcceptance

testWAS

Unit Test & CI

Test New

Test Old

AcceptancetestNOW

Unit test & CI

Test New

Test Old

Acctest

GOAL

Insufficient coverage, production incidents.

Beter coverage,testing still on critical path.

Faster, same coverage,less business capacity.


25








WoW cards

Lean Way of Work (WoW!)

• Two-sided with wow-effect

• Kanban: Japanese for ‘Visual card’

Lean, accessible, visual

• Color & appearance

• No bulky documents

Keep them alive

• Proces en ownership

• Expiration date26

27

Visualize

Kanban = visual card!

WoW: make testing colourful again!


28








Tool does the job

Faster, cheaper

Dependable, repeatable

Everybody does it

Demo impressed

me!

29

Top-down driver: bosses like it

Fiddling with tools

I like hightechstuff

Manual testingis boring

Automationis cool

Mymarketvalue

30

Bottom-up driver: boys will be boys (girls too)

Manage your Test Automation ‘just enough’!

Choose what to automate

• And what not to automate

Manage the top of the pyramid

• And choose ‘middle & top of the pyramid tools’

Let developers do it their way

• Trust & let go

• But be aware of ‘too many tools’

31


33





5. Manage your test automation tool landscape



Improve your improvement

Problem: continuous improvement is immature

Solution: agile working secures continuous improvement

Unfortunately it doesn’t work that way

34

What did we do?

We solved this problem by continuously organising ‘just do Scrum’

Stimulating, Challenging, Facilitating

Use guilds, audits, events, coaches => time to focus

Build a community

Keep looking for leverage opportunities (e.g. Cobit audit)

And make it fun!

35

Make it fun

Test community:40 members

‘pepernoten’ test‘broodje kroket’


37








The Cobit

audit framework:

Overview

38

Still a popular framework!

DNB uses a sub-set

Level 3: formalise & proof

Level 4: continuous improvement

Mapping Cobit at the agile/scrum process

39

• Include non-functional acceptance criteria

• Like Security and Performance

• Atention for out-of-the-box risks

Refinement and Sprint planning

• Security and Performancetest done

• Fall back plan available

• Anticipated and demo ’ed with your DoD

Review(= productevaluation)

• Did we follow the proces agreed upon?

• Do we do a sound risk management job?

• Is testing sufficiently Independent (separation of duties)?

Retro(= process evaluation)

This

example:

AI7.6

‘Testing of

Changes’

So far our story

41








43

Thank you

for [email protected]

www.valori.nl

[email protected]

www.dnb.nl

http://www.valori.nl/

http://www.dnb.nl/

agile testing in a formal organisation · mapping cobit at the agile/scrum process 39 • include...

Documents