Tester, get out of your cave!
Jan Jaap Cannegieter
Vice President SYSQA B.V.
© S
YS
QA
Alm
ere
3
Why are we here?
© S
YS
QA
Alm
ere
4
Agenda
• TMMi in under 10 minutes
• Data from 20 TMMi workshops
• Conclusions
• How to get out of your cave
© S
YS
QA
Alm
ere
6
TMMi – essentials
• Test maturity framework• Benchmark model for assessments• 5 levels• Based on:
• TMM• Evolutionary Testmodel (Gelperin / Hetzel)• CMMI for Development
• In line with ISTQB terminology• Developed by the TMMi-foundation• Public domain• www.tmmifoundation.org
© S
YS
QA
Alm
ere
7
TMMi levels
1. Initial
2. Managed
Test Policy and StrategyTest PlanningTest Monitoring and ControlTest Design and ExecutionTest Environment
3. Defined
Test OrganizationTest Training ProgramTest Life Cycle and IntegrationNon-Functional TestingPeer Reviews
4. Management and Measurement
Test MeasurementSoftware Quality EvaluationAdvanced Peer Reviews
5. OptimizationDefect PreventionTest Process OptimizationQuality Control
© S
YS
QA
Alm
ere
8
How I got my data
• Goals• Learn
• Presales
• Assignments
• 20 organizations• Telecommunication: 2
• Government: 7
• Financial services: 9
• Industry: 2
• Analysis
© S
YS
QA
Alm
ere
9
Quick scans – a demo
Results
Fasten your seatbelts
© S
YS
QA
Alm
ere
11
Test policy and strategy - goals
• Establish a test policy
• Establish a test strategy
• Establish test performance indicators
© S
YS
QA
Alm
ere
12
Test policy and strategy: A: 30% V: 14%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test policy and strategy
Organizations
Percen
tag
e o
f sp
ecif
ic g
oals
met
© S
YS
QA
Alm
ere
13
Test planning - goals
• Perform a product risk assessment
• Establish a test approach
• Establish test estimates
• Develop a test plan
• Obtain commitment to the test plan
© S
YS
QA
Alm
ere
14
Test planning: A: 41%, V: 13%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test planning
Organizations
Percen
tag
e o
f sp
ecif
ic g
oals
met
© S
YS
QA
Alm
ere
15
Test monitoring and control - goals
• Monitor test progress against plan
• Monitor product quality against plan and expectations
• Manage corrective action to closure
© S
YS
QA
Alm
ere
16
Test monitoring and control: A: 47%, V: 15%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test monitoring and control
Organizations
Percen
tag
e o
f sp
ecif
ic g
oals
met
© S
YS
QA
Alm
ere
17
Test design and execution - goals
• Perform test analysis and design using test design techniques
• Perform test implementation
• Perform test execution
• Manage test incidents to closure
© S
YS
QA
Alm
ere
18
Test design and execution: A: 60%, V: 8%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Test design and execution
Organizations
Percen
tag
e o
f sp
ecif
ic g
oals
met
© S
YS
QA
Alm
ere
19
Test environment - goals
• Develop test environment requirements
• Perform test environment implementation
• Manage and control test environments
© S
YS
QA
Alm
ere
20
Test environment: A: 59%, V: 9%
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Test environment
Organizations
Percen
tag
e o
f sp
ecif
ic g
oals
met
© S
YS
QA
Alm
ere
21
Overview
0
10
20
30
40
50
60
70
80
90
100
Per process area
© S
YS
QA
Alm
ere
22
My conclusion
60%: Test design and execution
30%: Test policy and strategy
41%: Test planning
47%: Test monitoring and control
59%: Test environment
Inside
Outside
© S
YS
QA
Alm
ere
23
We are in a cave!
© S
YS
QA
Alm
ere
24
© S
YS
QA
Alm
ere
25
In case you disagree…
© S
YS
QA
Alm
ere
26
Best practices ‘outside of your cave’
• Perform a product risk assessment and establish a test approach with relevant stakeholders
• Obtain commitment for your plan from relevant stakeholders
• Plan stakeholder involvement in your project
• Monitor stakeholder involvement in your project
• Make release advice with people outside of testing
• Establish and maintain a test strategy with relevant management
• Define, measure and report test performance indicators to relevant stakeholders
• Establish and maintain a test policy with higher management (business and IT)
Stakeholder management
27
The basics
© S
YS
QA
Alm
ere
28
Stakeholder
A person or group of persons who will be involved in or effected by your test project or improvement project
© S
YS
QA
Alm
ere
29
Stakeholder management
• Stakeholder identification
• Stakeholder analysis
• Stakeholder involvement
© S
YS
QA
Alm
ere
30
Stakeholder aspects
• Organizational level
• Power
• Business interest
• Mindset
• Ambition
• Behavioral style
• Stakeholder type
© S
YS
QA
Alm
ere
31
Type of stakeholders
Demanding Discretionary
Sleeping
Dependent
Dangerous Dominant
Definitive
Urgency Legitimacy
Power
Mitchell, Agle, Wood, 1997
© S
YS
QA
Alm
ere
32
Stakeholdercard – an example
Please think this over
I’ll turn on some inspiring music
© S
YS
QA
Alm
ere
34
© S
YS
QA
Alm
ere
35
© S
YS
QA
Alm
ere
36
Stuart Reid’s test competence model for testers
Test skills
IT skills
Domain knowledge
Soft skills
© S
YS
QA
Alm
ere
37
My test competence model for test consultants
Test skills
IT skills
Domain knowledge
Soft skills
Political skills
© S
YS
QA
Alm
ere
38
© S
YS
QA
Alm
ere
41
