agile testing days 2012 - lean mean bdd automation machine
DESCRIPTION
Agile Testing Days 2012 Presentation from Mike Scott & Tom RodenTRANSCRIPT
![Page 1: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/1.jpg)
SQS Group Limited
Mike Scott & Tom Roden
@MikeAScott @TomMRoden
Agile Testing Days
20 November 2012
Potsdam, Germany
Building a lean, mean, BDD
automation machine
![Page 2: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/2.jpg)
Caption Competition
2
Caption Competition
![Page 3: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/3.jpg)
A story about building an executable documentation system
• Large hedge fund COTS implementation
• Crippling regression test burden
Born out of necessity
• 6-8 month cycle daily release
• 80 times defect reduction Results:
• and again
• Same patterns automation platform
Then did it again…
Safia– Archetypes and Templates, a voyage in automating acceptance
3
![Page 4: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/4.jpg)
The problem quantified
Real data from the field
40
67.5
100
78
23 15
12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
4
![Page 5: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/5.jpg)
The confidence problem!
Where it came from
First Approach to Acceptance Testing:
UAT in a phase, after a period of development
5
![Page 6: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/6.jpg)
6
![Page 7: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/7.jpg)
What it felt like…
7
![Page 8: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/8.jpg)
Becoming acceptance driven…
Building Integrity in
8
Develop
Ready User
Story
Customer
Verification
Specify tests
/ examples
Stakeholder(s) & BA
Testers
Developers
Backlog
Demo /
Showcase
Automate
acceptance
tests
Session
Based
Testing
![Page 9: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/9.jpg)
The initial effect of automated acceptance testing
Real data from the field
40
67.5
100
78
23 15
12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
9
![Page 10: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/10.jpg)
Knowledge and experience to design a framework
• binding tests to investment apps
A set of design patterns
• from business specialists
Inject knowledge
• easy to customise
Vanilla acceptance tests
• tests as specifications
Living documentation
What we gained from this
10
![Page 11: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/11.jpg)
Illogical Architecture Diagram
How Safia Works
House
Adaptor
System
Adaptor
Trading system
Maps SAFIA terminology to
system terminology
e.g. Calypso, Beauchamps,
Summit
SAFIA automates at the
server level via the API
Safia fixtures
AP
I Fitnesse fixtures
Maps investment house
terminology to SAFIA
terminology e.g.
Trade Capture = Trade Entry
11
![Page 12: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/12.jpg)
e.g.
FX, Bond, Equities
IRS, FRA, CDS
e.g.
Run a P&L report
Update a legal entity
Amend SDI
Amend fees
Creating automated test cases using Safia
How Safia Works
Test Archetype
“Test Function”
Create a trade
Instrument
Template
IRS template
Test
Create a valid IRS trade
Apply Template &
Archetype
12
![Page 13: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/13.jpg)
Let’s put it to the test
Demo
13
![Page 14: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/14.jpg)
It brings benefits on different levels
Why use a framework like this
14
Benefits
Increased collaboration
Accelerating implementation
& learning
Anyone can interact with it
Teaching & coaching aid
Reduced Cost of Ownership
![Page 15: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/15.jpg)
It was easy, right?
In retrospect…
15
![Page 16: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/16.jpg)
The path to faster release
Real data from the field
40
67.5
100
78
23
15 12
5 3 0
28 64
151
217
311 371
393
479
545
601
0 0
25
106
284
371 393
479
545
601
0
100
200
300
400
500
600
700
0
20
40
60
80
100
120
February April August November February April May June July August
# T
es
t S
uit
es
Ma
n D
ays
Eff
ort
Month
Time taken for Release Regression testing
Man Days Effort for Regression Test # Test Suites # Automated Suites
16
![Page 17: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/17.jpg)
Tests as specs, the principles of test design
How we wrote the tests
17
What makes a good acceptance
test?
Self-documenting
About business intentions
Concise and granular
A specification not a script
Anyone can understand it
Based on testable statements
![Page 18: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/18.jpg)
~One third is system understanding
Reducing the Cost of Ownership
18
![Page 19: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/19.jpg)
Points to consider when automating
19
Theory ‘aint practice
Start early & iterate frequently
Distil and simplify
Tests or Documentation?
DON’T PANIC !!!
![Page 20: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/20.jpg)
“Brace yourself Daphne…”
20
![Page 21: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/21.jpg)
Questions
Thank you for listening
21
Please evaluate our
presentation by using the
evaluation booklets
which you can find in
your conference bag.
Thank you!
![Page 22: Agile Testing Days 2012 - Lean Mean BDD Automation Machine](https://reader034.vdocument.in/reader034/viewer/2022051610/5482653ab4af9fa00d8b47a2/html5/thumbnails/22.jpg)
SQS Group Limited
Mike Scott & Tom Roden
SQS Group Limited
7-11 Moorgate, London EC2R 6AF
Internet: www.sqs.com