@peterrhysthomas · way” to address the regression testing overhead. minimal testing with the...
TRANSCRIPT
![Page 1: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/1.jpg)
Testing the untestable
@peterrhysthomas
![Page 2: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/2.jpg)
Is testing worth it?
![Page 3: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/3.jpg)
![Page 4: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/4.jpg)
https://cucumber.pro/blog/2014/03/03/the-worlds-most-misunderstood-collaboration-tool.html
![Page 5: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/5.jpg)
![Page 6: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/6.jpg)
![Page 7: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/7.jpg)
![Page 8: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/8.jpg)
You know you have too few tests when….
…You get bugs that escape into production
![Page 9: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/9.jpg)
You know you have enough tests when….
…You rarely get bugs that escape into production
…You are rarely hesitant to change some code for fear it will cause production bug
TestCoverage – Martin Fowler
![Page 10: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/10.jpg)
You know you have the wrong sort of testing when….
…You get bugs that escape into production
…Preparing, Executing, Fixing or investigating tests causes a drag on the delivery of value
![Page 11: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/11.jpg)
![Page 12: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/12.jpg)
Deliberate Discovery
Don’t start with a backlog, start with a question
![Page 13: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/13.jpg)
In order to avoid incorrect or fraudulent payments As the Payments Manager I want to confirm a new payment instruction before
making the payment
![Page 14: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/14.jpg)
![Page 15: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/15.jpg)
![Page 16: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/16.jpg)
![Page 17: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/17.jpg)
![Page 18: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/18.jpg)
![Page 19: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/19.jpg)
![Page 20: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/20.jpg)
![Page 21: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/21.jpg)
![Page 22: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/22.jpg)
Given we are a development team When we have to learn something Then we develop software to help us learn it
![Page 23: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/23.jpg)
• Iteration 1 – What payment types should we be holding? – Build report on potential held payments and not
held payments
![Page 24: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/24.jpg)
• Iteration 2 – What details of held payments should we show to
the user? – Display potential held payments in target
application
![Page 25: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/25.jpg)
• Iteration 3 – How do we show the correct user their payments
to authorise? – Build assignment of potential held payments
![Page 26: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/26.jpg)
• Iteration 1 – Report on potential held payments and not held
payments • Iteration 2 – Display potential held payments in target application
• Iteration 3 – Allow assignment of potential held payments
• Iteration 4 – Enable hold, assignment and release of held payments
![Page 27: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/27.jpg)
• Discovery – Report on potential held payments and not held
payments • Discovery – Display potential held payments in target application
• Discovery – Allow assignment of potential held payments
• Delivery – Enable hold, assignment and release of held payments
![Page 28: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/28.jpg)
Spike and Stabilise
Only make production worthy the things that are worthy to be in
production
![Page 29: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/29.jpg)
In order to meet OFAC* sanctions As an Payments Manager I want to check all payments against the OFAC
sanctions list.
*Office of Foreign Assets Control
![Page 30: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/30.jpg)
![Page 31: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/31.jpg)
In order to reduce the manual overhead of checking all payments
As an Payments Manager I want to automatically check all payments
against the OFAC sanctions list.
*Office of Foreign Assets Control
![Page 32: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/32.jpg)
• Existing process manual spreadsheet comparison • New process built and deployed in 1 day – Minimal testing
• Users compared existing and new and decided whether new was:
1. Accurate 2. Better than the existing process
• Only then was effort made to fully productionise
![Page 33: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/33.jpg)
User Driven Blue-Green Deployment
Why not give the users the choice?
![Page 34: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/34.jpg)
![Page 35: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/35.jpg)
![Page 36: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/36.jpg)
Is Test First Always The Best Way?
Sometimes you can be too agile to test
![Page 37: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/37.jpg)
The Third Way
The system is not seen outside of the organisation. It is not used to make payments to external entities. The
users do not use the information solely to make decisions.
As a result the users are tolerant to bugs... provided they can be fixed quickly. The team can turn around bug
fixes within a few minutes to an hour which is acceptable to the business. So this presents a “third way” to address the regression testing overhead.
Minimal testing with the acceptable risk of introducing bugs that are fixed quickly
Chris Matts – Is TDD Always The Best Solution – The Third Way
![Page 38: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/38.jpg)
Your best testers…
…are your users
![Page 39: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/39.jpg)
Monitor Errors (in production) not Bugs
• Scan user logs for common errors • Identify errors before your users or support
team tell you
![Page 40: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/40.jpg)
Look (in production) For Abnormal Patterns
• Out of trend statistics can indicate problems • Helps to identify Black Swan events
![Page 41: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/41.jpg)
So what….?
• Think why you are testing • Use iterative delivery to uncover unknowns • Only test what deserves testing • Testing can only test what you know
![Page 42: @peterrhysthomas · way” to address the regression testing overhead. Minimal testing with the acceptable risk of introducing bugs that are fixed quickly Chris Matts – Is TDD Always](https://reader036.vdocument.in/reader036/viewer/2022070916/5fb6403a38fd1a4a10398e4b/html5/thumbnails/42.jpg)
@peterrhysthomas