![Page 1: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/1.jpg)
Beyond Agile Testing to Lean Development
James O. Coplien Gertrud & Cope
![Page 2: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/2.jpg)
![Page 3: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/3.jpg)
Processes and Tools over
Individuals and Interactions
![Page 4: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/4.jpg)
Usability Testing
System Testing
Exploratory Testing
Test Testing
![Page 5: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/5.jpg)
![Page 6: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/6.jpg)
Why doesn’t unit testing work?
We can’t test things of business value
Not knowing what to test, we test many things that don’t need testing
Automated crap is still crap
![Page 7: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/7.jpg)
Unit tests can’t test things of business value
Can you demonstrate that Method X of Object Y will ever be invoked in practice?
If a unit test fails, how does it affect ROI?
What’s amazing is that we continue to test… Paths that will never be exercised Methods that are never used Naive method invocation sequences
![Page 8: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/8.jpg)
Most unit tests are unnecessary
We test scenarios that will never be exercised in the real world
We test methods that will never be invoked
We supply data that a real application would never have to process
![Page 9: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/9.jpg)
Automated crap is still crap
Design requires intelligence: intelligence cannot be automated
Autonomation instead of automation
Testing is about exploratory inference
Get manual procedures working first and then automate the mundane parts
![Page 10: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/10.jpg)
Automated testing is not testing
![Page 11: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/11.jpg)
Nokia Test Score 2: Testing
No dedicated QA - 0
Unit tested - 1
Feature tested - 5
Feature tested as soon as completed - 7
Software passes acceptance testing - 8
Software is deployed - 10
![Page 12: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/12.jpg)
Weinberg’s Law of Decomposition
![Page 13: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/13.jpg)
Weinberg’s Law of Decomposition
Used
Int
erfa
ce
Used Interface
Must unit-test!
Must unit-test! Must
unit-
test!
![Page 14: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/14.jpg)
Weinberg’s Law of Decomposition
Use Size
Test Size
Wasted testing effort
![Page 15: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/15.jpg)
Part III: You Will Have Bugs!
Scenarios we test in the lab
Scenarios exhibiting
faults
Scenarios invoked in the field
![Page 16: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/16.jpg)
Testing waste
Scenarios we test in the lab
Scenarios exhibiting
faults
Scenarios invoked in the field
Type 2 Muda
Type 1 Muda
Type 4 Muda
Type 1: Defects
Type 2: Stuff customers
don’t ask for
Type 4: OverprocessingType
1 Muda (field)
Removing tests leaves field faults
undetected!
![Page 17: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/17.jpg)
Why do we unit test?
Because we can
It doesn’t require teamwork
We don’t have to talk with the business
We get to interact with tools instead of people
Our decisions are emotional, not rational
![Page 18: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/18.jpg)
Reduction of WasteType 1: Better testing (e.g., going beyond contextual testing to exploratory & experience-based testing) (and better yet, better analysis and design, where the real payoffs are)
Type 2: Don’t test dead code
Type 4: Remove redundant and tautalogical tests, and tests that give no information
Type 1 in the field: Ship your tests!
![Page 19: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/19.jpg)
Don’t take the tests out when you ship!Remember, you will have bugs in the field!
Deployment is your largest test driver
Ship your tests as part of the code
IN case of a failure, it saves you from Type 9 muda (unsafe execution)
![Page 20: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/20.jpg)
Be Customer-Focused
Assertions and Design by Contract!
![Page 21: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/21.jpg)
How Adding ASSERTS reduces work
0
500000
1000000
1500000
2000000
2500000
0 1 2 5 10 20 50 100
15 bugs p=0.01 15 bugs p=0.0210 bugs p=0.01 10 bugs p=0.025 bugs p=0.01 5 bugs p=0.02
number of ASSERTS
Wor
k, T
sing
lech
ecks
Overload Magazine,
October 2014
![Page 22: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/22.jpg)
Defensive Programming
… is largely the incorporation of tests in the code
Long known for high-reliability systems
The value of asserting the obvious
Design by Contract
![Page 23: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/23.jpg)
Assertions make code readable!
- (void) prepareForSegue: (UIStoryboardSegue*) segue sender: (id) sender { assert(segue != nil); if ([[segue identifier] isEqualToString: @"SegueAvatarPickerToPickOneLoginPlayer"]) { assert((UI_USER_INTERFACE_IDIOM() == UIUserInterfaceIdiomPhone)); // The field parentViewController_ (instance variable) is // maybe null if we are in iPhone mode and doing a real // segue, instead of a popover as we do on the iPad. So // get the right object from the segue record. KnowsyPlayerPickerCommonViewController *parentViewController = [segue destinationViewController]; [parentViewController setKnowsyData: knowsyData_]; } else { // iPad - no special processing - should be no popover assert(parentViewController_ == nil); } }
![Page 24: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/24.jpg)
Part IV: Beyond agile testing, Lean testingWhen a test finds a bug, it’s about fixing the process — not the bug
The long-term payoff is higher!
No bug-tracking systems: we fix problems NOW.
The tester should be the team’s conscience
![Page 25: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/25.jpg)
Other Lean-isms
Deming: Quality is inversely proportional to the number of testers
GM kept around its cars for 1 - 6 weeks for test and repair: Toyota drove them off the end of the assembly line onto the car carriers.
![Page 26: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/26.jpg)
Conclusion
Be humble about what tests can accomplish
It’s not about testing, but value
Design tests to assess properties of business value
Go beyond unit testing to defensive programming and design-by-contract
As agile testers, focus on process first and product second
![Page 27: [Rakuten TechConf2014] [G-4] Beyond Agile Testing to Lean Development](https://reader033.vdocument.in/reader033/viewer/2022052908/5595a0001a28ab14448b4603/html5/thumbnails/27.jpg)