holy test pyramid - it weekend · codepipestesting metrics (ctm ) metric name description ideal...
TRANSCRIPT
![Page 1: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/1.jpg)
Holy "Test Pyramid"How it ruins your life
![Page 2: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/2.jpg)
About
Yaroslav Pernerovsky
@ysparrow
![Page 3: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/3.jpg)
About
https://qaguild-slack.herokuapp.com/
![Page 4: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/4.jpg)
About
http://automation-remarks.com/podcast/
![Page 5: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/5.jpg)
Unit
Integration
E2E
![Page 6: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/6.jpg)
Unit
Integration
E2E
Manual
![Page 7: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/7.jpg)
a
![Page 8: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/8.jpg)
![Page 9: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/9.jpg)
![Page 10: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/10.jpg)
![Page 11: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/11.jpg)
• http://blog.codepipes.com/testing/software-testing-antipatterns.html
![Page 12: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/12.jpg)
Unit
Integration
E2E
White-boxIndependent unitsMocks/stubs
White/Black-boxSome deploymentExternal systems
Black-boxFully deployed systemReal life scenarios
![Page 13: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/13.jpg)
Integrationü uses databaseü uses network to call another component/application
ü external system (MQ, mail server etc.)ü reads/writes files or performs other I/Oü does not rely on source code, uses deployed binary instead
http://blog.codepipes.com/testing/software-testing-antipatterns.html
![Page 14: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/14.jpg)
What is wrong with the Pyramid?
![Page 15: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/15.jpg)
It is a theoreticalmetaphor
![Page 16: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/16.jpg)
It is almost unachievable in
real world
![Page 17: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/17.jpg)
Unit
Integration
E2E
Created by Developers
Sometimes by DevelopersSometimes by Testers
Usually by Testers
![Page 18: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/18.jpg)
![Page 19: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/19.jpg)
Contexts differences
Communication issues
Industry stereotypes
![Page 20: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/20.jpg)
Developers should create all the tests to build
the canonical test pyramid
![Page 21: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/21.jpg)
![Page 22: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/22.jpg)
Become a "developer"
Talk to each other
Brake stereotypes
![Page 23: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/23.jpg)
What is wrong with the tests in Pyramid?
![Page 24: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/24.jpg)
CC=3 CC=2 CC=4++
CC=3 CC=2 CC=4
=9
=24
![Page 25: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/25.jpg)
Having the "wrong" kind of tests
Unit
Integration
![Page 26: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/26.jpg)
Having the "wrong" kind of tests
Unit
Integration
![Page 27: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/27.jpg)
Having the "wrong" kind of tests
Unit
Integration
GUI
![Page 28: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/28.jpg)
Testing the wrong functionality
Core
Critical
Other
![Page 29: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/29.jpg)
20%
![Page 30: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/30.jpg)
Code coverage
![Page 31: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/31.jpg)
Codepipes Testing Metrics (CTM)Metric Name Description Ideal value Usual value Problematic
value
PDWT% of Developers writing tests
100% 20%-70% Anything less than 100%
PBCNT% of bugs that create new tests
100% 0%-5% Anything less than 100%
PTVB% of tests that verify behavior
100% 10% Anything less than 100%
PTD% of tests that are deterministic
100% 50%-80% Anything less than 100%
http://blog.codepipes.com/testing/software-testing-antipatterns.html
![Page 32: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/32.jpg)
0% 100%Code coverage
effo
rt n
eeded
![Page 33: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/33.jpg)
0% 100%Code coverage
Valu
e fr
om
tes
ts
![Page 34: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/34.jpg)
Treating TDD as a religion
ü you can write tests before
ü you can write tests at the same time
ü you can write tests after
ü you can never write tests
http://blog.codepipes.com/testing/software-testing-antipatterns.html
![Page 35: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/35.jpg)
Itogi podvedyom
ü Triangle is a mental abstractionü It should be treated as abstractionü Think before actü Don't blame yourselfü Use right metricsü Improve your programming skillsü Right tests over making trianglesü Do not hesitate to kill shitty tests
![Page 36: Holy Test Pyramid - IT Weekend · CodepipesTesting Metrics (CTM ) Metric Name Description Ideal value Usual value Problematic value PDWT % of Developers writing tests 100% 20%-70%](https://reader033.vdocument.in/reader033/viewer/2022042518/5fac9c5def16d81da707b3a2/html5/thumbnails/36.jpg)