holy test pyramid - it weekend · codepipestesting metrics (ctm ) metric name description ideal...
TRANSCRIPT
Holy "Test Pyramid"How it ruins your life
About
Yaroslav Pernerovsky
@ysparrow
About
https://qaguild-slack.herokuapp.com/
About
http://automation-remarks.com/podcast/
Unit
Integration
E2E
Unit
Integration
E2E
Manual
a
• http://blog.codepipes.com/testing/software-testing-antipatterns.html
Unit
Integration
E2E
White-boxIndependent unitsMocks/stubs
White/Black-boxSome deploymentExternal systems
Black-boxFully deployed systemReal life scenarios
Integrationü uses databaseü uses network to call another component/application
ü external system (MQ, mail server etc.)ü reads/writes files or performs other I/Oü does not rely on source code, uses deployed binary instead
http://blog.codepipes.com/testing/software-testing-antipatterns.html
What is wrong with the Pyramid?
It is a theoreticalmetaphor
It is almost unachievable in
real world
Unit
Integration
E2E
Created by Developers
Sometimes by DevelopersSometimes by Testers
Usually by Testers
Contexts differences
Communication issues
Industry stereotypes
Developers should create all the tests to build
the canonical test pyramid
Become a "developer"
Talk to each other
Brake stereotypes
What is wrong with the tests in Pyramid?
CC=3 CC=2 CC=4++
CC=3 CC=2 CC=4
=9
=24
Having the "wrong" kind of tests
Unit
Integration
Having the "wrong" kind of tests
Unit
Integration
Having the "wrong" kind of tests
Unit
Integration
GUI
Testing the wrong functionality
Core
Critical
Other
20%
Code coverage
Codepipes Testing Metrics (CTM)Metric Name Description Ideal value Usual value Problematic
value
PDWT% of Developers writing tests
100% 20%-70% Anything less than 100%
PBCNT% of bugs that create new tests
100% 0%-5% Anything less than 100%
PTVB% of tests that verify behavior
100% 10% Anything less than 100%
PTD% of tests that are deterministic
100% 50%-80% Anything less than 100%
http://blog.codepipes.com/testing/software-testing-antipatterns.html
0% 100%Code coverage
effo
rt n
eeded
0% 100%Code coverage
Valu
e fr
om
tes
ts
Treating TDD as a religion
ü you can write tests before
ü you can write tests at the same time
ü you can write tests after
ü you can never write tests
http://blog.codepipes.com/testing/software-testing-antipatterns.html
Itogi podvedyom
ü Triangle is a mental abstractionü It should be treated as abstractionü Think before actü Don't blame yourselfü Use right metricsü Improve your programming skillsü Right tests over making trianglesü Do not hesitate to kill shitty tests