who will test_your_tests_yahya poonawala- priti biyani
TRANSCRIPT
WHO WILL TEST YOUR TESTS ?
Yahya Poonawala @meetykp
Priti Biyani @pritibiyani
2https://flic.kr/p/9uZhDS 2
TESTSCode works
Reduce Bugs
Reducecost of change
AllowRefactoring
Faster development
Reduce fear
3
NOW
WHEN YOU HEAR ….
4
Oh, it failed?
Don’t worry. Just re-trigger it!
5
Oh, it failed?
Strange; it passed on my machine!
6
Oh, it failed?
Run it on agent 007!
It’s an agent issue!
7
Oh, it failed?
I know. It fails at 12AM!
8
9
Now, again,
WHY DO WE WRITE TESTS?
1
0
Tests prove that the code actually works
Tests prove that the code
sometimes works
1
1
Tests reduce bugs
Tests itself might have bugs
1
2
Tests reduce the cost of change
Change? Are you kidding?
1
3
Tests helps in faster development.
Testing is slowing me down
1
4
NON-DETERMINISTIC
TESTS
A.K.A “Flaky Tests”
1
5
1
6http://www.trovatten.com/long-game/
1
7
1
8
http://tom-doherty.com/wp-content/uploads/2014/02/craigslist-waste-time.jpg
1
9
ILLUSION OF SUCCESS!
2
0
ILLUSION OF FAILURE!
2
1
2
2
2
3
DO NOT DELETE YOUR
FLAKY TESTS!!
2
4
CAUSES OF FLAKINESS
2
5
Thread 1::
if( name is not Yahya ) {
…
fputs( name, file )
…
}
Thread 2::
Initialize name to Yahya
CONCURRENCY
2
7
Initialize lock;
Thread 1::
Acquire lock
if( name is not Yahya) {
. . .
fputs( name, file )
. . .
}
Release lock
Thread 2::
Acquire lock
Initialize name to Yahya
Release lock
TIME
2
9
http://imgs.xkcd.com/comics/bug.png
EXTERNAL SYSTEM CALLS
3
0
UNORDERED COLLECTIONS
3
1
3
2https://www.vardot.com/sites/default/files/2.jpg
# pseudo-code
makeAsyncCall;
sleep(2000);
readResponse;
ASYNCHRONOUS
WAITING
3
4
make asynchronous call;
While (response received?)
if timeout then
throw Test Timeout Exception
else
sleep(10)
. . .
read response
RESOURCE LEAKS
3
6
# //pseudo-code
FILE_NAME = “content.txt”
// Test 1:
FileReader.read(FILE_NAME).to be empty
// Test2: Write to the file
data = “XP Conf 2015”
FileWriter.write(FILE_NAME, data)
FileReader.read(FILE_NAME) == data
// Test 3: Updates the file
append_data = “Schedule for talk”
complete_data = “XP Conf 2015. Schedule for talk ”
FileWrite.append(FILE_NAME, append_data)
FileReader.read(FILE_NAME) == complete_data
TEST ORDER DEPENDENCY
3
8
INPUT / OUTPUT
3
9
4
0
WHAT’S YOUR STRATEGY?
4
1
STOP CALLING YOUR BUILD
FLAKY!!
4
2
Identify Flaky tests
Quarantine
Plan Fix Tests
Monitor
4
3
IT’S NOT ALWAYS YOUR
TESTS
4
4
DO NOT GET
OVERWHELMED BY FAILING
TESTS
4
5
REMEMBER!
4
6
LOVE YOUR TEST CODE !
4
7
There is no such thing as
FLAKY TEST!
If its flaky, its not a test !4
8
AND THE FUTURE…
4
9
REFERENCES
• An Empirical Analysis of Flaky tests
• Eradicating Non-Determinism in Tests – Martin
Fowler
• No more flaky tests on the GO team - Pavan
Sudarshan
• Top 12 reasons to write unit tests
• http://tech.just-eat.com/2015/03/30/how-to-fix-
flaky-tests/
5
0