how to test asynchronous code (v2)
DESCRIPTION
Slides from my unit testing talk at JSBerlin on May 19.TRANSCRIPT
How to Test Asynchronous Code
by Felix Geisendörfer
19.05.2011 (v2)
@felixge
Twitter / GitHub / IRC
Core Contributor
&
Module Author
node-mysql node-formidable
- Joined the mailing list in June 26, 2009- When I joined there where #24 people- Now mailing list has close to 4000 members members- First patch in September 2009
How to Test Asynchronous
Code
- Disclaimer: Don’t listen to me : )
The current approach
test('something', function(done) { doSomethingAsync(function() { assert.equals(...);
done(); });});
The current approach
test('something', function(done) { doSomethingAsync(function() { assert.equals(...);
done(); });});
If you have multiple tests, how will exceptions be linked?
db.query('SELECT A', function() { db.query('SELECT B', function() { db.query('SELECT C', function() { db.query('SELECT D', function() { // WTF }); }); });});
- Who here is running node in production?- Raise hands if you have a test suite for your node.js projects- Raise hands if you are happy with it- But why is it so difficult?
test('something', function(done) { doSomethingAsync(function() { assert.equals(...);
done(); });});
F%$! that shit
Forget that shit
What can we do?
- We looked at asynchronous testing frameworks- We really wanted to do TDD- But everything we tried was hard
Take out the I/O
Stub / Mock Dependencies
Synchronous Tests
Unit TestsIf you have nested callbacks (or any other I/O) in your tests, you are not unit testing
Microtestshttp://anarchycreek.com/2009/05/20/theyre-called-microtests/
• It is short, typically under a dozen lines of code.
• It is always automated.
• It does not test the object inside the running app, but instead in a purpose-built testing application.
• It invokes only a tiny portion of the code, most usually a single branch of a single function.
• It is written gray-box, i.e. it reads as if it were black-box, but sometimes takes advantage of white-box knowledge. (Typically a critical factor in avoiding combinatoric issues.)
• It is coded to the same standard as shipping code, i.e. the team’s best current understanding of coding excellence.
• It is vault-committed source, with a lifetime co-terminous with the functionality it tests.
• In combination with all other microtests of an app, it serves as a ‘gateway-to-commit’. That is, a developer is encouraged to commit anytime all microtests run green, and discouraged (strongly, even nastily) to commit otherwise.
• It takes complete control of the object-under-test and is therefore self-contained, i.e. running with no dependencies on anything other than the testing code and its dependency graph.
• It runs in an extremely short time, milliseconds per test.
• It provides precise feedback on any errors that it encounters.
• It usually (not always) runs entirely inside a single computer.
• It usually (not always) runs entirely inside a single process, i.e. with few extra-process runtime dependencies.
• It is part of a collection all or any subset of which is invokable with a single programmer gesture.
• It is written before the code-change it is meant to test.
• It avoids most or all usage of ‘awkward’ collaborators via a variety of slip-and-fake techniques.
• It rarely involves construction of more than a few classes of object, often one or two, usually under five.
Approach
• Load SUT in a Sandbox using v8 / node.js
• Inject dependencies using the global object of the sandbox
• Don’t use test doubles for internals, only for peers
Internal: An object that has the same life span as its hostPeers: Something that is being passed in / out of the SUT
Test Doubles
• Dummy Object
• Test Stub
• Test Spy
• Mock Object
• Fake Object
• Temporary Test Stub
http://xunitpatterns.com/Mocks,%20Fakes,%20Stubs%20and%20Dummies.html
node-fake
node-fake
var fake = require('fake')();var object = {};
fake.expect(object, 'method');
object.method();
node-fake
var fake = require('fake')();var object = {};
var objectMethodCall = fake.stub(object, 'method');
object.method();
assert.equals(objectMethodCall.calls.length, 1);
node-fake
var fake = require('fake')();
var MyClass = fake.class('MyClass');fake .expect('new', MyClass) .withArgs(1, 2);
var myClass = new MyClass(1, 2);
node-microtest
node-microtest
var fs = require('fs');
module.exports = function(path) { var file = fs.createReadStream(path); file.pipe(process.stdout);};
cat.js
node-microtest
var test = require('microtest').module('cat.js');
test.requires('fs');test.context.process = { stdout: test.object('stdout'),};
var cat = test.compile();
test-cat.js
node-microtesttest.describe('cat', function() { var PATH = test.value('path'); var file = test.object('file');
test .expect(test.required.fs, 'createReadStream') .withArgs(PATH) .andReturn(file);
test .expect(file, 'pipe') .withArgs(test.context.process.stdout);
cat(PATH);});
test-cat.js
Benefits
We take the position that the real benefit of extensive microtest-driven development isn't higher quality at all. Higher quality is a side effect of TDD. Rather, the benefit and real purpose of TDD as we teach it is sheer productivity: more function faster.
Mike Hill
Benefits
• Simpler implementations
• Short change / verification cycles
• Tests for edge cases (error handling, race conditions, etc.)
There really are only two acceptable models of development: "think and analyze" or "years and years of testing on thousands of machines".
Linus Torvalds
Disadvantages
• Requires gray / white box knowledge of implementation
• Lots of test code needs to be written
• Sometimes feels awkward
http://martinfowler.com/articles/mocksArentStubs.html#ClassicalAndMockistTesting
Numbers
node-mysql
node-mysql
0
425
850
1.275
1.700
library tests
Lines of code
library vs test code: 1x : 1.35x
Library: 1240 LoC (+600 LoC MySql constants)Tests:Tests: 1673 LoC
node-mysql
0
325
650
975
1.300
integration micro
integration vs micro tests: 1x : 2.89x
Lines of code
Micro Tests: 1243 LoCIntegration tests: 430 LoC
node-mysql
0
100
200
300
400
integration micro
Assertions
integration vs micro tests: 1x : 8.15x
Micro Tests: 375 assertsIntegration tests: 46 asserts
Transloadit
Transloadit
0
3.250
6.500
9.750
13.000
library tests
Lines of code
library vs test code: 1x : 2.04x
Library: 6184 LoCTests: 12622 LoC
Not included: Fixture data, server configuration, customer website, dependencies we developed
Transloadit
0
2.500
5.000
7.500
10.000
integration micro
integration vs. micro tests: 1x : 4.30x
Lines of code
Integration tests: 2254 LoCMicro Tests: 9695 LoC
Transloadit
0
750
1.500
2.250
3.000
integration micro
Assertions
integration vs. micro tests: 1x : 12.30x
Integration tests: 189 assertsMicro Tests: 2324 asserts
tl;dr
• Don’t use integration tests to show the basic correctness of your software.
• Write more microtests.
Questions?
☝Thank You
Thank You
Node.js Workshop Cologne
nodecologne.eventbrite.com
Use discount code ‘berlinjs’ for 10% off