geecon 2012 bad tests, good tests
DESCRIPTION
Slides from my GeeCON 2012 presentation. Few were removed so I do not spoil the fun for those of you who plan to attend my talk on Confitura.TRANSCRIPT
![Page 1: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/1.jpg)
Bad Tests, Good Tests
Tomek Kaczanowski
![Page 2: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/2.jpg)
Tomek Kaczanowski
• Developer
• Team lead
• Blogger
• http://kaczanowscy.pl/tomek
• Book author
• http://practicalunittesting.com
• Working at CodeWise (Krakow)
...we are hiring, wanna join us?
![Page 3: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/3.jpg)
Before we begin
• Most of the examples are real but:
Obfuscated
to protect the innocents
Truncated
imagine much more complex domain objects
• Asking questions is allowed
...but being smarter than me is not ;)
![Page 4: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/4.jpg)
A little bit of history
![Page 5: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/5.jpg)
Before we begin
• The tests were written in 2004-2006.
• No automation, no CI.
• Some tests do not compile.
• In some tests you can read a comment that "WARNING:
test requires the divide param to be set to 20" but the
code is so ugly, that there is no way to inject this value.
• Some test data are available in form of serialized objects
(*.ser) that can not be currently deserialized, because
the classes have changed.
• The project is now in maintenance.
Courtesy of Bartosz http://twitter.com/#!/bocytko
![Page 6: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/6.jpg)
We don't need no stinkin' asserts! public void testAddChunks() {
System.out.println("*************************************");
System.out.println("testAddChunks() ... ");
ChunkMap cm = new ChunkMap(3);
cm.addChunk(new Chunk("chunk"));
List testList = cm.getChunks("chunk",null);
if (testList.isEmpty())
fail("there should be at least one list!");
Chunk chunk = cm.getActualChunk("chunk",null);
if (chunk.getElements().isEmpty())
fail("there should be at least one element!");
if (cm.getFinalChunkNr() != 1)
fail("there should be at least one chunk!");
// iterate actual chunk
for (Iterator it = chunk.getElements().iterator();
it.hasNext();) {
Element element = (Element) it.next();
System.out.println("Element: " + element);
}
showChunks(cm);
System.out.println("testAddChunks() OK ");
}
![Page 7: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/7.jpg)
Success is not an option...
/**
* Method testFailure.
*/
public void testFailure() {
try {
Message message = new Message(null,true);
fail();
} catch(Exception ex) {
ExceptionHandler.log(ExceptionLevel.ANY,ex);
fail();
}
}
![Page 8: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/8.jpg)
What has happened? Well, it failed...
public void testSimple() {
IData data = null;
IFormat format = null;
LinkedList<String> attr = new LinkedList<String>();
attr.add("A");
attr.add("B");
try {
format = new SimpleFormat("A");
data.setAmount(Amount.TEN);
data.setAttributes(attr);
IResult result = format.execute();
System.out.println(result.size());
Iterator iter = result.iterator();
while (iter.hasNext()) {
IResult r = (IResult) iter.next();
System.out.println(r.getMessage());
...
}
} catch (Exception e) {
fail();
}
}
![Page 9: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/9.jpg)
What has happened? Well, it failed...
public void testSimple() {
IData data = null;
IFormat format = null;
LinkedList<String> attr = new LinkedList<String>();
attr.add("A");
attr.add("B");
try {
format = new SimpleFormat("A");
data.setAmount(Amount.TEN);
data.setAttributes(attr);
IResult result = format.execute();
System.out.println(result.size());
Iterator iter = result.iterator();
while (iter.hasNext()) {
IResult r = (IResult) iter.next();
System.out.println(r.getMessage());
...
}
} catch (Exception e) {
fail();
}
}
data is still null here. Ready or not, NPE is coming.
![Page 10: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/10.jpg)
Talk to me
//wait for messages
do {
input = "";
try {
System.out.print(">");
read = System.in.read(buf);
//convert characters to string
input = new String(buf, 0, read - newline.length());
System.out.println(input);
if (input.equals("end") || input.equals("exit")
|| input.equals("stop") || input.equals("quit")) {
System.out.println("Terminating Test please wait...");
System.out.println("******* Test terminated *******");
toStop = true;
}
else {
System.out.println("Commands:" + newline + "'end',
'exit', 'stop' or 'quit' terminates this test ");
}
} catch (Exception e) {
e.printStackTrace();
}
} while (!toStop);
![Page 11: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/11.jpg)
Tests are boring – let us autogenerate them! /**
* Generated by JUnitDoclet, a tool provided by
* ObjectFab GmbH under LGPL.
* Please see www.junitdoclet.org, www.gnu.org
* and www.objectfab.de for informations about
* the tool, the licence and the authors.
*/
public class AdapterTest
// JUnitDoclet begin extends_implements
extends TestCase
// JUnitDoclet end extends_implements
{
// JUnitDoclet begin class
Adapter adapter = null;
// JUnitDoclet end class
public AdapterTest(String name) {
// JUnitDoclet begin method AdapterTest
super(name);
// JUnitDoclet end method AdapterTest
}
public Adapter createInstance() throws Exception {
// JUnitDoclet begin method testcase.createInstance
return new Adapter();
// JUnitDoclet end method testcase.createInstance
}
protected void setUp() throws Exception {
// JUnitDoclet begin method testcase.setUp
super.setUp();
adapter = createInstance();
// JUnitDoclet end method testcase.setUp
}
protected void tearDown() throws Exception {
// JUnitDoclet begin method testcase.tearDown
adapter = null;
super.tearDown();
// JUnitDoclet end method testcase.tearDown
public void testMain() throws Exception {
// JUnitDoclet begin method testMain
Adapter.main(new String [] {"ADAPTER"});
// JUnitDoclet end method testMain
}
/**
* JUnitDoclet moves marker to this method, if there is not match
* for them in the regenerated code and if the marker is not empty.
* This way, no test gets lost when regenerating after renaming.
* Method testVault is supposed to be empty.
*/
public void testVault() throws Exception {
// JUnitDoclet begin method testcase.testVault
// JUnitDoclet end method testcase.testVault
}
public static void main(String[] args) {
// JUnitDoclet begin method testcase.main
junit.textui.TestRunner.run(AdapterTest.class);
// JUnitDoclet end method testcase.main
}
}
![Page 12: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/12.jpg)
Tests are boring – let us autogenerate them!
public void testSetGetTimestamp() throws Exception {
// JUnitDoclet begin method setTimestamp getTimestamp
java.util.Calendar[] tests = {new GregorianCalendar(), null};
for (int i = 0; i < tests.length; i++) {
adapter.setTimestamp(tests[i]);
assertEquals(tests[i], adapter.getTimestamp());
}
// JUnitDoclet end method setTimestamp getTimestamp
}
public void testSetGetParam() throws Exception {
// JUnitDoclet begin method setParam getParam
String[] tests = {"a", "aaa", "---", "23121313", "", null};
for (int i = 0; i < tests.length; i++) {
adapter.setParam(tests[i]);
assertEquals(tests[i], adapter.getParam());
}
// JUnitDoclet end method setParam getParam
}
![Page 13: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/13.jpg)
Conclusions
• Automation!
• Running
• Verification
• Tests are to be written not generated
• You should be informed why your test failed
• Master your tools
…at least learn the basics!
![Page 14: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/14.jpg)
Few words about tests
![Page 15: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/15.jpg)
Why bother with tests?
• System works as expected
• Changes do not hurt
• Documentation
http://twitter.com/#!/devops_borat
![Page 16: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/16.jpg)
Tests help to achieve quality
Not sure when I saw this picture – probably in GOOS?
![Page 17: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/17.jpg)
What happens if we do it wrong?
• Angry clients
• Depressed developers
http://www.joshcanhelp.com
![Page 18: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/18.jpg)
When I started out with unit tests, I was enthralled with the promise of ease and security that they would bring to my projects. In practice, however, the theory of sustainable software through unit tests started to break down. This difficulty continued to build up, until I finally threw my head back in anger and declared that "Unit Tests have become more trouble than they are worth."
Llewellyn Falco and Michael Kennedy, Develop Mentor August 09
![Page 19: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/19.jpg)
http://chrispiascik.com/daily-drawings/express-yourself/
![Page 20: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/20.jpg)
The worst kind of tests
![Page 21: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/21.jpg)
No smoke without tests class SystemAdminSmokeTest extends GroovyTestCase {
void testSmoke() {
// do not remove below code
// def ds = new org.h2.jdbcx.JdbcDataSource(
// URL: 'jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle',
// user: 'sa', password: '')
//
// def jpaProperties = new Properties()
// jpaProperties.setProperty(
// 'hibernate.cache.use_second_level_cache', 'false')
// jpaProperties.setProperty(
// 'hibernate.cache.use_query_cache', 'false')
//
// def emf = new LocalContainerEntityManagerFactoryBean(
// dataSource: ds, persistenceUnitName: 'my-domain',
// jpaVendorAdapter: new HibernateJpaVendorAdapter(
// database: Database.H2, showSql: true,
// generateDdl: true), jpaProperties: jpaProperties)
// some more code below, all commented out :(
}
![Page 22: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/22.jpg)
Let's follow the leader!
@Test
public class ExampleTest {
public void testExample() {
assertTrue(true);
}
}
![Page 23: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/23.jpg)
Uh-oh, I feel lonely...
@Test
public class ExampleTest {
public void testExample() {
assertTrue(true);
}
}
![Page 24: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/24.jpg)
Flickering tests
![Page 25: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/25.jpg)
Asking for troubles...
LoggingPropertyConfigurator configurator
= mock(LoggingPropertyConfigurator.class);
BaseServletContextListener baseServletContextListener
= new BaseServletContextListener(configurator);
@Test public void shouldLoadConfigProperties() {
baseServletContextListener.contextInitialized();
verify(configurator).configure(any(Properties.class));
}
@Test(expected = LoggingInitialisationException.class)
public void shouldThrowLoggingException() {
System.setProperty("logConfig", "nonExistingFile");
baseServletContextListener.contextInitialized();
}
Should load some default config
Should load this specific file
![Page 26: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/26.jpg)
Lets mock!
![Page 27: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/27.jpg)
Mock'em All!
public String getUrl(User user, String timestamp) {
String name=user.getFullName();
String url=baseUrl
+"name="+URLEncoder.encode(name, "UTF-8")
+"×tamp="+timestamp;
return url;
}
public String getUrl(User user) {
Date date=new Date();
Long time=(date.getTime()/1000); //convert ms to seconds
String timestamp=time.toString();
return getUrl(user, timestamp);
}
![Page 28: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/28.jpg)
Mock'em All!
public String getUrl(User user, String timestamp) {
String name=user.getFullName();
String url=baseUrl
+"name="+URLEncoder.encode(name, "UTF-8")
+"×tamp="+timestamp;
return url;
}
public String getUrl(User user) {
Date date=new Date();
Long time=(date.getTime()/1000); //convert ms to seconds
String timestamp=time.toString();
return getUrl(user, timestamp);
}
@Test
public void shouldUseTimestampMethod() {
//given
Util spyUtil = spy(util);
//when
spyUtil.getUrl(user);
//then
verify(spyUtil).getUrl(eq(user), anyString());
}
![Page 29: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/29.jpg)
Use the front door
@Test
public void shouldAddTimestampToGeneratedUrl() {
//given
util = new ....
TimeProvider timeProvider = mock(TimeProvider.class);
when(timeProvider.getTime()).thenReturn("12345");
util.set(timeProvider);
//when
String url = util.getUrl(user);
//then
assertThat(url).contains("timestamp=12345");
}
![Page 30: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/30.jpg)
Mock'em All!
@Test
public void shouldAddTimeZoneToModelAndView() {
//given
UserFacade userFacade = mock(UserFacade.class);
ModelAndView modelAndView = mock(ModelAndView.class);
given(userFacade.getTimezone()).willReturn("timezone X");
//when
new UserDataInterceptor(userFacade)
.postHandle(null, null, null, modelAndView);
//then
verify(modelAndView).addObject("timezone", "timezone X");
}
![Page 31: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/31.jpg)
Use the front door
@Test
public void shouldAddTimeZoneToModelAndView() {
//given
UserFacade userFacade = mock(UserFacade.class);
ModelAndView modelAndView = new ModelAndView();
given(userFacade.getTimezone()).willReturn("timezone X");
//when
new UserDataInterceptor(userFacade)
.postHandle(null, null, null, modelAndView);
//then
assertThat(modelAndView).constains("timezone", "timezone X");
}
![Page 32: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/32.jpg)
Single Responsibility Principle
![Page 33: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/33.jpg)
SRP for tests
A test should have one and only one reason to fail.
![Page 34: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/34.jpg)
Testing two things at once
@DataProvider
public Object[][] data() {
return new Object[][] { {"48", true}, {"+48", true},
{"++48", true}, {"+48503", true}, {"+4", false},
{"++4", false}, {"", false},
{null, false}, {" ", false}, };
}
@Test(dataProvider = "data")
public void testQueryVerification(String query, boolean expected) {
assertEquals(expected, FieldVerifier.isValidQuery(query));
}
![Page 35: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/35.jpg)
Testing two things at once
@DataProvider
public Object[][] data() {
return new Object[][] { {"48", true}, {"+48", true},
{"++48", true}, {"+48503", true}, {"+4", false},
{"++4", false}, {"", false},
{null, false}, {" ", false}, };
}
@Test(dataProvider = "data")
public void testQueryVerification(String query, boolean expected) {
assertEquals(expected, FieldVerifier.isValidQuery(query));
}
Data
Algorithm / Logic
![Page 36: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/36.jpg)
Testing two things at once
@DataProvider
public Object[][] data() {
return new Object[][] { {"48", true}, {"+48", true},
{"++48", true}, {"+48503", true}, {"+4", false},
{"++4", false}, {"", false},
{null, false}, {" ", false}, };
}
@Test(dataProvider = "data")
public void testQueryVerification(String query, boolean expected) {
assertEquals(expected, FieldVerifier.isValidQuery(query));
}
testQueryVerification1() {
assertEquals(true, FieldVerifier.isValidQuery(„48”));
}
testQueryVerification2() {
assertEquals(true, FieldVerifier.isValidQuery(„+48”));
}
testQueryVerification3() {
assertEquals(true, FieldVerifier.isValidQuery(„++48”));
}
testQueryVerification4() {
assertEquals(true, FieldVerifier.isValidQuery(„+48503”));
}
...
![Page 37: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/37.jpg)
Testing two things at once
@DataProvider
public Object[][] data() {
return new Object[][] { {"48", true}, {"+48", true},
{"++48", true}, {"+48503", true}, {"+4", false},
{"++4", false}, {"", false},
{null, false}, {" ", false}, };
}
@Test(dataProvider = "data")
public void testQueryVerification(String query, boolean expected) {
assertEquals(expected, FieldVerifier.isValidQuery(query));
}
![Page 38: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/38.jpg)
Concentrate on one feature
@DataProvider
public Object[][] validQueries() {
return new Object[][] { {"48"}, {"48123"},
{"+48"}, {"++48"}, {"+48503"}};
}
@Test(dataProvider = "validQueries")
public void shouldRecognizeValidQueries(
String validQuery) {
assertTrue(FieldVerifier.isValidQuery(validQuery));
}
@DataProvider
public Object[][] invalidQueries() {
return new Object[][] {
{"+4"}, {"++4"},
{""}, {null}, {" "} };
}
@Test(dataProvider = "invalidQueries")
public void shouldRejectInvalidQueries(
String invalidQuery) {
assertFalse(FieldVerifier.isValidQuery(invalidQuery));
}
![Page 39: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/39.jpg)
Are you satisfied?
![Page 40: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/40.jpg)
Happy path
testSum() {
assertEquals(Math.sum(2,2), 4);
}
http://mw2.google.com/mw-panoramio/photos/medium/68775332.jpg
![Page 41: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/41.jpg)
Happy paths are for wimps
2 + 2
2 + -2
2 + -5
0 + 2
2 + 0
Integer.MAX_VALUE + something
etc.
http://kidskidskids.tumblr.com/post/1145294997
![Page 42: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/42.jpg)
Avoiding happy paths
testSum() {
assertEquals(Math.sum(2,2), 4);
}
http://kidskidskids.tumblr.com/post/1145294997
sum(int x, int y) {
return 4;
}
And then listen to your code.
Because it tells you something.
Start with one:
Do the simplest thing that works:
![Page 43: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/43.jpg)
Avoiding happy paths
http://looneytunes09.files.wordpress.com/2010/07/lisa-yell.gif
You moron! Your test is so pathetic, that I can make it pass by doing such a silly thing. Try harder!
sum(int x, int y) {
return 4;
}
![Page 44: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/44.jpg)
Readability is the king
![Page 45: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/45.jpg)
Who the heck is “user_2” ?
@DataProvider
public static Object[][] usersPermissions() {
return new Object[][]{
{"user_1", Permission.READ},
{"user_1", Permission.WRITE},
{"user_1", Permission.REMOVE},
{"user_2", Permission.WRITE},
{"user_2", Permission.READ},
{"user_3", Permission.READ}
};
}
![Page 46: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/46.jpg)
Ah, logged user can read and write...
@DataProvider
public static Object[][] usersPermissions() {
return new Object[][]{
{ADMIN, Permission.READ},
{ADMIN, Permission.WRITE},
{ADMIN, Permission.REMOVE},
{LOGGED, Permission.WRITE},
{LOGGED, Permission.READ},
{GUEST, Permission.READ}
};
}
![Page 47: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/47.jpg)
Do not make me learn the API!
server = new MockServer(responseMap, true,
new URL(SERVER_ROOT).getPort(), false);
![Page 48: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/48.jpg)
Do not make me learn the API!
server = new MockServer(responseMap, true,
new URL(SERVER_ROOT).getPort(), false);
private static final boolean RESPONSE_IS_A_FILE = true;
private static final boolean NO_SSL = false;
server = new MockServer(responseMap, RESPONSE_IS_A_FILE,
new URL(SERVER_ROOT).getPort(), NO_SSL);
![Page 49: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/49.jpg)
Do not make me learn the API!
server = new MockServer(responseMap, true,
new URL(SERVER_ROOT).getPort(), false);
server = new MockServerBuilder()
.withResponse(responseMap)
.withResponseType(FILE)
.withUrl(SERVER_ROOT)
.withoutSsl().create();
![Page 50: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/50.jpg)
What is really important?
![Page 51: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/51.jpg)
What is really important?
@DataProvider
public Object[][] snapshotArtifacts() {
return new Object[][]{
{"a", "b", "2.2-SNAPSHOT", Artifact.JAR },
{"c", "d", "2.2.4.6-SNAPSHOT", Artifact.JAR},
{"e", "f", "2-SNAPSHOT", Artifact.JAR}
};
}
@Test(dataProvider = "snapshotArtifacts")
public void shouldRecognizeSnapshots(
String groupId, String artifactId,
String version, Type type) {
Artifact artifact
= new Artifact(groupId, artifactId, version, type);
assertThat(artifact.isSnapshot()).isTrue();
}
![Page 52: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/52.jpg)
Only version matters
@DataProvider
public Object[][] snapshotVersions() {
return new Object[][]{
{"2.2-SNAPSHOT"},
{"2.2.4.6-SNAPSHOT"},
{"2-SNAPSHOT"}
};
}
@Test(dataProvider = "snapshotVersions")
public void shouldRecognizeSnapshots(String version) {
Artifact artifact
= new Artifact(VALID_GROUP, VALID_ARTIFACT_ID,
version, VALID_TYPE);
assertThat(artifact.isSnapshot()).isTrue();
}
![Page 53: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/53.jpg)
Test method names
![Page 54: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/54.jpg)
Test methods names are important
• testFindTransactionsToAutoCharge()
• testSystemSuccess()
• testOperation()
![Page 55: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/55.jpg)
Test methods names are important
@Test
public void testOperation() {
configureRequest("/validate")
rc = new RequestContext(parser, request)
assert rc.getConnector() == null
assert rc.getOperation().equals("validate")
}
• testFindTransactionsToAutoCharge()
• testSystemSuccess()
• testOperation()
![Page 56: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/56.jpg)
“should” is better than “test”
• shouldRejectInvalidRequests()
• shouldSaveNewUserToDatabase()
• constructorShouldFailWithNegativePrice()
• shouldReturnOnlyUsersWithGivenName()
• testOperation()
• testQuery()
• testConstructor()
• testFindUsersWithFilter()
![Page 57: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/57.jpg)
“should” is better than “test”
• Starting test method names
with “should” steers you in
the right direction.
• “test” prefix makes your test
method a limitless bag
where you throw everything
worth testing
http://www.greenerideal.com/
http://jochopra.blogspot.com/
![Page 58: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/58.jpg)
Test methods names are important
@Test
public void testQuery(){
when(q.getResultList()).thenReturn(null);
assertNull(dao.findByQuery(Transaction.class, q, false));
assertNull(dao.findByQuery(Operator.class, q, false));
assertNull(dao.findByQuery(null, null, false));
List result = new LinkedList();
when(q.getResultList()).thenReturn(result);
assertEquals(dao.findByQuery(Transaction.class, q, false), result);
assertEquals(dao.findByQuery(Operator.class, q, false), result);
assertEquals(dao.findByQuery(null, null, false), null);
when(q.getSingleResult()).thenReturn(null);
assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0);
assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0);
assertEquals(dao.findByQuery(null, null, true), null);
when(q.getSingleResult()).thenReturn(t);
assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t);
when(q.getSingleResult()).thenReturn(o);
assertSame(dao.findByQuery(Operator.class, q, true).get(0), o);
when(q.getSingleResult()).thenReturn(null);
assertSame(dao.findByQuery(null, null, true), null);
}
![Page 59: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/59.jpg)
Assertions
![Page 60: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/60.jpg)
Asserting using private methods
@Test
public void testChargeInRetryingState() throws Exception {
// given
TxDTO request = createTxDTO(RequestType.CHARGE);
AndroidTransaction androidTransaction = ...
// when
final TxDTO txDTO = processor.processRequest(request);
// then
assertState(request, androidTransaction,
CHARGED, CHARGE_PENDING, AS_ANDROID_TX_STATE,
ClientMessage.SUCCESS, ResultCode.SUCCESS);
}
![Page 61: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/61.jpg)
Matchers vs. private methods assertState(TxDTO txDTO, AndroidTransaction androidTransaction,
AndroidTransactionState expectedAndroidState,
AndroidTransactionState expectedPreviousAndroidState,
ExtendedState expectedState,
String expectedClientStatus,
ResultCode expectedRequestResultCode) {
final List<AndroidTransactionStep> steps
= new ArrayList<>(androidTransaction.getTransactionSteps());
final boolean checkPreviousStep = expectedAndroidState != null;
assertTrue(steps.size() >= (checkPreviousStep ? 3 : 2));
if (checkPreviousStep) {
AndroidTransactionStep lastStep = steps.get(steps.size() - 2);
assertEquals(lastStep.getTransactionState(),
expectedPreviousAndroidState);
}
final AndroidTransactionStep lastStep = steps.get(steps.size() - 1);
assertEquals(lastStep.getTransactionState(), expectedAndroidState);
assertEquals(lastStep.getMessage(), expectedClientStatus);
assertEquals(txDTO.getResultCode(), expectedRequestResultCode);
assertEquals(androidTransaction.getState(), expectedAndroidState);
assertEquals(androidTransaction.getExtendedState(), expectedState);
if (expectedClientStatus == null) {
verifyZeroInteractions(client);
}
}
![Page 62: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/62.jpg)
Matchers vs. private methods
@Test
public void testChargeInRetryingState() throws Exception {
// given
TxDTO request = createTxDTO(CHARGE);
AndroidTransaction androidTransaction = ...
// when
final TxDTO txDTO = processor.processRequest(request);
// then
assertThat(androidTransaction).hasState(CHARGED)
.hasMessage(ClientMessage.SUCCESS)
.hasPreviousState(CHARGE_PENDING)
.hasExtendedState(null);
assertEquals(txDTO.getResultCode(),
ResultCode.SUCCESS);
}
![Page 63: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/63.jpg)
Assertion part is freaking huge public void shouldPreDeployApplication() {
// given
Artifact artifact = mock(Artifact.class);
when(artifact.getFileName()).thenReturn("war-artifact-2.0.war");
ServerConfiguration config
= new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH);
Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);
String destDir = new File(".").getCanonicalPath() + SLASH + "target" + SLASH;
new File(destDir).mkdirs();
// when
tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));
//then
JSch jsch = new JSch();
jsch.addIdentity(KEY_FILE);
Session session = jsch.getSession(USER, ADDRESS, 22);
session.setConfig("StrictHostKeyChecking", "no");
session.connect();
Channel channel = session.openChannel("sftp");
session.setServerAliveInterval(92000);
channel.connect();
ChannelSftp sftpChannel = (ChannelSftp) channel;
sftpChannel.get(TEMP_PATH + SLASH + artifact.getFileName(), destDir);
sftpChannel.exit();
session.disconnect();
File downloadedFile = new File(destDir, artifact.getFileName());
assertThat(downloadedFile).exists().hasSize(WAR_FILE_LENGTH);
}
![Page 64: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/64.jpg)
Just say it
public void shouldPreDeployApplication() {
Artifact artifact = mock(Artifact.class);
when(artifact.getFileName())
.thenReturn(ARTIFACT_FILE_NAME);
ServerConfiguration config
= new ServerConfiguration(ADDRESS, USER,
KEY_FILE, TOMCAT_PATH, TEMP_PATH);
Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config);
tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH));
SSHServerAssert.assertThat(ARTIFACT_FILE_NAME)
.existsOnServer(config).hasSize(WAR_FILE_LENGTH);
}
![Page 65: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/65.jpg)
What is asserted?
@Test
public void testCompile_32Bit_FakeSourceFile() {
CompilerSupport _32BitCompilerSupport
= CompilerSupportFactory.getDefault32BitCompilerSupport();
testCompile_FakeSourceFile(_32BitCompilerSupport);
}
![Page 66: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/66.jpg)
What is asserted?
@Test
public void testCompile_32Bit_FakeSourceFile() {
CompilerSupport _32BitCompilerSupport
= CompilerSupportFactory.getDefault32BitCompilerSupport();
testCompile_FakeSourceFile(_32BitCompilerSupport);
}
private void testCompile_FakeSourceFile(
CompilerSupport compilerSupport) {
compiledFiles
= compilerSupport.compile(new File[] { new File("fake") });
assertThat(compiledFiles, is(emptyArray()));
}
![Page 67: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/67.jpg)
Asserting everything
public void invalidTxShouldBeCanceled() {
String fileContent =
FileUtils.getContentOfFile("response.csv");
assertTrue(fileContent.contains(
"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));
}
![Page 68: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/68.jpg)
Asserting everything
public void invalidTxShouldBeCanceled() {
String fileContent =
FileUtils.getContentOfFile("response.csv");
assertTrue(fileContent.contains(
"CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));
}
public void invalidTxShouldBeCanceled() {
String fileContent =
FileUtils.getContentOfFile("response.csv");
TxDTOAssert.assertThat(fileContent)
.hasTransaction("123cancel").withResultCode(SUCCESS);
}
![Page 69: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/69.jpg)
Know your tool
![Page 70: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/70.jpg)
Expected exceptions
@Test(expectedExceptions = SmsException.class)
public void shouldThrowException() throws SmsException {
try {
String s = gutExtractor.extractGut(„invalid gut”);
System.out.println(s);
} catch (SmsException e) {
e.printStackTrace();
throw e;
}
}
![Page 71: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/71.jpg)
Expected exceptions
@Test(expectedExceptions = SmsException.class)
public void shouldThrowException() throws SmsException {
String s = gutExtractor.extractGut(„invalid gut”);
}
![Page 72: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/72.jpg)
Expected exceptions (with catch-exception)
@Test
public void shouldThrowException() throws SmsException {
when(gutExtractor.extractGut(„invalid gut”));
then(caughtException())
.isInstanceOf(SmsException.class)
.hasMessage("Invalid gut")
.hasNoCause();
}
http://code.google.com/p/catch-exception/
![Page 73: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/73.jpg)
Running SUT's code concurrently
@Test(threadPoolSize = 3, invocationCount = 10)
public void testServer() {
// this method will be run in parallel by 3 thread
// 10 invocations (in total)
}
![Page 74: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/74.jpg)
Dependent test methods
@Test
public void shouldConnectToDB() {
// verifying that you can
// estabilish a connection with DB
}
@Test(dependsOnMethods = „shouldConnectToDB”)
public void should…() {
// some operations on DB
}
![Page 75: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/75.jpg)
Know your tools
• Unit testing framework
Use of temporary file rule
Listeners
Concurrency
@Before/@After
Parametrized tests
Test dependencies
Additional libraries
Hamcrest, FEST, Mockito,
catch-exception, …
• Build tool
Parallel execution
CI
• IDE
Templates
Shortcuts
![Page 76: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/76.jpg)
What do you really want to test?
![Page 77: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/77.jpg)
What do you really want to test?
@Test
public void shouldAddAUser() {
User user = new User();
userService.save(user);
assertEquals(dao.getNbOfUsers(), 1);
}
![Page 78: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/78.jpg)
You wanted to see that the number increased
@Test
public void shouldAddAUser() {
Int nb = dao.getNbOfUsers();
User user = new User();
userService.save(user);
assertEquals(dao.getNbOfUsers(), nb + 1);
}
![Page 79: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/79.jpg)
Random
![Page 80: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/80.jpg)
Doing it wrong
public void myTest() {
SomeObject obj = new SomeObject(
a, b, c, productCode());
// testing of obj here
}
private String productCode(){
String[] codes = {"Code A", "Code B",
"Code C", "Code D"};
int index = rand.nextInt(codes.length);
return codes[index];
}
![Page 81: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/81.jpg)
The dream of stronger, random-powered tests
public void myTest() {
SomeObject obj = new SomeObject(
randomName(), randomValue(), ....);
// testing of obj here
}
Does it make your test stronger?
![Page 82: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/82.jpg)
The dream of stronger, random-powered tests
public void myTest() {
SomeObject obj = new SomeObject(
randomName(), randomValue(), ....);
// testing of obj here
}
Does it make your test stronger?
...or does it only bring confusion?
Test failed
Expected SomeObject(„a”, „b”, ....) but got
Expected SomeObject(„*&O*$NdlF”, „#idSLNF”, ....)
![Page 83: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/83.jpg)
Conclusions
![Page 84: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/84.jpg)
There is more to it
• Integration / end-to-end tests which are not parametrized
(so they all try to set up jetty on port 8080),
• Tests which should be really unit, but use Spring context
to create objects,
• Tests with a lot of dependencies between them (a
nightmare to maintain!),
• Tests which run slow
• Tests which try to cover the deficiencies of production
code and end up being a total mess,
• etc., etc.
![Page 85: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/85.jpg)
Test-last? No!
• makes people not write tests at all
• makes people do only happy-testing
• tests reflect the implementation
![Page 86: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/86.jpg)
For six or eight hours spread over the next few weeks I struggled to get the first test written and running. Writing tests for Eclipse plug-ins is not trivial, so it’s not surprising I had some trouble. [...] In six or eight hours of solid programming time, I can still make significant progress. If I’d just written some stuff and verified it by hand, I would probably have the final answer to whether my idea is actually worth money by now. Instead, all I have is a complicated test that doesn’t work, a pile of frustration, eight fewer hours in my life, and the motivation to write another essay.
Ken Beck, Just Ship, Baby
Always TDD?
![Page 87: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/87.jpg)
Treat tests as the first class citizens
• do it everyday or forget about it
• use the right tool for the job
• and learn to use it!
• do not live with broken windows
• respect KISS, SRP, DRY (?)
• write good code, and you will also write
good tests
• or rather write good tests and you
will get good code for free
• code review your tests
• do more than happy path testing
• do not make the reader learn the API,
make it obvious
• bad names lead to bad tests
• make tests readable using matchers,
builders and good names
• test behaviour not methods
• be pragmatic about the tests you write
• TDD always?
• what is the best way to test it?
unit/integration/end-to-end ?
• automate!
• always concentrate on what is worth
testing
• ask yourself questions like: 'is it
really important that X should send
message Y to Z?'
• use the front door – state testing before
interaction testing (mockc)
![Page 88: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/88.jpg)
…questions?
…rants?
…hate speeches?
…any other forms of expressing your ego?
P.S. If you have some „interesting” tests, I would be happy to see them. Send them to me, please!
![Page 89: GeeCON 2012 Bad Tests, Good Tests](https://reader034.vdocument.in/reader034/viewer/2022052310/554d2accb4c905c5208b50c8/html5/thumbnails/89.jpg)
Thank you for watching these
slides! You can learn more about
wirting high quality tests by
reading my book – „Practical Unit
Testing with TestNG and
Mockito”.
Regards,
Tomek Kaczanowski
http://practicalunittesting.com
Thank you!