How to express your requirements as tests, and vice versaJohannes Brodwall
Understanding the problemJohannes Brodwall
How to express your requirements as tests, and vice versaJohannes Brodwall
FitNesse eksempel #1
FitNesse eksempel #1
Understanding the problemJohannes Brodwall
Chief Scientist, Steria
Background: What’s a requirement?
© Steria| 11/04/23 Presentation titlep9 © Steria
”Why does the customer
want a solution?”
© Steria| 11/04/23 Presentation titlep10 © Steria
Understanding the need
© Steria| 11/04/23 Presentation titlep11 © Steria
Tools:
•Vision statements
•User analysis
© Steria| 11/04/23 Presentation titlep12 © Steria
”What do we implement
when?”
© Steria| 11/04/23 Presentation titlep13 © Steria
Planning
© Steria| 11/04/23 Presentation titlep14 © Steria
Tools:
•Product backlog
•User stories
© Steria| 11/04/23 Presentation titlep15 © Steria
”How should the
functionality work?”
© Steria| 11/04/23 Presentation titlep16 © Steria
Requirement specification
© Steria| 11/04/23 Presentation titlep17 © Steria
Tools:
•Acceptance tests
Tests emerge from discussions
Example: Electronic signature
© Steria| 11/04/23 Presentation titlep20 © Steria
Developer
Product owner
Tester
© Steria| 11/04/23 Presentation titlep21 © Steria
Developer
Product owner
Tester
© Steria| 11/04/23 Presentation titlep22 © Steria
Developer
Product owner
Tester
As a payment issuer,I want to sign my payments,So that nobody can impersonate me
© Steria| 11/04/23 Presentation titlep23 © Steria
Developer
Product owner
Tester
What about a file that contains both signed and failed payments?
© Steria| 11/04/23 Presentation titlep24 © Steria
Developer
Product owner
Tester
Ummm......(help?)
© Steria
© Steria| 11/04/23 Presentation titlep26 © Steria
Developer
Product owner
Tester
Given ....When ....Then ...
© Steria| 11/04/23 Presentation titlep27 © Steria
Developer
Product owner
Tester
Given a file with payment 1 and payment 2And payment 1 is signed correctlyAnd payment 2 is signed incorrectlyWhen the file is validatedThen payment 1 should be processed as usualAnd the payment issuer should receive a receipt indicating payment 2 was rejected
An anonymized example from a real project
© Steria
FitNesse eksempel #1
© Steria
© Steria| 11/04/23 Presentation titlep31 © Steria
Developer
Product owner
Tester
Given ....When ....Then ...
A practical example
© Steria| 11/04/23 Presentation titlep33 © Steria
Developer
Product owner
Tester
Given ....When ....Then ...
Cucumber(rspec)
© Steria
From my current project:Feature: ”Assign task to me”
Given I go to the work task screen ”team tasks”And I pick the first task with ”SSN” equal ”11111111”And I press the button ”Take task”When I go to the work task screen ”my tasks”Then the work task list should contain 1 taskAnd the ”SSN” of the task should be ”1111111”
© Steria
Executed automatically
Given I go to the work task screen ”team tasks”And I pick the first task with ”SSN” equal ”11111111”And I press the button ”Take task”When I go to the work task screen ”my tasks”Then the work task list should contain 1 taskAnd the ”SSN” of the task should be ”1111111”
Fictive example: Yahtzee
© Steria| 11/04/23 Presentation titlep37 © Steria
Developer
Product owner
Tester
Full house should give 25 point
© Steria
© Steria| 11/04/23 Presentation titlep39 © Steria
Developer
Product owner
Tester
Full house should give 25 point
© Steria| 11/04/23 Presentation titlep40 © Steria
Developer
Product owner
Tester
Is it a full house if five die_values all have the same pipcount?
© Steria| 11/04/23 Presentation titlep41 © Steria
Developer
Product owner
Tester
WTF?!?
© Steria
© Steria| 11/04/23 Presentation titlep43 © Steria
Developer
Product owner
Tester
For example five dice all reading 1 is not full house
Ok!
© Steria
Acceptance tests communicate requirements
Developers tests
Reflect functional tests
Should express requirements (but smaller)
© Steria
JUnit example: Repository
@Testpublic void shouldRetrieveSameInstanceForSameKey() throws Exception { Category inserted = new Category("A"); Serializable id = repository.insert(inserted); repository.flushChanges();
Category retrieved1 = repository.retrieve(Category.class, id); Category retrieved2 = repository.retrieve(Category.class, id); Category retrieved3 = repository.find(Category.class).iterator().next(); retrieved1.setCategoryName("Z"); assertEquals(retrieved1.getCategoryName(), retrieved2.getCategoryName()); assertEquals(retrieved1.getCategoryName(), retrieved3.getCategoryName());}
© Steria
@Testpublic void shouldRetrieveSameInstanceForSameKey() throws Exception { Category inserted = new Category("A"); Serializable id = repository.insert(inserted); repository.flushChanges();
Category retrieved1 = repository.retrieve(Category.class, id); Category retrieved2 = repository.retrieve(Category.class, id); Category retrieved3 = repository.find(Category.class).iterator().next(); retrieved1.setCategoryName("Z"); assertEquals(retrieved1.getCategoryName(), retrieved2.getCategoryName()); assertEquals(retrieved1.getCategoryName(), retrieved3.getCategoryName());}
JUnit: Repository
When I modify one of the retrieved instances
Then the others should be updated, too
Given a database with one object
Given I retrieve this object several times
© Steria
JUnit: Repository
@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();
assertNull(retrieveInNewThread(Category.class, category.getId()));
repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}
© Steria
JUnit: Repository
@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();
assertNull(retrieveInNewThread(Category.class, category.getId()));
repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}
Given I insert a new object while in a transaction
When I retrieve the object from another thread
Then I should not be able to see it
When I commit the transactionWhen I retrieve the object from another threadThen I should be able to see it
© Steria
JUnit: Repository
@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();
assertNull(retrieveInNewThread(Category.class, category.getId()));
repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}
Given I insert a new object while in a transaction
When I retrieve the object from another thread
Then I should not be able to see it
When I commit the transactionWhen I retrieve the object from another threadThen I should be able to see it
Uncommitted Inserts Should
Be Invisible For Other
Threads
© Steria
JUnit: Web tests
@Testpublic void listProductsPageShouldShowAll() throws Exception { Product product1 = new Product(uniqueName("product"), 12300); Product product2 = new Product(uniqueName("product"), 300); repository.insertAll(product1, product2); repository.flushChanges();
tester.beginAt("/products/"); tester.assertTextInElement("products", product1.getProductName()); tester.assertTextInElement("products", product2.getProductName());}
© Steria
JUnit: Web tests
@Testpublic void listProductsPageShouldShowAll() throws Exception { Product product1 = new Product(uniqueName("product"), 12300); Product product2 = new Product(uniqueName("product"), 300); repository.insertAll(product1, product2); repository.flushChanges();
tester.beginAt("/products/"); tester.assertTextInElement("products", product1.getProductName()); tester.assertTextInElement("products", product2.getProductName());}
Then I should see both products
Given two products with unique names in the database
When I go to the /products/ web page
© Steria
JUnit : Negative tests
@Test
public void priceMustBeNumeric() throws Exception {
String oldName = uniqueName("product"); int oldPrice = 1234;
Product product = new Product(oldName, oldPrice); Serializable id = repository.insert(product); repository.flushChanges();
tester.beginAt("/products/" + id + "/edit.html"); tester.setTextField("productName", uniqueName("product")); tester.setTextField("price", "this is not a price!"); tester.submit();
tester.assertMatchInElement("errorExplaination", "[Pp]rice .*numeric");
Product stored = repository.retrieve(Product.class, id); assertEquals(oldPrice, stored.getPrice()); assertEquals(oldName, stored.getProductName());}
© Steria
JUnit: Negative tests
@Testpublic void priceMustBeNumeric() throws Exception { String oldName = uniqueName("product"); int oldPrice = 1234;
Product product = new Product(oldName, oldPrice); Serializable id = repository.insert(product); repository.flushChanges();
tester.beginAt("/products/" + id + "/edit.html"); tester.setTextField("productName", uniqueName("product")); tester.setTextField("price", "this is not a price!"); tester.submit();
tester.assertMatchInElement("errorExplaination", "[Pp]rice .*numeric");
Product stored = repository.retrieve(Product.class, id); assertEquals(oldPrice, stored.getPrice()); assertEquals(oldName, stored.getProductName());}
Then I should see an error message
Given a product in the database
When I go to the /products/<id>/edit web pageAnd I go change the price to a negative valueAnd I press submit
And the product should be unchanged in the database
© Steria| 11/04/23 Presentation titlep58 © Steria
Developer
Given ....When ....Then ...
Good design can be grown
© Steria
JUnit: Yahtzee histogram
@Test public void simpleCategoriesShouldBeSumOfMatchingDice() { ... }
@Test public void smallStrait() { ... }
@Test public void largeStrait() { ... }
@Test public void threeOfAKind() { ... }
© Steria
JUnit: Yahtzee histogram
@Test public void threeOfAKind() {
assertEquals(0, scoreFor("three_of_a_kind", 1, 1, 2, 2, 3));
assertEquals(3, scoreFor("three_of_a_kind", 1, 1, 1, 2, 3));
assertEquals(6, scoreFor("three_of_a_kind", 2, 2, 2, 3, 3));
assertEquals(9, scoreFor("three_of_a_kind", 1, 1, 3, 3, 3));
}
© Steria
JUnit: Yahtzee histogram
@Test public void fullHouse() {
assertEquals(0, scoreFor("full_house", 1, 1, 2, 2, 3));
assertEquals(0, scoreFor("full_house", 1, 1, 1, 2, 3));
assertEquals(25, scoreFor("full_house", 1, 1, 1, 2, 2));
assertEquals(25, scoreFor("full_house", 1, 1, 2, 2, 2));
assertEquals(25, scoreFor("full_house", 5, 5, 6, 6, 6));
}
© Steria
JUnit: Yahtzee histogram
@Test public void histogramShouldReturnFrequencyOfEachDie() {
int[] roll = { 1, 1, 2, 3, 4 };
int[] histogram = new ScoreCard().histogram(roll);
assertEquals(7, histogram.length);
assertEquals(-1, histogram[0]);
assertEquals(2, histogram[1]);
assertEquals(1, histogram[2]);
assertEquals(1, histogram[3]);
assertEquals(1, histogram[4]);
assertEquals(0, histogram[5]);
assertEquals(0, histogram[6]);
© Steria
JUnit: Yahtzee histogram
scoreCalculators.put("four_of_a_kind", new ScoreCalculator() {
@Override
public int calculate(int[] histogram) {
for (int i=0; i<histogram.length; i++) {
if (histogram[i] >= 4) return i*4;
}
return 0;
}
});
Testing: Costs and benefits
© Steria| 11/04/23 Presentation titlep66 © Steria
Ron Jeffries
Only test what you want to work
© Steria| 11/04/23 Presentation titlep67 © Steria
Johannes
Find the defect where it’s cheapest
© Steria
Unscientific graph!
Compiler Junit FitNesse System test Prod
Cost
Realism
© Steria
Unscientific graph!
Compiler Junit FitNesse System test Prod
Cost
Realism
Chance of finding defects
© Steria
Unscientific graph!
Compiler Junit FitNesse System test Prod
Cost
Realism
Cost of running and changing
tests
© Steria
Unscientific graph!
Compiler Junit FitNesse System test Prod
Cost
Realism
Cost of a defect
© Steria| 11/04/23 Presentation titlep72 © Steria
Johannes
Unit tests can reduce maintaince costs. (How?)
© Steria| 11/04/23 Presentation titlep73 © Steria
Johannes
When will unit tests increase maintainance cost?
© Steria| 11/04/23 Presentation titlep74 © Steria
Brian Marick
Automated tests are expensive to maintainIf the system wasn’t written to be tested
But maintainability is only one reason to test!
How to create a good solution?
Understand the problem