how to express your requirements as tests, and vice versa johannes brodwall

How to express your requirements as tests, and vice versaJohannes Brodwall

Understanding the problemJohannes Brodwall

How to express your requirements as tests, and vice versaJohannes Brodwall

FitNesse eksempel #1

FitNesse eksempel #1

Understanding the problemJohannes Brodwall

Chief Scientist, Steria

Background: What’s a requirement?

© Steria| 11/04/23 Presentation titlep9 © Steria

”Why does the customer

want a solution?”

© Steria| 11/04/23 Presentation titlep10 © Steria

Understanding the need

© Steria| 11/04/23 Presentation titlep11 © Steria

Tools:

•Vision statements

•User analysis

© Steria| 11/04/23 Presentation titlep12 © Steria

”What do we implement

when?”

© Steria| 11/04/23 Presentation titlep13 © Steria

Planning

© Steria| 11/04/23 Presentation titlep14 © Steria

Tools:

•Product backlog

•User stories

© Steria| 11/04/23 Presentation titlep15 © Steria

”How should the

functionality work?”

© Steria| 11/04/23 Presentation titlep16 © Steria

Requirement specification

© Steria| 11/04/23 Presentation titlep17 © Steria

Tools:

•Acceptance tests

Tests emerge from discussions

Example: Electronic signature

© Steria| 11/04/23 Presentation titlep20 © Steria

Developer

Product owner

Tester

© Steria| 11/04/23 Presentation titlep21 © Steria

Developer

Product owner

Tester

© Steria| 11/04/23 Presentation titlep22 © Steria

Developer

Product owner

Tester

As a payment issuer,I want to sign my payments,So that nobody can impersonate me

© Steria| 11/04/23 Presentation titlep23 © Steria

Developer

Product owner

Tester

What about a file that contains both signed and failed payments?

© Steria| 11/04/23 Presentation titlep24 © Steria

Developer

Product owner

Tester

Ummm......(help?)

© Steria

© Steria| 11/04/23 Presentation titlep26 © Steria

Developer

Product owner

Tester

Given ....When ....Then ...

© Steria| 11/04/23 Presentation titlep27 © Steria

Developer

Product owner

Tester

Given a file with payment 1 and payment 2And payment 1 is signed correctlyAnd payment 2 is signed incorrectlyWhen the file is validatedThen payment 1 should be processed as usualAnd the payment issuer should receive a receipt indicating payment 2 was rejected

An anonymized example from a real project

© Steria

FitNesse eksempel #1

© Steria

© Steria| 11/04/23 Presentation titlep31 © Steria

Developer

Product owner

Tester

Given ....When ....Then ...

A practical example

© Steria| 11/04/23 Presentation titlep33 © Steria

Developer

Product owner

Tester

Given ....When ....Then ...

Cucumber(rspec)

© Steria

From my current project:Feature: ”Assign task to me”

Given I go to the work task screen ”team tasks”And I pick the first task with ”SSN” equal ”11111111”And I press the button ”Take task”When I go to the work task screen ”my tasks”Then the work task list should contain 1 taskAnd the ”SSN” of the task should be ”1111111”

© Steria

Executed automatically

Given I go to the work task screen ”team tasks”And I pick the first task with ”SSN” equal ”11111111”And I press the button ”Take task”When I go to the work task screen ”my tasks”Then the work task list should contain 1 taskAnd the ”SSN” of the task should be ”1111111”

Fictive example: Yahtzee

© Steria| 11/04/23 Presentation titlep37 © Steria

Developer

Product owner

Tester

Full house should give 25 point

© Steria

© Steria| 11/04/23 Presentation titlep39 © Steria

Developer

Product owner

Tester

Full house should give 25 point

© Steria| 11/04/23 Presentation titlep40 © Steria

Developer

Product owner

Tester

Is it a full house if five die_values all have the same pipcount?

© Steria| 11/04/23 Presentation titlep41 © Steria

Developer

Product owner

Tester

WTF?!?

© Steria

© Steria| 11/04/23 Presentation titlep43 © Steria

Developer

Product owner

Tester

For example five dice all reading 1 is not full house

Ok!

© Steria

Acceptance tests communicate requirements

Developers tests

Reflect functional tests

Should express requirements (but smaller)

© Steria

JUnit example: Repository

@Testpublic void shouldRetrieveSameInstanceForSameKey() throws Exception { Category inserted = new Category("A"); Serializable id = repository.insert(inserted); repository.flushChanges();

Category retrieved1 = repository.retrieve(Category.class, id); Category retrieved2 = repository.retrieve(Category.class, id); Category retrieved3 = repository.find(Category.class).iterator().next(); retrieved1.setCategoryName("Z"); assertEquals(retrieved1.getCategoryName(), retrieved2.getCategoryName()); assertEquals(retrieved1.getCategoryName(), retrieved3.getCategoryName());}

© Steria

@Testpublic void shouldRetrieveSameInstanceForSameKey() throws Exception { Category inserted = new Category("A"); Serializable id = repository.insert(inserted); repository.flushChanges();

Category retrieved1 = repository.retrieve(Category.class, id); Category retrieved2 = repository.retrieve(Category.class, id); Category retrieved3 = repository.find(Category.class).iterator().next(); retrieved1.setCategoryName("Z"); assertEquals(retrieved1.getCategoryName(), retrieved2.getCategoryName()); assertEquals(retrieved1.getCategoryName(), retrieved3.getCategoryName());}

JUnit: Repository

When I modify one of the retrieved instances

Then the others should be updated, too

Given a database with one object

Given I retrieve this object several times

© Steria

JUnit: Repository

@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();

assertNull(retrieveInNewThread(Category.class, category.getId()));

repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}

© Steria

JUnit: Repository

@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();

assertNull(retrieveInNewThread(Category.class, category.getId()));

repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}

Given I insert a new object while in a transaction

When I retrieve the object from another thread

Then I should not be able to see it

When I commit the transactionWhen I retrieve the object from another threadThen I should be able to see it

© Steria

JUnit: Repository

@Testpublic void uncommittedInsertsShouldBeInvisibleForOtherThreads() { repository.beginTransaction(); Category category = new Category("A"); repository.insert(category); repository.flushChanges();

assertNull(retrieveInNewThread(Category.class, category.getId()));

repository.commit(); assertEquals(category, retrieveInNewThread(Category.class, category.getId()));}

Given I insert a new object while in a transaction

When I retrieve the object from another thread

Then I should not be able to see it

When I commit the transactionWhen I retrieve the object from another threadThen I should be able to see it

Uncommitted Inserts Should

Be Invisible For Other

Threads

© Steria

JUnit: Web tests

@Testpublic void listProductsPageShouldShowAll() throws Exception { Product product1 = new Product(uniqueName("product"), 12300); Product product2 = new Product(uniqueName("product"), 300); repository.insertAll(product1, product2); repository.flushChanges();

tester.beginAt("/products/"); tester.assertTextInElement("products", product1.getProductName()); tester.assertTextInElement("products", product2.getProductName());}

© Steria

JUnit: Web tests

@Testpublic void listProductsPageShouldShowAll() throws Exception { Product product1 = new Product(uniqueName("product"), 12300); Product product2 = new Product(uniqueName("product"), 300); repository.insertAll(product1, product2); repository.flushChanges();

tester.beginAt("/products/"); tester.assertTextInElement("products", product1.getProductName()); tester.assertTextInElement("products", product2.getProductName());}

Then I should see both products

Given two products with unique names in the database

When I go to the /products/ web page

© Steria

JUnit : Negative tests

@Test

public void priceMustBeNumeric() throws Exception {

String oldName = uniqueName("product"); int oldPrice = 1234;

Product product = new Product(oldName, oldPrice); Serializable id = repository.insert(product); repository.flushChanges();

tester.beginAt("/products/" + id + "/edit.html"); tester.setTextField("productName", uniqueName("product")); tester.setTextField("price", "this is not a price!"); tester.submit();

tester.assertMatchInElement("errorExplaination", "[Pp]rice .*numeric");

Product stored = repository.retrieve(Product.class, id); assertEquals(oldPrice, stored.getPrice()); assertEquals(oldName, stored.getProductName());}

© Steria

JUnit: Negative tests

@Testpublic void priceMustBeNumeric() throws Exception { String oldName = uniqueName("product"); int oldPrice = 1234;

Product product = new Product(oldName, oldPrice); Serializable id = repository.insert(product); repository.flushChanges();

tester.beginAt("/products/" + id + "/edit.html"); tester.setTextField("productName", uniqueName("product")); tester.setTextField("price", "this is not a price!"); tester.submit();

tester.assertMatchInElement("errorExplaination", "[Pp]rice .*numeric");

Product stored = repository.retrieve(Product.class, id); assertEquals(oldPrice, stored.getPrice()); assertEquals(oldName, stored.getProductName());}

Then I should see an error message

Given a product in the database

When I go to the /products/<id>/edit web pageAnd I go change the price to a negative valueAnd I press submit

And the product should be unchanged in the database

© Steria| 11/04/23 Presentation titlep58 © Steria

Developer

Given ....When ....Then ...

Good design can be grown

© Steria

JUnit: Yahtzee histogram

@Test public void simpleCategoriesShouldBeSumOfMatchingDice() { ... }

@Test public void smallStrait() { ... }

@Test public void largeStrait() { ... }

@Test public void threeOfAKind() { ... }

© Steria

JUnit: Yahtzee histogram

@Test public void threeOfAKind() {

assertEquals(0, scoreFor("three_of_a_kind", 1, 1, 2, 2, 3));

assertEquals(3, scoreFor("three_of_a_kind", 1, 1, 1, 2, 3));

assertEquals(6, scoreFor("three_of_a_kind", 2, 2, 2, 3, 3));

assertEquals(9, scoreFor("three_of_a_kind", 1, 1, 3, 3, 3));

}

© Steria

JUnit: Yahtzee histogram

@Test public void fullHouse() {

assertEquals(0, scoreFor("full_house", 1, 1, 2, 2, 3));

assertEquals(0, scoreFor("full_house", 1, 1, 1, 2, 3));

assertEquals(25, scoreFor("full_house", 1, 1, 1, 2, 2));

assertEquals(25, scoreFor("full_house", 1, 1, 2, 2, 2));

assertEquals(25, scoreFor("full_house", 5, 5, 6, 6, 6));

}

© Steria

JUnit: Yahtzee histogram

@Test public void histogramShouldReturnFrequencyOfEachDie() {

int[] roll = { 1, 1, 2, 3, 4 };

int[] histogram = new ScoreCard().histogram(roll);

assertEquals(7, histogram.length);

assertEquals(-1, histogram[0]);

assertEquals(2, histogram[1]);

assertEquals(1, histogram[2]);

assertEquals(1, histogram[3]);

assertEquals(1, histogram[4]);

assertEquals(0, histogram[5]);

assertEquals(0, histogram[6]);

© Steria

JUnit: Yahtzee histogram

scoreCalculators.put("four_of_a_kind", new ScoreCalculator() {

@Override

public int calculate(int[] histogram) {

for (int i=0; i<histogram.length; i++) {

if (histogram[i] >= 4) return i*4;

}

return 0;

}

});

Testing: Costs and benefits

© Steria| 11/04/23 Presentation titlep66 © Steria

Ron Jeffries

Only test what you want to work

© Steria| 11/04/23 Presentation titlep67 © Steria

Johannes

Find the defect where it’s cheapest

© Steria

Unscientific graph!

Compiler Junit FitNesse System test Prod

Cost

Realism

© Steria

Unscientific graph!

Compiler Junit FitNesse System test Prod

Cost

Realism

Chance of finding defects

© Steria

Unscientific graph!

Compiler Junit FitNesse System test Prod

Cost

Realism

Cost of running and changing

tests

© Steria

Unscientific graph!

Compiler Junit FitNesse System test Prod

Cost

Realism

Cost of a defect

© Steria| 11/04/23 Presentation titlep72 © Steria

Johannes

Unit tests can reduce maintaince costs. (How?)

© Steria| 11/04/23 Presentation titlep73 © Steria

Johannes

When will unit tests increase maintainance cost?

© Steria| 11/04/23 Presentation titlep74 © Steria

Brian Marick

Automated tests are expensive to maintainIf the system wasn’t written to be tested

But maintainability is only one reason to test!

How to create a good solution?

Understand the problem