agile acceptance testing with f sharp
DESCRIPTION
TRANSCRIPT
AGILE ACCEPTANCE TESTING WITH F#
Zach Bray2010
Outline
Test Driven Development (TDD) Implementing TDD
Behaviour Driven Development (BDD) Implementing BDD .NET BDD frameworks Case study: Extending a C++ project
Test Driven Development (TDD) cycle
Write tests
Verify tests fail
Write minimal code needed to
pass testsVerify tests pass
Clean up code
Unit Tests - Good vs. Bad
Unit Testing
“A good unit test needs both to illustrate and define the behavioral contract of the unit in question. Behavior is more than just individual methods…” – Kevlin Henney
Na’vi
Behaviour Driven Development
“Is an agile software development technique that encourages collaboration between
Developers QA Business Participants
in a software project.”
“It extends TDD by writing test cases in a natural language that non-
programmers can read.”
User Stories/Features
Small chunks of business value. Creation and priority driven by stake
holders.
Acceptance Tests/Scenarios
Write in the language of the business domain i.e., high-level of abstractionRefunded items should be returned to stock• given a customer buys a black jumper• and I have 3 black jumpers left in stock• when he returns the jumper for a refund• then I should have 4 black jumpers in stock
Prerequisites for executing scenarios
Define an interpretable language. Domain Specific Language (DSL) Ubiquitous language Natural language
Choose a BDD framework
The BDD Framework (Unit Test generation)
Takes feature specifications + language context assembly
Matches scenario lines to functions
Compiles scenarios into tests
Compiles features into test fixtures
Scenario description line matching example
Types are tagged as language contexts.
Each line in the file, e.g., “given a customer buys 2 black jumpers”
Matches a member function. Use double-ticked for great readability
and DRYness
State within scenarios
In the previous example state was changed inside the language context.
Contexts are created on a per scenario basis. State doesn’t pass through to the next
scenario
Context Layer: Benefits of F# Discriminated Unions & Active Patterns
Domain Modelling Object Expressions
Implement interfaces Terse syntax
Readability Units of measure
Correctness Double ticked method names
Clear & DRY Async Workflows & Agents
Concurrency
BDD Patterns: Templates
Keeping things DRY with table driven templates. Useful for driving many similar test cases.
Refunded items should be returned to stock
given a customer buys a <colour> jumperand I have <x> <colour> jumpers left in stockwhen he returns the jumper for a refundthen I should have <y> <colour> jumpers in stockexamples:
| x | y | colour || 1 | 2 | green || 3 | 4 | black || 5 | 6 | red |
BDD Patterns: Tables
Using tables makes data input much more readable.
Tables are passed into functions.given the following bids exist in the Cape/Panamax market| qty@price | credit || 1000@ 12.9 | good || 1000@ 12.7 | bad || 800 @ 12.5 | good |and the following offers exist in the Panamax market| qty@price | credit || 1000@ 12.9 | good |when I observe implied orders for the Cape marketthen I should see the following bids| qty@price | credit || 1000@ 0.0 | good |
BDD Patterns: Templated Tables
given the following bids exist in the Cape/Panamax market| qty@price | credit || 1000@ <price1> | <cred1> || 1000@ 12.7 | bad || 800 @ 12.5 | good |and the following offers exist in the Panamax market| qty@price | credit || 1000@ <price2> | <cred2> |when I observe implied orders for the Cape marketthen I should see the following bids| qty@price | credit || 1000@ <calPrice> | <calCred> |examples:| price1 | price2 | calPrice | cred1 | cred2 | calCred
|| 1.0 | 1.0 | 0.0 | good | good | good || 1.0 | 0.0 | -1.0 | good | good | good || 2.50 | 2.50 | 0.0 | bad | good | bad |
Representation of complex scenarios
Plain text is dull. Rich text is cool.
It makes the point of the scenario pop out. Tables don’t look as ugly!
Confluence BDD tool It downloads features from a Confluence
Wiki. Then transforms them into executable
features.
Confluence BDD
BDD Patterns: More
Lots of examples online. Search for gherkin. Look at http://cukes.info/
Benefits of using BDD
Shared understanding of goals and progress.
It is independent of the implementation. Use the same tests on many implementations.
Regression suite. Nightly regression tests.
Reduces QA lag at the start of the project. QA can start specifying behaviour straight
away.
Costs of BDD
Requires more interaction with stake holders.
Requires more development effort than not testing at all.
Costs of not using BDD
Cost of wrong requirements. Cost of poor design. Cost of changing implementation
and having to refactor all of the tests.
The role of Unit Tests
Primary coverage should come from behaviour specifications.
Unit Tests should be used pragmatically. Documentation Edge cases
BDD Frameworks
GENERAL
RSpec Cucumber JBehave JSpec
etc.
.NET SPECIFIC
NBehave Nspec Cuke4Nuke SpecFlow StorEvil
SpecFlow: Integrated environment
Case Study: Indirect Aggression Project
Support indirect aggression of implied prices.
Requires extension to C++ Library. Verification is biggest problem. Not suited to Unit Tests.
Behaviour can be tested at API. F# made it easy to model the API in few
lines. Language interpretation was reused.
Starting to build up a regression suite.
Example scenario
Example matched function
Further Reading