xp days - acceptance test driven development

Licensed Under Creative Commons by Naresh Jain

Acceptance Test Driven Development

Naresh [email protected]

1

http://creativecommons.org/licenses/by-nc-sa/3.0/


http://agilefaqs.com/nareshjain.html


mailto:[email protected]

mailto:[email protected]


Tutorial Schedule

Part discussion, part exercise

Starts light and fluffy

You’ll get your hands dirty by the end

Questions welcomed all the time

2






Welcome

3






Welcome

Continuum

3






Welcome

Continuum

Hopes and Concerns

3






Warmup Scenarios

4






Warmup ScenariosPick one scenario and in relation to your scenario, what are the specific observable results that will tell you that the activity has been successfully completed?

4







Going out for Movie (THX sound and Digital projection)

4








Going out for meal (one veg.)

4









Going shopping ($50)

4









Going shopping ($50)[10 Minutes]

4









Going shopping ($50)[10 Minutes]

Present back to the group your findings. [3 minutes per group]

4






Acceptance Criteria and Tests: Definition

5






Acceptance CriteriaAcceptance Criteria & Tests: Definition

Are a set of conditions that the Story must meet for it to be accepted as complete

Are typically provided by the customer or product owner.

Are not a replacement for conversation. Are one of the results of the conversation

Acceptance Criteria are NOT tests

6






Acceptance Criteria should contain:

ACTOR

VERB – DESCRIBING A BEHAVIOR

OBSERVABLE RESULT

To accommodate pre-conditions Acceptance Criteria can be expressed as

Given [Precondition]

When [Actor + Action]

Then [Observable Result]

Writing acceptance criteria

7






Acceptance TestsAcceptance Criteria & Tests: Definition

8







Acceptance Criteria

8







Acceptance Criteria

+

8







Acceptance Criteria

Examples (data + scenarios)+

8







Acceptance Criteria

Examples (data + scenarios)

Acceptance Tests

+

8






DemoRoman Numerals to Decimal Conversion Example

9






Acceptance Criteria and Tests:

A Critical Piece of Agile

10






Analysis

Design

Implementation

DFDERD

DDST

1 May 1 Nov1 Jul 1 Sep

Traditional ApproachAcceptance Criteria & Tests: A Critical Piece of Agile

11






Key Questions

Are we building the right product?

Are we building the product right?

Business Facing

Technology Facing

12






Brian Marick’s Test Categorization

Business Facing

Technology Facing

Supp

orts

Pro

gram

min

gC

ritique product

13






Brian Marick’s Test Categorization

Business Facing

Technology Facing

Acceptance Testing Low-fi prototypes

Unit Testing

Exploratory TestingUI and Usability Testing

Performance Testing

Supp

orts

Pro

gram

min

gC

ritique product

14






The Agile Approach

May 1 Jul 1 Sep 1 Nov 1

Analysis

Design

Implementation

Test

Acceptance Criteria & Tests: A Critical Piece of Agile

15






The Agile ApproachAcceptance Criteria & Tests: A Critical Piece of Agile

16






The Agile Approach

Acceptance Criteria


16






The Agile Approach

Acceptance Criteria

Iteration


16






The Agile Approach

Acceptance Criteria

IterationAutomated Acceptance

Tests


16






The Agile Approach

Acceptance Criteria


Tests

AutomatedAcceptance

Tests


16






The Agile Approach

Acceptance Criteria

Acceptance Criteria


Tests

AutomatedAcceptance

Tests


16






TDD Rhythm - Test, Code, Refactor

Test Driven Development

17







Add a Test


17







Add a Test

Run the Test


17







Add a Test

Run the TestPass


17







Add a Test

Run the Test

Fail

Pass


17







Add a Test

Run the Test

Make a little change

Fail

Pass


17







Add a Test

Run the Test


Run the Test

Fail

Pass


17







Add a Test

Run the Test


Run the Test

Fail

Fail

Pass


17







Add a Test

Run the Test


Run the Test

Fail

Pass

Fail

Pass


17







Add a Test

Run the Test


Run the Test

Refactor

Fail

Pass

Fail

Pass


17






Mike Cohn’s Testing Pyramid

18







GUITests

18







GUITests

Small in NumberTools: Selenium, Sahi, Watir, Abbot, Frankenstein

18







AcceptanceTests

GUITests


18







AcceptanceTests

GUITests


At least one per storyTools: Fit, FitNesse, RSpec, JBehave

18







Unit Tests

AcceptanceTests

GUITests



18







Unit Tests

AcceptanceTests

GUITests



At least one per class or moduleTools: xUnit, TestNG

18






Every story must have at least one Acceptance Test

A story is not DONE until it passes it’s Acceptance Tests

Criteria for DONEAcceptance Tests: A Critical Piece of Agile

19






Manual Acceptance TestsAcceptance Tests: A Critical Piece of Agile

20






Manual Acceptance Tests

Manual Acceptance Tests

Acceptance Tests: A Critical Piece of Agile

20






Why Acceptance Tests?

Criteria for Completion

Great Collaboration tool

Source of Feedback

Real data to measure progress

21






Data From Acceptance Tests

0

18

36

54

72

90

1 2 3 4 5 6 7 8 9 10

Total ATs Failing ATs Passing ATs

22






Acceptance Tests Are Automated

23






The Button

How often would you press it?

When would you press it?

Who would press it?

Testers, Developers, Managers, Customers, Spectators, etc.


24





Licensed Under Creative Commons by Naresh Jain25






Criteria for DONE

25






Criteria for DONE

+

25






Criteria for DONE

Automated+

25






Criteria for DONE

Automated

Executable Specification

+

25






Executable Specification

A new paradigm for testing

Puts quality first

Removes ambiguity from requirements


26






Who Writes Acceptance Tests?

27






The Customer

The Customer Role

Stake holder

Business Analyst

Quality Assurance

Product Owner

Developer


28






Tests Get Technical

The “Customer” may need technical help to write tests

Developers and QAs are technical

Pair test authoring


29






Business Rules Get Fuzzy

Sometimes developers need help understanding tests

Customers know business rules

Pair test implementation


30






Exercise #1

31






The Login Test

Write a test plan, in plain text, for the business rules of logging in.

Web application

User credentials are stored in relational database

Successful login redirects to “Welcome” page

Exercise #1

32






Writing Good Acceptance Tests

33






Login Test PossibilitiesWriting Good Acceptance Tests

1. Direct browser to URL for login page

34








1. Enter the username ‘wallace’

34








1. Enter the username ‘wallace’

Build a Testable Environment First

34






Login Test Possibilities

1. Add some users to the system


35








3. Enter a value into the username field


35








3. Enter a value into the username field

Be Specific


35






Tests are Examples

Use concrete examples

Specify concrete behavior

No ambiguity allowed


36







1. Insert into User table values (’wallace’, ‘ilikecheeze’)


37








2. Open a browser to the URL http://localhost/myapp


37





http://localhost/myapp







2. Open a browser to the URL http://localhost/myapp

Avoid Implementation Details


37










Good Acceptable Criteria and Tests

38







S - SPECIFIC - Explicitly defined and definite

38








M - MEASURABLE - Possible to observe and quantify

38









A - ACHIEVABLE - Capable of existing or taking place

38










R - RELEVANT - Having a connection with the story

38










R - RELEVANT - Having a connection with the story

T - TIME-BOUND – When will the outcome be observed

38






Avoid Implementation Details

System

Database

Tests

DatabaseDatabase

UI


39






Login Test: Possible Solution

Add user to system: (’wallace’, ‘ilikecheeze’)

Process login with username ‘wallace’ and password ‘blah’

Check login failed

Process login with username ‘wallace’ and password ‘ilikecheeze’

Check login succeeded


40






Tools

41






Commercial Tools

WinRunner

Silk

RFT

Tools

Are not suitable for Acceptance Testing in an Agile environment

TestPartner

QTP

Squish

EggPlant

TestComplete

WindowTester

42






Open Source Options

FIT

FitNesse

Selenium

Among the few tools that support Test Driven Development

Sahi

Watir

Abbot

Frankenstein

SharpRobo

RSpec/JBehave

Tools

43






FIT

Framework for Integrated Tests

Created by Ward Cunningham

Open Source

The most accepted solution for agile acceptance testing

Tools

44






FitNesse

Environment build around FIT

Makes everything easier

Created by Object Mentor, Inc.

Open Source

Tools

45






FIT FitNesse

- Tests written in HTML- Tests are executed on the command line- Tables are executed- Non-table markup is ignored- Tables map to Fixtures- Fixtures are code that is aware of the system- Supplies foundational Fixtures- Implementations ported to many languages

- Stand alone web server- Is a wiki- Tests written in wiki text- Tests are executed from within the wiki- Translates tests into HTML- Uses FIT to execute tests- Supports test suites- Supports variables in tests- Supports test refactoring- Written in Java- Supports FIT implementations in any language

46






Thinking in Tables

47






Only Tables ExecuteThinking in Tables

Ignored

Executed

48






Foundational Table Structure

Name of Fixture

Interaction with Application

Table structure depends on type of Fixture

Thinking in Tables

49






Column FixtureThinking in Tables

51






Row FixtureAnalogous to comparing against rows in a database table

Thinking in Tables

52






Action Fixture

Think GUI window Counter WindowCounter:

Counter: 6 Count

public class CountFixture extends Fixture { private int counter = 0; public void count() { counter++; } public int counter() { return counter; }}

Thinking in Tables

53






FitLibrary

Extension to FIT

Written by Rick Mugridge

Adds some handy Fixtures

54






FitLibrary Fixtures

ArrayFixture for ordered lists

SetFixture for unordered lists

SetUpFixture

Supports

Graphics

Tree structures

Nested Tables

FitLibrary

55






Text

DoFixture

• Broken tables

• Highly readable

• Flexibility

FitLibrary

56






Wiki

57






What is it?

A collaborative web site

Editable by any

Created by Ward Cunningham

Every project should have one

http://c2.com/wiki

http://en.wikipedia.com

Wiki

58





http://c2.com/wiki

http://c2.com/wiki




Creating Tests

Use Wiki syntax to create a page with test tables Label the page as a Test Page

Use a page name of the form Test… Turn on the Test property

Make sure your Fixtures are in the classpath Use !path widget

Mechanics !path values are concatenated Java command to start FitServer is executed Testable HTML is passed to FitServer FitServer runs the tests Results are passed back to FitNesse

Wiki

59






Creating Suites

Set the Suite property Create a page with the Suite property Created test pages inside this page When the suite is executed, all child test pages will

be included in the suite execution Use the !see widget

!see <name of test page> All “included” tests pages will be included in the

suite execution

There are 2 ways to make Suites

Run a Suite by clicking the Suite button

Wiki

60






Hands-on Session

Conference Proposal Submission Portal

Some sample Stories

Should be able to submit new proposalShould be able to list all submitted proposalSubmitting proposal with same title should display appropriate error messageShould be able to delete submitted proposal based on the title Should be able to delete submitted proposal based on the titleShould be able to search proposals by titleShould be able to search proposals by IDShould be able to find all proposal by an author's name

61






Break

http://www.fitnesse.org

62





http://www.objectmentor.com



Patterns

63






Organizing Tests

Allowing customers to add new tests without breaking the build

Patterns

64






Version Control

Keeping the acceptance test in version control with the code.

Patterns

65






Cross-Functional Pairing

Using FitNesse based acceptance tests for collaboration between cross-functional team members.

Patterns

66






ATDD

Acceptance Test Driven Development

Patterns

67






CSTT

Cleanup, Setup, Test, Teardown

Patterns

68






Independent Tests

Tests shouldn’t depend on each other.

Tests leave the system in the same state it started in.

Patterns

69






Dynamic Stubbing

Avoiding complications of external systems.

Patterns

70






Non-Production Setup/Teardown

Using non-production light weigh code for setup and teardown.

Helps test only what you want to test.

Patterns

71






Suite Levels

Creating different levels of suites depending on the depth/level of feedback desired.

Smoke, Current Iteration/Sprint, Regression

Patterns

72






DRY

Using !include to avoid repeating yourself.

Patterns

73






Make it Real

Write ATs as close as possible to the real environment.

Patterns

74






Fixture Evolution

Allow Fixture implementation to evolve over time.

Treat fixtures as first class citizens.

Patterns

75






At Least One Test/Story

Every story should have at least one acceptance test

Avoid long/multipurpose tests.

Patterns

76






Anti-Patterns

77






Developer ATs

Developers writing acceptance tests by themselves, for themselves.

Anti-Patterns

78






Unit Testing

Don’t write ATs at the unit testing level

Unit tests are implementation specific

ATs are NOT implementation specific

Anti-Patterns

79






QA Testing Tool

Hard to write tests up front.

Perhaps only on large projects.

Anti-Patterns

80






Silver Bullet

Trying to use FitNesse for all types of Acceptance Tests

UI testing

XML testing

Anti-Patterns

81






Test After

Writing tests after the code is already written.

Anti-Patterns

82






Hidden Test Data

Hiding test data in the fixtures.

Anti-Patterns

83






Implementation Dependent ATs

Making test pages (tables) dependent on implementation details and data structures.

Anti-Patterns

84






Logging in Your Fixtures

Putting log statements or print statements in the fixture code.

Fixtures are probably too complicated.

Anti-Patterns

85






The End

http://www.fitnesse.org

86







xp days - acceptance test driven development

Spiritual

creative commons

naresh jain3

hands dirty

end questions

tutorial schedule