software testing with agile requirements practices

Copyright 2003-2005, Rally Software Development Corp

Software Testing with Agile Requirements Practices

Dr Syed Akhter HossainFall 2012


Agenda

•Context for Agile Testing•Technical Challenges•Organizational Challenges•Keys for Success


Popular Agile MethodsDynamic System Development Method (Dane Faulkner)

XP (Kent Beck)

Adaptive Software Development (Jim Highsmith)

Lean Software Development (Mary Poppendieck)

Crystal (Alistair Cockburn) Feature Driven Development (Jeff DeLuca)

Scrum (Ken Schwaber) Agile Rational Unified Process (RUP)


Excerpts from the Agile Manifesto• Our highest priority is to satisfy the customer through early and continuous

delivery of valuable software. • Welcome changing requirements, even late in development. Agile processes

harness change for the customer's competitive advantage. • Working software is the primary measure of progress. • Deliver working software frequently, from a couple of weeks to a couple of

months, with a preference to the shorter timescale. • Business people and developers must work together daily throughout the

project. • Build projects around motivated individuals. Give them the environment and

support they need, and trust them to get the job done.

DSAH@Fall 2012 4


A Generalized Agile Release Process

DSAH@Fall 2012 5

Release

Iteration 1 Iteration 2 Iteration 3 Iteration …

•Do Feature 1•Do Feature 2•Do Feature 3a

•Do Feature 3b•Do Feature 4a

•Do Feature 4b•Do Feature 5

•Do Feature 4c•Do Feature 6•Do Feature 7

Backlog

• Feature 8• Feature 9• Feature 10• ….

Backlog

• Feature 1• Feature 2• Feature 3•….• Feature 9


Agile Iteration Cadence

DSAH@Fall 2012 6

Dem

o &

Retro

Iteration N Iteration N+1Iteration N-1

Detailed

Iteration

P

lann

ing

& D

esign

Dev Feature Priority 1

Auto. TestsFeature 1





Auto. TestsFeature 4Dev Feature

Priority 5


Accep

t

Accep

t

Accep

tAuto. TestsFeature 2

Accep

t

Accep

t

Requirements Are Refined

Initial Elaboration Requirements

With Tests


What’s Different about Testing in Agile?

• Just-In Time Requirements Elaboration– No SRS-level waterfall documents to drive testing plan– Requirements and Test Cases developed in parallel or test

first strategy• More Frequent Iterations, More Frequent Releases

– Testing needs to happen Early and Often – Frequent to continuous regression testing– High need to automate nearly everything– Everyone needs to Test

• Two Levels of Testing– Iteration Vs. Release testing patterns

DSAH@Fall 2012 7


Technical Challenges•Requirements are changing fast. How does test keep up?

•Test early and often. How exactly do we move testing forward?

•Need to move off manual testing and more into automation. How does this happen?

•Different kinds of testing need to happen at different times. How do these get managed?


Requirements are Changing

DSAH@Fall 2012 9

Code& Deliver

GenerateTCs

UC/SR

UpdateTCs

RunTCs

Fail TCs

Code& Deliver

Pass& Accept

Code& Deliver

Updates

Updates


Requirements Changing is a Good Thing?

• Probably the hardest agile principle for the team to embrace.– Need to elaborate the feature ahead of time – There is minimal time to have the team review

before the start. – Sometimes you have to rewrite

• Bottom-line: everyone collaborates to make the feature as useful for the customer as possible.

DSAH@Fall 2012 10


Requirements to Test Cases• Use Case Scenario Tests are perfect Acceptance Tests• Use Case A

– Scenario 1 Test Case 1– Scenario 2 Test Case 2

• Declarative Requirements that further refine the Use Case may be better suited to going directly to automation– Have one Test Case be the container for all of the

automation results. – All automated tests have to pass before the Test Case

passes.

DSAH@Fall 2012 11


Need to Test early and often

• Need to test early in the Iteration – do not want mini-waterfalls

• Need to test on check-in – Don’t break the build

• Need to test nightly – Don’t wait for a Regression Iteration

DSAH@Fall 2012 12


Mike Cohn’s Testing Pyramid

DSAH@Fall 2012 13

GUI Acceptance

Tests

FitNesse

Unit Tests

•Small number•Automate many

•Find the right ones

•Largest numbers•Foster Test Driven Design

Start Stop ?Start Stop Look


Break the Manual Testing Paradigm

DSAH@Fall 2012 14

Manual GUI Acceptance

Tests

Unit Tests

Automated GUI Tests

•Easy to Create•Very familiar – what we always do•Typically tedious•How do we know coverage?

•Need Automation specialists•Automation good for performance•Seems like we always rewrite•Sometimes fragile

•What is Dev testing?•How do we know what these are?•How do we know when they fail?

Start Stop ?Start Stop Look


Manual Testing Conundrum

• “You can never have too many manual acceptance tests”– Manual tests are cute little bunnies, before you know it

you have hundreds or thousands in your regression suite– You inadvertently dig a hole you can never get out of– Whole team had to help run regression suite

• Defect count typically is high– Most defects were found as manual tests were elaborated– Regression tests typically didn’t find many defects– Commonly found defects – things we didn’t think of

DSAH@Fall 2012 15


Better, But Not Perfect Testing Architecture

DSAH@Fall 2012 16

Unit Tests

Manual GUI Acceptance

Tests

Automated GUI Tests& FitNesse

•Still too many here

•Add FitNesse

•Increase Coverage•Increase Capability

Start Stop Look


Testing Types and SchedulingAcceptance – GUI?

•Minimize Manual

•Generate off of Use Cases to get scenario tests.

•During the Iteration

Acceptance -Functional

•Combination of Unit tests, FitNesse

•Build Verification and Run Nightly

Load & Performance

•Profiling and Simulation Automation

•Do it periodically

•Don’t wait till the end of the Release cycle

Regression •Acceptance and Functional tests from previous Iterations

•Run Nightly

Exploratory •Manual Group Explore

•Roles and Personas

•Before Releasing


Keys to Overcome the Technical Challenges

• Continuous Builds

• Nightly Regression testing

• Make Unit Testing a priority

• From found defects – create automated tests that go into Regression

DSAH@Fall 2012 18


Organizational Challenges•Dev as Testers and Testers as Dev – how does that happen?

•Resistance to Change – how do we get the team to welcome and embrace changes and not feel threatened?

•Testers are an integral part of the team- do we need to re-organize to make this happen?


I’m a Developer, Not a Tester• Pretty typical to hear push back from developers that they

– Don’t have time to do all of this testing– Number of features delivered will go down– Don’t really want to do all this testing

• Testers can help – Provide guidance on how to break software, art of creative

destruction– Pair testing with developers works well

• Have developers help out with manual regression testing. – “Can’t I write a test for this instead of running it manually?”

DSAH@Fall 2012 20


I’m a Tester Not a Developer

• Pretty typical to hear from testers – That they don’t feel comfortable or knowledgeable

about coding– That they maybe won’t be needed anymore

• Developers can help– Developers can create the fixtures (code running the

test) needed to make FitNesse testing work– To make it easier to auto test the code at the GUI level

DSAH@Fall 2012 21


Resisting Change

• Resistance is common– It is easier to do what is familiar, than risk something

new– Time-challenges may keep you doing the old way– Fear of failing keeps you in the status quo

• Get the whole team involved in trying to change– Team needs to figure what works best– Don’t feel like you have to do everything all at once– Keep learning and adapting

DSAH@Fall 2012 22


Testers on the Team• Your organization may have testing as a separate

group – look for ways to integrate them into the team– Creating feature or component teams comprised of all

disciplines is one way

• Co-location is a great way to hear and share information

• Daily stand-ups with the whole team keeps the information current

DSAH@Fall 2012 23


Keys to Overcome the Organizational Challenges

• Have Dev help run manual Regression tests

• Pair Dev and Test on Unit and FitNesse Testing

• Co-location of all the team

• Daily Standups

• Do Retrospectives

DSAH@Fall 2012 24


Summary

• Agile Pulls Testing Forward– You need to change your tools and approaches to

move it forward – You might need to change the model/structure of

your team

• With Agile, you will create faster Release cycles, shorter Iterations, more satisfied customers, and team members that enjoy what they are doing

DSAH@Fall 2012 25

software testing with agile requirements practices

Documents

software testing

makethe feature

c feature

bdo feature

ado feature

iteration backlog feature

working software

dev testing