Optimizing Your Agile Testing Processes

September 23, 2008

www.uTest.com

1

Agile Manifesto

• Individuals and interactions over processes and tools• Working software over comprehensive documentation• Customer collaboration over contract negotiation• Responding to change over following a plan

Standard S/W Theory

3

Requirements Analysis

System Design

Coding

QA & Bug Fixes

Release!

The Reality…

4

Requirements Analysis

System Design

Coding

QA

Release!

New Requirements

Redesign

Coding

Boss’s Pet Feature

Redesign

Coding

QA is a defensive stance…

“Protect the user” mentality

5

Agile Development

Short Development Cycles(aka Iteration, Sprint, etc.)-Don’t over design: Only what you need for this sprint-Testing integral to process-Focus on software, not documentation-Design / Test / Code as a team-Business User is part of cycle

Iterative Releases-2-4 weeks apart-Unique value in each release

7

Is Agile New?

8

Move the deadline up!

Kill all documentation!

Keep coding till the day of release!

This is NOT Agile!

Agile is VERY orderly

9

Test Driven Development

Write Unit Tests

Run Tests

Write Some Code

Run Tests

Fail

Fail

Yes

Pass

More Features?

No

Pass

Check-in

Sample JUnit Test Case

11

@Test

public void testAddition() {

assertEquals( 12 , myMathClass.add(7,5) );

}

Method in JUnit that:•Checks if the two arguments are equal•Displays message if assertion fails

Method in JUnit that:•Checks if the two arguments are equal•Displays message if assertion fails

The functionality we are about to develop, which

needs to be tested

The functionality we are about to develop, which

needs to be tested

The JUnit Test Case that we are buildingThe JUnit Test Case that we are building

Sample Unit Test Code

12

@Test

public void testAddition() {

assertEquals( 12 , myMathClass.add(7,5) );assertEquals( 12 , myMathClass.add(5,7) );assertEquals( 3 , myMathClass.add(3,0) );

}

Sample Unit Test Code

13

@Test

public void testAddition() {

assertEquals( 12 , myMathClass.add(7,5) );assertEquals( 12 , myMathClass.add(5,7) );assertEquals( 3 , myMathClass.add(3,0) );assertEquals( 2 , myMathClass.add(-5,7) );assertEquals( -2 , myMathClass.add(5,-7) );

}

Sample Unit Test Code

14

@Test

public void testAddition() {

assertEquals( 12 , myMathClass.add(7,5) );assertEquals( 12 , myMathClass.add(5,7) );assertEquals( 3 , myMathClass.add(3,0) );assertEquals( 2 , myMathClass.add(-5,7) );assertEquals( -2 , myMathClass.add(5,-7) );assertEquals( 133332 , myMathClass.add(55555,77777) );

}

Sample Unit Test Code

15

@Test

public void testAddition() {

assertEquals( 12 , myMathClass.add(7,5) );assertEquals( 12 , myMathClass.add(5,7) );assertEquals( 3 , myMathClass.add(3,0) );assertEquals( 2 , myMathClass.add(-5,7) );assertEquals( -2 , myMathClass.add(5,-7) );assertEquals( 133332 , myMathClass.add(55555,77777) );assertEquals( 6 , myMathClass.add(1,2,3) );

}

16

Don’t worry about that Failure…

The Feature Is Not For

This Release

Don’t worry about that Failure…

It’s a Known Bug

Testing Phases

17

Compliance Matrix Complexity

• Testing Compliance Matrix keeps getting bigger• For web apps, who is the customer in the team?

18

1997 1999 2001 2003 2005 2007 2009

Medium Complexity: OS,

Hardware, Device Drivers

High Complexity:OS,

Browser, Browser Settings/Plug-ins,

Firewall / Anti-Virus / Filtering,Connection Speed,Language/Region

Low Complexity:Browser Version

Native-Code HTML Server-side Dynamic Web Apps

The Tester’s RoleA. Unit Tests = Specs

• QA and PM meet early• No more SRS TP Alchemy

B. Testing is part of development process• Team development calls for testing frame of

mind • In a programming team / pair, not everyone is

homogenous

C. Acceptance Testing by Customer• Coordinating (and participating in)

customer testing / community testing

Case Study:Testing in Short Iterations

2 week development cycle

Mon - FriDevelopmentDevelop features for current Sprint cycle

Fri 5PMRelease To Testing

Post the Sprint version and test plans to Testing arena

Fri 5PM - Mon 9AMTesting

Simultaneous worldwide testing by uTest Community

Mon 9AMComplete Bug Report

R&D Reviews Bugs Identified in Sprint version

Mon - FriBug Fixing

Stabilization and Bug Fixing.No new development

Fri 5PMRelease To Testing

Post the Release Candidate to Testing arena

Fri 5PM - Mon 9AMTesting

Simultaneous worldwide testing by uTest Community

Mon 9AMVersion Release

Showstopper fixes, release decision and

post to production site

Scoping each Sprint

22

Fri - Mon Testing

Mon - FriBug Fixing

Fri - Mon Testing

Mon 3PMBusiness Priority

MeetingSales, Mktg, Prod Mgmt choose se 6-8 candidate

features for upcoming cycle

Tues 11AMSprint Planning

MeetingProd Mgmt presents

candidates to R&D Mgmt

Fri 10 AMR&D Response to

Biz TeamR&D analysis is presented to

business team

Fri 3PMSprint Decision Closure

List of candidate features for upcoming Sprint cycle is finalized

Wed-ThursDev Speccing

R&D Team Leaders analyze candidates for code

complexity, cost of development and risk factors

Mon - FriDevelopment

23

Testing and the End-User

Thank you!

Q&A

24

For more info:http://www.utest.com/solutions_agile_testing.htm

Download the “Agile Testing with uTest” Case Study:http://www.utest.com/downloadcs.php