marriage of agile and test management

8/14/2019 Marriage of Agile and Test Management

1/40

The Marriage ofAg ile and Test

ManagementBrian Massey, Sr. Produc t Mana ger, IBM

March 31 April 2, 2009


2/40


3/40

3

Agenda

Ag ile Software Development Trad itiona l Software Testing

Everything is d ifferent but still the same

The fight is on

IBM Agile Projec t

Bring ing it a ll together


4/40


5/40

Ag ile Defined !

Agile software development refers to a group ofsoftwa re development methodolog ies that promotes

development iterations, open c ollabora tion, andproc ess adap tab ility throughout the life-c yc le of theproject.

Things a re done in sma ll inc rements, with minimalplanning , ra ther than p lan at length.

Project adapts to c hanges

Emphasis on stakeholder invo lvement.

5
http://en.wikipedia.org/wiki/Software_developmenthttp://en.wikipedia.org/wiki/Iterations#Project_managementhttp://en.wikipedia.org/wiki/Iterations#Project_managementhttp://en.wikipedia.org/wiki/Software_development


6/40

Agile Princ ip les

Working software is the princ ipa l measure of p rogress

Even la te c hanges in requirements a re welc omed

Close, da ily coopera tion between business peop le and developers

Fac e-to-fac e c onversa tion is the best form of communic a tion (Co-location)

Projec ts a re built a round motiva ted ind ividua ls, who should be trusted

Continuous a ttention to tec hnic a l exc ellenc e and good design

Simp lic ity

Self-organizing teams

Reg ula r adap tation to c hanging c irc umstances

Customer sa tisfac tion b y rap id , continuous delivery o fuseful software

Working software is delivered frequently (weeks ra therthan months)

6


7/40


8/40

Agile Planning8


9/40

9

Test First Development (TFD)

Automated acceptance testing drives yourdetailed requirements efforts

Automated developer testing drives the detaileddesign of the system

Tests form the majority of your detailedspecifications

Youll still need some high-level models toprovide overviews

This is the first time in IT history that developersare motivated to keep the specifications up-to-date

An example of single sourcing information

9


10/40

10Test Driven Development (TDD)

Instead of c ode d riving the testc ases; usec ases/ testc ases define the fea tures and c od e thata re d elivered for a milestone or itera tion

Deve lopment tea ms deliver a use c ase per fea ture ora use c ase tha t c overs multip le fea tures before anyc od ing is done

A g ood use c ase c onta ins the following: Overview Brief desc rip tion o f wha t w ill be delivered

Any known pre-cond itions

High leve l steps

Expec ted results

Onc e the use c ases a re c omp lete then cod ing b eg ins

Tea ms may choose to p rovide view lets of expec ted

behavior over use c ases

View lets p rovide a moc k up of the fea ture w ith adevelop er taking you through the expec ted behavior


11/40

11

Agenda

Ag ile Software Development

Trad itiona l Software Testing


The fight is on

IBM Agile Projec t



12/40


13/40

13

Agenda



Everything is d ifferent but still the same The fight is on

IBM Agile Projec t



14/40

Business Basic s Test Early, Test Often

Use Itera tive Approac h

Move Setup and Overhead into Earliest Itera tion

Minimize Doc umenta tion, Ma ximize Testing

Have defined ob jec tives to assess aga inst (sc ope)

14


15/40

Points to Pond er

What is your test p rojec t Vision?

What a re your deliverab les?

What a re your goa ls?

How do you know when those have been met?

Have defined ob jec tives to assess aga inst (sc ope)?

Are you effec tively utilizing your resourc es (peop le, lab , etc )?

15


16/40

Why do we test?

Find Bugs?

Valida te Spec s?

To b reak things?

To get d eveloper in troub le?

Ensure system is ready to use!

To c rea te release notes

16


17/40

Expense of Software Testing

Testing is Expensive. But

What is c ost o f support?

Wha t is c ost of a Crit Sit?

What is c ost o f business due to poor qua lity?

When say testing expensive what is tha t c ompared to?

17


18/40

18

Agenda




The fight is on

IBM Agile Projec t



19/40


20/40

The Extremes20


21/40

Approac hes to Quality

Linear, Sequential

QA planning occurs late

No test case review

Emphasis on bug fixing

21


22/40

Continuous Testing in an

Agile Environment

ReleaseDefinition

(RDD / PRD)

User Stories \Accept. Tests \

Tasks

Code

Feature

Author Feature

Test Cases

QA FeatureTest

Execution

System

Test

Dev Signoff

QA Signoff

QA Plan \Automation

Plan

Design

Handover

TestCa

seReview

Product

Release

CodeHandoff to

QA

Unit TestFeat.

WOs

Feat.

WOs

22


23/40

23

Agenda




The fight is on

IBM Agile Projec t



24/40

24IBM SVT Agile p rac tic es

Fully comp leted use c ases within an itera tion Ac tive user involvement/ c ontinuous stakeholder feedbac k Adaptable p lans and goa ls with self d irec ting teams Stab le c ode a t itera tion end Reflec tions/ Post mortems for c ontinuous proc ess improvement Automate Unit and Func tiona l Testing , System where appropria te Continuous builds and integration Continually verify quality Daily interac tions between teams and business peop le Customer demos a t eac h itera tion Develop itera tively with timeboxed functioning milestones Prioritize fea tures based on c ustomer feedbac k, elimina te waste Modeling a nd proto typ ing Test d riven d evelopment Doc ument temp la tes and itera tion assessments


25/40

25

Case Study - RPT Problem:

Large amount of dep end enc ies on underlying c od e

Globa l team with 2 loc a tions, sma ller than RAD team

Small c ode b ase b ut la rge sc a le test requirements Smaller 4 week itera tions tha t never truly func tioned

Solution: Constant Development and Test c ollabora tion

Daily SCRUM meetings Better use c ase deve lopment

Interna l c ustomer adop tion a s well as externa l deliverab les and stakeholders

Tools Used:

ReqPro fo r trac king integra tions and requirements CQ for de fec t managem ent

RFT and RPT fo r automation

RMT and CQTM for manua l testing

Reuse of a rtifac ts from RAD a nd the RAM environment as a system und er test

25


26/40

26More about testing itera tions

Establish func tion a nd system testing goals for iterations When d o you plan to run your performanc e benc hma rks? Sc a lab ility runs? Identify those

ma jor a rea s of foc us for ea c h iteration includ ing d oc umenta tion, insta ll, andcomsumability.

Target reg ression testing of va rious a reas ac ross iterations. Ensure tha t p reviously releasedfunc tion is working .

Release an assessment report upon c ompletion of any iteration or milestone test Did the tea ms meet the go a ls of the itera tion? Did we have any subjec tive feedbac k on

usab ility, consumability e tc . input from c ustome r group s? Are w e m aking our performa nce a nd sc a lab ility goa ls? Are a ll bloc king or high seve rity defec ts addressed ? Is there an a greed-to p lan for fixing the rema ining unresolved defec ts? How are w e ha ndling c ustomer feedba c k?

Test case d eve lopment bec omes iterative Think of the iterations as mini-relea ses and p lan func tion a nd system-level tests

accordingly. Enhanc e/ upd ate the testc ases ac ross itera tions to ma ke them more comp lex (and

c omplete ) as add itiona l fea tures are introd uced .

Higher risk items should be tested as soon as they are delivered If p erformanc e is a key item, it should be tested ea rly when there is time to fix prob lem s.

Performanc e tests tha t require long er to run will overlap multip le iterations. Defec ts foundmust be p rioritized with the other work (If defec ts do no t get fixed , then the va lue o f ea rlytesting is red uc ed .)

If a user inte rfac e is high risk, the goa l should b e to test it ea rly. Same p rinc iple asdevelop ing high risk items.

Some teams have run reliab ility test ea rly, but fo r a shorter durat ion (1st 2 days, then 5, 14

days a t the end)

For the last iteration, most development teams are only be fixing defec ts andreac ting to c ustomer change requests


27/40

27

Agenda




The fight is on

IBM Agile Projec t



28/40

28

The Importanc e of Test Management

Would you go on a roa d tripwithout a map?

How would you know where togo?

How would you know how long it

would take? How would you reroute a round

problems?

How would you know how to d oit better next time?


29/40


30/40

Agile Test Plans

Have heard this sta tement frommany.. We a re Ag ile, we don t d o test p lans

30

Wha t rea lly saying We d on t c rea te 90 page d oc uments ea rly on in projec t and hope we

meet la ter

Test Plan no a forma l deliverab le

Do not c rea te doc before sta rt testing

Not saying We have no process/strategy

Only do Spray and Pray testing

We have no deliverables

We have no goals


31/40

Agile Test Plans

Plan is ta rgeted for p roduc t / p rojec t

All effort needs op timized

Priorities based on c onc epts suc h asrisk

Plan is not full done before sta rtingtest c yc le

If test Plan is a deliverab le, then it is agenera ted doc ument/

Crea ted just in time (grows in deta ilas p roduc t evolves).

31


32/40

Agile Test Plans

Controls sc ope of Test Projec t

Conta in test Stra teg ies

Captures Reports and Metric s to bedelivered

Contains c overage ob jec tives

Requirements

PlatformsSc hedules

Risk

Most likely not c a lled a Test Plan

32

We shape our tools; therea fter our


33/40

33We shape our tools; therea fter, ourtools shape us. (McLuhan)

If you Test you ha ve a p lan May not a dmit it

Point to Information ra ther than repea t it Why best to use a tool or Wiki to c ap ture

Coverage Outline Produc t e lements, function, da ta , sc hedule, etc

Contains c overage ob jec tives Requirements

Platforms

Sc hedules

Risk


34/40

Best Prac tic es

Ensure you a re using a sampling of customers and notdesigning a p roduc t for one spec ific c ustomer. Design

fea tures tha t sa tisfy 80% of the c ustomer use c ases.

Provide c onstant and freq uent feed bac k and testing.Inc lude hard da ta and softer feed bac k about usab ilityissues and designs.

Do not bac k end loa d performance testing. Run ea rlyas pa rt of FVT and SVT. Run b enc hmarks on eac hrelease for comparison.

It s impera tive tha t deve lop ment c ommits to p rovideenough information for ea c h itera tion to ena b lesuc c essful test p repara tion and c ollabora tion. Testmust c ommit to help p rioritize defec ts and provide

build ident ific a tion.

34


35/40

Best Prac tic es Avoid using rec ord and p laybac k for GUI automation

for a 1.0 p roduc t relea se due to the amount o f user

interfa c e c ha ng es. Utilize head less tests or using SDKsor APIs to d rive automation. Test services andenterprise ha rdening for your p roduc t.

Use a good test ma nagement too l so you c an havemultip le versions of your testc ases and assoc ia te themwith a g iven itera tion or release.

Use c ollabora tion tools to ensure development a nd

test teams a re c onstantly in touc h and aware of thela test hot items.

Define release defining or high level requirements andtrac k them.

35


36/40

36Agile testing

Similarities to today Testc ases and test p lans a re living doc uments and a re enhanc ed

eac h itera tion as fea tures a re d elivered and c overage c hanges.

Limited use of forma l entranc e and exit c riteria and rely ongenera l inp uts required for milestone assessments (weekly builds,develop ment p ub lished themes and goa ls, c ustomer feedbac k

etc) Foc us on a ll test types as app lic ab le: integra tion, reliab ility long

runs, scalability, stress scenarios, functional capability regression,insta ll/ c onfig , system performanc e, end-to-end c ustomer

scenarios Provide input to fina l qua lity c ertific a tion o f the software

Provide softer experienc e report for eac h itera tion as well as

da ta from defec ts and exec ution


37/40

37Agile testing

Key Differences Test c overage sp read ac ross ite ra tions, with empha sis on

c ustomer sc ena rios evolving with the add ition of fea tures EARLYFEEDBACK

Less dead time a t the end of the c yc le a lthough you may stillha ve a fina l SVT pha se

More defec ts c an be addressed in the c urrent release SVT is better ab le to impac t new designs and fea tures and is

a lways working with the produc t and impac ting qua lity, evenwhen not doing an ac tual trac ked test pass


38/40


39/40

39

Page

39


40/40

Thank you!

For more information:

Brian MasseySenior Produc t Manager

IBM/ Rationa l Software678-248-4523

[email protected]/rational

marriage of agile and test management

Documents