agile testing

Agile Testing Guiding Principles & Enabling Practices

Raj Indugula raj.indugula@lithespeed.com

www.lithespeed.com  

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

Agile Approach

Cost   Time  

Scope  

Quality

Maximize value and quality within specified project constraints

Shared Context drives Quality

Individuals and interactions…

Working software…

Customer collaboration…

Responding to change…

Integrated Teams & Iterative Delivery Challenges

Ensure that the evolving software does not regress? Test in-cycle? Bridge the communication gap between Bus./Testing/Dev.? Adopt automation strategies? Deal with performance issues identified late? …

How  do  we…  

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

Move Quality Upstream

•  Testing not a trailing activity

•  QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product

Small paradigm shift with profound implications

Testing is Continuous, Not a Phase

http://testobsessed.com/

Continuous testing is the only way to ensure continuous progress  

Testing is Collaborative

•  Quality is everyone’s problem, not just of the testers

•  Testing is the responsibility of the whole

team

The whole team can only go as fast as the slowest part  

Tests represent Customer Expectations

•  Shared understanding of what it means

for a story to be done

Oops…that’s not what I intended!  

Quick Feedback

•  Faster feedback loops increase Agility – the ability to respond to change

•  Test automation provides rapid feedback on how the software is behaving

How long does the team have to wait for information about how the software is behaving?  

“Leave No Broken Windows”

•  Fix bugs as they are found

•  The sooner you find a defect, the cheaper it is to fix

Keep the code clean  

It isn't Done until it’s…

“Done Done”

Coded Tested Done

Test-Driven

Defining tests with the requirements guides development

http://testobsessed.com/

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

ATDD  

TDD  

Other  prac<ces  

Automated Testing is at the heart of Agility

Unit Tests/Component Tests

Cucumber, FitNesse, SpecFlow

xUnit, Mocks

Selenium

Developer-centric (Are we building the code right?)

Business-centric (Are we building the right code?)

Adaptation of Mike Cohn's test automation pyramid

Exploratory Testing

GUI Tests

Acceptance Tests

But, in reality…

Unit    Tests  

Acceptance    Tests  

 Automated  GUI  Tests  

Manual Scripted Testing

Automated Testing is NOT Selenium-type UI record & play tests

What are acceptance tests?

•  Tests that demonstrate business intent of

system from end user’s point of view

•  Black-box testing

What is Acceptance Test Driven Development (ATDD)?

A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson

1.  Elicit  details  from  the  business  stakeholders  about  their  expecta<ons  

2.  Dis<ll  acceptance  criteria  into  automatable  tests  expressed  in  a  natural  language  

3.  Wire  the  tests  to  SUT  with  “glue”  code  (as  part  of  implementa<on)  

The tactical act of getting a story ready is often performed as a two sprint look-a-head by an “amigos” (BA, QA, Dev) team

Select User Story 999 for Sprint n+2

Re-estimate it, sharpen story & acceptance criteria

Create testable example and other supporting material for 999 Develop

User Story 999

Sprint n Sprint n + 1 Sprint n + 2

The PO and 3+ Amigos look-a-head and select story 999

for inclusion for Sprint n + 2. They do cleanup on the story.

The 3+ Amigos further support the story and the PO

gets appropriate sign offs.

Story 999 makes it’s way into a sprint and it is built.

Sample Story Maturation Look-A-Head

Epics, Features, Stories

Epics

Features

Stories

Product backlog

Priority

Modern Agile Acceptance Model

Conditions of Satisfaction – Broad Terms

Acceptance Criteria – Further Refined

Examples – Actual scenarios or data

Executable Examples – Ready to automate

Testable Examples in Gherkin •  Executable Example, Making it Repeatable •  Examples that can be executed are the final step

Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage”

Can be used for both manual testing and automation

Automated Acceptance Tests

Tool (Cucumber, SpecFlow, FitNesse)

Specification expressed in

common language

“Glue” code that ties specification to

System Under Test

Business Analyst Develop usage scenarios

Developer

Create and maintain acceptance test “glue” code that ties test specification to system under test

Tester Create and execute tests to simulate usage scenarios; Automate regression testing

•  Should be an integral part of every iteration/sprint, not just a trailing activity

•  Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves

•  Success depends on cooperation & collaboration

Testing Collaboratively

Benefits of ATDD

•  Improved requirements elicitation

•  Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development

•  Reuse of Acceptance Scenarios for all phases of testing

•  Creates clear examples that everyone understands; discovers some problems before any development

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

ATDD  

TDD  

Other  prac<ces  

What are unit tests?

Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected Key Characteristics •  Quick to execute •  Easy to execute •  Isolated •  Leave the system under test unaltered

Test Doubles (Mocks)

Used to isolate unit tests form external dependencies

Why use them?

•  Ease of setup •  Fast executing •  To work with a subsystem that doesn’t exist •  Simulate various execution paths of external

system

What is Test Driven Development (TDD)?

Write a new test

Red (Failing Test)

Write Code

Green (Test Passes)

Benefits •  Better design •  Supports change •  Prevents gold-plating •  Prevent bugs •  Discover bugs  

ATDD/TDD Cycle

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

ATDD  

TDD  

Other  prac<ces  

Requirements gathering Application Development

Source Control

Continuous Integration (CI) Server

Continuously commit changes

and merge changes from others

Pull changes, Build, run tests

Deploy to higher environment

Poll for changes

Development tasks from requirements

test scenarios from business requirements for acceptance criteria

Acceptance Test Environment - Test harness - Tests (automated + manual)

Execute acceptance tests against the deployed application

QA Environment

Deployable artifacts

Build Automation & Continuous Integration

ON-­‐DECK   AT-­‐BAT   DONE  

What  is  the  problem  context?  

Concluding  thoughts.    Any  ques<ons?  

What  are  the  guiding  principles  of  Agile  Tes<ng  ?  

What  are  the  enabling  prac<ces?  

Some Barriers to Adoption

Developer : “I’m a developer, not a tester” Tester: “I’m a tester, not a developer” Management: “Why does Dev help with testing, when we have QA?”

•  Necessitates behavioral change •  Requires discipline •  Needs generalizing specialists •  Needs collaboration across roles

Agile Testing is supported by…

MINDSET   SKILLSET  

TOOLSET  

•  Everyone is responsible to ensure quality

•  Testing is not a phase

•  Testers are not sole gate-keepers of quality

•  Customer collaboration

•  Requirements elicitation

• Writing test requirements that assist developers •  Skills to do

different types of testing •  Automation •  Effective

communicator and team player

•  Development and build tools (source control, CI, code coverage) •  Requirements and examples (mock-ups, testable

examples) •  Multi-level test automation

Business  Req.  

User  Stories  

Unit  Tests  

Automated  Unit  &  Acceptance  Tests  

Con<nuous  Integra<on  

SoXware  Applica<on  

Fixtures  

Business-­‐facing  Tests  

Developer   Tester  PO/BA  

…the job of testing is to prevent defects, not just to find them

- Mary Poppendieck

Thanks!

40

Resources

41

Some Tools (open source)

•  Unit testing: JUnit, Mockito, EasyMock

•  Acceptance testing: FitNesse Cucumber

•  GUI testing: Selenium, Sahi

•  Javascript testing: Jasmine, Qunit

•  Testing the database: DBUnit, SQLUnit, DBFit

•  Performance testing: JUnitPerf, JMeter

•  Usability testing: Manual unfortunately

Some Tools (open source)

•  Unit testing: MSTest, NUnit, Moq

•  Acceptance testing: SpecFlow

•  GUI testing: Selenium

•  Javascript testing: Chutzpa

•  Testing the database: Tsqlt, SSDT SQL Test

•  Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest

•  Usability testing: Manual unfortunately

Agile Software Development

•  Extreme Programming Explained - Kent Beck •  TDD: A Practical Guide – Dave Astels •  Refactoring – Martin Fowler •  Agile Estimating and Planning – Mike Cohn •  Pragmatic Project Automation – Pragmatic Prog.

44

Acceptance Testing

•  Bridging the Communications Gap – Gojko Adzic •  Agile Testing – Lisa Crispin and Janet Gregory •  The Cucumber Book - Wynne M. and Hellesoy A. Gherkin

https://github.com/aslakhellesoy/cucumber/wiki/gherkin Cucumber

http://cukes.info/ https://github.com/aslakhellesoy/cucumber/wiki/

TDD/Refactoring

•  TDD: By Example - Kent Beck •  TDD: A Practical Guide – Dave Astels §  The Art of Agile Development – James Shore •  Refactoring – Martin Fowler •  Working Effectively with Legacy Code – Mike Feathers •  Refactoring to Patterns – Joshua Kerievsky •  xprogramming.com, refactoring.com, testdriven.com