agile testing

Agile Testing Guiding Principles & Enabling Practices

Raj Indugula [email protected]

www.lithespeed.com

ON-‐DECK AT-‐BAT DONE

What is the problem context?

Concluding thoughts. Any ques<ons?

What are the guiding principles of Agile Tes<ng ?

What are the enabling prac<ces?

Agile Approach

Cost Time

Scope

Quality

Maximize value and quality within specified project constraints

Shared Context drives Quality

Individuals and interactions…

Working software…

Customer collaboration…

Responding to change…

Integrated Teams & Iterative Delivery Challenges

Ensure that the evolving software does not regress? Test in-cycle? Bridge the communication gap between Bus./Testing/Dev.? Adopt automation strategies? Deal with performance issues identified late? …

How do we…

Move Quality Upstream

•  Testing not a trailing activity

•  QA leads the Sprint by providing guidance and feedback on the business needs of the emerging product

Small paradigm shift with profound implications

Testing is Continuous, Not a Phase

http://testobsessed.com/

Continuous testing is the only way to ensure continuous progress

Testing is Collaborative

•  Quality is everyone’s problem, not just of the testers

•  Testing is the responsibility of the whole

team

The whole team can only go as fast as the slowest part

Tests represent Customer Expectations

•  Shared understanding of what it means

for a story to be done

Oops…that’s not what I intended!

Quick Feedback

•  Faster feedback loops increase Agility – the ability to respond to change

•  Test automation provides rapid feedback on how the software is behaving

How long does the team have to wait for information about how the software is behaving?

“Leave No Broken Windows”

•  Fix bugs as they are found

•  The sooner you find a defect, the cheaper it is to fix

Keep the code clean

It isn't Done until it’s…

“Done Done”

Coded Tested Done

Test-Driven

Defining tests with the requirements guides development

http://testobsessed.com/






ATDD

TDD

Other prac<ces

Automated Testing is at the heart of Agility

Unit Tests/Component Tests

Cucumber, FitNesse, SpecFlow

xUnit, Mocks

Selenium

Developer-centric (Are we building the code right?)

Business-centric (Are we building the right code?)

Adaptation of Mike Cohn's test automation pyramid

Exploratory Testing

GUI Tests

Acceptance Tests

But, in reality…

Unit Tests

Acceptance Tests

Automated GUI Tests

Manual Scripted Testing

Automated Testing is NOT Selenium-type UI record & play tests

What are acceptance tests?

•  Tests that demonstrate business intent of

system from end user’s point of view

•  Black-box testing

What is Acceptance Test Driven Development (ATDD)?

A practice in which the whole team collaboratively discusses acceptance criteria, with examples, and then distills them into a set of concrete acceptance tests before development begins. - Elisabeth Hendrickson

1. Elicit details from the business stakeholders about their expecta<ons

2. Dis<ll acceptance criteria into automatable tests expressed in a natural language

3. Wire the tests to SUT with “glue” code (as part of implementa<on)

The tactical act of getting a story ready is often performed as a two sprint look-a-head by an “amigos” (BA, QA, Dev) team

Select User Story 999 for Sprint n+2

Re-estimate it, sharpen story & acceptance criteria

Create testable example and other supporting material for 999 Develop

User Story 999

Sprint n Sprint n + 1 Sprint n + 2

The PO and 3+ Amigos look-a-head and select story 999

for inclusion for Sprint n + 2. They do cleanup on the story.

The 3+ Amigos further support the story and the PO

gets appropriate sign offs.

Story 999 makes it’s way into a sprint and it is built.

Sample Story Maturation Look-A-Head

Epics, Features, Stories

Epics

Features

Stories

Product backlog

Priority

Modern Agile Acceptance Model

Conditions of Satisfaction – Broad Terms

Acceptance Criteria – Further Refined

Examples – Actual scenarios or data

Executable Examples – Ready to automate

Testable Examples in Gherkin •  Executable Example, Making it Repeatable •  Examples that can be executed are the final step

Given the “Unregistered User” user has navigated to the “register” page When entering “newuser” in the “Username” field And entering “abc123” in the “Password” field And entering “abc123” in the “Confirm Password” field And pressing the “Register” button Then the text “Thank you for Registering” should appear on page And the URL should end with “use/accountPage”

Can be used for both manual testing and automation

Automated Acceptance Tests

Tool (Cucumber, SpecFlow, FitNesse)

Specification expressed in

common language

“Glue” code that ties specification to

System Under Test

Business Analyst Develop usage scenarios

Developer

Create and maintain acceptance test “glue” code that ties test specification to system under test

Tester Create and execute tests to simulate usage scenarios; Automate regression testing

•  Should be an integral part of every iteration/sprint, not just a trailing activity

•  Vital not only to prove completeness of a feature in repeatable fashion, but also to prove that software does not regress as it evolves

•  Success depends on cooperation & collaboration

Testing Collaboratively

Benefits of ATDD

•  Improved requirements elicitation

•  Consensus between BA/QA/Dev on the story helps prevent bugs & gives clear target for development

•  Reuse of Acceptance Scenarios for all phases of testing

•  Creates clear examples that everyone understands; discovers some problems before any development






ATDD

TDD

Other prac<ces

What are unit tests?

Developer tests that determine whether the smallest piece of testable software in an application is behaving as expected Key Characteristics •  Quick to execute •  Easy to execute •  Isolated •  Leave the system under test unaltered

Test Doubles (Mocks)

Used to isolate unit tests form external dependencies

Why use them?

•  Ease of setup •  Fast executing •  To work with a subsystem that doesn’t exist •  Simulate various execution paths of external

system

What is Test Driven Development (TDD)?

Write a new test

Red (Failing Test)

Write Code

Green (Test Passes)

Benefits •  Better design •  Supports change •  Prevents gold-plating •  Prevent bugs •  Discover bugs

ATDD/TDD Cycle






ATDD

TDD

Other prac<ces

Requirements gathering Application Development

Source Control

Continuous Integration (CI) Server

Continuously commit changes

and merge changes from others

Pull changes, Build, run tests

Deploy to higher environment

Poll for changes

Development tasks from requirements

test scenarios from business requirements for acceptance criteria

Acceptance Test Environment - Test harness - Tests (automated + manual)

Execute acceptance tests against the deployed application

QA Environment

Deployable artifacts

Build Automation & Continuous Integration

Some Barriers to Adoption

Developer : “I’m a developer, not a tester” Tester: “I’m a tester, not a developer” Management: “Why does Dev help with testing, when we have QA?”

•  Necessitates behavioral change •  Requires discipline •  Needs generalizing specialists •  Needs collaboration across roles

Agile Testing is supported by…

MINDSET SKILLSET

TOOLSET

•  Everyone is responsible to ensure quality

•  Testing is not a phase

•  Testers are not sole gate-keepers of quality

•  Customer collaboration

•  Requirements elicitation

• Writing test requirements that assist developers •  Skills to do

different types of testing •  Automation •  Effective

communicator and team player

•  Development and build tools (source control, CI, code coverage) •  Requirements and examples (mock-ups, testable

examples) •  Multi-level test automation

Business Req.

User Stories

Unit Tests

Automated Unit & Acceptance Tests

Con<nuous Integra<on

SoXware Applica<on

Fixtures

Business-‐facing Tests

Developer Tester PO/BA

…the job of testing is to prevent defects, not just to find them

- Mary Poppendieck

Thanks!

40

Resources

41

Some Tools (open source)

•  Unit testing: JUnit, Mockito, EasyMock

•  Acceptance testing: FitNesse Cucumber

•  GUI testing: Selenium, Sahi

•  Javascript testing: Jasmine, Qunit

•  Testing the database: DBUnit, SQLUnit, DBFit

•  Performance testing: JUnitPerf, JMeter

•  Usability testing: Manual unfortunately

Some Tools (open source)

•  Unit testing: MSTest, NUnit, Moq

•  Acceptance testing: SpecFlow

•  GUI testing: Selenium

•  Javascript testing: Chutzpa

•  Testing the database: Tsqlt, SSDT SQL Test

•  Performance testing: JetBrains, VisualStudio 2013, SQLServerProfiler, SQLLoadTest

•  Usability testing: Manual unfortunately

Agile Software Development

•  Extreme Programming Explained - Kent Beck •  TDD: A Practical Guide – Dave Astels •  Refactoring – Martin Fowler •  Agile Estimating and Planning – Mike Cohn •  Pragmatic Project Automation – Pragmatic Prog.

44

Acceptance Testing

•  Bridging the Communications Gap – Gojko Adzic •  Agile Testing – Lisa Crispin and Janet Gregory •  The Cucumber Book - Wynne M. and Hellesoy A. Gherkin

https://github.com/aslakhellesoy/cucumber/wiki/gherkin Cucumber

http://cukes.info/ https://github.com/aslakhellesoy/cucumber/wiki/

TDD/Refactoring

•  TDD: By Example - Kent Beck •  TDD: A Practical Guide – Dave Astels §  The Art of Agile Development – James Shore •  Refactoring – Martin Fowler •  Working Effectively with Legacy Code – Mike Feathers •  Refactoring to Patterns – Joshua Kerievsky •  xprogramming.com, refactoring.com, testdriven.com