agile testing practice

Agile Testing

Principles and Practice

Test manager, Program and project

manager

11+ years’ delivery experience

6+ years’ agile experienceMary Jiang

姜融

About you?

Managers, testers, ScrumMasters…?

How long has your team used agile

principles/practices?

Which kind of testing your team are

working on?

Group Discussion (10 minutes)

What is your biggest concern regarding agile testing?

Or

What interests you most about agile testing?

Presentation Time

After discussion every Group needs to select and present

your top 3 items

Agile Context

(Image provided by Wikimedia, reproduced here

under a Creative Commons license.)

http://commons.wikimedia.org/wiki/File:Agile_Software_Development_methodology.jpg

Agile context –Manifesto

Individuals and interactionsover processes and tools

Responding to changeover following a plan

Customer collaborationover contract negotiation

Working software

over comprehensive documentation

Scrum Flow

Agile Testing

What is agile testing?

A software testing practice that follows the principles of agile software development

Involves all members of a cross-functional agile team, with special expertise contributed by testers

Testing from the customer perspective as early as possible

Includes everything beyond unit and component level testing

Does not just mean testing on agile project

Agile Testing vs. Traditional Testing

Traditional testing V-Model

The pain of traditional testing

Testers’ pain from traditional testing

Is it agile testing?

What does agile testing mean?

Challenge

Continuous Change

Less documentation

Keep everyone on the same page

Face to customer

More testing effort

Opportunity

Everyone owns quality

Earn business value earlier

Break the boundaries

Involve the customer

Quick feedback

Whole team approach

Scrum Team commitment to quality

Scrum Team responsible for testing

Product owner responsible for acceptance testing

Development team responsible for all level’s testing

Agile testing mind-set

Don’t be afraid

Enjoy learning new skills and taking on new challenges

Not limited in the areas you can test

No judgment

Results-oriented

Collaborative

Passionate

Ten principles for agile testers

Provide continuous feedback

Deliver value to the customer

Enable “face-to-face” communication

Have courage

Keep it simple

Practice continuous improvement

Respond to change

Self-organize

Focus on people

Enjoy

-Lisa Crispin and Janet Gregory

Change doesn’t come easy

Be patient

Let them feel pain

Build your credibility

Work on your own professional development

Beware of the quality police mentality

Typical activities

for agile testing

• PROJECT INITIATION AND

PRE-PLANNING

• SPRINT N-1 AND SPRINT N

• CLEAN UP AND DELIVERY

Testing activities in project initiation

and pre-planning

Definition of “Done”

Release planning

Test planning

Preparation for visibility

Team commitment

“

”

Nothing is done until it is tested

FAMOUS SAYING IN AGILE


Acceptance criteria

Agile tester through release planning

A big picture view

Sizing stories from the testing perspective

Testing considerations while prioritizing

What is in scope?

Sample big picture

Web Svc

SIEBEL SVCEIM LOAD CTI

UCM

REMEDY TOOLS

IVR

ESP

Portal

Siebel

UI

Alarm Etc

Web Interaction

Agent Ticket Application

Phone

Phone

W/S

Test

Perf

Load

Tech

UCM

Ubiquity

Portal

Web Interaction

GDC

US

Test planning

Type of testing

Infrastructure

Test environments

Test data

Test results

Lightweight test plans

Lightweight documents

Test matrix

Test spreadsheet

A white board

Online tools

Sample test plan

Preparation for visibility

Tracking test tasks and status

Communicating testing results/bugs

Release metrics

Test status

Traceability/coverage

Defect status

Consideration of Bugs

Is it a bug or is it a new feature?

Use a Defect Tracking System?

Which bugs need to be logged?

When to fix bugs?

Rules for dealing with bugs

Team commitment


Ensure enough involvement

Communication channels

Agreement for visibility

Collaboration in a distribution environment

Testing activities in Sprint N - 1 and

Sprint

Sprint planning

Driving Requirement

Driving development

Collaborate with developers

Facilitate communication

Completing testing tasks

Regression Tests

Sprint Metrics

Demo

Retrospective

Sprint N-1

Are the requirements for Sprint N ready or

not?

What will testers do during Sprint Planning?

Learn more details

Considering all viewpoints

Think about testability

Write task cards for testing

Ensure testing effort is considered in estimation

Samples of task cards for testing

Samples of testers sprint plan

Driving requirement

High level tests and examples

Flowchart diagram

Test/verification points

Test driven requirements

Q&A with customer/representative

High level tests review

Sample for high level flowchart

Places BidPurchases

Item

Creates

Account

Searches

listings for

item

Creates an

AuctionShips Item

Buyer

Seller

User

Samples of Verification Points

Q&A and review

Collaboration tools to track Q&A and review

comments

Online and offline review with

customer/business representative

Driving Development

Start Simple

Add complexity

Risk analysis

Coding and testing progress together

Focus on one story at a time

Collaborate with developers

Pair Testing

“Show Me”

Facilitate communication

Daily Stand-up

Bridge the gap between business and

development

Collaboration tools for distributed teams

Understand and sacrifice a little time

Completing Testing Tasks

Proactive - test as early as possible

Address the testing crunch

Anyone can do testing

Non-functional testing

Regression Tests

Automation will be preferred

Keep the build “Green”

Keep the build quick

Building a Regression Suite

Checking the “Big Picture”

Sprint metrics

Measuring progress Burn Down Chart

Is testing done?

Coverage Unit Test

Code

Test coverage

Defect Status

Samples for metrics – Test Status

Samples for metrics –

Traceability/Coverage

CategoryPriority

TC Covered Defects TC NON-Covered Defects Total

# % # %

4-Business Critical 25 60.98% 16 39.02% 41

3-High 34 58.62% 24 41.38% 58

2-Intermediate 28 62.22% 17 37.78% 45

1-Low 19 76.00% 6 24.00% 25

Total 106 62.72% 63 37.28% 169

Samples for metrics – Defect Status

0

50

100

150

200

250

300

350

400

450

500

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Week 8

Total Bugs

Total Fixed Bugs Ignore Reopen Situation(Forcast)

Incremeantal Won't Fixed Bugs

Total Fixed Bugs With Reopen Situation(Forcast)

Sprint Review (demo)

Conduct the demo

Listen to the customers

Ask questions if possible

Do not forget the “big picture”

Sprint Retrospective

Start, stop, continue

Identify test-related obstacles

Focus on a few action items for the next sprint

Clean Up and Delivery

Planning enough time for testing

Testing release candidate

Testing on a staging environment

Final non-functional testing

Integration with external applications

Product data conversion and load

Installation testing

Communication

Support customer testing

System Testing

Planning in advance

Test environment(QA/testing and staging)

Final testing including non-functional testing

Cooperation with third-party teams

ETL/data conversion

Communication constantly

Installation testing

Defect trend

What if it’s not ready?

Was the build broken in the last day?

Any show stopper bugs not detected until final

functional testing?

Any new non-functional issues found?

Any test data problems?

Integration issue?

User Acceptance Testing

High level plan and scope

Support and collaborate with customer

Get feedback from customer

Understand how customers view products

Deliverables

External Customer

Internal Customer

Team and Organization Assets

Transition from

traditional tester

to an agile one

Know yourself

Change your testing thinking

Values are more important than process and

practice

Existing testing skills are useful

Build you own credibility

Agile Testing Quadrants

Functional Tests

Examples

Story Tests

Prototype

Simulations

Exploratory Testing

Scenarios

Usability testing

User Acceptance Testing

Alpha/Beta

Unit Tests

Component Tests

Performance & Load testing

Security Testing

“ility” testing

Application under test

Business View

Technology View

Su

pp

ort

ing

th

e t

ea

m Critiq

ue

pro

du

ct

Business supporting test

Support Team

Right tools to elicit requirements

From big picture to details

Driving development with business – facing testing

Critique product

Demonstrate applications to stakeholders to get early feedback

Use scenarios and workflow to test from end to end

Business supporting testers’ role

As a bridge between business and development

Required skills and capability

Domain knowledge

Solid testing skills and technical skills as supplement

Critical and logic thinking

Soft skills

Personalities

Automation skills

Ashely and Mina's story

Technical Supporting Test

Support Team

Unit tests

Component Tests

Continuous build

Critique product

Performance/load testing

Security testing

“ility” testing

Technical supporting testers’ role

Driving development and non-functional needs

Required skills and capability

Solid testing and technical skills

Architecture level understanding as preferred

Developing and Programming skills

Requirement and results analysis skills

Alex and Colin’s Story