agile testing principles and practices - anil karade
DESCRIPTION
Traditional test processes are not adaptive to extensive changes in software. Agile process emphasizes on ability to adapt to changing business needs, customer collaboration, integrated teams and frequent delivery of business values. Agile is an umbrella term that describes a variety of methods including XP and Scrum. The talk will discuss pitfalls of the traditional testing process. Traditional testing process happens very late in the SDLC Where as Agile process focuses on test-first approach. The talk will explain benefits of going agile. Principles and practices of agile process will be discussed and agile methodologies Scrum and Extreme Programming will be discussed in detail. Purpose of Scrum, its effectiveness, timings and managing the scrum will be discussed. Some of the practices for XP like Pair Programming, Test Driven Development will be discussed. The Talk will also cover the QA role in agile world. The talk will cover the implementation issues while shifting from traditional to agile process. Talk will also include an interactive game for illustration of concepts.TRANSCRIPT
Agile Testing – Principles and Practices
Anil Karade
Talentica Software (I) Pvt. Ltd.
Agenda Overview of Traditional Processes Agile Methodology Scrum Agile Testing Principles Agile Testing Practices With Case Study Agile Management Tools Used Benefits
How Traditional Test Processes Work
Traditional Planning :
Design + DevAnalysis Testing
Analysis Design + Dev Testing
Release
Release
Actual :
Traditional Testing Practices
“Last Defender of Quality” Stance Detailed preparation and upfront planning Heavyweight documentation Strict entrance and exit criteria with signoffs Attempts at process enforcement
What is Agile ? Agile – A continuous stream of business values
Agile methods in contrast to traditional ones produce completely developed and tested features at frequent intervals of 2-4 weeks.
Iterative approaches mean we can trade features for time instead of sacrificing quality
S1 S2 S3 Sn
Release Release Release
Completed, potentially shippable FeaturesDesigned, Coded and Tested
Release
Waterfall Vs Agile
Feature Usage
Values Expressed in Agile Manifesto:
Individuals and interactions over processes and tools
Working software over comprehensive documentation.
Customer collaboration over contract negotiation Responding to change over following a plan.
Scrum Scrum in game of Rugby
Agile principles will not work without integrated team mindset
Scrum
Scrum Scrum - an Agile approach to software development Scrum projects make progress in a series of Sprints Sprint - Timeboxed Period (2 – 4 weeks) During Sprint, team does:
− Analysis, Design, Code and Test Product is potentially shippable after every Sprint
Agile Testing - Principles Shifting QA Focus From Being last line of defense to providing information,
feedback and visibility.
The adverse relationship between testers and developers must be replaced with a spirit of collaboration.
Principles Contd… Quality Responsibility –
• Traditional model – Responsibility of independent QA group
• Agile Model - Quality is everyone’s (team) responsibility”. QA execute test cases, developers do too.
Testing Phase-• Traditional model – Testing is a distinct phase that
happens in the end• Agile model - QA teams test continuously to measure the
quality and provide feedback.• Quality is not added later. It evolves gradually.
Principles Contd…
Shortening Feedback Loops
Agile Testing Practices with Case Study Scrum Implementation
• We follow 2 Weeks Sprint• Release sprint (1 week) before going to production• At the end of Release sprint final release to production is done
Agile Testing Practices with Case Study
Challenges Faced• Change in mindset• Fear to lose identity• Collaboration with programmers and customers• Not Understanding Agile Concepts – ‘Scrummerfall’
Scrummerfall
Testing Practices QA Role in Release Planning
In contrast to traditional processes testers are involved from day one of the project.
Sizing Stories – QA gives estimate for testing effort for the upcoming stories. In some cases testing a piece of functionality might take longer than coding it.
Prioritizing – Stories that have many unknown areas or have high business impact should be planned in an early iteration, so there is time to recover if story blows up.
Infrastructure – Plan for CI setup, test environment, test data set up
Testing Practices Contd...Clear Requirements = Half Done
Practice we followed -
Full Kit Teams -• Separate full kit team was formed that included BA, Dev, QA
experts and Customer. • This team worked on getting the requirements clarified from
everyone’s point of view. • This ensured that we are minimizing the risk of finding the
defects that will challenge the requirements in the end stages.
Testing Practices Contd...
QA Role at Iteration Kick -Off :• Learn stories in detailsExample Story
Story – 1As a customer I want to know how much my order will cost to Ship based on the shipping speed I select so that I can choose a different shipping speed if I want to.
Testing Practices Contd... QA Role at Iteration Kick -Off :
• Writing Task Cards –
• Review high level test cases with developers
Dev tasks for Story -1• Coding for Shipping API• UI Design• UI coding
Testing tasks for Story -1• Write high level test cases• Write detailed test cases• Acceptance testing for API• Detailed functional testing for API• Exploratory testing• Automation
Testing Practices Contd...
Coding and Testing :• QA works closely with programmers, customers and other team
members to produce stories in small testing-coding-reviewing-testing increments.
• Write detailed tests for a story as soon as coding begins. Follow conversational test case creation approach for writing test cases
Contd…
Testing Practices Contd... Conversational Test Case Creation –
• Create business facing test cases in conversation with BA. e.g. business flows overlooked by BA
• Create technology facing test cases in conversation with Developers. e.g. browser dependencies, performance, etc
• Creating test cases in isolation will not assure coverage.
Testing Practices Contd…
Identify Risks / Prioritize Test Cases • With Agile testing QA not only needs to test sooner, but they
need to uncover high risk areas faster. • Execute the test cases in order of the risk associated with it, so
that at the end of iteration we do not end up in finding critical bugs where we don't have sufficient time to fix.
Testing Practices Contd...
# Item Impact Probability Risk
1 Incorrect cost displayed 4 2 8
2 User can’t choose different shipping option
5 1 5
3 Estimated cost doesn’t match actual cost at checkout
3 4 12
4 Invalid postal code entered and not caught by validation
4 1 4
5 User can’t change shipping address
5 2 10
6 User changes shipping address, but cost doesn’t change accordingly
5 4 20
Risk Analysis for Story -1
Testing Practices Contd…
Test Early (Behind GUI) –• Always ask programmers how a small set of coding effort can be
tested earlier. One of the examples we followed is API testing.• With traditional approach QA used to test only in stable builds,
whereas with agile we are uncovering the risks earlier by API testing, database testing
Testing Practices Contd…
Behind GUI (API) Testing –
Testing for Shipping cost API
Assumption – only valid destinations are considered as input
Start with simple happy path testing-
Weight Destination Postal Code
Cost ($)
5 lbs 80104 7.25
Testing Practices Contd… Coding and Testing Progress Together –
Iterative back and forth in coding and testing.
Try Canadian postal code. Result – It returns exception
Revised happy path test –
Any Clarification - Programmer + QA + Customer = Power of Three
Weight Destination Postal code
Country Code Cost
5 lbs 80104 US 7.25
5 lbs T2J 2M7 CA 9.40
Testing Practices Contd…
• Tests that critique the product – Be aware of all nice to have, enhancement requests.
Don’t jeopardize other stories by spending too much of time adding things that doesn’t have a big ROI.
Pair Testing
Which bugs to Logs ?
Testing Practices Contd…
Exploratory Testing• Exploratory testing is done when we get a working model of a
functionality• Simultaneously develop and execute test cases while exploring
functionality
Testing Practices Contd... Lightweight Documentation
Instead of writing verbose, comprehensive test documentation focus on essence of test.
Use collaborative tools – wiki, Review board etc.. Centralize documents – e.g. centralize generic test cases Leverage documents for multiple purpose
Testing Practices Contd...
Automation Automation is the key to agile. Automation helps QA in focusing on other important
aspects of testing like functional, exploratory testing. Focus on automating business logic bypassing GUI when
UI is changing continuously. Have customizable smoke tests Use lightweight automation tools for automating unit tests
Tools used: JUnit, Selenium
Testing Practices Contd... Release Sprint Testing
• Following testing activities need to occur during release sprint before finally shipping the product.
• Final pass of end-to-end system integration test• Final performance/security test• Documentation touch ups• User Acceptance Test
• Time required is greatly reduced (from initial 3 weeks to 1 week, in our case ) as compared to traditional process and less likelihood of any surprises.
Testing Life Cycle Sprint 1 (2 Weeks)
Test Analysis, Acceptance Test,
Investigative Test, Automated Regression Test
Release Sprint (1 week)end-to-end system Test
Final Performance /Security TestUAT
Production
Sprint 2 (2 Weeks)Test Analysis,
Acceptance Test, Investigative Test,
Automated Regression Test
Investigative Test : - Exploratory Test - Scenario Test - System Test
Agile Tools
We used VersionOne and JIRA for scrum management.
Excel backlog sheets were also used. http://agilesoftwaredevelopment.com/scrum/simple-product-backlog
Agile Implementation Benefits
Increased customer satisfaction On time deliveries without compromising quality Iteration wise testing makes QA more focused on
specific functionality which results in effective testing.
Due to stable builds test cases execution was fast Reduction in number of defects observed in testing
Thank You