testing in agile
DESCRIPTION
Testing in agileTRANSCRIPT
http://qtp.blogspot.com 1
Testing in Agile
http://qtp.blogspot.com 2
Table of contents1. Traditional style QA2. Agile Approaches3. An Agile Tester4. Traditional vs. Agile5. Role Testers Play6. Testing & Testers on Agile Projects7. Question: While a developer is coding a task, it is impossible for a
tester to test it (it doesn't exist yet). What then is the role of a tester at this point.
8. Question: Is the tester now involved in unit testing? Is this done parallel to black box testing?
9. Question: What does the tester do during a sprint where primarily infrastructural changes have been made, that may only be testable in unit testing?
10. Specific Technical Skills For Agile Tester’s Toolkit
http://qtp.blogspot.com 3
Traditional Style Quality Assurance
Process and tools are a key part of QA and Testing.
QA people seem to love documentation.
QA people want to see the written specification.
And where is testing without a plan?
So, is there a role for QA in Agile projects?
Maybe, but the role is different, the tasks are different.
http://qtp.blogspot.com 4
Agile approaches are changing the conversation about software development
Agile shifted our attention to small teams..Incrementally delivering quality software.
The old ideas about testing at the end of the coding phase no longer applicable.
We need to think about the role of Quality.
Assurance in Agile Projects.
Definitely NOT business-as-usual.
http://qtp.blogspot.com 5
An Agile Tester
A professional tester who embraces change, collaborates well with both technical and business people, and understands the concept of using tests to document requirements and drive development.Agile testers tend to have good technical skills, know how to collaborate with others to automate tests, and are also experienced exploratory testers.They’re willing to learn what customers do so that they can better understand the customers’ software requirements.
http://qtp.blogspot.com 6
Traditional vs. Agile Testing
Requirements Specifications Code Testing Release
Time
A B
C
A B
C
A B
D
C
A B
D
E
FAgile:Iterative & Incremental
Each story is expanded, coded & testedPossible release after each iteration
Phased Model e.g. Waterfall
http://qtp.blogspot.com 7
Traditional vs. Agile Testing
Traditional Agile
In the phased approach diagram (see previous slide), it is clear that testing happens at the end, right before release. The diagram is idealistic, because it gives the impression there is as much time for testing as there is for coding. In many projects, this is not the case. The testing gets “squished” because coding takes longer than expected, and because teams get into a code-and-fix cycle at the end.
Agile is iterative and incremental (see previous slide). This means that the testers test each increment of coding as soon as it is finished. An iteration might be as short as one week, or as long as a month. The team builds and tests a little bit of code, making sure it works correctly, and then moves on to next piece that needs to be built. Programmers never get ahead of the testers, because a story is not “done” until it has been tested.
Tests are usually created from a requirements document.
Rather than creating tests from a requirements document that was created by business analysts before anyone ever thought of writing a line of code, someone will need to write tests that illustrate the requirements for each story days or hours before coding begins.
http://qtp.blogspot.com 8
Role Testers Play
Unit Testing Acceptance Testing & System Testers are no longer
required?
The role of the tester with agile methods is an area that has received increasing attention. Initially with a focus on unit testing and ‘acceptance’ testing it appeared the system tester did not have a role in agile.
1
•Facilitate communication between the technical & business stakeholders
2
•Support early validation of requirements
3
•Help the business stakeholders define acceptance criteria
4
•Create automated acceptance tests
5
•Perform manual/exploratory tests on early-stage code
As Cem Kaner put it:‘The nature of the tester's role changes in iterative projects. We are no longer the high-profile victims, we are no longer the lonely advocates of quality, we are merely (!) competent service providers, collaborating with a group that wants to achieve high quality.’
Typical responsibilities for testers in agile can include (not limited to):
http://qtp.blogspot.com 9
Testing and Testers on Agile Projects
http://qtp.blogspot.com 10
It comes as no surprise to testers that working software is not the same as code – the tester clearly needs to be involved in not only assessing the product, but in deciding how the product is to be assessed. However, with automated unit tests in the hands of the coders, and confirmation-focused acceptance testing driven by the customer, testers should be aware that they will not be the sole – or even the primary – owner of deciding what works, and what doesn’t.
http://qtp.blogspot.com 11
Testers need to be able to interact directly with designers and coders to understand the technological imperatives and restrictions that affect the software and its unit tests.
will take the place of
some documentation
Conversations and shared understandings
Overall less documentation
required
http://qtp.blogspot.com 12
Testing will be driven by what is important to a user, rather than to fulfill a procedural requirement. It is better to have communication between tester, customer and designer than to maintain independence of the test team.
In practice, it is common to find large-scale automated unit testing on agile projects, to confirm that code works as expected. The product will be judged by the customer typically by manual, confirmatory tests, with close observation for undesirable behaviors. Testing by testers is often driven by the need to measure the system’s performance and to find surprises – tools are very much in evidence, but rigid test scripts and procedures do not give the requisite opportunity for discovery, diagnosis and exploitation.
http://qtp.blogspot.com 13
Testers are key collaborators with the customer, and on some agile projects will take on much of the role of the customer in designing and executing confirmation-driven acceptance tests. However, although testers traditionally make good customer advocates, working closely with a customer is preferable to becoming a proxy.
Test strategies which lean heavily on an unchanging set of requirements (for example: designing and coding tests to be bought together with code late in the project; prioritizing tests based on a fixed risk assessment; testing only what has been agreed in the contract;
reporting bugs only against fixed requirements) may be considered to be fatally flawed in the light of this value. Iterative collaboration is favored over a negotiated bill of work.
http://qtp.blogspot.com 14
While a developer is coding a task, it is impossible for a tester to test it (it doesn't exist yet). What then is the role of a tester at this point.
http://qtp.blogspot.com 15
Testers can prepare their test plans, test cases, and automated tests for the user stories before (or while) they are implemented. This helps the team discover any inconsistency or ambiguity in the user stories even before the developers write any code.
http://qtp.blogspot.com 16
The tester could be working with the customer to fine tune the stories in the sprint.
http://qtp.blogspot.com 17
They can often be involved in designing the tests that the coder will write to perform TDD.
http://qtp.blogspot.com 18
If the agile team is fairly advanced then the tester would normally be writing the ATDD (Acceptance Test Driven Development) tests. These could be in a tool such as Fitnesse or Robot Framework or they could be more advanced ruby tests or even some other programming language. Or in some cases, simple record and playback can often be beneficial for a small number of tests.
http://qtp.blogspot.com 19
They would obviously be writing planning some exploratory testing scenarios or ideas.
http://qtp.blogspot.com 20
The tricky thing to comprehend sometimes for the team is that the story does not have to be complete in order to drop it to the test stack for testing. For example the coders could drop a screen with half of the fields planned on it. The tester could test this half whilst the other half is being coded and hence feedback in with early test results. Testing doesn't have to take place on "finished" stories.
http://qtp.blogspot.com 21
Is the tester now involved in unit testing? Is this done parallel to black box testing?
http://qtp.blogspot.com 22
Testers Unit TestsDon’t do Testers only test code that
passes all of the automated unit, integration and acceptance tests, which are all written by the developers. This split may be different elsewhere, though; for example your testers could be writing automated acceptance tests.
The whole point of unit tests is to test the software is correct to avoid wasting time later down the line. It's all about
instant feedback
http://qtp.blogspot.com 23
What does the tester do during a sprint where primarily infrastructural changes have been made, that may only be testable in unit testing?
http://qtp.blogspot.com 24
Testers workload will vary between sprints, but regression tests still need to be run on these changes...
http://qtp.blogspot.com 25
You may also find that having the testers spend the first couple of days of each sprint testing the tasks from the previous sprint may help, however it's better to get them to nail down the things that the developers are going to be working on by writing their test plans.
http://qtp.blogspot.com 26
Ensure that project or sprint requirements are clear, measurable and testable. In an ideal world each requirement will have a fit criterion written down at this stage. Determine what information needs to be automatically logged to troubleshoot any defects.
http://qtp.blogspot.com 27
Prepare a project specific test strategy and determine which QA steps are going to be required and at which project stages: integration, stress, compatibility, usability, performance, beta testing etc. Determine acceptable defect thresholds and work out classification system for defect severity, specify guidelines for defect reporting.
http://qtp.blogspot.com 28
Specify, arrange and prepare test environment: test infrastructure and mock services as necessary; prepare test data; write scripts to quickly refresh test environment when necessary; establish processes for defect tracking, communication and resolution; prepare for recruitment or recruit users for beta, usability or acceptance testing. Write test scripts.
http://qtp.blogspot.com 29
Ideally the tester would be working with the team and the customer (who by the way, is part of the team!) to define the planned stories and build in some good, detailed acceptance criteria. This is invaluable and can save loads of time later down the line. The tester could also be learning new automation techniques, planning test environments, helping to document the outcome of the planning.
http://qtp.blogspot.com 30
Specific Technical Skills For Agile Tester’s Toolkit
http://qtp.blogspot.com 31
For automation to succeed ->
-> we need to apply good design practices
We strive to keep each automated test to
a single &
clear purpose
Learn how to evaluate and choose the right tools, so youcan help your team create maintainable automated regression tests. You can free up time for essential testing activities such as exploratory testing.
Automation Skills
http://qtp.blogspot.com 32
Communication skills and good domain understanding enable testers to help business experts give good examples of both desired and undesired system behavior.
We can turn these examples into tests that
help the programmers understand what code to write. This is called acceptance test-driven
development, and it is a major step toward building quality into the code and preventing defects.
Acceptance Test-driven Development
http://qtp.blogspot.com 33
Learning Styles
auditory learners
learn by listening
visual learners
Learn by see pictures
We all have blind spots that may prevent us from learning or triggers where we shut down and don’t hear the message anymore. Keep your emotional “hot buttons” in mind and focus on what you can learn from instructors, material, or teammates to enhance your abilities. Mentors with different backgrounds or from other industries besides testing and software development might work best with your learning style. Don’t limit yourself to coaches, mentors, and instructors who work specifically in software testing.
Learning Styles
http://qtp.blogspot.com 34
Many good software
Testing books
Plenty of free material on the Internet
Resources your peers can provide
Communities of practice are another good place to find mentors and learn together
Conferences are an obvious way to get a lot of new ideas in a very short period of time
Mailing lists and social networks
Testing communities, such as Weekend Testers
Learning Resources Examples
http://qtp.blogspot.com 35
References
• AGILE TESTING A PRACTICAL GUIDE FOR TESTERS AND AGILE TEAMS by Lisa Crispin & Janet Gregory
• http://sqa.stackexchange.com/questions/1824/the-role-of-a-software-tester-in-an-agile-environment
• http://searchsoftwarequality.techtarget.com/news/1243805/Role-of-testing-in-agile-projects
• http://www.kohl.ca/blog/archives/000116.html• http://www.mcbreen.ab.ca/talks/CAMUG.pdf• http://newsweaver.ie/qualtech/e_article000847213.cfm
?x=b11,0,w• http://stackoverflow.com/questions/1640911/role-of-tes
ters-in-agile• LEARNING FOR AGILE TESTERS, Part 2 by Lisa Crispin and
Janet Gregor, Better Software Magazine Sept/Oct 2011