evolution of agile testing
TRANSCRIPT
Agile Days China
Evolution of Agile Testing (A tester's view)
Erik [email protected]: erik_petersen
Tech talk: http://bit.ly/80-20-rules
Familiar faces?
Methodologists meet in the snow
Mid February 2001, 17 people met in the Utah mountains over a weekend
They represented styles of lightweight software development
Only one, Brian Marick, had major interest in testing (a quiet person)
Another tester invitee, James Bach, could not attend (a loud person)
Non-light manifesto Individuals & interactions over processes &
tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following plan
Value in all, but items on left more valuable
Where's testing? Neither the agile manifesto or supporting
principles mention testing, focussing on ”sponsor, developer, user”
Principles do mention ”architecture, requirements & designs” but no testing or validation activity mentioned explicitly
Agile testing V1.0 Strong Extreme Programming slant
Developers do all tests, using TDD for design & automated unit tests, Junit mostly
No need for system tester role, ”all tests automated”
Developers to learn extra testing skills from testers then replace them
Kent Beck tells conference of testers they will all be out of jobs in a few years
Agile dev view
Agile tester view
A week of Ruby dev, test & talk Mid June 2003, a group of agile devs and
exploratory testers spend a week together talking and creating software at ”Agile Fusion” (Erik watches via webcam!)
Early version of Ruby seen as immature by devs (but great language for testing)
Some agile devs understand importance of exploratory testing, e.g Ward Cunningham
Agile Testing manifesto?
”It’s very important to understand that we in this Agile Fusion conference did not come to agreement on *any* specifically phrased idea about what agile testing is or isn’t. We tried to discuss it, but it quickly became apparent that
there are important philosophical & terminological differences among us.”
James Bach
“Important differences”?
Agile testing V2.0 Strong XP Programming slant weakening
Regression testing done by developers, using automated unit tests, now Xunit
Some appreciation for system tester role, e,g Thoughtworks, Mary Poppendieck
Some developers learn more about testing and importance of exploratory testing
Agile testing champions, e.g Elisabeth Hendrickson
Tools of the trade
Enter the test toolsmith Developer explicitly supporting less
technical testers, or more busy devs
Creating tools to support testability
Automating data creation, configuration table setup, security profiles, UI information fields only visible in test mode or hidden in html, etc
Focussed on manual & automated tests
Testability champion, Brett Pettichord
Test tools rule Rapid expansion of free functional tools
Framework for Integrated Test (FIT) 2002, java based implementation of customer examples as tests, lead dev Ward Cunningham
Selenium 2004, web application driver using javascript, lead dev Jason Huggins
Many other less popular tools
What does this mean?
Dependency Injection 1 What does this mean?
Imagine you run a window washers collective. Each worker has their own basic gear, buckets, sponges, scrapers, and can handle smaller jobs unassisted. Larger jobs need ropes, scaffolding or cherry pickers, so they need to be provided for the workers in those cases. (continued)
Dependency Injection 2 Dependency injection provides complicated
material to a worker function to help it do its job. This simplifies the design, and makes it more flexible allowing different types of materials to be provided in different cases. Another advantage is the ability to provide fake materials, when we are only interested in testing the function to check it still works properly.
From www.testingspot.net (Erik's link site)
Agile testing V3.0 Flavors Regression testing by developers, using
TDD & automated unit tests, with dependency injection & testability
Testsmiths create custom tools for their teams
Functional automation by developers or technical testers using tools like Selenium
Customer examples linked by dev tester to fixtures create simple tests, like FIT
ET in Agile Development Jim Shore & Chromatic write ”Art of Agile
Development”, 2007 with community review of each chapter
Lone voice of tester reviewer in dev cloud (Erik), pushing tester role & viewpoint
Book expanded to include ET Viewpoint: ET needed to find bugs missed by devs (to improve process to find next time!)
Included Exploratory testing chapter written by Elisabeth Hendrickson
Different stylistic views• 1) Does a agile team use a issue tracker?
Against: don't document just discuss & fixFor: bug history is valuable for bug clusters, empower user to raise issues or concerns
• 2) Is quality more important than velocity?Against: need to always deliver value over fixesFor: bugs may hide other bugs, more (potential) technical debt
Agile testing for testers & team Agile Testing stage at Agile 2009
conference
Exploratory testing sessions at Agile conference, Erik Petersen 2008, 2009, Jon Bach 2010
”Agile Testing” 2009, by Crispin & Gregory gets great reviews across all agile tribes
Strong agile tester community
Bughunts becoming popular in industry
Joining the cause
The 2nd wave of test tools Cucumber 2009, english-like fixture based
tool dominating functional test automation, lead developer Aslak Hellesoy
Selenium and Selenium Grid almost industry standard
Domain Specific Languages (e.g in Groovy), Robot framework, etc
AA functional test tools group discussing and leveraging tools, also GTAC (Google)
Key (system) testing skills Test techniques: e.g. boundary analysis,
equivalence partitioning, test intuition
Test modelling: domain, application, navigation, authorisation, etc.
Test case design and coverage
Defect management
Failure modelling (often innate skill)
Shadow architect & designer/ devils advocate
Key skill: Exploratory testing ET Framework: Three main parts
Investigation Wide across app, or deep to feature
Exploration Confirming app & other models, or trying
failure models
Reflection Maintain focus or change to next priority Investigate bug clusters
Attempt to make implicit skill visible
Agile testing today Still no agile testing manifesto, just many
flavors (like with agile)
New tools can be adopted quickly worldwide, e.g Cucumber
Many great developer testers but still specialist tester role needed
Risk based exploratory testing preferred technique for rapid testing
TDD ”checks” versus human ET ”tests”
Together (mostly)
Evolution of Agile Testing (A tester's view)
Erik [email protected]: erik_petersen
Tech talk: http://bit.ly/80-20-rules
Selected ReferencesTestability for Automation -Dave Catlett
presentation, 2004
http://www.nwcpp.org/Downloads/2004/ImprovingTestabilityNWCPP.ppt
Includes link to Brett Pettichord paper
AgileAlliance.org
Testingspot.net – Erik’s links site
Star Wars Posters – Cliff Chiang
Vader Portrait – Greg Peltz