raising the quality bar in agile teams by means of “early” test automation' by christian...
DESCRIPTION
When testing in agile projects, testers are required to deliver the best service possible to the developers. To be able to deliver this highly needed service to developers, the team needs already “done” features to keep on working, avoid known defects to resurface (reappear) as well as avoid builds to crash. I will present successfully applied methods and techniques used in agile software development projects. The focus on this approach for test automation is to let it become this needed “early” service to both developers and manual testers. By following these methods and techniques, testers will be able to impact the quality (positively) by contributing to test automation and test improvement via better design of test cases for test automation. It is a fundamental matter for high quality products that developers are able to continue keeping the high quality alongside new functionality is added. For this to be a success, test automation needs to be “pushed back” to as close as possible to the developers. In order to keep up with the pace of the developers and to avoid the “Death by Maintenance” syndrome, a proper approach to “early” test automation is required. I will introduce examples of a proper approach as well as some handy test techniques for easing the scripting of tests. The presentation is about how to incorporate successful test automation into an agile project. It is about the processes used daily by developers as well as incorporating our tests to the build processes and hence improving the quality of our products. I will also emphasize the importance of cooperation with the developers and their tool smiths, by pragmatic approaches.TRANSCRIPT
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
© Christian Nørlyng and PrettyGoodTesting®
By Christian Nørlyng
Raising The Quality Bar In Agile Teams By means Of Early Test Automation
1
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Who am I?
Test Automation Lead at PrettyGoodTesting
ISTQB certified Test Analyst (adv. level)
9+ years of test experience from• Small & Large Teams• Agile & Waterfall Projects• Medical – Directory – Telecom – Defense – Maritime
2
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Agenda
• Mission and Bugs• Goals, Context and Features for automation• Structural Test Patterns• Thinking Automation• Modeling for Data and more
3
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Most Important Mission
"Increase management
confidence in the product."
4
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Automation and bugs
Test Automation is looking forDefects found during regression testing
5
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Setting realistic goals
• Rules of thumb are difficult to apply• Conduct a small pilot project
• Less than a month in duration• After creating “just enough” infrastructure,
track time to automate each feature• Compare time to manual testing
• Use these data for subsequent test projects• Continue to update data• Automation will become more efficient
after the first few projects
6
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Getting the Context
”Features”
”Trade off’s”
”Possibilities”
”Context”
Starting with the intersection!
7
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Picking the first features
”Wants”
”Seems Quick”
”Musts Haves”
”Planned/ Implemented Features”
Starting with the intersection!
8
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Good Automation Candidates
• Short or simple transactions• Many data combinations• Expected results are stable
or easy to generate at runtime• Tests that are executed regularly• Tasks that are difficult to do manually• Highest priority features
9
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Poor Automation CandidatesPoor Automation Candidates• Long or complex transactions
• One-offs• Unstable or difficult to predict results• Tests that works across multiple applications
10
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Analysing the ”User Action” Patterns Under Test
• Breaking down the test cases andsystem under test into keywords• Each keyword should represent
a ”unique” set of actions to the system
11
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Designing Good Test Cases
• Test cases should have a single objective• Test cases should result in one of two
dispositions: PASS or FAIL• Test cases should be independent• No test case should rely on
the successful completion of another test• Test cases must start and stop
at a known state
12
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
What is ”Structural Test Patterns”?
• A break down method both for supporting• KeyWords, ActionWords or Primitives
• Can be used for manualand automated testing
• Can relate manualand automated testing
• Can ease maintenance of manualand automated testware
13
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
The Principle in short
• By thinking from the bottom up the supportfor automation is enabled from the test design.
• By breaking down the users behaviour intopatterns, the test procedures be simplified.
14
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Think AutomationHow and what to automate
Analyzed GUI/MMI
ActionWords
StructuringTest Conditions
Domainknowlegde Test Design
Structure
Requirements
Modular mindset, duringanalyzis and Test design
Test Conditions suitable for automation
Traceable identification
15
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
KeyWord Classification
NavigationGo from A to B (Start to adjust Time)
ActionInput to system (Set Time)
VerificationThe object of the test (Vary Time set)
16
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Work Process Description
Requirements
Test Design
Tests using Keywords
AutomatedManual
Modular mindset during test design
Review
Review
17
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Products of Module Based Test Design
Traditional SequentialTestCases
A pseudo test case broken down into ActionWords,each KeyWord is unique to the SUT (Software Under Test)
KW 6
KW 5
KW 4
KW 3
KW 2
KW 1 KW = KeyWord
18
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
KW 7
KW 6
KW 3
KW 2.1
KW 1
KW 7
KW 5.1
KW 4
KW 3
KW 2.2
KW 1
KW 7
KW 5.2
KW 4
KW 3
KW 2.3
KW 11
2.1 2.2 2.3
3
6 4
5.1
7Saving above 50% of work in just this example.
5.2
KW = KeyWordAn Advantage
19
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Why does this work?
• During test analysis, try to break down eachTestCase into unique and identifiable useraction sequences
• A unique sequence, is not to exist elsewhere• (stick to this…)
• The sequences are to be reused and combinedin any order, representing complete user actions
• Everything implemented is the rightcontext of the automation project
20
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Summary of test design
• Cooperation between Test Designer, Manual TestTeam and Automation Test Team reduces effort, by:- Thinking Automation First- Reusing Test Documentation/KeyWords
• KeyWords (aka ActionWords, Modules etc.):- Promotes flexible & easy maintainable Test Suites
• Exploratory Testing and Model Based Testing...still to be carried out (not a stand-alone thing)!
21
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Where to go from here!
• Data Drive the TestUse:
Parameters for the KeywordsKeywords as Data
Use Test Techniques and TestGeneration Tools for modelling!
Ex. Decision Tables,Classification Trees or State Machine Graphing
22
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
5.2
6
2.1 2.32.3
7
5.25.1
6 4
3
2.2 2.32.1
11
2.1 2.2 2.3
3
6 4
5.1
7
5.2
Simplified Example
23
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Remove the abstraction• Models can even be
madein a spread sheet!
• MS Excel (e.g.) can oftenbe used as a data source
• Designing test can now bedone from outside thetest automation tool!!
KeyWordParameter Set
Test 1KW1 KW1_P1KW2.2 KW2_P1KW3 KW3_P1KW4 KW4_P1KW5.1 KW5_P1KW7 KW7_P1
Test 2KW1 KW1_P1KW2.2 KW2_P2KW3 KW3_P1KW4 KW4_P1KW5.1 KW5_P1KW7 KW7_P1
Test 3KW1 KW1_P1KW2.2 KW2_P3KW3 KW3_P1KW4 KW4_P1KW5.1 KW5_P1KW7 KW7_P1
24
EuroSTAR 2011 : T6 : Raising The Quality Bar In Agile Teams By Means Of Early Test Automation
Questions and CommentsPlease feel free to contact me!
Christian NørlyngTest Automation Lead
Mob: (+45) 3163 [email protected]://www.PrettyGoodTesting.com
© Christian Nørlyng and PrettyGoodTesting®
25