software test automation squad february 2010 scott m. allman [email protected]
TRANSCRIPT
2http://xkcd.com/627
Classic
3
Outline
•This is an interactive session•What I am NOT covering today•Stages of software testing and tools for that stage•General observations on the tools•Your stories and questions about software test automation tools
4
Not Covered Today
•Commercial tools•Art of downloading and installing•Comparisons of different tools that play similar roles•People problems - technical problems
5
Universal and Easy Tools
•OpenOffice Freeware substitute for MS Office– http://www.openoffice.org/
•Yahoo Widgets– ScreenShot
•JEdit - Language sensitive editor– http://www.jedit.org/
•XAMPP– http://www.apachefriends.org/en/xampp-windows.html
6
7
8
9
Often Heard Reasons To Not Automate Software Testing
• I am not a programmer• Test automation tools do not apply to our
system• If I automate the testing I might lose my job• It takes too much time and not worth the
investment• Can you think of other reasons?
10
Stages of Software Testing
1.Design the experiments2.Create the data and inputs3.Set up the lab4.Execute tests5.Analyze results6.Report results7.Retrospect
11
Design the Experiments
•Few tools to help with experimental designs•Familiar: ECP and Boundary Value Analysis•FitNesse – a universal test runner–User inputs to tests–Worksheet inputs
12
FitNesse
•Installation– http://www.fitnesse.org/
•Key features– Wiki hierarchy of tests– History feature– Run Fitnesse
•Your stories?
13
Create the Data
•Generating data – XML•Generating loads•Mock objects for tough scenarios•Your Stories?
14
Set up the Lab
•Ensure tests run the same–Ant–Scripting languages
15
Execute the Tests
•Unit Testing - JUnit•Ant, again•Your stories ?
16
Analyze Results
•Visual capture of the screen and save the image•Chainsaw - Logs, custom logs, logs of all types
17
Chainsaw example
18
Chainsaw Log Analyzer
• Installation– http://logging.apache.org/chainsaw/index.html
• Replaces scripts – tail, grep• Originally designed for log4j but will work with
any well structured log• Run the Chainsaw
19
Report Results
•Installation– http://blog.testlink.org/
•Testlink– Reports are available anytime, anywhere– Archived results– Also has other test manager features
•Run TestLink
20
Retrospect
•How good are the tests ?•Coverage analyzers offer some data•Installation
– http://emma.sourceforge.net/•Emma Coverage Report
21
Summarize
•All tests need designs – Worksheets, TestLink•Data and inputs – simulators, mock object•Set up the lab – ant, TestLink•Execute the tests – Nunit, scripting languages•Analyze results – Chainsaw, screen capture•Report results – TestLink•Retrospect – coverage analyzers
22
Algorithm: Open Source Tools
Here is a simple approach:1.Find it2.Download and install it3.Learn it4.Adapt it5.Adopt it6.Try getting with FindBugs (for Java)
23
Your Stories
•Tools•Recommended tools?•Failures and disappointments ?•Success stories ?
24
Sources for the Tools– http://xkcd.com/627– http://www.openoffice.org/– http://www.jedit.org/– http://www.apachefriends.org/en/xampp-windows.html– http://www.fitnesse.org/– http://logging.apache.org/chainsaw/index.html– http://blog.testlink.org/– http://emma.sourceforge.net/
•My thanks to SQuAD for the opportunity to present today•[email protected]
25
Questions?