automating with open source testing tools corey mcgarrahan rsmart 01-july-08
TRANSCRIPT
Automating with Open Source Testing Tools
Corey McGarrahan
rSmart
01-July-08
Agenda
• Getting from Here to There• Test Procedures• Criteria for Tool Selection• Tools Discussed• And the winner is…• The Next Step
Where We Want to Be
• No Manual Test Scripts• No Testing Tool• Totally dependent on Sakai Scripts
Where We Were
• Acceptance tests for all tools• Regression Suite• Automated the Test Suites
Manual Test Procedures
• Writing Test procedures• Procedures vs. Scripts• Hire Testers• Automate Procedures
Test Script
Resources
1: Add Item
Test ID Test Objective Pass/Fail Notes
1.1 Add an empty folder
1.1.1 Make folder publicly available
1.1.2Restrict folder access to site participants
1.1.3Add more than one folder at one time
1.1.4Add a folder with spaces in the name
Test ProcedureStep ID Action Expected Result
1. Select the Announcements tool o The Announcements page is displayed
2. Select the Add link o The Add Announcement page is displayed with the following options:
oAnnouncement TitleoBodyoAccessoAvailabilityoAttachmentsoEmail Notification
3. Create an Announcement with the following information:
Title: This is a TestBody: Lorem ipsum …Access: No changeAvailability: No ChangeAttachments: NoneEmail Notification: No changeSelect the Add Announcement button
o The Announcements page is displayed showing the This is a Test announcement.
Criteria
• Functional Test tool• Open Source• Collaborative in nature• Easy to use, easy to learn• Ability to record and playback• Manual scripting capabilities• Command Line functionality• Maintenance of Scripts• User Community
Tools
• JMeter
• Push To Test– TestGen4Web
• Selenium– IDE, Remote Control, Core, Grid
Apache JMeter
Apache JMeter may be used to test performance both on static and dynamic resources. It can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. You can use it to make a graphical analysis of performance or to test your server/script/object behavior under heavy concurrent load.
Scorecard
• Ease of use - NO• Record and Playback - YES• Manual scripting – YES• Command Line – YES• Script Maintenance - NO• User Community - Established
Push To Test w/ TestGen4Web
PushToTest TestMaker is a functional, end-to-end testing platform that validates the service code from interface to database and back. PushToTest is the open-source platform for software developers, QA groups, and IT management to test, optimize, monitor, and govern information systems.
TestGen4Web is a Navigation recorder, and playback tool. It records the user action on the page, form fills, etc, and can save the recording to a file and replay it back for you on demand. Combined with translators, which can convert this recording to automated command line test tools, you can generate HTTPUnit, PHP-SimpleTest scripts.
Scorecard
• Ease of use - YES• Record and Playback - YES• Manual scripting – YES• Command Line – YES• Script Maintenance - NO• User Community - Growing
Selenium IDE
Selenium IDE is an integrated development environment for Selenium tests. It is implemented as a Firefox extension, and allows you to record, edit, and debug tests. Selenium IDE includes the entire Selenium Core, allowing you to easily and quickly record and play back tests in the actual environment that they will run.
Scorecard
• Ease of use – YES• Record and Playback - YES• Manual scripting – YES• Command Line – YES• Script Maintenance - YES• User Community - Growing rapidly
And the winner is…
• Selenium– How we use Selenium
• Sample Tests• Language Options• TestSuite.html
How We Use Selenium IDE
• Environment Setup• Site Creation and Data Seeding
• Acceptance Testing• Positive Path Functional Tests
• Regression Testing• Incorporating bug fixes and code
changes into the Acceptance Tests on an ongoing basis
TestSuite.html<html><head><title>Test Suite</title></head><body><table cellpadding="1" cellspacing="1" border="1">
<tbody><tr>
<td>Test suite for setting up environment and acceptance tests</td></tr><tr>
<td<a target="testFrame" href=“SS-01.htm" >Site Setup for 2.5</a></td></tr><tr>
<td<a target="testFrame" href=“ANNC-01.htm" >Create announcements</a></td></tr><tr>
<td<a target="testFrame" href=“CAL-01.htm" >Create calendar events</a></td> </tr><tr>
<td<a target="testFrame" href=“LGOUT-01.htm" >Log out</a></td> </tr>
</tbody></table></body></html>
Language Options
• HTML• Java• C#• Perl• PHP• Ruby
Support with Selenium
• Support Engineers Record steps to recreate bug and attach Selenium test to JIRA Issue
• QA uses Selenium test to verify bug fix
• Tests are added to Regression Suite
The Next Step
• Continuous Integration• Selenium Remote Control and Selenium
Grid