case study: qtp to selenium migration
DESCRIPTION
The presentation is from Selenium Conf 2014, where this case study was recently presented. The case study talks about a migration we did at TARLABS™ for one of our clients. The existing suite of QTP was migrated to Selenium. A custom C# based framework was developed. The framework is inspired from RobotFramework in Python. The execution time in QTP with Single machine was 160 min and was reduced to 19 min using SeleniumTRANSCRIPT
Case Study - QTP/UFT to Selenium
Migration - 80% reduced execution
By Tarun Lalwani
© TARLABS™ Ltd, 2013 - 14 1
About the Speaker
Founder at TARLABS™
Author of 3 books on Test Automation using QTP/UFT
Sold over 25000 copies world wide till date
Won various award by Infosys, HP, ATI
Worked with Infosys for 8 years as knowledge expert in areas of Telecom, Automation, Testing
Prime blogger at KnowledgeInbox.com
Helping SauceLabs Support on Directly.com
18000+ post on various forums
20K Regular followers
Education background – B.Engg from NSIT (Affiliate of Delhi College of Engineering)
© TARLABS™ Ltd, 2013 - 14 2
About TARLABS™
Founded in June 2013
Prime area of focus – Automation, API, Scripting, Product
building
We provide automated testing solution using QTP &
Selenium
Custom automation for different type of tasks, areas of
works
We have executed large projects in areas of Data mining,
scraping, Automated bots
© TARLABS™ Ltd, 2013 - 14 3
Agenda
About the Application Under Test
Existing Testing Suite
Existing Challenges
Why Migrate?
QTP v/s Selenium
Result
© TARLABS™ Ltd, 2013 - 14 4
AUT
Web app for hospitals
Managing patients, doctors, medications, diagnosis
Back End: ASP.NET, MS SQL
Front End: HTML, jQuery
75% AJAX autocomplete fields
© TARLABS™ Ltd, 2013 - 14 5
Existing Testing Suite
Tool: QTP
Test Cases: 30
Machine(s): 1
Run time: ~3 hours
Reporting: Native QTP
Execution Frequency: twice a day
Execution Tool: HP QC 11.00
Browsers: IE (primarily) & Firefox
Inconsistency Failure rate: 10%
© TARLABS™ Ltd, 2013 - 14 6
Existing Challenges
2 long test with 15-25 min execution time
2 long test - 40% failure chance because of inconsistent
app/AJAX behavior
Most failures re-attempted without re-testing
Extra debugging efforts as QTP native reports don't show
error/exception stack trace
© TARLABS™ Ltd, 2013 - 14 7
Idea of Migration
Created a single test proof of concept using Selenium and
C#
Selenium test with no framework and hard coded values
executed in 2.75 min against QTP's 4.5 min
© TARLABS™ Ltd, 2013 - 14 8
QTP v/s Selenium
QTP Selenium
Identifies browser at UI
level
Can connect to existing
browsers IE, Firefox,
Chrome
Identifies each object as
its own type – WebEdit,
WebTable, WebList
Identifies browser
launched using Selenium
Cannot connect to
existing browser
Identifies only
WebElement. Has only
support for Select
© TARLABS™ Ltd, 2013 - 14 9
Initial challenges
No functions for tables or specific object types
No method to check existence of object (in QTP,
Browser().Page().WebEdit().Exist(0))
No straight forward method to wait for object wait (in
QTP, Browser().Page().WebEdit().Exist(10))
No Object Repository management, DataTables
Lot of re-usable classes to be written
© TARLABS™ Ltd, 2013 - 14 10
Framework in C#
Inspired by Python RobotFramework
Aimed at below
Reduce logging effort
Easy to use
Object Repository
HTML reporting
Less code
Parameterization
Debugging from existing failures
© TARLABS™ Ltd, 2013 - 14 11
Key features
Automated reporting
[HandlerDocumentation("Take the screen shot of the page {screenshotName}")] public static string TakeScreenshot(string screenshotName)
Verify and assert Call,Click Link,${HomePage.tabAddProvider} Assert,VerifyExists,${AddProvider.btnAddNew} ${rowLocation}=,RowWithText, ${TabBilling.tblBilling}, ${BillingCode} Verify,HasValue,${rowLocation}
Object Repository ${Login.txtUserName}=name=username ${Login.txtPassword}=name=password ${Login.btnLogin}=name=submit ${PatientList.txtLastName}=id=txtLast
© TARLABS™ Ltd, 2013 - 14 12
Key features
Built-in jQuery and Sizzle selector support
Sizzle=, $$=, sz=
jquery=, $=, jq=
Injects jQuery or Sizzle libraries if not already present
Built-in label selector support
label=
<label for="User">Click me</label> <input type="text"
id="User" name="Name" />
Connect to existing firefox while testing LaunchBrowser, firefoxexisting
© TARLABS™ Ltd, 2013 - 14 13
Key features
Support for AJAX sites
AJAXClick
Works only if AJAX is done using jQuery
Counts the expected AJAX calls completed after clicking the
button
Doesn't work for sites like Facebook which continuously poll
data using AJAX
HTML Reports
Demo
© TARLABS™ Ltd, 2013 - 14 14
Execution Suite
© TARLABS™ Ltd, 2013 - 14 15
Excel Based (browser + script run time optimized)
Addon tools
© TARLABS™ Ltd, 2013 - 14 16
Sharp developer add-on (Intellisense + Syntax coloring)
Comparison
© TARLABS™ Ltd, 2013 - 14 17
Selenium v/s QTP Run-time: 19 min /165 min (-88%)
Selenium v/s QTP Development: 4 months /2 months
(+100%)
Add on Custom Performance Test
© TARLABS™ Ltd, 2013 - 14 18
Thank you!
Q&A?
© TARLABS™ Ltd, 2013 - 14 19
http://www.tarlabs.com
http://www.knowledgeinbox.com