selenium workshop - scqaa-oc.com · selenium workshop removing the mystery ... – selenium...
TRANSCRIPT
Selenium WorkshopRemoving the Mystery from Open Source Test Automation
Presented By
Chris Petrov, PMP
Leverage Point Corporation
SCQAA – Orange County
September 17th, 2013
Leverage Point Corporation
417B W. Foothill Blvd. #164
Glendora, CA 91741
626-857-9446
www.levpoint.com
Leverage Point Corporation
• Founded in 2006
• Specialize In Software Quality Assurance
"We will provide the best people, both technically and in customer focus, to deliver
specialized high-value and indispensable IT services. Each of our employees will
maintain a singular focus on using all of their experience, skill and creativity to ensure
the quality, performance and reliability of our customer’s critical business applications.”
• Specialize In Software Quality Assurance
Consulting and Staffing
• Outsourced Testing
• Test Automation Framework Development
• Testing Project Support
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
The Rise of Open Source Tools
• Cost cutting in IT departments
• Rise of lean startups
• Improved maturity of open source software
• Backing by major companies
– Google, IBM
• Cloud solutions versus in-house
• Price of commercial software increasing
• Value propositions of commercial solutions do not materialize
• Key factors for consideration in selecting Selenium as your
test automation tool versus commercial tools
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Is Selenium Right for You? – Cost/Support
• Cost
– FREE!!!
– The cost of learning curve and development may be significant
• Support
– Mature user groups
• Google group• Google group
• Stack Exchange
• SeleniumHQ
– Multitude of solutions just a search away
– Helpful people
– Commercial support available at various price points
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Is Selenium Right for You? - Functionality
• Supports major browsers
– Firefox, Chrome, IE, Safari, mobile browsers on Android
– Support of new versions of the browsers faster than commercial
tools
• Supports major web frameworks
– Rails, J2EE, ASP.NET, Etc.– Rails, J2EE, ASP.NET, Etc.
• Web Only – does not support:
– Desktop Applications
– Java Applets
– Flash
– ActiveX
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Is Selenium Right for You? - Testing
• Application Architecture
– Web based or mostly web based?
– Simple pages with MVC or complex workflows?
• Testing Methodology
– Unit Tests – over a single web page
– Business Cycle Tests– Business Cycle Tests
– System Tests
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Is Selenium Right for You? - Resources
• Resource Availability
– Do you have trained resources?
– Can your QA write simple code?
– Language Proficiency – Java, Python, Ruby, C#, JavaScript?
• Organization Capability
– Can you get resources?– Can you get resources?
– Can you get enough time?
– Collaboration with developers?
– Is organization committed to automation success?
– Ongoing support with time and money?
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Selenium Overview - Architecture
• Selenium IDE
– Simple Record and Playback Scripts
– JavaScript Based
– Available only in Firefox
• WebDriver
– Java Library giving you command of the browser– Java Library giving you command of the browser
– Supports all major browsers
– Code the scripts
• Extensions
– Custom JavaScript
– Additional Libraries
– Selenium Grid
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Selenium Overview – Use Models
• Selenium IDE Scripts
• Selenium IDE Scripts converted to WebDriver code
• Coded Tests with WebDriver
– Extract reusable parts as separate functions
• In-House Framework Design
– Rudimentary
– Data Driven
– Keyword Driven
• Low or High Level Keywords
• Third Party Frameworks– Many choices available with various pricing and capabilities
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Selenium IDE
• Resource Requirements
– Minimal training, some JavaScript knowledge
• Ease of Implementation
– Easy for simple pages, almost impossible for complex ones
• Ease of Test Development
– Very Easy for simple scripts, very difficult for complex scripts– Very Easy for simple scripts, very difficult for complex scripts
• Ease of Test Maintenance
– Very Difficult
• Cost and ROI
– Low initial cost, High over time - Low ROI
• Test Robustness
– Very Low
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Selenium IDE
• Who should consider this Use Model?
– No trained Resources
– No time to develop complex test frameworks
– Testing simple applications
– Mature GUI
• Who should avoid this Use Model? • Who should avoid this Use Model?
– Testing Complex Applications
– Volatile User Interface
– Organizations focused on long-term Automation ROI
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Coded Tests
• Resource Requirements
– Selenium Training, Programming, Tool Development
• Ease of Implementation
– Moderate for simple pages, moderate for complex ones
• Ease of Test Development
– Moderate for simple scripts, Moderate for complex– Moderate for simple scripts, Moderate for complex
• Ease of Test Maintenance
– Difficult
• Cost and ROI
– High initial cost, High over time - Low ROI
• Test Robustness
– Very High based on automation engineers abilities
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Coded Tests
• Who should consider this Use Model?
– QA with junior programming backgrounds
– Lack of Test Automation Architect
– Mature GUI
– More complex test requirements
• Reusability• Reusability
• Error handling
• Who should avoid this Use Model?
– Organizations with automation framework expertise
– Volatile User Interface
– Organizations focused on long-term Automation ROI
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – In-House Framework
• Resource Requirements
– Good Test Automation Architect, Dedicated Test Automation Team
• Ease of Implementation
– Very Difficult
• Ease of Test Development
– Easy– Easy
• Ease of Test Maintenance
– Easy
• Cost and ROI
– Very High Initial Cost, Low Over Time – High ROI
• Test Robustness
– Very High or as needed
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – In-House Framework
• Who should consider this Use Model?
– Availability of good Test Automation Architect
– Executive level commitment to long term test automation success
– Complex Application Architecture
• Who should avoid this Use Model?
– Organizations without test automation framework expertise– Organizations without test automation framework expertise
– Organizations focused on short term Automation gains
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Third Party Framework
• Resource Requirements
– Test Design and Application Knowledge
• Ease of Implementation
– Easy but depends on the framework
• Ease of Test Development
– Easy– Easy
• Ease of Test Maintenance
– Easy
• Cost and ROI
– Variable initial cost, Low over time - Very High ROI
• Test Robustness
– Variable but generally Very High
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Use Models – Third Party Framework
• Who should consider this Use Model?
– Organizations in need of fast improvement of quality
– Organizations without established test automation department
– Management commitment to long term test automation success
– Volatile GUI
– Complex Application Architecture– Complex Application Architecture
• Who should avoid this Use Model?
– Organizations with robust established open source test automation
solution
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Suitability of Selenium in a Framework
• Opportunities in using Selenium
– Extensibility by third party libraries
– Fast Patches
– Multitude of Languages
– Unified Library Interface
• Comparison with commercial tools• Comparison with commercial tools
• Challenges in using Selenium
– Desktop Interaction
– Somewhat harder debugging
– Uneven Language Support – C#
Copyright 2011-2013 Leverage Point Corporation www.levpoint.com
Questions?
Leverage Point Corporation – Copyright 2013 www.levpoint.com