Presented at PyconSG on 14th June 2013

● Coastal GeoScientist and Civil Engineer● 10+ years in Software Quality Assurance

and Functional Testing● HP Certified Professional in Quality Center● Lead Quality Assurance at Vistaprint● Contributor to Holmium Core● http://sajnikanth.com

About me

● Background to black-box testing● Manual vs Automated testing● Why python?● selenium and other uses● Lessons Learned

Contents

Black-box tester?

Source - http://innovation.lv/ino2/publications/leonardo_manual/en/www.innosupport.net/webhelp/wso/index.cfm@fuseactionlearnl_id3808pl_id3554.htm

● Prepare test plans and test cases● Test the application● Find and record defects● Re-test● Regression test

Black-box tester's task list

Qualities of a good black-box tester

How many test cases do you see?

What's the problem?

5 * 3 * 3 * 2 * 2 = 180Test Cases * Browsers * Countries * Languages * Environment

Time to test on a few platforms?Good luck sucker!!

● Repetitive manual regression = boring● Low test coverage● No common language● Ever growing Test Suites● Testing becomes a bottleneck● QA bites the bullet

What's the problem?

Source -http://en.wikipedia.org/wiki/Infinite_monkey_theorem

● "Fail fast; fail often" - Martin Fowler● Integration / Functional / Performance● Building block for continuous integration and

deployment● Higher Return on investment

Automated testing

QTP to the rescue

● Expensive● Proprietary● Windows-centric● "Ageing" community

● Familiar (HP Certified after all)● Popular● Supports Record and Play

I think it is time for QTP to die - http://paulhammant.com/blog/die-qtp-die.html/

python?

● Easy to setup● Readable code (easy for beginners)

○ forced indentations; better for the team○ consistency

● open source; active community / library

OK; Getting back to the problem5 * 3 * 3 * 2 * 2 = 180Test Cases * Browsers * Countries * Languages * Environment

Time to test on a few platforms?

● Record and Play (Firefox)● Multiple browsers● Mobile Support● Multiple ways to locate an element● Screenshots ● ...

Selenium Features

● Test APIs● Write scripts for simple UI tests● Create test data● Test Suites● Performance Testing● ...

Working with python

Lessons learned

Lessons learned

Web dev changes conventions - intrologin-name, intrologin-password etc.

1. Use Page ObjectsPageObjects by Adam Goucher -

http://www.pushtotest.com/create-robust-selenium-tests-with-pageobjects

Lessons learned

Lessons learned

2. Wait instead of SleepExplicit and Implicit Waits -

http://docs.seleniumhq.org/docs/04_webdriver_advanced.jsp

Lessons learned

3. pick

nose is nicer testing for python - https://nose.readthedocs.org/en/latest/

● Organize tests better● Run selected tests● Plugin Architecture● List all tests● Test Runner (less boilerplate code)

Lessons learned

● Run 30 tests on Firefox○ about 12 minutes

Lessons learned

● Run 30 tests on a Headless browser○ about 7 minutes

4. use headless

● step toward Continuous Integration○ jenkins / teamcity / Travis CI are phantomjs-ready

● better performance than Firefox / Chrome● works without GUI

○ no interruption○ ec2 micro instance○ vagrant

Lessons learned

● https://holmiumcore.readthedocs.org● plugin for nosetests

○ developed by Ali-Akber Saifee■ http://ali.indydevs.org/

● Page object definitions made easy● Other options:

○ Specify environment○ Select browsers○ Run tests remotely

Introducing Holmium

nosetests tests/vistamobi_tests.py --with-holmium--holmium-browser=firefox--holmium-environment=staging--holmium-useragent=iphone

View this presentation from here - http://sajnikanth.com

How many software testers does it take to change a lightbulb?

None; testers just report that the room is dark.

Questions?