Best Practices for Implementing Continuous Web App Testing

Olga Petrova

Agenda

• Why to Test?

• How do we test Ext JS Framework?

• Best practices of Continuous Testing

• Sencha Test in DevOps process

• Demo

Why to Test?

Good Application Quality …

• … increases customer satisfaction

• … decreases time to market

• … increases longevity of the application

• … boosts team morale

• … boosts value of business

Decisions to make

• Set Business Quality Goals

• Time to Market

• Application Longevity

• Repercussions of No-Automation

Strategic

• Define ROI

• Timeline

• Test Design

• Choose a Tool

Tactical

How do we test Ext JS Framework?

Ext JS Framework Testing

Total number of tests executed across different framework

versions

420,000 (approximately)

Types of tests executed Unit, Integration, End-to-End

Total number of virtual machines (VM) used for test execution

in parallel

50

Execution infrastructure Sauce Labs

Test Execution Time 20 Minutes – 1 Hour*

Tests Executed Every PR, Every Continuous Build, Nightly

CI tool TeamCity

Test Framework Used Jasmine, Selenium with Test Runner built in-house

(grandfather of stc)

Developer submits PR

Bot monitors for

source code

changes

Kicks offtests by

merging PR against branch

ApprovePR if no failure

Repeat actions for PRs from

all developers

Run continuous build once PRs are merged with the master

Sencha Process

Best Practices

Test Execution Tips

• Run concurrent tests to speeds up test execution

• Split up tests into smaller chunks into multiple VMs

• Aggregate results at the end of execution

Types of Tests

• Correct Price is Displayed

• Grid Shows the Right Value

• Form Displays the Correct Content

• Workflow Progression

Business Logic

• Rendered

• Visible

• Enabled\Disabled

• Visual Testing

UI Components

Finding Elements

• ID

• XPath

• CSS Selector

Common Traps

• xtype

• itemId

• title/fieldLabel*

• cls/iconCls*

Ext JS

Continuous Testing

Sencha Test in DevOps

• Run tests directly from Jenkins, TeamCity or other CI

• Execute tests unattended after every PR

• Run tests on browser farms leveraging multiple machines (internal or cloud-based)

• Store results in centralized location

• Analyze results in a matrix format

• Publish defects directly to the JIRA project

Demo

Sencha Test

http://sencha.com/products/test/