Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
ADVANCED AUTOMATED VISUAL TESTING FOR
MOBILE APPS
WHAT IS VISUAL TESTING?
A quality assurance activity aimed to verify that a Graphical User Interface appears correctly to users
WHY SHOULD IT BE AUTOMATED?
THE TEST MATRIX IS TOO BIG TO APPROXIMATE MANUALLY Web browsers Devices Operating systems Screen resolutions Responsive designs L10n
WHY SHOULD IT BE AUTOMATED?
NATIVE / HYBRID MOBILE APPS Harder to roll back changes
Can’t push daily
Updates take battery and data
Higher quality bar
WHY SHOULD IT BE AUTOMATED?
Many are already doing it (and sharing)…
PhantomCSS Fighting Layout Bugs
CSS Critc
Wraith
Needle
Grunt PhotoBox
dpdxt
WebdriverCSS
EyesHuxley
FBSnapshotTestCase
GeminiSelenium Visual Diff
VisualCeption
Specter
Snap And Compare
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
DRIVING AND CAPTURING
OTHER
PhantomCSS (CasperJS)
Fighting Layout Bugs (Java)CSS Critic (URLs)Wraith (URLs)
Needle (Python)Grunt PhotoBox (URLs)
WebdriverCSS (JS)
Eyes (All)
Huxley (RP)
FBSnapshotTestCase
(XCTest)Selenium Visual Diff (Java)
VisualCeption (PHP)
Specter (JS)
Wraith-Selenium (Ruby)
Snap And Compare (URLs)
Eyes (All)
Gemini (JS)
dpdxt (URLs)
Gemini (JS)
WebdriverCSS
(Appium – coming soon)
SCREENSHOTS
Real browsers? Full page? Frames? Regions? Viewport size? Page stabilization? Page preparation?
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
ANDMORE…
1 pixel offsets in element positioning
Dynamic content
Moving elements
Images of different size
ImageMagick A powerful command line tool for image processing. APIs are available for most programming languages. Fuzzing is used to eliminate slight color differences An error ratio is usually used to determine a match
$ compare –metric AE –fuzz 5% img1.png img2.png diff.png
2246
Resemble.js An image comparison tool implemented in Javascript
Used by PhantomCSS and other tools.
Good antialiasing support
An error ratio is usually used to determine a match
http://huddle.github.io/Resemble.js/
Applitools Eyes A specialized image processing stack designed to compare
computer generated UI images
Anti-aliasing
Partial and full pixel offsets
Images of different size
Dynamic content
Moving elements
Layout matching
Available as a cloud service
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
THE WORKFLOW
Drive the AUT and take screenshots
Compare screenshots with baseline images
Report differences
Update the baseline
Adam Carmi (@carmiadam)
Co-Founder and VP R&D at Applitools
THANK YOU