Download - Development and testing By: Stephen Frein
![Page 1: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/1.jpg)
Mobile DeviceTesting Strategies
Stephen FreinComcast
5/8/2014
![Page 2: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/2.jpg)
About Me• Director of QA for Comcast.com• Adjunct @ Drexel University• https://www.linkedin.com/in/stephenfrein• [email protected]• www.frein.com
![Page 3: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/3.jpg)
• Largest mass media and communicationscompany in the world (by $)
• Largest cable company / ISP in USA• Largest converged network in the world
(data / voice / video)• Comcast.com in April of 2014:
o 200 million + page viewso 25+ million unique visitors
![Page 4: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/4.jpg)
Stuff We'll Talk About• Importance of mobile• What should I test on?• Where do I keep devices?
![Page 5: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/5.jpg)
Mobile Is BIG
![Page 6: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/6.jpg)
![Page 7: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/7.jpg)
Mobile Matters (More Than Ever)• US retail mobile ecommerce sales increased
68% from 2012 to 2013 (1) *• US mobile web usage rose by a similar
amount (2)• More than half of Amazon holiday shoppers
used a mobile device in 2013 (3)
* Psst! The (1) is an endnote.
![Page 8: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/8.jpg)
But Desktop Still Rules Web• Most mobile Internet usage (80+%) is
driven by apps (4)• Mobile usage accounts for about 30% of
web traffic (5)
![Page 9: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/9.jpg)
Punchline
Mobile is growing in importance, but yourweb testing efforts should probablyemphasize desktop for the foreseeablefuture. (Apps are their own animal.)
Desktop > Mobile
![Page 10: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/10.jpg)
When Will That Change?
Not in the next year . . .
![Page 11: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/11.jpg)
Where Should I Focus?
7.1
7.06.1
Gingerbread
KitKat
Jellybean
![Page 12: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/12.jpg)
What My Team Does• Establish coverage thresholds (browser /
OS / device type must have x% traffic)• Tune test coverage based on usual factors:
o Historical defect discovery rateso Nature of changeso Risk of leakageo Schedule pressure
• Pray we got it right
![Page 13: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/13.jpg)
Balancing Stats
Your Stats(the focus)
Industry Stats(a counterweight)
Google Analytics, Adobe, Piwik, etc
StatCounter, Chitika, Android, etc
![Page 14: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/14.jpg)
Industry Stats Useful Linkshttp://gs.statcounter.com/#mobile+tablet-browser-US-monthly-201304-201404
http://www.netmarketshare.com/ (some $)
http://developer.android.com/about/dashboards/index.html
http://david-smith.org/iosversionstats/
![Page 15: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/15.jpg)
General Observations• Today, iOS and Android are the real show• New iOS versions have rapid adoption• Android versions more fragmented• Android world is heavily phone-skewed• “Default” browsers the focus• Mobile Chrome creeping up in usage• Don’t ignore Windows entirely, but it is a
lower priority• Other flavors may not be worth it
![Page 16: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/16.jpg)
Where to Test?
Physicals
Cloud
Emulators
![Page 17: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/17.jpg)
Help Me ChooseApproach Pros Cons
Physicals • Realism• Audio / Video
• Geo-lock• Management
Cloud • Geo-flex• Management• Automation
• Cost• Audio / Video• Speed
Emulator • Cheap• Fast
• Realism• Realism!!!
![Page 18: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/18.jpg)
We Use Cloud (Mostly)• International team• “Hide and seek” got old• Audio / video not a priority for us• Good automation support• Private cloud helps
o Automationo Test environment access
“Who has the &%*@! iPad?”
![Page 19: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/19.jpg)
Whose Cloud Offering?Start by looking at these two . . .
. . . but there are many others.
![Page 20: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/20.jpg)
Remember• Mobile matters, but desktop owns web• Stats key to your device selection
– Android phones and iPhone / iPad are must-do– Set percentage standards, but then tune based
on other factors• Don’t rely on emulators – consider cloud
solutions for device hosting
![Page 21: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/21.jpg)
Wrap-up
Questions?
Thanks!
![Page 22: Development and testing By: Stephen Frein](https://reader033.vdocument.in/reader033/viewer/2022060108/554dc694b4c905c7488b5116/html5/thumbnails/22.jpg)
Endnotes1. http://www.forbes.com/sites/chuckjones/2013/10/02/ecommerce-is-growing-nicely-while-mcommerce-is-on-a-tear/
2. http://bgr.com/2013/11/08/mobile-devices-internet-traffic-q3-2013/
3. http://phx.corporate-ir.net/phoenix.zhtml?c=176060&p=irol-newsArticle&ID=1886961&highlight=
4. http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/
5. http://www.walkersandsdigital.com/Walker-Sands-Mobile-Traffic-Report-Q3-2013