seattle at&t hackathon
TRANSCRIPT
Testing Mobile Apps
Testing Mobile Apps is Challenging
There are 19,000 Android User Agents reported by Akamai
There are Screen Size differences
And there are Processor/Memory constraints
It isn’t easy!
Many Do Not Invest Enough in Testing44% of all issues are reported by users
20% issues were only found because of complaints in app reviews
34% of Users use ratings as primary influence in choosing an app
http://info.perfectomobile.com/rs/perfectomobile/images/why-apps-fail-infographic.pdf
Top App Complaints from Consumers
1.User Interface Issues (58%)
2.Performance (52%)
3.Functionality (50%)
4.Device Compatibility (45%)
Performance Importance?How Important is Speed and Responsiveness of Mobile Apps?
https://t.co/7bMT4A1PIt
How Fast Should an App Startup?
72%
<3 seconds
49%
<2 seconds
91%
<4 seconds
https://t.co/7bMT4A1PIt
Testing For Performance
http://smarterappsguidelines.gsma.com/
http://knowmyapp.org/
http://www.appqualityalliance.org/aqua-performance-test-criteria
http://developer.att.com/application-resource-optimizer
Guidelines
Performance Testing With ARO
• ARO Records and Analyzes Network Interactions of App
• Grades Network interactions against 25 Best Practices
Network use
accounts for
40-70% of
battery drain!
ARO Requirements
• Open Source and Free
Developer.att.com/ARO
• iOS
- Requires iOS device tethered to Mac (with Xcode)
• Android
- Requires Android device tethered to Mac/PC (with
Android SDK)
Phone View of TraceARO establishes a VPN
connection on your Android
Device (click ok)
ARO shows Screen on your
computer (if video is being
recorded)
Click Stop to end ARO trace
Files copied to PC and
analyzed
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks
contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific
company or entity is not an endorsement by AT&T.
Duplicate Content
This 2.8 KB file should be cached for 1 year. yet
downloaded twice in ~2 minutes.
Other Considerations in Testing
Background traffic
Many Apps wake up in the background to update data or communicate with
servers.
Just 10 background connections = 0.5% battery/day
And it adds up FAST!
ARO For Mobile Apps
• Performance testing is crucial
• Good tools and test plans are essential
• AT&T Application Resource Optimizer (ARO)
• Testing manually
• Testing with automation
• Analysis and filing bugs
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks
contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific
company or entity is not an endorsement by AT&T.
http://bit.ly/HighPerfAndroidApps
© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks
contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific
company or entity is not an endorsement by AT&T.
Q&A
http://smarterappsguidelines.gsma.com/
http://knowmyapp.org/
http://www.appqualityalliance.org/aqua-performance-test-
criteria
http://developer.att.com/application-resource-optimizer
http://bit.ly/HighPerfAndroidApps
ARO Automation – Headless collectionBeta product: release planned July 2015
Command Line and Java APIs to run ARO
./aro --startcollector vpn_android --output /users/demo/MCoC_testing --video yes