aro for developers

AT&T Application Resource Optimizer (ARO)

A Guide for Developers

Doug Sillars

Bill Weir

April 2015

Mobile Development Is ChallengingThere are a lot of issues to consider… 19,000 Android User Agents reported by Akamai

There are Screen Size differences

Processor/Memory constraints

Battery constraints

And there are constraints inherent in Wireless Networks

It isn’t easy!

Don’t Let Users Do Your Quality Control44% of all issues are reported by users20% issues were only found because of complaints in app reviews

34% of Users use ratings as primary influence in choosing an app

http://info.perfectomobile.com/rs/perfectomobile/images/why-apps-fail-infographic.pdf

Users Expect Things to Just Work

This was an actual tweet.

Defects vs. PerformanceIs Performance REALLY that important? Yes!

https://t.co/7bMT4A1PIt

Top App Complaints from Consumers

1.User Interface Issues (58%)

2.Performance (52%)

3.Functionality (50%)

4.Device Compatibility (45%)

Performance Importance?How Important is Speed and Responsiveness of Mobile Apps?


From a Mobile User Survey conducted by Dimensional Research

How Fast Should an App Startup?

72%<3 seconds

49%<2 seconds

91%<4 seconds


From a Mobile User Survey conducted by Dimensional Research

Reactions to Poor Performance

Check Your App’s Performance

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-criteria

http://developer.att.com/application-resource-optimizer

http://tspace.web.att.com/files/app#/file/9ecf48ae-b5f7-4780-8672-8c53520c1fb3

Here Are Some Guidelines:

You Can Test Performance With ARO ARO Background

ARO Manual Test (Demo!)

ARO Analysis

ARO Automation (NEW!)

Find The Issues Early With AROARO Records and Analyzes Network Interactions of App

Network use accounts for 40-70% of battery drain!

ARO Requirements• Open Source and Free

Developer.att.com/ARO

• Required for AT&T Preload apps

• iOS

- Requires iOS device tethered to Mac (with Xcode)

• Android

- Requires Android device tethered to Mac/PC (with Android SDK)

Testing Your App With ARO1. Move at a speed that emulates real users 2. Create flexible (yet reproducible) test cases

a. Test cases can even reflect more of a feature checklistb. The Trace is not about the order of the features, but about the

content 3. Likely Problem Areas

a. Startupb. Streaming

4. Be Curiousa. If something seems odd, explore it b. Screen rotation best practice came out of tester noticing the

network lighting up each time they turned the device 5. Background Data:

a. Run long test with app running in background

Live Demo of ARO Test

Demo Backup Slides

In ARO Analyzer App: Start Data collector

Name the testCapture Video or not

Demo Backup SlidesARO establishes a VPN connection on your Android Device (click ok)

ARO shows Screen on your computer (if video is being recorded)

Click Stop to end ARO traceFiles copied to PC and analyzed

ARO Analysis

All network traffic mapped to processes

Stats about trace

Best Practices

ARO Simple Analysis

5 Best Practices Graded for Preloads:1. Text File Compression

2. Duplicate Content

3. Closing Connections

4. Periodic Connections

5. GPS/Bluetooth

© 2015 AT&T Intellectual Property. All rights reserved. AT&T, the AT&T logo and all other marks contained herein are trademarks of AT&T Intellectual Property and/or AT&T affiliated companies. All other marks contained herein are the property of their respective owners. Information contained herein is not an offer, commitment, representation or warranty by AT&T and is subject to change. Mention of a specific company or entity is not an endorsement by AT&T.

AT&T Defect MetricsDuplicate Content

<5% of KB

5-20% of KB

20+% of KB

Periodic Connections

0-20% of Power

20-50% of Power

50+% of Power

GPS/BT

0-25% Active (s)

25-65% Active (s)

65+% Active (s)

Text Compression

100-500KB

500-1000KB

1000+KB

Closing Connections

0-20% of Power

20-50% of Power

50+% of Power

Sev 3

Sev 2

Sev 1


Duplicate Content

AT&T preload says <5% is a pass.

This 2.8 KB file should be cached for 1 year. yet downloaded twice in ~2 minutes.

Download Less Stuff: Caching17% of HTTP content on mobile is duplicate data

– Reading from Cache is 75-99% faster than downloading again

“Web Caching on Smartphones: Ideal vs. Reality”, http://www.research.att.com/~sen/pub/Caching_mobisys12.pdf

6.7MB 6.7MB

4.8 MB duplicate!75% possible savings!

(data AND speed)

Caching Methods (How do I do it?)ETags

Cache Control Headers

•Each file has a Unique Tag

•Revalidated on server for each request– High Performance Web Sites:

Rule 1 – Make Fewer HTTP Requests (1)

– Adding a connection drains battery, adds 500-3,000 ms latency

• Important to carefully assign Max-Age times

•App will not check file on server until Max-Age is reached– Retrieval is strictly file processing time

(1) http://developer.yahoo.com/blogs/ydn/posts/2007/04/rule_1_make_few/

Comparing ETag vs. Cache Control

Network Usage Battery Drain Data Usage Speed

NO Cache YES entire file downloaded

Full network connection made

High Slowest

ETagYes – ETag in cache

is validated on server

Full network connection Low Slow

Cache Control No N/A N/A Fast

Any Caching is better than no caching, but for mobile, Cache Control is preferred

Closing Connections

< 20% Sev 3 defect

Blue Burst Color indicates Packets closing connections

Closing Connections: CODE

MultiRes Sample app from Android SDK

– Modified to download images

HttpURLConnection getimagecloseconn = (HttpURLConnection) urln.openConnection();

getimagecloseconn.setRequestProperty("connection", "close");

getimagecloseconn.connect();String cachecontrol = getimagecloseconn.getHeaderField("Cache-Control");InputStream isclose = getimagecloseconn.getInputStream(); bitmap = BitmapFactory.decodeStream(isclose); getimagecloseconn.disconnect();

https://github.com/attdevsupport/ARO/tree/master/2013DevSummitTurbocharge

https://github.com/attdevsupport/ARO/tree/master/2013DevSummitTurbocharge

GPS/Bluetooth

< 25% Sev 3 defect

Green Color Indicates active GPS usage


GPS: AT&T DriveMode for Android

3 min

480 GPS connections/24hr

~20% Battery


GPS: AT&T DriveMode for Android

Use activity APIs to identify when your users are in motion

11 minute trace

DetectedActivity.IN_VEHICLEDetectedActivity.ON_BICYCLE:DetectedActivity.ON_FOOT:

http://commons.wikimedia.org/wiki/File:Baby_goats_jan_2007_crop.jpghttp://commons.wikimedia.org/wiki/File:Goat_in_a_car.jpghttp://commons.wikimedia.org/wiki/File:2009_goat_and_bicycle_in_India_6512003001.jpg


Periodic Connections:

• We have a great new service• But we need to poll every 30 seconds• Is this bad for the network?

AT&T Sample App


AT&T Sample App (cont.)

• 2 concurrent connections• Extra Network utilization• If your capacity is set on # of connections – it just got halved


AT&T Sample App (cont.)

• Moved to persistent connection – is this better?• 10 minutes = 20 connections 0.67% of battery• 1 hour =120 connections 4% of battery• 24 hours = 2880 connections 96.7% of battery

Text Compression

> 1MB% Sev 1 defect

Periodic Connections in the Background!Background traffic

Many Apps wake up in the background to update data or communicate with servers.

Just 10 background connections = 0.5% battery/day

And it adds up FAST!

Look at Android’s JobScheduler API (5.0 and up)

Background Traffic

AT&T Corp Email

Facebook

Exercise app

ARO Automation – Headless collectionBeta product: release planned July 2015

Command Line and Java APIs to run ARO

./aro --startcollector vpn_android --output /users/demo/MCoC_testing --video yes

ARO Automation – Analysis• Open trace in Analyzer tool

• Analyze headlessly into JSON or HTML

./aro --analyze /users/demo/MCoC_testing --output /users/demo/MCoC_testing/report.html --format html

ARO for Developers• Performance testing is crucial

• Good tools and test plans are essential

• AT&T Application Resource Optimizer (ARO)

• Testing manually

• Testing with automation

• Analysis and filing bugs


Q&A

http://smarterappsguidelines.gsma.com/

http://knowmyapp.org/

http://www.appqualityalliance.org/aqua-performance-test-criteria

http://developer.att.com/application-resource-optimizer

http://bit.ly/HighPerfAndroidApps

aro for developers

Mobile

app complaints

user interface issues

app http

app reviews

lot of issues

app startup

performance importance

mobile user survey