measuring mobile web performance v2
DESCRIPTION
Measuring Mobile Web Performance presentation at the London Ajax Mobile Conference 2nd July 2011. Covers the basics of web performance measurement and looks specifically at the measurement of page load speed from mobile devices.TRANSCRIPT
Stephen Thair, Seriti Consulting, @TheOpsMgr
London Web Performance Meetup Group, @LDNWebPerf
MEASURING MOBILE PERFORMANCE
Before we start…
3Things to keep in mind
Desktop
≠
Mobile
A JAVA APP CALLING A JAVA APP CALLING A C++ APP USING JNI TO CALL A JAVA APP….
http://assets.en.oreilly.com/1/event/60/Analyzing%20the%20Performance%20of%20Mobile%20Web_%20Challenges%20and%20Techniques%20Presentation.pdf
Wifi
≠
Mobile
WIFI VS HSDPA PERFORMANCE
Mobile
≠
Mobile
DIFFERENT PHONES = DIFFERENT PERFORMANCE
http://www.anandtech.com/show/4144/lg-optimus-2x-nvidia-tegra-2-review-the-first-dual-core-smartphone/8
DIFFERENT OPERATING SYSTEMS
DIFFERENT BROWSERS…
http://www.webdevelopersnotes.com/articles/mobile-web-browsers-list.php
DIFFERENT NETWORK TYPES…NETWORK_TYPE_1xRTT Current network is 1xRTT
NETWORK_TYPE_CDMA Current network is CDMA: Either IS95A or IS95B
NETWORK_TYPE_EDGE Current network is EDGE
NETWORK_TYPE_EHRPD Current network is eHRPD
NETWORK_TYPE_EVDO_0 Current network is EVDO revision 0
NETWORK_TYPE_EVDO_A Current network is EVDO revision A
NETWORK_TYPE_EVDO_B Current network is EVDO revision B
NETWORK_TYPE_GPRS Current network is GPRS
NETWORK_TYPE_HSDPA Current network is HSDPA
NETWORK_TYPE_HSPA Current network is HSPA
NETWORK_TYPE_HSUPA Current network is HSUPA
NETWORK_TYPE_IDEN Current network is iDen
NETWORK_TYPE_LTE Current network is LTE
NETWORK_TYPE_UMTS Current network is UMTS
NETWORK_TYPE_UNKNOWN Network type is unknown
http://developer.android.com/reference/android/telephony/TelephonyManager.html
MOBILE PERFORMANCE IS LOCATION-DEPENDENT
http://mytests.3pmobile.com/context/?WyIyUFVDZHlWV2ErZHFISmxCSWFXRENnPT0iLCJEYnFYVVp3SEN4MD0iXQ%3D%3D
HSDPA… ON A #!%&$@*ING TRAIN
http://blog.davidsingleton.org/mobiletcp
Max RTT = 20266ms
How many ways are there to measure web performance?
And how useful are they on Mobile?
6
6 BROWSER PLUG-INSAs used in HTTPWatch, Dynatrace Ajax Edition etc
But only some mobile browsers support plug-ins…
5 PROXY TIMINGE.g. Fiddler Proxy (http://www.fiddler2.com/Fiddler2/version.asp)
There is no proxy settings in Android unless you root your phone (http://android-proxy.blogspot.com/)
Only useful for WiFi…
Beware the “observer effect”
4 WEB SERVER MODULEAs used by many APM solutions e.g. Dynatrace, New Relic
Install mod_ or ISAPI filter
Measure the RTT between web server and DB/Application tier
Great for measuring your back-end performance
Not much good for front end…
3 JAVASCRIPT TIMINGAs used in Boomerang, Episodes, GA, WebTuna etc
OK for web pages, better with new WebTiming and NavigationTiming APIs not that mobile browsers support them…
Relies on JavaScript and Cookies… which is problematic…
2 NETWORK LEVEL TIMINGAs used by PCAPPerf and by Atomic Labs Pion, Tealeaf, Coradiant
PCAPPERF limited to over WiFi (otherwise you can’t capture the packets)*
Pion, Tealeaf etc require network tap…
1 CUSTOM BROWSERAs used by 3PMobile.com (on the phone) and synthetic agents (by active monitoring providers e.g. Site Confidence)
At present this is the only way to get real OTA performance data.
So what to use?
Depends on your needs
Development vs
“Real World”
Network perspective vs
Browser Perspective?
PCAPPERF TESTING
PC acting as Wifi hotspot and capturing packets with Wireshark
http://code.google.com/p/pcaphar/wiki/CaptureMobileTraffics
Upload pcap file to pcapperf.appspot.com
AndView the waterfall
Upload
WiFi Connection to PC
MOBITEST – BLAZE.IO/MOBILE
“PCAPPerf as SaaS” + some on-device event capture + screenshots
WiFi only
Tethered machines in Ottawa, DC, Amsterdam
MOBITEST / PCAPPERF WATERFALL
3PMOBILE.COM “BROWSER2”
Android Native Browser code instrumented for performance measurement
Test Anywhere
Full Mobile context – device & location & network
Browser-level perspective
Screenshots
Automated Testing
DEVICE CONTEXT
LOCATION CONTEXT
NETWORK CONTEXT
3PMOBILE.COM “BROWSER2” WATERFALLNew “detail mode” hybrid waterfall
3PMOBILE OPEN BETA
Register at www.3pmobile.com
Download the APK & install
Test away!
The results from the tests shown today are available
Username = [email protected]
Password = ajaxmobile
Feel free to take a look (but please don’t delete any tests not your own)
So what else is there?
TCP DUMP ON ROOTED PHONEhttp://www.vbsteven.be/blog/android-debugging-inspectin-network-traffic-with-tcpdump/
JAVASCRIPT BOOKMARKLETS
Steve Souder’s Mobile Performance Bookmarlet
http://stevesouders.com/mobileperf/mobileperfbkm.php
None of these measure performance per se, but they can give you more information about the page
(but beware of a strong “observer effect”)
KEYNOTE MITE (EMULATION)
The free Keynote MITE emulator
Webkit-based emulation based on device profiles
DEBUGGING TOOLS
Most SDK’s include debugging tools and profilers that can help you understand what’s going on
But you are tethered to your PC (normally)
WATCHMOUSE “IAM” LIBRARY
In-App Monitoring (IAM) is a set of libraries to help instrument your web apps. Currently iPhone only.
Replace your app WebView calls with IAMWebView
It will gather the performance data and beacon it out for you
https://bitbucket.org/watchmouse/iam/
And that’s all I’ve got…
Questions?• Stephen Thair, Seriti Consulting, @TheOpsMgr
• www.seriticonsulting.com/blog
• London Web Performance Meetup Group, @LDNWebPerf
• Monthly Meetups in Central London (near Bank)
• http://www.meetup.com/London-Web-Performance-Group/