android performance bundle - disco · ieee, 2016, proceedings of the 13th international conference...
TRANSCRIPT
Android Performance Bundle
23.05.2017 Android Performance Bundle - Noah Hollmann 1
Performance on Android
23.05.2017 Android Performance Bundle - Noah Hollmann 2
Measuring is crucial!
1. Automated UI Testing
23.05.2017 Android Performance Bundle - Noah Hollmann 3
Measuring is crucial!
1. Automated UI Testing
2. Real world benchmarking of phones
23.05.2017 Android Performance Bundle - Noah Hollmann 4
Measuring is crucial!
1. Automated UI Testing
2. Real world benchmarking of phones
3. Benchmarking in the app store
23.05.2017 Android Performance Bundle - Noah Hollmann 5
Performance on Android
23.05.2017 Android Performance Bundle - Noah Hollmann 6
Energy leak GUI Lagging Memory Bloating
Performance on Android
23.05.2017 Android Performance Bundle - Noah Hollmann 7
Energy leak Memory BloatingGUI Lagging
What does GUI lagging mean?
• „App not responding“
23.05.2017 Android Performance Bundle - Noah Hollmann 8
What does GUI lagging mean?
• „Janky“ frames• 60 fps Framerate
• = 16-ms-per-frame
23.05.2017 Android Performance Bundle - Noah Hollmann 9
Vsync Vsync
What does GUI lagging mean?
• „Janky“ frames• 60 fps Framerate
• = 16-ms-per-frame
23.05.2017 Android Performance Bundle - Noah Hollmann 10
17ms UPDATE
Vsync Vsync
How to measure fluency?
23.05.2017 Android Performance Bundle - Noah Hollmann 11
How to measure fluency?
23.05.2017 Android Performance Bundle - Noah Hollmann 12
FPS
Rendering time
Issued frames
“Analyzing GUI Running Fluency for Android Apps“State Key Lab of Computer Science, Institute of Software, Chinese Academy of Sciences Beijing, China
23.05.2017 Android Performance Bundle - Noah Hollmann 13
Existing tools
23.05.2017 Android Performance Bundle - Noah Hollmann 14
Android Systrace Profile GPU Rendering
Existing tools
23.05.2017 Android Performance Bundle - Noah Hollmann 15
FPS Meter Instrument the source code
ARFluency - Implementation
<<Process>>SurfaceFlinger
<<Layer>>AppContentLayer
<<Layer>>StatusBar Layer
<<Layer>>Content Layer
23.05.2017 Android Performance Bundle - Noah Hollmann 16
Android App
draw
notify
Hardware Composer
ARFluency - Implementation
<<Process>>SurfaceFlinger
Code injection:libsurfaceflinger.so
<<Layer>>AppContentLayer
<<Layer>>StatusBar Layer
<<Layer>>Content Layer
Dumpsys
23.05.2017 Android Performance Bundle - Noah Hollmann 17
Android App
draw
Hardware Composer
notify
Experiments
23.05.2017 Android Performance Bundle - Noah Hollmann 18
1 1 1
4
16
35
21
15
24
26
2
4 4
14
13
CHROME SHADOWSOCKS GOOGLE TRANSLATE
GOOGLE PHOTOS
ANTUTU BENCHMARK
Issued frames (%) Frame rate (fps) Rendering time (ms)
Critique
23.05.2017 Android Performance Bundle - Noah Hollmann 19
Different perspectives
23.05.2017 Android Performance Bundle - Noah Hollmann 20
Many device configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 21
Many device configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 22
Many device configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 23
Android is unpredictable
23.05.2017 Android Performance Bundle - Noah Hollmann 24
Android is unpredictable
23.05.2017 Android Performance Bundle - Noah Hollmann 25
“Mining Test Repositories for Automatic Detection of UI Performance Regressions in Android Apps“13th International Conference on Mining Software Repositories (MSR'16), May 2016, Austin, Texas, United States. IEEE, 2016, Proceedings of the 13th International Conference on Mining Software Repositories.
23.05.2017 Android Performance Bundle - Noah Hollmann 26
App Version 1
“Mining Test Repositories for Automatic Detection of UI Performance Regressions in Android Apps“13th International Conference on Mining Software Repositories (MSR'16), May 2016, Austin, Texas, United States. IEEE, 2016, Proceedings of the 13th International Conference on Mining Software Repositories.
23.05.2017 Android Performance Bundle - Noah Hollmann 27
App Version 1
Automated Testing
23.05.2017 Android Performance Bundle - Noah Hollmann 28
Simulated device profile
Test Suite(UI events)
Test executionWith Robotium and
Genymotion
Dumpsys
Logcat
Test Repository
23.05.2017 Android Performance Bundle - Noah Hollmann 29
23.05.2017 Android Performance Bundle - Noah Hollmann 30
[v1, 4.1, samsung, S3, x86, wifi]
Automated Testing
23.05.2017 Android Performance Bundle - Noah Hollmann 31
Simulated device profile
Test Suite(UI events)
Test executionWith Robotium and
Genymotion
Dumpsys
Logcat
Test RepositoryCompare to
test repository
Finding similar configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 32
[v1; 4:1; samsung; S3; x86; wifi]
Finding similar configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 33
[v1; 4:1; samsung; S3; x86; wifi][v1:2; 4:1; samsung; S3; x86; wifi]
Finding similar configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 34
[v1; 4:1; samsung; S3; x86; wifi][v1:2; 4:1; samsung; S3; x86; wifi]
Finding similar configurations
23.05.2017 Android Performance Bundle - Noah Hollmann 35
[v1; 4:1; samsung; S3; x86; wifi][v1:2; 4:1; samsung; S3; x86; wifi][v1:1; 4:1; samsung; S3; x86; wifi][v1; 4:2; samsung; S3; x86; wifi][v1:2; 4:1; samsung; S3; x86; 4G]
Finding outliers
23.05.2017 Android Performance Bundle - Noah Hollmann 36
Most Similartest runs
OutliersSimilar outliers
(Outliers+, Outliers-)
Association Rule Mining
23.05.2017 Android Performance Bundle - Noah Hollmann 37
V5.0.0 => Outlier-Wifi => Outlier+
Similar outliersAssociationRule Mining
AssociationRules
Evaluation
Exo-Player Space Blaster K-9 Mail client
23.05.2017 Android Performance Bundle - Noah Hollmann 38
Evaluation
23.05.2017 Android Performance Bundle - Noah Hollmann 39
Evaluation
23.05.2017 Android Performance Bundle - Noah Hollmann 40
SDKs: 5.*, 6.0.0SDKs: 4.*
Device Lab Upgraded Device Lab
Experiment - Performance Degradation
23.05.2017 Android Performance Bundle - Noah Hollmann 41
Experiment - Performance Degradation
23.05.2017 Android Performance Bundle - Noah Hollmann 42
10/10
Experiment – Context isolation
23.05.2017 Android Performance Bundle - Noah Hollmann 43
Experiment – UI Event filtering
23.05.2017 Android Performance Bundle - Noah Hollmann 44
20 Events / 4 Janks
Precision: 53%Recall: 83%
Critique
23.05.2017 Android Performance Bundle - Noah Hollmann 45
What you could do…
23.05.2017 Android Performance Bundle - Noah Hollmann 46
23.05.2017 Android Performance Bundle - Noah Hollmann 47
“How Developers Detect and Fix PerformanceBottlenecks in Android Apps“
485 participants Sept 2015 > 1 year experience
Mario Linares-Vásquez, Christopher Vendome, Qi Luo, Denys PoshyvanykThe College of William and Mary, Williamsburg, VA, USA
23.05.2017 Android Performance Bundle - Noah Hollmann 48
Performance on Android
23.05.2017 Android Performance Bundle - Noah Hollmann 49
Energy leak Memory BloatingGUI Lagging
Datacollection
• 381,161 Java projects
• 16,331 potential repositories
• 24,340 contributors
• 628 survey responses
• 485 valid responses
Valid response rate: 2% (out of 24,340 contributors)
23.05.2017 Android Performance Bundle - Noah Hollmann 50
Preferred practices for detecting bottlenecks
23.05.2017 Android Performance Bundle - Noah Hollmann 51
11
5
11
1
71
56
21
17
R + M M R + M + T R M + T + O T
ReviewManualToolsOther
23.05.2017 Android Performance Bundle - Noah Hollmann 52
23.05.2017 Android Performance Bundle - Noah Hollmann 53
There are many tools available!
23.05.2017 Android Performance Bundle - Noah Hollmann 54
Preferred practices for detecting bottlenecks
23.05.2017 Android Performance Bundle - Noah Hollmann 55
11
5
11
1
71
56
21
17
R + M M R + M + T R M + T + O T
ReviewManualToolsOther
GUI Lagging: Solution Practices
23.05.2017 Android Performance Bundle - Noah Hollmann 56
74
47
40
32
17
4
THREADS GUI OPTIMIZATION
CACHING MEMORY MANAGEMENT
NETWORK RESOURCES
DELEGATION (CLOUD)
Number of people
Memory bloat: Solution Practices
23.05.2017 Android Performance Bundle - Noah Hollmann 57
45
30
3,5
2
GU I OPTIMIZ A TION MEMORY MGMT. NETWORK RESOURCES O N-D EMA ND RESOURCES
Number of people
Energy leak: Solution Practices
23.05.2017 Android Performance Bundle - Noah Hollmann 58
4
1 1
D ELEGA TION (CLOUD ) RED UCE GPS CA LLS RED UCE WA KELOCKS
Number of people
Critique
23.05.2017 Android Performance Bundle - Noah Hollmann 59
23.05.2017 Android Performance Bundle - Noah Hollmann 60
Papers & Sources
• Mining test repositories for automatic detection of UI performanceregressions in Android appsMaría Gómez, Romain Rouvoy, Bram Adams and Lionel Seinturier. MSR 2016.
• How developers detect and fix performance bottlenecks in Android appsMario Linares-Vásquez, Christopher Vendome, Qi Luo and Denys Poshyvanyk. ICSME 2015.
• Analyzing GUI running fluency for Android AppsTian Huang, Zhenyu Zhang and Xue-Yang Zhu. MSCC 2016.
• Android developershttps://developer.android.com/index.html
23.05.2017 Android Performance Bundle - Noah Hollmann 61
23.05.2017 Android Performance Bundle - Noah Hollmann 62