7 steps to pragmatic mobile testing · © 2017 splunk inc. © 2017 splunk inc. confidential...

© 2017 SPLUNK INC.

© 2017 SPLUNK INC. CONFIDENTIAL INFORMATION. DO NOT DISTRIBUTE.

© 2017 SPLUNK INC.

7 Steps to

Pragmatic Mobile Testing

@TomChavez [email protected] Sr. Manager, Developer Marketing

Feb 14, 2020

© 2017 SPLUNK INC.


7 Steps to Pragmatic Mobile Testing A path to faster, better testing

© 2017 SPLUNK INC.


▶ Expanding Test Plans

Introduction Changes affecting the testing process

© 2017 SPLUNK INC.



▶ Shorter Product Cycles


© 2017 SPLUNK INC.




▶ More Device Platforms


© 2017 SPLUNK INC.





▶ A Changing Focus


© 2017 SPLUNK INC.





▶ A Changing Focus

▶ An Emphasis on Analysis


© 2017 SPLUNK INC.


Time

Duration

Location

Intent

TIP: Work with your product owner, review market

research, conduct user focus groups, and verify/validate

with real user monitoring

Step 1 – Know your user Understand the most common user patterns

© 2017 SPLUNK INC.


Step 2 – Know your app Understand the application flow

Developers intended execution paths

User common use paths

© 2017 SPLUNK INC.


Step 2 – Know your app Understand the application flow

Developers intended execution paths

User common use paths

User revenue generating patterns

Current problem areas

TIP: Use real user data to create test cases

© 2017 SPLUNK INC.


RUM = Real User Monitoring Monitoring and measure all (or at least some) users

© 2017 SPLUNK INC.


Core functions and experiences

Core differentiators

Edge functions

Complicated

Repetitive

Measure manual effort

Know your person days per cycle, per area

TIP: Use your matrix to know which tests to automate

Step 3 – Know your test matrix All tests are not created equal

© 2017 SPLUNK INC.


Rank active devices

Analyze device usage

Weight matrix to device

Know problem areas per device

Core devices to own

Edge devices to borrow/rent

TIP: Choose which devices you must own

for testing and which might be optional but available

based on real user monitoring

Step 4 – Know your devices Understand your device matrix

© 2017 SPLUNK INC.


Using RUM to Build Device Matrix

© 2017 SPLUNK INC.


Build a Device Lab

© 2017 SPLUNK INC.


Start ASAP

Simple

Time Consuming

Mundane

Rhythm

TIP: Goal is Continuous Integration execution

Step 5 – Know how to Automate Return person days to your test cycle

© 2017 SPLUNK INC.


Device reaction to app functionality

Battery, CPU, Memory…

Common user path

Speed per screen or page

Separate back and front end performance

Performance to revenue correlation/transaction or task completion

TIP: Prioritize tests to highest revenue producing/completion paths

Step 6 – Know your mobile performance

© 2017 SPLUNK INC.


Time to Market

Functional

Competitive

TIP: Prioritize which strategy you will apply with the time saved from

test automation

Step 7 – Know your edge Understand ahead of time what you will do when you can

7 steps to pragmatic mobile testing · © 2017 splunk inc. © 2017 splunk inc. confidential...

Documents