Mobile quality and security issues affect the entire brand and cost more than a “Forced Closure”

Thomas Gronbach, Keynote Systems

19.November 2012

© 2012 Keynote Systems

Agenda

2

Mobile is a TOP Priority for Enterprises

Who is Enterprises?

What does it Mean TOP Priority?

What does it Mean for Mobile Development?

Effect of TOP Priority on Mobile Development

It is Happening: Case Studies

The Developer

Mobile Development Puzzle

© 2012 Keynote Systems

Mobile is a TOP Priority

3

© 2012 Keynote Systems

Some Mobile-Enterprise Profiles

Main website (HTML)

Mobile-optimized website (HTML)

Tablet-optimized website (HTML)

Commercial banking app

(Native: iOS, Android, BB)

Investment app

(Native: iOS, Android, BB)

Mobile payment app

(Native: iOS, Android)

Financial Institution

Retailer

Consumer Applications Internal Applications

Internal corporate website (HTML)

Mobile-optimized corporate website (HTML)

Trading app (Native: BB)

Expense app (HTML)

Type of Company

Main website (HTML)

Mobile-optimized website (HTML)

Mobile app (Hybrid – native with some web content: iOS, Android)

POS app (Native: iOS)

Time tracking (HTML)

Inventory tracking (HTML)

Who chooses this as TOP Priority? Who is Enterprise?

© 2012 Keynote Systems

What does it mean TOP Priority?

5

Enterprises Investments

Secure Mobile Apps

Qualitatively Sound Mobile Apps

Reliable Mobile Apps

Available Mobile Apps

Mitigate Mobile Risk

© 2012 Keynote Systems

What does it mean for mobile development?

6

Get SQRAM into your work

Good quality development

Test before production

Test well before production

Test a lot before production

Don’t forget that testing gets SQRAM into your work

© 2012 Keynote Systems 7

Humana

Fortune-100 healthcare company with over 40,000 employees Healthcare operations in all 50 US states Goal

Provides consumers and businesses with mobile access anywhere

any channel across various healthcare operations

HTML5-based web apps, mobile websites and native mobile apps

Approach

Technical drivers Support emerging mobile technologies

Tooling to support existing enterprise development/QA processes and professionals. Allow process improvement based on tooling capabilities

Support a highly regulated environment

Business driver

Humana corporate initiative: expand mobile access to patient information

Develop mobile process improvements to support upcoming mobile web and applications portfolio additions

Device flexibility supporting changing market and project requirements

Provide reliable and expedient access to consumers’ health information whenever needed

Humana: Mobile Quality for Business Return

© 2012 Keynote Systems 8

Solution

• Keynote’s DeviceAnywhere TCE Monitoring platform provides remote access to TomTom

devices, including its latest generation devices and the iPhone 4

• Connection to a live mobile network in France

• Housed in a data center in Paris, Keynote DeviceAnywhere created a customized

environment, providing GPS connectivity supporting functionality and performance

monitoring scenarios

Benefit

• TomTom’s operations receives real-time alerts about software, hardware and network

issues that could affect the end-user experience.

• Alerts are pre-defined and require no additional involvement from TomTom’s

development team

• Focus on developing the next-generation features and services

TomTom: Mobile Quality for Business Return

© 2012 Keynote Systems 9

Cloud Mobile Testing for Enterprises

App-Test-Factory

for quality assurance and risk mitigation

Internal Mobile App Store

at German Automobile Company

Target Mobile Cloud Testing for Enterprises

© 2012 Keynote Systems

The Mobile Developer

10

Developer are facilitator for Enterprise’s TOP Priority

Thought leader

Differentiated developer do things first

Real world testing

Real device testing

Simple, easy access for everyone

© 2012 Keynote Systems

Mobile Development Puzzle

11

Testing in a humongously heterogeneous and diverse environment

QA Effort Portion of Development Timeline

Effo

rt (

Tota

l Ho

urs

Sp

ent)

Traditional Products Mobile Products

Requirements/Design Development QA

App type: native, web, hybrid

OS variety

Device variety

Differing display sizes

Differences in processing speed, memory

Android customizations

Device life cycle – 12 to 18 months vs. 2 – 3 years

Connectivity

Network latency

Connectivity fluctuation

© 2012 Keynote Systems

Keynote DeviceAnywhere – Mobile Cloud Testing

© 2012 Keynote Systems

Rack mountable tabs

Live Mobile Device

Hardwiring Harness Device Standard I/O Status/Service

Indicators Power Switch

SIM Switcher (USB) Data Cable Connection (USB)

Power Server Connection (USB)

SIM Card Slot

RF Antenna

© 2012 Keynote Systems 14

Mobile Testing Data Center

© 2012 Keynote Systems

Booth 5 at MoMo National Summit, November 19th

15

Hands-on demo

Test your app, now!

Humana case study – request 3 pager for details

Get started

Learn about our free offering

Learn about Testing

Be invited to our Mobile Testing Event in Munich, February 19th 2013

Meet us at

HP-Discover, Dec 4-6, 2012, Frankfurt

Mobile World Congress, Feb 25-28, 2013, Barcelona

Get to know Keynote Systems, Inc. – 19 years of SaaS offerings, the global leader in Internet and mobile cloud testing & monitoring

Appendix

© 2012 Keynote Systems

Mobile is a TOP Priority

17

CIOs provided their feedback in a survey at Gartner’s 2012 CIO Leadership Forum, themed Amplify the Enterprise

Mark P. McDonald, Ph.D, Group VP & Head of Research, Gartner Executive Programs – January 2012

© 2012 Keynote Systems

Elements of a Test Strategy

18

Devices Device Rationalization

Optimize coverage (or minimize risk)

Control costs (time, equipment)

Testing Platform

Real device testing platform

Remote access to enable geographically diverse teams

Flexible device deployment options

Security and management

Test Automation

When and where to automate

Efficient automation of web, hybrid, and native apps

Flexible scripting options to enable QA engineers with different skillsets

© 2012 Keynote Systems

Real Device Testing Platform: Remote Access to Real Devices

19

What device input and output capabilities are required?

Keys; tapping and swiping touch screens

Opening and closing devices, connecting/disconnecting the battery, accelerometer support, audio, backlight, camera

Viewing the LCD with accurate pixel-by-pixel displays

Benefits

Enables management and storage of test assets and test results

Collaboration environment for QA

© 2012 Keynote Systems

Multiple Options for Accessing Real Devices

20

Devices Shared Devices

24X7 access to hundreds of smart devices

Use for compatibility testing, “untrusted device” testing

Private Devices

24X7 access to your devices

Enables geographically diverse team

Inside corporate firewall or external

Local Devices

Plug your smart devices on hand directly into your local computer

Make use of existing assets, great for local teams

Keynote Confidential

© 2012 Keynote Systems

Approach to Rationalizing Test Devices

Prioritize key criteria Choose the target # of devices The Device Planner generates a device list

based on device popularity, by finding devices that provide coverage across key criteria

Example: The Samsung Galaxy SIII is chosen first as the most popular Android device. The Galaxy Nexus is another popular device, but as its OS is also Android 4.0, it moves to another device, e.g. the LG Spectrum on Android 2.3, to maximize coverage across OS versions.

The Device Planner can be found at tce.deviceanywhere.com

Device Planner:

Optimized Device

List

Reduce required test devices by defining key criteria (e.g. OS version, screen resolution, CPU, etc.) that will impact the performance of your application and optimize for popular devices that provide coverage across these criteria

Device Planner:

Key Criteria

Device Planner: a free tool by Keynote

© 2012 Keynote Systems

Test Automation

22

As with software QA, test automation of mobile apps both save money and improve the effectiveness of your QA test process

Save Money

Automate smoke tests and regression tests to reduce cost of manual testing

Find bugs earlier in the process when it is cheaper to fix them

Execute more tests in less time, improving time to market and/or increasing test coverage

Execute 2.5 – 3X faster than manual tests

Schedule tests to run offline; utilize up to 24 hours of each day

Improve Quality

Find more bugs: increase test case coverage, reduce tester fatigue

Offload testers from repetitive tasks, allowing them to focus on product quality (e.g. exploratory testing)

Increase device coverage

Address quality in an increasingly fragmented device market

The more models you test against, the more efficiency you gain

© 2012 Keynote Systems

Mobile Application Testing Stages: Where to Automate?

23

Goals

Test Cases

Frequency

Methodology

Basic acceptance, build acceptance

Functional testing across all existing areas of product

Compatibility across devices

Very high value to automate!

High value to automate

Medium value to automate

# of Devices

Optimal Test Strategy

Automated Testing

Automated Testing

Manual or Automated Testing

Smoke Testing

Regression Testing

Compatibility Testing

Deep testing of new areas of functionality

Low value to automate

Manual Testing

New Feature Testing

Not automated

Exploratory Testing

Exploration of functionality from customer viewpoint

Manual Testing

November 22, 2012

© 2012 Keynote Systems

Use Case: Automation

24

One customer had a series of 100 test cases run across one Android and one iOS device that were required to verify each new build. Two QA engineers were effectively completing 40 test cases per day, or requiring 1 week of elapsed time (10 man-days)

Customer utilized DeviceAnywhere Test Center Enterprise Automation to automate all 200 test cases, which can now be scheduled to automatically run overnight when a new build is generated.

As a result, the customer improved time to market by 1 week, while freeing up QA resources for other testing efforts.

© 2012 Keynote Systems 25

Test hybrid and web applications, from a single platform, allowing you to:

• Test on real devices, ensuring test accuracy

• Enable all of your remote and local employees, with or without existing scripting skills

• Save time by automating native, web, and hybrid application testing

DeviceAnywhere Platform – Enabling Testing of Mobile Apps

THANK YOU!