hp perfecto webinar - uft mobile

35
© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. How to Build an Enterprise Grade Mobile Testing Strategy Eran Kinsbruner Product Marketing @Perfecto Mobile Antoine Aymer Product Marketing @HP A

Upload: perfecto-mobile

Post on 07-May-2015

503 views

Category:

Technology


1 download

TRANSCRIPT

Page 1: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

How to Build an Enterprise Grade Mobile Testing Strategy

Eran Kinsbruner Product Marketing @Perfecto Mobile

Antoine Aymer Product Marketing @HP

A

Page 2: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 2

How different is Mobile?

A

Page 3: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3

Type

Click Double click Right click Drag

Tap & Multi tap Double / long tap

Drag Swipe

Pinch & zoom

Locate

Type

Rotate / Tilt

Snapshot

Authenticate

Communicate

Speak

A

Page 4: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 4

CES 2014, Smart Innovation

BYOD Inflight Entertainment

Contactless payment

Smart

e-bike

Smart gardener

Smart clothes

Smart nappy

Smart appliance

Augmented reality

A

Page 5: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5

Starbucks’ app

Features: • Pay for your purchases • Track and redeem rewards • Reload Starbucks Card’s balance • Get directions to the nearest store • Explore coffees, beverages, food and

nutritional information • Send eGifts instantly to your contacts • Stay informed by being the first to

hear of special offers

Improve in-store customer experience

Generate customers loyalty

Increase sales

A

Page 6: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6

But this is not enough… A

Page 7: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7

Use case #1

SaaS-based applications are expected to generate a growth rate of 17.4%.

Sport App connected with smartwatch

Unit test

• Unit test • 100% emulation

Interoperability

• Data transfer via Bluetooth • 100% manual/human testing

Industrialisation

• Functional testing on major devices • 80% automated testing

A

Page 8: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8

Why Mobile Testing looks so difficult?

Page 9: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9

It is the jungle!

versus versus

Development platform

Various OS types, versions, device/tablet, OEMs

Evolving

Browsers

Page 10: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10

DEVICE

TESTING

MARKET

Performance Network (>400 operators)

Contextual issues Peripherals Impact on back-end Emulator vs real mobile

Automation vs manual Device management What Devices to test on? Bug-fix latency Unavailable services Off-shore

Time to market

Fragmented landscape (OS, version, input, screen, style)

USER Know the user Users’ feedback

Page 11: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11

Why Mobile Testing looks so difficult

A

48% lack

expertise

38% lack tools

Page 12: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12

Mobile testing disciplines

Functional

• Validation of Functionality

• Smoke / Regressions Testing

• Offline access testing

• Negative Testing

Non Functional

• Network Strength / Outage / Recovery

• Different NW Types

• Peripheral Testing

• Services

Interoperability (IOP)

• Voice / SMS interrupts

• Notifications

• Battery /Cable Removal

Memory Leak

• Memory Usage

• Memory Leaks

• Garbage Collection

Performance Testing

• CPU Usage testing

• Network Usage

• Page Render time or activity Render time

Usability Testing

• User Experience

• Competitive Analysis

• Expert Review

Installation Testing

• New App Install

• Uninstall and Reinstall

• Upgrade testing

Security Testing

• OWASP Vulnerabilities

• Dynamic Testing

• Static Code Analysis

• Data Encryption

Language Testing

• Validation for Locales

• Images and Text

• Currencies, time zones etc.

• Context

Page 13: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13

Definition of Fully Tested

Page 14: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14

The Truth Is Out There! Which of the following statements reflect your development team's approach to testing mobile apps?

4%

21%

27%

31%

34%

55%

55%

55%

60%

We don't formally test or instrument our appsbefore deploying them; we fix changes as needed

We integrate crash reporting/analytics into ourdeployed application to monitor and fix problems

We automate our testing process as part of acontinuous integration process

We use a cloud-based service to test multipledevices

We test devices that are deployed on multiplecellular carriers

We do a quick test on an emulator/simulator or ona single device

We test across multiple devices that run the sameversion operating system

We test across multiple OS versions

We test our app across multiple device orientations

Source: Forrester, Q1 2013 Base: 765 Software developers developing mobile applications

A

Page 15: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15

Mobile is Agile on Steroids

A

Page 16: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16

The ideal Mobile testing picture

Cum

ula

tive

bu

sin

ess

valu

e

Time

Sprint 4 Sprint 3 Sprint 2 Sprint 1

CI CI CI CI

V1.0 V1.1 V1.2

Release, Iteration, Nightly, Continuous

End-to-End test cycle

On any device, by any team, from any location

Longevity testing (independent of sprint cycles)

Page 17: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Agile and Testing -- 5 o’clock on Friday the 30th, do you know if your code is “done”?

Page 18: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18

The Truth Is Out There!

52% spend

more than an hour debugging/fixing

code

49% of

developers spend more than an hour a day testing new

code

Source: Forrester, Q1 2013 Base: 765 Software developers developing mobile applications

only 15% spend that much time writing test automation code

A

Page 19: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19

Agile mandates automation

Page 20: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20

Use case #2

SaaS-based applications are expected to generate a growth rate of 17.4%.

Top US Bank

Page 21: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21

Use case #3

SaaS-based applications are expected to generate a growth rate of 17.4%. Type of Testing

3 weeks

Mobile banking, 12 Millions unique users native application and mobile enabled website

Impact

manual

Deployment

In-house

A

Page 22: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22

Use case #3

SaaS-based applications are expected to generate a growth rate of 17.4%. Type of Testing

3 weeks

Mobile banking, 12 Millions unique users native application and mobile enabled website

Automated

Deployment

Cloud

48 hours

86% time savings

60% cost savings

Impact

A

Page 23: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23

Automation requirements

Page 24: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24

What can get in the way?

Silos App Style Cross-Platform

Differences Use cases

web store

Page 25: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25

Cross-platform differences

Menu panel

Text colours

Images

Name position

Labels

Button images

Page 26: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26

Building an Enterprise Grade Mobile Testing Strategy

Page 27: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27

Sprint 4 Sprint 3 Sprint 2 Sprint 1

CI CI CI CI

Start

Transform Grow

Predictability, scalability

What fits my workflow?

Coverage, Reliability & Governance

Page 28: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28

Finding the right balance

Coverage

Devices

Service Level

Governance

Existing Tools

Time to Market Locations &

Networks

A

Page 29: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29

HP UFT Mobile

Page 30: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30

na

tiv

e

#1 HP UFT Add-in #2 Deployment #3 Devices

Page 31: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31

HP UFT Extension Integrated with HP ALM

Real mobile devices (and emulators) Perform almost any operations

Device agnostic Develop, run and schedule one

test across multiple platform Visual & Object analysis

Manageable Full IT control on authentication, access permission and activities

Predictable availability

Global Accessible from any location (VPN mandatory with MCE)

Secured Full tractability of actions. Device cleanups and encryption abilities Continuous testing

Support fast, agile delivery

Scalable From single, small project to hundreds of simultaneous users

Complex scenario advanced built-in functionalities such voice injection and recognition, NFC, location based #1 HP UFT Add-in

Page 32: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32

#2 Deployment

Public Cloud

Virtual Private Cloud

MobileCloud Express

Page 33: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33

Use case #4: Virgin Media

Virgin Media UK Offices V

irg

in M

edia

Co

rpo

rate

Net

wo

rk

On-premise lab

Team members

Core team

Off-shore

Virtual Private Cloud Dedicated devices in various geographies

Multiple devices in various geographies

Public Cloud

Page 34: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34

#3 Devices

List of supported devices: http://www.perfectomobile.com/portal/cms/services/handsets.html

Wide device spectrum HTC, Apple, Nokia, Motorola, Samsung, SonyEricsson, LG,

BlackBerry, Palm, Kindle

Full device control Fast charging cycle, audio injection, reboot, clean/wipes

Scalable Additional handsets are added on-

demand

Global The handsets resides in various datacenters in the world: US, Canada, UK, France, Germany, Spain, South Africa, India.

Deploy your app Testers can deploy their application on the device, test it as if you were holding the handset in your hands.

Powerful replay all tests are video recorded to allow

easy recreation of defects

Page 35: Hp perfecto webinar - UFT Mobile

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 35

SCO

PE

AU

TOM

ATIO

N

Choose the right devices

Define relevant tests across all target devices

Traditional non-mobile automation rules still

apply

2

(before the demo)

Be intimate with manual testing

Think about future maintenance

Implement continuous integration

Gain mobile objects experience

Always sanity test your previous version

1

3

4

6

5

7

8