Live Webinar:Elevating Customer Experience with a Mobile QA Strategy That Works

2

Tal BarmeirCEO

Guido LubinskiQA Principal Manager

KumaresanNarayanaswamy

Head, Test Automation CoE

Today’s Speakers

1. How a mobile ecosystem is reshaping the organization priorities

2. Need of a comprehensive mobile testing strategy for superior customer experience

3. Key challenges and opportunities ahead for organizations in mobile-first world

4. Mobile transformation @Vodafone

5. Mobile QA tools – insights to action

6. Q&A

10 min

25 min

5 min

AGENDA 5 min

5 min

5 min

3

4

How a mobile ecosystem is reshaping the organization priorities (1/2)

Digital Channels for Connected Experience

Social Web Smart TVDigital Payments & Signage

WearablesKiosk

“Billion TouchPoints happening everyday on Mobile Apps”

How a mobile ecosystem is reshaping the organization priorities (2/2)

5

People & Culture Process & Governance Technology Diversity

• Competency• Customer centricity• Innovation

• Device policies, device ownership models

• Security, privacy• Light-weight approach

• Diversity of Apps and Devices

• Mobile enabled BizOps• Omni channel experience

Need of a comprehensive mobile testing strategy for superior CX

“High Velocity Quality”

Test strategy to be derived based on App analytics, Device Analytics and Monitoring as continuous process

Progressive automation with ever expanding coverage as continuous

journey throughout lifecycle of mobile apps

Comprehensive focus on functional and beyond-functional aspects like performance, security, compatibility,

accessibility, usability

6

Key Challenges Opportunities

7

Key challenges and opportunities ahead for organizations in mobile-first world

• Anytime, Anywhere Access

• Fragmentation – devices, OS, et al

• Recognize mobile app maturity curve

• “Someone can snoop over”

• Quality cannot be compromised for speed

• Personalized expectations

• Test beyond functional – cover all types of persona for CEx

• Monitor the app throughout its life-cycle

• Tools and automation

• Enterprise Mobile app management and governance

Vodafone Case Study

9

Company Profile Major Telecommunication Company

Headquartered in UK

More than 100,000 employees

Operates in around 80 countries

Customer base of above 400 million worldwide

• High quality demands• Cost pressure• Time-to-market – Agility?

• Location of testers globally distributed• Shipping Confidentiality • Local testing environments

• Variety of (device specific) Mobile Services• Various customer touchpoints and platforms• Country specific SIM cards / roaming

10

Is it easy to get into customers‘ shoes? No!Various constraints when it comes to simulation of real customers‘ behaviour:

Scope

Logistics

Business

Device Cloud(s) to drive Customer Experience!Around 250-300 devices in three flavors:

QA Cloud• QA Core business: ~ 70+ different devices• Acceptance Testing, Managed Test Service• Offshoring, Test Automation

Global App Viewer• 21 markets, 3 different SIMs, 130 devices• Manual access for development teams• Harmonization of local market look and feel

Vodafone Device Cloud• External vendors/suppliers: ~ 60+ devices• Integrate into CI frameworks (Jenkins), fully supporting agile delivery• Automated Device Certification

11

Automated Device Management: discharging, brightness, WiFi

12

Further improvements of the Vodafone Device CloudSome improvements already implemented:

Automated SIM Switcher: that in conjunction with SeeTestCloud, enables testing multiple SIMs on a single device (up to 16 SIMs)

Operational improvements:Floating licenses: allowing flexibility, a unified environment and enables everyone from anywhere to use the cloud.

Distributed Device Cloud: allows, on a need-basis, connecting to a satellite agent device in a different country and testing under real network conditions

Test Automation Framework: with full access for (external) teams and Jenkins integration

13

Apart from being close to the Customer – other benefitsWhat counts is the money:

Enablement of off-shore model with avoiding roaming costs

Reduce flying to different countries and locations

High utilization of devices, # devices reduced significantly

Result: tremendous cost reduction

14

Lessons learnedSome hurdles we have overcome in setup and during run times:

Plan for scalable architecture and optimal

network connectivity

Plan and prepare for appropriate User Management

and Access controls

Automate as much as possible operational support

processes

15

Mobile QA Tools -Insights to Actions

16

Centralized, Onsite Mobile Device LabKeep all your mobile devices in a centralized onsite location and allow remote access to them from anywhere at anytime

Simple set-up – software only solution running on standard Windows or Mac machines

Enterprise level security, reside within your VPN

High speed performance Full recovery, redundancy and no single point of failure

17

Remote Access to Mobile devices Fully control remote mobile devices from your browser anytime, anywhere

Web access from any browser

Immediate support for all mobile OS , incl. beta versions

Fully integrates to all testing, dev & CI environments

Support all mobile devices

18

Scalability Enable any number of mobile devices and users in any geography

Automation Team

Site A

Mobile device labSite A

Corporate Network

Cloud Server

1 2 15

Device Host 1

(MAC/ Windows)

1 2 15

Device Host 2

1 2 15

Device Host n

1 2 15

Device Host 1

1 2 10

Device Host n

Site B

1 2 15

Device Host 1

1 2 15

Device Host 2

1 2 7

Device Host n

Site N

Auto. engineer

(MAC/ Windows machine) Manual QA Team

Auto. engineer

DeveloperManual Tester

DeveloperManual Tester

Auto. engineer

Manual QA TeamAutomation Team

Site N

Auto. engineer

Auto. engineer

Manual Tester

DeveloperManual Tester

Manual Tester

Dev TeamDev Team

19

Management CapabilitiesEffectively manage large scale mobile projects with advanced management tools

Enhanced user, device and application management capabilitiesUsage reports

and Email alerts

Dashboard for quick overview

20

Manual TestingPerform manual testing on a remote mobile device, as though it was at the tester’s hand

Full control on device power cycle

Support for all manual gestures Fast interaction

Control all physical and virtual device buttons

Visual reports

21

Mobile Test Automation Test all types of mobile applications, and automate advanced mobile application scenarios

Run the same test on different mobile devices & OS’s

Integrate with any testing environment incl. Appium

Test Native, Web and Hybrid, system (setting, SMS) & 3rd party apps

Test barcode & check scanning, audio, GPS simulation, TouchID

Use the test recorder, view device screen and add commands

22

CI/CD Integration Accelerate your testing cycles with a fully automated 24/7 continuous testing environment

Seamless integration into CI environments

Centralized test reporting

Real time view of all executions

Large scale parallel execution (Grid)

Centralized, cloud-based execution, directly managed from the CI

23

Non-Functional Capabilities Test your application under different device conditions and network conditions

Monitor consumption of the mobile device CPU, memory and battery

Simulate different network conditions and on–the-move scenarios that are unique to mobile users

24

Development & DebuggingProvide developers with an advanced testing environment and access to real remote devices

Integrate with XCode, eclipse, Android Studio and more…

Integration with Appium, Calabash, Cucumber and more.

Simulators & emulators native support

Ability to share detailed reportsand live sessions

Support UI/unit testing using XCUITest, Espresso and UIAutomator

Demo CI/CD integration

QA

26