GALINA MAKAGONOVA

GADGET ON THE BUDGET

How to budget, create and operate a Device Test Lab

Average size of mobile development team is 20-70 people

GAMES MOBILE APPS MOBILE WEB

Our Agenda1. WORKSHOP

Device lab requirements

Device parameters

Planning: location, equipment, software, storage

Budget

Device lab technician: role description and responsibilities

Where to buy devices and overview of issues

Lab Maintenance

Device usage in Agile development team

How to present device lab during negotiations with the clients

2. PRACTICE and Q&A

OPTIONS TO EXPLORE

OR NO DEVICE LAB?

NO DEVICE LAB

Developers and testers are using their own devices

Application delivered to client without testing

Or Using device emulatorOr BYOD policy

External Test LabTYPES:1. Device Rental Labs2. Crowd solution (using private devices across the globe)3. Free device lab

PRO: no need to invest in device acquisition

CONS: 4. Cost:$15-25 per device per usage, $25-$30 per hour device+ tester5. Confidentiality: your app might be exposed to the market 6. Timeline: Sending out your testers at the end of test cycle7. No flexibility in OS update/Version8. Your target devices may not be available9. Booking in advance10. No way to reproduce bug on device11. Developers cannot investigate the bugs and validate features

Overview: Possible roadblocks

Do You Need Internal Test Lab1. What kind of mobile application are you developing?

Mobile Web? Games? Mobile apps?

2. How many projects do you have at the same time?

3. How many developers are in the company?

4. Are development and QA co-located?

5. Do you have a test department? How many testers?

6. Are you developing the app for Android/iOS market? Or for your clients?

7. Do you have contract obligations to test the app on mobile devices?

8. Agile or Waterfall?

9. When is Device test lab usage required? At the end of development cycle or for everyday usage?

Overview: various roadblocks

Business Focus1) Who are the users?

2) Who will pay (how to negotiate with the clients or get management

buy-in)?

3) How many devices do you need or could you afford?

4) Device selection/Device matrix

5) Device fragmentation (specifically, Android fragmentation)

6) Managing device inventory

7) What is the budget?

8) Device selection should be relevant for a year, preferably for 2

9) Return of investment: what is the value of your lab?

Who Are The Internal Users Of A Mobile Test Lab?

1. Development teams: How many?

2. Developers – how many?

3. Testers

4. Designers

5. Artists

6. Customer support / Publishing

7. Sales

8. User acceptance coordinators

Who: Target audience

Where to get info: from GD, Sales, Data analytics, GOOGLE

1. Market: countries

2. How secure the application is? Do you need to test hackers attacks?

3. Age and gender of the users

Overview: various roadblocks

And: Technical requirementsWhere to get info: from GD, Lead developers, game teams

1. IOS and Android? Android only? iOS only?

2. Any specific hardware requirements (i.e. gyroscope)

3. Screen requirements

4. Memory requirements

5. Graphic processor requirements

6. Other specific requirements

7. Phones and tablets? Or just phones? Or only tablets?

8. Pre-installed applications

9. Application to download

Overview: various roadblocks

Device Parameters:

• CPU• GPU• RESOLUTION• CHIPSET• SENSORS• YEAR RELEASED• PRICE• MARKET RELEASED

Device Lab PlanningCentralized or Distributed?

Static lab: Round table for testing (devices are co-located, anyone who want to test should come there.

Agile world: Devices are assigned per team and per tester, with visibility for the whole team: anyone who needs it could borrow for 1 hour or so.

• Location• Storage Space• Tracking tool: I’m using Meraki• Cables, cleaning, power supply• Inventory • Bookkeeping/Accounting• Internal Policy• Disasters happen

Overview: Common mistakes

Budget

• List of devices

• Cost of devices

• Cost of tools and supplies

• Cost of device lab (storage, etc)

• Cost approval process

• Lab technician (salary)

• Who will pay

Overview: typical mistakes

More About Device Lab Budget• Per project

• Per tester

• Track device life

• Maintenance materials

• Plastic gloves

• Cables

• Power supply

• Charging stations

• Tape for marking cables and devices

• SIM cards

• Budget for testing mobile data

Device Lab TechnicianKeep inventory changes

Track lost devices

Plan and manage OS updates

Root devices if needed

Sideload client build

Charge all devices

Gather devices spread across office

Create device test list per app

Perform device testing per app

Perform compliance testing

What To Buy And How Many

List devices to cover the technical spec (low end, high end, phones and tablets)

How many teams do you have?

How many developers per team? Total?

How many testers per team? Total?

Each QA should have 1 Android and 1 IOS gadget (1 phone, 1 tablet)

Each team should have 1 high end, 1 low end (both iOS and Android)

Who managed provisioning profile? How many devices are already in?

Some samples of requests to address

Buying devices*** No contract

*** New or used

*** Beware of jailbroken devices

• Amazon

• E-bay

• Classifieds

• Cell phone providers

• Buying old devices from staff (how much do you want to pay)

Lab Maintenance• Inventory (locate each gadget bi-monthly)

• Mark each device, cable and charger

• Cable replacement

• Monitor chargers

• Buy sleeves

• Monitor upcoming OS changes

• Monitor new devices on the market

• Manage device assignments to team members

Negotiation

Presenting options with cost and risk

Cost of device rent per test cycle, availability

Outsource – cost per cycle

Developers and QA need devices during development cycle

In-house, including

Cost of device tester

Quantity of test cycles

List of devices per app requirements

*** Who will own devices at the end of development

*** Add 1/6 of device budget every 6 months to keep current

Workshop

• List of 30 Devices with larger market share (samples)

• Operation system diversity

• Practical exercise: reviewing application requirements and

selecting 10 top devices

• Practical exercise: Working on device lab budget

• Practical exercise: Managing device test lab

• Practical exercise: Device compatibility checklist

Galina Makagonova

Gadget on the budget

Thank you very much for attending!

See you soon for the hands-on Training.

Galina Makagonova

QA Director of Roadhouse Interactions,

Vancouver, Canada