does really iop need to stand for inacceptable operation, pal?

www.comarch.com

Does really IOP need to stand for

Inacceptable Operation, Pal?

Piotr Madej Head of Embedded Solutions and Services

Global is a Largest Polish Software House offering products and services for:

Well diversified in order to ensure business stability and security, products and services

Comprehensive porfolio of Mobile Solutions & Services for Consumer Electronics Industry

Strong presence in Western Europe

Customers on 4 continents in more than 30 countries

3500 employees world-wide

Offices located in Europe, the Americas and APAC

Public Company - listed on the Polish Stock Exchange from 1999

Comarch in a Nutshell

Comarch HQ main building in Kraków, Poland

Comarch in a Nutshell

Segmentation

Common Grounds?

Not only a pair of devices to be interoperable, but very similarly constructed nowadays

„The ability of two or more systems or components to exchange information and to use the information that has been exchanged.”

Definition

Orienting Outside La

un

che

d:

20

01

Lau

nch

ed

: 2

01

1

Thanks to: gsmarena.com

All in one

From Consumer Electronic, through every day life Digital Gadgets, Mobile phones and Home Appliances, all the devices follow the paths of Personal Computers:

they actually become computers

consequently they carry a lot of Software

Convergencies

Matter of Importance

Cost

Quality assurance

Competences and domain knowledge

Interoperability at the level of a platform is about supportings standards Interoperability at the level of devices built on that platform is about simply working together

Focus

Convergences between devices in Consumer Electronics grow far beyond the borders of technologies nowadays, making whole industries influence, penetrate and interfere one another. It has already happened to cellular phones, where the world of PC software architecture and global Internet access blasted each other resulting in a supernova of advanced smartphones we face today. Many tend to believe the next galaxy of possibilities is round the corner due to direct impact of Automotive and IT industries in a big way.

But before that happen, hold on for a while: are we all satisfied with the interoperability of the devices and technologies, starting from the mobile world, connected home or consumer electronics? Are we happy with robustness, reliability and service level between products from various manufacturers? Is the ultimate conformance test, the , always satisfied? In case not, do we really believe the right step is now to blindly copy-paste all this mess to the Next Generation of Automotive? Let's have a look at the interoperability aspects first.

User Experience

Costs

$1 billion

Interoperability Cost Analysis of the U.S. Automotive Supply Chain

Report by NIST

„Solving interoperability problems can significantly reduce costs for the U.S. automotive supply chain. (...) This study estimates that imperfect interoperability imposes at least dollars per year on the members of the U.S. automotive supply chain. The majority of these costs are attributable to the time and resources spent correcting and recreating data files that are not usable by those receiving the files.”

Bluetooth HFP

Bluetooth A2DP

Bluetooth AVRCP

Bluetooth PBAP

Bluetooth HID

Bluetooth MAP

NFC

WiFi

UPnP

USB

Various devices: smartphones mid-range mobile phones accessories portable devices

Focus areas: audio handling and processing data sharing user experience

Management areas: Quality Assurance Error Management IOP Management Reporting Prototype Management

Interoperability – Examplary Approach

Technology Domains

FunctionalityDomains

Cross-field Management

Device Classes

Interoperability Assurance:

A carefully designed communication test bed

A comprehensive simulation environment for communication applications

A service platform or reference devices on which new services and applications can be tested and experienced under realistic conditions

Way to Achieve

Quick Figures

Number of test cases executed per one terminal-accessory pair: 30-100

Number of test cases executed per one IOP round: 2000-5000

Time needed for a single IOP round: 1-2 weeks

Example – Bluetooth

HF device allows multiple devices to be connected to itself at the same time

Multipoint supports a variety of Bluetooth profiles to provide different audio functionality including music playback (A2DP, AVRCP) and call handling (HFP) from multiple devices


17


State machine complexity (11 x 11 call states)

Errors on phone side

User experience issues

State machine complexity vs. UI simplicity

„Active” or „primary” phone definition

Mute synchronization

AG1 Idle Incoming Outgoing Active Active & Waiting Active & On Hold Active & On Hold & Waiting On Hold On Hold & Waiting On Hold & Outgoing On Hold & Outgoing & Waiting Incoming on Hold

Single Point with TWC

Multiple HFP without TWC

Multiple HFP with TWC

Multiple HFP + TWC (more then 3 calls on phone)

AG2 Idle Incoming Outgoing Active Active & Waiting Active & On Hold Active & On Hold & Waiting On Hold On Hold & Waiting On Hold & Outgoing On Hold & Outgoing & Waiting Incoming on Hold

Call states 121 States


One of the fundamental call indication problems is caused by the differences between messages for this operation. In principle call indication should be implemented based on +CIEV, +CLIP, +CCWA, +CHLD and +CLCC codes

Some of the AT commands and result codes are not supported by older HFP spec versions. Therefore, call manager has to support as many as possible messages exchanged between HF and AG and take care of profiles backwards compatibility


Number of test cases in the test bed – 1614

Number of new testcases implemented every week – 2-5

Number of reported errors – over 100 per year

Number of testcases run weekly – over 500

Example

Implementation of USB memory stick support in USB host stack

Functional tests passes with 100% passrate with one QE

Results of IOP

10 QE dvices were chosen for IOP

100% passrate for 6 QE devices

~10 new error found with 4 QE devices, including crashes

Example – USB

Reasons

Implementation is usualy done in „laboratory environment” with very limited number of QE devices for testing

Usually QE is randomly chosen from the market

If specification allows at least two ways of implementation we can be sure that functioality will be developed in *at least* two different ways

Different operating systems handle events/hardware in different way

Some QE devices have some additional, over-standard features which needs extra support (eg. HW encription, backup partition)

List of tipical probles found during IOP testing

External device is not supported

Voice quality is poor in audio devices

External device’s additional fetures causes the tested device’s crash

Example – USB

Parallel testing on multiple devices

Cross application, end-user’s perspective testing

Becoming open-source soon at http://tadek.comarch.com/

TADEK – Test Automation in a Distributed Environment Interoperability Tools

Client

daemon

daemon

daemon

TADEK – Test Automation in a Distributed Environment Interoperability Tools

RAF Gateways

• Physical terminal devices provided by Comarch,

usually WiFi routers supporting USB interface for

Broadband UMTS connection, making RAF

absolutely configuration free

Internet

RAF Server

• Hosted in Comarch Data Center in KRK, Poland

• VPN enabler

• User management and authorisation

• General connection and session management

• Web-based interface for end-users Network #2

• DUT, UPnP network, other

IP based network, wired or

wireless

Network #3 ...

Network #4 ...

Network #1



wireless

Remote Access Framework Interoperability Tools

RAF Gateways

• Physical terminal devices provided by Comarch,

usually WiFi routers supporting USB interface for

Broadband UMTS connection, making RAF

absolutely configuration free

Internet

RAF Server

• Hosted in Comarch Data Center in KRK, Poland

• VPN enabler

• User management and authorisation

• General connection and session management

• Web-based interface for end-users Network #2



wireless

Network #3 ...

Network #4 ...

Environemnt #1

• DUT or Bluetooth

environment

Remote Access Framework Interoperability Tools

Audio link, mic (cable)

Audio link, speaker (cable)

Bluetooth link (air)

USB connection (cable)

GSM connection (OTA)

Data link (ribbon)

=

ATAB

Master Phone

Managing PC

Slave Phone

DUT Audio Device (proto-board or RnD build)

QUIET BOX

Audibility Testing Automation Board Interoperability Tools

USB

Connection

Control connection

over USB

DUT

ATS USB Cable Manager Interoperability Tools

Inte

rop

era

bili

ty M

anag

em

en

t To

ol

Interoperability Management Tool (IOPMAN)

www.comarch.com

Thank You!

Piotr Madej Head of Embedded Solutions & Services, MSS BU [email protected] + 48 691 464 483

does really iop need to stand for inacceptable operation, pal?

Documents