@agatestudio

Windows Phone Quality Bar

Tedy

Quality Assurance

Agate Studio

Windows Phone

Quality Bar

Why do we need a

quality bar?

The existing Store certification is designed to provide a minimum quality measure and set of policy requirements that applications must fulfil to be listed in the Windows Phone Store.

Why do we need a

quality bar?

The Depth Partner Program carefully chooses partners to build Showcase applications that will delight customers with the highest standards of design and application quality. This document defines the high quality standards expected of these applications and is designed to encourage development of usable, sticky and innovative applications.

The Bar Defined

The bar pulls in quality requirements from several areas, such as user experience, functionality, design principles, and performance. Each application must pass all requirements. At the top level the requirements are defined as follows:

Design review feedback incorporated

o No MUST FIX items from design review are present

Quality review

o No MUST FIX bugs

o No more than 4 SHOULD FIX bugs

Look and Feel

The Windows Phone design philosophy is

exemplified by clean, uncluttered app screens that

operate quickly, minimize typing and surface new

info automatically. Interactions happen directly with

content and visual elements have great fit and

finish. The look and feel of the app should integrate

seamlessly with the design style of Windows Phone

Platform.

Look and Feel

Standard WP icons from the SDK are repurposed for another

meaning.

Content is not visible or reduced in visual appeal under light or dark

themes.

Page headers use inconsistent text case across the app.

App does not use a custom splash screen (default is used).

App does not use a custom app tile (default is used).

MUST FIX

Look and Feel

Page headers on secondary level pages are unnecessarily large.

Page headers do not fit the current context. Use a suitable page title,

instead of simply repeating the app name.

Icons that are not appropriate for the context or do not clearly indicate

intended purpose.

App bar icon graphics have an extra circular border making the circle

look thicker or misaligned.

SHOULD FIX

Look and Feel

Consider introducing brand images or iconography into the app‟s

background (if suitable).

Ensure that the system tray colour and transparency fits the branding

of the app

RECOMMENDATION

Layout

All actionable controls in the app should be designed for “fat

finger” and should have touch targets and padding such that

these are easy to tap accurately. Margins should be

consistent, and content should be aligned to a grid. Care

should be taken to avoid conflicting UI elements i.e.

horizontally actioned items displayed in horizontally scrolling

controls.

Layout

Layout

Inconsistent page margins used across the app (left, right and top).

Page content not left-aligned with header.

Touch gestures overlap: horizontally scrolling maps, horizontal scroll

viewers, toggle switches, etc. are placed within a pivot or panorama

(horizontally scrolling controls).

Touch target sizes are smaller than 9mm (34px).

Padding between touch targets is less than 2mm (8px), or touch

targets overlap.

MUST FIX

Layout

App shows empty pages when there is no content to display, e.g. if

there are no favourites added. Display placeholder text to explain what

will be shown here.

App shows blank page if no search results are returned. App should

show a clear „no results found‟ message on the page.

SHOULD FIX

App tiles

A Tile allows the app to present rich and engaging content to users on

the Windows Phone Start screen when the app isn‟t running. Tiles can

be live (updated through notifications), or left static. Tiles on the Start

screen come in three sizes—small, medium, and wide (wide being

optional).

It is good practice to avoid pinning content which is static or likely to

expire, such as individual news stories. Instead, allow pinning the news

category with a live tile to highlight the latest category story. To allow the

user to access individual pieces of content, consider whether an in-app

„favourites‟ feature would be more appropriate.

App tiles

App tiles

The tile content not consistent with the app, or is not of a sufficient

visual standard (in all sizes).

Wide tile does not display live content.

Duplicated app name on tile, text and logo.

Secondary tile takes the user to the app‟s main page.

App tiles use rounded corners, drop shadows and shine effects

(certain exceptions can be considered for branding justifications).

MUST FIX

App tiles

Tile without the app name if name is not integrated with the logo

The „pin‟ feature is not disabled or toggled to „unpin‟, when a page has

already been pinned.

SHOULD FIX

Avoid using full black (#000000) or full white (#FFFFFF) tile

backgrounds.

RECOMMENDATION

Navigation

The app navigation structure should be simple and shallow, the user

should be able to get to the content they want in 5 taps or less. Pressing

the back button should be a predictable experience and take the user

back to the previously visited page. Tapping the back button after

launching from an app tile must exit the app, even if this is a secondary

tile. Avoid navigating the user back through pages they have not visited.

Home, back or close buttons are not required in Windows Phone apps

as a physical back button is provided on every device allowing the app

design to remain clean and simple.

Navigation

Some special exceptions can be made when the app is navigating the

user through a „wizard‟ process, for example account registration or

completing a shopping checkout. The best practice is to allow the user to

back through the process stages or use a „cancel‟ button to quit the

process and return to the start. When the user has completed the

process, the app can provide a quick means to return home (e.g. via a

„done‟ button).

Navigation

App contains back or close buttons.

Back button does not close the app if tapped on the main page (even if

navigated to from secondary tile).

Back button does not close any transient UI such as keyboards, message

boxes, list pickers, app bar menu and context menus.

Back button does not cancel loading processes.

Pressing the hardware Back button after launching application from secondary

tile takes the user to main panorama or home page, rather than exiting the app.

Home buttons are used in the app. (special exceptions can be made for apps

with a valid business justification. Consider if a home button is truly required.)

App uses buttons or the hardware back button to sideways navigate across a

pivot or panorama.

MUST FIX

Navigation

Avoid using app exit confirmation message as these can become a

little irritating to the user. Alternatively, consider providing an in-app

setting to enable/disable this exit message to let the user choose.

RECOMMENDATION

Dialog and Message boxes

A message box is a prefabricated modal dialog box that displays a text

message to a user. Message boxes interrupt interaction with the app and

as such should only be used when critical information needs to be

communicated to the user (e.g. the app is offline), or the user‟s input is

required to continue using the app (e.g. confirm use of user‟s location).

Since these require an extra tap from the user, consider carefully if the

message box is really required or whether the information can be

presented in a less intrusive way.

Dialog and Message boxes

The user is unable to dismiss any popup message or dialog box by

pressing the back button.

Message box is not model (user is able to interact with the app

behind).

Message dialogues go beyond communicating simple messages or

questions. More complex content should be on a full page.

Message box do not follow the standard Window Phone message box

style. Single buttons should be left aligned, and multiple buttons should

be centre aligned and extended to the sides of the dialogue.

MUST FIX

Dialog and Message boxes

Language and Localisation

All content must be localised correctly to the languages that the app

claims to support. The language used throughout the app should be

grammatically correct, non-generic, and user friendly. The app should

also respect the user‟s regional settings as some data, such as dates

and numbers, can be misleading if presented in a format the user does

not expect.

Language and Localisation

Regional Settings are not respected, e.g. time and date formatting.

App is not correctly localized for the languages it supports.

Error messages are not contextually-relevant, or contain stack traces

or debugging information.

Application dialogs and other help items do not correctly reference the

platform and app type.

Spelling or grammar mistakes are present.

MUST FIX

Performance and Stability

Speed and responsiveness in apps have a high impact on

user's perception of the quality of the app. UI animations and

transitions between pages must be smooth and fluid.

Performance and Stability

The app fails to render the first page within 5 seconds after launch (does not

include the splash screen).

The app fails to respond to user input after 15 seconds.

Transitions are not smooth and fluid.

Progress indicators are not used to indicate when the app is loading content.

Crashing bugs are found.

App does not respond gracefully when its background agents are disabled.

Background tasks include auto-sync and download functionality, e.g. email and

update.

MUST FIX

App does not handle Battery Saver mode gracefully.

SHOULD FIX

Games

Game apps should follow good game UX design guidelines.

Please ensure the app makes full use of the graphics layout

capabilities and gesture support for an experience that

makes interaction easy.

Games

The user is unable to dismiss any popup message or dialog box by pressing

the back button.

When the Back button is pressed during gameplay, the game does not pause

or navigate the user to the prior menu screen.

When the Back button is pressed outside of gameplay, the user is not returned

to the previous menu or page.

Pressing Back must does not exit the game on the initial start screen / menu.

The game overrides the user's current music choice by turning off or changing

music that was playing before the game was launched.

If an application plays background music, the application does not provide the

user with configurable settings for both the background music, and the

background music volume.

Game does not pause or remember game state when an incoming call is

received.

MUST FIX

Games

On-screen home and back buttons used. Apps should rely on

hardware back button unless absolutely necessary.

SHOULD FIX

Automatically save the game state while the pause menu is shown, in

case the player exits the game (if suitable for the game type).

RECOMMENDATION

“Anyone Can Find a Crash Bug . . .”

David Price

(Test Lead, THQ, Inc.)

Will I play this game almost every day, for the next million

months, with a smile on my face, sharp eyes and ears,

faithfully reporting every bug I see?!

If you answered “yes,”

you just might make it as a professional video game tester.