test cases for windows...

Test Cases for Windows Applications Windows 10 Universal Windows Platform – Desktop and Mobile device families

Windows 8.1 Windows and Windows Phone

April 2016

Contents Introduction ........................................................................................................................................................................ 3

1. Launch-related Test Cases ......................................................................................................................................... 3

1.1 Low spec devices .................................................................................................................................................... 3

1.2 Launch on all required device families ............................................................................................................. 4

1.3 Multi-regional launch ............................................................................................................................................ 4

1.4 Offline launch .......................................................................................................................................................... 5

1.5 Launch time ............................................................................................................................................................. 5

2. Log In Test Cases .......................................................................................................................................................... 5

2.1 Offline log in ............................................................................................................................................................ 6

2.2 Invalid credentials .................................................................................................................................................. 6

2.3 All log in methods are functional ...................................................................................................................... 6

2.4 Switching away after logging in ........................................................................................................................ 7

2.5 Account creation ................................................................................................................................................... 7

2.6 Account switching ................................................................................................................................................. 7

3. Application Use Test Cases ........................................................................................................................................ 8

3.1 App Switching ......................................................................................................................................................... 8

3.2 Software Input Pad implementation ................................................................................................................ 9

3.3 Touch targets .......................................................................................................................................................... 9

3.4 Visual feedback ...................................................................................................................................................... 9

3.5 Gestures ................................................................................................................................................................. 10

3.6 Secondary tiles ...................................................................................................................................................... 11

3.7 Command bar with contextual commands ................................................................................................... 11

3.8 Application is stable ............................................................................................................................................ 12

3.9 Application performance is responsive ......................................................................................................... 12

3.10 Toast notifications are actionable ................................................................................................................. 13

3.11 Offline mid-use scenarios................................................................................................................................. 13

3.12 Saved status preservation ................................................................................................................................ 14

3.13 Portrait and landscape mode scenarios ...................................................................................................... 14

3.14 Core scenarios completed within the application .................................................................................... 15

3.15 In-App purchases............................................................................................................................................... 15

3.16 Geofencing .......................................................................................................................................................... 16

3.17 Cross-Platform adaptive layout ..................................................................................................................... 16

3.18 Application is feature complete ..................................................................................................................... 17

3.19 App Resume ........................................................................................................................................................ 17

3.20 Readability ........................................................................................................................................................... 18

3.21 Placeholder text and imagery ........................................................................................................................ 18

3.22 Application specific hardware ........................................................................................................................ 19

3.23 Cortana ................................................................................................................................................................ 19

3.24 Resuming after idling ....................................................................................................................................... 20

4. Phone-Only Test Cases ............................................................................................................................................ 20

4.1 Application resumes successfully from tombstoned state ....................................................................... 20

4.2 Back button for mobile ...................................................................................................................................... 21

4.3 Continuum for mobile ........................................................................................................................................ 21

5. Windows-Only Test Cases ....................................................................................................................................... 22

5.1 Split and windowed views and display resolutions ..................................................................................... 22

5.2 Application resumes from a suspended state ............................................................................................. 23

5.3 Back button for desktop ....................................................................................................................................24

5.4 Desktop and tablet mode switching ..............................................................................................................24

5.5 Multiple input methods ..................................................................................................................................... 25

5.6 Forward and backward navigation ................................................................................................................. 25

6. Social Networking Integration Test Cases ........................................................................................................... 26

6.1 Shared content ..................................................................................................................................................... 26

6.2 Share targets ......................................................................................................................................................... 26

7. Game Specific Test Cases ......................................................................................................................................... 27

7.1 Game leaderboard ............................................................................................................................................... 27

7.2 Game achievements and scores ..................................................................................................................... 27

7.3 Game multiplayer support ................................................................................................................................ 28

7.4 Roaming Game Saves ........................................................................................................................................ 28

8. Media Specific Test Cases ........................................................................................................................................ 29

8.1 Background audio playback .............................................................................................................................. 29

8.2 Background audio controls .............................................................................................................................. 29

8.3 Video pauses playback when the application is in the background ..................................................... 29

9. .......................................................................................................................................................................................... 30

10. WACK Test Findings ................................................................................................................................................. 30

11. Competitive Parity Test Cases ................................................................................................................................ 30

11.1 Feature parity – Competitive platforms ........................................................................................................ 30

11.2 Feature parity - Website ................................................................................................................................... 31

11.3 Performance parity............................................................................................................................................. 31

Introduction

Application functionality is the foundation of user satisfaction. The tests included here are designed to

test the most fundamental functional scenarios in Universal Windows Platform (Mobile and Desktop

device families) Store applications and games. These tests make no assumptions regarding an

application’s user interface and verify that an application’s behavior is consistent with user

expectations and the Windows design guidelines. The reader is encouraged to incorporate these test

cases into their software development projects to help ensure high user satisfaction – and high user

ratings.

1. Launch-related Test Cases

1.1 Low spec devices

The application should install and function properly on low spec devices.

Expected result:

The application should function properly on low spec devices.

Note: A ‘Low spec’ phone is defined as a 512MB Windows Phone 8.1 device, ex: Lumia 520, and/or a

1GB Windows 10 Mobile device, ex: Lumia 535.

1.2 Launch on all required device families

The application should install and launch properly on all device families the app is designed to run on.

With Windows 10 developers can write a single application that can be installed across a variety of

device families including Mobile, Desktop, and XBOX. By default, Windows 10 applications target all

device families. In the application manifest, this is called out as the ‘Universal’ Target Device Family.

Such applications will be tested against all device form factors where this application may be

deployed. Alternatively, developers can limit the device families for which an application can be

deployed. For example, if ‘Desktop’ is specified as the Target Device Family, that application can only

be installed on PC devices. When published, this application will only be available in the Store running

on PCs – it will not be available in the Phone store.

Steps to Reproduce:

1. Check the supported device families in the application’s manifest.

2. Attempt to install and launch the application on all required device families.

Expected result:

The manifest entry should be restricted only to supported devices when only one device is

supported.

More information:

For more information see DeviceTargetFamily.

Target Device Family Hardware

Desktop PCs and Tablets running Desktop OS SKU.

Mobile Phone and Tablets running Mobile OS SKU.

Team Surface Hub running Team OS SKU.

Universal All

1.3 Multi-regional launch

The application should launch properly on devices where the local username contains Unicode

characters.

Steps to Reproduce:

https://msdn.microsoft.com/en-us/library/windows/apps/dn986903.aspx

1. Launch the application on a device where the user local folder path has Unicode characters,

ex: c:\users\캔디\.

Expected result:

The application launches properly on devices where the local username contains Unicode characters.

1.4 Offline launch

The application should be stable and not crash or close unexpectedly if the device is offline. When the

application is offline, the application should warn the user for any requested action that cannot be

completed.

Steps to Reproduce:

1. Turn airplane mode on and wait 10 seconds.

2. Launch the application.

3. Perform an action that will cause a connection to be attempted.

Expected result:

The application will detect that the network is not available and alert the user.

Application should not crash and provide a relevant error message to user. Application should

not display the exception stack to the user.

The application can also implement an offline (cached) mode without alerting the user.

1.5 Launch time

The application should launch in a reasonable amount of time, and should display a loading or

progress indicator if the application takes a while to load.

Steps to Reproduce:


2. Record the start-up time.

Expected result:

The application displays a loading or progress indicator if the application takes a while to load,

and the content loads in a reasonable amount of time.

2. Log In Test Cases

The following test cases apply only to applications that support log in.

2.1 Offline log in

When the application is offline and the user attempts to log in, the application should warn the user

of the offline status.

Steps to Reproduce:



3. Attempt to log in with valid credentials.

4. Verify that an appropriate error message appears.

5. Turn airplane mode off and wait 10 seconds.

6. Return to the application.

7. Log in with valid credentials.

Expected result:

When attempting to log in without a network connection, an appropriate error message

should appear warning the user of the offline state.

When the network connection has been resumed, log in should work appropriately.

2.2 Invalid credentials

The application should warn the user appropriately when invalid credentials are entered.

Steps to Reproduce:


2. Attempt to log in with invalid credentials.

3. Verify that an appropriate error message appears.

4. Log in with valid credentials, changing the casing in the username field.

Expected result:

When attempting to log in with invalid credentials, an appropriate error message should

appear warning the user that the credentials are invalid.

The username field should not be case-sensitive.

When correct credentials are entered after a failed attempt, log in should work appropriately.

2.3 All log in methods are functional

All available log in options should be fully functional and appropriately implemented.

Steps to Reproduce:


2. Log in using all available options with valid credentials.

Expected result:

Application logs in when valid credentials are entered.

Social networking log in and/or sign up is processed using the services’ login page(s) and not

using application pages for this purpose.

2.4 Switching away after logging in

The application should load content successfully after switching away during the loading process.

Steps to Reproduce:


2. Log in with valid credentials.

3. While the application is loading, switch away from the application.

4. Wait one second and return to the application.

Expected result:

Content should load successfully after returning to the application.

2.5 Account creation

The creation of accounts within the application (when available) should function properly.

Steps to Reproduce:


2. Create an account if the option is available, entering valid information when prompted.

3. Exit the application.

4. Attempt to log in with the newly created account.

Expected result:

Accounts can be successfully created within the application when the option is available.

If an email verification is required, the application should state this and the verification email

should arrive within an hour of creating the account.

Username, email, and password requirements are made clear to the user.

2.6 Account switching

Users should be able to log into various accounts properly when the option is available.

Steps to Reproduce:

1. Launch the application and log into account #1.

2. Navigate through the application, saving information to the account whenever possible.

3. Log out and exit the application.

4. Launch the application and log into account #2.

5. Navigate through the application.

Expected result:

Logging into a second account should function properly after logging out of a first account.

Data from a first account should not appear when logged into a second account.

3. Application Use Test Cases

3.1 App Switching

When switching away from the application, network requests are cancelled. The application should

detect this situation and re-issue network requests when application is reactivated.

Steps to Reproduce:

3. Launch application.


5. During loading (when loading indicator is present), switch away from the application.

6. Wait one second and return to the application.

Expected result:

Functionality completes successfully and data is downloaded without error or exception.

More information:

For more information on Fast App Switching in Windows Phone applications, see Get Ready for Fast

Application Switching in Windows Phone and App activation and deactivation for Windows Phone 8

For Windows applications, see Managing app lifecycle so your apps feel "always alive"

http://channel9.msdn.com/events/MIX/MIX11/DVC09

http://channel9.msdn.com/events/MIX/MIX11/DVC09

https://msdn.microsoft.com/library/windows/apps/ff817008(v=vs.105).aspx

http://blogs.msdn.com/b/windowsappdev/archive/2012/04/30/10292044.aspx

3.2 Software Input Pad implementation

The Software Input Pad (SIP) should be contextually-aware of the types of input allowed per input

field and it needs to be dismissed when input field is not selected on an application or when the

application is deactivated.

Steps to Reproduce:

1. Switch the device to Tablet mode and ensure a keyboard is not connected.


3. Find an input textbox that allows alphanumeric characters.

4. Verify when this control has focus (is tapped with touch), the full keyboard and symbols are

displayed.

5. Find an input textbox that allows only numbers.

6. Verify when the control has focus (is tapped with touch), only the keypad SIP is displayed.

7. Move focus out of the input textbox (tap outside of the textbox with touch).

Expected result:

The SIP is dismissed when input controls lose focus.

Users can see what they are typing while they are typing it.

3.3 Touch targets

Touch targets should not be smaller than 7 mm or 26 pixels square.

Steps to Reproduce:



3. Identify actionable UI elements on the application.

Expected result:

Actionable UI elements in the application should be clearly visible and easily touchable by finger

without interfering with other neighboring touchable UI elements.

More information:

For more information on the Windows touch language, see Touch interaction design

3.4 Visual feedback

Visual feedback helps users recognize whether their interactions with your application are detected,

interpreted, and handled as intended.

http://msdn.microsoft.com/en-us/library/windows/apps/hh465415

Steps to Reproduce:


2. Explore and navigate the application. Tap on each actionable UI element.

Expected result:

There is visual feedback for actionable or interactive UI elements.

There is no visual feedback for non-actionable or non-interactive UI elements.

Note: Acceptable visual feedback can be a slight change in color and/or movement of the

button when selected by touch or mouse (not when hovering over with a mouse.)

More information:

For more information on visual feedback, see Guidelines for visual feedback

Touch interaction guidance: Touch interactions for Windows

Responding to user interactions: Responding to user interaction

3.5 Gestures

Primary actions should be enabled by directly interacting with content. For example, tap to open,

swipe to select, pinch to zoom, and drag to pan/paginate. If a panorama or pivot control contains

other controls which use the same left / right flick gestures, this may result in a poor user experience

due to gesture overlaps. UI elements, controls or other parts of an application should not interfere

with Windows edge gesture.

Steps to Reproduce:


2. Explore and navigate the pages and functionality of application with touch only.

3. Inspect the application’s panorama/pivot control for scrollable map, horizontal scroll viewer,

toggle switches, sliders, or other controls that use left / right flick (or pan) gestures.

4. Navigate to each page and test the edge gestures for commanding surfaces (Title bar, Task

bar, Action center, Task View). (For Windows 8 apps: app bar, navigation bar, charms bar,

recent app.)

Expected results:

The application should fully support touch input. For example, the user should be able to

navigate or interact with commands using touch input only.

Touch gestures should only be used for actions that match the Windows touch language.

Pivot and/or panorama controls do not have controls with competing gestures and can be

used reliably.

https://msdn.microsoft.com/en-us/library/windows/apps/hh465342.aspx

http://msdn.microsoft.com/en-us/library/windows/apps/hh465415.aspx


If the hub, panorama or pivot reacts by scrolling then the child control should not trigger.

Any UI elements, controls, or other parts of the application should not interfere with Windows

edge gestures.

More Information:

Gesture interaction guidance: Gestures, manipulations, and interactions

3.6 Secondary tiles

Secondary tiles should enable users to pin context specific content on the Start screen.

Steps to Reproduce:


2. Explore the application and identify functionality (if any) where the user has the option to ‘Pin

to Start’.

3. Select ‘Pin to Start’ to create a secondary tile on the Start screen.

4. Launch the application by tapping the primary tile of the application on the Start screen or

tapping the application name from the application list page.

5. Navigate to a second or third level page in the application.

6. Tap the Start button to return to the Start screen.

7. On the Start screen tap the secondary tile of the application.


9. Launch the application from the secondary tile.

Expected result:

Launching the application from the secondary tile results in displaying content related to the

secondary tile.

On pressing the Back button (if applicable), user exits the application or navigates to the main

page.

More information:

For more information on tiles refer below link: Secondary tiles overview (Windows Runtime apps)

3.7 Command bar with contextual commands

Command bar commands need to be functional as well as predictable and organized so users can

easily find them.

https://msdn.microsoft.com/en-us/library/windows/apps/xaml/JJ883700(v=win.10).aspx


Steps to Reproduce:


2. Explore and navigate the pages and functionality of application.

3. Select items with contextual commands.

Expected result:

All command bar buttons are functional, relevant, and contextual.

3.8 Application is stable

The application should be stable and not crash or hang. The application should not have functional

bugs that prevent completion of core scenarios.

Steps to Reproduce:



Expected result:

Application should not crash or hang.

3.9 Application performance is responsive

The application must appear responsive to the user at all times. Visual feedback is needed to indicate

the application is not frozen. Load times may be longer on slow connections resulting in the

application appearing to be frozen if no visual feedback is provided.

Steps to Reproduce:



3. Invoke functionality that requires time to load. E.g. network request to load data.

Expected result:

The application displays a loading or progress indicator for tasks that may take time to load,

and the content loads in a reasonable amount of time.

When the content has fully loaded, the loading indicator should no longer be present.

Application should be responsive and fluid.

Transition animations should be smooth.

The application should not display a blank screen, blank tiles, or empty white rectangles.

More information:

Guidelines for progress controls: Guidelines for progress controls

3.10 Toast notifications are actionable

Notifications should be used for time-sensitive or personally relevant notifications. Notifications are an

invite back into the application when it is not in the foreground. Tapping the notification should bring

the application to the foreground and in context to the notification.

Steps to Reproduce:


2. Invoke functionality to create a toast or notification. E.g. push notification or scheduled

reminder.

3. Switch away from the application and/or close the application.

4. Wait for the notification.

5. Select the notification.

Expected result:

When selecting the toast or notification, the application should be brought to the foreground

in the context of the toast or notification.

Toast notifications should not be displayed when the application is in the foreground.

More information:

For more information see Guidelines for toast notifications and Guidelines for toasts and badges

3.11 Offline mid-use scenarios

When the system is in an offline scenario (no cell coverage, no WiFi, airplane mode, etc.), the

application should alert the user appropriately.

Steps to Reproduce:





5. Perform an action that will cause a connection to be attempted.

6. Verify that the application notifies the user of the offline state.

7. Turn airplane mode off and wait 10 seconds.



http://msdn.microsoft.com/en-US/library/windows/apps/hh465403


Expected result:

The application will detect that the network is not available and alert the user.

Application should not crash and provide a relevant error message to user. Application should

not display the exception stack to the user.

The application can also implement an offline (cached) mode without alerting the user.

When resuming to the application in an online state after being previously using in an offline

state, the application should detect that the connection has been restored and work properly.

More information:

For more information, see Offline Sync for Mobile Services and Build offline apps with Azure Mobile

Services

3.12 Saved status preservation

Application progress and saved states should be maintained after navigating away, exiting, and

synching on another device.

Steps to Reproduce:

1. Launch the application and log in if available.

2. Explore and navigate the pages and functionality of application, saving whenever possible (for

games, clear 2~3 levels/stages/checkpoints).


4. Launch the application (if applicable, launch on a second device and log into the same

account as in step 1).

Expected result:

The application should retain saved states.

The application should retain saved information across devices when logged into the same

account.

3.13 Portrait and landscape mode scenarios

The application should adapt properly to portrait or landscape view if supported.

Steps to Reproduce:

1. Rotate the device to portrait or landscape view.


http://azure.microsoft.com/blog/2014/07/30/offline-sync-for-mobile-services/

http://channel9.msdn.com/Series/Windows-Azure-Mobile-Services/Build-offline-apps-Azure-Mobile-Services

http://channel9.msdn.com/Series/Windows-Azure-Mobile-Services/Build-offline-apps-Azure-Mobile-Services

3. If the application supports both portrait and landscape view (layout changes), navigate to

each page in the application. Rotate the device between all supported orientations at each

page.

Expected result:

When launched, the application remains in the same orientation as the device (if the

application supports multiple orientations).

All application pages render properly and adapt to the views supported. Ex: no text

truncation, all buttons are accessible, etc.

Applications should not display a logo or error message when the view is changed.

Applications should not launch in portrait mode on a Desktop device unless the application is

launched in portrait mode.

3.14 Core scenarios completed within the application

All core application scenarios are completed without errors.

Steps to Reproduce:


2. Navigate through the entire application.

Expected result:

All core scenarios are accessible from within the application, not from an external web

browser.

Note: It is acceptable for a privacy policy or a login page that redirects back to the application

to open in a web browser.

Note: It is acceptable for the application to contain an in-app web browser.

3.15 In-App purchases

In-App purchases should display the corresponding Store UI.

Steps to Reproduce:


2. Locate an in-app purchase.

3. Cancel the purchase.

Expected result:

The item for sale in the application should correspond to the one found in the Store UI.

The Store UI should take no more than 5 seconds to display.

Cancelling a purchase should not charge the user or reward them with the item.

After June 29, 2015, third party solutions for selling digital content consumed in the

application are no longer permitted. Microsoft IAP must be used to sell digital items that are

consumed or used within the application. The use of an external web browser or in-app

webview is not acceptable.

3.16 Geofencing

When geofenced sections of the application are accessed outside of the targeted area, the

application should display an appropriate error message.

Steps to Reproduce:


2. Locate or create the application scenario dependent on geofencing.

3. Perform an action that invokes the geofenced feature.

Expected result:

There should be an indication that the application is being used outside of the required region.

3.17 Cross-Platform adaptive layout

The application layout should adapt and display properly on all supported devices.

Steps to Reproduce:

1. Launch the application on all supported devices.

2. Navigate through the application and inspect the layout.

Expected result:

All application pages render properly and adapt across all supported devices. Ex: no text truncation,

all application buttons are accessible, no large blank areas, etc.

More information:

For more information see Introduction to Universal Windows Platform (UWP) applications for

designers and TargetDeviceFamily (Windows 10 Insider Preview).




3.18 Application is feature complete

The controls within the application should invoke the expected functionality. The application should

not display messages indicating features are ‘coming soon’ or do nothing in response to control

actions.

Steps to Reproduce:


2. Navigate through the application, interacting with buttons and tiles throughout.

Expected result:

The controls within the application should work as expected and should not present “coming soon”

messages.

3.19 App Resume

App Resume allows the previous suspended instance of the application to be resumed when user re-

launches the application from the Start screen.

Steps to Reproduce:

1. Launch the application by tapping the primary tile of the application on the Start screen or

tapping the application name from the application list page.

2. Navigate to a second or third level page in the application.

3. Tap the Start button to return to the Start screen.

4. On Start screen, tap the primary Tile again to resume the application.

Expected result:

The application resumes to the same page as in step 2 and pressing the back button takes the user

through the correct sequence of pages.

Or

The application resumes at the main page (without the splash screen and loading indicator) and

pressing the Back key exits the application. Pressing the Back key should NOT navigate to another

page.

More information:

For more information see: App lifecycle - Windows app development


3.20 Readability

All content in the application should be easily readable and adapt to a phone set to both light and

dark themes.

Steps to Reproduce:

1. If testing on a phone, set the device to the dark theme.


3. Navigate through the application, if an in-app light/dark theme is available switch between

the two.

4. If testing on a phone, set the device to the light theme.


Expected result:

All content in the application displays properly (ex: no light text against white backgrounds, etc.)

3.21 Placeholder text and imagery

The application should not display placeholder text or imagery.

Steps to Reproduce:

1. Locate the application in the device’s list of applications.

2. Pin the application to the Start screen.

3. Adjust the tile size.



Expected result:

• Tiles should be unique to the application and not display the default ⊠, a Unity logo, or be

blank.

• Splash screen should reflect the brand of the application. The splash screen should not

display the default icon ⊠ or be blank.

• The application should not display developer or placeholder text.

• Messaging and/or imagery in the application should not be from a different version of the

application (ex: Mobile imagery or tutorial instructions on a Desktop application,

iOS/Android imagery or text, etc.)

3.22 Application specific hardware

The application should function properly with all supported hardware.

Steps to Reproduce:

1. Connect supported hardware (ex: Bluetooth earphones, fitness band, etc.).


3. Navigate through the application, interacting with supported hardware as needed.

4. Disconnect or turn off the hardware.

Expected result:

• The application should connect with supported hardware and interact properly, registering

any changes made with the hardware in the application.

• When the hardware is disconnected, the application should not crash and should alert the

user that the hardware has been disconnected.

• The hardware device connection should not be intermittent and stay reliably connected to

the host device.

• In-application instructions on how to sync/connect the device to the application should be

provided if the connection is not done automatically.

3.23 Cortana

Actions performed using supported Cortana commands should execute properly.

Steps to Reproduce:

1. Launch the application (this will ensure the commands are populated).

2. Select the Cortana button in the lower left corner of the Windows taskbar (on Mobile devices,

launch the Cortana application).

3. Perform the command, “What can I say?”.

4. Scroll down through the list of applications and locate the application under test.

5. Select the application under test and perform each Cortana command listed with voice

(example: say “News, show me the latest stories”), ensuring that the application is running

before each command is spoken.

6. Close the application.

7. Perform each Cortana command from step 5 with voice, ensuring that the application is not

running when the commands are spoken (show all running apps by displaying the task

manager on Desktop or the running apps list by holding down the back key on Mobile).

Expected result:

Spoken commands should perform the action that is implied by the listed commands.

Cortana should not perform a Bing search or simply launch the application.

Commands should function when the application is running and closed.

More information:

• Cortana Voice Commands: http://aka.ms/cortanadocs

• Speech interactions in your app: http://aka.ms/speechdocs

• App Linking: http://dev.bing.cofm

3.24 Resuming after idling

Applications should support the application lifecycle and resume successfully after the screen is turned

off.

Steps to Reproduce:


2. Navigate to a page other than the default landing page for the application. Change the state

of the page. E.g. fill in a text box.

3. Let the device go idle until the screen automatically turns off.

4. Turn the screen back on and resume use of the application.

Expected result:

• The application should be in the same state as when the application was suspended.

• Interaction with the application should resume as normal.

More information:

For more information see Application lifecycle overview

4. Phone-Only Test Cases

The following test cases apply only to Windows Phone applications.

4.1 Application resumes successfully from tombstoned state

Application should resume successfully from tombstoned state.

Steps to Reproduce:

http://aka.ms/cortanadocs



http://aka.ms/speechdocs

http://aka.ms/speechdocs

http://dev.bing.cofm/


1. Using a low-memory device if possible, launch a different application (ex: Internet Explorer).

2. Press the Start button.

3. Launch the application under test.

4. Perform some steps to get the application into a state with data present.

5. Run memory consumption application(s) to use all of the phone's memory (on 512mb devices

this is generally 5-6 games). This will cause application under test to tombstone.

6. Press the back button until the application under test is brought back.

Expected result:

The application should resume to the same page and state that it was in before entering

tombstoned state. There should be no data loss.

Application should not crash on resuming and user should be able to continue using the

application as normal.

More information:

For more information on tombstoning, see How to preserve and restore page state for Windows

Phone 8, App lifecycle - Windows app development, and ApplicationExecutionState enumeration.

4.2 Back button for mobile

Applications should properly navigate back through pages when using the back key.

Steps to Reproduce:


2. Navigate to a second or third page within the application.

3. Press the back key until reaching the Start menu.

Expected result:

The application should not bring up the Start screen or exit the application unless back key is

pressed while on the main menu.

The application should navigate back through the previously selected pages in the correct

order and eventually arrive at the Start menu.

More information:

Guidelines for back buttons

4.3 Continuum for mobile

Applications should display properly on external monitors and support keyboard and mouse when

using the continuum dock.

https://msdn.microsoft.com/en-us/library/windows/apps/ff967548(v=vs.105).aspx

https://msdn.microsoft.com/en-us/library/windows/apps/ff967548(v=vs.105).aspx


https://msdn.microsoft.com/en-us/library/windows/apps/windows.applicationmodel.activation.applicationexecutionstate.aspx


Steps to Reproduce:


2. Navigate to a second or third page within the application.

3. Dock the device.

4. Select the application from the application list on the external monitor and ensure that the

application resumes where it was in step 2.

5. Navigate through the application and check the functionality.

6. Undock the device.

7. Select the application from the application list and ensure that the application resumes where

it was in step 5.


9. Exit the application by holding down the back key and selecting the x in the upper right

corner and re-dock the device.

10. Launch the application from the external monitor while docked.


Expected result:

The application displays properly on larger external screens, ex: no truncation, stretched out

images, or large empty spaces.

The application functions properly with keyboard and mouse, including any login screens.

More information:

For more information on Continuum, see Optimizing Windows Apps for Continuum and Optimizing

apps for Continuum for phone

5. Windows-Only Test Cases

The following test cases apply only to Windows Client applications.

5.1 Split and windowed views and display resolutions

The application should support a variety of screen sizes and split view(s).

https://channel9.msdn.com/Events/Build/2015/2-703

https://blogs.windows.com/buildingapps/2015/12/07/optimizing-apps-for-continuum-for-phone/

https://blogs.windows.com/buildingapps/2015/12/07/optimizing-apps-for-continuum-for-phone/

Steps to Reproduce:

1. Set display resolution to minimum recommended resolution of 800x600 (1024x768 if testing a

W8.1 application on a W8.1 device).

2. Ensure that the device is in Desktop mode if testing on a W10 device.


4. Select the window button in the title bar to resize the application screen if testing on a W10

device.

5. On each page in the application, adjust the horizontal and vertical size of the application if

testing on a W10 device.

6. Inspect the layout and application functionality.


8. Set display resolution to 1280x800 (1366x768 if testing a W8.1 application on a W8.1 device).

9. Switch the device to Tablet mode if testing on a W10 device.


11. Split the screen by swiping down from the top of the screen.

12. On each page in the application, inspect the layout and application functionality.

Expected result:

All application pages render properly and adapt at minimum resolution and split and resized

view(s). Ex: no text truncation, all command and app bar buttons are accessible, etc.

Consumer non-game applications should not display a logo or error message (ex: "this app

does not support split screen view") in split screen view.

All non-game applications should support windowed views and not force the user to full

screen view.

When snapping an application, the currently viewable position and state of the application

should still be visible when the application is in split screen view and full screen view.

Navigation between pages is still functional when in split screen view.

More information:

For more information see: Defining layouts and views and Guidelines for snapped and fill views

5.2 Application resumes from a suspended state

Applications should support the application lifecycle and resume successfully from a suspended state.

Steps to Reproduce:



http://msdn.microsoft.com/en-us/library/windows/apps/hh465371

2. Navigate to a page other than the default landing page for the application. Change the state

of the page. E.g. fill in a text box.

3. Switch to the desktop.

4. Open Task Manager (enable View | Status values | Show suspended state) and verify

application is suspended.

5. Switch back to the application.

Expected result:

Application should be in the same state as when the application was suspended.

More information:

For more information see Application lifecycle overview

5.3 Back button for desktop

Windows 10 applications should properly navigate back through pages when using the Windows 10

system back button.

Steps to Reproduce:

1. Set the device to Tablet mode.


3. Navigate to the second or third page within the application.

4. Select the system back button located on the taskbar until reaching the Start menu.

Expected result:

The application should not bring up the Start menu unless it is a Windows 8.x application or if

the system back button is pressed while on the main menu.

The application should navigate back through the previously selected pages in the correct

order and eventually arrive at the Start menu.

More information:

Guidelines for back buttons

5.4 Desktop and tablet mode switching

Applications should seamlessly support switching between Desktop and Tablet modes.



Steps to Reproduce:

1. Set the device to Desktop mode.



4. Switch to Tablet mode.



7. Switch to Desktop mode.


9. Navigate through the application and check the functionality

Expected result:

The application should not crash when switching between desktop and tablet mode.

The application should adapt to the change in screen format and handle the differing

functionality (Software Input Pad, split view, etc.)

5.5 Multiple input methods

The application should properly handle multiple input methods.

Steps to Reproduce:


2. Navigate through the application using a mouse. Checking boxes, right clicking on items, and

dragging if possible.

3. Navigate through the application using touch (and Xbox controller and Pen if applicable).

Attempt to perform all of the same functionality as the mouse (ex: right-clicking highlighted

text displays an additional menu).

Expected result:

Application fully supports navigation and selection using a mouse and touch.

Listbox items should not change state if there are no actions associated with marking or

‘ticking’ the item.

More information:

For more information on Touch: Gestures, manipulations, and interactions Keyboard: Responding to

keyboard interactions Mouse: Responding to mouse interactions

5.6 Forward and backward navigation

There should be UI that clearly indicates how to backward navigate within the application.





Steps to Reproduce:

1. Launch the application in Desktop mode.


3. Attempt to return to a previous page.

Expected result:

Applications should be easy to navigate (ex: in-app back buttons or menus on every page).

6. Social Networking Integration Test Cases

The following test case applies only to applications that have Social Network Service integration.

6.1 Shared content

Sharing should function properly when available.

Steps to Reproduce:


2. Navigate through application where social networking or sharing tied features are provided

(also check the “Share” option located in the command bar or Share Charm if applicable.)

3. Verify that actions taken in the application are properly reflected on the social network or

target application.

Expected result:

The contents and status provided by the social network or target application function properly.

6.2 Share targets

Applications that are a share target should be able to have content shared to them from another

application.

Steps to Reproduce:


2. Launch another application that is a share source.

3. Share to the application under test.

4. Verify that the actions taken in the application are properly reflected on the selected share

target.

Expected result:

The application in test appears as a share target when appropriate content is selected in

another application, ex.: a news article to be shared to the application in test Facebook.

The content shared using another application is properly shared to the application, ex.: no

references to a link that does not appear in the post, no cut off text, etc.

7. Game Specific Test Cases

Game applications may have special user scenarios where additional attention or different

interpretation of requirement is needed. The following test cases ensures applications meet general

quality bars commonly expected from game applications.

7.1 Game leaderboard

Implementation of a leaderboard is not required. If a leaderboard is implemented, ensure the

leaderboard is updated for single and multiplayer scenarios.

Steps to Reproduce:


2. Play the game long enough to post to the leaderboard for both single and/or multiplayer

sessions if offered on the application.

3. Look at all the leaderboards in the game.

Expected result:

With an active connection to the game’s backend service, the application should post to the

leaderboard showing both the gamer info such as gamer name and the associated score that

is the same as the one from the game session played.

Without an active connection to the game’s backend service, the application should not show

stability issues when not posting to a leaderboard and should inform users about the

connection issue to refresh leaderboard when users access leaderboards.

7.2 Game achievements and scores

Implementation of achievements and score are not required. If implemented, posted achievements

and scores should be recorded correctly.

Steps to Reproduce:


2. Play the game and earn as many achievements and high scores as possible.

Expected result:

The achievements and high scores are posted on the application and the application’s online

service accurately.

The achievements and high scores are earned while the device has no network connection

are saved accurately on the application and the application’s online service when connection

becomes available.

7.3 Game multiplayer support

Implementation of multiplayer support is not required. If implemented, multiplayer game functionality

should gracefully handle any player leaving the game.

Steps to Reproduce:


2. Select multiplayer game mode and start the game playing.

Expected result:

The game is played in a multiplayer mode with valid user.

The application gracefully handles the case where other user(s) playing the game left the

multiplayer game session.

The application gracefully handles the case where the user lost network connection during

multiplayer game session.

7.4 Roaming Game Saves

UWP games that can be used across multiple device families should support roaming game saves.

Steps to Reproduce:

1. Launch the application on the first device family (ex: desktop).

2. Complete a level and/or earn an achievement.

3. Launch the application on the second device family (ex: mobile), ensuring that this device is

logged into the same Microsoft account as the device in step 1.

4. Complete another level and/or earn an achievement.

5. Launch the application on the first device family (ex: desktop).

Expected result:

The game should retain achievements and level completions across multiple device families.

8. Media Specific Test Cases

The following test cases only apply to applications that support background audio or play video.

8.1 Background audio playback

Media applications that play audio (music, radio, podcasts, etc.) should continue audio playback when

the application is not in the foreground.

Steps to Reproduce:


2. Start audio playback.

3. Switch away from the application.

Expected result:

Audio continues to play after switching away from the application.

8.2 Background audio controls

Media applications that play audio (music, radio, podcasts, etc.) should support the system transport

control when the application is not in the foreground.

Steps to Reproduce:


2. Start playback.


4. Use hardware hot keys for volume to activate the system transport control.

5. Switch tracks or pause using the system transport control.

Expected result:

Audio playback should be controlled using the system transport control.

8.3 Video pauses playback when the application is in the background

Media applications that display video should pause playback when the application is not in the

foreground.

Steps to Reproduce:

1. Launch the application (in Tablet mode for desktop).

2. Start video playback.



Expected result:

Video playback should be paused when switching away from the application.

More information:

For more information, see Application lifecycle (Windows Runtime apps) [Activated and suspending

events]

9.

10. WACK Test Findings

11. Competitive Parity Test Cases

11.1 Feature parity – Competitive platforms

The application should have feature parity with competitive platforms (Android and iOS if applicable).

Steps to Reproduce:

1. Launch application on Windows, Android, and iOS devices (if applicable).



2. Check the application on Windows for feature gaps when compared to Android or iOS.

Features in Android or iOS versions not available in Windows version:

Expected result:

Non-OS specific features that are found in the Android or iOS versions of the application are also

found in the Windows version.

11.2 Feature parity - Website

The application should have feature parity with its respective website.

Steps to Reproduce:

1. Launch application on a Windows device and open the website the application is based on.

2. Check the application on Windows Phone for feature gaps when compared to the website.

Features in the web version not available in Windows version:

Expected result:

Features that are found on the website are also found on the Windows application.

11.3 Performance parity

The application should perform similarly to competitive platforms using comparably capable devices

(Android and iOS if applicable).

Steps to Reproduce:


2. Record the start-up time for the initial launch.

3. Record the data loading time between a page or level.

4. Close the application.


6. Record the start-up time for subsequent launches.

7. Observe the graphical performance and overall application responsiveness.

8. Repeat steps 1-7 on competitive platforms (Android or iOS).

Expected result:

Applications should start up within 5 seconds. Games within 10. If this is not the case then the

start-up time should be no more than the time found on competitive platforms.

Load times between pages should be no more than the time found on competitive platforms.

The graphics and responsiveness of the application is comparable across platforms.

test cases for windows...

Documents