+

The Difference Between General Software Testing and Game Testing?

Theorycrafting based on my brief experience with game testing

+Introduction

In this presentation I explore the difference between general software testing and game testing

I have previously worked with software testing of mobile devices, and now work with game testing

This presentation is an attempt create a theoretical view of the differences before I have the experience to evaluate it myself

+Software Testing

General Software Testing

Game Testing

+General Software Testing

Black Box Software Testing [1]

Test Automation [2]Context-Driven

Testing [3]

Testing at Google & Microsoft [4][5]

ISO29119[6]Testing & Checking

[7]

+What makes Game Testing Unique

Fun Factor Testing[8] Balance Testing[8] Game Level/World Testing[8]

AI Testing[8][10]Multiplayer/Network

Testing[8] Audio Testing[8]

Physics Testing[8] Realism Testing[9]Modification API

Testing

+There are many similarities

Software is software whether it is a game or not

Software testing is an engineering discipline whether it is games, applications or other types of software

Functionality, performance, stability, compatibility, localization, etc. are mostly the same, but with different focus depending on context

A game tester should have the same general knowledge base as any software tester, but with a special focus on what makes games unique

+Fun Factor Testing

User experience and usability are valid testing areas for all types of software

What differs games from other types of software is that it has to be fun – it is entertainment and it needs to have a fun factor that other types of software may not require

To be able to work with Fun Factor Testing requires a large domain knowledge of games and what makes them fun

+Balance Testing

Balancing different options is something that is much more prevalent in games compared to other software Balancing different character options such as race, class,

and attribute choices Balancing weapons, boosters, equipment, and other

paraphernalia Balancing levels Balancing difficulty

+Game Level/World Testing

Somewhat unique to games is the number of instances in which assets are used An application may use a feature in one place, while a game might use it

in 20 different levels or parts of the world This means that you need to test that the asset is used correctly in all

these instances

Especially 3D worlds require a unique approach Stuck/Sticky spots Invisible walls Map holes Missing geometry

+AI Testing

Facing a computer controlled opponent is common in games, but less common in other types of software

Testing the different attributes and of the AI, and how it reacts in different situations is critical in many games How well it mimics human behavior Survival instinct – such as looking for cover in a fire fight Hunting state – not waiting to react to the player’s actions, but

proactively performing actions Infighting – how it reacts to other AI Pathfinding

+Multiplayer / Network Testing

Playing against multiple other opponents is quite common in games, and even though other software also communicates with other users and servers through different mediums, it is often to a lesser extent, and less sensitive

Many things can go wrong in multiplayer Failed connections Dropped connections Lag Invisible players Scoring errors Unaccepted invitations

+Audio Testing

Obviously almost all software has some kind of audio However in games it is often more detrimental to the user experience if

something goes wrong Often much more complex in games, with many sounds playing

simultaneously

Many things can go wrong Audio drop Skipping Distortion Missing sound effects Volume level

+Physics Testing

Some games have physics engines, which affect both gameplay and animations

It requires a specific skill to spot physics bugs

Breakable geometry must be tested to assure that it is destroyed in a way consistent with the desirable physics

Dynamic behavior such as boxes moving when you walk into them is another example of physics in action

+Realism Testing

This type of testing is also related to how the game managed to mimic the real world in a desirable way

Is the car handling like a real car?

Does the weapon feel like a real weapon?

Running animations and jumping must have the right look and feel

This type of testing requires a lot of domain knowledge – to know if an airplane is realistic enough, you must know about airplanes

+Modification API Testing

If the game allows for user to create their own mods, and it is a competitive game like an mmo this requires a unique approach

Open APIs are common in software, but competitive games add a dimension of not allowing mod users to gain unfair advantages or being able to exploit the game using their mods

Imagining how the APIs will be used is critical to understanding if there will be future problems or not

+Player Types [9]

You can categorize gamers in a slightly different way than users of other applications and technology in general

Killer Achiever Explorer Socializer

Casual Gamer

Hardcore Gamer

Button Masher

Customizer

Exploiter

+Prioritization based on Player Types How you prioritize your tests could be influenced by

which player types you are aiming the game for

If you want to please all types, then making sure you have run sufficient tests in each category will go a long way when it comes to reducing critical bugs

+Combinatorial Testing [9]

Of course combinatorial testing is nothing unique to games, but when looking at a large, sprawling 3D game world, one can imagine the size of the combinatorial explosion

A larger game world, a large amount of actors, and a large amount of actions for these actors to perform, all results in a combinatorial nightmare

+Conclusion

There are many similarities between game testing and other software testing

There are some aspects of game testing that are unique and requires more or less domain knowledge

How you categorize gamers could differ from how you categorize other software or technology users

All types of software testing require experts to perform it well and in a structured way

+References

[1] BBST

http://www.testingeducation.org/BBST/

[2]The A Word

https://leanpub.com/TheAWord

[3]Context-Driven Testing

http://context-driven-testing.com/

[4] How Google Tests Software

http://www.amazon.com/Google-Tests-Software-James-Whittaker/dp/0321803027

[5]How We Test Software at Microsoft

http://www.amazon.com/How-We-Test-Software-Microsoft/dp/0735624259/

[6]ISO29119

http://www.softwaretestingstandard.org/

[7] Testing & Checking

http://www.satisfice.com/blog/archives/856

[8] Game Development Essentials: Game QA & Testing

http://www.amazon.com/Game-Development-Essentials-QA-Testing/dp/1435439473

[9] Game Testing: All on One

http://www.amazon.com/Game-Testing-Second-Charles-Schultz/dp/1936420163/

[10] Artificial Intelligence (Video Games)

http://en.wikipedia.org/wiki/Artificial_intelligence_(video_games)