GameHouse Confidential 1

Isolation/Generalization &

Ad-Hoc Testing

February 2013

GameHouse Confidential 2

Isolation

Isolation is the process of examining the root causes of a defect.

• While the exact root cause might not be determined it is important

to try and separate the symptoms of the problem from the cause.

• Isolating a defect is generally done by reproducing it multiple times

in different situations to get an understanding of how and when it

occurs.

GameHouse Confidential 3

Generalization

Generalization is the process of understanding the broader impact of

a defect.

• Because developers reuse code elements throughout a program a

defect present in one element of code can manifest itself in other

areas.

• A defect that is discovered as a minor issue in one area of code

might be a major issue in another area.

• Individuals logging defects should attempt to extrapolate where else

an issue might occur so that a developer will consider the full

context of the defect, not just a single isolated incident.

• A defect report that is written without isolating and generalizing it, is

a half reported defect.

GameHouse Confidential 4

Severity

The importance of a defect is usually denoted as its “severity”. There are many

schemes for assigning defect severity – some complex, some simple. In JIRA,

severity is synonymous with Priority.

• Almost all feature “Severity-1” (Blocker) and “Severity-2” (Critical)

classifications which are commonly held to be defects serious enough to delay

completion of a project or build release.

• “Severity-3” (Major) is the middle ground that is important enough to be fixed

relatively quickly, but not immediately.

• In many other companies, developers and testers get into arguments about

whether a defect is “Severity-4” (Normal) or “Severity-5” (Trivial) and time is

wasted.

GameHouse Confidential 5

Priority

Bugs should be assessed in terms of impact and probability to

determine Priority.

• Impact is a measure of the seriousness of the defect when it occurs

and can be classed as “high” or “low” – high impact implies that the user cannot complete the task at hand,

– low impact implies there is a workaround or it is a cosmetic error.

• Probability is a measure of how likely the defect is to occur and

again is assigned either “Low” or “High”.

Impact/Probability

= Priority

High/High High

High/Low Medium

Low/Low Low

GameHouse Confidential 6

BUG

Trivial

Minor

Major

Critical

Blocker

PRIORITY

GameHouse Confidential 7

Pokédex 000 item duplication glitch

The Pokédex 000 item duplication glitch (commonly referred to as the Rare

Candy glitch due to the preferred item chosen to duplicate) is an

infamous glitch in the Pokémon Generation 1 games. It allows the player to

duplicate items in their bag.

• Through the development of Pokémon Yellow, which occurred in the two years

following the release of Pokémon Red and Green in Japan, the old man

glitch was disabled by blanking the data for wild Pokémon before overwriting it,

and reprogramming shore tiles to not have any Pokémon. However, a user can

still encounter the glitch Pokémon and exploit the item duplication glitch using

the Ditto glitch or the Cable Club escape glitch.

GameHouse Confidential 8

• RESULT: The sixth item in the bag is duplicated upon encountering the glitch Pokémon, and again if it is caught. The quantity of this item will be increased by 128, provided that the quantity is less than 128 before performing the glitch. This means that the player is free to perform the glitch again by swapping the item, or using/tossing the duplicated item to reduce its quantity back under 128.

• CAUSE: Every Pokémon has two separate bit lists that tells the game whether it has been seen or caught. If the bit is off, that means it hasn't been caught or seen.

• Missingno.'s  Pokédex seen bit is in the same location as the bit that stores how many of the 6th item is in the bag, as well as 'M (00)'s. This is why, when Missingno. or 'M (00) is encountered, the sixth item slot is increased by 128 if the quantity of the item is less than 128. The glitchy box symbol is a result of the game attempting to display a number greater than 99, which causes it to grab sprites from beyond the number sprites. Sometimes it can appear to be a blank tile, but if one goes somewhere else or out of battle it will revert back to an unusual tile.

• A way to tell whether " 9" is in fact 9 or [blank tile]9 is to select Toss. The quantity is displayed with a leading zero if the amount is actually 9, and as simply " 9" if not.

Bug Regression &

Halo Testing

February 2013

GameHouse Confidential

10

Bug Regression• Bug regression is among the most crucial qualities to have as a

tester. There is a broad generalization of regression being only verifying fixed bugs. Actually there are two facets to regression

– Bug Fix Verification

– Validation of Reproduction Steps

• The first is simple. Fully comprehend the reproduction steps. If steps are not entirely clear, we assign the bug back to the Reporter with comments as to why it is not clear. As we do regressions, these are the ones sent back as CONFUSED.

• As a Tester, our bugs need to be the most clear and concise bugs in the JIRA database. Developers and artists will write bugs and tasks specific to their production tracking.

GameHouse Confidential

GameHouse Confidential 11

Regression Priorities

• Provide adequate coverage without wasting time should be a

primary consideration when conducting regression tests. – Sort and assign groups of bugs to regress.

• Casino example: Slots, Blackjack, etc.

– If attempting to isolate a new issue, have a fellow team member try and

reproduce the issue with you.

• Try to spend as little time as possible doing regression testing

without reducing the probability that you will detect new failures in

old, already tested code.– Make sure to check or keep a mental note of issues as you regress them.

• If time permits, check closed bugs of the same type and make sure

they aren’t reopened without us being aware.

GameHouse Confidential 12

New Issues from Regression

• It is important to remember that when a bug is fixed and a new build

is released, that new bugs are likely to be introduced.

• Check surrounding issues through functionality testing. Look for

general problems within the game itself or its user interface, such as

stability issues, game mechanic issues, and game asset integrity.– Example: Purchase Testing

GameHouse Confidential 13

Halo Testing

• ha·lo  (hl)n. pl. ha·los or ha·loes

A circular band of colored light around a light source, as around the sun or moon,

caused by the refraction and reflection of light by ice particles suspended in the

intervening atmosphere.

Abstract Exercise: Halo Test around this photo

GameHouse Confidential 14

Smoke Testing

Initially was used to refer to physical tests made to test for leaks by running non-

toxic smoke through a closed system of pipes.

By metaphorical extension, the term is also used for the first test made after

assembly or repairs to a system, to provide some assurance that the system

under test will not catastrophically fail.

• Smoke testing performed on a particular build is also known as a build

verification test. Microsoft claims that after code reviews, "smoke testing is the

most cost effective method for identifying and fixing defects in software

• Smoke tests can be broadly categorized as functional tests or unit tests.

Functional tests exercise the complete program with various inputs.