heuristics, mnemonics and other greek words in the exploratory testing of mobile apps

Software quality assurance days20 Международная конференция по вопросам качества ПОsqadays.com

Минск. 24–26 ноября 2016

Владислав РоманенкоIntellectsoft. Минск, Беларусь

Эвристики, мнемоники и другие греческие слова в исследовательском тестировании мобильных приложений

Тестирую мобильные и веб приложения

ISTQB Certified Test Analyst

QA-инженер в Intellectsoft

[email protected]

2

@laechoppe

Reasons forExperience-based testing:

- lack of time - lack of documentation - issues could hardly be located using more systematic approaches

3

Experience-based techniquesError guessing

Checklist-based testing

Exploratory testing

4

Error guessing

5

Checklist-based testing Checklist-based testing

6

Exploratory testing- tests are designed and executed concurrently- usually guided by a charter- lightweight documentation (if any)

7

+ more interesting+ more apparent bugs first+ good if time is limited+ works if docs are absent+ more checks due to variations of tests+ ….

Exploratory testing

- less experienced testers won't get as much as more experienced

- could be hard to reproduce the issues- not suitable if requirements are mandatory- hard to collect metrics- …

8

Heuristic - a set of research methods to facilitate the discovery of a previously unknown

Mnemonic - a learning technique, allowing to remember the sequence and structure of something.

Richard Of York Gave Battle In Vained

range

ellow

reen

lue

ndigo

iolet

9

Mnemonic

Not Every Cat Eats Sardines (Some Are Really Yummy)

?

10

MnemonicNotEveryCatEatsSardines(SomeAreReallyYummy)

11

Goldilocks Principle - in any dataset there’s always a U-shaped distribution

12

Goldilocks Principletoo short text too long text just right text

13

SFDPOT (San Francisco Depot)

S StructureF FunctionD DataP PlatformO OperationsT Time

14


What files does it have? Can I download the app?Can I download an update?Does the app write to an SD card? What if the card is filled, removed or changed?

SFDPOT

15


SFDPOT

16


Does the app perform the tasks or features it was designed to?Does the app or site prevent, block or not include features not intended on a mobile device?Does the app direct me to turn on necessary related services?

SFDPOT

17


SFDPOT

18


What kinds of input does it process?What does its output look like?Does it come packaged with preset dataDoes the app integrate with any hardware such as bar code scanners for payment or other purposes?

SFDPOT

19


SFDPOT

20


What operating systems does it run on?Does the environment have to be configured in any special way? Does it depend on third-party components?Does the app depend on any device settings?

SFDPOT

21


SFDPOT

22


Where and how the app will be used?Does the app function when I am moving and traveling around (switch wi-fi <-> 3G)?What happens when a mobile interruption occurs such as an incoming call or text message?

SFDPOT

23


SFDPOT

24


What happens if the time zone is switched?What happens when my location is switched?Does the device auto detect a change in time zones and the day and date?Does the app find time-related data based on my device time?

SFDPOT

25


SFDPOT

26

SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface

Does the app work well on phones and tablets?Does the app work well when used with “one hand?”Is the mobile version consist or complementary to the desktop version?Is the app consistent with UI current constructs in the marketplace?

27

SFDPOTIS StructureF FunctionD DataP PlatformO OperationsT TimeI Interface

28

SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors

Does the app continue gracefully after an error?Is the message helpful, instructive?If a user provokes multiple errors in succession, does the app continue?Can I get to an error state that forces the app to shut down?What happens when I restart the crashed app – is previous data or state saved?Is there a method to send an error report?

29

SFDPOTIES StructureF FunctionD DataP PlatformO OperationsT TimeI InterfaceE Errors

30

I SLICED UP FUNI InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network

31

I InputsS StoreL LocationI InteractionsC CommunicationE ErgonomicsD DataU UsabilityP PlatformF FunctionU User ScenariosN Network

I SLICED UP FUN

32


I SLICED UP FUN

33


I SLICED UP FUN

34


I SLICED UP FUN

35


I SLICED UP FUN

36


I SLICED UP FUN

37


I SLICED UP FUN

38


I SLICED UP FUN

39


I SLICED UP FUN

40


I SLICED UP FUN

41


I SLICED UP FUN

42


I SLICED UP FUN

43


Start from

I SLICED UP FUN

44

Heuristics help to:

find out what to start testing fromremember the context of app usageavoid repeating the mistakes…

TO SUM UP:

45

эвристикамнемоникатехникатеориядиапазонсистемныйпараллельнометодсимвол

идеяэргономикателефонбиблиотекаграфикмикрофонпараметрриск…

Some of the Greek words:

46

http://www.rockynook.com/press-release-software-test-engineers-handbook-2nd-edition/

http://www.computing.dcu.ie/~ray/teaching/CA358/dorothy_graham.pdf

http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/

http://karennicolejohnson.com/2012/12/applying-the-sfdpot-heuristic-to-mobile-testing-adding-i/

http://karennicolejohnson.com/wp-content/uploads/2012/11/KNJohnson-2012-heuristics-mnemonics.pdf

https://www.utest.com/articles/rotation-tour-in-mobile-testing

http://www.kohl.ca/articles/ISLICEDUPFUN.pdf

http://www.satisfice.com/articles/sfdpo.shtml

http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html

https://goritskov.com/blog/heuristics-in-testing.html

Pictures from: mycutegraphics.com

References

47

http://www.rockynook.com/press-release-software-test-engineers-handbook-2nd-edition/

http://www.computing.dcu.ie/~ray/teaching/CA358/dorothy_graham.pdf

http://karennicolejohnson.com/2012/05/applying-the-sfdpot-heuristic-to-mobile-testing/

http://karennicolejohnson.com/2012/12/applying-the-sfdpot-heuristic-to-mobile-testing-adding-i/

http://karennicolejohnson.com/wp-content/uploads/2012/11/KNJohnson-2012-heuristics-mnemonics.pdf

https://www.utest.com/articles/rotation-tour-in-mobile-testing

http://www.kohl.ca/articles/ISLICEDUPFUN.pdf

http://www.satisfice.com/articles/sfdpo.shtml

http://michaeldkelly.com/blog/2005/9/20/touring-heuristic.html

https://goritskov.com/blog/heuristics-in-testing.html

http://mycutegraphics.com

[email protected]

48

@laechoppe

heuristics, mnemonics and other greek words in the exploratory testing of mobile apps

Technology