THE COMPLETE GUIDE FOR NEGATIVE TESTING

DAVID TZEMACHWWW.MACHTESTED.COM

MAY 12 2017

AGENDA

• OVERVIEW

• SO WHAT IS “NEGATIVE” TESTING ANYWAY?

• GOALS OF NEGATIVE TESTING

• NEGATIVE TESTING PROCESS

• ADVANTAGES OF NEGATIVE TESTING

• WHEN TO STOP NEGATIVE TESTING?

• WHY YOU CANNOT IGNORE NEGATIVE TESTING?

OVERVIEW

A QUALITY ASSURANCE PROCESS CONTAINS MANY ACTIVITIES SUCH AS LEARNING, DESIGNING, AND TESTING THAT ARE PERFORMED TO VALIDATE THAT THE

CUSTOMER WILL RECEIVE THE APPLICATION WITH THE HIGHEST QUALITY AS

POSSIBLE.

DURING THESE ACTIVITIES, THE TESTING TEAMS WILL TEST THE APPLICATION WITH

DIFFERENT TEST TECHNIQUES, METHODOLOGIES, AND TOOLS THAT WILL ALLOW

THEM TO INCREASE THE CHANCES TO FIND THE MAXIMUM DEFECTS AS POSSIBLE TO

ENSURE THAT THE CLIENT CAN USE THE SOFTWARE AS EXPECTED.

NOW, FINDING DEFECTS IN ANY APPLICATION IS PROBABLY THE MAIN TASK FOR

ANY TESTER, DUE TO THE IMPORTANCE OF THIS TASK, THE TESTER MUST COVER

TWO MAIN ASPECTS:

THE POSITIVE SIDE

TEST THE APPLICATION AGAINST THE PRELIMINARY REQUIREMENTS AND

SPECIFICATIONS, THIS TESTS WILL DETERMINE IF THE APPLICATION PERFORMS AS

EXPECTED.

THE NEGATIVE SIDE

WHILE THE POSITIVE SIDE OF TESTING IS CRUCIAL AND IMPORTANT, THE NEGATIVE

SIDE IS SOMETIMES EVEN MORE IMPORTANT, DURING THIS TESTS, WE WILL

VALIDATE THAT THE APPLICATION CAN HANDLE FAILURES OR ANY ABNORMAL

SITUATIONS THAT CAN CAUSE UNNECESSARY RISKS OR AFFECT THE REQUIREMENTS

AND SPECIFICATIONS.

SO WHAT IS “NEGATIVE” TESTING ANYWAY?

NEGATIVE TESTING IS….

NEGATIVE TESTING IS A TESTING PROCESS WHERE THE TESTER WILL USE AN

INVALID DATA AS AN INPUT TO THE APPLICATION FUNCTIONALITIES. DURING

THIS TESTS, THE TESTER WILL VALIDATE WHETHER THE APPLICATION CAN

HANDLE THIS TYPE OF INPUTS AND IF THE APPLICATION BEHAVES AS EXPECTED

AND DEFINED IN THE PRELIMINARY REQUIREMENTS AND SPECIFICATIONS.

GOALS OF NEGATIVE TESTING

THE REAL PURPOSE OF NEGATIVE TESTING IT IS VERY IMPORTANT TO UNDERSTAND THE REAL PURPOSE OF NEGATIVE TESTING, AND WHY YOU MUST COMBINE

THEM IN ANY TESTING PROCESS, THIS LIST WILL ALLOW YOU TO UNDERSTAND WHAT NEGATIVE TESTING IS ALL ABOUT:

• VALIDATE THAT THE APPLICATION WILL NOT DO ANYTHING THAT IT IS NOT SUPPOSED TO DO.

• VALIDATE HOW THE SOFTWARE WILL RECOVER FROM FAILURES THAT CAUSED BY INVALID DATA.

• TO DETERMINE UNDER WHICH SCENARIOS THE APPLICATION CAN CRASH.

• IDENTIFICATION OF THE WEAKNESSES POINTS OF THE APPLICATION.

• VALIDATE THAT THE APPLICATION CAN ABNORMAL SITUATIONS WHERE THE USER USES AN INVALID INPUT

(NOTIFICATIONS, RESPONSE TIME, ERRORS, LOGS….).

• VALIDATE THAT THE DIFFERENT ASPECTS THAT ARE NOT DOCUMENTED IN THE POSITIVE REQUIREMENTS AND

SPECIFICATIONS.

NEGATIVE TESTING PROCESS

THE FOUR PHASES OF NEGATIVE TESTING PROCESSPHASE 1: KNOWLEDGE GATHERING

THE TESTER WILL GATHER ALL INFORMATION THAT IS NEEDED TO DESIGN

THE NEGATIVE TEST CASES, DURING THIS ACTIVITY, THE TESTER SHOULD

READ THE APPLICATION DOCUMENTATION, REVIEW THE APPLICATION DESIGN

AND GATHER ALL INFORMATION THAT HE CAN USE PRIOR TO DESIGN THE

TEST CASES.

PHASE 2: TEST DESIGN

BASED ON THE INFORMATION GATHERED IN THE FIRST ACTIVITY, THE TESTER

HAS THE RELEVANT INFORMATION THAT HE CAN NOW USE TO DESIGN THE

NEGATIVE TEST SCENARIOS INCLUDING THE TESTING DATA THAT HE WILL USE

THROUGHOUT THE TEST PROCESS.

PHASE 3: TEST EXECUTION

THE TESTER WILL RUN THE TESTS AND REPORT ANY INCIDENT THAT WAS

FOUND DURING THIS PROCESS.

PHASE 4: EXAMINE THE TEST RESULTS

ONCE ALL TESTS ARE DONE, THE TEST RESULTS ARE REVIEWED AND

ANALYZED TO DETERMINE IF FURTHER TESTS ARE NEEDED.

ADVANTAGES OF NEGATIVE TESTING

THE MAIN ADVANTAGES OF NEGATIVE TESTING(A)

1. NEGATIVE TESTING WILL ALLOW YOU TO INCREASE THE

CONFIDENCE IN THE PRODUCT, THE QUALITY OF THE

APPLICATION IS ALWAYS DETERMINED BY BOTH POSITIVE

AND NEGATIVES SCENARIOS.

2. USING NEGATIVE TESTING, WE CAN SHOW THAT THE

APPLICATION DOES NOT WORK.

3. NEGATIVE TESTING WILL DEMAND MORE CREATIVITY AND

THINKING FROM THE TESTERS TO THINK ABOUT TEST CASES

THAT ARE NOT SPECIFIED IN THE BASIC REQUIREMENTS AND

SPECIFICATION.

4. ONCE A NEGATIVE SCENARIO THAT CAUSED THE

APPLICATION TO FAIL IS FIXED, WE WILL REMOVE ONE

MORE RISK THAT WILL INCREASE THE STABILIZATION OF THE

SOFTWARE.

THE MAIN ADVANTAGES OF NEGATIVE TESTING(B)

1. NEGATIVE TESTING WILL ALLOW US TO

UNDERSTAND THE LIMITATIONS OF THE

APPLICATION.

2. WE CAN USE NEGATIVE TESTING IN ALL

LAYERS OF TESTING (UNIT, COMPONENT, INTEGRATION, SYSTEM AND END-TO-END).

3. NEGATIVE TESTS ARE USED TO COVER

MANY ASPECTS THAT ARE NOT COVERED IN

THE PROCESS OF POSITIVE TESTING.

4. NEGATIVE TESTING WILL ALLOW US TO

GRACEFULLY HANDLE ERRORS AND

NOTIFICATIONS THAT MAY POP-UP TO THE

USER.

WHEN TO STOP NEGATIVE TESTING?

SO WHEN SHOULD WE STOP RUN NEGATIVE SCENARIOS..?

THERE IS NO “END” DATE TO STOP NEGATIVE TESTING, SIMILAR TO ANY OTHER TESTING PROCESS, YOU WILL

STOP TESTING ONLY WHEN THE RISKS ARE REMOVED

AND THERE ARE NO REMAINING ISSUES THAT CAN

AFFECT THE USER EXPERIENCE AND THE STABILITY OF

THE APPLICATION.

ANOTHER TESTING STOPPER THAT IS WELL KNOWN IS

THE PROJECT DEADLINES, ALTHOUGH YOU MAY HAVE

THOUSANDS OF TEST CASES YOU WILL NEVER SUCCEED

TO RUN THEM ALL DUE TO THE TIME LIMITATIONS AND

THE RELEASE COMMITMENTS, THEREFORE YOU WILL

NEED TO PRIORITIZE YOUR TESTS IN ORDER TO FIND THE

MOST IMPORTANT ISSUES IN THE SPECIFICALLY

AVAILABLE TIME FRAME.

WHY YOU CANNOT IGNORE NEGATIVE TESTING?

THE IMPORTANCE OF NEGATIVE TESTING (A)

1. NEGATIVE TESTING WILL ALLOW YOU TO FIND

PROBLEMS THAT ARE UNIQUE FOR SPECIFIC

INTEGRATION.

2. POSITIVE TESTING WILL NOT ALLOW YOU TO

UNDERSTAND HOW EXTERNAL ARTIFACTS CAN

AFFECT THE APPLICATION FUNCTIONALITY (LOW

DISK SPACE, LOW MEMORY, LOW BANDWIDTH, SERVICE CRUSHES ETC.)

3. AS MANY EXCEPTIONS THAT ARE FOUND AND

REMOVE, THE BETTER QUALITY YOU ACHIEVE.

4. NEGATIVE TESTING WILL ALLOW US TO IMPROVE

THE NOTIFICATIONS IN CASE OF FAILURES.

5. IF YOU FAIL TO FIND A SOFTWARE EXCEPTION, THE

SYSTEM WILL BE RELEASED AS IF NO PROBLEM HAS

BEEN FOUND, THIS CAN LEAD TO A MAJOR ISSUE

ON ALL SYSTEM LEVELS.

THE IMPORTANCE OF NEGATIVE TESTING (B)

1. USING NEGATIVE TESTING, WE WILL ENFORCE THE

APPLICATION GUIDELINES AND DEMANDS, THESE

ENFORCEMENTS WILL PREVENT UNNECESSARY AND

UNEXPECTED SCENARIOS THAT MAY RAISE WILL THE

CUSTOMER INTERACTS WITH THE APPLICATION.

2. NEGATIVE TESTING WILL ALLOW

3. NEGATIVE TESTING WILL ALLOW YOU TO STOP THE

APPLICATION EXECUTION IN CASE OF A FAILURE UNTIL

THE PROBLEM IS FIXED.

4. IF YOU RUN ONLY “POSITIVE” SCENARIOS, YOU WILL

FAIL TO UNDERSTAND WHAT ARE THE APPLICATION

UPPER AND LOWER LIMITS.

5. A SMALL PROBLEM THAT WE FAIL TO FIND DURING THE

TESTING CYCLES CAN (AND PROBABLY WILL) BE

ACCELERATE AND BECOME A MAJOR ISSUE.

FOR ADDITIONAL KB’S PLEASE VISIT MY BLOG

WWW.MACHTESTED.COM