learn test automation & qtp · 2017-12-21 · principles of software testing • the goal of...

Software TestingSoftware Testing

Paratus Systems, Aundh

AgendaAgenda

• Principles of Software Testing

• Testing Strategies

• Advanced Techniques and Tools

• Assignments


2

Principles of Software TestingPrinciples of Software Testing

Testing DefinitionTesting Definition• Testing

• It is the process of exercising or evaluating a system or system

component by manual or automated means to verify that it

satisfies specified requirements – [IEEE 83a]

• Process

• Sequence of steps performed for a given purpose [IEEE]

• Software Process

• A set of activities, methods, practices and transformations that

people use to develop and maintain software and associated

products [SEI-CMM]


3


Testing ObjectivesTesting Objectives• Primary:

• Execute a program with a intent of finding errors to

• Determine whether system meets specification

• Determine whether system meets user’s needs

• Secondary:

• Continuously improve the testing process


4


Misunderstanding about TestingMisunderstanding about Testing• Testing is debugging

• Testing is not the job of a programmer

• If programmers are more careful testing would be unnecessary

• Testing never ends

• Testing activities start only after the coding is complete

• Testing is not a creative task


5

Myths and Facts about testingMyths and Facts about testing


Testing is a single phase in SDLC.

Testing is easy.

Development is worth more than testing.

Complete testing is POSSIBLE.

Anyone can be a Tester.

It’s a continuous process.

Test cases are designed manually.

Testing works throughout SDLC.

All inputs & lines cannot be tested.

Training & skills are required.

6


Theory vs RealityTheory vs Reality

• Theory:• Need Sufficient Time• Base-lined and Frozen Requirements• Automation Testing• Have trained testers: test process, business flow,…

• Reality:• Time Pressures to keep delivery date• Continuously updated requirements• Don’t know when to stop: Update Requirements

continuously (CRs) -> Testers update TC and re-tests• Manual Testing• Lack of dedicated software test environment: share, steal• Inexperienced testers without appropriate training


7


• The goal of testing is to find defects before customers find them out• Exhaustive testing is not possible; program testing can only show the presence

of defects, never their absence• Testing applies all through the software life cycle and is not an end-of-cycle

activity• Understand the reason behind the test• Test the tests first• Tests develop immunity and have to be revised constantly• Defects occur in convoys or clusters and testing should focus on these

convoys• Testing encompasses defect prevention• Testing is a fine balance of defect prevention and defect detection• Intelligent and well-planned automation is key to realizing the benefits of testing• Testing requires talented, committed people who believe in themselves and

work in teams


8


Verification and ValidationVerification and Validation


9

Criteria Verification Validation

Definition The process of evaluating work-products (not the actual final product) of a development phase to determine whether they meet the specified requirements for that phase.

The process of evaluating software during or at the end of the development process to determine whether it satisfies specified business requirements.

Objective To ensure that the product is being built according to the requirements and design specifications. In other words, to ensure that work products meet their specified requirements.

To ensure that the product actually meets the user’s needs, and that the specifications were correct in the first place. In other words, to demonstrate that the product fulfills its intended use when placed in its intended environment.

Question Are we building the product right?

Are we building the right product?

Evaluation Items

Plans, Requirement Specs, Design Specs, Code, Test Cases

The actual product/software.

Activities •Reviews•Walkthroughs•Inspections

•Testing

10

1. Verification is a static practice of verifying documents, design, code and program.

1. Validation is a dynamic mechanism of validating and testing the actual product.

2. It does not involve executing the code. 2. It always involves executing the code.3. It is human based checking of documents and files.

3. It is computer based execution of program.

4. Verification uses methods like inspections, reviews, walkthroughs, and Desk-checking etc.

4. Validation uses methods like black box (functional) testing, gray box testing, and white box (structural) testing etc.

5. Verification is to check whether the software conforms to specifications.

5. Validation is to check whether software meets the customer expectations and requirements.

6. It can catch errors that validation cannot catch. It is low level exercise.

6. It can catch errors that verification cannot catch. It is High Level Exercise.

7. Target is requirements specification, application and software architecture, high level, complete design, and database design etc.

7. Target is actual product-a unit, a module, a bent of integrated modules, and effective final product.

8. Verification is done by QA team to ensure that the software is as per the specifications in the SRS document.

8. Validation is carried out with the involvement of testing team.

9. It generally comes first-done before validation.

9. It generally follows after verification.

Audit-Quality check Inspection-Author, moderator, reader,

tester Walkthrough Review

11


Vtest ModelVtest Model


12


Defect ManagementDefect Management


13

Testing StrategiesTesting Strategies

• It includes identifying

• What type of testing would you use for testing the functionality?

• What are the configurations or scenarios for testing the features?

• What integration testing would you do to ensure these features work

together?

• What localization validations would be needed?

• What “non-functional” tests would you need to do?


14


Unit TestingUnit Testing

• It is a level of the software testing process where individual units/components

of a software/system are tested

• The purpose is to validate that each unit of the software performs as designed

• It is a method by which individual units of source code are tested to determine if

they are fit for use

• It is concerned with functional correctness and completeness of individual

program units

• It is typically written and run by software developers to ensure that code meets

its design and behaves as intended

• Its goal is to isolate each part of the program and show that the individual parts

are correct


15


Integration and System TestingIntegration and System Testing

• Testing the interaction between the modules and interaction with other systems

externally is called integration testing

• Integration testing starts when two of the product components are available and

ends when all component interfaces have been tested

• Integration testing is both a type of testing and a phase of testing

• The testing conducted on the complete integrated products and solutions to

evaluate system compliance with specified requirements on functional and

nonfunctional aspects is called system testing

• It is the first level of software testing where the application is tested as a whole

• Non-Functional aspects include Performance/Load testing, Scalability testing,

Reliability testing, Stress testing, Interoperability testing, Localization testing


16


Acceptance TestingAcceptance Testing

• Acceptance testing is done by the customer or by the representative of the

customer to check whether the product is ready for use in the real-life

environment

• Following test cases can be included for acceptance testing• End-to-end functionality verification• Domain tests• User scenario tests• Basic sanity tests• New functionality• A few non-functional tests• Tests pertaining to legal obligations and service level agreements• Acceptance test data


17


Alpha and Beta TestingAlpha and Beta Testing

• Alpha testing performed at developers site but not by developing team

• Testing a software product which is not the final version.

• This software does not have to necessarily contain the full functionality

required for an application, however core functionality to accept input an

generate output is required.

• Here virtual environment will be created to simulate real time environment.

• Beta or field testing is done by customers at their locations

• Beta Testing is last stage of testing where a product is sent outside the

company or offer the product for free trial download


18


Performance TestingPerformance Testing

• The testing performed to evaluate the response time, throughput and utilization

of the system, to execute its required functions in comparison with different

versions of the same product(s) or different competitive product(s) is called

performance testing

• It is done to ensure that a product• Processes the required number of transactions in any given interval

(throughput)• Is available and running under different load conditions (availability)• Responds fast enough for different load conditions (response time)• Delivers worthwhile return on investment for the resources – hardware

and software – and deciding what kind of resources are needed for the

product for different load conditions (capacity planning)• Is comparable to and better than that of the competitors for different

parameters (competitive analysis and bench marking)


19


Security TestingSecurity Testing

• Security testing is a process to determine that an information system protects

data and maintains functionality

• To check whether there is any information leakage

• To test the application whether it has unauthorized access and having the

encoded security code

• To find out all the potential loopholes and weakness of the system that can

cause loss of important information

• To identify the vulnerabilities and subsequently repair them

• It helps in improving the current system and ensuring that the system will work

for longer time


20


White Box and Black Box TestingWhite Box and Black Box Testing

• White box testing is a way of testing the external functionality of the code by

examining and testing the program code that realizes the external functionality

• It is also known as clear box, or glass box or open box testing

• It takes into account the program code, code structure and internal design flow

• Black box testing is done without the knowledge of the internals of the system

under test

• It is done based on requirements

• It addresses the stated requirements as well as implied requirements

• It encompasses the end user perspectives

• It handles valid and invalid inputs


21


Basis path testingBasis path testing• Basis path testing, a structured testing or white box testing technique used for designing

test cases intended to examine all possible paths of execution at least once. • Creating and executing tests for all possible paths results in 100% statement coverage

and 100% branch coverage.• Example: Function fn_delete_element (int value, int array_size, int array[]){

1 int i;location = array_size + 1;

2 for i = 1 to array_size3 if ( array[i] == value )4 location = i; end if; end for;

5 for i = location to array_size6 array[i] = array[i+1];end for;7 array_size --;

}


22


Basis path testingBasis path testing


• Steps to Calculate the independent paths• Step 1 : Draw the Flow Graph of the Function/Program under consideration as

shown below:

• Step 2 : Determine the independent paths.• Path 1: 1 - 2 - 5 - 7 • Path 2: 1 - 2 - 5 - 6 - 5 - 7 • Path 3: 1 - 2 - 3 - 2 - 5 - 6 - 5 - 7• Path 4: 1 - 2 - 3 - 4 - 2 - 5 - 6 - 5 - 7 23


Equivalence testingEquivalence testing

• Equivalence partitioning is a software testing technique that involves identifying a small

set of representative input values that produce as many different output conditions as

possible

• It is useful to minimize the number of test cases when the input data can be divided into

distinct sets, where the behavior or outcome of the product within each member of the set

is the same

• Example: Identify equivalence classes for this requirement “If a pupil has total score

>=75, he will pass the exam, otherwise will fail (total score is an integer)”

• Search Function Requirement: User fills in search field then clicks search. The system will

find all search strings in the document and highlight them. If none is found, message on

form to indicate no results found (search like Notepad)

• Savings Account: 3% rate of interest is given if the balance in the account is in the range

of $0 to $99.99, 5% rate of interest is given if the balance in the account is in the range of

$100 to $999.99, and 7% rate of interest is given if the balance in the account is $1000 and

above, identify three valid equivalence partitions and one invalid partition.


24


Graph Based testingGraph Based testing


25

• Graph based testing methods are applicable to generate test cases for state machines

such as language translators, workflows, transaction flows and data flows


Test MetricsTest Metrics• Metrics derive information from raw data with a view to help in decision making.• Metrics can be classified as product metrics and process metrics

• Product metrics can be further classified as • Project metrics

• Effort variance, Schedule variance, Effort distribution• Progress metrics

• Testing Defect Metrics• Defect find rate, Defect fix rate, Outstanding defects rate, Priority

outstanding rate, Defects trend, Defect classification trend, Weighted

defects trend, Defect cause distribution• Development Defect Metrics

• Component-wise defect distribution, Defect density and defect removal

rate, Age analysis of outstanding defect, Introduced and reopened defects

rate• Productivity metrics

• Defects per 100 hrs of testing, Test cases executed per 100 hrs of testing, Test

cases developed per 100 hrs, Defects per 100 test cases, Defects per 100

failed test cases, Test phase effectiveness, Closed defect distribution


26


Test ReportingTest Reporting• Test incident report

It is a communication that happens through the testing cycle as and when defects

are encountered

• Test cycle report

It gives• A summary of activities carried out during that cycle• Defects that were uncovered during that cycle, based on their severity and

impact• Progress from the previous cycle to the current cycle in terms of defects fixed• Outstanding defects that are yet to be fixed in this cycle• Any variations observed in effort or schedule

• Test summary report• Phase-wise test summary, which is produced at the end of every phase• Final test summary reports, which has all the details of all testing done by all

phases and teams, also called as “release test report”


27

Advanced Techniques and ToolsAdvanced Techniques and Tools

GUI testingGUI testing

• Graphical User Interface [GUI] Testing – methods used to identify and conduct GUI tests,

including the use of automation tools

• Elements of GUI Testing –

• A process• A GUI Test Plan• A set of supporting tools

• Running the Test Cases –

• Mouse position capture• Event capture


28


Functional testingFunctional testing

• It involves product features and functionality

• It tests product behavior

• Result conclusions are simple steps written to check expected results

• Results vary due to product implementation

• Testing focus is on defect detection

• Product and domain knowledge are required for testing

• Failures are normally due to code

• Testing phases include unit/component, integration and system testing

• One-time setup is required for a set of test cases

• Test cases are executed many times


29


Automated Testing Tools featuresAutomated Testing Tools features• Reliable : Tests perform precisely the same operations each time they are run, there by

eliminating human error.

• Repeatable: We can test how the application reacts after repeated execution of the same

operation

• Comprehensive : We can build a suite of tests that covers every feature in our application

• Reusable : We can reuse tests on different versions of an application, even if the user

interface changes.

• Better Quality Software: Because you can run more tests in less time with fewer resources

• Fast: Automated Tools run tests significantly faster than human users.

• Cost Reduction: As the number of resources for regression test are reduced


30


Automated Testing Tools selectionAutomated Testing Tools selection

Step 1: Define Tool requirements based on Project’s testing strategy and:Business requirementsTool audienceBudget constraints

Step 2: List possible Tool Candidates in the required category & collect Tool Data

Step 3: Compare each tool with its basic features against the Tool requirements defined in Step 1 and the cost

Step 4: Evaluate/Rank the Tools

Step 5: Select or recommend Tool with highest ranking / best pricing

Step 6: Arrange a Demo of Tool in Test Environment

Step 7: Finalize


31


Mobile TestingMobile Testing

• Mobile Application Testing is a process by which a developed application of a mobile

device is tested for its functionality, consistency and usability

• Challenges involved include:• Wide varieties of mobile devices• Different mobile operating system• Different mobile network operators• Input method• Hardware compatibility

• Type of testing include:• Functional testing• Laboratory testing• Performance and Load testing• Security testing• Localization testing• Usability testing


32


Testing Tools SeleniumTesting Tools Selenium

• Selenium is a Functional Automation Tool for Web Applications

• It is an open source tool (no cost involved with it)

• It supports the languages like HTML, Java, PHP, Perl, Python, Ruby and C#

• It supports the browsers like IE, Mozilla Firefox, Safari, Google Chrome and Opera

• It supports the operating systems like Windows, Linux and Mac

• It is very flexible when compared to QTP and other functional tools, because it supports

multiple languages

• It is mainly built on 3 components:• Selenium IDE• Selenium RC• Selenium Grid


33


Testing Tools JUnitTesting Tools JUnit

• JUnit has become the standard tool for Test-Driven Development in Java

• JUnit test framework is a package of classes that lets you write tests for each method,

then easily run those tests

• TestRunner runs tests and reports TestResults

• You test your class by extending abstract class TestCase

• To write test cases, you need to know and understand the Assert class

• To start using Junit, create a subclass of TestCase, to which you add test methods• Name of the class is important – should be of the form TestMyClass or MyClassTest• This naming convention lets TestRunner automatically find your test classes

• Each assert method has parameters like: message, expected-value, actual-value


34


Testing Tools – monkey talkTesting Tools – monkey talk

• Previously named “FoneMonkey”

• Tool for automated testing of iOS, Android, HTML5 and Adobe Flex applications

• It is an integrated environment for recording, customizing, running and managing test

suites

• Free and Open Source

• Powerful and robust record and playback functions for mobile applications

• Powerful IDE

• Fully cross platform

• No jailbreaking needed


35

learn test automation & qtp · 2017-12-21 · principles of software testing • the goal of...

Documents