Automation Concepts & QTP

Demo

Name : Dheenan Ganesan

2

Please mute your mobile phones

Do not come late OR while the session is ON

Help each other in learning – as learning is a continuous process

Participate actively to make the session interactive

No side conversations

Do not beat the dead horse

Ground Rules

3

Agenda

Automation Concept

1. Test Automation – Why?

2. Test Automation – Qualifying Criteria

3. Test Automation - Prerequisites

4. Automation Readiness Check

5. Challenges in Automation

6. Phases of Automation

7. Automation Framework

8. Skills required for scripting

9. Availability of Automation tools

Automation Tool - QTP

1. Tool Selection Process

2. Costs involved in automation

3. ROI

4. Automation Benefits

5. Automation Drawbacks

6. Case Study

7. Test

QTP Demo

3

Test Automation – Why?

Reduced Test Cycle Time

Earlier defect reporting

Reduction in Test Execution Effort

Remove Tester fatigue

Earlier informed decision making

Detailed Test Log Archives

Reduced criticality of Functional/Domain expertise

4

Test Automation – Qualifying Criteria

Repetitive nature of tests:

If the tests are repetitive in nature like regression tests, portability tests,

compatibility tests etc, then automation can be considered.

Regression tests – Tests that are run for every build/release

Portability tests – Tests that are run on different OS or webservers

Compatibility Tests – Tests that are run on different browsers or software

components

Tests run on production:

The smoke tests( without data change) have to be completed within a very

short duration of time to minimize the downtime of the production servers.

5

Test Automation – Prerequisites

Application stability

Stable GUI, stable functionality

Tool readinessTool decided, tool compatibility

established, availability of licenses

People readiness

Automation team available with

required skills

Project Readiness

The project has reached a stage when test automation is estimated,

The approval of project management team for going ahead

with test automation.

Process Readiness

The right automation framework is identified and communicated to the

automation team.

6

Automation Readiness Check

• Includes Test Calls, Test data arrangement, scripting standardsFramework should be

functional

• automation specific tasks like GUI objects maintenance and synching, local to shared staging, reviewing, configuration control etc.

Tasks allocation is understood

• Individual Roles and responsibilities are understoodRoles and Responsibilities

are identified

• Distribution lists, communication matrix established. Escalation mechanism understood.

Communication and Escalation channels defined

• Environment readiness checklist is doneAutomation Environment is

ready

• Formal – Automation plan signoff, kick off meeting; informal – stand up meeting

Project Management Team informed

7

Test Methodology - Automated

Top 5 Best Practices for Automation Success

Focus on the methodology, not the tool Choose extensible test tools Separate test design and test automation Lower costs Jumpstart with a pre-trained team

Assess

Environment

Identify

Gaps

Establi

sh

Framew

ork

Tool

Evaluatio

n, PoC

Script

Develop

ment

Stabilizati

on

Application

Architecture,

Stability

Test Pack Analysis

Financial

Resources

Personnel

Time-line

Dependencies

Completed

Environmental

Assessment

Determine priorities

Identify Strategic

Outlook

Prioritize Test Pack

Test Strategy

Test Pack Re-

engineering

Research, Evaluate,

Score test tool

candidates

Object Identification

Proof of Concept /

Prototyping

Approval

Test Scripts Stds

Learning, Metrics from

PoC re-used

Automation Framework

Design

Test Scripting

Script Integration,

Reviews, Testing, Re-

work

Dry Run

Documentation

Training

Metrics (% Automation

achieved)

Process Improvements

Maintenance Phase

1 2 3 4 5 6, 7

Key Considerations for automation

Application/functionality stability, release plans Application architecture Test pack analysis for automatable test conditions/cases Interfacing application dependency Analysis of automation ROI

Early Involvement

Define Performance Testing Requirements

Structured and Organized Test Case Repository

Software Testing Life Cycle (STLC)

Requirement Analysis

Testing Activities

Manual / Automation/ Performance Test Automation

Specific Activities

Test Strategy

Test Planning

Test Scenarios/

Conditions

Identify features which can/ cannot be automated

High level Automation Design / Framework

Automation Strategy

Design & Development of Automation Scripts

Test Script Execution

Test Script Maintenance

Design & Coding

Testing

Software Development Life Cycle (

SDLC )

Performance Testing

Specific Activities

Test Cases

Traceability

Test Environment

Test Data

Test execution

Defect Tracking & closure

Test Summary Report

Defect Summary Report

Design & Development of Performance Test Scripts

Performance Testing

Analysis and Fine Tuning

Performance Test Reports

•Identify business

scenarios for

performance testing

Framework Driven Automation

Modularized

Automation

Test Suite

Automation Framework

Manual Test Case Repository Manual Test Procedures

Automation

Design Considerations

Test

Automation

Standards /

Guidelines

Business

Scenarios

Automation

Test Scripts

Re-usable

Library

Automation

Design

Document

Test Automation

Results /

Metrics

Test Strategy

Test

Automation

Configuration

Control

Application Test Automation Tool Test Management Tool

Test Environment

Ap

plicatio

n M

ainten

ance

Reg

ression

Testing

En

han

cemen

t

Automation Frameworks in Practice

Modularity Driven

Data Driven

Keyword Driven

Hybrid

Types of Automation Framework

The Test Automation Life Cycle

Customer Speaks…..

Automated test generation reduces

manual effort up to 90%

Test planning time reduced by up to

50%

Test Execution Cycle reduced by

30%

Reduced QA effort by 25%

Modularity and Reusability - The same

tests can be reused on different

versions of an application, even if the

user interface changes

Process Expertise

Client References spanning across

geographies

Cost Effective solutions enriched by

stable processes

World class methodologies & tools

Standards Complying with IEEE, SW-

CMM/CMMI, ISO 9000:2000

Certified testing consultants & lead

assessors with domain expertise

Back

Approach to Test Automation

• Analyze business functionalities, application architecture • Economic feasibility – ROI analysis• Identify features which can/

cannot be automated

Test Planning & Design2

• Automation Effort Estimation• Test Strategy • Test Framework • Prepare Test Conditions/Cases• Prepare Traceability Matrix

Development of Scripts 3

• Test Scripts • Identify & place re-usable components in reuse library • Add checkpoints, synchronization points& functional validations• Dry Run

Decision to Automate1 Maintenance of Scripts

• Periodic Updation of scripts with changes to existing functionalities / new requirements• Label and maintain the changes • Versioning scripts & other related Testware

5

Test Execution4

• Test Script Execution • Reporting Results • Defect Management • Test Automation metrics• Update scripts • Prepare Operations Manual

Value Proposition on Test Automation

Fast – Runs Tests significantly faster with reduced time of testing lifecycle

Reliable – Tests perform precisely the same operations each time they are run

Repeatable – Repeat the same set of execution as many times desired.

Programmable – Program sophisticated tests that bring out hidden information.

Comprehensive – Build a suite of comprehensive tests that covers every feature

Reusable – Reuse test cases created in various situations

Maintainable – Easy maintainability and minimizes effort of maintenance

Long Term Savings – Phased ROI based Automation - in testing effort & cycle time

Improved Planning and Focused testing effort

Enhanced Coverage

Minimal Manual Intervention and coding and hence reliable

Regression Pack for Future Product Releases

Reduces Go-To-Market time for an application

Reduced Defect Density

Challenges in Automation - Process

15

Challenges in Automation – Application

Automation to keep pace with the rapidly changing application

Standards not followed during development.

Operations cannot be performed on some objects.

Batch processes – wait time and processing time

Environment – Shared environment, unstable

Test Data – Test data issues, modification of test data

16

Challenges in Automation - Tool

Objects not recognizable

by tool.

Backward compatibility

of tool versions

Difficult to use, non-

user friendly

Memory cache size

Non-availability of

support

17

Challenges in Automation - Skill

18

Test Automation – Framework Architecture

QUICK TEST

PROFESSIONAL

TERMINAL

`

TERMINAL

VPN

``TERMINAL

QUICK TEST PROFESSIONAL

TEST

RESULTS

SCRIPT

OBJECT

REPOSITORY

REUSABLE

ACTIONSLIBRARY FILES

QUALITY CENTER

DATA FILESRECOVERY

MANAGER

APPLICATION UNDER TEST

Tools Availability

20

•HP-QTP

•HP-WinRunner

•Rational Robot

•Compuware TestPartner

•Segue Silktest

Selenium

Floyd

Watir

Frankenstein SWTbot

JMeter

About QTP

QuickTest Professional enables you to

test

Standard Windows applications

Web objects, ActiveX controls

Visual Basic applications

Note:

Add-ins for special environments such as Java, Oracle, SAP

solutions, .NET Windows and Web Forms, Siebel,

PeopleSoft, Web services, and Terminal emulator

applications.

QTP Versions

QTP 10.0

QTP 9.5

QTP 9.2

QTP 9.0

QTP 8.2

QTP 8.1

QTP 8.0

QTP 6.5

QTP 6.0

QTP 5.6

QTP – Add in Manager

Keyword & Expert View

Keyword View:

Enables to create and view the steps of the test in a keyword-

driven.

Table-like view, in which each step is a separate row in the

table, and each column represents different parts of the steps.

Expert View:

Enables to create scripts in the form of VBscript for each

operation performed on the application.

Script editor and for each object and method in an Expert

View statement, a corresponding row exists in the Keyword

View

Keyword View

Expert View

Creating QTP Script

Record a session

Build an object repository

Insert checkpoints into your test

Broaden the scope of your test or component by replacing fixed

values with parameters.

Enhance your test and/or add programming statements

Test Results

QTP Recording Modes

Recording Modes:

Standard Recording Mode

Records the objects in your application and

the operations performed on them.

Low-Level Recording Mode

Records mouse clicks and keyboard input

with the exact x- and y-coordinate

QTP Default Mode : Standard Recording Mode

Synchronizing your Tests

When you run tests, your application may not always respond

with the same speed. For example, it might take a few seconds:

for a progress bar to reach 100%

for a status message to appear

for a button to become enabled

for a window or pop-up message to open

You can handle these anticipated timing problems by

synchronizing your test to ensure that QuickTest waits until your

application is ready before performing a certain step.

Synchronization Methods

Synchronization point

Exist or Wait statements

Increase the default timeout settings

Synchronization Point

A synchronization point is a

line in the test script that

instructs QTP to wait for a

certain response from the

application during playback.

Insert ->Synchronization Point

Synchronization point, which instructs QuickTest to pause the test

until an object property achieves the value you specify.

When you insert a synchronization point into your test, QuickTest

WaitProperty step is displayed in the Tree View with the icon for

the selected object. And also it generates the WaitProperty

statement in the Expert View.

Usage of Comments

Comments - A comment is a free text entry that spans an entire row.

Object Repository

When you create a test by recording, QuickTest learns

the properties of test objects and are stored in Object

repository. When the test runs, QuickTest searches for

the object that matches the description it learned.

Object Identification

37

Object Identification – Best Practices

Object Repository

Object Spy

Constructs

Conditional Statements

If .. Then

Else

End If

ElseIf … Then

Loop Statements

For … Next

Do… while

Do… until

If..Else Example

If window("Notepad").Exist(2) Then

reporter.ReportEvent micPass, "Notepad dialog",

"Notepad dialog exists"

Else

reporter.ReportEvent micFail, "Notepad dialog",

"Notepad dialog exists"

End If

Checkpoints

A Checkpoint is a verification point that compares a current value

for a specified property with the expected value for that property

How Checkpoint Works ?

When you add a checkpoint, QuickTest adds a

checkpoint with an icon in the test tree, and

adds a Check CheckPoint statement in the

Expert View.

When you run the test, QuickTest compares

the expected results of the checkpoint to the

current results. If the results do not match, the

checkpoint fails

Checkpoint Types

Output Value

An Output Value is a value captured during the test run and

entered in the run-time Data Table for use at another point in the

test run.

Parameterization

QuickTest enables you to expand the scope of

a basic test by replacing fixed values with

parameters. This process, known as

parameterization, greatly increases the power

and flexibility of your tests.

Parameter Types

Test/action parameters

Data Table parameters

Environment variable parameters

Random number parameters

Descriptive program - Advanced

'Record Method

Browser("Untitled Page").Page("Untitled Page").Link("Login To HCL LOS").Click

'One way

'Browser("Name:=Untitled Page").Page("Title:=Untitled Page").Link("text:=Login To HCL

LOS","html tag:=A").Click

'Other Way

Set MyObject = Browser("Name:=Untitled Page").Page("Title:=Untitled

Page").Link("text:=Login To HCL LOS","html tag:=A")

Myobject.Click

To connect QTP to Quality Center Choose Tools > Quality Center Connection or click the Quality Center Connection

toolbar button. The Quality Center Connection dialog box opens

In the Server box, type the URL address of the Web server where Quality Center is installed

To view the current Quality Center connection, point to the Quality Center icon. To open

the Quality Center Connection dialog box, double-click the Quality Center icon

Connecting QTP to TestDirector: The connection process has two stages. First, you connect QTP to a local or remote

Quality Center Web server. This server handles the connections between QTP and the Quality Center project

Next, you choose the project you want QTP to access. The project stores tests or components and run session information for the Web site or application you are testing.

Integration of QC with QTP:

Note: Quality Center projects are password protected, so you must provide a user name and a password

QC Server Name

Working with Quality Center Connectivity Add-in

If you are working with Quality Center 8.0, this add-in is installed automatically

To work with TestDirector 7.6 or earlier, you must manually install the TestDirector

Connectivity Add-in that corresponds to your TestDirector version on your QTP

Computer

Integration of QC with QTP (Contd):

Note that the Save Test to Quality

Center dialog box opens only when QTP

is connected to a TestDirector Project

To save a test directly in the file system,

click the File System button to open the

Save QTP Test dialog box. (From the

Save QTP Test dialog box, you can

return to the Save Test to TestDirector

Project dialog box by clicking the

Quality Center button)

Saving Tests to a QC Project

Integration of QC with QTP (Contd):

Running the Test Scripts

Mapping the test cases into Test lab

In the Run Stage, for the Host

Tab we have to specify the Host

Number on which you want to

run the test scripts, The Status

Tab will give the status of the

Test Script during the execution

of the Test Script and after

executing the script

Host Number

Integration of QC with QTP (Contd):

When you click on the left side run button in the test lab, you will get one window

which is shown on the right side of this slide. In this window we have a tab called

“Launch Report”; with this tab we will get the Results Summary of that particular

test script.

Disconnecting QTP from Quality Center:

You can disconnect from a Quality Center project or a Web server. Note that if

you disconnect QTP from a Web server without first disconnecting from a

project, QTP’s connection to that project database is automatically disconnected

Integration of QC with QTP (Contd):

54

DEMO

Demo

55

QC Access – IV & VS Lab

IP addresses of the USAA Test lab machines

1) 10.104.57.92 2) 10.104.57.179 3) 10.104.57.123 4) 10.104.57.100

URL to access Quality Center

http://10.103.91.73/qabin/

Login credentials

Login ids - Tester2, Tester3, Tester4 (case insensitive)

Passwords – (do contact Dheenang@hcl.in, Phone 9791037245)

Constraints of usage

1. 15 concurrent licenses

2. Training licenses, Can not be used for evaluation or project purposes

3. Do not use at the cost of impacting assigned work or schedule

56

What is expected from you

Practice all learnt knowledge

Practice the features extensively using such as Synchronization,

Recovery Manager, Verification, Custom report etc.

Bring the learning to use in assigned work / schedule, as applicable

When in doubt about a feature or functionality – please get the

clarifications by

Contacting others around

Contacting speakers of this presentation

Contacting software testing community on arKMedis

Browsing on internet

When you accomplish some thing new – please share the knowledge

On software testing community on arKMedis

With IV&VS academy which takes care of Learning / Growth

57

What is expected from you

A few exercises

Problem – Automate LOS – Module (Create, Edit & Verify) – Mandatory for all

Demonstrate the creation of Automation scenarios for LOS – Module which includes Create, Edit & Verify.

Problem – Usage of Check Points

Demonstrate the usage of SQL, XML check points

Problem – Usage of Recovery Manager

Demonstrate the usage of Recovery Manger features

Problem – QTP Integration with QC

Demonstrate the integration of QC with logging defects via QTP scripting, passing dynamic parameters from

QC to QTP

Problem – Develop Custom Defined Functions to connect RDBMS Data Base

Demonstrate the creation of Custom functions to connect RDBMS Data base & execute queries

Problem – Develop Customized HTML/XML Result file

Demonstrate the creation of Custom functions to create/update user defined result into HTML/XML file

Problem – Result notification via email

Demonstrate the creation of Custom functions to notify the stake holders on Test Execution status via email

58

What is expected from you … continued

Conditions :

Do not combine multiple solutions into one document. Submit separate documents for responses to

each problem

1.No more than 2 people can be teamed up for answering a question. Please mention “the names of the

people” and “problem you are submitting the response for” on the first page.

2.Each response must be provided with the step-by-step explained along with screen shots wherever

possible

3.Use the demo/training version of the licenses for practice purpose. Do not use any client provided

licenses as far as possible

4.Last Date for completion: 20 days from circulation of materials

59

Questions and Answers