©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

Intelligent automated testing with HP Business Process Testing

Susan BockhoffSoftware QA ManagerMcKesson Provider Technologies

Agenda

McKesson and Project Background

Project Goals and Test Automation ROI

Why BPT was Selected

Test Automation Approach and Results

Behind the scenes: the BPT architecture

Creating Business Components

Setting up Parameters

Creating automated tests using BPT

Components

2

McKesson: Who Are We?

Largest healthcare services company in the world

Fortune 15 – $106 billion in revenues (FY09)

More than 32,000 employees dedicated to healthcare

Oldest U.S. healthcare company

Established 1833

Only company offering solutions at every point of care

Deep clinical, IT and process expertise

3

Horizon Clinicals Software Products Manage information across the healthcare lifecycle

4

Patient& Family

Pharmacy

Local

HospitalPediatrician

Primary Care

Physician /

Medical Home

Grandma’s

Physician

Alternative

Medicine

Practitioner

Our Software Is Complex ….

We develop software in multiple locations

─ Westminster, Colorado

─ Alpharetta, Georgia

─ Roseville, MN

─ Bangalore, India

Using many development tools

─ Java

─ .Net

─ GWT

─ Standard HTML

New release contains 35 different applications and over 40,000 tests

5

Why Automate Our Tests?

Revenue: Faster time to adoption will speed license revenue recognition, market penetration

Cost: Reduced cost to execute tests

Agility: Ability to run regression tests more quickly to speed time to market of our products.

Repeatability: Detect regression errors more quickly without human errors or variability.

Coverage: Increase platform coverage on each release.

Increased confidence in our application quality.

6

Status at the Start of the Project

2000 working automated tests using QTP

These took 3 years to develop

The team was using a keyword-driven

framework

Tests were scripted using VB script (no record

and playback)

Tests required a lot of maintenance and many

had to be abandoned

There were many false failures

7

The Challenge!

5000 new working automated tests in one year

─ Must be in production and executed successfully

─ Broad product coverage

Be able to execute the tests in 3 days

Tests must be re-usable over multiple releases

with a minimum of maintenance

Automate first – SME Centric approach

Use automated testing as part of agile

development process

8

Business Process Testing

HP’s Business Process Testing (BPT) product

was selected for this project

The QTP framework had some problems we

wanted to resolve:

─ Tests still required a lot of maintenance

─ Tests required automation engineers to write, and

they did not know the applications well enough to

know what the tests should do

─ When tests failed the failures could not be resolved

by the automation engineers (did not know enough

about the app to know if it was a bug)

9

Advantages of BPT

Components are created by automation

engineers who can create them without really

knowing everything the application needs to do

Tests themselves are created by subject matter

experts who know what the tests need to do and

how best to prioritize them.

Test failures are analyzed by subject matter

experts who know if it is a software defect or a

problem with the automated script itself.

10

How to Select Good Candidates for Automated Tests

App must be in a technology that is automatable

(no apps requiring reading of bitmaps).

Frequently run tests are automated first since

they will give the greatest payback

This requires that a full risk-based analysis be

performed on your regression suite to prioritize

all regression tests

High-priority regression tests (wide and shallow)

are the ones that should be automated for

highest ROI.

11

Frequency of Automated Tests

Environmental Health Check – 1 hr. Can you log

in to all apps? Executed on every build.

Smoke tests – 4 hrs. Can all main pages be

accessed? Executed every two weeks.

Critical automated regression – 6 hrs. can all

major transactions be performed? Executed at

start of regression testing and also last thing

before release.

High priority regression - executed several

times on a release.

12

Test Automation Architecture

Application Under Test

Business Process Testing (BPT)

Component Component Component

TestTest Test TestTest

Resources:

-Functions

-Object Rep.

-Environment

HP Quality Center (QC)

HP QuickTest Pro (QTP)

SME-centricTestAutomation Process

Auto

mation

Engin

eer

Automation Architecture Design

Create Function Libraries

Create Object Repositories

Create Business Components

Drag Components to create tests in QC

Configure Input/Output parameters

Debug tests in QC Test Plan module

Add tests to test set in Test Lab module

Sub

ject M

att

er

Expert

or

Te

st E

ngin

eer

SME-centric Process Advantages

Responsibilities are clearly defined:─ Automation developers define architecture and create

components

─ Subject matter experts create automated test cases

SME doesn’t need to know/use QTP.

Uses very flexible Keyword/Data-Driven approach.

Reduced learning curve due to same approach across different applications/technologies

Applicable to all areas of automated testing, like regression, smoke, acceptance, etc.

Designed for Maintainability

Automated Testing with Agile

Automation Engineers SME Testers

Shared between Agile teams

Create BPT components

during each sprint

Modify components as

needed

Embedded in Agile teams

Write automated tests using BPT

components during each sprint

Execute automated tests at the end

of each sprint

Use BPT tests as regression tests

during later sprints

Benefits of New Approach

Leverage product knowledge of SMEs to write

more meaningful and reliable tests

Increase test coverage -> fewer product

escapes

Ability to keep up with increased product

workload without increase in staff

Reduce time spent analyzing automation failures

Faster execution of automated tests

The Results

The team created 5500 new, working BPT tests

in one year using this approach.

With the same budget, the team increased the

number of automated tests created by 5 times

using this approach.

Automated tests now comprise 30% of our total

different tests, and 60% of the total tests

executed on a given release.

The current project is to get test execution even

faster.

18

Igor Gershovich

President

Connected Testing

19

Automated Testing Process

Quality

Center

QTP AUTBPT Components,

Automation Resources

Results

AUT

Testing

Test case

Data

SETTING UP THE BPT ARCHITECTURE

Step 1

21

Test Automation Elements

IDE: Quality Center with BPT/Mercury QuickTest

Professional (QTP)

Environment Files

Application Areas

Function Libraries

Object Repositories

Business Components and Test Scripts

Environment (BPT Test) for Computer Name–Test Environment Mapping

Computer name – Test Environment mapping

Environment.xls

Function Libraries

Technology libraries─ Web

─ GWT

─ .Net

─ Win32

Common libraries─ Utility

─ DB

─ CommProtocols

Project Specific libraries

CREATING REUSABLE BPT COMPONENTS

Step 2

26

Business Components

A component is a reusable module to perform

tasks on logically independent application parts

(Page or Window).

Components use Input/Output parameters for

data input/verification and navigation within

Page or Window.

Input/Output parameters also are used to pass

data between components.

Components can use iterations.

Login Component Steps(Example for Web Application)

Component initialization

─ Load environment variables like URL, login

credentials, DB connection strings, etc.

Close all browsers except QC

Invoke an application

Login

Verification that login is successful

Functional Component Steps (Example for Web Application)

Component initialization – ComponentInit() function

─ Load environment variables like URL, login

credentials, DB connection strings from

Environment.xls file using QC OTA

Verify that Expected page is displayed

Side or Top menu navigation

Web Page internal flow

BPT Component Script and AUT

AUT

QTP

CREATING AUTOMATED TESTS BY CONFIGURING PARAMETERS WITHIN COMPONENTS

Step 3

31

BPT Test Case in Quality Center

Data

Parameter Naming Conventions

Prefix Object Example

lst List lst_User

txt Edit txt_User

chk Checkbox chk_User

rad Radiobutton/RadioGroup rad_User

tab Tab tab_User

tbl Table tbl_User

cell Cell in Table cell_User

file WebFile file_User

ele WebElement ele_User

cmb Combobox cmb_User

st Static Text (Read Only) st_User

.Apply Method

Object.Apply Parameter(“Parameter_Name”)

.WebEdit(“Name”).Apply Parameter(“txt_Name”)

Apply method functionality:

─ Performs data substitution

─ Verifies if object exists and visible

─ Makes decision what to do with “Name” edit object

based on Parameter data

Input Constraints for Edit (Text Box) Object

Value Action Example

No action

<CLEAR> Clear edit field <CLEAR>

text Enter text Hello

[text] Verify text [Hello]

[<ENABLED>], [<DISABLED>] Verify if object is

Enabled/Disabled

[<EXIST>],[<NOEXIST>] Verify if object exist/no-

exist

[<REGEX>Pattern] Verify that Pattern

matches text

[<REGEX>TestCase.*]

Working with Objects in Grids

row=WebTable().GetRowIndex(Parameter(“search_”),clmn)

WebTable().CellApply row, clmn, objType,

Parameter(“cell_”)

iRow = WebTable().GetRowIndex(Parameter(“search_Name”),2)

WebTable().CellApply iRow,3,“WebElement”,”Parameter(cell_Age”)

Parameter prefixed with the word “search_” indicates

which row to work with.

─ Row can be selected by using row number: #n

─ the text contained in the specified Column

─ Pattern

Parameters for objects in table cells are prefixed by

“cell_”

Example: Working with Grids

Data Substitution

Special tags for Dates, Times, SSNs, Unique

values, DB Queries, Random, etc.

<SSN> - get substituted with unique SSN

<TODAY> - get substituted with today date (03/19/10)

<RNDDIG 3> get substituted with 3 random digits

<DateTime YYYYMMDDHHmmss>

<DB DBEnvironment::SQLStr>

Tags can be combined

[<REGEX><TODAY> at .* AM]

Summary

This method provides a way for test automation

engineers to create re-usable components without

having to know all the ways the application will be used.

Subject matter experts can use the components to easily

create tests that can do anything that the application can

do without having to be software engineers.

This method allowed us to create a great number of

working automated tests in a relatively short time.

The project has been a major success and has helped

us achieve greater confidence in our release quality.

39

Q&A

40

41 ©2010 Hewlett-Packard Development Company, L.P.

To learn more on this topic, and to connect with your peers after

the conference, visit the HP Software Solutions Community:

www.hp.com/go/swcommunity