1

Performance Optimization In QTP Execution Over Video Automation Testing

Speaker : Krishnesh Sasiyuthaman Nair Date : 10/05/2012

2

Importance Of Performance Optimization

• Prevent unnecessary wastage of test resources.• Get faster test results thus reducing cost in

automated testing.

3

Video Testing

• Involves different interlinked components like streaming servers, settop box racks, IR blasters etc.

• Need support from multiple applications and frameworks which provides OCRs, Image comparisons, SNMPs, trace processing etc

4

HPQC-QTP Framework In Video Testing

• Requirements and test cases will be available in QC Test Plan. QC provides a template in Test Plan for storing the test scripts, allows them to be added to QC Test Set, executes and saves result to QC test set.

• Provides feature to store defects in the defects area of QC.

5

QTP Test Execution From QC

6

Problems : QC-QTP Automation Development & Execution

• Interruptions during long execution due to QC timeouts and network issues.

• In the case of multi-location, its not possible to sync the local agile development version repository with mapping and execution of test cases in QC test plan.

7

Problems : QC-QTP Automation Development & Execution Continued

• Manual testers and other QC users will also be able to view the scripts and edit configuration files. This is not necessary.

• Network bandwidth utilization is too much during execution due to frequent connecting local machine QC client to QC server for transferring reports, initiating new runs etc as execution needs multiple active QC sessions.

8

Long Test Execution Cycle Time

In video testing each script will take around 10-15 minutes and a typical release testing will contain around 200-400 scripts , will be around 20-25 box types to validate. On a minimum there will be 4000 test runs and will take around 27 days if executed in one machine.

9

No Support For Agile Development & Coupled Execution

• There is no continuous integration available in QC-QTP default framework.

• In rigorous script development environment when multiple people works on the same library files, there are chances of bugs getting added to the impacted areas of code change and need to executed and validated frequently, with minimal effort, which is not possible currently.

10

Need For Common Local Subversion For Execution

• Since video automation framework using QTP is huge(contains lot of images, xmls , data tables, vbs file, csv files etc ), the execution need to be as local as possible because of multiple reasons.

• Its always an overhead in maintaining script and framework in two locations(subversion & QC) has wastage of time and space.

11

Network Bandwidth Usage Due To Active QC Sessions

As discussed in previous slide an execution can extend up to 27 days, which need to be shortened through parallel executions in multiple machines, say 10 machines it will come down to 2.7 days. But, this will need 10 active QC sessions polling frequently to client machines for execution triggering and result upload. Since framework itself is big, the file transfer between server to client will eat up lot of network bandwidth.

12

Solution : Performance Optimization In Framework Root causes for the problems mentioned in the

previous slides were found out and the converged causes are the following.

– Continuous integration support not available.– Execution need to be triggered locally with minimum

interaction with the QC server.

A test driver created with QTP AOM and QC OTA will be sufficient to solve the major bottle necks.

13

Continuous Integration In Script Development

Continuous integration involves integrating code early and often so as to avoid the pitfalls of ‘integration hell’. Aim is to reduce rework and reduce cost and time. There is a need to have version based execution and this support is added with the help of test driver.

14

Minimal Interaction With QC Server

With local execution framework, the long download times of script and framework files from QC is reduced along with reducing the interaction of QC during execution with local client executing machines. Also a configurable tool reboot time is introduced which removes the unexpected memory leaks associated with long duration executions.

15

Video Test Automation With Test Driver

16

Video Test Automation With Test Driver Continued

• Driver retrieves the details of test set created in QC.• Test lab added test cases with supporting files are

downloaded to local QTP execution machine will be the latest version or specific version in subversion.

• Execution trigger will be added to windows scheduler for daily runs.

• Execution results and logs will be available in QC Test Set on completion of execution.

17

Test Execution Duration Difference

Execution initiated Without Driver Execution Initiated With Driver

Total Scripts Duration Hrs

200 34

400 67

Total Scripts Duration Hrs

200 28

400 54

There is more than 10 hrs execution duration saving during execution. This is a definite advantage in video test automation space, since we can free up the hardware and software tools and resources blocked for this execution cycle & advantage is thus tremendous.

18

Script Level :Performance Optimization

• Each individual scripts need to be optimized separately for better results.

• There are a set of guidelines to be followed in the scripts and tool settings for better performance .

19

Conclusion

Performance optimized framework provides excellent foundation for automated testing in video test automation space. As manual execution is really tiring, the cost benefit a well optimized framework provides is really immense.

20

Question & Answers

21

Reference

• QC, QTP Documentation Library & HP ALM Open Test Architecture Reference.

• Tata Elxsi Automation Information Database.

22

Thank you

ITPB Road, Whitefield

Bangalore, India 560 048 Tel +91 80 22979123Fax +91 80 28411474

e-mail info@tataelxsi.com

www.tataelxsi.com