8/6/2019 2006 05 18 Testing Tools

1/22

Software Reliability and Testing 2006 1

Advanced Testing

Testing Tools

8/6/2019 2006 05 18 Testing Tools

2/22

Software Reliability and Testing 2006 2

Plan

Testing activities and tools

Some open source testing tools

Some commercial/closed source testing tools

Summary

8/6/2019 2006 05 18 Testing Tools

3/22

Software Reliability and Testing 2006 3

Motivation

Use tools to

Reduce time and cost of testing

Increase quality of testing

Ultimately get better quality software for thesame amount of money

8/6/2019 2006 05 18 Testing Tools

4/22

Software Reliability and Testing 2006 4

[Abrain and Moore, 2004]

8/6/2019 2006 05 18 Testing Tools

5/22

Software Reliability and Testing 2006 5

Tools for all test activities

Planning

Coordinate of personnel, test facilities and equipment, ...

Test-case generation

Defined test cases based on level of testing and testing techniques

Test environment development

Support development and execution of test cases, logging, recovery, ...

Execution

Run the actual test cases in the test environmentTest results evaluation

Determine whether the test has been successful

Problem reporting/Test log

Record relevant information on testDefect tracking

Track defects to improve and measure development process

8/6/2019 2006 05 18 Testing Tools

6/22

Software Reliability and Testing 2006 6

Some commercial tools

Some Jolt Awards and finalists

2006

Award VMTN Subscription 2005 (VMware) Runner-ups

Agitator 3.0 (Agitar Software) AQtime 4.7 (AutomatedQA) Clover 1.3 (Cenqua) Parasoft Jtest 7.0 (Parasoft) TestComplete 4.0 (AutomatedQA)

Previous 2005: Agitator and Dashboard 2.0, FogBugz 2004: TestComplete 3.0

2003: TestTrack Pro 2002: JProbe Suite

2000: Parasoft Jtest 1997: SQA Suite

8/6/2019 2006 05 18 Testing Tools

7/22

Software Reliability and Testing 2006 7

JTest

Demonstrationhttp://www.parasoft.com/jsp/products/support/present

ation/flash/jtest/demo/7.0/JTD.html

8/6/2019 2006 05 18 Testing Tools

8/22

Software Reliability and Testing 2006 8

Some open source testing tools

http://www.opensourcetesting.org lists 280tools...

A great variety, e.g., HttpUnit

JMeter

Mock objects

FitNesse

JIRA

8/6/2019 2006 05 18 Testing Tools

9/22

Software Reliability and Testing 2006 9

HttpUnit

http://httpunit.sourceforge.net/

Java framework for integration, and system testing of

web applications

Supports test-driven development for webapplications

Emulates relevant portions of browser behaviour

E.g., form submissions and JavaScript

Example...

8/6/2019 2006 05 18 Testing Tools

10/22

Software Reliability and Testing 2006 10

AccountServer

8/6/2019 2006 05 18 Testing Tools

11/22

Software Reliability and Testing 2006 11

AccountServlet

8/6/2019 2006 05 18 Testing Tools

12/22

Software Reliability and Testing 2006 12

http://localhost:8080/account

8/6/2019 2006 05 18 Testing Tools

13/22

Software Reliability and Testing 2006 13

Using HttpUnit

8/6/2019 2006 05 18 Testing Tools

14/22

Software Reliability and Testing 2006 14

Using HttpUnit

8/6/2019 2006 05 18 Testing Tools

15/22

Software Reliability and Testing 2006 15

JMeter

http://jakarta.apache.org/jmeter/ Performance measurements on use of static and

dynamic resources Files, Servlets, scripts, Java objects, data bases, ...

GUI for defining and visualizing test plansThread groupsemulate concurrent users

Samplersdefine input from thread groups (e.g., HTTPrequests)

Listenersused to capture, analyze, and visualize test runs

Example...

8/6/2019 2006 05 18 Testing Tools

16/22

Software Reliability and Testing 2006 16

Using JMeter

8/6/2019 2006 05 18 Testing Tools

17/22

Software Reliability and Testing 2006 17

Using JMeter

8/6/2019 2006 05 18 Testing Tools

18/22

Software Reliability and Testing 2006 18

Mock Objects

http://www.mockobjects.com//http://www.jmock.org/

Library for testing Java code using mock objects

Mock objects

Given an interface create an advanced stub atruntime using reflection

May define expected values on mock objects using

constraints

Example...

8/6/2019 2006 05 18 Testing Tools

19/22

Software Reliability and Testing 2006 19

Using jMock

8/6/2019 2006 05 18 Testing Tools

20/22

Software Reliability and Testing 2006 20

FitNesse

http://fitnesse.org/

Wiki-based collaborative tool for defining acceptance tests

E.g., testing a calculator thrugh a ColumnFixture

eg.Division denotes a fixture class that is run by the

FitNesse server

8/6/2019 2006 05 18 Testing Tools

21/22

Software Reliability and Testing 2006 21

JIRA

http://www.atlassian.com/software/jira/ Issue and defect tracking and management

A JIRA installation Is web-based Covers a number of projects

Projects have issues, versions, components

8/6/2019 2006 05 18 Testing Tools

22/22

Software Reliability and Testing 2006 22

Summary

The is a huge variety of testing tools for all parts of thetesting process

Many open source tools exist also of relatively high quality Complements commercial testing tools

Many are domain-dependent

We saw/have seen some examples JUnit, AETG, JTest, HttpUnit, JMeter, FitNesse, JIRA

No substitute for common sense and hard work though...