Automatic test automationAutomatic test automationWhat we do, how far we got, and where do we want to go

2

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

3

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

4

Targets: TestautomationTargets: Testautomation

Testautomation of the essential business processes in order to confirm the correct functionality of the software

to simulate the essential business processes covered by the software to apply fully automatic tests of very complex business processes to significantly reduce staff required for the manual test Shorter lead times, optimizing the use of available testing ressources Precise repeatability of test scenarios across multiple releases

Testresult reliabilty Increased product stability and quality Increase the reliabilty of the test performance Increasing the test coverage level

Documentation of test results Automatic generation of test results documentation

5

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

6

How far did we manageHow far did we manage

Testautomation of the essential business processes in order to confirm the correct functionality of the software

to simulate the essential business processes covered by the software to apply fully automatic tests of very complex business processes to significantly reduce staff required for the manual test Shorter lead times, optimizing the use of available testing ressources Precise repeatability of test scenarios across multiple releases

Testresult reliabilty Increased product stability and quality Increase the reliabilty of the test performance Increasing the test coverage level

Documentation of test results Automatic generation of test results documentation

7

How far did we manageHow far did we manage- the essentials- the essentials

We had to find solutions to these problems: Business processes do not pay attention to the input methods of the

software- Some of these input methods are not supported by eCatt ( IDOC and

batch processing)

In order to achieve fully automatic regression tests we must find a way to produce test data dynamically without human interference

In order to achieve absolute test result reliability, we had to find a way to remove factors which would reduce the sharpness of the test processes

The complicated and complex business processes consist of a large number of steps which need to be performed in a controlled sequence and to exchange parameters in a controlled and secure manner

8

How far did we manageHow far did we manageinput methods input methods

Examples of test processes with various input methods

Dialogue – „normal“ online user transaction – easily supported by eCatt

Batchprocess – Dunning ( occasionally with a user interface to type in the information to be processed – but the processing itself is batch and asynchronous

Creation and processing of input files

Creation and processing of IDOC files

Automatic database inspection checks

9

How far did we manageHow far did we manageDynamic supply of test dataDynamic supply of test data

Some „trivial“ test data problems

Which date do I need for this dunning test proces? Which date is it in two months? How do I know if this person has already been „inserted“ into the system How do I know, which date it is in 20 days – also if it needs to be a bank day

10

How far did we manageHow far did we managetest result reliabilitytest result reliability

How to reach test reliability:

You need to know everything about the test data How old it is Which attributes it has Each and every attribute Each and every connection or link to other data elements

11

How far did we manageHow far did we managecomplex business processescomplex business processes

The means of referenced eCatt scripts does not offer a configurative flexibility

The referenced eCatt scripts would not be the right means for the input methods which can not be processed with eCatt

12

How far did we manageHow far did we manageRequirements - summaryRequirements - summary

If you want to control your test process, you must create and process all required data yourself. You cannot rely on the quality of data already in the system.

You need to be able to produce valid test data automatically and supply this test data also automatically to your test process.

You need to be able to service different kind of input methods in order to be able to offer test of entire business processes.

If you want to provide a fully automatic test service you must focus on relatively low maintenance.

13

How far did we manageHow far did we manageSolution – head linesSolution – head lines

Our test processes generate and, where required, process their test data themselves.

Our test processes support different input methods.

The steps of our test processes are linked together and exchange parameters through configurations (rather simple table entries).

Due to the configuration ability we concentrate our maintenance on changes in the software products which we test.

14

How far did we manageHow far did we manageSolution – head linesSolution – head lines

We developed additional SAP functions in order to embed the SAP eCatt standard functions into a solution which would fit our requirements.

- The addtitionals provide several test configuration functions

- Test process definition

- Test steps parameter exchange

- IDOC configuration

- Setup of test files

- Etc Test performing functions

- Start up and scheduling of test processes (not available in the eCatt environment)

- Utilities to start up the test steps in the correct sequence Test result displaying functions

- Various views on the results of the performed tests

15

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

16

Fully automatic test of business processesFully automatic test of business processesBasic exampleBasic example

One testprocess contains several test steps

The steps of the testprocess are linked together through table entries

Each test step produces parameters

These parameters can be picked and processed up by succeeding steps

The test steps are also supplied with test data from the standard eCatt test data container

17

Fully automatic test of business processesFully automatic test of business processesTestprocesses – Monitoring overviewTestprocesses – Monitoring overview

18

Fully automatic test of business processesFully automatic test of business processestestprocess – test stepstestprocess – test steps

19

Fully automatic test of business processesFully automatic test of business processestestprocess – test stepstestprocess – test steps

20

Fully automatic test of business processesFully automatic test of business processestestprocess – test parameters and valuestestprocess – test parameters and values

21

FullyFully automatic test of business processesautomatic test of business processesThe parameters are picked up by suceeding stepsThe parameters are picked up by suceeding steps

22

Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots

To minimize maintenance efforts, it is very important, to create the test scripts as modular as possible. For example, there is the business partner dialogue; in oscare ®, there are 39 different roles for bp’s and according to most of them, the dialogue is different, mainly concerning the number of tab strips, which ones are to process and their order.

Hereinafter two examples are shown by screenshots, at first a school and then an employer are created.

Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of a school as a business partnerCreation of a school as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

Fully automatic test of business processesFully automatic test of business processesCreation of an employer as a business partnerCreation of an employer as a business partner

38

Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots

To cover all different requirements with one script, it is neccessary, to encapsulate every tab strip within one single script. All of those scripts are referred within a so called ‚masterscript‘. The testdatacontainer is used, to manage the process by different flags, like for example ‚I_DO_PARTNERINFO‘. If the flag is set, the relevant tab strip is processed, otherwise not.

Moreover, the same tab strips can have different positions, depending on the role. To handle that fact, a table had to be developed, where the information is stored, which tab strip can be found at what place (according to the role), what is it‘s name and what is it‘s ID.

39

Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots

Below all entries for the ‚Partnerinfo‘-tab are shown:

40

Fully automatic test of business processesFully automatic test of business processesSome examples – screen shotsSome examples – screen shots

Here the protocol entry, that was originated while choosing the ‚Partnerinfo‘-tab strip during runtime, when name and id were determined:

41

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

42

Fully automatic test of output management Fully automatic test of output management What was the problem and how it was solvedWhat was the problem and how it was solved

A precise and reliable test of the correct generation and formation of RDI and XSF files is rather problematic.

The requirements describing how these fields supposedly should be populated were defined and stored in testautomation.

Functions to examine the correct population of the fields of these output files were developed.

Most of the required business processes had already been test automated.

Eventually we just had to add the output function and connect them to the field inspection functions.

43

Fully automatic test of output management Fully automatic test of output management Two kinds of field inspectionTwo kinds of field inspection

Mainly in order to avoid loosing control of the automated output inspection checks we split the checks up in two categories.

Appearance checks – does the output field actually occur in the output stream?

Content checks – does the output field hold exactly the expected value?

44

Appearance checksAppearance checksRDIRDI

Target: Do all required fields occurr? Did unexpected fields occurr?

How it works: We test automate the business process The steps of the business process

populate the data fields, which should eventually show up in the output streams.

The OPM field documentation of the data streams is transferred to a simple R/3 table

The appearance checks are appended to the business and print part process

OPM checks

Business

process

Printing part

45

Appearance checksAppearance checksExamplesExamples

Comparison with the OPM documentation in order to check that no unexpected fields occurred

Selective check in order to inspect if these three fields occurred.

46

Content ChecksContent ChecksHow it worksHow it works

The content check contains two components

Extraction of the expected field content Configuration table entries set up exactly from which teststeps, exactly which parameter

the expected content of this output field holds. The function that extracts the expected field content reads these configuration entries

and picks out the parameter values.

Comparison - expected with actual output stream fields This function compares the expected field content with the actual stream fields.

47

Content CheckContent Check

OPM checks

Businessprocess

Print part

48

Content ChecksContent Checks

Extraction of expected values

49

Content chekcsContent chekcs

Comparison – expected - actual

50

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

51

JAVA

ABAP

ORACLEFORMS

ORACLE

JSF

WebDynpro Visual

ComposerWD4J

JSP

WD4A

CRM2007WebClient

UI

BSP

CompositionEnvironment

Composition Tools

AS Java

WebDynpro

SAPGUIWebDyn

pro

Webbrowser

NWBC

Portal

PerspectivesWhere do we go from here

52

AgendaAgenda

Our original target

SAP Tools or non SAP tools for testautomation

Fully automatic test of business processes

How far did we manage

Fully automatic test of output management

Perspectives

53

SAP-EcattSAP native free to use toolVery integrated automation capabilities and interfaces

No serious support of non SAP GUI DialoguesTosca

Acquisition and / or licensing costs

„Videocam“-Technology Films everything also WEB dialogues (understands nothing)No SAP-Integration No SAP InterfacesVery limited (if at all available) test result checks

Compuware und MercuryHigh priced and licensing costs

Films everything also WEB dialogues (understands nothing )

No SAP-IntegrationNo SAP Interfaces

Very limited (if at all available) test result checks

SAP or non SAP tools for testautomation

54

No non SAP product provides a professional SAP integration

With these products you won‘t be able to conduct

Regression tests without manual processing of test data - (this is already the first knock-out)

A detailed query or inspection of the SAP Database content, in order to evaluate the result of yout test operation. (this is how it got the second knock-out)

supply an exact repeatability without sacrifizing the test reliability

If you want to try it anyway, it would require extreme ressources, furthermore the question: "Why don‘t we do the whole test manually anyway" would have to be answered

SAP or non SAP tools for testautomation

55

eCATT or different SAP tools for testautomation?eCATT or different SAP tools for testautomation?

Supported protocols : SAP GUI WebDynpro Java Web, SAP-Web (Portal) Win32 .NET Terminal (3270, 5250) PowerBuilder Oracle Forms, Oracle Applications, Siebel,

PeopleSoft MS Office, VB, Visual Age

License costs : depends

SAP Quick Test Professional by HP

Supported protocols: SAP GUI WebDynpro Java WebDynpro ABAP (only selected objects) SAP Web Services

License costs: none

SAP eCATT