test management for business suite on hana migration projects using sap solution manager 7.1

7/21/2019 Test Management for Business Suite on HANA Migration Projects using SAP Solution Manager 7.1

http://slidepdf.com/reader/full/test-management-for-business-suite-on-hana-migration-projects-using-sap-solution 1/75



© 2013 SAP AG. All rights reserved.

Agenda: Test Management for Sui te on HA NA migra t ion pro j

Introduction and Project Phases

Phase 1: Test PreparationBusiness BlueprintTest Scope IdentificationFunctional Regression TestsPerformance and Load TestsService and Interface Tests

Phase 2: Test Execution

Phase 3: Test Activities after Go-Live

Key Take Aways



Introduction and Project Phases




Test Management for Sui te on HA NA migra t ion pro jec ts HANA migration project phases and involvement of test management

Preparation phase Suite on HANA migration project aft

Suite on HANA Migration Project

Test Management

Status assessment

Tool & approachdecisons

Test managementsetup

Test planning

Test execution

Issue resolution

Retest and sign-off



Phase 1: Test Preparation


http://slidepdf.com/reader/full/test-management-for-business-suite-on-hana-migration-projects-using-sap-solution 6/75© 2013 SAP AG. All rights reserved.

Test Management for Sui te on HA NA migra t ion pro jec ts Test Management status assessment and setup activities (1)

WorkstreamBusiness Blueprint

WorkstreamTest Scope

Identification

WorkstreamTest Planning

• Coverage of business processesdocumented

• Identification of Business Analysts

Assessment & Decision Setup

Interviews with business

teams to identify prio 1

business processes

Business processes and involved• RBPD service or

• SAP SolMan: Solution Documenta

• SAP SolMan: Scope and Effort An

Metrics for test scope identification - examples:

• priority or criticality of business process

• # of affected users

• usage frequency of business process

Functional Test Plan 1 Test case selection frombusiness processes with

class A metrics

Ftebucu

•

••

• Status test system (PRD copy, test data , …)

• Availability of business users for tests

• Reporting requirements

Test system setup• Definition of Test Packages f

• Tester assignment

Optional: metricassignment to business

processes(ABC classification)

WorkstreamFunctional

Regression Tests

• Coverage assessment of manual andautomated tests

• Target level for test automation, e.g. 30%

• Test tool evaluation and tool selection +. ROIexpectations

Optional: Implementation

project for selected test

automation tool

Test case definition

• selected functional tests for executi

• E2E integration tests for execution



Test Management for Sui te on HA NA migra t ion pro jec ts Test Management status assessment and setup activities (2)

WorkstreamPerformance Tests

Load Tests

WorkstreamService &

Interface Tests

WorkstreamReporting & Sign-off

Assessment & Decision Setup

Metric selection for load profile definition• Top (15 ) transactions, batch jobs, reports

creating 80% of system load

• performance critical transactions

• Throughput of critical transactions

• peak hours and maximum number of users

• Transactions with high number of affected users

• Processes with custom code or modifications

Definition of Load Profile• high number of online users p

in parallel

• throughput of processes (with

• critical processes including C

Performance Baseline define baseline for comparisons

• Workload Monitor (ST03N)

• SAP Workload (STAD)

• Performance Analysis (ST05)

Creation of Performance TesDefinition of scaling factorfor PRD versus TST system

using SAPS ratioOptional: DB Stress Testsperform high volume DB acce

Identification of non-assigned systems

• Identification of SAP and non-SAP systemsrequired, but not connected to HANA test system

• Tool selection for Service (system) Virtualization

Identification of critical Services and APIs

• Identification of (web) services and APIs whichrequire tests without UI (e.g. w/o calling App) andmultiple use cases / data variants

• Tool selection for service testing

Service (system) Virtualization

• SAP recommendation: connect required system if possible and avoid syste

• Optional: virtualization of not connected, but required systems using syste

Service Testing

Creation of service test cases to perform automated tests for (web-) services application. Data variation to perform various regression tests of the service /

• Definition of Reporting Requirements

• Decision: use of add. reporting featuresSelection of reports / optional: setup of add.reporting




1a - Business Blueprint



SAP Solution ManagerSolution Documentation of System Landscape and Business Processes

System landscape documentation

SAP ERP

DEV

TST

PRD

TST

DEV

PRD

Single source oftruth

Infrastructure

SAP CRM

Business process documentation

Non-SAPbusiness

apps

http://localhost/var/www/apps/conversion/tmp/scratch_5/Demos/01_Demo_4_1_EBB_short_demo.exe

http://localhost/var/www/apps/conversion/tmp/scratch_5/RunSAP%20Partner/MM_PPT_Demos/Kurzdemos/01_Demo_4_1_EBB_short_demo.exe



Business BlueprintE2E Business Processes and automated generation of SAP Module oriented Blueprint

Business Blueprint with E2E business processes

Alternatives to create E2E Business Blueprint

1. File upload using existing process documentation

2. Solution Documentation Assistant (tool) and ReverseBusiness Process Documentation (service)

3. Data transfer from ARIS or HP QC (tool by ALM COE EMEA)

4. Manual documentation supported by BPR (SAP content)

Automated generation of SAP Module oriented B

1. Application: Scope and Effort Analyzer

Availability: SAP Solution Manager 7.1 SP

Blueprint items generated in background bstatistics of executables in PRD systems

Grouping by SAP Application Component

Business Blueprint with process steps by SASales

Order to Cash

Quotation

Sales Order

Delivery

Goods Issue

BillingProcurement

Procure to Pay

LE Logistic Execution

LE-SHP ShippingDelivery (VL01N)

Goods Issue (VL02

SD Sales & Distribution

SD-BIL BillingBilling (VF01)

SD-SLS Sales

Quotation (VA21)

Sales Order (VA01

E2E business process



Business BlueprintCreation of Business Blueprint using Solution Document Assistant of SAP Solution Manager 7.1

SAP Solution Manager

Projects

SAPCRM

SAPSolMan

OtherOther

Solution DocumentationAssistant

SAP Solution LandSAPERP

Content Interface

1

3 4

1 - Upload business process structure

2

2 - Analyze managed systems3 - Create and complete project (cust4 - Hand over to solution

RBPDContent




Business Blueprint Automated generation of SAP-Module oriented Blueprint - Result

Result

• 3 level Blueprint hierarchy

• Level 1: SAP Module

• Level 2: ACH L2

• Level 3: Transaction orReport

• Tab „Transaction“ withassigned executable

Custom Code

• Node „Customer“

• Custom Code transactionsand reports are grouped bydevelopment class




Business BlueprintComposition of E2E Business Processes based on SAP-Module oriented Process Step Library

From Process Step Library ….

… to E2E Business Processes

Automatic generation

Manual composition of E2E bu




1b - Test Scope Identification




Test Scope IdentificationFunctional Test Plan #1: Metrics used for identification of important business transactions

Step 1: Identification of metrics

• Prioritiy / criticality of business process• Number of affected users of business process• Usage frequency of business process• …

Step 2: BPx guidance

• ABC classification of business processpriorities through interviews with Businessprocess Experts (BPx) of all business areas

Step 3: Usage frequency analysis• Assess the usage frequency with S

Workload Monitor (ST03N)• ABC classification for top n busin

transactions

Step 4: ABC classification for Bus

• Definition of custom attributes for B

Blueprint in SAP Solution Manage• Assign attribute value for business

„A“classification (from step 2 and

• high priority / criticality

• high number of users

• high usage frequency




Test Scope IdentificationFunctional Test Plan #1 for important business transactions

Test Option 1:

SAP Solution Manager with integrated partner products

B u s i n e s s B l u e p r in t Manual and au tomated Tes ts

Manual Test

Automated Test

Test Plan generat ion - alternatives:

1. filter all available test cases by Blueprint process attribute

2. filter Blueprint by keywords which are assigned to test cases

Functional Test Plan #1

Test Option 2:

SAP Quality Center by HP plus SAP TA

Bus iness Bluepr in t StrC

QC T

• Ma

• HP

QC Test Set generat io

Functional Test Plan




Optimize custom code and modificationsMotivation

Main drivers for optimizing custom code for the Suiteon HANA migration:

• Discover the potential and take the opportunity todrastically improve the performance of long runningtransactions using HANA functionality

• Restore the performance of transactions due toperformance impacts of HANA migration

• Fix functional degradation due the HANA migration

There is no need to adapt everything!Identify what needs to be done and whereimprovement potential is available




Test Scope IdentificationFunctional Test Plan #2 for Custom Code

ABAP Test Cockpit (ATC) Usage and Procedure Logging (UPL)

Approach to identify Custom Code with potential performance improvements

SQL M

SAP recommendation: define Regression Test Plan #2 for adjusted code objects

• find violating SQL statementswith static code analysis

• fix functional & performancedegradation by adaption of

ABAP/OpenSQL Statements, likeSELECT for Pool/Cluster-Tables w/oORDER BY

Native SQL code (Critical Statements)

DB hints (Critical Statements)

ATC results: functional prio 1issues

• UPL logs all called and executed ABAPunits like programs, function modulesdown to classes, methods andsubroutines

ATC results + UPL:

• exclude unused custom code fromanalysis

• limit analysis to high usage areas

• Dynamic runstatements

• Aggregated lbased SQL e

• Ability to idetables with h

Performanc

Narrow dow




Test Scope IdentificationFunctional Test Plan 2 for custom code: ATC

ABAP Test Cockpit (ATC)




Test Scope IdentificationFunctional Test Plan 2 for custom code: ATC





Test Scope IdentificationFunctional Test Plan #2 for custom code: Usage and Procedure Logging (UPL)

Usage and Procedure Logging (UPL)

UPL is a kernel based logging technologyproviding runtime usage information of ABAPprocedure units like methods, function modules,subroutines and much more …

UPL complements the standard ST03N workloadstatistics of ABAP executables

UPL provides 100 % reliable usage analysiswithout measurable performance impact

UPL is available as of SAP Netweaver 7.01SP10 with Kernel 720 Patch 94

1

2

Eva

Ex

p

L3

4

SAP Business Suite

System like SAP ERP

(system role: PRD)

SAP Solution

Manager




Test Scope IdentificationFunctional Test Plan #2 for custom code: Usage and Procedure Logging (UPL)

Usage and Procedure Logging (UPL)




Test Scope IdentificationFunctional Test Plan #2 for custom code: customer example (modified)

ATC scan pattern FindinSELECT for Pool/Cluster-Tables w/o ORDER BY

Native SQL code (Critical Statements)

DB hints (Critical Statements)

Analysis of WHERE Condition for SELECT 1-1

Analysis of WHERE Condition in UPDATE and DELETE 0

Changing Database Accesses in Loops

EXIT or no statement in SELECT...ENDSELECT loop

SELECT Statements with Subsequent CHECK 0

Incomplete evaluation. <x>% of fields used <10

SELECT Statements That Bypass the Table Buffer >10

Table Attributes Check

SAP recommendation: define Regression Test Plan #2 for adjusted code objects

1

2

Customer example (modified)

• The company activated UPL well in advance

to the Suite on SAP HANA migration project• > 1000 used Custom Code objects detected

by UPL

• ABC classification for CC usage (3 months)

• Class A: > 100 mio executions

• Class B: > 1 mio executions

• ATC code scans results limited to CustomCode of usage frequency class A and B

ABC classification of CC usage frequency v

CC adjustments for prio 1 ATC results limit




Test Scope IdentificationFunctional Test Plan 2 for custom code: SQL Monitor

SQL Monitor

# executions # records selected

table name

High # executions

Zero # records

High # records

High mean time




Test Scope IdentificationFunctional Test Plan #2 for Custom Code


Approach to identify Custom Code with potential performance improvements

SQL Monitor (SQL-M)

SQL Performance Tuning Worklist

• Combines ATC and SQL-M in one application

• Transaction code: SWLT

• Availability with SAP Basis 7.02 SP14 / 7.38 SP09 / 7.40 SP0




Test Scope IdentificationFunctional Test Plan 2 for custom code: SQL Performance Tuning Worklist (T-Code: SWLT)

SQL Performance Tuning Worklist• Entry Screen




Test Cope IdentificationFunctional Test Plan 2 for custom code: SQL Performance Tuning Worklist (T-Code: SWLT)

SQL Performance Tuning Worklist

correlates• Static Code Checks covering golden rules from

ABAP Test Cockpit• Runtime information (numbers of execution,

execution time, fetch count,…) from SQLMonitor

Create a weighted work list of codeoccurrences that need attentionProvide solution proposals




Test Cope IdentificationExample for prioritization and resolution using SQL Monitor

SQL Performance Tuning

Worklist provides informationabout• Runtime information (from SQL

Monitor)• Static Checks from ABAP Test

Cockpit (ATC)

… and includes • starting point of executable,

URL, RFC,… (e.g. transactionVA01)

• detailed information on how toimprove the ABAP statement

d f




Test Scope IdentificationFunctional Test Plan 2 for custom code: Business Process Change Analyzer (BPCA)

SAP ApplicationGoal Activ

Optimal performanceafter HANA m igrat ion

• ABAP Test Cockpit (ATC)

• Usage and Procedure Logging(UPL)

• SQL Monitor

• SQL Performance TuningWorklist (SWLT)

• IdeCowista

• Adpri

No negat ive impacton c r i t i cal bus iness

processes incomb ina tion wi theffort opt imized

regression test scope

• Business Process Change Analyzer (BPCA) - SAPSolution Manager

• SAP and partner test tools

• Idebuimadj

• Te

• Reim

T S Id ifi i




Test Scope IdentificationFunctional Test Plan 2 for custom code: Business Process Change Analyzer (BPCA)

1 2

3

4

Business Blueprint Custom Code and Modification adjustme

Tranadjusted Custom Co

BPCA

TBOM (*)Tests (**)

(*) Semi-dynamic TBOMs generation fully automated with SAP SolMan 7.1 SP10 (**) Test cases for HP QC integrated with Business Blueprint and BPCA via SAP-HP interfaces

Change impact analysis

• selection of adjusted custom code / modifications from transports

• Selection of business processes, TBOMs, test cases

• BPCA change impact analysis plus test scope optimization

Test Plan generation (semi-automatic) in SAP Solution Manager or SAP Quality Center




1c - Functional Regression Tests

T t M t




Test Option 1

Test Option 3

SAP Connector, Quality Manager, Requirements Composer, Team Co

Business Blueprint

BPCA

Business BlueprintTest Planning

Test System Setup Test Execution

Change Impact AnalysisSAP Solution

Update

New SAPSolution

Test ManagementProducts and capabilities integrated with SAP Solution Manager 7.1

Test Workbench Test Automation FHPCBTA

Quality Center Business Process TeHPSAP TAO

Test AutomationTest Management

Test AutomationTest Management

Test Option 2

http://www.notebookcheck.com/typo3temp/pics/11b0cf7872.jpg





Test Ooption1 using SAP Solution Manager 7.1

Test Management Test Option 1 using SAP Solution Manager 7




Test Management – Test Option 1 using SAP Solution Manager 7.Capabilities of Test Option 1 with SAP Solution Manager 7.1 (SP07 and beyond)

Business Blueprint

Business ProcessChange Analyzer

(BPCA)

WoC Test Management

• Manual Tests

• Automated Tests

SAP TDMS

Test Planning

eCATT

WoC Test Managmen

Test Workbench

BW Reporting

Partner Logs and Repor

Test Status Reporting

Test script creation

Test system assignment

Test data assignment

Test Automation Framework

Test Case Assignment

other 3rd party test automation tools

Test Scope Optimization

Test Execution

HP QTP

SAP Solution Manager 7.1

Lean test system setup

CBTA (Component based Test Automation)

Business Blueprint

Test PlanningTest System Setup Test Execution and Reporting

Change Impact Analysis

WS Certify

WoC Test Management

• Test Plan generation

• Tester assignment

Manual Testing




Manual TestingManual Test Case for business transaction with description of execution steps and access to test dat

Note: manual test case is assigned to process step in Business Blueprint via T-Code SOLAR02

Manual Testing




Manual TestingConvinient access for manual Testers

12

3

Tester Worklist

1) Easy access formanual testers fromWork Center TestManagement

2) Test Packagesassigned to Testerincluding status

3) Test Cases of selectedTest Package- Start of execution- Test status- Test Notes- Incidents

1

2

3

Manual Testing




Manual TestingTest Execution: user reads test case decription

Manual Testing




Manual TestingTest Execution: user views test data to be used during test execution

Second window with view of available teTDC stays open during test execu

Manual Testing




Manual TestingTest Execution: guided access to System Under Test (SUT) to start execution of business transactio

Screen 1: test script Screen 2: test data

Test execution

Screen 3: automatic launch of transaction that shbe tested by user in preselected test system

T A i i h SAP S l i M 7 1




Test Automation with SAP Solution Manager 7.1

Test Design

Seamless integrationbetween SAP Solution

Manager and test toolsfrom SAP and Partners

Test Data Container(TDC) provide test datafor test script parameters

Central management ofSystem under Tests

Test Automation Framework

Test Execution

Start of automated tests

from Workcenter Test

Management

Ad-hoc start or scheduler

for unattended execution

Integrated test execution

logs from partner tools

Test Result Analysis

Test Manager can view

test results from manual

and automated tests in

one environment

Status and progress

resports

Integrated partner reports

Accel

Wor

and T

trigg

Rich

Engi

prob

repa

Non-SAP

Test Automation Framework in SAP Solution Manager 7.1




Test Automation Framework in SAP Solution Manager 7.1 Automated tests assigned to process steps in Business Blueprint

Test ScriptBusiness Process

HP QTP

Test Data

Test Configuration

other ISV testautomation tools

Order Type Customer Product

OR C1000 P-100

OR C2000 P-200

Test Data Container

SAP customers have a choice of test automation tools integrated with SAP SolutionManager 7.1 through the Test Automation Framework

CBTA, eCATT

Component Based Test Automation ( CBTA ) is shipped with SAP Solution Manager 7.1 SP07

Customers with SAP Enterprise Support contract can download 2 HP QTP licenses from SMP (https:/service.sap

Test Automation Framework provides 2 interfaces to integrate ISV test automation tools. SAP ICC offers certification services

Component-based Test Automation (CBTA)




p ( )Test Automation of E2E Business Processes using SAP Solution Manager - CBTA

Component-based Test Automation (CBTA)

Business An alys t executes b us iness

t ransac t ion v ia CBTA

Test Composition Environment

Business Process “Order to Cash” Quotation Sales Order Delivery Goods Issu

CBTA crea tes tes t sc r ip t b ased on tes t

c o m p o n e n t s d u r i n g e x ec u t i o n

Attributes• Business Trans• Systems under

Parameters • Parameters for• Dynamic hand

ScriptComponents

• SAP delivered• On the fly gen

Test Data• Data defined in• Test Data Assi

Automated Tests

http://localhost/var/www/apps/conversion/tmp/900_Rollout_CDs/mySAP_ERP_CGV_Rollout_CD_intern/1_Corporate_Governance/MIC/P2P_3_Purchase_Requisition.ppt

http://localhost/var/www/apps/conversion/tmp/900_Rollout_CDs/mySAP_ERP_CGV_Rollout_CD_intern/1_Corporate_Governance/MIC/P2P_4_Purchase_Order_Processing.ppt

http://localhost/var/www/apps/conversion/tmp/900_Rollout_CDs/mySAP_ERP_CGV_Rollout_CD_intern/1_Corporate_Governance/MIC/P2P_5_Goods_Receipt.ppt








Flow to create new automated test using Component-based Test Automation (CBTA)

Business Process Test Composition Environment (TCE)create new Test Configuration

Component Based T(C

CBTA – Test C

CBTA – proce

CBTA – Test Com

QuotationTest Composition Environment (TCE)

finalize Test Configuration

Test Data Container

Test Data Assignment Wizardassign test data

Store Test Configurationat process step orbusiness process

Functional Regression Tests







gTest data planning

21 Test Data Container (TDC): Data Structure

Test Data Container(s) act as cent ra l repos i tory for your test dataCustomer benefit : test data changes can be done in one central location leading to

significant lower maintenance effort and faster availability of test data

How : create TDC parameter for all input fields of

your business process used in test script

Who : Test Engineer

TDC: test data planning

test data entry

Who : Business Analysts

Use : Test data stored in a TDC can beby multiple Test Configurations.





Test Configuration

TesSAP CBT

System

Test Data Container (TDC)

Logon abusiness

Test dataselection

TDC

Test execution:handover of Test Script+ Test Data + SUT info

User startstest execution

gTest execution of automated tests

Test Script Test Data System Data

Tester Worklistin Workcenter Test Management

Test results and logs

1

2

3

5

4





gTest Reporting with SAP Solution Manager: Gap, Status and Progress Analysis

Test Status Reports Test Tool LTrend and Progress Reports

Gap Analysis Quick Status Overview

Test Resu

Key benefits: Leveraging SAP Solution Manager for all status

and results reporting, limiting investment in 3rd party tools





Test Ooption2 using SAP Solution Manager 7.1,HP Quality Center and SAP TAO





Products and capabilities of Test Option 2 with SAP Solution Manager 7.1

Quality Center• Requirement• Test Plan• Business Process Tests

SAP TDMS

Test Option 2

Business Blueprint

Test PlanningTest System Setup

Test ExecutioReporting

Change Impact Analysis

Solution ManagerBusiness Blueprint• Process documentation• Business requirements

Solution ManagerBPCA• Change Impact Analysis• Test Scope Optimization

Adapter

Quality C• Test La• Sprinte• QTP• Defect

• Blueprint

• Defects / test results SAP TAO• Optimized SAP test scripts

• Accelerated maintenance• Load and

performance

LoadRu

• Setup of lean test systems





Manual Testing: HP Sprinter

Floating windows with test stepguidance for Tester

Smart Defects

Recording of User Actions (list) andStory Board (screenshots)

Full integration with Quality Center forTest Script selection and results





Test Automation of Business Processes using SAP TAO

SAP TAO Process Flow Analyzer

Bus iness Analys t executes

bus in ess t ransac t ion f rom SAP TAO

Test Case* cTest Componenparameters for

Test ComponeSAPGUI subsparameters for

File with Tesinput paramet

SAP Quality Center by HP

upload

Business Scenario “Order to Cash” Quotation Sales Order Delivery Goods Issu

(*) Note: Further test case refinement might be needed e.g. Test engineer may add more validation steps and consolidate the test case forperformance improvement.

SAP TAOh b l











Test Execution with SAP TAO built test scripts


SAP Test System

During test exis entered inthe SAP bu

Test Script execution

Test data file

Iteration Result

# 1 Passed

# 2 Passed

# 3 Failed

Test results

SAP TAO T A i f E2E B i P




Test Automation of E2E Business Processes

Variant 1Quotation Sales Order V1

C1 C2 C3

SAP TAOTest Components

Test data

Delivery Goods Issue Billing


C1 C2 C3


C4

Test data

SAP TAO test scripts

Test script variants can be built easily via additional Test Components, Flows and Test Data

Scenario tests can be built easily through composition of the appropriate single test

SAP TAO Test Automation of E2E Business Processes





C1 C2 C3

SAP TAOTest Components

Test data



C1 C2 C3

Delivery Goods Issue Billin

C4

Test data

Typical software changes affecting automated tests

1. Customizing changes for process step (optional field mandatory field, ...)

2. Custom Code changes for process step (additional mandatory fields, additional screens, ...)

Customer benefit: SAP TAO provides acceleratedrepair functionality through rapid and semi-

automated exchange of central test components

Test exec

Functional Regression Tests T t ti i id t g t d t t lt




GoalTest Execution of manual and automated tests

* additional license required

Test Lab – execution and documentation of

Defects – managing incidents

Test execution, incident management and test results

Functional Regression TestsSAP Quality Center by HP Dashboards and Reporting




SAP Quality Center by HP – Dashboards and Reporting

GoalReporting of test status and progress suitable for different types of users

Scope Custom and role-based reports

Coverage graphs with drilldown

Benefits

Comprehensive and holistic overviewabout test progress

Out of the box reports

Various reporting types usable for a

wide range of users from manual

testers to senior management

* additional license required




1d - Performance Tests and Load Tests

Performance & Load TestsD fi i i f KPI




Level 1 – Performance & Load Test of key business processesEnd-to-end execution time including specific known queriesResource consumption of end-to-end scenarios including long running queriesCPU consumptionMemory consumptionDegree of parallelizationThroughput

Volume restrictionsLevel 2 - Runtime behavior and resource consumption of HANA DB Statements

Execution Time of specific known queries (SQLs / HTTPs)Resource Consumption of long running queriesMeasure CPU and memory consumption, and degree of parallelization

Definition of KPIs

Definition of KPIs and target values

Performance and Load TestsD fi iti f l d fil (1)




Goal: Identification of Top 15 Processes causing ca. 80% of system load

Approach 1ST03N: Workload Monitor – what’s used in the system, elapsed time

Download ST03N table to MS ExcelSelection of Top Transactions causing ca. 80% of system loadDouble-Click: How many users

STAD: Business Transaction Analysis – what‘s happening in the system right nowST05: Performance AnalysisSAT: Runtime Analysis

Definition of load profiles (1)

Definition of Load Profiles

Creation of SAP LoadRunner by HP scripts for these business processes

Performance and Load TestsDefinition of load profiles (2)




Goal: Identification of Top 15 Processes causing ca. 80% of system load

Approach 2Interviews

Frequently used business processesBusiness-critical business processesCustom developed business transactions„Long Runners“, but rarely executed (MRP Run, Fixed Asset Depreciation)How may users execute those processes in parallel (Average/Peak)?How often are they executed (Throughput: 100.000 Sales Order / year, 10.000 / month)

Definition of load profiles (2)

Definition of Load Profiles

Creation of SAP LoadRunner by HP scripts for these business processes

Load & Performance TestsE2E Test script example for CRM on HANA using SAP Loadrunner by HP




E2E Test script example for CRM on HANA using SAP Loadrunner by HP

E2E ScriptCRM on HAN

Single Actions – CRM on HANA

Runtime Settings – CRM on HANA

Paramters – CRM on HANA

Load & Performance TestsExample results for CRM on HANA using SAP Loadrunner by HP




Example results for CRM on HANA using SAP Loadrunner by HP

Execution of the SAP LoadRunneaccordance to the load profiles def

• here: 1000 Vusers (= processe• here: step wise ramp-up of Vu• Think Times: 5 s

Online resource consumption monexecution for defined monitoring pLoadRunner by HP

• here: Transaction Reponse Tim

Test Results:• here: SQLs / hour• here: HTTPs steps / hour• Here: identified issue: Load Balancing

Load & Performance TestsExample results for CRM on HANA using SAP Loadrunner by HP




p g y

Load & Performance TestsRuntime behavior & resource consumption of HANA DB




p

Measuring the HANA Engines portion of the overall query response timeOption 1: Expensive SQL statementsOption 2: SQL trace

Analyzing SQL execution of pure SQL statement in detail Analyzing SQL execution with the profile traceCapturing performance traces for Attribute, Analytic & Calculation Views

Capturing a performance trace from the HANA StudioX-Server connection to the HANA database server and starting the HDB admin toolCapturing a performance trace with the HDB admin toolFinding and displaying a specific SQL query in the HDB admin toolUnderstand which engine executes which plan operationsUnderstand the degree of parallel executionDrill down into single plan operations

Analyzing the performance trace of an Analytic View Analyzing the performance trace of a Calculation View

SAP Recomendation: Drill Down Analysis on SAP HANA DB based on results of E2E Performance &LoadTests executed with SAP LoadRunner by HP




1e - Service and Interface Tests

Service and Interface Tests Approach




UI Layer /Application

Layer

Custom Application A

DBLayer

Custom Application B

Custom API

Custom

Table 1

Custom

Table 2

SAP

Table…

HANA

Relevance for SAP HANA: Ability to test multiple use cases of custom-built interfaces with read / write access to custom andSAP defined tables residing in the SAP HANA database layer after technical migration

multiple use casesmultiple use cases

Service and Interface TestsTool: SAP Quality Center by HP – Service Test Module



Phase 2: Test Execution

Test Management for Sui te on HA NA migra t ion pro jec ts Test execution during migration project



© 2013 SAP AG All rights reserved

Functional Testsby QM teams

Reporting (test progress reports, status reports: open problems / resolution status)

• incidents

• corrections

• retest

Q Gate IntegrationTests by Business AnalystsFunctional

Tests

Automated Regression Tests for E2E Business Processes

Performance Tests for mission critical processes

Optional: Load Tests and DB Stress Testsfor large systems with very high throughput requirements

Performance& LoadTests

Service and Interface Tests Functional tests of APIs / Services / Interfaces without UI

Service &Interface

Tests

Comparison actual vs. baseline performance

Adjustments and Optimization of Custom Code / Modifications for SAP HANA



Test Activities after Go-live

Test Management for Sui te on HA NA migra t ion pro jec ts Part 3: Regression Tests for HANA revision deployments




Error corrections for SAP HANA via SAP HANA RevisonsSituation

Regression tests required at technical and / or process levelComplete regression test not feasable due to time and effort constrains

Regression Tests

Business Processes: Identify subset of functional tests for mission criticprocessesTechnical level: Identify important business operations implemented vViews or Stored ProceduresDefine automated tests via functional test automation and service testsExecute automated tests as minimum regression tests to verify HANA

Approach



Key Take Aways

Key Take Away




Business Blueprint Use SAP Solution Manager capabilities,in case your blueprint is incompleteFunctional test cases shall be assigned tobusiness processes or process steps

Test Scope 1 – critical business processes Identify Business Process ExpertsIdentify priority 1 business processes andassign custom attribute in your blueprint

ABC classification of frequently usedbusiness transactions

System preparation Activate UPL as early as possible in yourPRD systems

Test Scope 2 - Custom Code ABAP Test Cockpit (ATC) plus

UPL for scope limitationSQL Monitor to identifyimportant custom tables

Functional RegConsider te

Setup automprio 1 businPrepare test

Test Execution Assess how QM teams, test engineers and BPx can efficien

Test ManagementOverview and Detail Presentations and Offline Demos




SAP Service Marketplace : http://service.sap.com/testing

Informatio

Presentat

Offline D

More Information : http://service.sap.com/testing -> Additional Information

Best Practices / Next Practice:Regression Testing of SAP-centric Business Processes




Leverage the experience of other SAP customers and

out what works best for them.

Content

Regresstion tests as integral part of the software chang

9 Recommendations for regression testing

Customer use cases from Colgate-Palmolive, Ferrero, ConAgra Foods, Soka-Bau

Best Practice available at

SDN: http://www.sdn.sap.com/irj/sdn/alm-test-management

SMP: http://service.sap.com/testing More Information Te

Best Practices: Regression testing of SAP-centric Business Processes9 Recommendations

http://www.sdn.sap.com/irj/sdn/alm-test-management

http://service.sap.com/testing

http://service.sap.com/testing









Test System

Business Blueprint

Creationof

Regression Tests

Governance

1 Lean documentation of important business processes

2 Setup of lean test systems

3 Test automation for critical business processes

4 Comprehensive test data for automated regression tests

5 Provisioning of easy to use manual tests

7

Manual test execution with guidance for business users

9 Management of the change process

Change Impact

Analysis6 Change Impact Analysis: Risk-based test scope identification

Test Execution8

Unattended (‚lights out‘) execution of automated regression te

test management for business suite on hana migration projects using sap solution manager 7.1

Documents