SAP Knowledge Base Article

Symptom

The results of a query that uses BWA are not returned to users within an acceptable time or the service level agreements are not being met.

Environment

SAP NetWeaver Business Warehouse Accelerator (BWA)

Resolution

WHAT TO CHECK The following steps can be taken to analyze and try to improve the performance of a query running on BWA:

1. Check whether the query uses BWA or not First, execute the query using transaction RSRT -> "execute & debug" and select the flag "Display Statistics Data" and “Do not use cache." Transaction: RSRT

    1871604 - BWA Troubleshooting: Analyse Poor Query performance  

Version   1     Validity: 11.06.2013 - active   Language   English

-> Execute the query. ->When the results are displayed press the back button and the statistics will be shown.  The query only uses the BWA index if the name of the InfoProvider contains the suffix "$X":

In case of a MultiProvider scenario, check if the slowest runtime is for an InfoProvider that has a BWA index. In the statistics, it is possible to tell which query accesses run against the BWA by the looking at the Aggregate column.  The suffix $X after the cube name signifies that this subquery accessed the BWA server. To display InfoCubes that have been indexed in BWA and check the status of the indexes: BW < 7.3: Go to transaction rsddv  ->  “Display BIA indexes”

 BW 7.3 and greater: Use transaction rsddb

Possible solution: If the slowest runtime is due to a missing BWA index, then evaluate the feasibility of creating and filling the BWA index for that InfoCube and then do the measurement again.

2. Check where most of the runtime is spent Execute the query using transaction RSRT -> "execute & debug" and select the flag "Display Statistics Data.”

BWA_KERNEL_TIME represents time spent in the BWA:

 

 

If the difference between values "ABAP_RFC_TIME" and “BWA_KERNEL_TIME" is high compared to the overall runtime, check if:

 a)     The network throughput between BW and BWA is acceptable. Send a ping from the BWA to all BW application servers: /usr/sap/<BWA_SID>/TRX00>ping -s 65000 -c 10 <BW server> (this command sends ten pings and then terminates automatically; afterwards a summary is displayed with minimal, maximal and average ping time).

 

 b)   The flag "Use Selection of Structure Elements" is activated.  The Use Selection of Structure Elements option influences system performance and decreases system runtime. This function should therefore normally be activated. It does not have any effect on the data displayed. Technically speaking, the system only passes to the database the selections and key figures of the columns (or structure elements, to be more precise) currently used to filter the query. If this function is deactivated, the data is read for the entire structure or for both structures from the database. Therefore this function cannot be activated if the query has the read mode “A.”

 c)     The performance feature FEMS_COMPRESSION has been activated.  By setting FEMS_COMPRESSION = on, the data transport costs can be reduced and duplicates in query results that occur directly on the BWA side can be eliminated when it is enabled. Use of this feature can improve query performance on BWA.  See SAP Note 1118425 for details on how to activate this feature.

 d)     Dimension function index (DFI) for a BWA index can also be used to improve query performance or to reduce excessive main memory usage.  See SAP Note 1133742 for details on how to activate DFI.  ** DFI is supported with BWA 7.00 Revision 54 or higher 

The DFI is an internal data structure that stores the data objects (DimFunctions) created during a query execution. These data objects are used for join operations during query execution. Any subsequent query that needs to execute the same join operations can reuse the DFI. DFI caching and reuse can improve query performance significantly in either of the following cases:

When the execution of the join operation is expensive (in CPU time). When the cached DFI contains huge data objects that no longer need to be transferred between blades during query execution.

If users are running similar queries, then the idea is that there is no need to perform the join operations again. However, do not activate DimFn for all cubes, as then all cubes would use up the cache. Go to BW statistics and search for long running queries and search for high runtime from the BWA (i.e. high kernel runtime) and activate DFI for the associated BWA indexes.

Possible solutions to the above: 

(a)   For the required 1GB network speed the observed values should be below 2.5ms. If the values are high, please contact the responsible hardware vendor to evaluate further. 

(b)   Activate “Use Selection of structure elements” for the query in transaction: RSRT and go the “Properties” tab for the query:

                                              

(c)   Steps to activate "FEMS compression" 

1.  Start the TREX admin tool (standalone/Python). 

2.  Navigate to the screen area Landscape  -> Ini. 

3.  Open the configuration file "TREXIndexServer.ini" and go to the section [OLAP]. 

4.  Add the line "fems_compression = on" to this section. 

5.  Use the "Save to all hosts" button to publish the changed TREX configuration file to all blades of the

     BWA landscape.

(d)   Activation of DFI TREX Standalone Admin tool -> View "Usage" -> Tab "DimFn Index" -> select any indexes that should use DFI -> right mouse click -> "Switch on"

3. Check for any errors using the checkBIA.py python script SAP delivers the Python script checkBIA.py to check for hard and software requirements and to test the basic functionality of a BIA installation as BWA system check.  It is also possible to start the BWA system check from BI application side by means of the transaction RSDDBIAMON2 (BI Accelerator Monitor). If the BWA system check is executed successfully, the BWA has been installed properly, the configuration files contain the necessary entries, and the BWA installation meets the requirements for correct functioning. The BWA system check produces a result list with the status of the BWA system check which had been performed. 

Possible solution 

To start the system check in transaction RSDDBIAMON2:

**It is recommended that this check is executed when the system is idle.

  

The most recent checkBIA.py script can also be ran by following the instructions in SAP Note 992064.  When the system check is complete the results will be displayed in a new window. Analyze any errors highlighted from the system check.

4. Check if a reorg is proposed by BWA The process called "reorg" is a reorganization that optimizes the distribution and settings of already existing BWA indexes. The reorg algorithm calculates an execution plan that moves BWA indexes from one blade to another. The goal is to optimize the distribution of BWA indexes regarding memory and CPU usage. 

If TREX calculates that the BWA landscape would profit from execution of reorg, the TREX alert server raises an alert message suggesting execution of a reorg.  On the BW system, check if a reorg alert has been raised using transaction RSDDBIAMON2

  

In the TREXAdmin tool the alert can also be seen in “Landscape” -> “Alerts”:

 

 Possible solution:  

By going to the “Reorg” tab, it is possible to view the potential improvement that performing the reorg would provide. The reorg can be executed from this screen (it can also be started from transaction: RSDDBIAMON2).

 

5. Check for partitioning of a huge F-Index A query gets slow if the Fact index of the involved InfoProvider is very big and wrongly partitioned or not partitioned at all.                          

Possible Solution 

The splitting of the fact index and dimension index is not done randomly but in a smart way that is optimized for joining the two indexes. This is called “Partitioning.” Joining a dimension index and a fact index means that the dimension index keys get related with those rows of the fact index that have the same values in the corresponding dimension column. So, partitioning a fact index by a given dimension means that: 

(a) All rows of the fact index that have the same value for this dimension column go to one part index (on one blade). 

(b) The dimension index is also split: in such a way that the rows that go to a certain blade have the same key values (=Dimension IDs) as the rows of the fact index part on this blade. 

(c) With such a distribution of index rows it is possible to do the complete join of a fact and a dimension index by joining each part of the dimension index to the corresponding part of the fact index. These parts can be joined locally on each blade. This will allow the high memory and network load to be avoided and improve the query performance. 

With the standard settings, a fact index will be split if it exceeds the size given in the standalone TREXAdmin tool (“Landscape” -> “Configuration”, tab “Index”). Steps on splitting the Index:

1) In the TREXAdmin Tool goto into Tab Index -> Landscape

2) Select index Id that needs to be split. Choose Split/Merge Index from the context menu.

    Choose with roundrobin = N where N equal number of active (not backup) blades in the landscape.

6. Review BWA Statistics index The BWA statistics index collects detailed information about all calls received by a BWA installation. This includes search calls, indexing calls, and administration calls. The information is collected in a logical BWA index (an OLAP cube) with several physical index parts. These indexes are always located in the "_trex" namespace. They can be queried like any other BWA OLAP index. SAP Note 1163732 – BWA 7.00: BWA statistics index - provides further information regarding the configuration of the BWA statistics index. With the installation of BWA 7.00 Rev 60 the statistics index is switched off by default. To switch the statistics index off manually modify TREXIndexServer.ini in section [statistics] trace = off. To switch it on set trace = on The TREX statistics index can be used to answer such queries as: ->What were the top queries in a particular time period? ->Which user sent the most queries? ->Which queries took longer than t seconds to process? ->Which errors occurred? How to use the BWA index statistics Possible Solution Within the TREX standalone admin tool, the BWA statistics index can be used to analyze statistical data about indexing and query processes that run on the BWA.   From the view "Mining", it is possible to create any query on the OLAP index of the statistics index. By choosing the index "_trex:statistics" in the mining view the default queries can be executed by right-clicking to open the context menu, choosing "TREX statistics" and then selecting a query from the submenu:

 By right clicking the attribute above and selecting, for example, the query “What users caused highest load?” the following screen is displayed:

From here, for example, it is possible to see the user and what query they were running. To get the technical name of the query from the “queryid” above simply go to the table rsddstat_dm in the BW system and insert the “query id” in the field DMUID and press execute:

7. Review the Alert server for any errors/warnings In the TREXAdmin tool (transaction trexadmin -> Alerts) can open the Trex Alert server to check for any critical alerts.

Recommendation: Recommendations can be given based on the warning and errors displayed in the Admin tool.

8. Execute a Performance trace If the above solutions do not help to improve the performance and a deeper analysis is required, capture a performance trace. A performance trace records time spent in different areas of the BWA kernel. Mostly it is used to analyze the runtime of a query in the BWA kernel. There are two ways to record the performance trace. One is from python Admin Tool and the other is from transaction RSDDBIAMON2. It is not recommended to run the trace for over one hour. A.  Python Admin Tool:        1. Go to python Admin Tool -> Perf. Trace -> Start -In the popup window please specify:     A. The BW user executing the job (e.g. query),     B. The time when performance trace recording is to stop ("0" is default, which means endless runtime) and the timeline precision (1 second should be chosen) and     C. A flag for "Plan Execution Details" (to analyze query performance this flag is recommended). Pressing the "OK" button starts performance trace recording.         (If the query runtime is known, then it is possible to specify the time to stop. Otherwise use "0" to stop manually.)

2.  Reproduce the problem scenario. 3. To manually stop the trace, go to python Admin Tool -> Perf. Trace -> Stop. 4. Save the trace in python Admin Tool -> Perf. Trace -> Save (it is not possible to see the trace until it is saved).

       B. Transaction RSDDBIAMON2           1. Go to transaction RSDDBIAMON2 -> Performance Trace -> Start Trace Recording.           2. Specify the user and the time to stop the trace recording, if necessary.           3. Execute "Start Trace Recording."           4. Reproduce the problem scenario.           5. Save the trace in RSDDBIAMON2 -> Performance Trace -> Save Trace File.          Now that the trace is recorded and saved, it is possible to load the trace file in order to perform analysis. To do this, follow these steps:           1. Go to python Admin Tool -> Perf. Trace -> Load.           2. Choose the .tpt file recorded before.           3. After loading, go to python Admin Tool -> Perf. Trace -> Calls. Here it is possible to see that different Services/Methods take different Duration/Processing time like the following:

                     4. Double click one line in step 3 i.e. the long running step, in order to see the details under tab Call Pattern:

                   By clicking on “Help” an explanation of the graph colors appears which will help to identify where the majority of the time is spent during this particular call and where further analysis is necessary.           5. Under python Admin Tool -> Perf. Trace -> Call Patterns, it will show all the Call Patterns details

9. Execute Python Trace

A Python Trace can be used to analyze the calls which are sent from BW to BWA. It can be recorded for all of the BWA services. Using a python trace, it is possible to replay the query and do further analysis. With the python trace it is possible to reduce the trace by commenting out the code not causing the problem and finally then get to the real issue. The Python trace can be switched on/off from the Python TREX Admin Tool -> Trace tab. The trace is stored in a trace file on the file system. The Python trace is revision independent and can be copied to another system and replayed there for analysis. To record a python trace, go to transaction rsrt and select the following:

Analyzing a Python trace: The recorded Python trace can be run with the following Linux command: >python <tracefile_name.py> Note that if the Python trace is to be run on a test system, the so called communication method needs to be changed first. To change the communication method, change the setCommunicationMethod parameter in the .py file from 0 to 1. Please note that due to the different BWA data on the test system, the problem might not be reproducible there, even after running the Python trace. To write the Python trace execution results to the command-line, add the following piece of code to the end of the trace: so.olapSearch() -> fuzzypy.writeResult(so.olapSearch()) Possible solution: Analyzing the Python trace and commenting out parts of the Python trace code can lead to identifying a specific part of a query that has led to the performance problem. In order to reproduce a query issue the index is also necessary.  If further help with analyzing the python trace is required open an SAP Customer Message on component BC-TRX-BIA and attach the .py trace file.

See Also

General SAP Notes for Reference 

992064     BIA 7.00 + BIA 7.20: BIA system check 1118425   BIA 7.00: Activate performance feature FEMS 1133742   BIA 7.00: DimFunction Index (DFI) 1163732   BWA 7.00: BWA statistics index 1318293   BWA: Recording a performance trace 1533891   TREX 7.00/7.10: Recording a Python Trace

Keywords

BWA Query Performance, Query Performance Slow, Query with BWA takes long time

Header Data

Product

This document is not restricted to a product or product version

Released On 05.08.2013 11:32:31

Release Status Released to Customer

Component BC-TRX-BIA TREX BI Accelerator

BW-BEX-OT-BIA BW Accelerator

Priority Normal

Category How To

Other Components