obtaining explain result sets - sap help portal · obtaining explain result sets automated...
TRANSCRIPT
CUSTOMER
SAP BusinessObjects Predictive Analytics 3.12017-10-26
Obtaining Explain Result SetsAutomated Analytics Configuration Guide
Content
1 About Explain Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.1 Native Explain Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Configuring Automated Analytics to use Explain Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.1 Where to Find the Configuration Files. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.2 Activating or Deactivating the Explain Feature. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62.3 Analyzing Explain Result Sets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72.4 Setting the Classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
3 Using Explain Mode. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.1 Availability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103.2 Activation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103.3 Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113.4 Using the Workload Attribute in SAP BusinessObjects Predictive Analytics. . . . . . . . . . . . . . . . . . . . 113.5 Default Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.6 Setup Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.7 Runtime Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123.8 Dynamic Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133.9 Using Dynamic Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 C U S T O M E RObtaining Explain Result Sets
Content
1 About Explain Mode
This feature allows you to categorize the performance of SQL requests generated by Automated Analytics in several classes defined by the user. This categorization is done without actually executing the full SQL request in order to be as fast and as light as possible.
The objective is to allow you to estimate the work load of SQL requests before using the results (view data, train a model, …) and then decide (or apply an explicit IT Corporate Policy) if the SQL request can actually be used.
For example, an IT corporate policy may favor interactivity and then define three classes of SQL requests, each with its maximum time.
Class Duration Result
Immediate < 1 s The request is accepted and executed immediately.
Batched 1s <= duration < 2 s The request is accepted but will be executed on next idle time.
Rejected >= 2 s The request will never be executed.
The number, names and limits of classes are defined by the user so these values match the current DBMS configuration and DBMS usage policy.
1.1 Native Explain Mode
Some Database Management Systems (DBMS) can analyze an SQL request and instead of returning the resulting values, can return a specific result set describing step by step how the SQL request will be executed and how much time each step will take. This is called 'explaining the request', 'get the request plan', …
Obviously, each DBMS has its own way of describing a step and its own units to measure time and cost to run a step (the unit may even depend on the actual configuration of the server running the DBMS).
However, even with 'unknown' units, the global process is still valid:
1. Define properly the limits of the classes.2. Use the native Explain mode to get costs for each step.3. Aggregate all costs.4. Find the proper class using a set of intervals defined by the user (with help of DBA, which is the only one to
know the actual measure units of the DBMS).
Whatever way the DBMS sets up its native Explain mode, results are provided the same way: the DBMS returns an Explain result set. Available fields and values are not related to the fields and values returned by the normal SQL request but to the capacity of the DBMS to explain any SQL request.
This result set is called the Explain result set and categorizing a SQL request amounts to finding the proper segment for the value of a specific explain field in the Explain result set.
Obtaining Explain Result SetsAbout Explain Mode C U S T O M E R 3
ExampleWith SQLServer 2005: SELECT * FROM adultid_5000 returns the classic result set:
ID 1 …
age 42
workclass State-gov
fnlwgt 55764
education Assoc-acdm
educationnum 12
maritalstatus Divorced
occupation Prof-specialty
relationship Not-in-family
race Black
sex Male
capitalgain 0
capitalloss 0
hoursperweek 40
nativecountry United-States
classe 0
Using Explain mode is done with:
SET SHOWPLAN_ALL ON
SELECT * FROM adultid_5000
and returns the Explain result set:
StmtText select * from adultid_5000 |--Clustered Index Scan(OBJECT:…
……….
StmtId 1 1
NodeId 1 2
Parent 0 1
PhysicalOp Clustered Index Scan
LogicalOp Clustered Index Scan
Argument 1 OBJECT:…
DefinedValues [kxen].[Greedy].[adultid_5000].[ID], …
EstimateRows 5001 5001
EstimateIO 0.06460648
EstimateCPU 5.66E-03
4 C U S T O M E RObtaining Explain Result Sets
About Explain Mode
AvgRowSize 1074
TotalSubtreeCost 7.03E-02 7.03E-02
OutputList [kxen].[Greedy].[adultid_5000].[ID], …
Warnings
Type SELECT PLAN_ROW
Parallel 0 0
EstimateExecutions 1
Automated Analytics uses the correct defaults in relation to the DBMS when choosing a field in the Explain result set and analyzing its values. In the example above, the default is to use the first value of EstimateCPU field. If needed, setup can be done so that the sum of all EstimateIO values will be used.
Obtaining Explain Result SetsAbout Explain Mode C U S T O M E R 5
2 Configuring Automated Analytics to use Explain Mode
The Explain Mode is set up with the standard Automated Analytics option syntax in the option file.
The granularity of the Explain Mode is set by store. The common syntax for all Explain options is:
DataAccessExplanation.<DSN>.<Explain option>=<Value>
where <DSN> is the name of the store.
NoteThe wildcard * can also be used instead of <DSN>, meaning "for every connection".
2.1 Where to Find the Configuration Files
To configure the Explain mode, you may need to edit one or several of the following configuration files:
Filename OS Location
KxCORBA.cfg UNIX ${INSTALLATION DIR}/KxenServer/
Windows ${INSTALLATION DIR}\Server <version>\EXE\Servers\CORBA\
KJWizard.cfg UNIX (*) ${INSTALLATION DIR}/KJWizardJNI/
Windows ${INSTALLATION DIR}\Desktop <version>\EXE\Clients\KJWizardJNI\
(*) If delivered
Note${INSTALLATION DIR} is the folder where the application has been installed.
2.2 Activating or Deactivating the Explain Feature
By default the Explain Mode is deactivated for all Data Source Name.
● To activate this feature for a DBMS, use the Activation option in your configuration file:
○ For standalone workstation, edit the file KJWizard.cfg.
6 C U S T O M E RObtaining Explain Result Sets
Configuring Automated Analytics to use Explain Mode
○ For a server, edit the file KxCORBA.cfg.● Comment the following statement: DataAccessExplanation.*.Activated=false should become
#DataAccessExplanation.*.Activated=false or DataAccessExplanation.*.Activated=true
NoteIt is possible to manage Explain feature for a specific Data Source Name. In the above line, replace the * by the Data Source Name.
ExampleTo activate the Explain feature only for DSN DevServer and DSN TestServer, the correct lines would be as follows:
Sample Code
DataAccessExplanation.*.Activated=false DataAccessExplanation.DevServer.Activated=true DataAccessExplanation.TestServer.Activated=true
2.3 Analyzing Explain Result Sets
The mandatory options ExplainField and ExplainFieldOperator allow overriding the default values used by the application when analyzing an Explain result set.
ExplainField is the name of a column in the Explain result set. A check is done to verify that the type of the ExplainField is correct (for example, on Oracle and SQLServer 2005 it must be a number).
ExplainFieldOperator is the operation to apply on the values of ExplainField.
Available Operators
ExplainFieldOperator Description
First First non null value
Last Last non null value
Max Maximum of non null values
Min Minimum of non null values
Sum Sum of all non null values
Default settings
The application engine has built-in defaults for major DBMS (SQLServer 2005, Oracle, Teradata) so these two parameters have to be set only in special circumstances.
Obtaining Explain Result SetsConfiguring Automated Analytics to use Explain Mode C U S T O M E R 7
DBMS ExplainField ExplainFieldOperator
SQLServer 2000,2005 EstimateCPU First
Oracle 9,10 COST First
Teradata V2R5.1 to 12 Explanation Special text extractor
NoteTeradata uses a specific Explain mode. ExplainField and ExplainFieldOperator cannot be overwritten for this DBMS
2.4 Setting the Classes
Each class is identified by its <ClassID> and has <Name>, <Description>, <Min>, <Max> and <Workload> attributes.
● <ClassID> is a free string and must not contain any blank, tab or . (dot) characters.● <Min> and <Max> attributes must be expressed in the unit used by the <ExplainField> . If <Min> is not
specified, 0 is used. If <Max> is not specified, a special value 'infinite' is used.● <Workload> is mandatory and can be Small , Medium or Huge . This attribute quantifies the cost in
DBMS associated to each class. The application automatically uses the value of this attribute to execute without confirmation, to execute with confirmation or to reject the execution of this request. For more information, refer to the related content section.
● <Description> is mandatory. It is the text that will be displayed by the application as the result of Explain mode.
Some special keywords in the description text are automatically replaced:
Keyword Replacement
#FIELD# Explain field used to make the categorization
#VALUE# Actual value used to make the categorization
#MIN# Minimum attribute of the matching class
#MAX# Maximum attribute of the matching class
#WORKLOAD# Workload attribute of the matching class
#CLASSID# ClassID of the matching class
Sample CodeThe following options describe three Explain classes: immediate, batched and rejected
DataAccessExplanation.MyDataMart.Immediate.Max=1 DataAccessExplanation.MyDataMart.Immediate.Workload=SmallDataAccessExplanation.MyDataMart.Immediate.Description="This request costs only #VALUE# on the DBMS and can be executed immediately."
8 C U S T O M E RObtaining Explain Result Sets
Configuring Automated Analytics to use Explain Mode
DataAccessExplanation.MyDataMart.Batched.Min=1DataAccessExplanation.MyDataMart.Batched.Max=2DataAccessExplanation.MyDataMart.Batched.Workload= MediumDataAccessExplanation.MyDataMart.Batched="This request costs #VALUE# on the DBMS and can only be executed with a low priority." DataAccessExplanation.MyDataMart.Rejected.Min=2DataAccessExplanation.MyDataMart.Rejected.Workload=HUGEDataAccessExplanation.MyDataMart.Rejected="This request costs too much (#VALUE#) to be executed on the DBMS."
Related Information
Using the Workload Attribute in SAP BusinessObjects Predictive Analytics [page 11]
Obtaining Explain Result SetsConfiguring Automated Analytics to use Explain Mode C U S T O M E R 9
3 Using Explain Mode
3.1 Availability
The space parameter CanExplain is set to true if the Explain mode is managed by the application engine for the current Database Management System (DBMS).
Example
createStore KXEN.ODBCStore st st.openStore "kxsrvmulti2_sqlserver2005" "kxenodbc" 'kxenodbc"st.newSpace "select * from adultid_5000" spsp.getParameter "" sp.checkParameter "Parameters/CanExplain" true
3.2 Activation
The space parameter Explain is used to trigger the Explain mode.
The next validateParameter operation will immediately launch the Explain mode for the space. After the results of Explain mode have been analyzed, this parameter is automatically reset to false.
Example
createStore KXEN.ODBCStore st st.openStore "kxsrvmulti2_sqlserver2005" "kxenodbc" 'kxenodbc"st.newSpace "select * from adultid_5000" spsp.getParameter ""sp.changeParameter "Parameters/Predict" "true"sp.validateParameter The Explain mode is immediately launched.
10 C U S T O M E RObtaining Explain Result Sets
Using Explain Mode
3.3 Results
When successfully analyzed, the space parameter subtree DataAccessExplanation is filled with the following information:
Parameters/DataAccessExplanation Description
ValueForExplain Value used to categorize the request
SmartValueForExplain Truncated (more user-friendly) value of ValueForExplain
ClassID ClassID of the matching class
Min Minimum authorized value of the matching class
Max Maximum authorized value of the matching class
ExplainStatus OK, BadConfig or RuntimeError
ExplainIssues List of all issues encountered when checking setup or analyzing the Explain result set
Description Description of the matching class
Example
Sample Code
st.openStore "kxsrvmulti2_sqlserver2005" "kxenodbc" 'kxenodbc"st.newSpace "select * from adultid_5000" spsp.getParameter ""sp.changeParameter "Parameters/Predict" "true"sp.validateParametersp.getParameter "" sp.getParameterSubEntries "Parameters/DataAccessExplanation" string
3.4 Using the Workload Attribute in SAP BusinessObjects Predictive Analytics
Each class is associated with its cost in DBMS, that is, the workload attribute.
This cost can be:
● Small: the application will allow executing the SQL request.● Medium: the application will ask a confirmation before executing the SQL request.● Huge: the application will display a warning message and never execute the SQL request.
The applicartion checks this before:
● Executing a "View Data", which avoids stressing the DBMS and waiting an unexpected long time when doing data exploration or designing a complex data manipulation.
● Materializing a data manipulation in a table.
Obtaining Explain Result SetsUsing Explain Mode C U S T O M E R 11
3.5 Default Setup
If no explicit setup of Explain classes is provided for the current DBMS, the application engine automatically uses a default Explain class. This class with ClassIDKxenCatchAllNoSetup catches all SQL requests (min is 0 and max is infinite) and has an associated workload of MEDIUM.
In SAP BusinessObjects Predictive Analytics, this default setup will make all "View Data" done on SQLServer 2005, Oracle and Teradata ask for a confirmation before execution.
In such a case, the confirmation request clearly indicates that a default setup is used and that the administrator can tune it.
3.6 Setup Errors
To ensure that the Explain setup is correct prior to any categorization, Automated Analytics checks the following:
● All Explain classes have a DESCRIPTION attribute● All Explain classes have a WORKLOAD attribute● Only one Explain class has a MIN attribute set to 0● Only one Explain class has a MAX attribute set to 'INFINITE'● Min attribute is less than the MAX attribute● There is at least one class defined● No unknown option is used
In case of any setup error, Automated Analytics does the following:
● Forces classification of the request to a special Class with CLASSID KXENCATCHBADCONFIG, workload MEDIUM and with a clear description, so the user has to confirm execution
● Sets EXPLAINSTATUS parameter to BADCONFIG● Sets EXPLAINISSUES parameter to the list of issues encountered
3.7 Runtime Errors
To ensure that the categorization is possible and that its result has a meaning, each time the categorization is done Automated Analytics checks the following:
● Explain field is actually in the Explain result set and has the proper type● One and only one Explain class matches the value returned by analysis of the Explain result set● There is at least one usable value in the Explain result set
In case of any run-time setup error, Automated Analytics does the following:
● Forces classification of the request to a special class with CLASSID KXENCATCHRUNTIMEERROR, workload MEDIUM and a clear description, so the user has to confirm execution
12 C U S T O M E RObtaining Explain Result Sets
Using Explain Mode
● Sets EXPLAINSTATUS parameter to RUNTIMEERROR● Sets EXPLAINISSUES parameter to the list of issues encountered
3.8 Dynamic Configuration
Proper setup of Explain mode needs to be done with respect to the actual DBMS, its setup and IT policy, this is why the application is delivered without setup for Explain mode.
In such a mode, each request asks for a confirmation (with a clear indication that this is due to a pending setup of Explain mode). As a convenience, it is possible to do a basic setup on the fly.
When no setup is available, a special pseudo class is available where the CLASSID is set to KXENSMALLFORCEDBYUSER and the workload is set to SMALL. It is therefore possible to use the API setConfiguration to change the MAX attribute of this class and so, to dynamically set up a set of 2 classes that will be used for further Explains. One class is named KXENSMALLFORCEDBYUSER with a workload SMALL, the other is KxenMediumForcedbyUser with a workload MEDIUM.
This dynamic configuration is valid only for the current user session.
3.9 Using Dynamic Configuration
1. Without any setup, the Explain mode for the request SELECT * FROM ADULT on a DBMS identified by an ODBC connection named MyDBMS returns a cost of 42 and classifies everything as (KxenCatchAllNoSetup,Medium).
2. The application recognizes this KxenCatchAllNoSetup classID and decides to propose a dynamic configuration. If accepted, the application calls the API:
setConfiguration DataAccessExplanation.MyDBMS.KxenSmallForcedByUser.Max 42
3. This automatically sets up Explain mode with two classes:
Sample Code
ClassID: KxenSmallForcedbyUser, Workload: Small, Max: 42 ClassID: KxenMediumForcedbyUser, Workload: Medium, Min: 42
4. After that, any SQL request with a cost below 42 will be executed without confirmation, otherwise a confirmation is asked.
5. Each time the user emits a SQL request costing more than 42, a confirmation is asked which can be painful again. The application can then decide to manage all classifications with ClassID KxenMediumForcedbyUser and propose to extend the level of automatically executed SQL requests. If the user accepts this extension, using the same call with new cost value will change the separation of the 2 classes.
Obtaining Explain Result SetsUsing Explain Mode C U S T O M E R 13
Important Disclaimers and Legal Information
Coding SamplesAny software coding and/or code lines / strings ("Code") included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended to better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, unless damages were caused by SAP intentionally or by SAP's gross negligence.
AccessibilityThe information contained in the SAP documentation represents SAP's current view of accessibility criteria as of the date of publication; it is in no way intended to be a binding guideline on how to ensure accessibility of software products. SAP in particular disclaims any liability in relation to this document. This disclaimer, however, does not apply in cases of willful misconduct or gross negligence of SAP. Furthermore, this document does not result in any direct or indirect contractual obligations of SAP.
Gender-Neutral LanguageAs far as possible, SAP documentation is gender neutral. Depending on the context, the reader is addressed directly with "you", or a gender-neutral noun (such as "sales person" or "working days") is used. If when referring to members of both sexes, however, the third-person singular cannot be avoided or a gender-neutral noun does not exist, SAP reserves the right to use the masculine form of the noun and pronoun. This is to ensure that the documentation remains comprehensible.
Internet HyperlinksThe SAP documentation may contain hyperlinks to the Internet. These hyperlinks are intended to serve as a hint about where to find related information. SAP does not warrant the availability and correctness of this related information or the ability of this information to serve a particular purpose. SAP shall not be liable for any damages caused by the use of related information unless damages have been caused by SAP's gross negligence or willful misconduct. All links are categorized for transparency (see: http://help.sap.com/disclaimer).
14 C U S T O M E RObtaining Explain Result Sets
Important Disclaimers and Legal Information
Obtaining Explain Result SetsImportant Disclaimers and Legal Information C U S T O M E R 15
go.sap.com/registration/contact.html
© 2017 SAP SE or an SAP affiliate company. All rights reserved.No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company. The information contained herein may be changed without prior notice.Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary.These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies.Please see http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.