copyright 2007, information builders. slide 1 relational efficiencies: part ii overview and join /...

Copyright 2007, Information Builders. Slide 1

Relational Efficiencies: Part IIOverview and JOIN / Sort Scenarios

Renee Teatro

Information Builders

Optimization Overview Definition Command syntax and settings Optimization Hierarchy Projection and Selection Tracing

JOIN ScenariosSort Scenarios

Relational Efficiencies Agenda

Interface optimization is the degree to which a TABLE request is translated to SQL

In other words, the process in which the interface translates projection, selection, JOIN, sort, and aggregation operations of a report request into its SQL equivalent and passes it to the RDBMS for processing

TABLE, MODIFY, MAINTAIN SQL Direct SQL Passthru Passing JOINs

Relational Efficiencies: Overview Definition of Optimization


TSOCMSSQL

DB2SQLDSSQLDBCSQLORASQLMSSSQLIDMS

SETOPTIMIZATIONSQLJOIN

ONOFFSQLFOCUS

*

Components Description

Target Database Engine DB2 for DB2SQLDS for SQL/DSSQLDBC for TeradataSQLORA for Oracle

SQLMSS for MS SQL Server SQLIDMS for IDMS SQL

OUTER

Relational Efficiencies: OverviewOptimization Command

SQL DB2 SET OPTIMIZATION ON

SQL SQLORA SET OPTIMIZATION ON

Relational Efficiencies: OverviewOptimization Hierarchy

JOIN

SORT

AGGREGATION

If the JOIN optimization fails, so will the SORT and

AGGREGATION.

If the SORT optimization fails, so will AGGREGATION.

AGGREGATION will occur only ifJOIN and SORT operations wereoptimized (and AGGREGATION

requested).


Relational Efficiencies: Overview Selection and Projection

What is selection and projection?

Regardless of the optimization setting, the data adapter will always try and translate selection and projection.

Why are selection and projection so important?

They are both processes that reduce the volume of data being returned from the RDBMS, which helps to improve performance, efficiency and report response time.

Selection The process of retrieving the table rows that meet the request criteria.

Projection The process of retrieving the table columns that meet the request criteria.

SET TRACEON=component//destination Component

SQLDI - FSTRACE - All Interface-RDBMS activitySQLAGGR - FSTRACE3 - Optimization messagesSTMTRACE - FSTRACE4 - SQL onlySQLCALL - commands and data exchange between

the physical and the logical layers of the data adapter Destination

FSTRACE - allocation for the ddname of FSTRACE CLIENT - displays client session to the screen

NOTE: Make sure to SET TRACEUSER=OFF afterwards

Relational Efficiencies: OverviewData Adapter TRACE Facility

Relational Efficiencies: OverviewData Adapter TRACE Facility

SET TRACEOFF=ALL SET TRACEUSER=CLIENTSET TRACEON=SQLAGGR//CLIENTSET TRACEON=STMTRACE//CLIENTSET XRETRIEVAL=OFFTABLE FILE …

SET XRETRIEVAL=[ON | OFF]ON – the data adapter sends the request to the

RDBMS and it processes the requestOFF – the data adapter attempts to optimize the

request and trace output is generated but no RDBMS processing is done


JOIN Scenarios

Relational Efficiencies JOIN: Performance related to KEYS attribute

JOINing 3 ORACLE tables results in very poor performance.

These are newly created MFDs and AFDs via CREATE SYNONYM

and the first time any requests / JOINs are being issued.

Scenario

Enable data adapter tracing to see if the JOIN is being

passed over and if not, why.

Suggestion

SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN COURSE_NO IN COURSE TO ALL COURSE_NO IN SECTION AS J1JOIN SECTION_ID IN COURSE TO ALL SECTION_ID IN ENROLLMENT AS J2TABLE FILE COURSE …


(FOC2510) FOCUS-MANAGED JOIN SELECTED FOR FOLLOWING REASON(S):

(FOC2505) A SEGMENT IN THE STRUCTURE IS NON-KEYED: : SECTION

(FOC2517) FST. OR LST. WHERE SORT FIELDS DO NOT COVER KEY

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:

(FOC2592) RDBMS-MANAGED JOIN HAS BEEN DISABLED

SELECT T1."COURSE_NO",T1."DESCRIPTION" FROM STUDENT.COURSE T1;

SELECT T2."SECTION_ID",T2."SECTION_NO",T2."START_DATE_TIME" FROM STUDENT.SECTION T2 WHERE (T2."COURSE_NO" = :0001);

SELECT T3."STUDENT_ID" FROM STUDENT.ENROLLMENT T3 WHERE (T3."SECTION_ID" = :0001);

Output

The .acx (ACCESS FILE DESCRIPTION) should have KEYS= with a

NON-ZERO value to indicate the PRIMARY KEY for the table in

question. The PRIMARY KEY is made up of the FIRST n fields

in the .mas (MASTER FILE DESCRIPTION).

Correction


SEGNAME=COURSE, TABLENAME=STUDENT.COURSE,

CONNECTION=STUDENTA.IBI.COM, KEYS=1,$

SEGNAME=SECTION, TABLENAME=STUDENT.SECTION,

CONNECTION=STUDENTA.IBI.COM, KEYS=0,$ *** Change to KEYS=1

SEGNAME=ENROLLMENT, TABLENAME=STUDENT.ENROLLMENT,


Correction

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."COURSE_NO",T1."DESCRIPTION",T2."SECTION_ID", T2."SECTION_NO",T2."START_DATE_TIME",T3."STUDENT_ID", T3."SECTION_ID" FROM STUDENT.COURSE T1,STUDENT.SECTION T2, STUDENT.ENROLLMENT T3 WHERE (T2."COURSE_NO" = T1."COURSE_NO") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."COURSE_NO", T2."SECTION_ID",T3."STUDENT_ID",T3."SECTION_ID";

Relational Efficiencies JOIN: Performance related to MULTI-PATH


Certain JOIN combinations using these same tables do not

have this issue.

Scenario

Suggestion

SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN GRADE AS J2TABLE FILE STUDENTPRINT SECTION_ID ENROLL_DATE GRADE_TYPE_CODEBY STUDENT_ID BY LAST_NAME BY FIRST_NAMEEND



(FOC2512) MULTI-PATH REQUEST



SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME" FROM

STUDENT.STUDENT T1;

SELECT T2."GRADE_TYPE_CODE" FROM STUDENT.GRADE T2 WHERE

(T2."STUDENT_ID" = :0001);

SELECT T3."SECTION_ID",T3."ENROLL_DATE" FROM

STUDENT.ENROLLMENT T3 WHERE (T3."STUDENT_ID" = :0001);

Output

Adjust JOIN field(s) to create a SINGLE PATH structure SQL Passthru Create relational VIEW on RDBMS – single segment MFD Create HOLD files & JOIN HOLD files vs. FOCUS-managed JOIN

Correction


CHECK FILE STUDENT PICTURE


JOIN CLEAR *

JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1

JOIN ENROLLMENT.STUDENT_ID AND SECTION_ID IN STUDENT TO ALL

STUDENT_ID AND SECTION_ID IN GRADE AS J2

-*CHECK FILE STUDENT PICTURE

Correction

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME", T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE", T3."GRADE_TYPE_CODE" FROM STUDENT.STUDENT T1,STUDENT.ENROLLMENT T2,STUDENT.GRADE T3 WHERE (T2."STUDENT_ID" = T1."STUDENT_ID") AND (T3."STUDENT_ID" = T2."STUDENT_ID") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."STUDENT_ID",T1."LAST_NAME", T1."FIRST_NAME";

Relational Efficiencies JOIN: Separate Node / Subsystem


Certain JOIN combinations using these same tables do not

have this issue.

Scenario

Suggestion

SET TRACEOFF=ALLSET TRACEUSER=CLIENTSET TRACEON=STMTRACE//CLIENTSET TRACEON=SQLAGGR//CLIENTSET XRETRIEVAL=OFFJOIN CLEAR *JOIN STUDENT_ID IN STUDENT TO ALL STUDENT_ID IN ENROLLMENT AS J1JOIN ENROLLMENT.STUDENT_ID AND SECTION_ID IN STUDENT TO ALL STUDENT_ID AND SECTION_ID IN GRADE AS J2TABLE FILE STUDENT…



(FOC2519) THE JOINED SEGMENTS RESIDE IN MORE THAN ONE NODE OR SUBS



SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME" FROM

STUDENT.STUDENT T1;

SELECT T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE" FROM

STUDENT.ENROLLMENT T2 WHERE (T2."STUDENT_ID" = :0001);

SELECT T3."GRADE_TYPE_CODE" FROM STUDENT.GRADE T3 WHERE

(T3."STUDENT_ID" = :0001) AND (T3."SECTION_ID" = :0002);

Output

CONNECTION / HOST values in ACX file is not the same for all tables involved in JOIN structure

Another MFD/AFD by the same name higher in the application path list with the SAME name as one of the JOINed tables that did indeed point to a different node.

Correction


SEGNAME=STUDENT, TABLENAME=STUDENT.STUDENT,


SEGNAME=GRADE, TABLENAME=STUDENT.GRADE,

CONNECTION=STUDENTB.IBI.COM, KEYS=4,$ *** Change to STUDENTA.IBI.COM

SEGNAME=ENROLLMENT, TABLENAME=STUDENT.ENROLLMENT,


Correction

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."STUDENT_ID",T1."FIRST_NAME",T1."LAST_NAME", T2."STUDENT_ID",T2."SECTION_ID",T2."ENROLL_DATE", T3."GRADE_TYPE_CODE" FROM STUDENT.STUDENT T1,STUDENT.ENROLLMENT T2,STUDENT.GRADE T3 WHERE (T2."STUDENT_ID" = T1."STUDENT_ID") AND (T3."STUDENT_ID" = T2."STUDENT_ID") AND (T3."SECTION_ID" = T2."SECTION_ID") ORDER BY T1."STUDENT_ID",T1."LAST_NAME", T1."FIRST_NAME";


Sort Scenarios

Relational Efficiencies Sort Management by the RDBMS

Relational Efficiencies Sort Management by the RDBMSBY and ACROSS equate to SQL ORDER BY clauseBY / FOR ROWS OVER syntax will translateJOIN COURSE_NO IN COURSE TO ALL COURSE_NO IN SECTION AS J1TABLE FILE COURSEPRINT COSTBY SECTION_IDFOR COURSE_NO103 OVER 100 OVER150 OVER136 OVER121END

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON: (FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USED SELECT T1."COURSE_NO",T1."COST",T2."SECTION_ID" FROM STUDENT.COURSE T1,STUDENT.SECTION T2 WHERE (T2."COURSE_NO" = T1."COURSE_NO") AND (T1."COURSE_NO" IN(100, 103, 121, 136, 150)) ORDER BY T2."SECTION_ID",T1."COURSE_NO";

Relational Efficiencies Sort Management by FOCUS / WebFOCUS

Relational Efficiencies Sort Management by FOCUS / WebFOCUSDEFINE FILE COURSESHORT_DESC/A5=EDIT(DESCRIPTION,'99999');COST_INC/P8=COST * .10;ENDTABLE FILE COURSEPRINT COSTBY COURSE_NOBY COST_INC-*BY SHORT_DESCEND

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USEDSELECT T1."COURSE_NO",T1."COST" FROM STUDENT.COURSE T1;

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2594) AGGREGATION IS NOT APPLICABLE TO THE VERB USEDSELECT T1."COURSE_NO",T1."DESCRIPTION",T1."COST" FROMSTUDENT.COURSE T1;

Relational Efficiencies Sort Management by the RDBMSDEFINE FILE COURSESHORT_DESC/A5=EDIT(DESCRIPTION,'99999');COST_INC/P8=COST * .10;ENDTABLE FILE COURSESUM COSTBY COURSE_NOBY COST_INC-*BY SHORT_DESCEND

AGGREGATION DONE ...SELECT T1."COURSE_NO",(T1."COST" * .1), SUM(T1."COST") FROMSTUDENT.COURSE T1 GROUP BY T1."COURSE_NO",(T1."COST" * .1)ORDER BY T1."COURSE_NO",(T1."COST" * .1);

AGGREGATION DONE ...SELECT T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5)),SUM(T1."COST") FROM STUDENT.COURSE T1 GROUP BY T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5)) ORDER BY T1."COURSE_NO",(SUBSTR(T1."DESCRIPTION", 1, 5));

Relational Efficiencies Sort Management by the RDBMSDEFINE FILE SSSALESMONTH_COMPONENT/A3=HNAME(ORD_DATE,'MONTH','A3');ENDTABLE FILE SSSALESSUM QTYBY MONTH_COMPONENT-*BY ORD_DATEEND

Tracing via DEFINE field MONTH_COMPONENT sort:

(FOC2590) AGGREGATION NOT DONE FOR THE FOLLOWING REASON:(FOC2609) CANNOT AGGREGATE BY NON-SQLIZABLE SORT KEY : MONTH_COMPSELECT T1."ord_date",T1."qty" FROM pubs.dbo.sales T1;

Tracing via REAL field ORD_DATE sort:

AGGREGATION DONE ...SELECT T1."ord_date", SUM(T1."qty") FROM pubs.dbo.sales T1GROUP BY T1."ord_date" ORDER BY T1."ord_date";

Relational Efficiencies Sort Management by the RDBMSDEFINE FILE SSSALESMONTH_COMPONENT/A3=HNAME(ORD_DATE,'MONTH','A3');ENDTABLE FILE SSSALESSUM QTYBY ORD_DATEM-*BY MONTH_COMPONENT-*BY ORD_DATEEND

AGGREGATION DONE ...SELECT T1."ord_date", SUM(T1."qty") FROM pubs.dbo.sales T1GROUP BY T1."ord_date" ORDER BY T1."ord_date";

FILENAME=SSSALES, SUFFIX=SQLMSS , $ SEGMENT=SSSALES, SEGTYPE=S0, $ FIELDNAME=STOR_ID, ALIAS=stor_id, USAGE=A4, ACTUAL=A4, $ FIELDNAME=ORD_NUM, ALIAS=ord_num, USAGE=A20, ACTUAL=A20, $ FIELDNAME=TITLE_ID, ALIAS=title_id, USAGE=A6, ACTUAL=A6, $ FIELDNAME=ORD_DATE, ALIAS=ord_date, USAGE=HYYMDs, ACTUAL=HYYMDs, $ FIELDNAME=ORD_DATEM,ALIAS=ord_date, USAGE=MT, ACTUAL=DATE,$ FIELDNAME=QTY, ALIAS=qty, USAGE=I11, ACTUAL=I4, $

From a performance standpoint, consider using TABLEF instead of TABLE for requests that you know are being returned in the requested sort order. This will free FOCUS / WebFOCUS from having to verify the sort order by loading its internal matrix. FOCUS / WebFOCUS will act directly on the data returned by the RDBMS.

ACROSS phrases are not supported. It must be verified that an ORDER BY statement has been generated in

the SQL for the correct columns prior to using TABLEF. Multi-verb requests are not supported. RETYPE is not available. TABLEF can hold locks on data pages until the data adapter issues a

COMMIT (usually when the report displays). Locks may prevent access to the data by other applications. (online core FOCUS)

Certain prefix operators are not available (TOT., PCT.) Computes that use prefix operators are not supported.

Relational Efficiencies Sort: Using TABLEF

copyright 2007, information builders. slide 1 relational efficiencies: part ii overview and join /...

Documents

join optimization

sort optimization

sql sqlora set optimization

relational efficiencies

interface optimization

overview selection

set traceuser

joins relational efficiencies