b211 parser
TRANSCRIPT
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 1/22
Module 11: SQL Parser
After completing this module, you will be able to:
Describe internal, channel, and LAN parcels.
Explain software cache functionality.
List and identify the function of the main components
(phases) of the parser.
Describe the functionality of Request-To-Steps cache.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 2/22
Internal, Channel and LAN Parcels
• A REQUEST parcel is followed by zero or one DATA parcel plus one
RESPOND parcel.
• The RESPOND parcel identifies response buffer size.
• A RESPOND parcel may be sent by itself as a continuation request for
additional data.
• A SUCCESS parcel may be followed by RECORD parcels.
• Every REQUEST parcel generates a SUCCESS/FAIL parcel.
Teradata
RDBMS
REQUEST
PARCEL
1
DATA
PARCEL
RESPOND
PARCEL
RECORD
PARCEL
RECORD
PARCEL
Success / Fail
PARCEL
HOST
2 3
123
INTERNAL CHANNEL PARCELS
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 3/22
Request Parcel
A Request Parcel:
• Must contain at least one SQL statement.
• May contain two or more SQL statements (multi-statement request).
• May be a transaction by itself (default).
• May be one parcel of a multi-request transaction.
A Request Parcel is the parsing unit.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 4/22
The Data Parcel
• Explicitly declare a USING data parcel in BTEQ.
• FastLoad and MultiLoad do not use the SQL protocol, but use this conceptthrough the DEFINE or LAYOUT statements.
• The Preprocessor generates a USING data parcel from macro parameters or
SQL statements that reference host program variables.
• CLI programs must create and manage their own DATA parcels.
USING x INTEGER, y CHAR(5), z INTEGER
EXEC My_Macro (:x, :y, :z); 100, 'ABC', 321
CREATE MACRO My_Macro(var1 INTEGER
,var2 CHAR(5)
,var3 INTEGER )
AS ( INSERT INTO Table_1
VALUES (:var1, :var2, :var3);
UPDATE Table_2
SET Col_C = :var3 + 1WHERE Col_A = :var1; ) ;
User executes
Request Parcel USING x INTEGER, y CHAR(5), z INTEGER
EXEC My_Macro (:x, :y, :z);
Within Teradata
100, 'ABC', 321 Data Parcel
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 5/22
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 6/22
Software Cache
• ALL Teradata vprocs use some SMP memory as software cache.
• Software cache retains data dictionary information and/or processing
steps.
– Dictionary Cache - hold DD/D information in memory
– Requests-To-Steps Cache - holds processing steps in memory
• Cache eliminates regenerating and re-fetching needed information.
• Larger memory gives the processors more software cache and reduces
disk accesses.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 7/22
Dictionary Cache
• The Parser needs data from the DD to convert SQL into AMP steps.
• DD cache holds the most recently used items in PE memory:
– SQL names and their related numeric IDs.
– Statistical information for the Optimizer.
• SQL statements that change the DD generate “spoiling” messages.
• PEs drop the “spoiled” entries from their cache.
• The system purges cache every four hours:
– Data demographics can change significantly within four hours.
– Purging cache forces the Parser to re-optimize all current requests.
•
Processors purge their cache on a sequential basis.
• DD tables used by the resolver to parse a request:
– Dbase – TVM – AccessRights
– TVFields – Indexes – RoleGrants (V2R5)
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 8/22
Request-to-Steps Cache
• Cache stores the SQL text and the AMP steps generated by the Parser
without binding in data from the Using DATA parcel.
– Plans do not have to be re-parsed by the user.
– Requests with hard-coded values are not immediately cached.
• For cached plans, the plan is retrieved and the SYNTAXER, RESOLVER,
OPTIMIZER and GENERATOR steps are bypassed.• Cached steps may be shared by sessions/logons within a Parsing Engine.
• Cache is maintained in Least Recently Used sequence so that lower activity
plans swap out to disk.
• The system purges unmarked entries from cache every four hours.
– All PEs also purge affected cache entries on receipt of a DDL “Spoiling” message.
• Demographically independent plans (UPI, USI and some Nested Joins) are
marked when placed into cache.
• DDL requests are never cached.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 9/22
Request-to-Steps Cache Check
REQUEST
Parcel
Request/Steps
CACHE
REQUESTParcel
PlasticSteps
Cached?
To SYNTAXER To SECURITY and APPLY
No Yes
If an identical Request exists in Request-To-Steps cache:
• Call SECURITY and APPLY and pass them the memory address of the AMP steps.
• These steps do not have DATA parcel values bound into them; they are called Plastic
steps.
Otherwise, the Request Parcel passes the request to the SYNTAXER.
The larger the Request Parcel, the longer these steps take.Macros reduce parcel size, dramatically improving performance.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 10/22
Request-To-Steps Cache Logic
CREATE
HASH of
PARCEL
HASH
in REQ.AREA
?
TEXT
MATCH?
PARCEL
CONTAIN
: var
?
HASH
in TEXT
AREA
?
Move HASH
and TEXT to
REQUEST AREA
Y
N
Y
N
Y
N
Y
N
PARSE
Request
STORE HASH
and TEXT in
REQUEST AREA
STORE HASH
in TEXT AREA
SECURITY
PARSE
Request
OPTIMIZE Request and GENERATE Steps
STORE
Steps
Apply data parcels and dispatch steps to AMP(s)
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 11/22
Syntaxer
REQUEST
Parcel
InitialPARSE TREE
To RESOLVER
SYNTAXER
• This module checks the syntax of an incoming Request Parcel.
• If no errors are found, it produces an initial parse Tree and calls the
RESOLVER.
• The larger the Request Parcel, the longer these steps take.
•
Macros reduce parcel size, dramatically improving performance.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 12/22
Resolver
Initial
PARSE TREE
AnnotatedPARSE TREE
To SECURITY
RESOLVERViews and Macros
From SYNTAXER
DD Cache
DD Dbase
AccessRights
RoleGrants (V2R5)
TVM
TVFieldsIndexes
• RESOLVER replaces all View and Macro names with their underlying text.
• Nesting of Views and macros could add substantial time to this process.
• Views and/or Macros can be nested 10 levels (V2R3) or 64 levels (V2R4).
• The RESOLVER uses information from the DD cache when possible.
• It accesses the DD tables if additional information is needed.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 13/22
Security
SECURITY verifies that the requesting User ID has the necessary permissions
on the referenced objects.
Permissions can be granted or revoked dynamically.
Each execution of a Request includes a SECURITY check.
• Initial parse and execute checks ALL access rights.
• Execution from cache checks only user rights (i.e., EXEC).
Annotated
PARSE Tree
Plastic
Steps
To OPTIMIZER To APPLY
AnnotatedPARSE Tree
SECURITY
PlasticSteps
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 14/22
Optimizer
• DD/D operations replace DDL statements in the Parse tree.
• The OPTIMIZER evaluates DML statements for possible access paths: – Available indexes referenced in the WHERE clause.
– Possible join plans from the WHERE clause.
– Full Table Scan possibility.
• It uses COLLECTed STATISTICS or dynamic samples to make a choice.
•It generates Serial, Parallel, Individual and Common steps.
• OPTIMIZER output is passed to either the Generator or the Explain facility.
Annotated
PARSE TREE
OptimizedPARSE TREE
To GENERATOR
OPTIMIZER
From SECURITY
Statistics
EXPLAIN
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 15/22
Generator
• Plastic Steps are AMP steps without data values from the DATA parcel boundin.
• Hard-coded literal values are embedded in the Plastic Steps.
• Plastic Steps are stored in Request-to-Steps cache.
Optimized
PARSE TREE
PlasticSteps
To APPLY
GENERATOR
From OPTIMIZER
R-T-S CACHE
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 16/22
Apply
APPLY binds the DATA parcel values into the Plastic Steps:
• This produces Concrete Steps.
•
The DISPATCHER sends the Concrete Steps to the AMPs.• a.k.a., GncApply in V2 or OptApply in V1
Plastic
Steps
ConcreteSteps
APPLY
From SECURITY or
GENERATOR
DATA
Parcel
DISPATCHER
AMPs
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 17/22
SQL Parser Review
DATA parcel
STATISTICS
GENERATOR OPTIMIZER
APPLY
EXPLAIN
Plastic STEPS
Concrete STEPS
R-T-S Cache
Teradata’s “Late Binding” Parser
provides maximum flexibility. DISPATCHER
RESOLVER
SYNTAXER
CACHED? No
SECURITY
REQUEST Parcel
Yes
DD Cache
DD Dbase
AccessRights
RoleGrants
TVM
TVFields
Indexes
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 18/22
Parser Summary
• Plastic Steps for Requests with DATA parcels are cached immediately.
• Views and Macros cannot be nested beyond 10 levels (V2R3) or 64 levels
(V2R4).
• Nested Views and macros take longer for initial parsing.
• Multi-statement requests (including macros) generate more Parallel and
Common steps.
• Execution plans remain current in cache for up to four hours.
• DDL “spoiling” messages may purge DD cache entries at any time.
• Requests against purged entries must be re-parsed and re-optimized.
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 19/22
Review Questions
1. What must a REQUEST parcel contain? ________________
2. Which statement(s) about the RESPOND parcel are true? ___
a. Identifies response buffer size.
b. Generates a SUCCESS/FAIL parcel.
c. Always followed by one or more DATA parcels.
d. May be sent by itself as a continuation request.
3. Match the six SQL Parser sections (phases) listed below with its correct description.
__ Syntaxer a. Determines whether the Requesting User ID has the necessary permissions
__ Resolver b. Create concrete steps
__ Security c. Checks the Request for valid syntax.
__ Optimizer d. Creates the steps for execution.
__ Generator e. Breaks down Views and Macros into their underlying table references
__ Apply f. Chooses the execution plan.
4. Select the Parser phase that benefits the most from the use of macros? ___
a. Generator
b. Resolver
c. Syntaxer
d. Apply
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 20/22
Module 11: Review Question Answers
1. What must a REQUEST parcel contain? At least 1 SQL statement
2. Which statement(s) about the RESPOND parcel are true? ___
a. Identifies response buffer size.
b. Generates a SUCCESS/FAIL parcel.
c. Always followed by one or more DATA parcels.
d. May be sent by itself as a continuation request.
3. Match the six SQL Parser sections (phases) listed below with its correct description.
C Syntaxer a. Determines whether the Requesting User ID has the necessary permissions
E Resolver b. Create concrete steps
A Security c. Checks the Request for valid syntax.
F Optimizer d. Creates the steps for execution.
D Generator e. Breaks down Views and Macros into their underlying table references
B Apply f. Chooses the execution plan.
4. Select the Parser phase that benefits the most from the use of macros? ___
a. Generator
b. Resolver
c. Syntaxer
d. Apply
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 21/22
Review Questions (cont.)
5. What is the function of the Request-to-Steps (R-T-S) Cache? ____
a. Stores the SQL text and AMP steps generated by the Parser.b. Resolves View and Macro references down to table references.
c. Stores the most recently used DD information including SQL names, their related numeric IDs
and Statistics.
d. Analyzes the various ways an SQL Request can be executed and determines which of these is
the most efficient.
6. Teradata's Late Binding Parser refers to Apply, which acts upon the _____________ from the Generator and produces __________ by binding in the data values from the DATA parcel.
a. Interpretive Steps / Compiled Steps
b. Processing Steps / Execution Steps
c. AMP steps / Request-to-Steps Cache
d. Plastic Steps / Concrete Steps
7/27/2019 B211 Parser
http://slidepdf.com/reader/full/b211-parser 22/22
Module 11: Review Question Answers
5. What is the function of the Request-to-Steps (R-T-S) Cache? ____
a. Stores the SQL text and AMP steps generated by the Parser.b. Resolves View and Macro references down to table references.
c. Stores the most recently used DD information including SQL names, their related numeric IDs
and Statistics.
d. Analyzes the various ways an SQL Request can be executed and determines which of these is
the most efficient.
6. Teradata’s Late Binding Parser refers to Apply, which acts upon the _____________ from the Generator and produces __________ by binding in the data values from the DATA parcel.
a. Interpretive Steps / Compiled Steps
b. Processing Steps / Execution Steps
c. AMP steps / Request-to-Steps Cache
d. Plastic Steps / Concrete Steps