05 rollup and scan components
DESCRIPTION
ComponentsTRANSCRIPT
AccentureAccenture Ab Initio TrainingAb Initio Training 11
Introduction to Ab Initio
Prepared By : Ashok Chanda
AccentureAccenture Ab Initio TrainingAb Initio Training 22
Ab initio Session 5Ab initio Session 5
RollupRollup ScanScan
AccentureAccenture Ab Initio TrainingAb Initio Training 33
RollupRollup
Rollup evaluates a group of input records Rollup evaluates a group of input records that have the same key and then generates that have the same key and then generates data records that either summarize each data records that either summarize each group or select certain information from group or select certain information from each group. each group.
Rollup gives you more control over record Rollup gives you more control over record selection, grouping, and aggregation than selection, grouping, and aggregation than Aggregate . Rollup does not produce Aggregate . Rollup does not produce intermediate summary records. If you want intermediate summary records. If you want intermediate summary records, use Scan. intermediate summary records, use Scan.
AccentureAccenture Ab Initio TrainingAb Initio Training 44
Alphabetical List of Rollup Alphabetical List of Rollup ParametersParameters
check-sort check-sort key key key-method key-method limit limit logging logging log_input log_input log_intermediate log_intermediate log_output log_output log_reject log_reject max-core max-core ramp ramp reject-threshold reject-threshold sorted-input sorted-input transform transform
AccentureAccenture Ab Initio TrainingAb Initio Training 77
Parameter Descriptions for Parameter Descriptions for RollupRollup
sorted-inputsorted-input :(boolean, required) :(boolean, required) When set to When set to In memory: Input need not be In memory: Input need not be
sortedsorted, Rollup accepts ungrouped input, and , Rollup accepts ungrouped input, and requires the use of the requires the use of the max-coremax-core parameter. parameter.
When set to When set to Input must be sorted or Input must be sorted or groupedgrouped, Rollup requires grouped input, and , Rollup requires grouped input, and the the max-coremax-core parameter is not available. parameter is not available.
Default is Default is Input must be sorted or groupedInput must be sorted or grouped..
AccentureAccenture Ab Initio TrainingAb Initio Training 88
Parameter Descriptions for Parameter Descriptions for RollupRollup
key-method key-method :(choice, optional) :(choice, optional) Method by which the component Method by which the component
groups records. Choose one of the groups records. Choose one of the following: following: Use key specifierUse key specifier — the component uses — the component uses
a key specifier. a key specifier. Use key_change functionUse key_change function — the — the
component uses the component uses the key_changekey_change transform function. transform function.
AccentureAccenture Ab Initio TrainingAb Initio Training 99
Parameter Descriptions for Parameter Descriptions for RollupRollup
keykey :(key specifier, optional) :(key specifier, optional) Names(s) of the key field(s) Rollup can use Names(s) of the key field(s) Rollup can use
to group or define groups of data records. If to group or define groups of data records. If the value of the the value of the key-methodkey-method parameter is parameter is Use key specifierUse key specifier, you must specify a , you must specify a value for the value for the keykey parameter. parameter.
If you specify If you specify Use key_change functionUse key_change function for the for the key-methodkey-method parameter, the parameter, the keykey parameter is not available. parameter is not available.
For an alternate method Rollup can use to For an alternate method Rollup can use to define groups, see the define groups, see the key-methodkey-method parameterparameter
AccentureAccenture Ab Initio TrainingAb Initio Training 1010
Parameter Descriptions for Parameter Descriptions for RollupRollup
max-core max-core :(integer, required) :(integer, required) Maximum memory usage in bytes. Maximum memory usage in bytes. Only available when the Only available when the sorted-inputsorted-input parameter parameter
is set to is set to In memory: Input need not be In memory: Input need not be sortedsorted..
The default value of The default value of max-coremax-core is is 6710886467108864 (64 (64 megabytes). megabytes).
If the total size of the intermediate results Rollup If the total size of the intermediate results Rollup holds in memory exceeds the number of bytes holds in memory exceeds the number of bytes specified in the specified in the max-coremax-core parameter, Rollup parameter, Rollup writes temporary files to disk. writes temporary files to disk.
AccentureAccenture Ab Initio TrainingAb Initio Training 1111
More on :The sorted-input More on :The sorted-input parameterparameter
Rollup can process either grouped or Rollup can process either grouped or ungrouped input, depending on the ungrouped input, depending on the setting of the sorted-input parameter setting of the sorted-input parameter .There are two choices for the sorted-.There are two choices for the sorted-input parameter:input parameter:
Input must be sorted or grouped Input must be sorted or grouped (default)(default)
In memory: Input need not be sortedIn memory: Input need not be sorted
AccentureAccenture Ab Initio TrainingAb Initio Training 1212
Input must be sorted or Input must be sorted or groupedgrouped
When you leave the sorted-input When you leave the sorted-input parameter set to this default, Rollup parameter set to this default, Rollup requires data records grouped requires data records grouped according to the key parameter . If according to the key parameter . If you need to group the records, use you need to group the records, use Sort with the same key specifier that Sort with the same key specifier that you use for Rollup.you use for Rollup.
AccentureAccenture Ab Initio TrainingAb Initio Training 1313
Input must be sorted or Input must be sorted or groupedgrouped
With sorted-input set to Input must be sorted or grouped, With sorted-input set to Input must be sorted or grouped, Rollup defines groups of data records in one of the following Rollup defines groups of data records in one of the following ways:ways:
According to the key parameter.According to the key parameter.
By the key_change transform function , if you set the key-By the key_change transform function , if you set the key-method parameter to Use key_change function.method parameter to Use key_change function.
Rollup only supports the key_change transform function when Rollup only supports the key_change transform function when the sorted-input parameter is set to Input must be sorted or the sorted-input parameter is set to Input must be sorted or grouped.grouped.
Rollup considers records as belonging to the same group if Rollup considers records as belonging to the same group if they have the same key value, or if the key_change transform they have the same key value, or if the key_change transform function returns 0 (false).function returns 0 (false).
AccentureAccenture Ab Initio TrainingAb Initio Training 1414
Input must be sorted or Input must be sorted or groupedgrouped
The key_change transform function takes two arguments: The key_change transform function takes two arguments:
A previous input recordA previous input record
The current input recordThe current input record
The key_change transform function should return 0 (false) if The key_change transform function should return 0 (false) if the key values in these two records are the same, or non-0the key values in these two records are the same, or non-0
(true) if they are not the same.(true) if they are not the same.
If you want Rollup to treat all records as one group, specify If you want Rollup to treat all records as one group, specify the value in the key parameter as {}.the value in the key parameter as {}.
AccentureAccenture Ab Initio TrainingAb Initio Training 1515
In memory: Input need not In memory: Input need not be sortedbe sorted
When you set the sorted-input parameter to this choice, When you set the sorted-input parameter to this choice, Rollup accepts ungrouped input, and groups all records Rollup accepts ungrouped input, and groups all records according to the key parameter .With sorted-input set to In according to the key parameter .With sorted-input set to In memory: Input need not be sorted, Rollup maximizes memory: Input need not be sorted, Rollup maximizes performance by holding intermediate results in main memory.performance by holding intermediate results in main memory.
The size of the table of intermediate results (until it reaches The size of the table of intermediate results (until it reaches the number of bytes specified in the max-core parameter) the number of bytes specified in the max-core parameter) depends on:depends on:
The number of distinct key valuesThe number of distinct key values
The size of the input record format The size of the input record format
The size of temporary_typeThe size of temporary_type
AccentureAccenture Ab Initio TrainingAb Initio Training 1616
Built-in Functions for Built-in Functions for RollupRollup
The following aggregation functions The following aggregation functions are predefined and are only available are predefined and are only available in the rollup component:in the rollup component:
avgavg maxmax
countcount minmin
firstfirst productproduct
lastlast sumsum
AccentureAccenture Ab Initio TrainingAb Initio Training 1717
Modes of Roll upModes of Roll up
Template ModeTemplate Mode
Use of functions such as sum() ,count() ,min() ,max()Use of functions such as sum() ,count() ,min() ,max()
Expanded ModeExpanded Mode
To find other details about min or max value record.To find other details about min or max value record.
Example : Example : you want to determine the price of you want to determine the price of the largest single purchase and the item that the largest single purchase and the item that was purchased was purchased
AccentureAccenture Ab Initio TrainingAb Initio Training 1818
Rollup : Template Mode Rollup : Template Mode (Grid View)(Grid View)
AccentureAccenture Ab Initio TrainingAb Initio Training 1919
Rollup : Template Mode Rollup : Template Mode (Text View)(Text View)
Out ::rollup(in) =begin out.cust_id :: in.cust_id; out.first_date :: min(in.dt); out.last_date :: max(in.dt); out.total_amount :: sum(in.amount);end;
AccentureAccenture Ab Initio TrainingAb Initio Training 2121
Runtime behavior of Runtime behavior of RollupRollup
Rollup reduces each group of data records Rollup reduces each group of data records to a single aggregate output record, using to a single aggregate output record, using a series of transform functions as follows:a series of transform functions as follows:
With the first data record of each group, With the first data record of each group, Rollup creates a temporary aggregate Rollup creates a temporary aggregate record. record.
With each following data record of the With each following data record of the same group, Rollup updates the temporary same group, Rollup updates the temporary aggregate record. aggregate record.
AccentureAccenture Ab Initio TrainingAb Initio Training 2222
Runtime behavior of Runtime behavior of RollupRollup
When you set sorted-input to Input must be When you set sorted-input to Input must be sorted or grouped, Rollup writes the sorted or grouped, Rollup writes the temporary aggregate record to the out port temporary aggregate record to the out port after processing the last data record of each after processing the last data record of each group, and repeats the preceding process group, and repeats the preceding process with the next group.with the next group.
When you set sorted-input to In memory: When you set sorted-input to In memory: Input need not be sorted, Rollup processes Input need not be sorted, Rollup processes all the data records, and then writes all the all the data records, and then writes all the temporary aggregate records to the out temporary aggregate records to the out port.port.
AccentureAccenture Ab Initio TrainingAb Initio Training 2323
At runtime, Rollup executes At runtime, Rollup executes the following steps:the following steps:
Input selection : Input selection : optionaloptional Temporary initialization :Temporary initialization :optionaloptional RollupRollup FinalizationFinalization Output selection : Output selection : optionaloptional
AccentureAccenture Ab Initio TrainingAb Initio Training 2424
Runtime : Input selectionRuntime : Input selection
If you have defined input_select, it filters the input If you have defined input_select, it filters the input records. records.
If the expression evaluates to 0 for a particular If the expression evaluates to 0 for a particular record , Rollup does not process the record (i.e., record , Rollup does not process the record (i.e., the record does not appear on any output port).the record does not appear on any output port).
If the expression produces NULL for a particular If the expression produces NULL for a particular record , Rollup:record , Rollup:
Writes the record to the reject port Writes the record to the reject port Writes a descriptive error message to the error Writes a descriptive error message to the error
portport
AccentureAccenture Ab Initio TrainingAb Initio Training 2525
Runtime : Input selectionRuntime : Input selection
If you do not connect flows to the reject If you do not connect flows to the reject or error ports, Rollup discards the or error ports, Rollup discards the information. information.
If the expression evaluates to anything If the expression evaluates to anything other than 0 or NULL for a particular other than 0 or NULL for a particular record, Rollup processes the record.record, Rollup processes the record.
If you have not defined input_select, If you have not defined input_select, Rollup processes all records.Rollup processes all records.
AccentureAccenture Ab Initio TrainingAb Initio Training 2626
Runtime : Temporary Runtime : Temporary initializationinitialization
Rollup passes the first record in each Rollup passes the first record in each group to the initialize transform group to the initialize transform function.function.
The initialize transform function The initialize transform function creates a temporary record for the creates a temporary record for the group, with record type group, with record type temporary_type. temporary_type.
AccentureAccenture Ab Initio TrainingAb Initio Training 2727
Runtime :ComputationRuntime :Computation
Rollup calls the rollup transform Rollup calls the rollup transform function for each record in a group, function for each record in a group, using that record and the temporary using that record and the temporary record for the group as arguments.record for the group as arguments.
The rollup transform function returns The rollup transform function returns a new temporary record. a new temporary record.
AccentureAccenture Ab Initio TrainingAb Initio Training 2828
Runtime :FinalizationRuntime :Finalization
Rollup repeats this procedure with each group.Rollup repeats this procedure with each group.
If you set sorted-input to In memory: Input need If you set sorted-input to In memory: Input need not be sorted:not be sorted:
After Rollup processes all the input records, it After Rollup processes all the input records, it calls the finalize transform function with the calls the finalize transform function with the temporary record for each group and an arbitrary temporary record for each group and an arbitrary input record from each group as arguments.input record from each group as arguments.
The finalize transform function produces an The finalize transform function produces an output record for each group.output record for each group.
AccentureAccenture Ab Initio TrainingAb Initio Training 2929
Runtime :Output selectionRuntime :Output selection
If you have defined the output_select transform function, it If you have defined the output_select transform function, it filters the output records. filters the output records.
The output_select transform function takes a single The output_select transform function takes a single argument - the record produced by finalization - and argument - the record produced by finalization - and returns a value of 0 (false) or non-0 (true).returns a value of 0 (false) or non-0 (true).
Rollup ignores records for which output_select returns 0; it Rollup ignores records for which output_select returns 0; it writes all others to the out port. writes all others to the out port.
If you have not defined the output_select transform If you have not defined the output_select transform function, Rollup writes all records to the out port.function, Rollup writes all records to the out port.
AccentureAccenture Ab Initio TrainingAb Initio Training 3030
Explanation of types and Explanation of types and transform functionstransform functions
temporary_typetemporary_type : Rollup requires temporary : Rollup requires temporary storage. The first section of the example defines storage. The first section of the example defines temporary storage as a record with a type named temporary storage as a record with a type named temporary_type: temporary_type:
type temporary_type type temporary_type recordrecord date (“DDMMYYYY") first_dt;date (“DDMMYYYY") first_dt; date (“DDMMYYYY") last_dt;date (“DDMMYYYY") last_dt; decimal(4) total_amt;decimal(4) total_amt; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 3131
Explanation of types and Explanation of types and transform functionstransform functions
initializeinitialize - The initialize transform function - The initialize transform function initializes temporary storage by setting the initializes temporary storage by setting the total_amount field of the temporary record to 0 and total_amount field of the temporary record to 0 and by saving the value from the dt field of the first by saving the value from the dt field of the first input record to the first_date and last_date fields of input record to the first_date and last_date fields of the temporary record:the temporary record:
Out :: initialize(in) =Out :: initialize(in) = beginbegin out.first_dt :: in.dt;out.first_dt :: in.dt; out.last_dt :: in.dt;out.last_dt :: in.dt; out.total_amt :: 0;out.total_amt :: 0; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 3232
Explanation of types and Explanation of types and transform functionstransform functions
rolluprollup - The rollup transform function fills the fields - The rollup transform function fills the fields in the temporary record for each group as follows:in the temporary record for each group as follows:
It accumulates, in the total_amt field, a sum of the It accumulates, in the total_amt field, a sum of the values in the amount field of the input records. values in the amount field of the input records.
It selects the smallest value of the dt field from the It selects the smallest value of the dt field from the group of input records, and inserts it in the first_dt group of input records, and inserts it in the first_dt field. field.
It selects the largest value of the dt field from the It selects the largest value of the dt field from the group of input records, and inserts it in the last_dt group of input records, and inserts it in the last_dt field.field.
AccentureAccenture Ab Initio TrainingAb Initio Training 3333
Explanation of types and Explanation of types and transform functionstransform functions
Out :: rollup (tmp, in) =Out :: rollup (tmp, in) =beginbeginout.first_dt :: if (in.dt < tmp.first_dt)out.first_dt :: if (in.dt < tmp.first_dt) in.dtin.dt elseelse tmp.first_dt;tmp.first_dt;
out.last_dt :: if (in.dt > tmp.last_dt)out.last_dt :: if (in.dt > tmp.last_dt) in.dtin.dt elseelse tmp.last_dt;tmp.last_dt;out.total_amount :: tmp.total_amt + in.amount;out.total_amount :: tmp.total_amt + in.amount;end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 3434
Explanation of types and Explanation of types and transform functionstransform functions
finalize finalize - The finalize transform function gathers the - The finalize transform function gathers the values of the customer_id field from the last record of the values of the customer_id field from the last record of the group, and the value of the first_date, last_date, and group, and the value of the first_date, last_date, and calculated total_amount fields from the temporary record calculated total_amount fields from the temporary record for the group, and produces the output record:for the group, and produces the output record:
out :: finalize (tmp, in) =out :: finalize (tmp, in) = beginbegin out.customer_id :: in.customer_idout.customer_id :: in.customer_id out.first_date :: tmp.first_dt;out.first_date :: tmp.first_dt; out.last_date :: tmp.last_dt;out.last_date :: tmp.last_dt; out.total_amount :: tmp.total_amt;out.total_amount :: tmp.total_amt; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 3535
SCANSCAN
Scan generates a series of Scan generates a series of cumulative summary records - such cumulative summary records - such as successive year-to-date totals - for as successive year-to-date totals - for groups of data records.groups of data records.
Scan produces intermediate Scan produces intermediate summary records. If you do not want summary records. If you do not want intermediate summary records, use intermediate summary records, use Rollup .Rollup .
AccentureAccenture Ab Initio TrainingAb Initio Training 3636
Alphabetical List of Scan Alphabetical List of Scan ParametersParameters
check-sort check-sort key key key-method key-method limit limit logging logging log_input log_input log_intermediate log_intermediate log_output log_output log_reject log_reject max-core max-core ramp ramp reject-threshold reject-threshold sorted-input sorted-input transform transform
AccentureAccenture Ab Initio TrainingAb Initio Training 3939
Parameter Descriptions for Parameter Descriptions for ScanScan
sorted-input : sorted-input : (Boolean, required) (Boolean, required) When set to When set to In memory: Input need not be In memory: Input need not be
sortedsorted, Scan accepts ungrouped input, and , Scan accepts ungrouped input, and requires the use of the requires the use of the max-coremax-core parameter. parameter.
When set to When set to Input must be sorted or Input must be sorted or groupedgrouped, Scan requires grouped input, and the , Scan requires grouped input, and the max-coremax-core parameter is not available. parameter is not available.
Default is Default is Input must be sorted or groupedInput must be sorted or grouped..
AccentureAccenture Ab Initio TrainingAb Initio Training 4040
Parameter Descriptions for Parameter Descriptions for ScanScan
key-method : key-method : (choice, optional) (choice, optional) Method by which the component Method by which the component
groups records. Choose one of the groups records. Choose one of the following:following: Use key specifierUse key specifier — the component uses — the component uses
a key specifier. a key specifier. Use key_change functionUse key_change function — the — the
component uses the component uses the key_changekey_change transform function. transform function.
AccentureAccenture Ab Initio TrainingAb Initio Training 4141
Parameter Descriptions for Parameter Descriptions for ScanScan
key : key : (key specifier, optional) (key specifier, optional) Name(s) of the key field(s) Scan can Name(s) of the key field(s) Scan can
use to group or define groups of data use to group or define groups of data records.records.
For an alternate method Scan can For an alternate method Scan can use to define groupsuse to define groups
AccentureAccenture Ab Initio TrainingAb Initio Training 4242
Parameter Descriptions for Parameter Descriptions for ScanScan
max-core : max-core : (integer, required) (integer, required) Maximum memory usage in bytes. Only Maximum memory usage in bytes. Only
available when the available when the sorted-inputsorted-input parameter parameter is set to is set to In memory: Input need not be In memory: Input need not be sortedsorted. The default value of . The default value of max-coremax-core is is 1048576010485760 (10 megabytes). (10 megabytes).
If the total size of the intermediate results If the total size of the intermediate results Scan holds in memory exceeds the number Scan holds in memory exceeds the number of bytes specified in the of bytes specified in the max-coremax-core parameter, Scan writes temporary files to parameter, Scan writes temporary files to disk. disk.
AccentureAccenture Ab Initio TrainingAb Initio Training 4343
Specifying a Transform for Specifying a Transform for ScanScan
Input selection : Input selection : optionaloptional Temporary initialization : Temporary initialization : optionaloptional RollupRollup FinalizationFinalization Output selection : Output selection : optionaloptional Key_change : Key_change : optionaloptional
AccentureAccenture Ab Initio TrainingAb Initio Training 4444
Determining Whether to Determining Whether to Scan Sorted or Unsorted Scan Sorted or Unsorted
InputInput Scan can process either sorted or Scan can process either sorted or
unsorted input, depending on the unsorted input, depending on the setting of the setting of the sorted-inputsorted-input parameter parameter. . The setting can be one of the following: The setting can be one of the following:
Input must be sorted or groupedInput must be sorted or grouped (default) (default)
In memory: Input need not be In memory: Input need not be sortedsorted
AccentureAccenture Ab Initio TrainingAb Initio Training 4545
Runtime behavior of Runtime behavior of ScanScan
Scan produces a series of n output records for each group of Scan produces a series of n output records for each group of n input data records, computing each output record as an n input data records, computing each output record as an aggregation of the input records processed so far, as follows:aggregation of the input records processed so far, as follows:
With the first data record of each group, Scan creates a With the first data record of each group, Scan creates a temporary aggregate record. temporary aggregate record.
With each following data record of the same group, Scan With each following data record of the same group, Scan updates the temporary aggregate record, producing an updates the temporary aggregate record, producing an aggregate record of the records processed so far for each aggregate record of the records processed so far for each group.group.
After processing each data record, Scan writes the aggregate After processing each data record, Scan writes the aggregate record to the out port , using a series of transform functions.record to the out port , using a series of transform functions.
AccentureAccenture Ab Initio TrainingAb Initio Training 4646
At runtime, Scan executes At runtime, Scan executes the following steps:the following steps:
Input selection : Input selection : optionaloptional Temporary initialization :Temporary initialization :optionaloptional ComputationComputation FinalizationFinalization Output selection : Output selection : optionaloptional
AccentureAccenture Ab Initio TrainingAb Initio Training 4747
Runtime : Input selectionRuntime : Input selection
If you have defined input_select, it filters the If you have defined input_select, it filters the input records. input records.
If the expression evaluates to 0 for a If the expression evaluates to 0 for a particular record , Scan does not process particular record , Scan does not process the record (i.e., the record does not the record (i.e., the record does not appear on any output port).appear on any output port).
If the expression produces NULL for a If the expression produces NULL for a particular record , Scan :particular record , Scan :
Writes the record to the reject port Writes the record to the reject port Writes a descriptive error message to the Writes a descriptive error message to the
error porterror port
AccentureAccenture Ab Initio TrainingAb Initio Training 4848
Runtime : Input selectionRuntime : Input selection
If you do not connect flows to the reject If you do not connect flows to the reject or error ports, Scan discards the or error ports, Scan discards the information. information.
If the expression evaluates to anything If the expression evaluates to anything other than 0 or NULL for a particular other than 0 or NULL for a particular record, Scan processes the record.record, Scan processes the record.
If you have not defined input_select, Scan If you have not defined input_select, Scan processes all records.processes all records.
AccentureAccenture Ab Initio TrainingAb Initio Training 4949
Runtime : Temporary Runtime : Temporary initializationinitialization
Scan passes the first record in each Scan passes the first record in each group to the initialize transform group to the initialize transform function.function.
The initialize transform function The initialize transform function creates a temporary record for the creates a temporary record for the group, with record type group, with record type temporary_type. temporary_type.
AccentureAccenture Ab Initio TrainingAb Initio Training 5050
Runtime :ComputationRuntime :Computation
Scan calls the Scan transform Scan calls the Scan transform function for each record in a group, function for each record in a group, using that record and the temporary using that record and the temporary record for the group as arguments.record for the group as arguments.
The Scan transform function returns The Scan transform function returns a new temporary record. a new temporary record.
AccentureAccenture Ab Initio TrainingAb Initio Training 5151
Runtime :FinalizationRuntime :Finalization
Scan can calls the finalize transform Scan can calls the finalize transform function once for every input record.function once for every input record.
Scan passes the temporary record and the Scan passes the temporary record and the input record to the finalize transform input record to the finalize transform function.function.
The finalize transform function produces an The finalize transform function produces an output record for each input record.output record for each input record.
AccentureAccenture Ab Initio TrainingAb Initio Training 5252
Runtime :Output selectionRuntime :Output selection
if you have defined the output_select transform function, it if you have defined the output_select transform function, it filters the output records. filters the output records.
The output_select transform function takes a single The output_select transform function takes a single argument - the record produced by finalization - and argument - the record produced by finalization - and returns a value of 0 (false) or non-0 (true). returns a value of 0 (false) or non-0 (true).
Scan ignores records for which output_select returns 0; it Scan ignores records for which output_select returns 0; it writes all others to the out port. writes all others to the out port.
If you have not defined the output_select transform If you have not defined the output_select transform function, Scan writes all records to the out port.function, Scan writes all records to the out port.
AccentureAccenture Ab Initio TrainingAb Initio Training 5353
Explanation of types and Explanation of types and transform functionstransform functions
temporary_typetemporary_type : Scan requires temporary : Scan requires temporary storage. The first section of the example defines storage. The first section of the example defines temporary storage for Scan as a record with a type temporary storage for Scan as a record with a type named temporary_type. This type contains a field named temporary_type. This type contains a field named amount_to_date, which provides a place for named amount_to_date, which provides a place for Scan to store the running total it calculates: Scan to store the running total it calculates:
type temporary_type =type temporary_type = recordrecord decimal(8.2) amount_to_date;decimal(8.2) amount_to_date; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 5454
Explanation of types and Explanation of types and transform functionstransform functions
initializeinitialize - The initialize transform - The initialize transform function initializes temporary storage by function initializes temporary storage by setting the amount_to_date field of the setting the amount_to_date field of the temporary record to 0 :temporary record to 0 :
temp :: initialize(in) =temp :: initialize(in) = beginbegin temp.amount_to _date :: 0;temp.amount_to _date :: 0; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 5555
Explanation of types and Explanation of types and transform functionstransform functions
scan scan - The scan transform function accumulates, - The scan transform function accumulates, in the amount_to_date field of the temporary in the amount_to_date field of the temporary record for each group, a running total of the record for each group, a running total of the values in the amount field of the input records values in the amount field of the input records from each group processed so far:from each group processed so far:
out :: scan(temp, in) =out :: scan(temp, in) = beginbegin out.amount_to_date :: out.amount_to_date ::
temp.amount_to_date + in.amount;temp.amount_to_date + in.amount; end;end;
AccentureAccenture Ab Initio TrainingAb Initio Training 5656
Explanation of types and Explanation of types and transform functionstransform functions
finalize finalize - The finalize transform function gathers - The finalize transform function gathers the values of the customer_id and dt fields from the the values of the customer_id and dt fields from the current record, and the value of the current record, and the value of the amount_to_date field from the temporary record, amount_to_date field from the temporary record, and produces an output record:and produces an output record:
out :: finalize(temp, in) =out :: finalize(temp, in) = beginbegin out.customer_id :: in.customer_id;out.customer_id :: in.customer_id; out.dt :: in.dt;out.dt :: in.dt; out.amount_to_date :: out.amount_to_date ::
temp.amount_to_date;temp.amount_to_date; end;end;