bi extractors

© Copyright IBM Corporation 2006

IBM Global Business Services

BI EXTRACTORS

LIS to LO | Confidential | SAP MM - BW - 001 | Apr 22, 2023 2



Agenda

What are Extractors (Datasources)

Types of Datasources

Overview on LO Data Extraction

Overview on CRM Data Extraction

Overview on HR Data Extraction

Overview on FI Data Extraction

Overview on COPA Data Extraction

Overview on Generic Datasources




Types of Extractors




LO Data Extraction

LO Cockpit Extractor is mainly used to extract the data from logistics modules

IMP Transations :

- To view the LO data sources : LBWE- To delete the setuptable : LBWG- To fill the setuptables : SPRO – IMG – Filling the setup tables- To view the extraction Queue : LBWQ- To view the delta Queue : RSA7




Conti….




Conti….

Full Update / Initialization :

For full update the data will be taken from setup tables

Delta Update : For capturing the changed /new records

Once a new transaction happens/an existing record is modified, upon saving it goes to the respective application table. From here depending on the update mode Direct/Queued/Unserialized V3 update the data will be populated in delta queue (RSA7) and finally reaches to BW.




CRM extraction

Customer relationship management (CRM) is broadly about managing the relationships with customers, and is useful to analyze customer, vendor, partner, and internal process information.




BW Adapter – Interface Between CRM and BW




Conti….

SAP BW calls up the BW Adapter using the Service API.

The BW Adapter reads the data from the respective database.

The selected BDoc data is converted into the extract structure from a mapping module that is also entered in the BW Adapter metadata.

The type of Business Add-In (BAdI) that is called up by the BW Adapter depends on the BDoc type

The requested data package is transferred to SAP BW using the Service API.




Delta Mechanism for the Loading Process

Any new postings/uptation of old postings from the source sytem (CRM )side will be communicated via Middleware in the form of a BDoc.

The flow controller for Middleware calls up the BW Adapter.

The BW Adapter first checks whether the change communicated via the BDoc is relevant for SAP BW. A change is relevant if a Data Source for the BDoc is active.

If the change is not relevant, it is not transferred to SAP BW and the process is complete.

If it is relevant, then the BW Adapter calls up the corresponding mapping module and BAdi (the type of BAdi that needs to be called up in turn depends on the type of BDoc).

And finally these will help in converting the BDoc data into the extract structure.




Conti….

The mapping module and the BAdis that are called up during delta upload are same as those called up during the initialization of the delta process.

The change is transferred to SAP BW using the Service API. CRM standard data sources support AIMD (After-Images with Deletion

Indicator Delta Queue)




HR Extraction

The HR module enables customers to effectively manage information about the people in their organization, and to integrate that information with other SAP modules and external systems

HR delivers a rich set of business content objects that covers all HR sub-functional areas.




Conti….

The data will be extracted from an info type (PA, PD, time management etc) and for few other applications it is from the cluster tables (Payroll, compensation etc.)

- An info type is a collection of logical and/or business-related characteristics of an object or person

HR is basically master data centric because it is always related to people related Info Objects, such as Employee, Person. In most of the cases HR master data is defined as Time Dependent to enable historical evaluation. HR R/3 system records a specific period of validity for each Info type.

Except for payroll and time management rest all sub-functional areas supports only full load.




FI Data Extraction

FI Module deals with accounting and financial needs of an organization.

Depending on the business requirement we can use either FI-SL or standard BW content extractors (FI-AR, FI-AP, and FI-GL) to fetch FI data

Here we are going to discuss about AP/AR/GL.

We can extract the financial data at totals level / line item level.

In general, we will use R/3 line item tables as the data source for extracting the data to allow drill down capability.

Financial Accounting data can be extracted directly from the tables.




Conti….




Conti….

FI Data sources writes the entries into the time stamp table BWOM2_TIMEST in the SAP R/3 system with a new upper limit for the time stamp selection.

Initialization: The status of the initialization of the Data source can be identified with the “X” for LAST_TS field in BWOM2_TIMEST table.

Delta load:- The delta identification process works differently for new financial records and for

changed financial records.- New Financial accounting line items which are posted in SAP R/3 system will be

identified by the extractor using the time stamp in the document header Table BKPF-(field) CPUDT.




Time Stamp Mechanism: New Documents




Time Stamp Mechanism: Changed Documents




Time Stamp Mechanism: Recording of Changed Documents




Conti….

FI Data sources will use after-image delta (AIE) method. This delta method is not suitable for filling Info Cubes directly in the BW system

It uses delta type E(pull) means the delta data records are determined during the delta update by the data source extractor, updated to the delta queue and passed on to BI directly from there.




CO-PA Extractor

Profitability analysis allows Management to review information with respect to the company's profit or contribution margin by business segment.

It can be obtained by the following methods:

- Account-Based Analysis - Cost-Based Analysis




Database Structure in CO-PA




Data Extraction in CO-PA




Conti….

When the data is requested from SAP BW, the extractor determines which data source the data is to be read from. This depends on the

- Update mode (full, initialization of the delta method, or delta update) - On the definition of the DataSource (line item characteristics (apart from field

REC_WAERS) or calculated key figures) - On the available summarization levels.

The extractor always tries to select the most appropriate data source, that is, the one with the smallest data volume.




Conti….

Once an Info-Package is executed, the SAP BW Staging Engine calls the CO-PA transaction data interface. CO-PA extraction program for the SAP BW uses the same replication method as the update program for CO-PA updating summarization levels. On the BW side, only data that is "at least 30 minutes old" is received .This is to secure data integrity. Because the time stamps from different application servers can be slightly different.

This retention period of 30 minutes is often described as a "security delta/Safety delta" The system only extracts data that is at least 30 Min Old.




Account-Based Analysis

Initialization: The initialization must be performed from a summarization level.

Delta update: Delta will be read from line items. - During the delta load controlling area, fiscal period fields should be mandatory.

Note: If the data needs to be read from a summarization level, then the level must also contain all the characteristics that are to be extracted using the Data Source (entry * in maintenance transaction KEDV). Furthermore, the summarization must have status ACTIVE.




Cost-Based Analysis:

In the case of costing-based CO-PA, data can only be read from a summarization level if no characteristics of the line item are selected apart from the Record Currency (REC_WAERS) field, which is always selected.

An extraction from the segment level, that is, from the combination of the tables CE3XXXX / CE4XXXX (where XXXX stands for the operating concern), is only performed for Full Updates if no line item characteristics are selected (as with summarization levels).




Conti….

Initialization: - There are two possible sources for the initialization of the delta method. One is from

Summarization levels (if no characteristics of the line item are selected) and the other one is from line item level.

- In case of Summarization level, it will also record the time when the data was last updated / built.

- If it is not possible to read data from a summarization level, data is read from line items instead.

Delta update: Data is always read from line items.




CO-PA Delta Mode:

Extraction is based on Timestamp.

When data is extracted from CO-PA, a "safety delta" of half an hour is used with the initialization and the delta upload. This always ensures that only records that are already half an hour old since the start of the upload are loaded into SAP BW. Half an hour was chosen as the safety delta to overcome any time differences between the clocks on the different application servers.




Generic extractors

Generic Extractors will be used when

- None of the SAP- predefined extractors meeting the business demand

- When Business content does not include a data source for your application

- Business content requires additional enhancements that need data that is not supplied by SAP BW.

- When the requirement demands to use your own programs to fill your tables in SAP Systems




Generic Delta

The generic delta will be depends on:

- Timestamp, - Calendar day, - Numeric pointer (e.g. document number, counter)

Generic Data Sources for transaction data, master data attributes or texts will be created from

- Transparent tables- Database views or SAP query- Functional areas or via a function module




Conti….

Specify the delta-specific field and the type for this field.

Maintain the settings for the generic delta: Specify a safety interval.

NOTE: Safety interval should be set so that no document is missed – even if it

was not stored in the DB table when the extraction took place.