] sherryanne meyer [ asug installation member member since: 2000 anup maheshwari [ asug installation...
TRANSCRIPT
]
SHERRYANNE MEYER[ASUG INSTALLATION MEMBER MEMBER SINCE: 2000
ANUP MAHESHWARI[ASUG INSTALLATION MEMBER MEMBER SINCE: 2008
AJAY VONKARERY[ASUG INSTALLATION MEMBER MEMBER SINCE: 1999
ASUG Webcast: Exploring the Capabilities of SAP Data Integration
and Data Cleansing Tools
Bjarne BergDirector SAP BI
Real Experience. Real Advantage.
[
2 2
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
3
SAP data service capabilities delivered in the SAP Data Integrator and SAP Data Quality Management tools. Learn what is new, what is simple to implement and what requires a bit more effort based on experiences from real projects and lessons learned from the field.
Explore the limitations and benefits of the tools, as well as what options each tool provides from a technical and business perspective.
This Webcast will explore the new capabilities and the roadmap for integrating new features and functions in 2009 and 2010 and what can realistically be achieved by organizations. The session is intended for beginner and intermediate level attendees.
Learning Points
Real Experience. Real Advantage.
[
4 4
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
5 5
The 3-Tiers of Information Management
Information management from an SAP perspective is six distinct efforts with different tools and some overlapping of functionality.
Therefore the SAP BOBJ tools are many with various capabilities
Applications
ERP, SCM, CRM
Business Intelligence
Data Synchronization &
Migration
Performance
Management
Information Management
Data Federation
Data Integration
Text Analysis
Metadata Mgmt.
Masterdata Mgmt.
Data Quality
Structured UnstructuredData Data
RDBMS
ERP
RDBMS
ERP
Notes
Web
Docs
Real Experience. Real Advantage.
[
7
The total BOBJ Data Management toolset
http://www.sap.com/solutions/sapbusinessobjects/large/information-management/index.epx
There are many BusinessObjects data quality and integration tools that are not specific to SAP.
The tool landscape can be very confusing and the best approach is to examine this SAP site.
Real Experience. Real Advantage.
[
8 8
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
9
SAP BusinessObjects Data Services XI 3.1
BusinessObjects Data Services XI 3.1 is a data movement, cleansing & integration tool.
1. Data Services Designer allows you to create jobs (applications) that include transformations and data mappings
2. The Data Services XI 3.1 RealTime tool supports real-time data movement for integration to web pages, applications and other systems.
3. Previously you had these functions in BusinessObjects Data Integrator XI 2 and Data Quality XI 2
Image: SAP AG, Aug. 2009
Real Experience. Real Advantage.
[
10
The XI 3.1 Data Services Architecture
The tool architectural view of SAP BusinessObjects Data Services XI 3.1
ProcessData
ValidationData
CleansingData
Auditing
Data Profiling
SourceData
PeopleSoft
Oracle Apps
Data Services Engine
Siebel
SAP R/3
Oracle DB
SAP BI NetWeave
r
SQL DB
DB2
XML
Files
Mainframe Excel
OthersSAP ECC
TargetData
PeopleSoft
Oracle Apps
Siebel
SAP R/3
Oracle DB
SAP BI NetWeave
r
SQL DB
DB2
XML
Files
Mainframe Excel
OthersSAP ECC
Impact Analysis
Data Lineage
Real Experience. Real Advantage.
[
11
Pre-delivered connectors to systems and databases
Databases1. Oracle2. SQL Server3. IBM DB24. Sybase & IQ5. MySQL6. Informix7. Teradata8. Netezza9. ODBC
Applications1. SAP R/3 & ECC
– ABAP– BAPI– Idoc
2. SAP NetWeaver BI3. JD Edwards4. Oracle Apps5. Siebel6. Salesforce.com7. PeopleSoft
Transports & File formats1. XML2. SOAP -Web Service3. Cobol4. HTTP5. JMS6. Excel7. EBCDIC8. Text fixed width9. Text delimited
MainFrames1. Enscribe2. ADABAS3. IMS/DB4. RMS5. VSAM6. ISAM
Non-Structured Data• 30+ languages• Any fileformat
All major platforms are supported with pre-delivered connectors that can be installed for data movement
The high-performance parallel data processing also supports grid computing platforms for batch and real-time execution
Real Experience. Real Advantage.
[
1212
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
13
The Components
Data Services Job ServerThis application launches the Data Services processing engine and provides an engine interface and access to other components.
Data Services engineThis engine executes jobs defined in the application and creates the needed engines for maximum performance.
Central Repository
Local Repository
Data Services Designer
Web Administrator
Job Server & Engine
Access Server
Data target Server
Data SourceServers
Web Applications
Data Services DesignerThis GUI is where you design ETL and cleansing jobs. The interface is intended to be used to develop applications that are specifying work flows (job execution definitions) & data flows (data transformation definitions).
Note: A Workflow may consist of many data flows. Data-flows are source-target focused, while Workflows are an entire job (think process chains in SAP BI)
Real Experience. Real Advantage.
[
1414
The Components
Data Services RepositoryA local database that contains pre-delivered and user-defined objects (i.e. transformation rules). You can also create a central repository for version control and to share objects,
Central Repository
Local Repository
Data Services Designer
Web Administrator
Job Server & Engine
Access Server
Data target Server
Data SourceServers
Web Applications
Data Services Access ServerProvides reliable processing on request-response messages between applications, engines and the Job Server.
Data Services AdministratorWeb browser-based administration of Data Services (i.e. kick-off batch jobs, scheduling and performance monitoring).
Real Experience. Real Advantage.
[
15
How Does it Work
There are several steps to implement Data Services XI 3.1, In the following slides we will highlight the major tasks
1) Create a local repository for the install
2) Add a job server in the Data Service – Service Manager
3) Associate the local repository with the job server
Central Repository
Local Repository
Data Services Designer
Web Administrator
Job Server & Engine
Access Server
Data target Server
Data SourceServers
Web Applications
Real Experience. Real Advantage.
[
16
The Data Service Designer
The Data Service Designer is the nerve center of the Data Services. This is where most of the time is spent during the development projects.
Image: SAP AG, Aug. 2009
Projects
Object Library (local)
Tools
Work area
Real Experience. Real Advantage.
[
17
The Administrator Interface
From the administrator Interface you can monitor jobs, start and stop web services, manage repositories, servers, connection and source system definitions.
This is where you spend most of your time after the system has been developed.
Real Experience. Real Advantage.
[
18
Impact Analysis and Lineage
Lineage is an end-user view that shows how Calculated Key Figures (CKF) are calculated from the source to the target.
This tool increases the likelihood that people will trust your data.
Impact analysts is a tool to determine who will be affected by a change in the IT system (i.e. who is using this measure or characteristic)
Real Experience. Real Advantage.
[
1919
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
20
Data Cleansing Capabilities
The Profile This tab in the “view data” screen contains data profile statistics on each column that can help you decide on the quality of the input data.
The system automatically captures the following statistics in a profile grid.
1. Column Name2. Number of distinct values in a column3. Number of records with a NULL value in this column4. Maximum & Minimum value of the column
Real Experience. Real Advantage.
[
21
Data Cleansing Capabilities
The ValidationValidation allows you to create rules for cleaning data prior to loading it to the system. You can have a pass rule and an Action on Failure that can provide complex logic.
Real Experience. Real Advantage.
[
22
Data Cleansing Capabilities
The AuditThe Auditing selectionallows you to take complex actions when the data quality is poor. You can:
1.Send an email to an administrator
2.Load the data to a table for later correction
3.Modify the data through scripts
4.Create custom functions for your own processing logic
Real Experience. Real Advantage.
[ Universal Data Cleansing: Example of Enhanced Party Masterdata
Source: SAP AG, 2009
You can also add new items such as geocodes for visualization in SAP BI I.e. maps
You can add new characteristics to the data such as:
1) Legal tax jurisdictions 2) Census track ID3) Block group ID4) Insurance rating
territories5) Tax authority name6) Tax authority FIPS codes7) Longditude & Latitude8) City type9) ...
GREAT FEATURE: The Census track ID allows you to analyze your customers and partners using government census information
Real Experience. Real Advantage.
[ Universal Data Cleansing: Customer Aggregating and Discovery
Source: SAP AG, 2009
A common way to look at customer data is by Households instead of single records.
BOBJ DQ allows you to look at customer's addresses and create shared master records, customer mapping keys, aggregating data (i.e. aggregated sales data for the household), check "no-call" lists, examining churn (apparent customer turn-over).
You can also integrating all masterdata from many records into a single "super record" that contains all the unique masterdata you have about a single customer or partner.
Real Experience. Real Advantage.
[ Universal Data Cleansing: Data integration & BAS
SAP Data Quality Management has pre-delivered content for many solutions including CRM -> ECC integration. This include:
1) Across platform search capabilities2) Automated address correction 3) De-Duplication of records4) Direct system connection (no file extraction)5) Supported for all major releases: R/3 4.6c; ECC 5 and 6; CRM 4 and 5
BAS is the Business Address Service feature. With this you can:
1) Use Postal reference files from 190 countries to clean address, including suggestion lists
2) Data scans and searches in SAP for duplicate records using partial user input.
"Data Quality Management for SAP provides a prepackaged native integration of data quality best practices within the SAP environment using the BOBJ Data Services platform"
SAP AG, 2009
Real Experience. Real Advantage.
[
2626
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
2727
Interesting use for SAP NetWeaver BI
Using BOBJ Data Services you can consolidate data from many source systems, cleanse and integrate them before you send it to SAP BI. This avoids multi-nested DSOs and complex load logic.
Source systems- Oracle- JDE- Peoplesoft- Baan- Siebel- Custom- Hyperion- Other.
Real Experience. Real Advantage.
[
2828
Interesting use BOBJ Data Service XI 3.1 for SAP ECCUsing BOBJ Data Services you integrate, cleanse and merge data from source systems during
1) ECC implementation projects, 2) Retirement of legacy systems, 3) Mergers and Acquisitions.
Source systems- Oracle- JDE- Peoplesoft- Baan- Siebel- Custom- Hyperion- Other.
Real Experience. Real Advantage.
[
2929
What is New in XI 3.1
Expanded matching capabilities to allow the business user to select other fields (beyond street name and zip code) within the generation of break keys.
An improved method to install the functionality of this product into your IC WebClient or CRM IC WebClient environment. To do so, you add a Component Usage to the Component to which you want to add Postal Validation.
If you have purchased the geocoding option for this product, geocoding allows you to return latitude, longitude, and relevant status information for a U.S. address record
Real Experience. Real Advantage.
[
3030
What is New in XI 3.1
The Business Add-Ins are supported on SAP CRM 2007 (Basis version 7.00).
The RFC Server is supported on the following operating systems:
HP-UX 11i v2 (11.23) (Itanium)IBM AIX 5.2 and 5.3Red Hat Linux Enterprise Server 4 and 5Red Hat Advanced Server 4 and 5Solaris 9 and 10SuSE Enterprise Server 9 SP3 and 10Windows XP (32 bit)Windows 2003 Server (32 bit)
On Windows, the ability to install the RFC Server as a Windows Service or a stand-alone program.
Use of BusinessObjects Data Services XI 3.1 SP1 (v12.1.1) for its data quality operations.
Real Experience. Real Advantage.
[
3131
Agenda
Introduction
BOBJ Data Management Tool overview
SAP BusinessObjects Data Services XI 3.1 Overview
Components and capabilities of SAP BusinessObjects Data Services XI 3.1
Data Cleansing
Some Ideas & What is New
Wrap-up
Real Experience. Real Advantage.
[
32
ResourcesCOMERIT Inc. Downloadshttp://www.comeritinc.com/Downloads.htm
SAP BusinessObjects Data management web site:http://www.sap.com/solutions/sapbusinessobjects/large/information-management/index.epx
SAP Data Quality web site:http://www.sap.com/solutions/sapbusinessobjects/large/information-management/data-quality-management/index.epx
SAP BOBJ - Data Insight:http://www.sap.com/solutions/sapbusinessobjects/large/information-management/data-quality-management/datainsight/index.epx
Real Experience. Real Advantage.
[
33
Questions and Answers
How to contact me:Dr. Bjarne Berg