oracle warehouse builder to oracle data integrator 12c migration utility

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Warehouse Builder to Oracle Data Integrator 12c Migrations

Noel Sidebotham – Oracle Data IntegrationHolger Dresing – Oracle Data IntegrationAshish Kashalkar – Oracle ConsultingAugust, 2015

Oracle Confidential – Internal/Restricted/Highly Restricted


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Confidential – Internal/Restricted/Highly Restricted 2

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |3

Agenda

• ODI 12c - Introduction

• Migration Utility Overview

• Migration Factory Services

• Demonstration

• Q&A


Oracle Data IntegrationIntroduction



Oracle Data IntegratorBulk Data Processing and Data Transformation

Oracle Data Integrator

OLTP

Applications

Legacy

Unstructured

Certified for leading technologies to deliver fast time to value

High-performance, low cost of ownership E-LT architecture

Lightweight deployment

Flexible, easy to enrich functionality

High Performance E-LT

Declarative Design

CEP Data Services

Extensible Knowledge Modules

Capable of Integrating Structured & Unstructured Data

Copyright © 2014 Oracle and/or its affiliates. All rights reserved.

Big Data: Hive, HBase, Sqoop

Best for Oracle: Merge, Spatial, Multi-Table Insert, Optimizer Hints and more

Real-Time: CDC with GoldenGate

Applications: E-Business Suite, Siebel, PeopleSoft, JD Edwards Enterprise One, JD Edwards World, SAP ERP and SAP BW

Heterogeneous: Optimizations for all major RDBMS: IBM DB2, Microsoft SQL Server, Teradata, Netezza, etc.

Applications

Legacy

Databases

Big Data

Cloud

Extended ConnectivityLeverages Existing IT, Faster Implementation


Reverse Engineer Metadata

Journalize (CDC)

Load from Source to Staging

Check Constraints

Integrate, Transform

Data Service

Knowledge ModulesSimpler Physical Design and Shorter Implementation Time

9

Oracle Sqoop Hive HBaseOracle Merge

SAP ERP

SAP BWOracle

DatapumpOracle DBLink

JMSExternal Tables

Teradata

Oracle Spatial

SiebeleBusines

SuiteIBM DB2 Netezza SCD

Key Architecture Benefits Faster development and simpler

maintenance using templates Easy to extend and add new best practices Enforces predictability and reduces cost of

ownership

Pluggable Knowledge Modules Architecture

Sample out-of-the-box Knowledge Modules

Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |

• Key Architecture Benefits: 100% Java, Open APIs, fast E-LT

DA

B

FileC

C$_0

C$_1

LKM

LKM

IKM

I$ E$ (Errors)

CKMIKMRKM

JKM

Check-LoadTransformExtract-Load

ODI Agent

Application & SOA Business Intelligence& Data Warehouse

ODI Agent may be deployed in any part of the architecture

How ODI Works: E-LT ArchitectureHigh Performance, Flexible, Lightweight Architecture

10


• Declarative Flow-Based User Interface

• Reusable Mappings

• Step By Step Debugger

Improved User Experience and Productivity

• Parallel Target Table Load

• In-Session Parallelism

• Low Overhead Session

Core Features for Better Performance

• Integration with Common Admin Model

• Integration with OPSS Enterprise RolesEnterprise Deployment

• Integration with Enterprise Manager Cloud ControlImproved Manageability

• Integration with Oracle Warehouse Builder

• Integration with Oracle GoldenGate

Deeper Oracle Stack Integration

New Features in Oracle Data Integrator 12c


Quick Overview of ODI StudioUI Overview

Object Tree

Source and Target Area

Navigators

Accordions

Toolbar

Components

Expression


A mapping can contain multiple targets

Any component can map result to multiple downstream components

Split component allows to split input logically based on expressions

Multiple Targets


• Capture common logic (sources, targets, transformations)

• Reuse logic in mappings

Reusable Mappings


Provide control to the user on the Session Execution Flow

Pause, Resume, Run to next Step, etc.

Define breakpoints on Steps in Packages

Toggle breakpoints in Sessions

Query data through Agent (gives access to uncommitted data)

Step By Step Debugger


Oracle Warehouse Builder to Oracle Data IntegratorMigration ToolIntroduction



Remarks

Oracle Data Integrator is the strategic go-forward ETL platform from Oracle

Statement of Direction: http://www.oracle.com/us/products/middleware/data-integration/odi-statement-of-direction-1922235.pdf

Existing OWB customers are NOT being forced to migrate to ODI

Oracle offers “Lifetime Support” option for Sustaining Support on all OWB features

Oracle Data Integrator 12c supports a phased migration approach

Execute and monitor existing OWB 11gR2 jobs from ODI 12c

Use the migration utility to convert existing OWB 11gR2 objects into their ODI equivalents

Start new developments using ODI 12c


OWB Support Policies

Release GA Date

Premier

Support Ends

Extended

Support Ends

Sustaining

Support Ends

OWB 9.2 Jul 2003 Oct 2007 Not Available Oct 2010

OWB 10.1 Apr 2004 Jul 2007 Not Available Jul 2010

OWB 10.2 Jul 2005 Jul 2010 Jul 2013 Indefinite

OWB 11.1 Aug 2007 Aug 2012 Aug 2015 Indefinite

OWB 11.2 Sep 2009 Jan 2015 Jan 2018 Indefinite

For more Information about Support Levels:http://www.oracle.com/us/support/lifetime-support/index.html

Customer Downloadable OWB Support Policies:http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf

Lifetime Sustaining Support for OWB Customers

http://www.oracle.com/us/support/lifetime-support/index.html

http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf


Goal Provide a migration path to OWB users Integrate OWB and ODI runtime for

phased migration

How OWB repositories can now be managed in

Topology OWB Jobs can be executed and monitored

in ODI OWB to ODI migration utility

Execute and monitor existing OWB 11gR2 jobs from ODI 12c

Oracle Warehouse Builder Integration


Migration Utility Overview

Versions supported

OWB 11.2.0.4 or 11.2.0.3 to ODI 12c

Requires an ODI-EE license

Design-time metadata migration only

Data Objects

Mappings

Separate command line utility

Uses OWB SDK and ODI SDK


Migration Scope Details

Module and Location

Data Objects

Table (Columns, Keys, Indexes)

View (Columns, Keys)

Materialized View (Columns, Keys, Indexes)

External Table (Columns)

File (Records, Fields)

Sequence

Supported OWB Design Time Metadata


Mappings

Classic Mappings, Code Template Mappings, Pluggable Mappings

Most Mapping Operators

Aggregator, Constant, Deduplicator

Expression, External Table, Filter

Flat File, Joiner, Key Lookup

Mapping Input Parameter, Materialized View, Pivot

Pre/Post Mapping Process, Sequence, Set

Sorter, Splitter, Subquery

Table, Table Function, Transformation

Unpivot, View


Supported OWB Design Time Metadata



Process Flows

Dimensional modeling metadata

Oracle Discoverer metadata and derived OBIEE metadata

Data Quality components

Custom PL/SQL

OMB*Plus scripts

OWB Experts

Unsupported OWB Design Time Metadata (sample)


ODI Related Enhancements

New components developed for ODI 12c (12.1.3)

Pivot

Unpivot

Table Function

Subquery

Enhancements to existing ODI Objects

Variable

Sequence (support for CURRVAL)

Lookup

Oracle Knowledge Modules


Migration Utility Syntax and Options

Command:

>$OWBHOME/bin/unix/migration.sh <odi_master_repos_pwd> <odi_user_pwd> <owb_wksp_owner_pwd> <config_file>

Configuration File

Connection Information to OWB and ODI

OWB design metadata to migrate

Migration options

3 running modes

Fast Check, Dry Run, Run


How to Get the Migration Utility?

Apply the following patches

OWB: #17830453

ODI: #17053768

Both are available on My Oracle Support

Supported Platforms: Windows 64-bit and Linux 64-bit

Supported Versions: OWB 11.2.0.3+ and ODI 12.1.2.0.0+

Migration documentation included in ODI documentation: http://docs.oracle.com/middleware/1212/odi/ODIMG/index.html


Migration Screenshots

Customers Mapping in OWBCustomers Mapping in ODI


Migration Screenshots

Customer Sales Mapping in ODICustomer Sales Mapping in OWB


Customer Experiences

• Average 80% of the mappings can be migrated automatically: customer with OWB in-database ETL up to 95% .

• Performance of the migrated SQL statements is comparable to the OWB processes. You have to keep in mind, that OWB generated long complex SQL-Statements with inline queries. ODI generated shorter SQL Statements, sometimes it generates more than 1 SQL Statements.

• The development of ODI mappings is faster than development of OWB mappings, about 30 %, reason is the use of the knowledge modules.

• ODI12c allowed us to maintain investment in OWB to execute non-migrated legacy code, whilst allowing us to use the new fantastic platform support like Hadoop.


Customer Examples

• International service provider in the area of:

• Supply Chain & Manufacturing,

• Solutions are based on Oracle technology

• Oracle Business Intelligence Enterprise Edition

• Endeca

• Oracle Data Integration Suite

• E-Business Suite

• Benefits

• Development time needed for EDW mappings decreased

by half

BI Infrastructure:

Enterprise dataware-

houseEDW

(Data Vault)

Analyticscubes

Reports

Statistics

Dashboards

Data-mart

DM

Staging

(pull)STG

SelfServicequeries

Pre-staging(push)

PRE

Process administration ADM

Data Sources


Oracle ConsultingMigration Factory ServicesOWB to ODI 12c



Op

erate

/ Main

tain

Migration Factory Approach

Dis

cove

r

Programme Governance

Platform Delivery

Tran

siti

on

Mig

rate

An

alyz

eto

to

to

Stre

am G

ove

rnan

ce

Further Migration Types

OWB ODI

BO OBIEE

SAP BW Oracle BI


OWB –ODI Migration: List of steps

Step DescriptionVerify your system environment

Before running the migration utility, verify that your system meets

requirements.

Review supported and unsupported objects

Make sure you understand what will and will not be migrated.

Edit the migration utility configuration file

Edit the migration utility configuration file and make sure the settings are

correct for your environment.

The configuration file contains connection information and other details

required for migration.

Run the migration utility

Run the migration utility to migrate OWB objects to ODI using the settings in

the migration utility configuration file.

Review the migration utility log file

After migration is complete, review the migration utility log file.

The file contains details about objects that were migrated, and error messages

if errors occurred.

Review the migration utility exclusion report

After migration is complete, review the migration utility exclusion report.

The report provides a summary of objects that were migrated, and lists

whether migration succeeded or failed for each object.

Update manually and migrate objects in Exclusion report

Manually update the interfaces to add functionality of objects that are not

covered by Utility

Verify your migration In ODI Studio, connect to your ODI environment and perform post-migration

testing to verify your migration.

Verify your system environment

Review supported and unsupported

objects

Edit the migration utility configuration

file

Run the migration utility

Review the migration utility log file

Review the migration utility exclusion

report. Manually fix.


Questions & Answers

oracle warehouse builder to oracle data integrator 12c migration utility

Software