oracle data integration - · pdf filehardware & siebel migrations ... high availability...

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1

Oracle Data Integration Real Time Access to Real Time Information

Tony Velardo DIS Sales Manager

Valarie Bedard Principle Sales Consultant

Oracle Data Integration Moving Data to Transform Business


OLTP & ODS Systems Data

Warehouse, Data Mart

Oracle PeopleSoft, Siebel, SAP

Custom Apps

Files Excel XML

SOA Applications

Custom Reporting Packaged

Applications

Business

Intelligence

Analytics

Data

Federation

Data

Warehousing

Custom

Data Marts Data Access

Data Silos

SQL Java

Batch Scripts

Data Hubs

Data

Migration

Data

Replication

Fragmented

Data Silos

Slow

Performance

Lack of

Trusted Data

OLAP

Out of sync What’s the

cost?

IT Obstacles to Unifying Information Avoiding the Accidental Architecture


Choosing the Right Data Integration Platform Key Criteria for Maximum ROI from Cloud

• Comprehensive set of capabilities

• Flexible to address multiple use cases

• Support for heterogeneous platforms Enterprise-Class

• High performance to handle spikes

• Easy and cost-efficient to scale

High Performance

• Easy to deploy and manage

• Pre-integrated, certified solutions

• Low cost of ownership

Fast Time to Value


Modernization MDM SOA Big Data

Oracle Data Integration Complete Offering for Enterprise Data Integration

Complete and best-of-breed

approach for enterprise data

integration

Maximum performance with

lower TCO, ease of use and

reliability

Certified for leading

technologies to deliver fast

time to value

Oracle Data Integrator

Oracle GoldenGate

Oracle Enterprise Data Quality

Active Data Guard

OLTP

Applications

Legacy

Unstructured Oracle Customers Report:

80% lower TCO

Five times higher performance

70% reduction in development costs

Synchronization Custom BI


Data Integration Solutions

Business Intelligence

Consolidation / Migration

High Availability

Need fresh data 24X7 to optimize operations?

Mission-critical OLTP?

Minimize impact on source systems?

Need operational visibility in real time?

Growing data volumes and shrinking batch windows?

• Oracle GoldenGate provides real-time data for analysis

• Oracle Data Integrator (ODI) populates DW with high performance and low TCO

Do you need 24/7 system availability?

Is your standby database sitting idle?

Is planned downtime at least as costly as unplanned?

Need to synchronize data across geographies?

• Oracle Active Data Guard ensures business continuity and resource utilization

• Oracle GoldenGate reduces planned downtime and enables Active-Active

Want to get onto lower cost standard infrastructure?

Can’t afford downtime?

Does your data cost $$ or make $$?

Why not fix data assets as well as HW/SW assets during an

infrastructure move?

• Oracle GoldenGate enables zero downtime migrations and upgrades

• Oracle Data Quality for ODI moves and improves data so it can be trusted


Oracle GoldenGate Real-time Data Integration

High-performance, low-

impact real-time data

integration

Timely data for improved

business insight

Continuous availability for

24/7 operations

Oracle GoldenGate

OLTP

Applications

Legacy

Unstructured

Log-based Change Data

Capture and Delivery

CEP Integration,

Real-time Events

Heterogeneous Sources

and Targets

Reliability and

Transaction Integrity

Bidirectional Replication


Who’s Using GoldenGate?

• Leader in real-time data integration solutions

• Over 3,000 customers including Fortune 500

companies, representing Financial Services,

Communications, Healthcare, Public Sector,

Retail, Utilities industries

• 4 of top 5 largest commercial banks

• 3 of top 3 busiest ATM networks

• 5 of top 5 financial data services

companies

• 5 of top 5 telecommunications providers

• 3 of top 5 largest food & drug stores

http://brandsoftheworld.com/brands/0002/6850/Liberty_Mutual.eps

http://www.google.com/imgres?imgurl=http://www.credit-card-logos.com/images/visa_credit-card-logos/visa_logo_6.gif&imgrefurl=http://www.credit-card-logos.com/&h=113&w=180&sz=2&tbnid=6RgFXMad8jPZ6M:&tbnh=63&tbnw=101&prev=/search?q=Visa+logo&tbm=isch&tbo=u&zoom=1&q=Visa+logo&usg=__gUcOb1jLcLWi-BDQpHJ5scq4BUo=&sa=X&ei=DfVqTq_PDonXiAL6k9DBDg&ved=0CBUQ9QEwAQ

http://www.google.com/imgres?imgurl=http://www.yourlogoresources.com/wp-content/uploads/2011/08/walmart-logo.png&imgrefurl=http://www.yourlogoresources.com/walmart-logo/&h=787&w=2846&sz=70&tbnid=38GNj6G-fd-PLM:&tbnh=41&tbnw=150&prev=/search?q=Walmart+logo&tbm=isch&tbo=u&zoom=1&q=Walmart+logo&usg=__XIZZd_QfnsMe8X81zdMNviqsJGo=&sa=X&ei=pfdqTtv_CMniiALb-Ki-Dg&ved=0CBcQ9QEwAg

http://www.sbi.co.in/index.htm?lang=0&id=0


That’s the Magic of Macy’s Shopping at the worlds largest retails and your credit card transactions are moved in real-time to data

centers around the world, before being approved, using Oracle GoldenGate (Retail Decision)

Good TV. Better TV. DirecTV. If you call DirecTV customer service, your call data will move to their data warehouse via Oracle

GoldenGate, so the rep can provide you with personalized recommendations and promotions.

Say it with Flowers Ordering from 1-800-Flowers, Oracle GoldenGate will move your order to their reporting system

to help process and track your order seamlessly

Get Sick in Pittsburgh If you get a health check at UPMC, the clinical applications that doctors use are supported by

Oracle GoldenGate to enable high-availability

No Charge ATM Fees 3 of the 3 busiest ATM Networks use Oracle GoldenGate for Continuous Availability.

Oracle GoldenGate Under the Covers


GoldenGate for Banking / Financial Services

Bank of America High availability for ATM network Eliminate downtime during database,

hardware & Siebel migrations

High availability for app. that assesses

online access risk

State Street Bank Offload reporting from hosted

applications

High availability for private Exa cloud Integrate relational data with

enterprise service bus

JP Morgan Chase Eliminate planned downtime for

retail banking app.

Eliminate planned downtime for

treasury app.

Transform production data and

synchronize with reporting database

UBS Standardize to reduce complexity &

cost

Distribute data among internal

applications

High availability for planned &

unplanned downtime

Fidelity Eliminate planned downtime for

Siebel app.

High availability for tax accounting

app.

High availability for trade confirmation

system

SunTrust Fraud detection for online banking

app.

Integrate relational data with

enterprise service bus

Reduce or eliminate batch extracts

Capital One High availability for enterprise

imaging app.

Eliminate downtime during database

upgrades

Off-load reporting for car loan

origination app.

Chicago Mercantile Exchange, Deutsche Bank, US Bank, Swedebank, Visa, Prosa, FNBO, First Data, Retail Decisions, Zagreb

Bank, ADP, American Express, VISA, Desjardins Bank, Hyundai Securities, Bombay Stock Exchange, Nomura, RBS, Bankart


Databases O/S and Platforms

Oracle GoldenGate Capture: Oracle

DB2 (all platforms)

Microsoft SQL Server

Sybase ASE

MySQL

Teradata

Enscribe

SQL/MP

SQL/MX

JMS message queues

Oracle GoldenGate Delivery: All listed above, plus:

PostgreSQL

TimesTen, Coherence

Flat Files for Netezza, Greenplum, ETL products

Linux

Sun Solaris

Windows 2000, 2003, XP

HP NonStop

HP-UX

HP OpenVMS

IBM AIX

IBM z Series

IBM i Series

zLinux

Oracle GoldenGate Across the Enterprise


Oracle GoldenGate Use Cases Low-Impact Real-Time Data Integration & Transactional Replication

New DB/HW/OS/APP

Fully Active Distributed DB

Reporting Database

Data Warehouse

Global Data Centers

ODS Data Integrator

Zero Downtime Upgrade & Migration

Query Offloading, Disaster Recovery

Data Synchronization across the Enterprise

Real-time BI, Operational

Reporting, MDM

Event Driven Architecture,

SOA

Highly Available / Disaster Recovery

Log-based, changed data

Database

Message Bus

Mainframe

Message Bus


Sample GoldenGate Mainframe Customers

Chicago Mercantile Exchange (DB2/zOS to/from

Oracle)

– Migration off mainframe

AARP (DB2/zOS to Oracle and JMS)

– Integrate mainframe data with downstream

applications and SOA/EDA

Liberty Mutual (DB2/zOS & DB2/LUW to Teradata)

– Real-time feed to data warehouse for “single view

of the business”

Macy’s (DB2/zOS to Exadata)

– Real-time integration of inventory for faster

shipping & replenishment

Discover (DB2/zOS to Oracle)

– Mainframe data integration with downstream

applications

ACS-Xerox (DB2/zOS to Oracle)

– Report off-loading to reduce cost of reports

AutoZone (DB2/zOS to DB2/LUW DPF &

GreenPlum)

– Report off-loading to save mainframe cycles and

speed reporting

Citigroup (DB2/zOS to Oracle)

– PeopleSoft application migration off mainframe

Zagreb Bank (DB2/zOS to Exadata via ODI &

DB2/zOS to DB2/zOS)

– Data center consolidation, Data warehouse feed

Sears (DB2/zOS to GreenPlum)

– Report off-loading

Intercontinental Hotels Group (DB2/zOS to Oracle)

– Mainframe data migration and sharing

CIBC (DB2/zOS to HP NSK)

– Data integration with mainframe and HP Non-stop


Across the Enterprise Market Transparency with Real-Time Data Access

Live Reporting & Real-Time Data Integration

“GoldenGate has provided the data integration solution

we were looking for. We look forward to rolling this

product out for other projects across the organization.” -John Bubb, Product Manager, Business Intelligence

Products The New York Independent System Operator

Business Challenges:

High rate of analysis required to ensure the market is functioning

properly

The ‘freshest’ data is required to track pricing, bidding and

purchasing so the correct ‘buy’ and ‘sell’ price is displayed

Trades must be monitored in ‘real-time’ to ensure market

visibility and integrity

Performing analysis against the production system negatively

impacts system performance

Alternatively, relying on nightly ‘batch’ loads caused delays in

uncovering potential issues

GoldenGate Solutions:

Live Reporting from Comprehensive Bid Management System to

an ODS

Real-time Data Integration to monitor market performance

Zero-downtime database upgrade from Oracle 9i to Oracle 11g

(not depicted)

Results:

Real-time transaction management, issue resolution and

automated analysis

Better marketplace performance and lower cost of ownership

Real Time Pricing,

Load, Demand (OISR) Market Data (MIS)

ODS for Reporting &

Analytics

Market Monitoring

Performance

Database DW

Historical Data


Across the Enterprise Site License – Multiple Implementations of GoldenGate

Pervasive Use Today

Initially had handful of uses in 2008. Has grown to over 50 implementations since then. Combination of reducing barriers to use technologies and spread of GoldenGate knowledge.

CHALLENGES / OPPORTUNITIES

• Globally distributed bank with needs to integrate data and ensure business resiliency

• Multiple types of databases requiring common data sharing in all facets of bank

• Experienced online verification outage that increased risk to bank and impacted customer satisfaction

Some of the GoldenGate Implementations

• Supports sign-on authentication optimizing performance, risk avoidance, and supporting “always there” policy

• Multi-master global wire transfer application

• Zero downtime DB migration to new version using phased approach (e.g. Exadata consolidation project)

• Consolidation of wholesale credit from 50+ disparate data sources

• Always available, zero data loss reference architecture

• ATM application fault tolerance

• Real-time feed to data warehouse and business intelligence systems

• Payments application integration with messaging technology for mass broadcast


Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-Oracle

Database(s) Bi-directional

Oracle GoldenGate Runtime Architecture

Capture: committed transactions are captured (and can be filtered) as they occur

by reading the transaction logs.

Trail: stages and queues data for routing.

Pump: distributes data for routing to targets.

Route: data is compressed, encrypted for routing to targets.

Delivery: applies data with transaction

integrity, transforming the data as required.

Manager Manager

Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16 16 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.

Oracle GoldenGate Adapters

Target System

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE

TRAIL

FILE

Source System

LAN / WAN / INTERNET

OVER TCP/IP

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE

TRAIL

FILE

OGG for JAVA JMS CAPTURE: captures data from JMS text messages and creates an OGG trail for normal routing and delivery

OGG for JAVA DELIVERY: delivers change data from any OGG Capture to Java APIs or directly to JMS as formatted messages

JMS CAPTURE

Target SystemSource System

LAN / WAN / INTERNET

OVER TCP/IP

ANY OGG CAPTURE: captures data from any source and creates an OGG trail for normal routing and delivery

OGG for FLAT FILE DELIVERY: generates data and control files from trails for batch consumption by third party systems

CAPTURE

DATAFILES

CONTROLFILE

FILE DELIVERY

JMS DELIVERY

JAVA DELIVERY

Oracle GoldenGatefor Flat File

Oracle GoldenGatefor Java

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE

TRAIL

FILE

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE - UDF

TRAIL

FILE

TRAIL

FILE


Oracle GoldenGate Flexible Deployment Models Unidirectional Query Offloading

Zero-Downtime Migration

Bi-Directional Hot Standby or

Active-Active for HA

Peer-to-Peer Load Balancing

Multi-Master

Broadcast Data Distribution

Integration/Consolidation Data Warehouse

BPM

BAM

CEP

Data Distribution via Messaging


GoldenGate’s flexibility allows for multiple integration schemes

The Capture process is always the same

Delivery can be through

– Staging tables

– Intermediate flat files

– Messaging

Capture

ETL

Tool

Trail

Delivery

Data

Pump

+ UE

Data

Pump

+ UE

JMS / MoMQueue or

Topic

ETLTool

data

filesdata

filesdata

files

ETL

Tool

Integration via Staging Tables

Integration via Files

Integration via Messaging

Oracle GoldenGate – Deliver Many Integration Example


Centralized management of

GoldenGate modules

Advanced monitoring, alerting

(SNMP support), lag graphs,

and historical repository.

Browser based access to error

logs and process reports

Enterprise Manager Plug-in

Management Pack for Oracle GoldenGate

Oracle GoldenGate Monitor & OEM

Administration & Monitoring


identifies and reports data discrepancies between heterogeneous databases

Reduce financial/legal risk exposure

Speed and simplify IT work in comparing data sources

No disruption to business systems

Improved failover/migration

Confident decision-making and reporting

Oracle GoldenGate Veridata

Oracle GoldenGate Veridata

Data Comparison Without Downtime


Streams Statement of Direction – Oracle Database Upgrade Guide, Version 12c

http://docs.oracle.com/cd/E16655_01/server.121/e17642/deprecated.htm#BABEAJJE

Oracle Streams is deprecated in Oracle Database 12c and may be

desupported and unavailable in a later Oracle Database release.

– Oracle’s stated direction is to use Oracle GoldenGate for replication

implementations in place of Oracle Streams.

– Oracle Database Advanced Queuing is independent of Oracle Streams

and continues to be enhanced.

– Oracle Database Advanced Replication is deprecated in Oracle

Database 12c. Use Oracle GoldenGate to replace all features of

Advanced Replication, including multi-master replication, updatable

materialized views, hierarchical materialized views, and deployment

templates.

21

http://docs.oracle.com/cd/E16655_01/server.121/e17642/deprecated.htm


Join the Data Integration Community

Twitter or Facebook twitter.com/ORCLGoldenGate facebook.com/OracleDataIntegration

Oracle GoldenGate YouTube channel youtube.com/oraclegoldengate

Oracle.com/goto/dataintegration

LinkedIn “Oracle Data Integration”

Oracle’s Data Integration blog blogs.oracle.com/dataintegration

Oracle Technology

Network

Oracle’s Public Help Forum http://forums.oracle.com/forums/forum.jspa?forumID=860&start=0

Oracle Technology

Network

OTN: GoldenGate http://www.oracle.com/technetwork/middleware/goldengate/overview/index.html

Oracle Learning Library http://apex.oracle.com/pls/apex/f?p=44785:1:0::NO

MOS Knowledgebase http://support.oracle.com/

oracle data integration - · pdf filehardware & siebel migrations ... high availability...

Documents