deep dive with oracle goldengate 11g

8/11/2019 Deep Dive With Oracle GoldenGate 11g

1/70

1 Copyright 2011, Oracle and/or its affiliates. All rightsreserved.

Deep Dive with Oracle GoldenGate 11 gKamran Hussain ([email protected])

Data Integration Solution Specialist


2/70


The following is intended to outline our general productdirection. It is intended for information purposes only, andmay not be incorporated into any contract. It is not acommitment to deliver any material, code, or functionalitand should not be relied upon in making purchasingdecisions. The development, release, and timing of any

features or functionality described for Oracle

s proremains at the sole discretion of Oracle.


3/70


Program Agenda

Introduction to Oracle GoldenGate

Technology Overview Continuous Availability Architecture Deep Dive: Differentiators

Component Architecture Performance & Scalability Transaction Integrity & Reliability Heterogeneity

Summary Q&A


4/70

Copyright 2012, Oracle and/or its affiliates. All rights reserved.4

Oracle Data IntegrationComplete Offering for Enterprise Data Integration

Modernization MDM Big Data SynchronizatiCustom BI

Comple

approacintegratiMaximulower TreliabiliCertifiedtechnolo

time to v

Oracle Data Integrator

Oracle GoldenGate

Oracle Enterprise Data Quality

Active Data Guard

OLTP

Applications

Legacy

Unstructured Oracle Custom 80% lower T Five times h 70% reducti


5/70Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 135

Whos Using Oracle GoldenGate?

Leader in real-time data integration solution

Over 2100 customers as of 2012 includincompanies, representing Financial Services,Communications, Healthcare, Public Sector,industries

4 of top 5 largest commercial banks 3 of top 3 busiest ATM networks 5 of top 5 financial data services com

5 of top 5 telecommunications provid 3 of top 5 largest food & drug stores
http://www.sbi.co.in/index.htm?lang=0&id=0http://www.google.com/imgres?imgurl=http://www.yourlogoresources.com/wp-content/uploads/2011/08/walmart-logo.png&imgrefurl=http://www.yourlogoresources.com/walmart-logo/&h=787&w=2846&sz=70&tbnid=38GNj6G-fd-PLM:&tbnh=41&tbnw=150&prev=/search?q=Walmart+logo&tbm=isch&tbo=u&zoom=1&q=Walmart+logo&usg=__XIZZd_QfnsMe8X81zdMNviqsJGo=&sa=X&ei=pfdqTtv_CMniiALb-Ki-Dg&ved=0CBcQ9QEwAghttp://www.google.com/imgres?imgurl=http://www.credit-card-logos.com/images/visa_credit-card-logos/visa_logo_6.gif&imgrefurl=http://www.credit-card-logos.com/&h=113&w=180&sz=2&tbnid=6RgFXMad8jPZ6M:&tbnh=63&tbnw=101&prev=/search?q=Visa+logo&tbm=isch&tbo=u&zoom=1&q=Visa+logo&usg=__gUcOb1jLcLWi-BDQpHJ5scq4BUo=&sa=X&ei=DfVqTq_PDonXiAL6k9DBDg&ved=0CBUQ9QEwAQhttp://brandsoftheworld.com/brands/0002/6850/Liberty_Mutual.eps


6/70Copyright 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 136

Data Integrator

New DB/HW/OS/APP

Fully Active Distributed DB

Reporting Database

Data Warehouse

Private Cloud

ODS

Ze

Qu

Re

Ev

Hi

Message Bus

Log-based,Changed Data

Oracle & Non-OracleDatabase(s)

Message Bus

Legacy Systems

How Are They Using GoldenGate?Low-Impact, Real-Time Data Integration & Transactional Replicati


7/70


Databases O/S and PlatformsOracle GoldenGate Capture:

Oracle DB2 W/L/U DB2 - z/OS & iSeries (AS/400) Microsoft SQL Server Sybase ASE Teradata Enscribe SQL/MP SQL/MX

MySQL JMS message queues

Oracle GoldenGate Delivery: All listed above, plus: TimesTen, IBM System I, Netezza & Greenplum ETL product

Linux

Sun SolarisWindows 2000, 2003, XP, 2008

HP NonStop

HP-UX

IBM AIX

IBM z Series

zLinuxIBM i Series

Oracle GoldenGate 11g Supported Platforms


8/70


Performance

Flexible & Extendable

Reliable

Sub-second latency for data movement

Low-impact capture, routing, transformatidelivery of transactional data

Open, modular architecture Supports heterogeneous sources and target Agile and adaptable to modern data integr

architectures Maintains transactional integrity Resilient against interruptions and failures Fast Recovery after process interruptions.

Oracle GoldenGate Differentiators


9/70

Technology Overview


10/70


How Oracle GoldenGate Works

SourceOracle & Non-OracleDatabase(s)

Capture: committed transactions are captured (and can be filtered) as they occur byreading the transaction logs.


11/70





Trail: stages and queues data for routing.


12/70


13/70






Pump: distributes data for routing to target(s).

Route: data is compressed, encrypted for routing to


14/70








Delivery: applies data with traintegrity, transforming the data


15/70






Delivery: applies data with traintegrity, transforming the data

SourceOracle & Non-OracleDatabase(s) Bi-directional



16/70


Oracle GoldenGate Parameter File Example


17/70


Oracle GoldenGate Parameter File Example


18/70


Unidirectional Query OffloadingZero-Downtime Migration

Bi-Directional Hot Standby or

Active-Active for HA

Peer-to-PLoad BalanMulti-Mast

Broadcast Data Distribution

Integration/Consolidation Data Warehouse

Data Disvia Messa

Oracle GoldenGate Differentiator: Flexibility


19/70

2010 Oracle Corporation Proprietary and Confidential

GoldenGates flexibility allows for multiple integration schemes The Capture process can be the same

Delivery can be through Staging tables Intermediate flat files Messaging

Capture

Trail

Delivery

DataPump+ UE

DataPump+ UE

JMS / MoMQueue or

Topic

ETLTool

datafilesdata

filesdatafiles

ETLTool

Integration via Staging Tables

Integration via Files

Integration via Messaging

Non-Database Integration


20/70


Oracle GoldenGate for Java

Through Oracle GoldenGate Application Adapter for Java,transactional data captured by Oracle GoldenGate can be delivered to

targets such as BPM / CEP & ETL through JMS or integrating with acustom applications Java API

l ld d


21/70


A high-speed, low impact datacomparison solution

Identifies and reports data discrepanciesbetween heterogeneous databaseswithout interrupting their availability

Benefits: Reduce financial/legal risk exposure Speed and simplify IT work in comparing

data sources No disruption to business systems Improved failover to backup systems Confident decision-making and reporting

Oracle GoldenGate Veridata Data Comparisonwithout Downtime


22/70


Advanced monitoring, alerting (SNMPsupport), solution discovery, laggraphs, and historical repository.

Based on new integrated core agentinfrastructure. Supports GoldenGate

11.1.1.1.1+

Included in Management Pack

Management Pack for Oracle GoldenGate

Oracle GoldenGate Monitor

Monitoring


23/70


Oracle Preferred monitoring tool, coversentire Oracle stack: Database, Storage,FMW etc.

For customers who have standardizedon EM, or prefer to use the EMsextensive capabilities: DB & OS

resource monitoring, incidents, RemedyPlug-in, etc.

Included in Management Pack

Management Pack for Oracle GoldenGate

Enterprise Manager Plug-in

Monitoring

O l G ld G t f C ti A il bilit


24/70


Zero Downtime Upgrades, Migrations and Maintenance

Migration from non-Oracle databases to Oracle 11gR2 & Exadata

Upgrade Oracle Database 8 i , 9 i , 10g to 11gR2 Upgrade/Migrate Database Server/OS

Perform Database Maintenance

Application Upgrade (Siebel CRM and JD Edwards)

Disaster Recovery for Non-Oracle Databases and Cross- Oracle Databaseversions

Continuous Availability via Active-Active Databases

Offloading Queries from Legacy Systems to Oracle Databases for MaximPerformance

Oracle GoldenGate for Continuous Availability

A Certified App Upgrade Solution for


25/70


A Certified App Upgrade Solution forJD Edwards and Siebel CRM

Upgrade/migrate/maintain database,hardware, OS and/orapplication

Minimize risk withfailback option

Improve success withphased user migration

Real-time changed data

Failback Data Flow

Non-OracleDatabase/Server

Siebel / JDELower Version

Eliminate Downtime During Upgrades to Oracle 11g R


26/70


Eliminate Downtime During Upgrades to Oracle 11g R

Zero databasedowntime forupgrades from 8i, 9i,10g to 11g

Leverage newfeatures of OracleDatabase 11g withoutimpacting businessoperations

Minimize risks withfailback option

Failback Data FlowOracleDatabase8i/9i/10g

ApplicationSwitchover

Compare & VerifyWith Oracle GoldenGate

Veridata

Eliminate unplanned downtime with


27/70


Eliminate unplanned downtime withActive Data Guard & GoldenGate

Active Data Guard fordisaster recovery anddata protection for:

Oracle-to-Oracle Oracle GoldenGate for

disaster recovery anddata protection for:

Non-Oracle platforms Active-Active Cross-OS and Oracle

database versionrequirements

Post-Switchover Data Flow

Application

Production

Switchover

Compare & VerifyWith Oracle GoldenGate

Veridata


28/70


29/70

Eli i t All D ti f C ti A il bilit


30/70


Eliminate All Downtime for Continuous Availability

Available no issues

Planned outage Upgrade Migration Maintenance

Unplanned outage System failure Data failure

Semi-a

Fi S M i i A il bili


31/70


Storage failure Data recovery Backups

Instance failure Server failure RAC rolling

maintenance Performance

scale-out Consolidation

Continuous point-in-time recovery

Granular repairof logicalcorruptions

Transaction Table Database

Database failure System failure Site failure Zero data loss Automatic database

failover Lost-write protection Database rolling

upgrade Offload read-only

workload andbackups

Some migrations

Onlineapplica

Bi-diremulti-m Distrib

read-w An alte

physicfor site

Flexib

and hemigrat Zero d

upgradmigrat

Five Steps to Maximize Availability

ASM,

RMAN,OSB

Oracle RAC

Flashback

ActiveData Guard

Oracle


32/70

Architecture Deep Dive

A hit t C t (P )


33/70


Architecture Components (Processes) Log-based VAM-based Local/Remote Queuing Filtering Parallel Coordination Local/Remote*

Native database API ODBC Batched Operations Filtering Transaction/Operations Grouping Parallel Coordination Local/Remote*

CAPTURE DELIVERY

PUMP MANAGER Filtering Local/Remote TCP/IP Compression Encryption

Queue (Trail) Management Process Management (Dynamic) Monitoring and Administration (Lag reports) Reporting (Events, Errors, Thresholds) Receive/Route requests from UI

On-disk Components Trail Files Checkpoint Files (Capture, Pump, Delivery) Data Definition Files (SOURCEDEFS/TARGETDEFS) Configuration Files Discard Files Report/Log Files Executables

Oracle GoldenGate Components


34/70


Oracle GoldenGate ComponentsCommunications and Startup

TargetSourceTCP/IP

SourceDatabase

Process StartShared Memory (for Monitoring)

GGSCIInteractiveInterface

Manager Manager

LAN / WAN / INTERNETOver TCP/IP



35/70



TargetSourceTCP/IP

SourceDatabase



Manager Manager




36/70



TargetSourceTCP/IP

SourceDatabase



Manager Manager



37/70



38/70



TargetSourceTCP/IP

SourceDatabase



Manager Manager



39/70

Differentiator: Reliability


40/70


Differentiator: Reliability

Decoupled architecture Individual processes can be restarted automatically

Tolerance to network outages (configurable)

Recovery Recovery ensures that no operations are skipped or duplicated after

failure of any kind Recovery of the Capture process is more involved than recovery the

Delivery process Although GoldenGate processes are completely decoupled, the

Delivery process must be aware of a recovery performed by theCapture process or pump

GoldenGate Checkpointing


41/70


GoldenGate Checkpointing Captu

and D

positcheckthey in ca

SourceDatabase



42/70



and D


SourceDatabase



43/70



and D


TargetDatabase

SourceDatabase

GoldenGate Lag Measurement


44/70


Source DB

extract lag

extract

pump lag

replicat lag

WANreplipump

Source Trail Target Trail

end -to- end latency S o u r c e

C o m m

i t T i m e s

t a m p

S y s t e m

T i m e

W r i t e

t o T r a

i l

S y s

t e m

T i m e

W r i t e

t o T r a

i l


45/70


Q&A

Advantages of Oracle GoldenGate Architecture


46/70


g

Captures once, delivers to many targets for different uses Non-invasive, log-based capture Moves only committed data, reduces bandwidth needs

Reduced Overhead and TCO

Subsecond latency even with high data volumes Preserves transaction integrity Ensures data recoverability

High Performance with Reliability

Provides decoupled, modular architecture Supports heterogeneous sources and targets, and different latency needs Coexists and integrates with ELT/ETL and messaging solutions

Flexibility and Ease of Use


47/70

GG + ODI

Complementary and Used Together


48/70


Fastest real-time solution Sub-second latency for real-time feeds Guaranteed delivery eliminates data loss Eliminates down-time for migration and upgrades Least intrusive to source systems

Oracle GoldenGate

Fastest E-LT Solution Optimized SET-based transformation for high volume transformations Data lineage for improved manageability Integrates to Data Quality

Oracle Data Integrator

p y gOracle GoldenGate and Oracle Data Integrator

Next Generation Data Warehousing


49/70

EMP

OracleGoldenGate

Oracle DataIntegrator

EMP DEPT

DIM

FACT

DIM

ODS SchemaCanonicalSchema

On-Disk Logs

Source 1

On-Disk Logs

Source 2

OracleGoldenGate

EMP DEPT

DEPT

ODI Knowledge Modules Integrate ODI & GoldenGate

Generates GoldenGate configuration from ODI UI

Delivers out of box pattern for Real-Time Data Integration

g

Faster decis ion m aking w ith f resher data in h is to r ical co ntext


50/70



51/70

GG 11gR2 New Features

Integrated Capture


52/70


OGold

1

Integrated Capture

Overview

A unique, industry-leading addition to the productthat moves GoldenGates capture capabilities

closer to the Oracle Database engine.

Integrated Capture


53/70


OGold

1

Integrated Capture

Overview

Support for Oracle Compression

With this support, GoldenGate provides 100%support for capture and delivery of compressed

objects within Oracle Database and OracleExadata

Enhanced Conflict Detection & Resolution


54/70


OGold

1


Overview

Enhancements provide automated detectionand resolution of data conflicts for active-active

or multi-master deployments.

Robust, easier to code, and less time toimplement.



55/70

74 Copyright 2012, Oracle and/or its affiliates. All rights

reserved.

Syntax for 11gR1 CDR

MAP source.Order, TARGET target.Order,REPERROR (21000, DISCARD),

SQLEXEC (ID lookup, ON UPDATE,QUERY "select count(*) conflict from Order where ID =

? and & Modified_TS > ?",PARAMS (p1 = ID, p2 = Modified_TS), BEFOREFILTER,

ERROR REPORT, TRACE ALL),

FILTER (lookup.conflict = 0, ON UPDATE, RAISEERROR21000);

7 Lines of Code & SQL

Syntax for 11gR2 CDR

MAP source.Order, TARGET target.O

RESOLVECONFLICT (UPDATERO(DEFAULT, USEMAX (Modif

3 Lines of Code, Automatica

Example

Security


56/70


reserved.

OGold

1

Security

Overview

The Most Secure Version of GoldenGate

Oracle GoldenGate 11gR2 supports FederalInformation Protection Standard (FIPS) and BLOWFISH

encryption algorithms to provide secure datamovement across systems and regions

Performance


57/70


reserved.

OGold

1

Performance

Overview

Performance improvements for data replicationacross geographically separated systems in high

latency networks.

Extensibility


58/70


reserved.

OGold

1

Extensibility

Overview

Using Event Markers allows GoldenGateimplementations to be extended to kick-off

actions based on an incident (DML,DDL, datavalue)

Manageability & Monitoring


59/70


reserved.

OGold

1

Monitoring

Overview

The new Oracle GoldenGate Monitor product offers arich graphical user interface (GUI) client that interactswith core Oracle GoldenGate components to provide a

complete view of the GoldenGate implementations

Also now available: Enterprise Manager 12c Integration

Monitoring


60/70


reserved.

Oracle GoldenGate Monitor Lightweight tool, caters to GoldenGate customers For customers, who prefer not to install additional Oracle

database as management server. Simple SNMP integration

Oracle Enterprise Manager Oracle preferred monitoring tool, covers entire Oracle stack

Database, Storage, FMW etc For customers, who prefer to drill into DB sessions, OS stats,

etc, apart from monitoring GoldenGate process

Overview

Expanded Heterogeneity


61/70


reserved.

OGold

1

Expanded Heterogeneity

Overview

The new release brings enhancements to existingsupported platforms such as: Oracle MySQL, MicrosoftSQL Server, Sybase, Teradata, IBM DB2/iSeries and IBM

DB2 z/OS including expanded data type support

Stay tuned for announcements for Coherence &PostgreSQL support


62/70


63/70

Zero Down-time Upgrade/Migration

GoldenGate Usage Zero Downtime MigrationOLTP UsersOLTP d

S G ld G

I i d b i h

O i i i i l

N fi h

O i i l

S i h OLTP


64/70

- Capture / Extract - Delivery / Replicat - Trail

Oracle

8i, 9i, 10g

Initial Instantiation

OLTP users are connected toprimary server. The new

database is empty.

Start GoldenGate captureprocess & store transactions intrail. Eliminates need for point-

of-consistency.

Instantiate new database withprimary data while OLTP users

are active. Can use anymethod preferred.

Once instantiation is complete,deliver the real-time

transactions to new databasehandling collisions.

Now testers can confirm thatthe application will work

correctly with live data and notaffect production.

Once testing is complete, startreplication back to primary

from new database. Enables afallback option.

Switch OLTP users to newdatabase. If problems occur,

can immediate fallback toold database.

Start change data capture on Source Database

GoldenGate Instantiation


65/70

Start change data capture on Source Database Current source Commit Sequence Number (CSN) is 222

Capture PumpCommit Ordered

Source TrailCommit Ordered

Target TrailSourceDatabase

Current CSN is222

CSNs 222through

Wait for any open transactions to close

GoldenGate Instantiation


66/70

Wait for any open transactions to close Use a backup/export/TTS to copy source to target as of a consistency point

Current source Commit Sequence Number (CSN) is 245

Capture PumpCommit Ordered

Source TrailCommit Ordered

Target TrailSourceDatabase

Take Backup asof CSN 245

Apply Backupas of CSN 245

CSNs 222through

Current CSN is245


67/70

Case Study: Research in Motion Blackberry DivisionZero Downtime OS and DB Migrations and Application Maintenance

d


68/70

Business Challenges : RIM BlackBerry has a very small monthly

window to take downtime for upgrades,migrations, and planned maintenance

Strict SLA limits with the major Telcos!

GoldenGate Solution : Zero-Downtime Upgrade and OS change for

business-critical Oracle databases GoldenGate Veridata to ensure that all data

from source is in-synch with target and thatmigration has been successful

GoldenGate Benefits : Fail-back capability to reduce business risk and

downtime No manual intervention to reduce possibility oferrors Assurance of database consistency with

Veridata.

Zero-Downtime OS and DUpgrade with Application Ma

Oracle 10.2.0.3HP-UX

Waterloo

Application Mainten

ProvisioningDatabase migrati

with failback

Business Challenges :

Case Study: First Data CorporationZero Downtime Migration for Merchant Services


69/70

Business Challenges :Minimize total downtime to business during upgradeof their Merchant Services Settlement system from:

Oracle 8i/HP-UX to 10g/IBM AIXNo tolerance for downtime - operates 24x7x365

GoldenGate Solution:Zero-Downtime Migration solution works withdifferent application, database, and hardwareversions and vendors, across dispersed locationsPhased migration approach spread over 3 months(work on production system during weekends only)

Benefits:Reduced downtime during the massive migrationeffort (both environments remained online for 1month before cutting over)Simplified data conversions without interruption toregular business processes

Zero-Downtime Oracle 8Database Migratio

IIP Infrastructure

81,000 ConcurrentMerchants

Merchant Services InfrastruWeb; Automated & Call

Oracle 8iHP/UX

Case Study: Home Shopping Network (HSN)Zero Downtime for Major Oracle Siebel & Database UpgradeB i Ch ll g


70/70

Business Challenges : Needed to minimize total downtime to business

during upgrade of their order management systemSiebel 6.2 to Siebel 8.0.

Additionally in project: upgrades from Oracle 8i toOracle 10g and hardware.

No tolerance for downtime - operate 24x7x365

GoldenGate Solution: Zero-Downtime Operations solution works with

different application, database, and hardwareversions and vendors.

Phased upgrade approach spread over 3 months.(50-100 agents per day)

Reduce downtime during the massive migrationeffort.

Simplify data conversions without interruption toregular business processes.

Both environments remained online for 3 monthsbefore cutting over in September 2008 to the newenvironment.

Zero-Downtime ApplicatioDatabase Upgrade

Siebel 6.2Oracle 8i

Sun Solaris 8

1,500 users,phased switchove

Single Order Management SWeb; Automated & Call Cen

" Being a l ive 24/7 bus iness, HSN sim ply afford to t ake any o utage, so wh en o ther vsugges ted we take the sys tems dow n for ato com plete the migrat ion , we knew w e hfind an alternate solutionwe selected GoldenGate

- Anitha Reddy, Vice President of IT A

deep dive with oracle goldengate 11g

Documents