doag data replication with oracle golden gate: looking behind the scenes

Data Replication With Oracle GoldenGate – Looking Behind The Scenes –

Robert BialekPrincipal ConsultantPartner

Who Am I

Principal Consultant, Partner and Trainer at Trivadis GmbH in Munich

– [email protected]

Focus:

– Oracle Database High Availability

– Database Architecture/Internals

– Backup/Recovery

– Troubleshooting/Performance Tuning

– Linux Administration

Trainer for the following Trivadis courses

– Oracle Grid Infrastructure (O-GRINF)

– Oracle Real Application Cluster (O-RAC)

– Oracle Data Guard (O-DG)

2 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Agenda

3

1. Introduction

2. Oracle GoldenGate Architecture

3. Oracle GoldenGate Topologies

4. Demo – Integrated Extract/Replicat

5. Oracle GoldenGate Licensing

6. Core Messages

21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Introduction


What is Oracle GoldenGate?

Oracle strategic Middleware software product used for data extraction and replication

– Use-cases: data/service high-availabillity, “zero-downtime” upgrades/migrations, easy fallback (minimize risk)

Moves only committed transactions across TCP/IP network without Oracle Net Services

– In contrast to Oracle Streams…

Supports

– asynchronous data transfer (DML, DDL)

– heterogeneous sources and targets (databases, operating systems)

– different extraction, propagation and apply methods

– different topologies, mappings, transformations, filtering, error handling, …


Why Oracle GoldenGate?

Oracle Streams is deprecated in the Oracle 12c version (but still supported!)

– https://docs.oracle.com/database/121/UPGRD/deprecated.htm

According to the Oracle GoldenGate Statement of Direction

– http://www.oracle.com/technetwork/middleware/goldengate/overview/statement-of-direction-gg-132120.pdf

Beginning with the version 12.1.0.2, the non-CDB architecture is also deprecated

– As of now, Streams is not supported with the new CDB architecture

Oracle Streams continues to be supported but will not be enhanced to support any new database features in future releases. This means that Streams will continue to support exactly what it supports today (11.2) in future releases. It also means that any new database features, data types, or capabilities will not be supported by Streams.


Oracle GoldenGate Architecture


Oracle GoldenGate Architecture: Big Picture


Source DB Target DB

Primary (Regular)Extract

Primary (Passive)Extract

Secondary Extract

Data Pump

Collector

Replicat

Local Trail Remote Trail

Primary (Alias)Extract

Optio

nal

Oracle GoldenGate Instance Oracle GoldenGate Instance

Static Configuration

RuntimeConfigurati

on

RuntimeConfigurati

onStatic

Configuration

OptionalOptional

Manager (VIP)

Manager

(VIP)MONITORMONITOR

Primary Extract Group: Capture

Primary Extract can run in either

– Classic Capture mode (all database versions)

– Integrated Capture mode (full support: 11.2.0.3+)

Extracts and optionally filters/transforms the transactions from the source database

– Only for configured objects

Many Extracts per source database can operate at the same time

Only committed transactions (CSN) will be propagated to the target system

– Optionally written to a local trail (recommended)

– Without a local trail, the primary Extract send the data to the target system

– Not committed transactions will be discarded from memory or dirtmp directory

9


Primary (Passive)Extract

21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Source DB

Primary Extract Group: Classic Capture

Classic Capture mode

– Traditional mode, reads directly Oracle database redo logs

– Able to mine redo logs in ASM (two methods)

– All database versions supported

With some restrictions. For example no support for:

– BASIC, OLTP, Exadata Hybrid Columnar compression

– PDML on RAC database

– CDB architecture!

DDL, TDE support – no out-of-the box support

Not transparent to RAC configuration changes (#threads)


Source DB


Secondary Extract

Data PumpLocal Trail

Redo Logs

OS System Calls

Primary Extract Group: Integrated Capture

Integrated Capture mode

– Interacts with the database logmining server

– Required to capture data from a CDB architecture

– Fully transparent with RAC, TDE, ASM

– Full support for databases 11.2.0.3+

– Source database 11.2.0.4+ does not require supplementary objects for DDL support

– Supports more complex data types than classic capture

– Integrated capture deployments

• Local or a downstream deployment

The best choice for most deployments


Source DB


Secondary Extract


Redo LogsLogmaining Server

LCR

LCR

Secondary Extract Group (Data Pump): Propagation

Optional but recommended component

– Reads the local trail and propagates the data to the remote trail (via target collector process)

– Data can be transferred “as is” (PASSTHROUGH)

– Optionally filters/transforms the transactions from the source database (NOPASSTHROUGH)

– Isolates the primary extract from TCP/IP timeouts

The network transfer can be optionally encrypted

– Not necessary, in case the trail is already encrypted


Encryption

Secondary Extract


Replicat Process: Apply

The Replicat process can run in either

– Classic Replicat mode (nonintegrated) - all database versions

– Coordinated Replicat mode (nonintegrated) - all database versions

– Integrated Replicat mode – database versions 11.2.0.4+

The Replicat process reads the remote trail, reconstructs the SQL and applies them to the target database

– Optionally filters/transforms the data

The application of the data can be optionally delayed (DEFERAPPLYINTERVAL parameter)


Target DB

Replicat

Remote Trail

Replicat Group: Classic/Coordinated Mode

Classic Replicat

– Constructs SQL statements from the trails and applies the SQL through OCI

– Supported for all database versions

– Transactions are applied sequentially (single-threaded)

Coordinated Replicat

– Transactions are applied in parallel (multi-threaded)

– There is 1 coordinator plus many apply threads

– Supported for all database versions

– GoldenGate 12c required


OCI

SQL

Target DB

Replicat

Remote Trail

OCI

SQL

Target DB

Remote TrailCoordinat

or

Thread 1

Thread 2

Thread n

Replicat Group: Integrated Mode

Integrated Replicat

– Supported for target database versions 11.2.0.4+

– Constructs LCRs and transmits DMLs to the database inbound server

Some changes (e.g. DDL) are not supported directly by the inbound server

– In those cases the Replicat process disconnects from the inbound server and executes the transaction in direct apply mode through OCI

With many transactions applied in direct apply mode it might be necessary to switch to nonintegrated Replicat


LCR

Target DBReplicat

Remote Trail

Database

Inbound Server

SQL

Capture/Apply: CDB & Pluggable Databases

Extract must operate in integrated mode

– Extract must connect to the CDB$ROOT container as a common user (c##)

– One Extract can capture from one or many PDBs to a single trail

Replicat can connect and apply only to one PDB (one Apply/PDB, connect using service)

Source DB – CDBCDB$ROOT PDB1 PDB2

Target DB – CDB

CDB$ROOT PDB1 PDB2

Primary ExtractsINTEGRATED

Data Pumps CollectorsReplicats


common_user_prefix (12.1.0.2)

Capture & Apply: Different Combinations Possible

Different combinations are possible

– Classic Extract with integrated or classic/coordinated Replicat

– Integrated Extract with integrated or classic/coordinated Replicat

You can switch between the modes with some configuration changes17 21.05.2015 Data Replication With Oracle GoldenGate - Looking Behind The Scenes

Source DBRedo Logs

Logmaining Server

LCR

LCR

LCR

Target DB

Replicat(s)

Database

Inbound

Server

OS System Calls

Primary Extract(

s)

OCI

SQL

Classic

Integrated

Integrated

Classic

Coordinate

d

Oracle GoldenGate Topologies



Active-Passive

– Data replication only in one direction

– Only one set of primary Extract, Data Pump and Replicat groups

Active/Standby

– Additional inactive primary Extract, Data Pump and Replicat groups

– Switchover: Stop one set of process Start the inactive one



Active-Active

– Data replication in both directions

– Two active sets of primary Extract, Data Pump and Replicat groups

– With some restrictions: e.g.: DDL-support restricted to two databases with identical metadata

Additionally broadcasting, cascading, consolidation is possible


Demo – Integrated Extract/Replicat


Oracle GoldenGate: Setup Flow


ArchitectureDesign

Software Installation/Patches Database Preparation

Manager Configuration

Primary Extract, Data Pump, Replicat

Configuration Files

Target Database Instantiation

Register/Start Primary Extract, Data

Pump

Register/Start Replicat

Cluster Integration / INIT Scripts

Tuning/Customizing/ Maintenance/…

1 2 3

6 5 4

7 8 9

n

Oracle GoldenGate: Demo Environment


Source DB: 2-nodes RAC 11.2.0.4Schema: APP1GoldenGate version: 12.1.2.1.2Active/Passive data replication

CDB$ROOT APP1 APP2

Primary ExtractINTEGRATED

Target DB: CDB 2-nodes RAC 12.1.0.2PDB: APP1, Schema: APP1GoldenGate version: 12.1.2.1.2Active/Passive data replication

APP3

Schema: APP1

Local Trail

Secondary ExtractData Pump

RemoteTrail

Collector

ReplicatINTEGRATED

Licensing


GoldenGate Licensing

Please contact your Oracle sales representative for additional/more detailed information!

Licensed customers of Oracle Database Enterprise Edition who purchase Oracle GoldenGate are entitled to a full use of

– Oracle Active Data Guard

– Oracle Global Data Services (12.1.0.1+)

Oracle Store (https://shop.oracle.com)

– 304.00€ / Named User Plus (min. 25 NUPs per physical CPU)

– 15,194.00€ / Processor

You need to fully license the source and the target environment

– Independent of the topology

In a cluster environment all nodes need to be licensed

– Soft-Partitioning cannot be used to limit the amount of licenses!


GoldenGate Licensing – Example


DB1 DB2

APP1

APP2

APP3

APP1

APP4

APP5

GoldenGate replication for the schema APP1

DB1

Standby database

GG replication will be activatedafter loss of the primary system only

GG license3 nodes

GG license3 nodes

Oracle 10-day failover rule

applies

Core Messages


Core Messages

GoldenGate is a very powerful replication software framework

For most deployments the integrated Extract/Replicat is the best choice

Supplemental logging generates more undo and redo

– In some cases you need even to supplementally log all columns!

Script all the configuration steps

– Very useful if you need to re-setup the whole environment

You don’t need to replace your current Streams installation ASAP by Oracle GoldenGate , but

– Streams is deprecated in the version 12c and will not be enhanced in future releases

– There is a framework to ease the migration from Streams to GoldenGate (streams2ogg)

– As of now Streams is not supported with CDB architecture

– For new projects it might be a better choice (licensing)!


Questions and answers…Robert BialekPrincipal ConsultantPartner

Tel. +49 89 [email protected]


doag data replication with oracle golden gate: looking behind the scenes

Presentations & Public Speaking