collaborate 2012 - rman eliminate the mystery

Eng. Nelson Calero, OCPUYOUG

High Availability Boot Camp: RMAN - Eliminate the Mystery

About me:

http://www.linkedin.com/in/ncalero

Working with Oracle tools and Linux environments since 1996

DBA Oracle (since 2001) & MySQL (since 2005)

Oracle University Instructor since 2011

Co-founder and President of the Oracle user Group of Uruguay (UYOUG) since 2009

Computer Engineer. OCP DBA 10g

High Availability Boot Camp: RMAN - Eliminate the Mystery

Uruguay

• Introduction• RMAN Concepts• Backups • Recovery• Integrity validation• Monitoring• Maintenance• When errors• Optimization• Examples – time permitted

AGENDA

Utility with many features to use in backup and recovery tasks. Included for free with Oracle database installation ($ORACLE_HOME/bin) Command line interface.

some functionality available inside Enterprise Manager Written in Pro * C, executes PL / SQL Since Oracle 8 – 1997 Documented bugs in all versions: less than 1283, more than 170. Known RMAN Performance Problems (MOS note 247611.1)

32 bugs, all fixed by 11g (some even before)

What is RMAN?

Best practices of backup & recovery Today?

Some. Those using RMAN

More technology is needed apart from RMAN in a DR/HA policy:• Datapump• Flashback• Secure backup• ….

RMAN is one solution among 20 from Oracle for unplanned downtime, used for site and storage failures, and data corruptionhttp://docs.oracle.com/cd/E11882_01/server.112/e17157/planned.htm#CIHDCJCE

Backups without Rman (user managed)

Consistent (cold):cp path/* dest-bkp

Inconsistent (hot):alter tablespace nnn begin backup;cp path/datafile.dbf destbkpalter tablespace nnn end backup;cp /archivelogs-path/* dest-bkp

RMAN backups

Consistent (cold): rman target / startup mount; backup database plus archivelog;

Inconsistent (hot): rman target / backup database plus archivelog;

RMAN Backup - files

• Which files were generated by a backup?– list backup;

• Which files are copied?– datafiles– controlfiles– archived logs– spfile

• Which files are not copied?– temporary files (tempfiles) and online redo logs (not needed for recovery)– files from external tabes or bfiles– database binary files– database network configuration files

RMAN backups

Differences between RMAN and user-managed backups:

– RMAN sets the rules, but is flexible

– RMAN commands implement sophisticated tasks, which lead to less code to be written to perform the same tasks, when possible.

– RMAN has many functionalities

– RMAN requires training to use effectively

RMAN Concepts

– Device– Channel

– Backupset and Copies• Backup pieces

• Status: expired/available– Retention policy

• Obsolete

– Incarnation

RMAN Configuration

RMAN> show all; using target database control file instead of recovery catalog RMAN configuration parameters for database with db_unique_name XE are: CONFIGURE RETENTION POLICY TO REDUNDANCY 2; CONFIGURE BACKUP OPTIMIZATION ON; CONFIGURE DEFAULT DEVICE TYPE TO DISK; # default CONFIGURE CONTROLFILE AUTOBACKUP OFF; CONFIGURE CONTROLFILE AUTOBACKUP FORMAT FOR DEVICE TYPE DISK TO '%F'; # default CONFIGURE DEVICE TYPE DISK PARALLELISM 1 BACKUP TYPE TO BACKUPSET; # default CONFIGURE DATAFILE BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default CONFIGURE ARCHIVELOG BACKUP COPIES FOR DEVICE TYPE DISK TO 1; # default CONFIGURE MAXSETSIZE TO UNLIMITED; # default CONFIGURE ENCRYPTION FOR DATABASE OFF; # default CONFIGURE ENCRYPTION ALGORITHM 'AES128'; # default CONFIGURE COMPRESSION ALGORITHM 'BASIC' AS OF RELEASE 'DEFAULT' OPTIMIZE FOR LOAD TRUE ; # default CONFIGURE ARCHIVELOG DELETION POLICY TO NONE; # default CONFIGURE SNAPSHOT CONTROLFILE NAME TO '/u01/app/oracle/product/11.2.0/xe/dbs/snapcf_XE.f'; # default

RMAN Configuration

Per target (in controlfile or Catalog): Device Parallelism Retention Policy Backup optimization Encryption

If using Fast Recovery Area, database parameters: DB_RECOVERY_FILE_DEST_SIZE ( DB_RECOVERY_FILE_DEST is set)

If not using a recovery Catalog, set according to your policy CONTROL_FILE_RECORD_KEEP_TIME

NOTE: Since 11.2.0.3 destination can be ASM Cluster File System (ACFS)

Back to RMAN Backups

See what has been backed up:

RMAN> connect target connected to target database: ENT11G (DBID=410442782)

RMAN> list backup summary; List of Backups =============== Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag ------- -- -- - ----------- --------------- ------- ------- ---------- --- 1 B F A DISK 13-FEB-12 1 1 NO TAG20120213T122058 2 B F A DISK 13-FEB-12 1 1 NO TAG20120213T122238 3 B F A DISK 13-FEB-12 1 1 NO TAG20120213T122421 4 B F A DISK 13-FEB-12 1 1 NO TAG20120213T122440

RMAN Backups

Details:

RMAN> list backup tag TAG20120213T122058; using target database control file instead of recovery catalog

List of Backup Sets =================== BS Key Type LV Size Device Type Elapsed Time Completion Time ------- ---- -- ---------- ----------- ------------ --------------- 1 Full 1.03M DISK 00:00:01 13-FEB-12

BP Key: 1 Status: AVAILABLE Compressed: NO Tag: TAG20120213T122058 Piece Name:

/u01/app/oracle/fast_recovery_area/ENT11G/backupset/2012_02_13/o1_mf_nnndf_TAG20120213T122058_7ml72cnz_.bkp

List of Datafiles in backup set 1 File LV Type Ckp SCN Ckp Time Name ---- -- ---- ---------- --------- ---- 5 Full 1044143 13-FEB-12 /u02/oradata/ent11g/prueba.dbf

RMAN Backups

TIP - change date format inside RMAN

[oracle@oraculo ~]$ export NLS_DATE_FORMAT='DD/MON/YYYY HH24:MI:SS' [oracle@oraculo ~]$ rman target / Recovery Manager: Release 10.2.0.3.0 - Production on Wed Dec 21 20:31:46 2011 Copyright (c) 1982, 2005, Oracle. All rights reserved. connected to target database: Ent11g (DBID=943234298)

RMAN> list backup summary tag bkp_prod_121511060003; using target database control file instead of recovery catalog

List of Backups =============== Key TY LV S Device Type Completion Time #Pieces #Copies Compressed Tag ------- -- -- - ----------- -------------------- ------- ------- ---------- --- 1 B F A DISK 13/FEB/2012 03:21:03 1 1 NO TAG20120213T122058 2 B F A DISK 13/FEB/2012 03:25:42 1 1 NO TAG20120213T122238 3 B F A DISK 13/FEB/2012 03:29:28 1 1 NO TAG20120213T122421

RMAN backups

Using Oracle XE:

$ORACLE_HOME/config/scripts/ • backup.sh • restore.sh

RMAN backups - Incremental

Differential: copy only the changes since the last incremental backup Cumulative: copy all changes from the last full backup.

Incremental backups are differential by default

To restore a database one of this is needed: all differential backups since the last full backup, or last cumulative backup following the last full backup

RMAN incremental backups

Examples

First incremental backup must be complete, and is identified with the level 0.backup incremental level 0 tablespace users;

Next is level 1. If no level 0 backup exists,it creates a level 0 backup:backup incremental level 1 tablespace users;

Cumulative backup: backup incremental level 1 cumulative tablespace users;

RMAN incremental backups

(more)

Incremental updated Backups (or merged)apply an incremental backup to the last full backup to get a new full backup

run { backup incremental level 1 for recover of copy with tag 'BKP_L0' database; recover copy of database with tag 'BKP_L0'; }

requires more disk space

to have more than redundancy 1, add UNTIL TIME – MOS note 745798.1

RMAN space usage

Built in functionality, no need to enable NULL BLOCK COMPRESSION (8i) - does not copy empty blocks never used

(ie: unformatted above High Water Mark (HWM)) UNUSED BLOCK COMPRESSION (10.2) - does not copy unused blocks

(ie: empty under HWM) UNDO OPTIMIZATION (11.1) – does not copy undo segments not needed

Must be explicitly enabled

– BINARY COMPRESSION (10g)

Compress data before sending it to the destination. Supports incremental backups. Algorithm can be changed using the Advanced compression option (11.2)

NULL COMPRESSION in action

oracle@oraculo:~> du -hs oradata/XE 1.5G .oradata/XE

oracle@oraculo:~> ls -lrt $ORACLE_BASE/flash_recovery_area/XE/backupset/2010_06_03/total 1176688-rw-r----- 1 oracle dba 1203748864 2010-06-03 00:57 o1_mf_nnndf_TAG.._60g9xpkz_.bkp

Free space on the database:

01:41:16 XE> select sum(bytes)/1024/1024 mb from dba_free_space; MB

---------- 135.8125

RMAN COMPRESSION

Two ways to enable:

backup as compressed backupset ...CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET;

Files generated:

-rw-r----- 1 oracle dba 1203986432 2010-06-03 01:31 o1_mf_nnndf_TAG.._60gcywnl_.bkp-rw-r----- 1 oracle dba 235642880 2010-06-03 01:38 o1_mf_nnndf_TAG.._60gdd066_.bkp

Almost 1GB saved and more CPU used

RMAN COMPRESSION – XE

Can we configure compression as the default backup method?

rman target / CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO BACKUPSET; exit; ./backup.sh

-rw-r----- 1 oracle dba 1203748864 o1_mf_nnndf_TAG.._60g9xpkz_.bkp

rman target / CONFIGURE DEVICE TYPE DISK BACKUP TYPE TO COMPRESSED BACKUPSET; exit ./backup.sh

-rw-r----- 1 oracle dba 120398643 o1_mf_nnndf_TAG.._60gcywnl_.bkp

backup.sh code:

echo "Backup in progress..." rman target / >> $rman_backup << EOF set echo on; shutdown immediate; startup mount; configure retention policy to redundancy 2; configure controlfile autobackup format for device type disk clear; configure controlfile autobackup on; sql "create pfile=''$rman_spfile2init'' from spfile"; backup as backupset device type disk database; configure controlfile autobackup off; alter database open; delete noprompt obsolete;

backup.sh needs to be modified to enable compression:

echo "Backup in progress..." rman target / >> $rman_backup << EOF set echo on; shutdown immediate; startup mount; configure retention policy to redundancy 2; configure controlfile autobackup format for device type disk clear; configure controlfile autobackup on; sql "create pfile=''$rman_spfile2init'' from spfile"; backup as compressed backupset device type disk database; configure controlfile autobackup off; alter database open; delete noprompt obsolete;

Really necessary?

– maximum amount of user data is 11G in XE 11.2 (was 4Gb in 10.2)

– backup.sh uses redundancy 2

Corruption detection

Backup validates block checksum when writing (physical) by default

To detect logical corruption while taking the backup:– backup blocks all check logical database;

To validate already taken backups, and their availability for the restore operation– restore validate database;

– restore validate controlfile to 'c:\temp\control01.ctl';

– restore validate archivelog from sequence N1 until sequence N2;

Validation – integrity and corruption

VALIDATE command (11.1) – independent of backup/recovery

validate database;VALIDATE BACKUPSET 5;validate datafile 3;validate datafile 3 BLOCK 5 TO 20;

If some detected, see which ones:

select * from V$DATABASE_BLOCK_CORRUPTION;

NOTE: This view is also populated by other Oracle process (11.2)

Corruption prevention

To prevent and detect corruption, these initialization parameters must be used:

– DB_BLOCK_CHECKING – in memory checks. Overhead up to 10%.

OFF / LOW / MEDIUM / FULL– DB_BLOCK_CHECKSUM – calculated and stored checksum of data and logs.

Overhead up to 5%. OFF / TYPICAL / FULL– DB_LOST_WRITE_PROTECT – detect lost writes on standby.

NONE / TYPICAL / FULL

– DB_ULTRA_SAFE: set the proper combination of the previous three. • NONE: no changes are made• DATA_ONLY: MEDIUM / FULL / TYPICAL• DATA_AND_INDEX: FULL / FULL / TYPICAL

Recovery scenarios

basic – 2:– complete and tablespace

advanced – 12:– Recovering After the Loss of Datafiles: Scenarios– Recovering Through an Added Datafile with a Backup Control File: Scenario– Re-Creating Datafiles When Backups Are Unavailable: Scenario– Recovering Through RESETLOGS with Created Control File: Scenario– Recovering NOLOGGING Tables and Indexes: Scenario– Recovering Read-Only Tablespaces with a Backup Control File: Scenario– Recovering Transportable Tablespaces: Scenario– Recovering After the Loss of Online Redo Log Files: Scenarios– Recovering After the Loss of Archived Redo Log Files: Scenario– Recovering from a Dropped Table: Scenario– Performing Media Recovery in a Distributed Environment: Scenario

Basic recovery examples

Recover the last full backup, using the existing controlfile

RMAN> STARTUP MOUNT; RMAN> RESTORE DATABASE; RMAN> RECOVER DATABASE; RMAN> ALTER DATABASE OPEN;

Recover only one tablespace in an open instance

RMAN> SQL 'ALTER TABLESPACE mytbs OFFLINE IMMEDIATE'; RMAN> RESTORE TABLESPACE mytbs; RMAN> RECOVER TABLESPACE mytbs; RMAN> SQL 'ALTER TABLESPACE mytbs ONLINE';

Summary of RMAN benefits

– Incremental backups

– automated recovery

• block media recovery

– compression

• skipping blocks and binary

– encryption

– backup history and catalog

– logical and physical corruption detection

– backup optimization

– improved and new features in new releases

• 11 in version 11.2, 21 in version 11.1 (from Backup and Recovery User's Guide, “What's New in Backup and Recovery?”)

using RMAN for the first time

– Learn RMAN functionality and usage

– Practice recovery scenarios

NOTE: RMAN generated files are not portable between platforms.

– Intel 32-bit (x86) backup cannot be restored on Intel 64-bit (x86_64).

– CONVERT DATABASE command can be used over platforms which share endian format.

Define the backup policy

– acceptable level of service

• time frame for implementation and execution

• disk consumption and CPU usage to define using of compression

– retention

– destination

– file names, size limits

– parallelism

– use of encryption

Create complete scripts– Configuration can be used to recreate the database– Delete obsolete backups – List catalog (documentation) – Validate logical integrity – Detect errors in the execution of the script and notify operators

Test recovery periodically– automated to validate health of backup media– manually to practice different recovery scenarios

• one team to break, one to recover

Monitoring RMAN operations

Running tasks: V$SESSION_LONGOPS

SELECT SID, SERIAL#, CONTEXT, SOFAR, TOTALWORK, ROUND(SOFAR/TOTALWORK*100,2) "%_COMPLETE"FROM V$SESSION_LONGOPSWHERE OPNAME LIKE 'RMAN%'AND OPNAME NOT LIKE '%aggregate%'AND TOTALWORK != 0AND SOFAR <> TOTALWORK;

History: – V$BACKUP_*– V$RMAN*

If using catalog: – RC_*

Maintenance

CrossCheck Delete obsolete

If using standby

Above 11.1: ARCHIVELOG DELETION POLICY TO APPLIED ON STANDBY

Older versions: Split backup from delete obsolete tasks

Features not available in Standard Edition:

– allocate disk parallel channels

– block change tracking

– encryption -- Advanced Security option (disk) or Oracle Secure Backup license (tape)

When errors

• Manuals: http://otn.oracle.com

• Support: http://support.oracle.com

• Debug: rman target / log rman.log trace rman.trc run{ allocate channel t1 type sbt………trace=2; allocate channel t2 type sbt………trace=2; allocate channel t3 type sbt………trace=2; debug on; restore database; debug off; }

When errors

Media manager libraries (MML) are from third parties Trace generates sbtio.log file Simulate and compare:

run { allocate channel t1 type sbt parms 'SBT_LIBRARY=oracle.disksbt,ENV=(BACK UP_DIR=d:\temp)' trace=2; backup database; }

Optimizing backup operations

mainly I/O activity detailed process architecture: MOS note 360443.1 use async I/O (O.S. Configuration) channels adequate to the hardware in use performance of the MML comparing the time of "backup validate" (read-only operation) with the time of backup. enable block change tracking for incremental backups (Enterprise Edition) use standby database if Active DataGuard is available.

Optimizing Recovery

• Scripts commands are executed serially

• This does not work in parallel:

run { allocate channel t1 type sbt....; allocate channel t2 type sbt....; allocate channel t3 type sbt....; restore datafile 2; restore datafile 3; restore datafile 5; restore datafile 7; restore datafile 11; }

Optimizing Recovery

• This works:

run { allocate channel t1 type sbt....; allocate channel t2 type sbt....; allocate channel t3 type sbt....; restore datafile 2,3,5,7,11; }

• For big datafiles, from 11.1, use SECTION SIZE:

CONFIGURE DEVICE TYPE sbt PARALLELISM 3;CONFIGURE DEFAULT DEVICE TYPE TO sbt;RUN { BACKUP SECTION SIZE 300M TABLESPACE prueba; }

Next steps?

– Exercise recovery scenarios

– Use Enterprise Manager whenever possible

– Use Data Recovery Advisor

– Incorporate the use of Dataguard and Flashback

Questions?

nelson.calero@gmail.com

Some time for examples?

– Instance cloning• Classic (10g)• From active instance (11.1)• RAC

– Using Data Recovery Advisor – only on the white paper

Instance cloning

Classic (10g) using pre-existing backups

1) Set destination database environment: parameter file, password file and directories – db_file_name_convert y log_file_name_convert if using different directory structure

2) Reboot the destination database in nomount mode

3) Get the SCN to which you want to restore from source. There are several ways to do it. One is:

select next_change# from v$archived_log where recid = (select max(recid) from v$archived_log where backup_count>0);

Instance cloning

Classic (10g) using pre-existing backups

4) execute the duplication on destination server :

rman catalog rman/clave@rman target sys/clave@origen connect auxiliary / run { allocate auxiliary channel dupdb1 type disk; set until scn $MAX_SCN; duplicate target database to COPIA NOFILENAMECHECK; }

5) disable archivelog on the cloned database

Instance cloning

From active instance (11.1)

Same steps 1 and 2, then 4th is:

rman nocatalog target sys/clave@origen connect AUXILIARY sys/clave@copia run { DUPLICATE TARGET DATABASE TO 'COPIA' FROM ACTIVE DATABASE; } exit;

Instance cloning

RAC databases

MOS notes 452868.1 and 461479.1 changes to single instance duplication:

destination database configured as single instance (removing RAC configuration) clone the same way as single instance pfile parameters are adjusted, adding RAC settings removed before:

correct names for control files and the new database. restart the database to use the new parameters configure the new database in all the remaining RAC nodes:

pfile, pwfile, tnsnames.ora register the new database in CRS

collaborate 2012 - rman eliminate the mystery

rman backup files

rman backupsdetails

rman backupsdifferences

functionalities rman

rman backupssee

recovery files

rman backupsconsistent

flexible rman commands

Technology

oracle rman demo

rman recipes doag2014 - doag deutsche … · rman recipes...

asigra - rman interface_description

rman command document

rman tutorial p

rman for dummies

rman booklet

ppoug, 05-oct-01 agenda rman architecture why use rman?...

les 10 tune rman

rman hot bkp

oracle rman best practices

les 07 rman rec

rman notes

less 04 config rman

dbaii rman backups

rman - recovery manager concepts

backuprecovery with rman

migrate databases to exadata using rman...

rman cloning theory

rman guide