Backup & Recovery with RMAN

Backup & Recovery with RMAN CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/it1AgendaLCG 3D Workshop, Bologna, June 2007 - 2Types of failures and backups in OracleRMAN ArchitectureManual vs. RMAN backupsOn-tape backups with RMANRMAN ConfigurationRMAN backup strategiesRMAN backups at Tier0Exemplary recovery scanariosDemo

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itTypes of failuresLCG 3D Workshop, Bologna, June 2007 - 3Instance FailureUsually connected with an Oracle process failureMedia FailureDisk failure, storage array controller failure etc.Block CorruptionUsually caused by bugs in Oracle softwareHuman errorIn most cases accidentally deleted/updated dataDatabase user or DBADisasterFire, flood, earthquake, plane crash etc.CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBackup options in OracleLCG 3D Workshop, Bologna, June 2007 - 4Physical backups Cold (off-line) backupsFull database onlyRequire downtimeDo not provide flexibility for point in time recovery (PiTR)Hot (on-line) backups Different types of backups: full, incr. (cumulative, differential), archivelogs Different scopes: full database, tablespace(s) or datafile(s)Do not require database downtimeCan be used to recover full database, single/multiple tablespace(s)/datafile(s) or a corrupted blockDatabase can be recovered to any point in time within assumed backup retention period

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBackup options in Oracle (2)LCG 3D Workshop, Bologna, June 2007 - 5Logical backupsLogical copy of data in the databaseSupport for different backup granularityCan be taken either with legacy Export/Import tools or with Data Pump (10g)

Standby systems (Data Guard)Physical and logical standby databases

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itHow hot backups are possible?LCG 3D Workshop, Bologna, June 2007 - 6

SGAServer proc.DBWRLGWRRedoLogRedoLogDb fileDb fileDb fileDb fileLog archiverArchived redo logsBackup proc.Tapes or disks systemCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itRMAN architectureLCG 3D Workshop, Bologna, June 2007 - 7TargetDatabase

RMAN ClientTargetDatabaseTargetDatabaseAuxiliaryDatabaseRMANCatalogSchemaCatalog DatabaseCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itTypes of RMAN hot backupsLCG 3D Workshop, Bologna, June 2007 - 8Copy or backupsetFull database backupIncremental backups (in 10g 2 levels available: 0 and 1)Cumulative, differentialCan be used to update a copy of the databaseArchivelog backupsTablespace(s), datafile(s) backups

Differential incrementalCumulative incrementalCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itManual vs. RMAN backupsLCG 3D Workshop, Bologna, June 2007 - 9RMAN advantages:Supports incremental backup strategiesRMAN on-line backups are not so heavy for the system as manual on-line backupsRMAN can detect corrupted blocksRMAN automatically track database structure changesProvides easy, automated backup, restore and recovery operationsKeeps invenotory of taken backupsCan seamlessly work with third party media managers

Disadvantage: something new to learnRMAN concepts and command syntax sometimes are not intuitive

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itOn-tape backups with RMANLCG 3D Workshop, Bologna, June 2007 - 10RMAN allows to take on-disk backups out of the boxFlash recovery area, if configured, further simplifies such backupsOn disk backups are interesting but usually not sufficient for a disaster recoveryOn-disk backups can be manually sent to tapesRecovery can be very troublesomeRMAN can seamlessly work with third party Media ManagersMedia Manager Library (MML) is requiredDifferent configuration tasks for different MMLsCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itOn-tape backups with RMAN (2)LCG 3D Workshop, Bologna, June 2007 - 11Many vendors of Media Management software provide MMLsMost popular are:Tivoli Storage ManagerVeritas NetBackupEMC NetWorker

Oracle Secure BackupVery new and probably not yet reliable enoughCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itRMAN ConfigurationRMAN can be preconfiguredConfiguration is stored in the control file and in the recovery catalog (if used)Can facilitate backup automationMost useful settings:

SettingDefaultRecommendedControlfile autobackupoffonRetention policyto redundancy 1to recovery window of 31 daysDevice typedisk parallelism 1 ...disk|sbt prallelism 2 ...Default device typeto diskto sbtBackup optimizationoffoffChannel device typenonesbt parms=...Maxsetsizeunlimiteddepends on your database sizeLCG 3D Workshop, Bologna, June 2007 - 12CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itRMAN Configuration (2)LCG 3D Workshop, Bologna, June 2007 - 13Example:

configure RETENTION POLICY TO RECOVERY WINDOW OF 31 DAYS;configure DEFAULT DEVICE TYPE TO 'sbt';configure DEVICE TYPE 'sbt' PARALLELISM 2;configure CHANNEL DEVICE TYPE sbt parms='ENV= (TDPO_OPTFILE=/opt/tivoli/tsm/client/oracle/bin/tdpo.opt)';configure DEVICE TYPE DISK PARALLELISM 2;configure MAXSETSIZE TO 200 G;configure archivelog backup copies for device type 'sbt' to 1;configure controlfile autobackup on;

The SHOW ALL command lists all RMAN configuration settingsTo clear a given settings append CLEAR at the end of the CONFIGURE command

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itRMAN backup strategiesLCG 3D Workshop, Bologna, June 2007 - 14RMAN allows many types of backupsIt possible to build own backup strategy that suits given database bestThere are also Oracle recommended backup strategies:Incremental backup strategy: level 0 backups + level 1 backups (cumulative and/or differential) and archivelog backups inbetween.Incrementally updated database copy: 1 backup as copy of the whole database + incremental backups used to update the copy + archived redo logs.CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBackup strategy used at Tier0LCG 3D Workshop, Bologna, June 2007 - 15Both Oracle-recommended strategies implemented for all production systemsIncremental backup strategy:Backups go to tapes Weekly or biweekly level 0 backups (depending on the DB size)A level 1 cumulative backup inbetweenDaily incremental level 1 differential backupsArchivelog backup every 30 minutesIncrementally updated DB copy strategy:daily incremental differential backups applied with 2 days of delayCopies, incremental backups and archived redo logs stored in the Flash Recovery AreaCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBackup operations at Tier0LCG 3D Workshop, Bologna, June 2007 - 16Central machine to schedule and run all the backupsCentral RMAN catalog exported on regular basisExamples of RMAN commands being used:run { crosscheck archivelog all;backup force tag some_tag_0T' incremental level 0 check logical database force format '%d_%T_%U_lvl0T' plus archivelog format '%d_%T_%U_lvl0Tarch'; delete noprompt force archivelog all completed before sysdate - 2";}run { crosscheck archivelog all; backup tag some_tag_1D' incremental level 1 database format '%d_%T_%U_lvl1D' plus archivelog format '%d_%T_%U_lvl1Darch'; delete noprompt force archivelog all completed before sysdate 2;}CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBackup operations at Tier0 (2)LCG 3D Workshop, Bologna, June 2007 - 17run { backup tag some_tag_AR' archivelog all format '%d_%T_%U_arch'; delete noprompt force archivelog all completed before sysdate - 2; }run {backup maxsetsize 2047G tag ' DB_Copy_tag' device type disk incremental level 1 for recover of copy with tag DB_Copy_tag database;

recover device type disk copy of database with tag DB_Copy_tag until time sysdate -2 ; delete force noprompt backupset device type disk tag DB_Copy_tag completed before sysdate -2;}CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itComplete database recoveryLCG 3D Workshop, Bologna, June 2007 - 18Needed when: All datafiles are lost or the SYSTEM tablespace datafiles are lostAt least one member of each redo log group survivedRequires:Control file recovery (if its lost)Datafile restore from a backupDatabase recovery using incremental backups and/or archived redo logs and online redo logsstartup mountrun { allocate channel c1 device type disk|sbt; allocate channel c2 device type disk|sbt; restore database check readonly; recover database; alter database open;}CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itDatabase point-in-time recoveryLCG 3D Workshop, Bologna, June 2007 - 19Needed when: all datafiles are lost All copies of the current control file are lost Or all online redo log group members are lostThe most typical recovery in case of systems implementing SAME approachIf done after a disaster it has to be preceded by:Hardware configurationOS and Oracle software installationRe-creation or restore from non-RMAN backup of listener.ora, tnsnames.ora and other important configuration filesASM instance and diskgroup configuration (if needed)MML installation and configuration...RequiresSpfile restoreControlfile restoreDatafiles restore and recovery

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itLCG 3D Workshop, Bologna, June 2007 - 20run { set until time = "to_date('24-AUG-2006 00:00:00','dd-mon-yyyy hh24:mi:ss')"; allocate channel c1 device type disk|sbt; allocate channel c2 device type disk|sbt; restore database check readonly; recover database; alter database open resetlogs;}set dbid=xxxxxxxstartup nomount;run {allocate channel c1 device type disk|sbt;restore spfile to some_location from autobackup;recover database; alter database open resetlogs;}shutdown immediate;startup nomount;run { allocate channel c1 device type disk|sbt; restore controlfile from autobackup;alter database mount; } CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itTablespace point-in-time recoveryLCG 3D Workshop, Bologna, June 2007 - 21NeededMainly to address a human errorOracle makes efforts to automate itCan be done with few clicks in OEMRequiresPoint in time recovery of the whole database (an auxiliary instance is created)Export/import of selected tablespaces schemas or objects

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itBlock media recoveryLCG 3D Workshop, Bologna, June 2007 - 22Needed when:Database reports either single or multi block corruptionCan be done with an open databaseOnly the table(s) containing corrupted blocks are not availableDatabase corruptions can be discovered with RMAN backup validate database commandCorrupted blocks can be found in V$DATABASE_BLOCK_CORRUPTION

run {allocate channel c1 device type disk|sbt;blockrecover datafile 19 block 44;}ORA-01578: ORACLE data block corrupted (file # 19, block # 44)ORA-01110: data file 19: d:\oracle\oradata\data\mydb_maintbs_01.dbfCERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itSingle/multiple datafile/tablespace recoveryLCG 3D Workshop, Bologna, June 2007 - 23Needed whenSingle/multiple tablespaces or datafiles have been lostSYSTEM tablespace is intactControlfiles and online redo logs are intactRequiresTo put offline datafiles and tablespaces being recoveredThe database can be open and available to usersrun { sql alter tablespace users offline; allocate channel c1 device type disk|sbt; restore tablespace users; recover tablespace users; sql alter tablespace users online;}CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/itMore infoLCG 3D Workshop, Bologna, June 2007 - 24Oracle Database 10g RMAN Backup & Recovery (by Mathew Hart and Robert G. Freeman)

Oracle DocumentationBackup and Recovery Basics Backup and Recovery Advanced User's Guide Backup and Recovery Reference

CERN - IT DepartmentCH-1211 Genve 23Switzerlandwww.cern.ch/it