1

Chapter 17Troubleshooting RMAN

2

Background

Authors thought this topic was often glazed over or not covered well

Knew that every topic can’t be covered, that could be a very long list

Tried to cover the most common types of RMAN issues

3

Sources of Information

Backup and recovery problems are stressful Usually you can find somebody else who has

already encountered the issue Start at bottom of error stack and work your

way up Syntax issues are common Check alert.log and trace files

4

Sources of Information (continued)

Ask for help Search Oracle’s documentation Use RMAN forum Open a SR with Oracle Enable output logging (Oracle Support will

often request this)

5

Resolving Connection Issues

Common problem Usually caused by not using an

authenticated user If using a password file, ensure correct

username/password are being used Attempt to connect to SQL*Plus to

troubleshoot

6

Handling Disk Space Issues

Usually caused by running out of disk space To correct:

– Change location of the backup– Add disk space (if possible)– If using a FRA, either add space or move– Change retention policy– Delete old files

7

Dealing with RMAN-06059 Error

Common problem, occurs when RMAN doesn’t know where old archive redo log files are located

Due to switching from user managed to RMAN Due to not using RMAN to delete old files “loss of archived log compromises recoverability”...

Yikes!

RMAN> crosscheck archivelog all;

8

Terminating RMAN Processes

Sometimes necessary when backup or restore hangs

Try control + C Manually kill process from operating system Kill RMAN session from SQL*Plus Query v$session_longops to see if the job is

actually making progress

9

Diagnosing NLS Character Set Issues

ORA-12705 error Sometimes caused by a mismatch between

NLS character set on client and server Sometimes caused by an erroneously set

NLS-related operating system parameter

SQL> select value from v$nls_parameters where parameter = 'NLS_CHARACTERSET';

10

Logging RMAN Output

Extremely helpful for debugging/diagnosing issues From OS prompt:$ rman target / log=rman_output.log

From within RMAN:RMAN> spool log to rman_output.logRMAN> set echo onRMAN> backup database;RMAN> spool log off;

11

Viewing RMAN Command History

Use v$rman_output

SQL> select sid, recid, output

from v$rman_output

order by recid; Holds 32,768 rows

12

Enabling Debug Output

You can turn on debugging in several different ways:– From the OS prompt– When allocating a channel– When configuring a channel– From the RMAN command-line prompt

13

Enabling Debug Output

Enable all debugging

$ rman target / debug=all log=rman_output.log

Enables debugging just for I/O activities:

$ rman target / debug=io

14

Information Types of Debugging

io sql plsql rcvman rpc

15

Enabling Granular Time Reporting

Oracle Support will often recommend that you enable this before sending them output

Allows you to view down to the second when operations occurred

Authors recommend that you always enable this Korn shell example:

$ export NLS_DATE_FORMAT='dd-mon-yyyy hh24:mi:ss'

16

Working with Oracle Support

Some problems will require help from Oracle Support To facilitate response:

– Test case that illustrates step-by-step details to reproduce the problem

– Complete RMAN script or command(s) that were run– Complete RMAN output log– Database alert.log file– Remote Diagnostic Agent (RDA) output (optional)

17

Resolving RMAN Compatibility Issues

Ideally, all databases and catalog will be at the same level

Reality, almost never the case Most up to date source RMAN compatibility

matrix information in MetaLink note 73431.1 Checking the recovery catalog version (if

using):SQL> select * from rcver;

18

Dealing with an ORA-19511 Error

Vexing issue Caused by MML mis-configuration Caused by incorrectly set OS variables Work with your MML vendor to resolve Check sbtio.log file

19

Dealing with an ORA-27211 Error

Vexing issue Usually related to MML not being correctly

installed Work with MML vendor to resolve Look in log and trace files for more

information

20

Dealing with an ORA-04031 Error

Enabled I/O slaves RMAN will throw this if it can’t allocate

enough memory Setup a large pool area Use ASMM See MetaLink note 73354.1

21

Managing Files in an ASM Environment

Background: Technical editor wrote this material in the book, often received questions regarding ASM at Oracle Support

Using ASM (Automatic Storage Management) Need to view RMAN files Use ASMCMD Command line utility$ asmcmd –p Unix like commands